← Every exhibit

Table 1From “What is an A/B test? randomized controlled trials, statistical power, and governance”

Decision Errors and Statistical Power

Cite Embed

Table 1Decision Errors and Statistical Power

DecisionNull hypothesis trueTreatment effect real (lift)
Reject the null (declare a winner)Type I error (alpha): a false positive, about 5% of the timeCorrect decision (power, one minus beta): a true discovery, 80% to 90% of the time
Fail to reject (retain the control)Correct decision (one minus alpha): a true negative, about 95% of the timeType II error (beta): a false negative, 10% to 20% of the time

Swipe or scroll horizontally if the table is wider than your screen.

Source: Table from this essay. Sources and interpretation are given in the article.

PNG4:3, 1600 × 1200, with the title, the source line and the attribution set in the image isoglu-2026-what-is-an-ab-test-fig02.png

From the essay

Cite or embed

APA 7

Isoglu, S. (2026). Decision Errors and Statistical Power [Table]. isoglu.com. https://isoglu.com/insights/exhibits/isoglu-2026-what-is-an-ab-test-fig02/

BibTeX
@misc{isoglu-2026-what-is-an-ab-test-fig02,
  author = {Isoglu, Sinan},
  title = {{Decision Errors and Statistical Power}},
  year = {2026},
  howpublished = {Table 1, isoglu.com},
  url = {https://isoglu.com/insights/exhibits/isoglu-2026-what-is-an-ab-test-fig02/},
  note = {From the essay {What is an A/B test? randomized controlled trials, statistical power, and governance}}
}
Embed this exhibit

Copy this HTML to use the PNG exhibit on your site with its source line and a link back to the exhibit page.

<figure>
  <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-an-ab-test-fig02/">
    <img src="https://isoglu.com/social/isoglu-2026-what-is-an-ab-test-fig02.png" alt="Table 1: Decision Errors and Statistical Power" width="3200" height="2400">
  </a>
  <figcaption>Table 1: Decision Errors and Statistical Power. Source: Table from this essay. Sources and interpretation are given in the article. <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-an-ab-test-fig02/">View exhibit</a>.</figcaption>
</figure>

Related exhibits

Figure 1Figure

The A/B test experimental architecture

What is an A/B test? randomized controlled trials, statistical power, and governance

Figure 2Figure

Taxonomy of Experimentation Architectures

What is an A/B test? randomized controlled trials, statistical power, and governance

Table 2Table

The Metric Hierarchy in Controlled Experiments

What is an A/B test? randomized controlled trials, statistical power, and governance

Every exhibit

Contact

Bring the question you're working on.

Research, a guest lecture, or a commercial question.