Figure 16 Sept 2026From “What is statistical conclusion validity? A significant result can still be wrong”

The synthetic statistical-inference review

Significance is one field in an inference record. Effect, uncertainty, testing family, model, and permitted wording must remain visible together.

Cite Embed

Figure 1The synthetic statistical-inference review

ID	Reported result	Uncertainty and information	Testing or model context	Conclusion permitted	Still not established
T-01	Estimate +2.0 points, p = 0.01	95% interval +0.5 to +3.5; adequate planned power	One declared outcome and model	Evidence of a positive association in the studied sample	Practical importance or causality
T-02	Estimate +0.2 points, p < 0.001	Narrow interval +0.15 to +0.25	Large sample; outcome scale is small	Precisely estimated small difference	Material business value
T-03	Estimate +8 points, p = 0.08	Wide interval -1 to +17	One outcome; limited information	Data are insufficient to rule in or out the declared effect	No-effect conclusion
T-04	Estimate +5 points, p = 0.03	Interval +0.4 to +9.6	Twelve outcomes tested; no family rule stated	At most a flagged result pending multiplicity review	Confirmed discovery
T-05	Estimate +10 points, p = 0.02	Interval +2 to +18	Model changed after inspecting outcome	Conditional post hoc result	Ex ante test interpretation
T-06	Estimate +6 points, p = 0.01	Interval +2 to +10	Studied in one segment and 14-day window	Positive result in that segment and window	Transfer to all customers or 90-day renewal

Swipe or scroll horizontally if the table is wider than your screen.

Source: Author's synthetic inference review grounded in Cohen (1994) and Ioannidis (2005). Estimates, intervals, testing histories, and permitted conclusions are illustrative.

PNG4:3, 1600 × 1200, with the title, the source line and the attribution set in the image isoglu-2026-what-is-statistical-conclusion-validity-fig02.png

Source and evidence

Each line names a claim related to this exhibit, its source, and how it was checked.

A COH94-C1 "the ritual of null hypothesis significance testing mechanical dichotomous decisions around a sacred 05 criterion still persists", and the article reviews the problems with itCohen (1994), article
A COH94-C2 What he recommends instead: "an emphasis on estimating effect sizes using confidence intervals and the informed use of available statistical methods", with graphic methods and better measurementCohen (1994), article
A IOANNIDIS05P-C1 "the probability that a research claim is true may depend on study power and bias the number of other studies on the same question" and the ratio of true to no relationships in the fieldIoannidis (2005), PLoS Medicine article
A IOANNIDIS05P-C2 Bias is the lever: "with increasing bias the chances that a research finding is true diminish considerably this is shown for different levels of power and for different pre study odds"Ioannidis (2005), PLoS Medicine article
B R08-OWN-C1 Statistical conclusion validity asks whether the inference from data and model to the stated statistical conclusion is warranted under its assumptions and error controlsAuthor framework grounded in COH94-C1 to COH94-C3
B R08-OWN-C2 A review should report effect estimate, uncertainty interval, sample or information basis, testing family, model specification, and conclusion wording togetherAuthor framework
B R08-OWN-C7 The six-row inference table contains illustrative estimates and labels only; it is not an observed study result or benchmarkAuthor synthetic object
B R08-OWN-C9 A significant result should be bounded by the population, outcome, model, testing family, and observation window that produced itAuthor operating rule

Grades: A, verified against the printed page of the primary source · B, primary source, text layer only · C, authoritative secondary · D, reported.

From the essay

From the research bench

What is statistical conclusion validity? A significant result can still be wrong

Statistical conclusion validity asks whether the data and model warrant the stated result. Check effect, uncertainty, power, and testing before deciding.

6 Sept 2026

Read the essay

This exhibit belongs to an essay: the numbers live here, the argument lives there.

Cite or embed

APA 7

Isoglu, S. (2026). The synthetic statistical-inference review [Figure]. isoglu.com. https://isoglu.com/insights/exhibits/isoglu-2026-what-is-statistical-conclusion-validity-fig02/

BibTeX

@misc{isoglu-2026-what-is-statistical-conclusion-validity-fig02,
  author = {Isoglu, Sinan},
  title = {{The synthetic statistical-inference review}},
  year = {2026},
  howpublished = {Figure 1, isoglu.com},
  url = {https://isoglu.com/insights/exhibits/isoglu-2026-what-is-statistical-conclusion-validity-fig02/},
  note = {From the essay {What is statistical conclusion validity? A significant result can still be wrong}}
}

Embed this exhibit

Copy this HTML to use the PNG exhibit on your site with its source line and a link back to the exhibit page.

<figure>
  <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-statistical-conclusion-validity-fig02/">
    <img src="https://isoglu.com/social/isoglu-2026-what-is-statistical-conclusion-validity-fig02.png" alt="Figure 1: The synthetic statistical-inference review" width="3200" height="2400">
  </a>
  <figcaption>Figure 1: The synthetic statistical-inference review. Source: Author&#39;s synthetic inference review grounded in Cohen (1994) and Ioannidis (2005). Estimates, intervals, testing histories, and permitted conclusions are illustrative. <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-statistical-conclusion-validity-fig02/">View exhibit</a>.</figcaption>
</figure>