Figure 218 Oct 2026From “What is p-hacking?”

An analysis decision record

Write down the result, the decisions behind it, when they were fixed, every version run, the plausible effect and the label before the result goes into a report.

Cite Embed

Figure 2An analysis decision record

Source: Author's decision worksheet. The decisions field follows the practices listed in Head et al. (2015), p. 1, and Munafò et al. (2017), p. 3; the fixed field follows Munafò et al. (2017), p. 3; the versions and label fields follow Head et al. (2015), p. 12; the outside size field follows Gelman and Carlin (2014), pp. 642 and 648. The fields are reader inputs; no company data is supplied.

PNG4:3, 1600 × 1200, with the title, the source line and the attribution set in the image isoglu-2026-what-is-p-hacking-fig02.png

Source and evidence

Each line names a claim related to this exhibit, its source, and how it was checked.

B PHK-OWN-C4 A team with a handful of analyses on one question cannot draw a p-curve, so its check is the record of decisions; Figure 2 records per reported result the estimate, the decisions behind it (stopping, outcome, exclusions, groups, covariates), whether they were fixed before or after seeing results, how many versions were run and whether all are shown, the plausible effect from other data, and the prespecified or exploratory label; illustrated with a hypothetical discount-policy review in which, after a planned comparison shows no significant difference, twelve versions are run and the mid-market deal-size result at p = 0.03 is reported as exploratory with all versions listed, and the outside size is taken from an earlier policy change only if that change was not itself picked for its significance; the record does not make a before-and-after comparison causal.Author worksheet grounded in HEAD15-C3, HEAD15-C9, MUN17-C1, MUN17-C4 and GELC14-C4

Grades: A, verified against the printed page of the primary source · B, primary source, text layer only · C, authoritative secondary · D, reported.

From the essay

From the research bench

What is p-hacking?

P-hacking is trying data and analysis choices until a result is significant, then reporting that one. It can leave an excess of p-values just below 0.05.

18 Oct 2026

Read the essay

This exhibit belongs to an essay: the numbers live here, the argument lives there.

Cite or embed

APA 7

Isoglu, S. (2026). An analysis decision record [Figure]. isoglu.com. https://isoglu.com/insights/exhibits/isoglu-2026-what-is-p-hacking-fig02/

BibTeX

@misc{isoglu-2026-what-is-p-hacking-fig02,
  author = {Isoglu, Sinan},
  title = {{An analysis decision record}},
  year = {2026},
  howpublished = {Figure 2, isoglu.com},
  url = {https://isoglu.com/insights/exhibits/isoglu-2026-what-is-p-hacking-fig02/},
  note = {From the essay {What is p-hacking?}}
}

Embed this exhibit

Copy this HTML to use the PNG exhibit on your site with its source line and a link back to the exhibit page.

<figure>
  <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-p-hacking-fig02/">
    <img src="https://isoglu.com/social/isoglu-2026-what-is-p-hacking-fig02.png" alt="Figure 2: An analysis decision record" width="3200" height="2400">
  </a>
  <figcaption>Figure 2: An analysis decision record. Source: Author&#39;s decision worksheet. The decisions field follows the practices listed in Head et al. (2015), p. 1, and Munafò et al. (2017), p. 3; the fixed field follows Munafò et al. (2017), p. 3; the versions and label fields follow Head et al. (2015), p. 12; the outside size field follows Gelman and Carlin (2014), pp. 642 and 648. The fields are reader inputs; no company data is supplied. <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-p-hacking-fig02/">View exhibit</a>.</figcaption>
</figure>