Figure 118 Oct 2026From “What is p-hacking?”

P-values text-mined from Results sections, in the two bins just below 0.05

In each of the three disciplines with the most p-values, the bin just below 0.05 held more p-values than the bin below it, which the authors do not expect without p-hacking.

Cite Embed

Figure 1P-values text-mined from Results sections, in the two bins just below 0.05

Source: Head et al. (2015), p. 7, Table 1: mean number of p-values from Results sections of open access papers in PubMed, based on 1,000 bootstraps of one p-value per Results section and rounded to whole numbers, in the bins 0.04 to 0.045 and 0.045 to 0.05, for the three disciplines the authors name as having good statistical power (p. 8). One-tailed binomial test for p-hacking from the same table.

PNG4:3, 1600 × 1200, with the title, the source line and the attribution set in the image isoglu-2026-what-is-p-hacking-fig01.png

Source and evidence

Each line names a claim related to this exhibit, its source, and how it was checked.

A HEAD15-C6 Text-mining of p-values in all open access papers in PubMed, Results sections and Abstracts separately; pooled, a strong right skew (researchers appear to study mainly nonzero effects) and strong evidence for p-hacking: upper-bin share 0.546 (lower CI 0.536) across 14 disciplines in Results, 0.537 (0.518) across 10 in Abstracts; more p-values in the upper bin in most disciplines; in Results significant in every discipline with good statistical power (medical and health sciences, biological sciences, multidisciplinary), in Abstracts only for the multidisciplinary and information and computing sciences categories. Table 1 (p. 7), Results-section mean counts from 1,000 bootstraps, lower and upper bin with the one-tailed binomial p: medical and health sciences 1,477 and 1,785 (below 0.001), multidisciplinary 638 and 750 (0.001), biological sciences 350 and 423 (0.005), psychology and cognitive sciences 29 and 50 (0.012); nine of the 14 disciplines have more p-values in the upper bin (counted from Table 1). The authors read the result as p-hacking being widespread, note a strong right skew in all curves and that text-mining pools many types of questions; the proportion 0.5 means an equal number of cases in the two bins; they also report that an earlier text-mining study of five medical journals found no evidence of p-hacking and was criticised for using p-values from Abstracts, which are more likely to contain only the strongest results (pp. 4 to 9): "used text-mining to search for p-values in all Open Access papers available in the PubMed database"; "researchers appear to be predominantly studying phenomena with nonzero effect sizes"; "there was also strong evidence for p-hacking in both the Results"; "estimated proportion of p-values in the upper bin (0.045 < p < 0.05) (lower CI) = 0.546 (0.536)"; "values in the upper bin (0.045 < p < 0.05) (lower CI) = 0.537 (0.518)"; "In most disciplines, there were more p-values in the upper than the lower bin"; "in every discipline where we had good statistical power"; "only the multidisciplinary and Information and Computer Science categories were significant"; "Number of p-values in each bin is the mean number based on 1,000 bootstraps of one p-value per Results section, rounded to the nearest whole number."; "Our text-mining suggests that p-hacking is widespread."; "there was still a strong right skew in all the p-curves we examined"; "we are combining many different types of questions to generate our p-curves"; "differed from 0.5 (i.e., equal number of cases in the two bins)"; "found no evidence of p-hacking in a text-mining study of five medical journals"; "they were criticized for using p-values from Abstracts"; "so they are more likely to contain only the strongest"Head et al. (2015) · Head et al. (2015)

Grades: A, verified against the printed page of the primary source · B, primary source, text layer only · C, authoritative secondary · D, reported.

From the essay

From the research bench

What is p-hacking?

P-hacking is trying data and analysis choices until a result is significant, then reporting that one. It can leave an excess of p-values just below 0.05.

18 Oct 2026

Read the essay

This exhibit belongs to an essay: the numbers live here, the argument lives there.

Cite or embed

APA 7

Isoglu, S. (2026). P-values text-mined from Results sections, in the two bins just below 0.05 [Figure]. isoglu.com. https://isoglu.com/insights/exhibits/isoglu-2026-what-is-p-hacking-fig01/

BibTeX

@misc{isoglu-2026-what-is-p-hacking-fig01,
  author = {Isoglu, Sinan},
  title = {{P-values text-mined from Results sections, in the two bins just below 0.05}},
  year = {2026},
  howpublished = {Figure 1, isoglu.com},
  url = {https://isoglu.com/insights/exhibits/isoglu-2026-what-is-p-hacking-fig01/},
  note = {From the essay {What is p-hacking?}}
}

Embed this exhibit

Copy this HTML to use the PNG exhibit on your site with its source line and a link back to the exhibit page.

<figure>
  <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-p-hacking-fig01/">
    <img src="https://isoglu.com/social/isoglu-2026-what-is-p-hacking-fig01.png" alt="Figure 1: P-values text-mined from Results sections, in the two bins just below 0.05" width="3200" height="2400">
  </a>
  <figcaption>Figure 1: P-values text-mined from Results sections, in the two bins just below 0.05. Source: Head et al. (2015), p. 7, Table 1: mean number of p-values from Results sections of open access papers in PubMed, based on 1,000 bootstraps of one p-value per Results section and rounded to whole numbers, in the bins 0.04 to 0.045 and 0.045 to 0.05, for the three disciplines the authors name as having good statistical power (p. 8). One-tailed binomial test for p-hacking from the same table. <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-p-hacking-fig01/">View exhibit</a>.</figcaption>
</figure>