Figure 2From “What is publication bias?”
A test register
Register every test before it starts, then record its result, its write-up and whether it reached the report.
Figure 2A test register
Source: Author's decision worksheet. The register as a known population follows Franco et al. (2014), pp. 1 and 2 of the Science Express version; the planned size field follows their note that TESS requires adequate statistical power (p. 1), and the effect sought beside the size is this guide's addition; the write-up and report fields follow their finding that most null results were never written up (pp. 1 and 2, Table 3 on p. 6) and Munafò et al. (2017), p. 3; the reason field follows the authors' explanations Franco et al. report (p. 2). The fields are reader inputs; no company data is supplied.

Source and evidence
Each line names a claim related to this exhibit, its source, and how it was checked.
- B PUB-OWN-C3 In a pricing or sales analytics team, a review that shows only tests with significant results is a dashboard of winners and carries the selection Franco et al. found at the write-up, a step the analyst controls and which, as in TESS, can anticipate what the review is expected to want; Sterling's chain can run inside a team when a test with a null result leaves no note and is rerun until a chance success is reported; a register of every test made before it starts makes the team's file drawer countable, so Rosenthal's question, how many hidden studies it would take to overturn a result, need not be asked, because the number of tests left out is known (Rosenthal computed a threshold, not an estimate of any drawer); Figure 2 records the question and registration date, the planned size with the effect sought (power is defined for a stated effect, so a size alone cannot show whether a null could have detected the effect; recording the effect is this guide's addition), the result as strong, mixed or null against the plan, the write-up, the review that shows it and the reason it was left out; illustrated with a hypothetical 24 price tests, each with its question, sample, run length and smallest effect worth finding fixed before the start, 7 significant, all shown by result with notes on the nulls; a null from a test too small for the effect sought says little either way; the mapping is untested.Author mapping grounded in STE59-C2, FMS14-C2, FMS14-C4, ROS79-C2 and MUN17PB-C1; untested
Grades: A, verified against the printed page of the primary source · B, primary source, text layer only · C, authoritative secondary · D, reported.
From the essay
From the research bench
What is publication bias?
Publication bias is when a study's result decides whether it reaches print. In one known set of studies, most null results were never even written up.
Read the essayThis exhibit belongs to an essay: the numbers live here, the argument lives there.
Cite or embed
Contact
Bring the question you're working on.
Research, a guest lecture, or a commercial question.


