← Every exhibit

Table Figure 1 From the research bench

The advertising test-value screen

Release additional advertising data when it can inform a named action, not simply because the sample is larger.

Review fieldRequired inputPermitted interpretationStop signal
DecisionAction that could change“This test informs this choice.”The test has no stated decision.
Relevant effectMinimum commercially meaningful difference“This effect size matters at the stated threshold.”Any statistically detectable difference is treated as useful.
Variance and powerOutcome noise and detection target“The design has power for the effect that matters.”Sample size is copied from another campaign.
Observation costData, time, and follow-up burden“More data is worth collecting under this cost.”Data volume is treated as free.
Action thresholdResult that changes the recommendation“This estimate crosses or does not cross the rule.”A confidence interval is reported without a decision rule.
Claim boundaryStudy, platform, horizon, and metric“The evidence supports this bounded statement.”One experiment becomes a universal sample-size or ROI rule.

Swipe or scroll horizontally if the table is wider than your screen.

Cite Embed

Reference & Evidence

Source: Author's decision framework grounded in Johnson, Lewis and Reiley (2017) and Wernerfelt, Tuchman, Shapiro and Moakler (2025). The thresholds and example prompts are synthetic; the reported platform findings remain bounded to their studies.

Each line is a claim from the register this journal publishes against, resolved from the register at build time.

  • A The design, in the authors' words: "We use a randomized field experiment on 3 million Yahoo! users who are also past customers of the retailer", and the paper is about what the data buy: "We show that control ads boost measurement precision by identifying and removing the half of in-campaign sales data that are unaffected by the ads" Johnson, Lewis & Reiley. (2017) · JLR17-C1
  • A More data is not the lever the title implies: "Less data give us 31% more precision in our estimates", while "we only improve precision by 5% when we include additional covariate data to reduce the residual variance in our experimental regression" Johnson, Lewis & Reiley. (2017) · JLR17-C2
  • A So the design question is power and decision value rather than observation count, which the authors demonstrate by the contrast between removing unaffected data and adding covariates: "Less data give us 31% more precision in our estimates" Johnson, Lewis & Reiley. (2017) · JLR17-C3
  • A The experiment, verbatim: "we conduct a large-scale, randomized experiment that includes more than 70,000 advertisers on Facebook and Instagram" Wernerfelt, Tuchman, Shapiro & Moakler. (2025) · WTSM25-C1
  • A The two costs and the change between them, verbatim: "We find a median cost per incremental customer at baseline of $38.16 that under the median loss in effectiveness would rise to $49.93, a 31% increase" Wernerfelt, Tuchman, Shapiro & Moakler. (2025) · WTSM25-C2
  • A And the distributional finding: "we find ads targeted using offsite data generate more long-term customers per dollar than those without, and losing offsite data disproportionately hurts small scale advertisers" Wernerfelt, Tuchman, Shapiro & Moakler. (2025) · WTSM25-C3

Grades: A, verified against the printed page of the primary source · B, primary source, text layer only · C, authoritative secondary · D, reported.

Related exhibits