# Coding sheet: counted cells

Coded by **S. Isoglu**, 2026-07-27, by hand against section 6 of the protocol.
Second-coder pass on a 25% sample: **RUN 2026-07-28, by a language model rather than an independent human reader**: see `second-coder-pass.md`. The protocol's human requirement was waived by the author on 2026-07-28 and the deviation is in the changelog.
**Transcription check.** Every `Figure` and `Named as the source` field below is tested against the answer text in `responses.jsonl`. 41 of 72 are verbatim and are marked `[verbatim]`; the rest are the coder's paraphrase, are marked `[PARAPHRASE]`, and carry the record's own sentence underneath. **Quote from the `[verbatim]` fields or from `responses.jsonl`, never from a paraphrase.**


`citations_n` and `domains_unique` are computed from `responses.jsonl`, not typed. `domains_unique`
counts registrable domains, so `media-publications.bcg.com` and `www.bcg.com` are one domain.
`interested_share` is null on every row: the protocol requires each domain to be classified from its
own homepage and that pass has not been run.

## Q1

| System | Run | figure_given | source_named | source_class | cit | uniq | int_share | contra | version | search |
| --- | ---: | --- | --- | --- | ---: | ---: | --- | --- | --- | --- |
| chat_gpt | 1 | range | yes | tertiary | 3 | 3 |: | no | n/a | yes |
| chat_gpt | 2 | range | no | tertiary | 2 | 1 |: | no | n/a | yes |
| chat_gpt | 3 | range | no | tertiary | 2 | 1 |: | no | n/a | yes |
| claude | 1 | range | yes | tertiary | 5 | 4 |: | no | n/a | yes |
| claude | 2 | range | yes | tertiary | 6 | 4 |: | no | n/a | yes |
| claude | 3 | range | yes | tertiary | 5 | 4 |: | no | n/a | yes |
| gemini | 1 | range | yes | tertiary | 10 | 6 |: | no | n/a | yes |
| gemini | 2 | range | yes | tertiary | 12 | 8 |: | no | n/a | yes |
| gemini | 3 | range | yes | tertiary | 18 | 7 |: | no | n/a | yes |
| perplexity | 1 | range | yes | tertiary | 19 | 17 |: | yes | n/a | yes |
| perplexity | 2 | range | yes | tertiary | 20 | 18 |: | yes | n/a | yes |
| perplexity | 3 | range | yes | tertiary | 19 | 17 |: | yes | n/a | yes |

**Q1 · chat_gpt · run 1**

- Figure [verbatim]: up to five times; 5–7 times; 5–25 times
- Named as the source [verbatim]: According to Optimove … Joy Subscription notes … Artisan Strategies further elaborates
- Domains: artisangrowthstrategies.com, joysubscription.com, optimove.com
- `provenance_error`: none
- Coder note: Three vendor pages offered in answer to a request for the original source. No document, author or study is named as an origin; the named entities are the pages it read.

**Q1 · chat_gpt · run 2**

- Figure [PARAPHRASE]: 5 to 25 times higher; SaaS CAC $1,200 vs CRC $35
- Named as the source [verbatim]: Industry benchmarks indicate
- Domains: artisangrowthstrategies.com
- `provenance_error`: none
- Coder note: No origin named at all. Both citation slots resolve to one growth-consulting blog; domains_unique 1 against citations_n 2. `source_class` codes the highest class of document offered in support, whether or not it is named as an origin; `none` is reserved for answers returning no citations at all. Under the alternative reading: `none` wherever no origin is named: this would be the one cell in the study where `source_class` varies across runs, which is the protocol's falsification trigger. The rule applied is stated so a reader can apply the other one.

**Q1 · chat_gpt · run 3**

- Figure [PARAPHRASE]: 5 to 25 times higher; SaaS CAC $1,200 vs CRC $35
- Named as the source [verbatim]: depending on the industry and business model
- Domains: artisangrowthstrategies.com
- `provenance_error`: none
- Coder note: Same shape as run 2, same single domain. `source_class` codes the highest class of document offered in support, whether or not it is named as an origin; `none` is reserved for answers returning no citations at all. Under the alternative reading: `none` wherever no origin is named: this would be the one cell in the study where `source_class` varies across runs, which is the protocol's falsification trigger. The rule applied is stated so a reader can apply the other one.

**Q1 · claude · run 1**

- Figure [verbatim]: 5-25 times more
- Named as the source [verbatim]: The original source for this widely referenced statistic is **Bain & Company research conducted by Frederick Reichheld**
- Domains: artisangrowthstrategies.com, businessdasher.com, stealthagents.com, yotpo.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] Names Bain & Company / Reichheld research as the origin of the 5–25x cost ratio. No Bain or Reichheld publication reports this ratio; ground truth established 2026-07-27 from primary documents.
- Coder note: The sentence tracks stealthagents.com, which states: 'The foundational figure here comes from Bain and Company research by Frederick Reichheld: acquiring a new customer costs 5 to 25 times more than retaining an existing one.' Checked at source 2026-07-27. The attribution is the page's, reproduced.

**Q1 · claude · run 2**

- Figure [PARAPHRASE]: 5-25 times more; 5x floor / 25x ceiling; retention +5% -> profits +25% to 95%
- Named as the source [verbatim]: The statistic originates from research by **Frederick Reichheld at Bain & Company**
- Domains: artisangrowthstrategies.com, businessdasher.com, stealthagents.com, yotpo.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] Names Bain & Company / Reichheld as the origin of the 5–25x cost ratio; no such publication exists.
    - [not_established] '25% to 95%': checked against the full text of Reichheld & Sasser, 'Zero Defections' (HBR Sept-Oct 1990), supplied by the author 2026-07-28. The article gives the range three ways and none is 25-95%: the exhibit is titled 'Reducing Defections 5% Boosts Profits 25% to 85%'; the body reads 'Reducing defections by just 5% generated 85% more profits in one bank's branch system, 50% more in an insurance brokerage, and 30% more in an auto-service chain'; and the introduction says 'Companies can boost profits by almost 100% by retaining just 5% more of their customers.' The 95% ceiling is downstream drift.
- Coder note: The 5x-floor / 25x-ceiling sentence is near-verbatim stealthagents.com: 'The 5x floor is typical for transactional consumer businesses. The 25x ceiling shows up in complex B2B environments with long sales cycles and substantial onboarding costs.' Checked at source. The 5x-floor/25x-ceiling sentence is near-verbatim stealthagents.com. Separately, the full text of 'Zero Defections' was checked on 2026-07-28: it contains no acquisition-to-retention ratio of any kind, and its only acquisition-cost statement is 'In credit cards, for example, companies spend an average of $51 to recruit a customer and set up the new account.'

**Q1 · claude · run 3**

- Figure [verbatim]: 5-25 times more
- Named as the source [verbatim]: The original source for this widely-referenced statistic is **Bain & Company research conducted by Frederick Reichheld**
- Domains: artisangrowthstrategies.com, businessdasher.com, stealthagents.com, yotpo.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] Names Bain & Company / Reichheld research as the origin of the 5–25x cost ratio; no such publication exists.
- Coder note: Identical in substance to run 1. Same four domains, same named origin.

**Q1 · gemini · run 1**

- Figure [PARAPHRASE]: 5 to 25 times more expensive; 5 times (1990 HBR)
    - the record says: “cquiring a new customer can be anywhere from 5 to 25 times more expensive than retaining an existing one.”
- Named as the source [verbatim]: a 1990 Harvard Business Review article by Frederick Reichheld, cited the cost as 5 times more
- Domains: churnkey.co, optimove.com, postaffiliatepro.com, prescientai.com, uplandsoftware.com, yotpo.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] Attributes a 5x cost ratio to a 1990 HBR article by Reichheld. Reichheld & Sasser, 'Zero Defections: Quality Comes to Services' (HBR Sept–Oct 1990) contains no acquisition-to-retention cost ratio; acquisition cost appears only in absolute terms ('an average of $51 to recruit a customer').
    - [not_established] Names Reichheld alone; the 1990 article is co-authored with W. Earl Sasser Jr.
- Coder note: Names a real, checkable document, and the document does not contain the claim. Every citation resolves through a vertexaisearch redirect; the reader cannot see the destination without following it.

**Q1 · gemini · run 2**

- Figure [verbatim]: 5 to 25 times more expensive
- Named as the source [verbatim]: widely cited and supported by research from Bain & Company
- Domains: 2wtech.com, optimove.com, postaffiliatepro.com, prescientai.com, proceragroup.com, thefundworks.com, uplandsoftware.com, yotpo.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] Attributes the ratio to Bain & Company research. No Bain publication reports it.
- Coder note: Organisation named, no document. 12 citation slots resolve to 8 domains, all vendor or agency pages.

**Q1 · gemini · run 3**

- Figure [PARAPHRASE]: 5 to 25 times more; CAC $200–$1,500 vs retention $15–$85; 80-95% less; retention +5% -> profits +25% to 95%; 60-70% vs 5-20%
- Named as the source [verbatim]: stemming from research such as a 1990 Harvard Business Review article by Frederick Reichheld
- Domains: artisangrowthstrategies.com, churnkey.co, forbes.com, optimove.com, postaffiliatepro.com, prescientai.com, yotpo.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] Attributes the ratio to the 1990 Reichheld HBR article, which does not contain it.
    - [not_established] '25% to 95%': downstream drift from the 25–85% range in Reichheld & Sasser 1990.
- Coder note: Longest answer in the cell and the one with the most unsourced numbers: five further figures appear with no attribution of any kind.

**Q1 · perplexity · run 1**

- Figure [verbatim]: 5 times more; 5 to 25 times
- Named as the source [verbatim]: The original source most often traced for this rule of thumb is **Frederick Reichheld's Bain & Company research**
- Domains: artisangrowthstrategies.com, churnkey.co, churnmiser.com, forbes.com, hashtagpaid.com, invespcro.com, linkedin.com, optimove.com, outboundengine.com, postaffiliatepro.com, psu.edu, regulr.ai, releva.ai, stealthagents.com, uplandsoftware.com, upside.com, yotpo.com
- `provenance_error`: none
- Coder note: Hedged correctly: it reports the attribution as an attribution ('most often traced'), not as a fact, so no provenance error is coded. contra_as_support: three cited documents dispute the claim they are cited for. hashtagpaid.com ('the stat is…well, kind of old… one source traces its statistics to another source, which quotes another source'); churnmiser.com ('rarely to a controlled study you could reproduce'); releva.ai ('the specific number is dangerously outdated'). All three checked at source 2026-07-27.

**Q1 · perplexity · run 2**

- Figure [verbatim]: 5 times more; 5 to 25 times
- Named as the source [PARAPHRASE]: **Bain & Company / Frederick Reichheld** research, which later summaries say Harvard Business Review repeated as "anywhere from 5 to 25 times more expensive"
    - the record says: “he commonly cited figure is that acquiring a customer costs **5 times more** than retaining one, though some sources put the range at **5 to 25 times** depending on the industry and business model.[1][16]

The original source most often traced for the **5x** claim is **Bain & Company / Frederick Reichheld** research, w”
- Domains: artisangrowthstrategies.com, churnkey.co, churnmiser.com, forbes.com, hashtagpaid.com, invespcro.com, linkedin.com, optimove.com, outboundengine.com, postaffiliatepro.com, prescientai.com, psu.edu, regulr.ai, releva.ai, stealthagents.com, uplandsoftware.com, upside.com, yotpo.com
- `provenance_error`: none
- Coder note: The HBR paraphrase is accurate: Gallo's 2014 opening sentence reads 'anywhere from five to 25 times more expensive.' Same three contra documents cited.

**Q1 · perplexity · run 3**

- Figure [verbatim]: 5 times more expensive; 5–25 times
- Named as the source [verbatim]: later articles trace it to **Bain & Company research associated with Frederick Reichheld**, with subsequent writeups noting the claim was popularized through Harvard Business Review
- Domains: artisangrowthstrategies.com, churnkey.co, churnmiser.com, forbes.com, hashtagpaid.com, invespcro.com, linkedin.com, optimove.com, outboundengine.com, postaffiliatepro.com, psu.edu, regulr.ai, releva.ai, stealthagents.com, uplandsoftware.com, upside.com, yotpo.com
- `provenance_error`: none
- Coder note: Third consecutive hedge. This cell is the only one in the question that never asserts an origin.

## Q4

| System | Run | figure_given | source_named | source_class | cit | uniq | int_share | contra | version | search |
| --- | ---: | --- | --- | --- | ---: | ---: | --- | --- | --- | --- |
| chat_gpt | 3 | yes | yes | primary | 4 | 1 |: | no | preprint | yes |
| claude | 3 | yes | yes | primary | 2 | 2 |: | no | vor | yes |
| perplexity | 3 | yes | yes | primary | 6 | 6 |: | no | preprint | yes |
| gemini | 3 | yes | yes | primary | 22 | 8 |: | no | preprint | yes |
| chat_gpt | 4 | yes | yes | primary | 4 | 1 |: | no | preprint | yes |
| chat_gpt | 5 | yes | yes | primary | 4 | 1 |: | no | preprint | yes |
| claude | 4 | yes | yes | primary | 4 | 2 |: | no | vor | yes |
| claude | 5 | yes | yes | primary | 2 | 2 |: | no | vor | yes |
| gemini | 4 | yes | yes | primary | 17 | 6 |: | no | preprint | yes |
| gemini | 5 | yes | yes | primary | 15 | 6 |: | no | vor | yes |
| perplexity | 4 | yes | yes | primary | 7 | 7 |: | no | preprint | yes |
| perplexity | 5 | yes | yes | primary | 6 | 6 |: | no | preprint | yes |

**Q4 · chat_gpt · run 3**

- Figure [PARAPHRASE]: approximately 14%; 34% for less experienced agents
- Named as the source [verbatim]: the working paper "Generative AI at Work" … published by the National Bureau of Economic Research in April 2023
- Domains: nber.org
- `provenance_error`: none
- Coder note: Four citation slots, one domain. Third consecutive preprint answer; the QJE version is never mentioned.

**Q4 · claude · run 3**

- Figure [PARAPHRASE]: 14% (NBER WP); 15% (QJE 140(2), 889–942)
- Named as the source [verbatim]: **Published Paper:** … *The Quarterly Journal of Economics*, Volume 140, Issue 2 (May 2025): 889–942
- Domains: nber.org, oup.com
- `provenance_error`: none
- Coder note: Reports both versions, gives correct pagination, cites academic.oup.com, and explains the 14%/15% difference. Third clean pass out of three.

**Q4 · perplexity · run 3**

- Figure [verbatim]: about 14% on average
- Named as the source [verbatim]: their NBER working paper, **"Generative AI at Work"**
- Domains: bls.gov, danielle.li, inigomedina.co, linkedin.com, mit.edu, nber.org
- `provenance_error`: none
- Coder note: Six domains including the author's own PDF and BLS. No QJE link in any of the three runs.

**Q4 · gemini · run 3**

- Figure [PARAPHRASE]: 14%; 5,179 agents
- Named as the source [PARAPHRASE]: NBER Working Paper No. 31161, in April 2023
    - the record says: “. Raymond, published by the National Bureau of Economic Research (NBER Working Paper N”
- Domains: edwardconard.com, hrdive.com, itif.org, mit.edu, nber.org, nngroup.com, policycommons.net, stanford.edu
- `provenance_error`: none
- Coder note: Gives no novice percentage at all, where the narrative runs in Appendix C recorded drifting values ('35% or 38%', '34% to 35%'). Those runs are not counted; the drift reappears in counted run 4 as '13.8%'. 22 slots, 8 domains.

**Q4 · chat_gpt · run 4**

- Figure [PARAPHRASE]: approximately 14%; 34% for less experienced agents
- Named as the source [verbatim]: the working paper "Generative AI at Work" … published by the National Bureau of Economic Research in April 2023
- Domains: nber.org
- `provenance_error`: none
- Coder note: Four slots, one domain. Second consecutive preprint answer; the QJE version is never mentioned.

**Q4 · chat_gpt · run 5**

- Figure [verbatim]: approximately 14%; 34%
- Named as the source [verbatim]: the working paper titled "Generative AI at Work" … published by the National Bureau of Economic Research in April 2023
- Domains: nber.org
- `provenance_error`: none
- Coder note: Third consecutive preprint answer, four slots, one domain, near-identical wording to runs 3 and 4.

**Q4 · claude · run 4**

- Figure [PARAPHRASE]: 14% (NBER WP); 15% (QJE); 5,179 agents
- Named as the source [verbatim]: Erik Brynjolfsson, Danielle Li, Lindsey Raymond, "Generative AI at Work," The Quarterly Journal of Economics, Volume 140, Issue 2, May 2025, Pages 889–942
- Domains: nber.org, oup.com
- `provenance_error`: none
- Coder note: Names the QJE article as the original source outright, with correct pagination, and cites academic.oup.com. Reports the preprint's 5,179 agents rather than the published 5,172: the figure count still comes from the working paper.

**Q4 · claude · run 5**

- Figure [PARAPHRASE]: 14% (NBER WP); 15% (QJE)
- Named as the source [verbatim]: NBER Working Paper No. 31161 (April 2023)**, later published in The Quarterly Journal of Economics, Volume 140, Issue 2 (May 2025)
- Domains: nber.org, oup.com
- `provenance_error`: none
- Coder note: Cites academic.oup.com and gives both figures, but LEADS with 14% and names 'NBER Working Paper No. 31161 (April 2023)' as the original source, flagging the QJE second. Coded `vor` because the answer surfaces and links the version of record; under a stricter reading of 'which version the answer is built on' this row is `preprint`. The call is recorded here because it is the one that makes this cell 3/3 rather than 2/3.

**Q4 · gemini · run 4**

- Figure [PARAPHRASE]: 14% on average; 34% for novices; "Some sources specify the average increase as 13.8%"
- Named as the source [PARAPHRASE]: NBER Working Paper 31161, in April 2023 and revised in November 2023
    - the record says: “Raymond, published by the National Bureau of Economic Research (NBER Working Paper 31161) in April 2023 and revised in November 2023.”
- Domains: google.com, hrdive.com, itif.org, nber.org, nngroup.com, stanford.edu
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] '13.8%' appears in neither version of the paper. Attributed only to unnamed 'some sources'.
- Coder note: The drifting-figure behaviour returns in a new form: a third decimal variant, hedged to unnamed sources. `google.com` appears as a citation domain.

**Q4 · gemini · run 5**

- Figure [PARAPHRASE]: 15% on average; 5,172 agents
- Named as the source [verbatim]: circulated as an NBER working paper (No. 31161) and later published in the Quarterly Journal of Economics
- Domains: arxiv.org, itif.org, policycommons.net, revartis.com, richmondfed.org, stanford.edu
- `provenance_error`: none
- Coder note: The one result that breaks the pattern. Reports BOTH version-of-record figures: 15% and 5,172, and names the QJE publication. But it cites no oup.com slot: its domains are stanford.edu, arxiv.org, revartis.com, itif.org, policycommons.net, richmondfed.org. Correct numbers, no link to the paper that contains them.

**Q4 · perplexity · run 4**

- Figure [PARAPHRASE]: about 14% on average; about 34% for novices; 5,179 agents
- Named as the source [verbatim]: their NBER working paper, *Generative AI at Work*
- Domains: bls.gov, danielle.li, inigomedina.co, linkedin.com, mit.edu, nber.org, nngroup.com
- `provenance_error`: none
- Coder note: Seven domains, none of them the QJE. Reports the preprint's agent count.

**Q4 · perplexity · run 5**

- Figure [PARAPHRASE]: about 14% on average; about 34% for novices
- Named as the source [verbatim]: their NBER working paper, **"Generative AI at Work"**
- Domains: bls.gov, danielle.li, inigomedina.co, linkedin.com, mit.edu, nber.org
- `provenance_error`: none
- Coder note: Third consecutive preprint answer with a near-identical citation set.

## Q5

| System | Run | figure_given | source_named | source_class | cit | uniq | int_share | contra | version | search |
| --- | ---: | --- | --- | --- | ---: | ---: | --- | --- | --- | --- |
| chat_gpt | 3 | yes | no | tertiary | 2 | 2 |: | no | preprint+ | yes |
| claude | 3 | yes | yes | primary | 6 | 3 |: | no | preprint | yes |
| perplexity | 3 | yes | yes | tertiary | 13 | 9 |: | no | preprint | yes |
| gemini | 3 | yes | yes | primary | 12 | 7 |: | no | preprint+ | yes |
| chat_gpt | 1 | yes | no | tertiary | 2 | 2 |: | no | preprint+ | yes |
| chat_gpt | 4 | yes | no | tertiary | 2 | 1 |: | no | preprint+ | yes |
| claude | 1 | yes | yes | primary | 3 | 3 |: | no | preprint | yes |
| claude | 4 | yes | yes | primary | 6 | 2 |: | no | preprint | yes |
| gemini | 1 | yes | yes | primary | 7 | 6 |: | no | preprint | yes |
| gemini | 4 | yes | yes | primary | 33 | 10 |: | no | preprint | yes |
| perplexity | 1 | yes | yes | tertiary | 18 | 13 |: | no | preprint | yes |
| perplexity | 4 | yes | yes | tertiary | 18 | 13 |: | no | preprint | yes |

**Q5 · chat_gpt · run 3**

- Figure [PARAPHRASE]: 12.2% more tasks; 25.1% faster; 40.2% higher quality; 23% worse
- Named as the source [verbatim]: A Boston Consulting Group (BCG) study involving 758 consultants
- Domains: bcg.com, jobcannon.io
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] 40.2% appears in neither the preprint nor the version of record. Traced to jobcannon.io, a career-assessment site, which is also the cited source here.
    - [routed] 23% is BCG's own marketing figure; the paper reports 19 percentage points.
- Coder note: Second counted run at 40.2%. The paper is never named in any version. Two slots, two domains: a career-quiz site and the commissioning sponsor.

**Q5 · claude · run 3**

- Figure [PARAPHRASE]: 12.2% more tasks; 25.1% quicker; 40% higher quality; 19% less likely
    - the record says: “he BCG study, conducted in collaboration with Harvard Business School and other institutions, found several key performance improvements when consultants used GPT-4:

Consultants who used GPT-4 completed on average 12.2 percent more tasks, 25.1 percent quicker, and produced 40% higher quality work compared to those without AI access”
- Named as the source [verbatim]: the working paper titled "Navigating the Jagged Technological Frontier" published on September 22, 2023
- Domains: aibusiness.com, linkedin.com, thecrimson.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] '750 Boston Consulting Group consultants' against the true 758, cited to the same LinkedIn Pulse post as in counted runs 1 and 4.
    - [routed] 40% quality figure was removed in peer review; the published paper reports 33.9% and 29.9%.
- Coder note: Names the preprint correctly, including its date. Drops the Crimson headcount conversion ('40 percent of the trial group') seen in counted runs 1 and 4 and gives the magnitude form instead. Six slots, three domains, none academic.

**Q5 · perplexity · run 3**

- Figure [verbatim]: 12.2% more tasks; 25.1% faster; about 40% higher quality; 23% worse
- Named as the source [verbatim]: The original source is BCG's report *How People Create and Destroy Value with Generative AI*
- Domains: bcg.com, facebook.com, linkedin.com, medium.com, mi-3.com.au, thecrimson.com, themelan.com, venturebeat.com, youtube.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] Names the commissioning sponsor's own marketing publication as the original source of its own experiment.
    - [routed] 23% is BCG's figure; the paper reports 19 percentage points.
    - [routed] 40% was removed in peer review.
- Coder note: Second counted run naming BCG's own publication as the original source. Sponsor substitution is the stable behaviour of this stack on this question.

**Q5 · gemini · run 3**

- Figure [PARAPHRASE]: 12.2% more tasks; 25.1% faster; 40.2% higher quality; 758 consultants
- Named as the source [verbatim]: a Harvard Business School Working Paper titled "Navigating the Jagged Technological Frontier: …" by Dell'Acqua et al., published in 2023
- Domains: bcg.com, innovationtraining.org, jobcannon.io, mi-3.com.au, northernlight.com, qualitydigest.com, youtube.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] 40.2% appears in neither version of the paper. jobcannon.io, checked at source 2026-07-27, states '…produced results rated 40.2% higher in quality…' and is this run's top citation slot.
- Coder note: Run 3 returns 40.2%: the fabricated decimal: with jobcannon.io in its top citation slot, where runs 1 and 4 both return 40%. Coded as a single occurrence, NOT as the decimal crossing stacks: run 4 went back to 40% with jobcannon.io still in the citation set. One in three is a fluctuation. Names the preprint correctly and cites nothing academic.

**Q5 · chat_gpt · run 1**

- Figure [PARAPHRASE]: 40.2% higher quality; 23% worse
- Named as the source [verbatim]: a study conducted by the Boston Consulting Group
- Domains: bcg.com, jobcannon.io
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] 40.2% appears in neither version of the paper; cited to jobcannon.io, which states '…rated 40.2% higher in quality…': checked at source 2026-07-27.
    - [routed] 23% is BCG's marketing figure; the paper reports 19 percentage points.
- Coder note: Two slots, two domains: a career-assessment quiz site and the commissioning sponsor. The paper is never named.

**Q5 · chat_gpt · run 4**

- Figure [PARAPHRASE]: 12.2% more tasks; 25.1% faster; 40.2% higher quality; 23% decline
- Named as the source [verbatim]: A Boston Consulting Group (BCG) study, in collaboration with Harvard Business School, MIT Sloan, and Wharton
- Domains: bcg.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] 40.2% appears in neither version, and this run cites BCG's own PDF for it, a document that states 40%.
    - [routed] 23% is BCG's marketing figure; the paper reports 19 percentage points.
- Coder note: Third counted run at 40.2%, and the second time the decimal is attributed to a BCG document that does not contain it. Both slots resolve to bcg.com. The 'Harvard Business School, MIT Sloan, and Wharton' attribution is broadly right: Mollick is at Wharton, Kellogg at MIT, Lifshitz-Assaf at Warwick, and is NOT coded as an error.

**Q5 · claude · run 1**

- Figure [PARAPHRASE]: "40 percent of the trial group produced higher quality results"; 19%
- Named as the source [PARAPHRASE]: a working paper titled *Navigating the Jagged Technological Frontier*, September 22, 2023
    - the record says: “Original Source: The study was published in a working paper titled "Navigating the Jagged Technological Frontier" on September 22, 2023, by Harvard Business School researchers who conducted an experimental study of GPT-4 use by 750 Boston Consulting Group consultants.”
- Domains: aibusiness.com, linkedin.com, thecrimson.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] '40 percent of the trial group' converts a magnitude into a headcount: a claim about a different quantity. The Harvard Crimson states exactly this, verbatim; the error is the newspaper's and is reproduced faithfully.
    - [routed] 40% was removed in peer review.
    - [not_established] '750 Boston Consulting Group consultants' against the true 758, cited to a LinkedIn Pulse post: the same post cited in counted runs 3 and 4.
- Coder note: Three slots, three domains, none academic. The Crimson headcount conversion appears in counted runs 1 and 4 (2 of 3); the separate 750-vs-758 count error appears in all three, cited to the same LinkedIn Pulse post each time.

**Q5 · claude · run 4**

- Figure [PARAPHRASE]: 12.2% more tasks; 25.1% quicker; "40 percent of the trial group produced higher quality results"; 19% less likely
- Named as the source [verbatim]: a working paper titled **"Navigating the Jagged Technological Frontier"** published by Harvard Business School researchers on September 22, 2023
- Domains: linkedin.com, thecrimson.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] The Crimson headcount conversion, reproduced again.
    - [not_established] '750 Boston Consulting Group consultants' against the true 758, cited to the same LinkedIn Pulse post as runs 1 and 3.
    - [routed] 40% was removed in peer review.
- Coder note: The headcount conversion returns after being absent in run 3, so it is the majority behaviour of this cell (2 of 3), not a one-off. Five of six slots are the Crimson.

**Q5 · gemini · run 1**

- Figure [PARAPHRASE]: 40% higher quality
- Named as the source [verbatim]: a working paper … published by Harvard Business School. The lead author … is Fabrizio Dell'Acqua
- Domains: hindustantimes.com, innovationtraining.org, jobcannon.io, legaldive.com, mi-3.com.au, venturebeat.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] 40% was removed in peer review; the published paper reports 33.9% and 29.9%.
- Coder note: Names the working paper and cites none of it: six domains, all trade press, vendor or national newspaper.

**Q5 · gemini · run 4**

- Figure [PARAPHRASE]: 12.2% more tasks; 25.1% faster; 40% higher quality; 758 consultants
- Named as the source [verbatim]: "Navigating the Jagged Technological Frontier: Field Experimental Evidence…" was published on September 22, 2023
- Domains: bcg.com, hindustantimes.com, innovationtraining.org, jobcannon.io, legaldive.com, mi-3.com.au, qualitydigest.com, shelf.io, substack.com, venturebeat.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [routed] 40% was removed in peer review.
- Coder note: Back to 40% after run 3's 40.2%, so the fabricated decimal did NOT stick on this system. jobcannon.io is still in the citation set. 33 slots resolving to 10 domains: the largest repetition-as-breadth gap in the study.

**Q5 · perplexity · run 1**

- Figure [PARAPHRASE]: 40% higher quality; 23% worse
- Named as the source [PARAPHRASE]: The original BCG source is **How People Create and Destroy Value with Generative AI** from Boston Consulting Group
    - the record says: “The original BCG source is **“How People Create and Destroy Value with Generative AI”** from Boston Consulting Group.”
- Domains: bcg.com, facebook.com, forbes.com, hbs.edu, innovationtraining.org, jobcannon.io, linkedin.com, medium.com, mi-3.com.au, thecrimson.com, themelan.com, venturebeat.com, youtube.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] Names the commissioning sponsor's own marketing publication as the original source of its own experiment.
    - [routed] 23% is BCG's figure; the paper reports 19 percentage points.
    - [routed] 40% was removed in peer review.
- Coder note: Eighteen slots, thirteen domains, zero academic.

**Q5 · perplexity · run 4**

- Figure [PARAPHRASE]: 12.2% more tasks; 25.1% faster; about 40% higher quality; 23% drop
    - the record says: “he BCG study found that consultants using **GPT-4** did **12.2% more tasks**, worked **25.1% faster**, and produced work rated **about 40% higher in quality** than the control group on the creative/product-innovation task.[1][3]

For tasks *outside* GPT-4’s capability frontier, the same consultants performed worse: BCG”
- Named as the source [verbatim]: The original source is BCG's publication **"How People Create and Destroy Value with Generative AI"**
- Domains: bcg.com, facebook.com, forbes.com, hbs.edu, innovationtraining.org, linkedin.com, medium.com, mi-3.com.au, reddit.com, thecrimson.com, themelan.com, venturebeat.com, youtube.com
- `provenance_error` (route: routed = a cited document is established to contain it):
    - [not_established] Third counted run naming the sponsor's marketing publication as the original source.
    - [routed] 23% is BCG's figure; the paper reports 19 percentage points.
    - [routed] 40% was removed in peer review.
- Coder note: Reddit and Facebook appear in the citation set; hbs.edu appears via an HBS AI-institute blog post, not the working paper. Still zero links to Organization Science.