← Every exhibit
Table 1From “Survey bias is a decision error before it is a questionnaire flaw”
Survey bias pathway map
Trace the threat to the decision before choosing a sample-size or wording fix.
Table 1Survey bias pathway map
| Path | Threatened object | Possible distortion | Evidence check | Decision remains unresolved when |
|---|---|---|---|---|
| Frame | Population and eligibility | Relevant units are absent or the wrong units are included | Compare frame, reachable population, and decision population | The inclusion boundary is not recorded |
| Response | Who is visible | Responders differ from nonresponders on relevant experience or outcome | Record reachability, timing, response, partial completion, and known differences | Nonresponse direction is assumed |
| Wording | Interpretation of the item | Framing, recall, or social desirability changes the answer | Pilot alternatives and inspect item meaning in the decision context | The construct is not defined |
| Measurement | Construct and scale | A reliable response measures the wrong object or period | Name construct, scale, reference period, and validity check | The measure cannot be tied to the decision |
| Interpretation | From answer to conclusion | A pattern is treated as population, causal, or predictive evidence | Separate description, association, prediction, and causation | The inferential step is hidden |
| Decision | Action and threshold | Evidence is collected without a rule for what changes | Name action, threshold, comparison, and follow-up outcome | No action or outcome is specified |
Swipe or scroll horizontally if the table is wider than your screen.
Source: Author's framework grounded in Guest, Bunce, and Johnson (2006), Hennink, Kaiser, and Marconi (2017), Hagaman and Wutich (2017), Malterud, Siersma, and Guassora (2016), and Schoonenboom and Johnson (2017). This is an evidence review, not a coded survey dataset.
PNG4:3, 1600 × 1200, with the title, the source line and the attribution set in the image isoglu-2026-survey-bias-is-a-decision-error-fig01.png
Source and evidence
Each line names a claim related to this exhibit, its source, and how it was checked.
- B GUEST06-C1 The number is conditional on that study's design: "they found that saturation occurred within the first twelve interviews, although basic elements for metathemes were present as early as six interviews"Guest, Bunce & Johnson. (2006)
- B GUEST06-C2 The 12-interview result is conditional and not a universal rule, and the authors bound it themselves: saturation occurred within the first twelve interviews, "at least based on the codebook we devel"oped, in a study of relatively homogeneous respondentsGuest, Bunce & Johnson. (2006)
- B HENN17-C1 Two different closure questions, named: "Our study compared two approaches to assessing saturation: code saturation and meaning saturation", and "we found that code saturation was reached at nine interviews" while meaning saturation took longerHennink, Kaiser & Marconi. (2017)
- B HENN17-C2 Code saturation and meaning saturation answer different questions, and the paper reports different sample sizes for each: more interviews "were needed to reach meaning saturation where we developed a richly textured understanding of issues", and the authors present the finding as conditions rather than a number, since "We used our results to develop parameters that influence saturation"Hennink, Kaiser & Marconi. (2017)
- B HAG17-C1 In a four-site cross-cultural study, larger samples were needed for metathemes: the purpose was "to reach data saturation for metathemes in multisited and cross-cultural research", the analysis "Analysis of the data yielded 240 site-specific themes and nine cross-cultural metathemes", and the authors report larger sample sizes "ranging from 20 to 40 interviews"Hagaman & Wutich. (2017)
- B HAG17-C2 Homogeneity is what moves the number: "We found that 16 or fewer interviews were enough to identify common themes from sites with relatively homogeneous groups"Hagaman & Wutich. (2017)
- B MAL16-C1 Information power is defined by five items, verbatim: "information power depends on (a) the aim of the study, (b) sample specificity, (c) use of established theory, (d) quality" of dialogue, and (e) analysis strategyMalterud, Siersma & Guassora. (2016)
- B MAL16-C2 Higher information power can support a smaller sample, but the concept is an appraisal and not a formula: "Information power indicates that the more information the sample holds, relevant" for the study, the lower the number of participants neededMalterud, Siersma & Guassora. (2016)
- B SCHOON17-C1 The design dimensions are the authors' own list: "We explain the seven major design dimensions: purpose, theoretical drive, timing (simultaneity and dependency), point of integration, typological versus interactive design approaches, planned versus emergent design, and design complexity"Schoonenboom & Johnson. (2017)
- B SCHOON17-C2 A supplemental strand still owes its own quality criteria, and the authors argue exactly that: "the idea of decreased rigor is in conflict with one key theme of the present article, namely that mixed methods designs should always meet the criterion of multiple validities legitimation"Schoonenboom & Johnson. (2017)
- B B10-C1 Survey bias is actionable only when the threatened decision, population boundary, measurement object, and interpretation step are named togetherAuthor framework grounded in the methods sources above
- B B10-C2 A sample-size rule cannot repair a frame, nonresponse, wording, construct, or interpretation problem that was never specifiedAuthor framework grounded in Guest, Hennink, Hagaman, Malterud, and Schoonenboom
Grades: A, verified against the printed page of the primary source · B, primary source, text layer only · C, authoritative secondary · D, reported.
From the essay
From the research bench
Survey bias is a decision error before it is a questionnaire flaw
Survey bias becomes a business risk only when its path to a named decision, population, measure, and interpretation can be shown and checked.
Read the essayThis exhibit belongs to an essay: the numbers live here, the argument lives there.
Cite or embed
Contact
Bring the question you're working on.
Research, a guest lecture, or a commercial question.