Table 5From “What is an A/B test? randomized controlled trials, statistical power, and governance”
The Experimentation RACI Matrix
Table 5The Experimentation RACI Matrix
| Experimentation Lifecycle Milestone | Product Manager / Growth Lead | Lead Data Scientist / Statistician | Software Engineering Lead | Executive Decision Committee |
|---|---|---|---|---|
| Problem Identification & Hypothesis Formulation | Accountable | Consulted | Consulted | Informed |
| Metric Selection & OEC Definition | Accountable | Responsible | Consulted | Informed |
| Prospective Power Calculation & Sizing Signoff | Consulted | Accountable | Informed | Informed |
| Variant Implementation & Code Deployment | Informed | Consulted | Accountable | Informed |
| Pre-Flight QA & A/A Instrumentation Verification | Consulted | Responsible | Accountable | Informed |
| Daily SRM Monitoring & Health Surveillance | Informed | Accountable | Responsible | Informed |
| Statistical Result Synthesis & FDR Correction | Consulted | Accountable | Informed | Informed |
| Rollout / Rollback Production Decision | Accountable | Consulted | Responsible | Accountable |
| Code Cleanup & Feature Flag Retirement | Informed | Informed | Accountable | Informed |
| Institutional Knowledge Base Codification | Accountable | Responsible | Consulted | Informed |
Swipe or scroll horizontally if the table is wider than your screen.
Source: Table from this essay. Sources and interpretation are given in the article.

From the essay
From the research bench
What is an A/B test? randomized controlled trials, statistical power, and governance
An A/B test is a randomized experiment comparing two concurrent variants to measure the causal impact of a single change on commercial performance.
Read the essayThis exhibit belongs to an essay: the numbers live here, the argument lives there.
Cite or embed
Related exhibits

Figure 2Figure
Taxonomy of Experimentation Architectures
What is an A/B test? randomized controlled trials, statistical power, and governance

Table 4Table
How do you audit your own experimentation programme?
What is an A/B test? randomized controlled trials, statistical power, and governance

Figure 1Figure
The A/B test experimental architecture
What is an A/B test? randomized controlled trials, statistical power, and governance
Contact
Bring the question you're working on.
Research, a guest lecture, or a commercial question.