← Every exhibit
Guardrail Metric Verification and Downstream Economic Translation
| Metric Category | Metric Name | Control (Variant A) | Treatment (Variant B) | Statistical Verdict | Operational Status |
|---|---|---|---|---|---|
| Primary OEC | 21-Day Checkout Conversion Rate | 2.500% | 2.900% | $p = 0.0025$, Lift: +16.0% | Statistically Significant WIN |
| Guardrail 1 | Page Load Latency (p95) | 480 ms | 465 ms | $p = 0.12$, Neutral (-15ms) | PASS: No latency penalty |
| Guardrail 2 | Credit Card Gateway Decline Rate | 3.20% | 3.15% | $p = 0.74$, Neutral | PASS: Payment hygiene intact |
| Guardrail 3 | Onboarding Ticket Escalation Rate | 4.80% | 5.10% | $p = 0.41$, Neutral | PASS: Support queue stable |
| Downstream 1 | Day-30 User Activation Rate | 64.2% | 63.8% | $p = 0.68$, Neutral | PASS: Healthy cohort quality |
Swipe or scroll horizontally if the table is wider than your screen.
Reference & Evidence
Source: Table from this essay. Sources and interpretation are given in the article.
Related exhibits
-
The Metric Hierarchy in Controlled Experiments
From the essay What is an A/B test? randomized controlled trials, statistical power, and governance
-
The A/B test experimental architecture
From the essay What is an A/B test? randomized controlled trials, statistical power, and governance
-
Decision Errors and Statistical Power
From the essay What is an A/B test? randomized controlled trials, statistical power, and governance