Table 24 Sept 2026From “What is an A/B test? randomized controlled trials, statistical power, and governance”

The Metric Hierarchy in Controlled Experiments

Table 2The Metric Hierarchy in Controlled Experiments

Metric Class	Operational Purpose	Primary Examples	Decision Rule in Experimentation
1. Overall Evaluation Criterion (OEC)	The unified strategic objective function that defines organizational success	Composite score balancing 30-day net customer retention and annualized margin contribution	Primary criterion for declaring experimental victory and rolling out code to production
2. Direct Driver Metrics	Local transactional metrics measuring immediate behavioral response	Call-to-action click rate, checkout form start rate, onboarding completion rate	Diagnostic indicators to understand why the treatment impacted the primary OEC
3. Guardrail / Health Metrics	Invariant system and business health indicators that must not be degraded	Server latency (p99), checkout error rate, customer support ticket volume, refund rate	Non-negotiable veto power: if any guardrail metric breaches thresholds, the test is aborted
4. Long-Term North Star Metrics	Macro-economic business metrics tracking durable shareholder value	Customer Lifetime Value (LTV), Net Revenue Retention (NRR), brand equity indices	Evaluated retrospectively across long-term holdout cohorts to verify sustained impact

Swipe or scroll horizontally if the table is wider than your screen.

Source: Table from this essay. Sources and interpretation are given in the article.

PNG4:3, 1600 × 1200, with the title, the source line and the attribution set in the image isoglu-2026-what-is-an-ab-test-fig04.png

From the essay

From the research bench

What is an A/B test? randomized controlled trials, statistical power, and governance

An A/B test is a randomized experiment comparing two concurrent variants to measure the causal impact of a single change on commercial performance.

4 Sept 2026

Read the essay

This exhibit belongs to an essay: the numbers live here, the argument lives there.

Cite or embed

APA 7

Isoglu, S. (2026). The Metric Hierarchy in Controlled Experiments [Table]. isoglu.com. https://isoglu.com/insights/exhibits/isoglu-2026-what-is-an-ab-test-fig04/

BibTeX

@misc{isoglu-2026-what-is-an-ab-test-fig04,
  author = {Isoglu, Sinan},
  title = {{The Metric Hierarchy in Controlled Experiments}},
  year = {2026},
  howpublished = {Table 2, isoglu.com},
  url = {https://isoglu.com/insights/exhibits/isoglu-2026-what-is-an-ab-test-fig04/},
  note = {From the essay {What is an A/B test? randomized controlled trials, statistical power, and governance}}
}

Embed this exhibit

Copy this HTML to use the PNG exhibit on your site with its source line and a link back to the exhibit page.

<figure>
  <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-an-ab-test-fig04/">
    <img src="https://isoglu.com/social/isoglu-2026-what-is-an-ab-test-fig04.png" alt="Table 2: The Metric Hierarchy in Controlled Experiments" width="3200" height="2400">
  </a>
  <figcaption>Table 2: The Metric Hierarchy in Controlled Experiments. Source: Table from this essay. Sources and interpretation are given in the article. <a href="https://isoglu.com/insights/exhibits/isoglu-2026-what-is-an-ab-test-fig04/">View exhibit</a>.</figcaption>
</figure>