Marketing mix modeling is a calibration problem

Marketing mix modeling is a calibration problem, not a dashboard problem. It can organise an aggregate budget decision, show how demand and spend moved together, and expose a set of scenarios. It cannot certify causal lift just because the regression is sophisticated.

The distinction is easy to lose because every instrument returns a number. A multi-touch attribution model assigns credit to observed touches. A marketing mix model relates aggregate outcomes to aggregate inputs. An incrementality test asks what changed relative to a counterfactual. The numbers look comparable only after the question has been silently changed.

Which three distinct measurement questions do attribution, experimentation, and MMM answer?

The first discipline is to name the output before choosing the model. If the decision is whether a channel created demand that would not otherwise have existed, the model must contain a credible counterfactual. If the decision is how to plan a portfolio under several demand scenarios, an aggregate model may be the right instrument even when it cannot identify causal lift on its own.

This is the narrower claim behind the incrementality illusion. The problem is not that an observed association is useless. It is that an organisation starts making a counterfactual decision from an instrument that never observed the counterfactual.

Why do marketing mix models require continuous experimental calibration against lift tests?

An aggregate model sees time. Time carries seasonality, promotions, product launches, macroeconomic shocks and the company’s own decision to spend more when demand looks promising. The model must separate those movements with data and assumptions that are never perfectly observed.

The transformations matter too. Adstock, saturation, lag and interaction terms are not decoration. They are claims about how exposure becomes demand. Different assumptions can produce different allocations while fitting the past in a similar way.

Calibration is the discipline that keeps this from becoming a debate about whose chart looks more plausible. It uses experiments, external shocks, stable control series, known pricing changes or other observations to constrain the model. It does not make the model experimental. It tells the reader which parts of the model have an anchor outside the model.

Gordon and colleagues¹ show why this matters when different advertising measurement approaches are compared against field experiments. Blake, Nosko and Tadelis show how paid search can capture inframarginal buyers. Lewis and Rao³ show how noisy outcomes make return measurement expensive. The lesson is not to discard every model. It is to stop treating model fit as proof of lift.

How can commercial leadership apply MMM coefficients without econometric overreach?

Start with the portfolio question. Which channels need a directional bound? Which channels have enough scale for a designed holdout? Which decisions are defensive and should be governed by a budget ceiling rather than a claim of incremental demand?

Then show the range. A model that produces one allocation without a sensitivity view has hidden its uncertainty. Vary the baseline, the carryover and the saturation assumptions. If the recommended portfolio changes when a reasonable assumption moves, the decision is sensitive to the assumption. That is an output worth knowing.

The next step is to connect the model to a test. A holdout can calibrate a channel or a family of channels. It may not transfer to every market or period, but it gives the aggregate model an external point. The evidence over anecdote worksheet is useful here: write down what would count as evidence before the allocation is shown.

Finally, separate the model’s scenario from the operating decision. The model can say that a portfolio looks better under one set of assumptions. Leadership still has to decide how much uncertainty it is willing to buy, what cash payback is acceptable and which learning matters more than this quarter’s apparent efficiency.

Why is executive econometric governance distinct from dashboard monitoring?

The dashboard is a view. Governance is the rule that says what happens when the view conflicts with a test, a margin threshold or a customer constraint. The funnel bottleneck nobody’s measuring shows the same pattern outside advertising: the visible metric is often the output, while the unmeasured interval decides whether the output can be trusted.

A marketing mix model is worth keeping when it makes assumptions explicit, supports scenarios, accepts outside calibration and changes a decision. It is not worth keeping as a decorative source of precision. The right question at the next review is not “What did the model say?” It is “Which part of the model has been tested, which part is a scenario, and what decision is allowed to rely on each?”

How should marketing measurement governance set owners, thresholds, and escalation rules?

Governance turns a calibrated model into a decision instrument only when ownership, thresholds, and escalation are written before the next conflict arrives. Use the following rules when attribution, MMM, and lift tests disagree:

1. Name the owner for each instrument. Marketing analytics owns the model run and the test design; the budget owner owns the spend decision; finance owns recognition and contribution boundaries. Do not leave “the dashboard” as the unnamed owner.
2. Separate thresholds by question. Set one threshold for when a lift test must recalibrate the model, one for when a scenario may change budget, and one for when a result is display-only. Do not apply a single ROAS cutoff to every instrument.
3. Escalate on conflict, not on preference. If a test contradicts the model, or the model contradicts the P&L contribution boundary, escalate with both results visible and the decision frozen until the conflict is named. Do not average the conflict into one executive score.
4. Record what may rely on each part. State which decisions may use historical fit, which may use calibrated lift, and which remain scenario-only. Do not let an untested coefficient authorize a permanent reallocation.
5. Review on a cadence with an exit. Schedule recalibration and holdout reviews, and name who can retire a model that no longer changes decisions. Do not keep a decorative model because the dashboard still looks complete.

These rules stay inside the article’s evidence limits. They add no private budget threshold, universal ROAS target, or portable uplift. The calibration and governance sections above remain the operating instruments.