Measuring what a channel caused

Why this exists

What clients ask next

Our companion page, Creating demand, or collecting it, makes the case that a borrowed incrementality figure says little about your own account. Clients who accept that argument ask two things next: how to measure their own number, and whether a media mix model would do the job instead.

Both have become easier to buy. Google released a free geo testing library in September 2026 that feeds results into its own mix model, and in November 2025 it cut the minimum spend for a platform incrementality study to $5,000.¹ Two steps have not become easier. One is designing a test that can detect the effect you care about. The other is getting the result into the numbers people use to make budget decisions. This page is about both.

1.1 The right tool

Which method answers which question

Each measurement method answers a narrower question than its sales pitch suggests. The table pairs common budget questions with the method that answers them.

Platform lift studies split people into groups rather than markets, so they reach a clear answer on far fewer conversions. Google’s documentation sets out the trade. A geo test counts every conversion in a region, whether the platform tracked it or not, which makes it noisier and usually more expensive. In return it captures in-store sales and CRM outcomes, compares results across publishers, and works for any channel that can be targeted by region.⁴

1.2 Mix models

Where mix models fall short

A media mix model estimates how sales respond to spend in each channel, after accounting for seasonality, price, promotions and the economy. It covers every channel at once, television and out-of-home included, and it needs no user-level tracking, so privacy changes do not weaken it. Three problems limit what it can prove.

It works from too little data

A typical model is built on three years of weekly national data, which is 156 data points, often spread across 20 or more channels. Each channel needs three or four settings to describe how long its effect lasts and where its returns flatten, so the model can end up with more unknowns than observations.² Google’s own researchers found that with samples this small, the assumptions built into the model drive much of the answer.⁵

Spend follows demand

Budgets rise into peak season, when sales would rise anyway. Search and retargeting ads reach people who already intended to buy, and television prompts searches that then raise search spend. In each case spend moves together with sales that would have happened anyway, and the model credits the spend.²

A 2026 simulation put a size on the problem. With the true answer known, a standard model reported paid search ROAS of 10.61× against a true 4.20×. Given information about demand that no advertiser would have, it still reported 8.41×. Estimated from four geo tests instead, it came out at 4.14×.³ The study used simulated data, so the size of the gap belongs to that simulation. The direction matches what large field experiments at Meta found when they checked modeled estimates against randomized ones.

A good fit can still be wrong

In a Google example, five equally plausible models fitted the same data with an R² between 0.98 and 0.99. They differed by up to 50% in predicted sales and disagreed on how to fund the largest channel.² Researchers at Wharton and London Business School showed that diminishing returns and changing effectiveness are often impossible to tell apart in mix model data, and that the two readings recommend different budgets.⁶ In both cases, a test that changes spend on purpose settles the question.

Where the two methods meet

Near current spend, a mix model estimates the return on the next dollar reasonably well. Their estimate of average return is weak, because it requires guessing what would happen at zero spend.² A switch-off test measures that average return directly, so each method covers a gap in the other.

1.3 When to buy a model

When a mix model is worth it

A mix model earns its cost when splitting budget across several channels is the main question, offline media is a large share of spend, two or more years of weekly data with enough variation in spend exist, and there is a plan to run the tests that check it.

Without tests behind it, a model produces confident advice that nobody can check. A business running on two or three digital channels usually learns more from a series of geo tests on its largest channel.

That second route is our default. We build a matrix of tested incremental shares, one per channel, and set bids and budgets from it. Every number in the matrix traces to a test the client can inspect, where a mix model’s output rests on assumptions few people outside the modeling team can check. An incrementality matrix you can bid from shows the approach on two client accounts.

Google’s Meridian calibrates against tests by taking a test result and its margin of error as the starting estimate for that channel, and its 2026 release also suggests which channels most need a test.^{7, 1} Meta’s Robyn scores candidate models down when they disagree with test results.⁸ One caution applies to both. A test reported as a single number tells the model little about how fast the effect fades or where returns flatten. Tests run at different spend levels give it far more to work with.³

2.1 The question

Four test designs

A matched-market test compares markets where advertising changed with similar markets where it did not. The change you make decides what you learn.

The gap between average return and next-dollar return is the one most often missed. A channel’s first dollars usually reach the easiest customers. A switch-off result can look healthy while the last dollars in the budget buy little, so it should not be used on its own to justify scaling. Research on advertising experiments reaches the same conclusion: a standard test shows whether current spend works, and deciding how much to spend takes a design with more than one spend level.⁹

Agree the decision rule before the test, in writing. For example: if incremental ROAS on a heavy-up beats 2.0 and the low end of its range beats 1.0, raise the national budget 20%. If the range includes break-even, hold. If even the top of the range sits below break-even, cut. The figures are illustrative. Agreeing them up front fixes the smallest effect the test has to detect, and it settles what each result will mean before anyone sees one.

2.2 Markets

Choosing test and control markets

The control markets stand in for what the test markets would have done without the change. Before launch, check how well they do that on history they were not fitted to.

To check, build the comparison on one stretch of past data and see how well it predicts a later stretch.¹⁰ Google’s geo testing tool recommends a fit score (R²) of 0.8 or higher on this check, and treats anything under 0.5 as unreadable.⁴ Simple correlation between markets is a weak substitute. Two markets that both grow through the year will correlate whether or not they move together week to week.

• Keep the largest markets out of the test group. The comparison is a weighted blend of control markets, and no blend of smaller markets can reproduce New York or Los Angeles.^{10, 11}
• Leave out control markets that share commuters, broadcast signal or delivery areas with the test markets. Spillover makes the control group behave like the test group and shrinks the measured effect.¹²
• Use enough markets. Google recommends at least 10 for a simple test and 50 or more for its preferred way of assigning them.⁴

Write down the markets, dates and analysis method before launch. A result is credible only if nobody could have chosen the method after seeing the numbers.¹⁰

2.3 Test size

Sizing the test

The smallest lift a test can detect depends on how closely the control markets track the test markets when nothing has changed. If the two usually differ by 7% in a given week, a 3% lift disappears into that gap.

Budget does not move that limit. It moves the size of the lift your spend can produce, and so whether the lift clears it.⁴ Measuring the normal gap on the same history used to build the comparison makes it look smaller than it is. Google’s 2026 library switched to measuring it on held-out history for that reason, and its recommended test budgets went up as a result.⁴ The calculator below runs a simplified version of that check.

Reading the calculator

The three scenarios change one thing at a time. The small program cannot detect the lift its own spend would plausibly produce. The mid-size program at four weeks has slightly better than even odds, which is a poor basis for a decision the business will treat as settled. Running the same weekly spend for eight weeks makes the answer reliable.

When no realistic combination works, skip the test and put the money into media until volume supports one.

2.4 Running it

Common leaks and their fixes

Most contamination in geo tests is self-inflicted, and much of it comes from platform defaults.

Run the test for at least one full purchase cycle, then allow one to two weeks of cooldown to catch late conversions.^{11, 4} Set the end date in advance and keep to it. Checking results along the way and stopping when they look good produces false positives far more often than the stated confidence level suggests.¹⁴ During the test, watch delivery, leakage and data quality, and leave the results alone until the end date.

3.1 The baseline

Building the baseline

Every readout works the same way. Predict what the test markets would have done without the change, compare that prediction with what they did, and add up the difference over the test window.

Analysts call the prediction the counterfactual. Most tools build it as a weighted blend of control markets, chosen so the blend mirrors the test markets’ history, a method known as synthetic control.¹⁵ Meta’s GeoLift and Google’s GeoX both automate it.^{16, 11}

When reviewing a result, start with how closely the baseline tracked the test markets before launch. If the two lines in a chart like the one below did not sit together for months beforehand, the gap after launch carries little information.

3.2 Confidence

How confident to be in a result

The most common error in reading a geo test is treating each customer as a separate data point.

Thousands of customers make a result look precise. The test assigned markets, though, and customers in the same market move together. A local news story, a regional promotion or a storm affects all of them at once. The effective sample size is much closer to the number of markets than to the number of customers.

The standard check is a placebo test. Treat each control market in turn as if it had been the test market, rebuild its baseline, and record how large a gap appears when nothing happened. The test result counts only if it stands out from those.¹⁰ This sets a hard limit on small designs. With one test market and fewer than 19 control markets, no result can reach 95% confidence, however large the effect.

Some geo tools default to 90% confidence rather than 95%, Google’s GeoX among them.¹ For a budget decision that can be reversed, that is a reasonable trade, provided it is chosen before the test.

3.3 The return

From lift to return

Two numbers matter to the budget. Incremental ROAS is the extra revenue the test caused, divided by the extra spend it took.¹² The incremental share is the extra conversions divided by the conversions the platform claimed in the same markets and weeks, which tells you how much of the platform’s figure to believe.

A common mistake divides incremental conversions by all conversions in the test markets. That ratio is the lift, the share of total business the channel caused, and it is small for any channel running alongside others. Read as an incrementality rate, a healthy 10% lift makes a channel look as if 90% of its reported conversions were false.

The platform reports $131 per conversion. Each extra conversion cost $250. Until the channel is tested again, its reported conversions should be multiplied by 0.52 in reporting.

Report the range as well as the midpoint. With the calculator’s default settings, the lift could plausibly sit anywhere from about 3.7% to 16.3%, which puts the cost per incremental conversion between about $153 and $682. The midpoint of $250 sits near the bottom of that range. A result at the low end of the lift range more than doubles the cost.

4.1 Data

The four data feeds

A geo test joins the business’s own sales or lead data with each ad platform’s spend, market by market. Four feeds need to exist before anyone picks markets.

The outcome feed usually takes longest to build, so start it first. Once a test is running, check that feed against a second source, such as payment processor totals, every week. A tracking tag that stops working in one region opens a gap between the groups that looks like a result.

4.2 Reporting

Putting the result into weekly reporting

A test changes decisions only if its result shows up where decisions are made, which for most teams is the weekly report.

1. Show both numbers. Multiply each channel’s reported conversions by its incremental share, and put the adjusted cost per conversion next to the platform’s.
2. Set bid targets from the adjusted number. Automated bidding hits its target on the platform’s own count, so if half of those conversions would have happened anyway, the account can hit target and lose money. The formula below gives the lowest target that still breaks even.
3. Date every adjustment. Each incremental share is measured at one spend level in one season. Re-measure when spend, creative strategy, competition or attribution settings change. A change to the platform’s attribution window makes the old share invalid.

In that example, any platform ROAS target below 6.4× loses money once the conversions that would have happened anyway are removed. The companion page works through the same arithmetic in more detail.

Media should not wait for a test before it is judged. Launch it, read it on the best numbers available, and correct those numbers as test results come in. Holding spend until a result arrives costs months, and a new channel has no history to test against anyway.

Test cadence

With a mix model, test on the model’s refresh cycle: refresh it, find the channels with the most spend and the least certainty, test one or two, and feed the results back in. Without a model, start with the largest channel, carry its incremental share into reporting, and test again when conditions change. That is the path we run. Each test costs less than the one before, because the data feed, the market history and the test log already exist.

A.1 Formulas

The math on one page

Lift is the share of business in the test markets that the change caused.

Incremental ROAS and cost per incremental conversion are the return on the money the test moved.

The incremental share converts a platform’s reported conversions into the conversions the advertising caused.

The smallest detectable lift depends on the normal weekly gap between actual results and the baseline, and on how many weeks the test runs. If weekly misses were independent, it would be:

Weekly misses usually run in streaks, which widens the range. The calculator includes that allowance and shows 9.0% for the same inputs. The lift a test can expect is:

Average return is total return divided by total spend. Return on the next dollar is the extra return from a small increase. In the chart in section 1.2 they are 2.9 and 1.6 conversions per $1,000. Budget should move toward the channels where the next dollar returns most, so a model’s table of average returns is the wrong input for a budget shift.

A.2 Measurement notes

Limits of the evidence

Much of the method literature comes from the companies whose media it measures. Google built Meridian and GeoX and wrote the mix model reviews cited here. Meta built GeoLift and Robyn. Open code makes these methods checkable, but their default settings deserve the same scrutiny as any vendor claim.

The 10.61× result comes from a single-author preprint using simulated data, and the result on diminishing returns comes from a working paper. Google’s launch claims for GeoX, including budget savings above 31%, come from internal benchmarks with no published method and are not used here. The GeoX confidence default is reported from its documentation by trade press.

The calculator in section 2.3 is a planning estimate. GeoLift and GeoX both estimate test power by simulating on the advertiser’s own history, and that is the method to rely on before committing budget.

A.3 Terms

Terms used on this page

A.4 Sources

References

Numbered references appear inline throughout. Each entry gives the publication, the type of evidence and any commercial interest the publisher has in the method.