Treatment Spillovers: Who Is Actually Untreated?
October 2026 | Applied Economics | Marketplace Causal Inference | ← Back to Blog
Every causal estimate depends on a claim that is easy to forget: that the comparison group stayed untouched. This post explains how treatment can reach units labeled as untreated, how to check whether it did, and why the same question matters when a benchmark is used in merger analysis.
The Core Problem: Who Is Actually Untreated?
In a randomized experiment, some units are assigned to treatment and the rest to control. The standard analysis compares the two groups, and the difference is read as the effect of the treatment. That reading depends on assumptions, including that a unit's outcome depends only on its own assignment, not on anyone else's. This is the no-interference part of the Stable Unit Treatment Value Assumption, or SUTVA.
When the assumption fails, the label “control” describes the assignment but not what the unit experienced. A control unit can be affected by a treatment it was never assigned. The difference between the groups then no longer measures the effect of the treatment, because the comparison group is not the world without it.
An earlier post on switchback experiments covered one response to this problem, which is to change the unit of randomization. This post is about a step before that: checking whether the comparison group was actually untouched.
“Untreated” describes the assignment, not the experience.
A Concrete Case: The Concert Discount
Say a ticketing platform shows a discount on a popular concert to a random half of its users. You see it and tell a friend who happens to be in the control group. They cannot get the discount on their own account, so they ask you to buy their ticket with yours.
Your friend's behavior changed because of something assigned to you, not to them. The purchase is recorded under the treatment group, and it disappears from the control group. The experiment sees a gain, but if your friend was going to buy anyway, the platform simply sold the same ticket at a lower price.
A hypothetical example shows the size of the distortion. Suppose the platform has 1,000 treated and 1,000 control users, and each user buys at most one ticket, so counts and purchase rates tell the same story. With no spillover, treated users buy 100 tickets and control users buy 50, so the estimated effect of the discount is 50 extra tickets. Now suppose 10 control users ask treated friends to buy for them. Treated users now buy 110 tickets and control users buy 40. The estimated effect is 70 tickets, 40% larger than before. Total sales are still 150 tickets in both cases. The experiment reports a bigger win only because ten purchases moved from one column to the other, at a lower price.
The treatment group took from the control group. That is the sense in which the control group was not untreated.
Exposure Comes in Several Forms
The concert example is a social spillover, but exposure can run through several channels, and the bias does not have a fixed sign.
Geographic. A program run in one area can reach neighboring areas that were assigned to control, through shared markets, shared services, or people moving between them. If the neighbors benefit too, a treated-versus-control comparison can understate the effect of the program.
Social. Treatment can travel through friends, referrals, or shared information, as in the concert example. How much of it a unit receives depends on how many treated units it is connected to.
Marketplace. When units compete for the same buyers, jobs, or inventory, a gain for the treated can come at the expense of the untreated. A comparison that ignores this can overstate what the intervention would do if it were rolled out to everyone.
How to Check: Compare Control Outcomes by Exposure
One way to check is to compare control outcomes by how exposed each control unit was to the treated ones. The steps could be as simple as the following. These are not the only steps, and the right ones depend on the setting.
First, define an exposure measure that fits the setting. It could be distance in miles, the number of treated friends, or the share of treated listings in the same category.
Second, group the control units by exposure, or regress their outcomes on the exposure measure. Only control units belong in this comparison, because the question is whether units that were never assigned treatment still move with how much treatment is nearby.
Third, read the pattern. If control outcomes are flat across exposure, the control group is behaving like one. If control outcomes differ systematically with exposure, that is a sign the “untreated” units may be contaminated.
The check has limits. Exposure is often not randomly assigned, so outcomes that move with distance are a warning sign, not a measure of how big the spillover is. Control units near the treated ones may differ from distant ones for other reasons, such as busier locations or heavier users. And finding no pattern does not prove there was no spillover, especially with few units. The exposure measure may also miss the channel through which treatment actually travels.
The check is still worth running. It tells you whether there is visible evidence against the comparison group before you build a decision on it.
Why This Matters Beyond Experiments
The same logic applies outside experiments. In merger analysis or regulatory matters, non-merging rivals or unaffected markets often serve as the benchmark. But a merger changes the competitive landscape, and that can change how those rivals behave. Then they may no longer show what would have happened without the deal.
For example, if rivals raise their prices in response to a merger, a comparison of merging firms against rivals can understate the effect of the merger on the merging firms' prices, and it will miss the rivals' own price response entirely. Whether that happens is an empirical question, and the same exposure check applies. Do outcomes at non-merging rivals vary with how closely they compete with the merging firms?
In all of these settings, the estimate is only as credible as the comparison group behind it.
What to Do If You Find Contamination
Finding a pattern does not mean the experiment is lost, but it changes what can be claimed. There are two common responses.
The first is to change the unit of randomization so that interference stays inside units, for example by randomizing at the level of regions, networks, or time windows. The second is to change the question and model the interference explicitly, estimating the effect of treatment under different levels of exposure instead of a single treated-versus-control difference. Each trades some power or simplicity for a more credible comparison.
Further Reading
Hudgens, M.G. and Halloran, M.E. (2008). Toward Causal Inference with Interference. Journal of the American Statistical Association. On causal inference when one unit's treatment may affect another unit's outcome.
Aronow, P.M. and Samii, C. (2017). Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment. Annals of Applied Statistics. On estimating causal effects under general interference, including in social networks.
Miguel, E. and Kremer, M. (2004). Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities. Econometrica. On measuring program impacts when treatment has externalities on untreated units.
Crepon, B., Duflo, E., Gurgand, M., Rathelot, R. and Zamora, P. (2013). Do Labor Market Policies Have Displacement Effects? Evidence from a Clustered Randomized Experiment. Quarterly Journal of Economics. On whether labor market policies displace outcomes for people who are not assisted.
Blake, T. and Coey, D. (2014). Why Marketplace Experimentation Is Harder than It Seems: The Role of Test-Control Interference. Proceedings of the ACM Conference on Economics and Computation. On test-control interference in marketplace experiments.
Johari, R., Li, H., Liskovich, I. and Weintraub, G. (2022). Experimental Design in Two-Sided Platforms: An Analysis of Bias. Management Science. On bias in experiments on two-sided platforms.
