A Predictive Model Can Be Right for the Wrong Reason

November 2026 | Pricing | Industrial Organization | Causal Inference | ← Back to Blog

Earlier posts covered why price is endogenous and why consumers do not treat every product as a substitute. This post brings the two together. A predictive demand model can forecast sales well and still get the behavior behind them wrong, and the fix is to build the model from how consumers actually choose.

Two Kinds of Prediction

A forecast and a pricing decision both ask about future demand, but they ask under different conditions. A forecast asks what demand will be if the market keeps working the way it did in the data. A pricing decision asks what demand will be after you change something: a price, a product, the set of options customers see.

A flexible model trained on history can be very good at the first question. It learns whatever relationships produced past sales, and as long as those relationships hold, its forecasts are accurate. The second question is different. A pricing decision moves the market away from the conditions the model learned, and a model that only learned past relationships has no way to know what happens next.

Forecast accuracy shows the model learned the past. It does not show the model understands the behavior that produced it.

How a Good Forecast Learns the Wrong Behavior

Suppose a firm raises prices in weeks it expects to be busy: a holiday weekend, a popular event, a stretch of good weather. The firm knows something about demand that the data may not fully capture, and it sets prices accordingly. Those same conditions also raise sales.

A predictive model trained on that history sees high prices next to high sales. Because price carries information about demand, the model can learn a positive relationship between price and sales, even though customers, on the whole, buy less when prices rise. The forecast is accurate. The behavior behind it is backwards.

A held-out test won't catch this. The held-out data comes from the same history, where prices still rose in busy weeks, so the predictive model looks just as accurate there. The trouble starts when the firm uses that predictive model to set prices. Now prices no longer track expected demand, and the pattern the model relied on is gone.

You see the same thing outside pricing. If promotions mostly went to customers who were going to buy anyway, a predictive model credits the promotion with sales that would have happened without it. Stockouts are another example. When a product sells out, the sales data only records what was available to buy, not how many people wanted it. A model trained on sales reads those weeks as low demand, when demand was actually high enough to empty the shelves.

Demand Comes From Choices

Demand is the sum of many individual choices. A customer weighs the price against what the product offers, its uses and quality, and against the alternatives, including not buying at all.

Say you are deciding whether to see a movie. You look at the ticket price, how comfortable the seats are, the movie itself and its reviews, and what the concessions cost. If none of it appeals, you might pick a different movie, or skip it and take a walk in the park or get ice cream instead.

That one decision contains the building blocks of demand:

Price sensitivity. A higher price makes the option less attractive, and some customers care about price much more than others.
Preferences for product features. Seat comfort, the movie, the reviews. Customers value these differently.
Substitution between similar options. When an option gets worse or more expensive, customers move first to the closest alternatives, like another movie, before leaving the category.
The outside option. Not buying at all is always on the menu.

Economic demand models, from logit and nested logit to random-coefficients models, are ways of writing these pieces down and estimating them from data.

What the Building Blocks Make Possible

A model built from those pieces gives forecasts that come with a reason. Because price sensitivity, feature preferences, and substitution are explicit, the model can answer questions the historical data never showed directly: what happens to demand if this price changes, if a new product enters, or if a rival cuts its price.

This only works if the estimates are credible. The price response has to come from price changes that had nothing to do with demand, like a randomized price test, or a cost shock that moves price without affecting demand directly and can serve as an instrument. Even then, the answers are only as good as the assumptions about how customers behave.

There is also a limit to what a demand model alone can say. It tells you how customers respond to a rival's price cut. How the rival decides to set that price, and how the market settles afterward, requires modeling the firms as well.

Where Flexible Models Fit

Once the behavior is right, flexible models are useful for everything around it. Seasonality, weather, events, and local conditions all shift demand in ways that are hard to specify by hand and easy for a flexible model to learn. Double machine learning uses a related idea. Flexible models absorb the many factors that shift demand, which helps isolate the effect of price. It estimates that price effect, though, not a full picture of how customers substitute between options.

Flexible models also make a good check on the economic model. If a flexible forecast keeps beating the underlying economic model in some part of the data, that is a sign an assumption about customer behavior is missing, wrong, or needs to be adapted. The gap is a place to investigate, not proof of a specific error. It might point to a substitution pattern the model rules out, a group of customers with different price sensitivity, or a demand shifter that was left out.

A Practical Order of Work

Define the choice. What options does a customer compare, and what does not buying look like?
Specify the building blocks. Price sensitivity, the product features that matter, and how customers substitute.
Find credible price variation. Experiments, cost shocks, or policy changes that moved price independently of unobserved demand conditions.
Let flexible models handle the demand shifters. Seasonality, weather, events, and other factors around the behavior.
Benchmark against a flexible forecast. Where it consistently does better, revisit the behavioral assumptions.
Test the decision. Before a full rollout, run the recommendation as an experiment and compare the result with what the model predicted.

The Takeaway

Accuracy on historical data tells you a model has learned the past. A pricing decision asks about a market that has changed, and answering that requires knowing how customers choose. Build the behavior first and use the most powerful tools you have to refine it.