Combining Structural Models And Causal Inference Methods
How to “localize” structural models
Causal inference methods focus on the “effect” on metrics and other high-level outcomes, but we need a model to translate treatment effects into a decision.
Structural econometric methods help us make decisions because they fully specify the economic nature of the decisions we’re making. This blog post shows how to combine these two approaches to make decisions grounded in both economic theory and data.
For example, if we know the causal effect of a price increase on demand for a product, that’s useful information, but it doesn’t — by itself — tell us what price we should charge. We need a model to turn that treatment effect into a decision.
If our cost of providing our products is roughly zero — we’re making websites, selling SaaS — then maybe we want to maximize revenue:
max P’Q(P)
Now we’re working in the space of structural parameters, the economic primitives. The next step is to link these to treatment effects.
The revenue-maximizing price solves:
DQ(P) P + Q(P) = 0
DQ(P) is the derivative of demand with respect to price, which maps onto the more causal inference problem of getting the treatment effect of shifting price on quantity, allowing us to translate treatment effects into pricing decisions.
We always put treatment effects through a model to make decisions. Whether we use an explicit economic model or not, we have some model in mind when we say that it’s a “good thing” to increase this metric and hurt that other metric. What structural methods do is make those assumptions explicit and ground those assumptions in economic theory.
But there is a downside to structural methods as they’re frequently used in the academic literature.
A common approach you’ll see:
Estimate a demand function Q(P) that we can evaluate at any price, given some functional form assumption.
Then, solve the first-order conditions DQ(P)P + Q(P) = 0 explicitly, which involves evaluating the demand function globally, at prices far from those in the data.
Hot take: this analysis is not credible enough for making business decisions at the accuracy level we need. The further we move from prices we have seen in the data, the less we’re using data to inform pricing (or other) decisions, and the more we’re relying on arbitrary functional form assumptions.
Anyone who’s tried the above approach in the wild has seen the model suggest wild prices that end up (rightly) ignored by stakeholders as unreasonable.
The trick to marrying more credible causal inference methods and structural methods is to use structural methods that can make use of local results — use methods that don’t rely on identifying the structural function globally.
Instead of trying to solve DQ(P)P + Q(P) = 0, instead recognize that DQ(P)P + Q(P) gives the marginal change in revenue for a small price change for each product. So we can “localize” the revenue maximization problem by doing something like the following:
Estimate the demand function at current prices Q(P) by using various causal inference methods — or experimentation, if feasible. This is more credible because we don’t need to try to map out the full demand function. We just need to measure the impact of increasing prices a little relative to current prices, i.e., estimate treatment effects.
Then, compute F = DQ(P) P + Q(P) at current prices.
If F(j) > 0, increase the price P(j) a little bit. If F(j) < 0, decrease the price P(j) a little bit. In practice, because F is estimated, it makes sense to add other restrictions, like: “we’ll only change prices if |F(j)| > e”.
Periodically re-run the analysis to update the demand function now evaluated at the new prices, and then we march towards the optimal prices.
So, we can combine more credible causal inference methods with economic models to help us make decisions that are both informed by economic theory and don’t overly rely on arbitrary functional form or other modeling decisions.
The key is to localize the structural model, using either the first-order method above or constraining decisions to lie within a tight neighborhood around the status quo.
Over time, we’ll move further and further from the initial status quo as we gather more evidence about other parts of the decision space.
And that’s it. A pragmatic way to use structural economic models and causal inference methods together.
Thanks for reading!
Zach
Connect at: https://linkedin.com/in/zlflynn
Take my little Udemy course (on causal inference methods) at: https://www.udemy.com/course/identifying-causal-effects-for-data-scientists/?couponCode=CHEAPCAUSALINF3
If you want my help with any Experimentation, Analytics, etc. problem, click here.