field note · method
a forecast tells you who resembles past leavers. it cannot see who leaves when you change the conditions.
readout · churn risk score vs fragility
the question every model answers
Every churn model in production answers the same question: which of my current customers look like the customers who already left. Survival curves, gradient boosted classifiers, engagement decay scores. Different machinery, same question. And to be fair, they answer it well.
The problem is not the model. The problem is an assumption hiding inside the question: that tomorrow is drawn from the same distribution as yesterday. Under steady state, that roughly holds. Prices stable, packaging unchanged, competitors behaving, macro flat. In that world, historical churn is a defensible prior for future churn, and your dashboard is telling you something true.
Almost no decision that matters to revenue is made in that world.
the churn your data cannot contain
Consider the moves that actually change retention numbers. Raising prices. Restructuring plans. Moving a feature behind a higher tier. Sunsetting a legacy discount. Each one is an intervention, and for each one your historical data contains exactly zero examples of your customers responding to it. There is no feature engineering trick around this. You cannot train on a world that has not happened.
There is a second problem underneath, and it is quieter. Your current customer base is not a sample of the market. It is the filtered subset that tolerated every condition you have imposed so far. The customers most sensitive to price already left at the last increase, so the survivors make your base look calmer than the population it came from. Tolerance estimated on survivors is biased upward at exactly the point you want to probe.
churn is a boundary crossing
Here is the reframe that makes the problem tractable. A customer occupies a position on a decision landscape. Staying is a basin. Leaving is a basin. Between them runs a boundary. Churn is the moment a customer crosses it, and the crossing is the least informative part of the whole event. By the time it lands in your data warehouse, everything interesting has already happened.
The quantity that matters is not the probability of leaving under current conditions. It is distance to the boundary, and how much force it takes to push someone across. Call that fragility. The two are different measurements, and they routinely disagree. A customer can score low on churn risk and high on fragility at the same time: perfectly retained today, logging in daily, one four dollar price move from gone. Under current conditions, nothing in their behavior distinguishes them from a loyal customer, which is why no model trained on behavior under current conditions will ever surface them. That is the upper left quadrant in the chart above. It is invisible by construction, not by accident.
susceptibility rises before the cascade
Physics has a result worth borrowing. Systems approaching a phase transition show rising susceptibility before the transition: they respond disproportionately to small perturbations while still looking stable to a casual observer. Populations of customers behave the same way. Before a churn cascade, sensitivity climbs. Small price adjustments draw sharper reactions than they used to. Alternatives start entering the discourse unprompted. Friction that was silently absorbed for years suddenly gets named. Each signal is noise on its own. Taken as a population level measurement, they are a leading indicator, and they are readable in public discourse before you touch anything in production.
This is the same asymmetry we described in why your dashboard turns red too late: the dashboard reports the cascade. The fragility underneath it was climbing long before.
stress testing instead of forecasting
Which brings us to the method. If the churn you care about has never happened, you cannot forecast it. You can simulate it. Build a behavioral population from the observable discourse of your category, the same arguments, thresholds, and defection grammar real customers leave in public. Place the modeled segments on the landscape. Then apply the intervention synthetically. Raise the price in simulation. Move the feature behind the tier in simulation. Count who crosses the boundary, in which order, and at what magnitude. That is the run behind our Netflix price increase stress test, which flagged a roughly 14% at-risk cohort from public signal alone, before a single cancellation was posted.
The output is not a per customer risk score. It is a fragility map of your base: which cohorts are load bearing, what perturbation size produces a shrug and what size produces a cascade, and where the cliff sits relative to the move you are actually considering.
This instrument has limits, and we would rather name them than have you discover them. A simulated population will not tell you which named individual leaves, only how much of each segment crosses and what pushes them. And it is only as good as the discourse the population leaves behind; a category that does not argue in public is a category we can read only dimly. The two rules of predicting a crowd apply here in full. What the instrument does that no forecast can: it measures a future that has not happened yet, before you run the experiment on production revenue.
A forecast extrapolates the past. A stress test simulates the future you are about to create. If there is a price move on your roadmap this year, only one of these measurements contains it.