field note · method
what Foundation got right about modeling people, and where the honesty has to come in.
readout · a population under a named stress
Asimov gave the fantasy its cleanest form. Hari Seldon's psychohistory could chart the next thousand years of a galactic civilization, not because it knew any single person, but because it had learned the equations of the mass. It is a seductive idea, and most churn tooling is a watered-down version of it where feeding in enough transaction logs makes the future seem to drop out. The seductive part is half right. The half that is wrong is the half that matters, and the honest version of crowd prediction lives in the gap between them. Foundation got two things right. It needs three corrections to survive contact with a real population.
rule one: the crowd, never the person
The first premise is the one that holds. Individuals are not predictable. The same person cancels on a Tuesday and renews on a Wednesday for reasons neither of you could write down. But put enough of them together and the noise cancels: the fraction of a segment that crosses a line under a given pressure is stable even when no single crossing is. This mirrors the thermodynamics of a physical gas; you cannot track a single molecule's trajectory, but you can calculate the exact volume and pressure of the whole chamber. This law of large numbers lets an insurer price a hundred thousand policies without blinking. dfmchn predicts the cohort, not the customer. Anyone selling a per-person crystal ball is selling the part Asimov knew was fiction.
rule two: the stress, never the open future
The second thing Foundation got right is quieter, and people miss it. Seldon did not forecast events. He forecast responses to crises, focusing on specific, bounded perturbations the plan was built around. That is the tractable question. Asking what this population will do over the next decade yields no answer. Asking what this population will do when this price moves, when this mandate lands, or when this policy reverses yields a clear one, because a named stress collapses a thousand open futures into a single conditional. Prediction works the moment you stop asking what happens and start asking what happens if. The unit is not the future. It is the response to a move you can name.
Open-ended forecasting is astrology with a dashboard. A named stress is the only question a crowd will answer.
where the honesty comes in
Now the corrections, because this is where the fiction was fiction, and where a method earns trust by saying so.
First, the crowd is not ignorant of you. Asimov knew his own weak point. Psychohistory only held if the population stayed unaware of the predictions, which is why he had to invent a Second Foundation to keep the plan secret. There is no Second Foundation. Your population reads the same price increase you do, sees the competitor's ad, and adapts while you watch. So you cannot model a frozen crowd. You model a reacting one, over a short horizon, and you re-run it when the crowd moves. The prediction has a shelf life measured in weeks, not centuries.
Second, the regime is an assumption, not a guarantee. Seldon's plan broke on the Mule, the one variable the statistics did not contain. Real populations have Mules: a viral moment, an exogenous shock, or a single product decision that rewrites the board. An honest model does not predict the shock. It predicts the response to your stress, given the world holds roughly still around it, and it labels that assumption in plain sight. When the regime turns, the number is void, and pretending otherwise is how you get surprised twice.
Third, we are not reading the equations of the soul. Seldon had a mathematics of human behaviour. We have what a population says about itself, which is a sample: self-selected, loud in some places and silent in others, where the people who post stand in for the people who pay. That is a real signal and a directional one. It is not omniscience, and a method that admits the shape of its own bias is worth more than one that claims to have none.
Foundation's mistake was confidence: a crowd predicted forever, in the dark, by equations that knew everything. The two rules survive that. Predict the crowd, not the person. Predict the response to a named stress, not the open future. Then say where the model stops. A prediction that knows its own edge is the only kind worth acting on.