The first essay ended at the trade-off every regime detector faces: alarm early and misfire, or alarm late and stay credible. This one holds that trade-off still and looks at how the four main method families actually work — machinery first, judgement second, in that order deliberately.

The setup: returns from a shifting process

Formally, a detection model watches a series of observations — returns, ranges, realized correlations — and asks which of two worlds produced today’s data:

The same world as before. Today’s numbers came from the same distribution as the recent past.
A changed world. Something in the process — variance, mean, tail weight, coupling between assets — has shifted, and the past is no longer a fair guide.

Every method below machinery-wise picks a different parameterization of “changed”: hidden discrete states, a sharp break-point, structural rewiring, or slow distributional drift. None of them is magic; each is a careful way of asking one version of the question.

Hidden Markov models: markets with hidden gears

James Hamilton’s 1989 model of the American business cycle is the ancestor most practitioners meet first. The idea: the market occupies a small number of hidden states — commonly labelled calm, trending, stressed — and each state produces returns with its own statistical personality. The actual state generates the only data we see; the model infers back. After every close, a filtered probability updates: perhaps 4% chance we are in the stressed state today, up from 1% yesterday.

Each closing day the model also updates its odds of switching states through a transition matrix, which is where the craft enters. Fit the same market with two states instead of five and you will get different answers, because the state count is a modelling choice, not something the data dictates. A model whose states are “calm” and “panic” simply has no seat at the table for a kind of market it was never taught to name.

Bayesian changepoint detection: the run-length trick

Changepoint methods attack the problem from the other side: forget states, watch continuity. In the formulation Adams and MacKay published in 2007, the model tracks a probability distribution over the run length — how long the current stretch of statistical stability has lasted. As long as new observations fit the recent pattern, probability mass accumulates on longer runs: the world looks continuous. The instant observations stop fitting, mass shifts toward short runs: the stretch may just have ended.

One knob matters more than the rest: the hazard — the model-maker’s prior belief about how long regimes tend to survive. A short-hazard model assumes breaks are frequent and alarms readily; a long-hazard model is patient. The honesty of the output therefore depends on an assumption the operator chose before seeing the data, which is exactly why this series insists the assumption be stated out loud.

Correlation networks: watching the web

The February 2018 and 2022 episodes in the first essay had something in common: before prices did anything headline-worthy, the relationships between assets moved. Correlation-network methods make that observation systematic. Represent the market as a graph — nodes are assets, edges encode how strongly two things move together — then monitor the graph itself for ruptures: hedges that quietly decouple; safe havens that stop being safe; clusters that suddenly fuse.

Mantegna’s 1999 study of hierarchical structure in equity markets is the classic reference here: minimum-spanning-tree pictures of the market whose topology reorganizes when the environment changes. The standing weakness is noise: correlations measured over short windows are unstable, so single snapping threads mean little. Experienced readers wait for several independent edges to agree.

Drift statistics: staleness alarms for anything learned

The last family comes from the industrial side of machine learning, where it keeps credit-scoring and fraud models honest. A statistic such as the population stability index compares the distribution a model sees in production against the one it was trained on; a KS test does similar work for two samples. When today’s inputs or outputs drift far from the training window, the alarm says: your model is stale — the world it learned is gone.

In investment AI, drift monitors are the minimum-maintenance detector: simple, cheap, and usually the slowest of the four, because drift statistics confirm a change only after enough new-reality data accumulates to measure. What they lack in speed, they return in credibility — they are evidence, not vibes.

How a working desk actually reads them

In production, none of these runs alone. The Markov layer speaks fast but with fuzzy labels; the changepoint layer pins moments but needs its hazard tuned; the network layer sees structure the others cannot; the drift layer sits at the back and confirms. A reader of this site should treat any single-detector claim — “the model says we’ve switched regimes” — as one instrument reading, not a weather forecast. Instruments get read by someone who knows their quirks; that someone, and the discipline they bring, is the subject of the third essay.