The Grand Scheme of Madness
Hidden Markov models  ·  a visual guide

The regime you can’t see

A market drifts between a bull regime and a bear regime. Nobody announces which one is in force. All you get is the tape: up, flat or down, one day at a time. A hidden Markov model is the machinery for reasoning from the tape back to the regime.

Hidden regimeveiled
Observed move: up at the top, flat in the middle, down at the bottomtoday →
A live simulation of the model in section 1. The hatched band is what an observer never sees. Reveal it and notice how hard the regime is to guess from the dots alone.
  1. 00The idea in two minutes
  2. 01The model
  3. 02Three questions
  4. 03The trellis: forward and Viterbi
  5. 04Forward–backward
  6. 05Two ways to decode
  7. 06Baum–Welch
  8. 07Why EM works
  9. 08The likelihood surface
  10. 09Gaussian emissions
  11. 10How many states
  12. 11Dwell times
  13. 12Numerical practice
  14. 13The Bayesian route
  15. 14The family
  16. 15Failure modes
  17. 16Trading on it
  18.  Sources
00

The idea in two minutes

If you read nothing else, read this one figure from top to bottom. You need to be comfortable multiplying probabilities, and nothing more.

Forty days from the model in section 1. Row 1 is the truth, which an observer never sees. Row 2 is all the observer gets. Rows 3 and 4 are what the algorithms in sections 3 to 5 recover from row 2 alone. Row 5 is Baum–Welch from section 6: each dot is one round, the open circle is the starting guess and the black tick is the truth. It reads 600 days of the same tape, because forty are too few to learn from.
  1. A market is always in one of two regimes, bull or bear. Nobody tells you which. That is row 1.
  2. All you see is the daily move: up, flat or down. Each regime makes some moves more likely than others. That is row 2.
  3. Regimes last. Tomorrow’s regime is usually the same as today’s, which is why one odd day should not change your mind.
  4. From the moves alone, the model works out how likely each regime was on each day (row 3) and the single most likely story for the whole stretch (row 4).
  5. It can also learn each regime’s habits from the tape itself, with no labels. Row 5 starts from a blind guess about how long each regime lasts and ends near the true values. The rest of this guide shows how each of these steps works.
01

Two dice and a switch

An HMM has three ingredients. A starting distribution \(\pi\) says which regime day one begins in. A transition matrix \(A\) says how regimes follow each other: \(a_{ij}\) is the chance of moving from state \(i\) to state \(j\). An emission model \(B\) says what each regime tends to produce: \(b_j(x)\) is the chance that state \(j\) emits observation \(x\).

Two assumptions make everything tractable. Tomorrow’s regime depends only on today’s. Today’s move depends only on today’s regime. Together they let the probability of a full story, hidden path \(z\) and observations \(x\), factor into a chain of small terms:

\[P(z,x)=\pi_{z_1}\,b_{z_1}(x_1)\prod_{t=2}^{T}a_{z_{t-1}z_t}\,b_{z_t}(x_t)\]
Bull Bear

Bull regime

Emits flat

Bear regime

Emits flat
The model behind sections 1 to 6 and 13. Arrow weight follows probability, and the filled circle is the regime the tape above is in right now. Day one starts bull with probability 0.6. Drag an arrow up or down, or move a slider, and every dependent figure recomputes.
02

Everything you do with an HMM is one of three questions

How likely is this tape?

Sum the probability of the observations over every hidden path. This scores a model against data and lets you compare two models.

Forward algorithm

What was the regime?

Find the single hidden path that best explains the observations, or the probability of each regime on each day.

Viterbi · forward–backward

What are the parameters?

Given only the tape, estimate \(A\) and \(B\). Nobody hands you labelled regimes, so counting transitions directly is impossible.

Baum–Welch
03

Forward sums. Viterbi takes the max.

Eight days and two regimes give \(2^8=256\) possible hidden paths. A trading year gives \(2^{252}\). Enumeration is hopeless, but the Markov assumption means a path’s past matters only through the state it is in now. So work one day at a time and keep one number per state.

The forward algorithm keeps \(\alpha_t(j)\), the probability of the tape so far, ending in state \(j\). Viterbi keeps \(\delta_t(j)\), the probability of the best single path so far ending in \(j\), plus a back-pointer to where that path came from. The two recursions differ by one operator:

\[\alpha_t(j)=b_j(x_t)\sum_i\alpha_{t-1}(i)\,a_{ij}\qquad\qquad\delta_t(j)=b_j(x_t)\,\max_i\,\delta_{t-1}(i)\,a_{ij}\]
Click any observation along the top to change it. In Viterbi mode, solid edges are the surviving back-pointers and dotted edges are the candidates that lost. The heavy black line is the traced-back answer. In forward mode, edge weight shows how much each predecessor contributes to the sum.
04

Using tomorrow to judge today

The forward pass, normalised, gives the filtered belief \(P(z_t\mid x_{1:t})\): what you could know in real time. With the whole tape in hand you can do better. A backward pass computes \(\beta_t(i)\), the probability of everything after day \(t\) given state \(i\) on day \(t\):

\[\beta_t(i)=\sum_j a_{ij}\,b_j(x_{t+1})\,\beta_{t+1}(j),\qquad\beta_T(i)=1\]

The two messages meet at each day. Their product, normalised, is the smoothed belief:

\[\gamma_t(i)=P(z_t=i\mid x_{1:T})=\frac{\alpha_t(i)\,\beta_t(i)}{P(x)}\]
Each bar splits belief between bull (teal) and bear (rose). The forward message sweeps left to right and carries the past. The backward message sweeps right to left and carries the future. Their product is sharper than either, and on the last day the backward message has nothing to say.

Over a longer tape the difference is plain. The smoothed belief reacts earlier at regime changes and shrugs off isolated odd days, because later evidence confirms or contradicts each move.

In live use you cannot wait for the whole tape, but you can wait a little. Fixed-lag smoothing reports \(P(z_t\mid x_{1:t+L})\), the belief about day \(t\) formed \(L\) days later. Most of the benefit of hindsight arrives within the first few days of lag.

Smoothed, whole tape Fixed lag Filtered, past only Bull Bear
Top: probability of the bull regime on one 120-day tape, against the true regime and the Viterbi path. Hover for values. Bottom left: the share of days labelled correctly at each lag, measured on a separate 4,000-day simulation of the same model.
05

Two decoders, two different questions

Viterbi returns the single most probable path as a whole. Posterior decoding picks, for each day separately, the state with the largest \(\gamma_t\). The second gets more individual days right on average, and that is all it promises. Nothing forces its day-by-day choices to form a path the model allows.

The example below adds a third, range-bound regime and one structural rule: the market never jumps straight between bull and bear. It has to pass through the range.

Circles show the smoothed probability \(\gamma_t\) of each state, with thin lines for the transitions the model permits. Black rings mark the Viterbi path. Dots mark the most probable state on each day. Use Viterbi when the path has to make sense as a story, and posteriors when you need calibrated beliefs about individual days.
06

Learning the model from the tape alone

If the regimes were labelled, estimation would be counting: the share of bull days followed by bear days, the share of bear days that closed down. They are not labelled, so Baum–Welch counts in expectation. It is the EM algorithm applied to an HMM, and it alternates two steps.

E-step. Run forward–backward under the current guess. This yields \(\gamma_t(i)\), the probability of state \(i\) on day \(t\), and \(\xi_t(i,j)\), the probability of the transition \(i\to j\) between days \(t\) and \(t+1\):

\[\xi_t(i,j)=\frac{\alpha_t(i)\,a_{ij}\,b_j(x_{t+1})\,\beta_{t+1}(j)}{P(x)}\]

M-step. Re-estimate each parameter as a ratio of expected counts. It is the counting you would have done with labels, with soft weights in place of hard ones:

\[a_{ij}\leftarrow\frac{\sum_t\xi_t(i,j)}{\sum_t\gamma_t(i)}\qquad\qquad b_j(k)\leftarrow\frac{\sum_{t:\,x_t=k}\gamma_t(j)}{\sum_t\gamma_t(j)}\]

Below, 400 days are simulated from the model in section 1. The algorithm starts from a random guess and sees only the tape.

True regime Model’s belief γ
Bars are the current estimates and black ticks are the true values, which the algorithm never sees. The belief strip starts as mush and sharpens into regimes as the parameters improve. Estimated states are matched to bull and bear by their emissions before display.
07

Why every EM step goes uphill

The likelihood \(\log P(x\mid\theta)\) is hard to maximise directly because it sums over every hidden path. EM sidesteps the sum. For any distribution \(q\) over paths, the log-likelihood splits into a tractable piece and a gap:

\[\log P(x\mid\theta)=\underbrace{\mathbb{E}_q\big[\log P(x,z\mid\theta)\big]+H(q)}_{\text{lower bound }\mathcal{L}(q,\theta)}+\underbrace{\mathrm{KL}\big(q\,\Vert\,P(z\mid x,\theta)\big)}_{\text{gap}\;\ge\;0}\]

The E-step sets \(q\) to the posterior under the current parameters \(\theta_k\). The gap closes and the bound touches the likelihood at \(\theta_k\). The M-step moves \(\theta\) to the top of that bound, which is easy because the bound is a sum of weighted logs. The true likelihood at the new point is at least as high as the bound there, so

\[\log P(x\mid\theta_{k+1})\;\ge\;\mathcal{L}(q_k,\theta_{k+1})\;\ge\;\mathcal{L}(q_k,\theta_k)=\log P(x\mid\theta_k)\]

The figure shows this on a one-parameter slice: the probability that bull stays bull, with every other parameter held at its true value.

Black: the log-likelihood of a 300-day tape as the bull persistence varies. Teal: the lower bound built at the current estimate. The bound touches the curve where it was built and lies below it everywhere else, so climbing the bound can only climb the curve. Click anywhere on the chart to start from a different value.
08

The likelihood surface has a mirror in it

With two free parameters the likelihood becomes a landscape. Take a stripped-down model: two equally sticky states and a tape of ups and downs, where the only unknowns are each state’s probability of emitting an up day.

Nothing in the likelihood says which state is called “state 1”. Swap the two labels and the fit is identical, so the surface is symmetric about the diagonal and has two equally good peaks. Along the diagonal the two states are identical twins, and EM started there has no reason to prefer either, so it stays. Everywhere else it runs to whichever peak is on its side.

0P(up | state 1) →1
Vertical axis: P(up | state 2), from 0 at the bottom to 1 at the top. Darker bands are higher likelihood. The dashed diagonal is where the two states are identical.
Click anywhere on the map to launch EM from that starting guess. Hollow circles are starts and filled circles are where EM stops. This two-parameter slice has only the mirror-image peaks. With every parameter free, as in figure 7, the surface also has inferior local peaks, which is why “New random start” there sometimes ends lower.
09

From three symbols to real returns

Nothing in the algorithms requires a table of discrete emissions. Replace it with a density and the recursions are unchanged: wherever \(b_j(x_t)\) appeared, evaluate the density of state \(j\) at the observed value. The usual choice for returns gives each regime its own mean and volatility,

\[x_t\mid z_t=j\;\sim\;\mathcal{N}(\mu_j,\sigma_j^2)\]

and the M-step becomes a weighted mean and a weighted variance, with the smoothed beliefs as weights:

\[\mu_j\leftarrow\frac{\sum_t\gamma_t(j)\,x_t}{\sum_t\gamma_t(j)}\qquad\qquad\sigma_j^2\leftarrow\frac{\sum_t\gamma_t(j)\,(x_t-\mu_j)^2}{\sum_t\gamma_t(j)}\]

This is the standard regime-switching model. Below, three years of daily returns are simulated from three regimes: calm, choppy and crisis. A Gaussian HMM is then fitted with Baum–Welch from the returns alone. Now the regimes overlap. A quiet day can happen in a crisis, and only persistence lets the model tell them apart.

Top: 750 simulated daily returns, coloured by the fitted model’s most probable regime, between the true regimes above and the fitted ones below. Bottom left: the return histogram with each fitted state’s density scaled by its share of days, and their sum dashed. Fitted states are ordered by volatility. Try fitting two or five states to three-regime data, then switch to pure noise.
10

How many states

In-sample likelihood cannot choose the number of states, because it never goes down when you add one. Two standard checks can. Held-out likelihood scores each fitted model on data it has not seen. The Bayesian information criterion penalises the in-sample fit by the number of free parameters \(p\):

\[\mathrm{BIC}=-2\log\hat{L}+p\,\log T,\qquad p=\underbrace{K(K-1)}_{A}+\underbrace{(K-1)}_{\pi}+\underbrace{2K}_{\mu,\,\sigma}\]

The chart scores the fits from section 9, for one to five states, on the data shown there and on a second, unseen tape drawn the same way.

Log-likelihood by number of fitted states. Each line is drawn relative to its own best value, so zero marks that criterion’s preferred model. The BIC line is the in-sample log-likelihood minus \(\tfrac{p}{2}\log T\). Filled markers are each criterion’s choice. The shaded column is the model shown in figure 10. Values far below the rest are clipped at the bottom edge.
11

How long a regime lasts

An HMM decides each day whether to leave its state with the same coin. That makes the length of a stay geometric:

\[P(D=d)=a_{ii}^{\,d-1}\,(1-a_{ii}),\qquad\mathbb{E}[D]=\frac{1}{1-a_{ii}}\]

The implication is easy to miss. Whatever the persistence, the single most likely duration is one day, and a regime that has lasted a hundred days is no closer to ending than one that began this morning. If real regimes have a typical length, the model cannot express it. A hidden semi-Markov model fixes this by drawing each stay’s length from an explicit duration distribution, at the cost of heavier inference.

Bars: the distribution of stay lengths an HMM implies. Line: a peaked duration distribution with the same mean, of the kind a semi-Markov model can represent and an HMM cannot.
12

The numbers vanish, so rescale them

Each step of the forward recursion multiplies by a probability, so \(\alpha_t\) shrinks geometrically. Double-precision arithmetic bottoms out near \(10^{-308}\), and a naive implementation returns exactly zero on any tape of realistic length. There are two standard remedies. Work with logarithms and replace sums by log-sum-exp, or rescale \(\alpha\) to sum to one at every step and keep the scaling constants:

\[c_t=\sum_j\tilde\alpha_t(j),\qquad\hat\alpha_t=\tilde\alpha_t/c_t,\qquad\log P(x)=\sum_{t=1}^{T}\log c_t\]

Rescaling is the cheaper of the two and gives the filtered beliefs \(\hat\alpha_t\) for free. Viterbi needs only the log transform, since a max of products becomes a max of sums. Every figure after the trellis uses these forms.

The order of magnitude of the forward variable on a long tape from the two-state model, computed in ordinary double precision. The naive product (black) falls off the bottom of the number system. The rescaled computation (teal) recovers the same quantity from its scaling constants and keeps going.
13

Beyond a single answer

Viterbi gives one path and Baum–Welch gives one set of parameters. Both hide how uncertain the answer is. The Bayesian treatment keeps the uncertainty by sampling.

Sampling paths. Run the forward filter, then walk backwards drawing each state from its distribution given the state just drawn: \(P(z_t=i\mid z_{t+1}=j,\,x_{1:t})\propto\hat\alpha_t(i)\,a_{ij}\). Every draw is a complete, coherent path from the posterior. The spread across draws shows which regime changes are certain and which are a matter of opinion.

True regime Viterbi path 40 posterior samples
Forty paths sampled from the posterior for the 120-day tape of figure 5, one per row. Columns where every row agrees are days the model is sure about. Ragged edges are regime changes it can only date approximately, and stripes that appear in some rows only are regimes it is unsure existed at all.

Sampling parameters. Put Dirichlet priors on the rows of \(A\) and \(B\) and alternate: sample a path given the parameters, then sample the parameters given the path, which is again just counting. The result is a posterior distribution for every parameter, which says how much the tape actually pins each one down.

Posterior distributions of the two persistence parameters from a Gibbs sampler with flat Dirichlet priors, run on the 400-day tape of figure 7. The solid tick is the true value, the dashed tick is the Baum–Welch estimate, and the bracket spans the central 90% of the posterior.
14

Where the HMM sits among its relatives

Most sequence models you will meet are an HMM with one assumption changed. Knowing which assumption tells you which of the algorithms above still apply. Each panel opens a short interactive guide to that model.

15

How this goes wrong in practice

AlgorithmAnswersRecursionCost
Forward\(P(x)\), filtered beliefssum over predecessors\(O(S^2T)\)
Viterbibest single pathmax over predecessors, then trace back\(O(S^2T)\)
Forward–backward\(\gamma\) and \(\xi\), smoothed beliefsforward message times backward message\(O(S^2T)\)
Baum–Welch\(\pi\), \(A\), \(B\)forward–backward, then re-count, repeated\(O(S^2T)\) per iteration
Forward filter, backward sampledraws from the posterior over pathsforward pass, then sample backwards\(O(S^2T)\) per draw
16

Trading on it

Sections 3 to 6 work on a tape that is already complete. A strategy stands at the end of the tape and has to act before the next move prints. Used without care, the same algorithms hand it information from days it has not lived through. There are four ways this happens.

Smoothed labels (uses the future)Filtered, parameters fitted on the full sampleFiltered, refit walk-forwardBuy and hold
One simulated tape of 1,000 days from the model in section 1, where an up day earns a point and a down day loses one. Each strategy is long when it expects the next move to be up and short when it expects it to be down. Smoothed labels are scored against the same day they label. The two filtered lines set each day’s position from the previous close. Walk-forward refits every 100 days on the data so far. Trading starts on day 200. Change the model in figure 2 and this recomputes.

The walk-forward line is the only one of these that could have been run. Compare it with buy and hold before believing the regime model added anything.

Sources and further reading