The regime you can’t see
A market drifts between a bull regime and a bear regime. Nobody announces which one is in force. All you get is the tape: up, flat or down, one day at a time. A hidden Markov model is the machinery for reasoning from the tape back to the regime.
- 00The idea in two minutes
- 01The model
- 02Three questions
- 03The trellis: forward and Viterbi
- 04Forward–backward
- 05Two ways to decode
- 06Baum–Welch
- 07Why EM works
- 08The likelihood surface
- 09Gaussian emissions
- 10How many states
- 11Dwell times
- 12Numerical practice
- 13The Bayesian route
- 14The family
- 15Failure modes
- 16Trading on it
- Sources
The idea in two minutes
If you read nothing else, read this one figure from top to bottom. You need to be comfortable multiplying probabilities, and nothing more.
- A market is always in one of two regimes, bull or bear. Nobody tells you which. That is row 1.
- All you see is the daily move: up, flat or down. Each regime makes some moves more likely than others. That is row 2.
- Regimes last. Tomorrow’s regime is usually the same as today’s, which is why one odd day should not change your mind.
- From the moves alone, the model works out how likely each regime was on each day (row 3) and the single most likely story for the whole stretch (row 4).
- It can also learn each regime’s habits from the tape itself, with no labels. Row 5 starts from a blind guess about how long each regime lasts and ends near the true values. The rest of this guide shows how each of these steps works.
Two dice and a switch
An HMM has three ingredients. A starting distribution \(\pi\) says which regime day one begins in. A transition matrix \(A\) says how regimes follow each other: \(a_{ij}\) is the chance of moving from state \(i\) to state \(j\). An emission model \(B\) says what each regime tends to produce: \(b_j(x)\) is the chance that state \(j\) emits observation \(x\).
Two assumptions make everything tractable. Tomorrow’s regime depends only on today’s. Today’s move depends only on today’s regime. Together they let the probability of a full story, hidden path \(z\) and observations \(x\), factor into a chain of small terms:
Bull regime
Bear regime
Everything you do with an HMM is one of three questions
How likely is this tape?
Sum the probability of the observations over every hidden path. This scores a model against data and lets you compare two models.
Forward algorithmWhat was the regime?
Find the single hidden path that best explains the observations, or the probability of each regime on each day.
Viterbi · forward–backwardWhat are the parameters?
Given only the tape, estimate \(A\) and \(B\). Nobody hands you labelled regimes, so counting transitions directly is impossible.
Baum–WelchForward sums. Viterbi takes the max.
Eight days and two regimes give \(2^8=256\) possible hidden paths. A trading year gives \(2^{252}\). Enumeration is hopeless, but the Markov assumption means a path’s past matters only through the state it is in now. So work one day at a time and keep one number per state.
The forward algorithm keeps \(\alpha_t(j)\), the probability of the tape so far, ending in state \(j\). Viterbi keeps \(\delta_t(j)\), the probability of the best single path so far ending in \(j\), plus a back-pointer to where that path came from. The two recursions differ by one operator:
Using tomorrow to judge today
The forward pass, normalised, gives the filtered belief \(P(z_t\mid x_{1:t})\): what you could know in real time. With the whole tape in hand you can do better. A backward pass computes \(\beta_t(i)\), the probability of everything after day \(t\) given state \(i\) on day \(t\):
The two messages meet at each day. Their product, normalised, is the smoothed belief:
Over a longer tape the difference is plain. The smoothed belief reacts earlier at regime changes and shrugs off isolated odd days, because later evidence confirms or contradicts each move.
In live use you cannot wait for the whole tape, but you can wait a little. Fixed-lag smoothing reports \(P(z_t\mid x_{1:t+L})\), the belief about day \(t\) formed \(L\) days later. Most of the benefit of hindsight arrives within the first few days of lag.
Two decoders, two different questions
Viterbi returns the single most probable path as a whole. Posterior decoding picks, for each day separately, the state with the largest \(\gamma_t\). The second gets more individual days right on average, and that is all it promises. Nothing forces its day-by-day choices to form a path the model allows.
The example below adds a third, range-bound regime and one structural rule: the market never jumps straight between bull and bear. It has to pass through the range.
Learning the model from the tape alone
If the regimes were labelled, estimation would be counting: the share of bull days followed by bear days, the share of bear days that closed down. They are not labelled, so Baum–Welch counts in expectation. It is the EM algorithm applied to an HMM, and it alternates two steps.
E-step. Run forward–backward under the current guess. This yields \(\gamma_t(i)\), the probability of state \(i\) on day \(t\), and \(\xi_t(i,j)\), the probability of the transition \(i\to j\) between days \(t\) and \(t+1\):
M-step. Re-estimate each parameter as a ratio of expected counts. It is the counting you would have done with labels, with soft weights in place of hard ones:
Below, 400 days are simulated from the model in section 1. The algorithm starts from a random guess and sees only the tape.
Why every EM step goes uphill
The likelihood \(\log P(x\mid\theta)\) is hard to maximise directly because it sums over every hidden path. EM sidesteps the sum. For any distribution \(q\) over paths, the log-likelihood splits into a tractable piece and a gap:
The E-step sets \(q\) to the posterior under the current parameters \(\theta_k\). The gap closes and the bound touches the likelihood at \(\theta_k\). The M-step moves \(\theta\) to the top of that bound, which is easy because the bound is a sum of weighted logs. The true likelihood at the new point is at least as high as the bound there, so
The figure shows this on a one-parameter slice: the probability that bull stays bull, with every other parameter held at its true value.
The likelihood surface has a mirror in it
With two free parameters the likelihood becomes a landscape. Take a stripped-down model: two equally sticky states and a tape of ups and downs, where the only unknowns are each state’s probability of emitting an up day.
Nothing in the likelihood says which state is called “state 1”. Swap the two labels and the fit is identical, so the surface is symmetric about the diagonal and has two equally good peaks. Along the diagonal the two states are identical twins, and EM started there has no reason to prefer either, so it stays. Everywhere else it runs to whichever peak is on its side.
From three symbols to real returns
Nothing in the algorithms requires a table of discrete emissions. Replace it with a density and the recursions are unchanged: wherever \(b_j(x_t)\) appeared, evaluate the density of state \(j\) at the observed value. The usual choice for returns gives each regime its own mean and volatility,
and the M-step becomes a weighted mean and a weighted variance, with the smoothed beliefs as weights:
This is the standard regime-switching model. Below, three years of daily returns are simulated from three regimes: calm, choppy and crisis. A Gaussian HMM is then fitted with Baum–Welch from the returns alone. Now the regimes overlap. A quiet day can happen in a crisis, and only persistence lets the model tell them apart.
How many states
In-sample likelihood cannot choose the number of states, because it never goes down when you add one. Two standard checks can. Held-out likelihood scores each fitted model on data it has not seen. The Bayesian information criterion penalises the in-sample fit by the number of free parameters \(p\):
The chart scores the fits from section 9, for one to five states, on the data shown there and on a second, unseen tape drawn the same way.
How long a regime lasts
An HMM decides each day whether to leave its state with the same coin. That makes the length of a stay geometric:
The implication is easy to miss. Whatever the persistence, the single most likely duration is one day, and a regime that has lasted a hundred days is no closer to ending than one that began this morning. If real regimes have a typical length, the model cannot express it. A hidden semi-Markov model fixes this by drawing each stay’s length from an explicit duration distribution, at the cost of heavier inference.
The numbers vanish, so rescale them
Each step of the forward recursion multiplies by a probability, so \(\alpha_t\) shrinks geometrically. Double-precision arithmetic bottoms out near \(10^{-308}\), and a naive implementation returns exactly zero on any tape of realistic length. There are two standard remedies. Work with logarithms and replace sums by log-sum-exp, or rescale \(\alpha\) to sum to one at every step and keep the scaling constants:
Rescaling is the cheaper of the two and gives the filtered beliefs \(\hat\alpha_t\) for free. Viterbi needs only the log transform, since a max of products becomes a max of sums. Every figure after the trellis uses these forms.
Beyond a single answer
Viterbi gives one path and Baum–Welch gives one set of parameters. Both hide how uncertain the answer is. The Bayesian treatment keeps the uncertainty by sampling.
Sampling paths. Run the forward filter, then walk backwards drawing each state from its distribution given the state just drawn: \(P(z_t=i\mid z_{t+1}=j,\,x_{1:t})\propto\hat\alpha_t(i)\,a_{ij}\). Every draw is a complete, coherent path from the posterior. The spread across draws shows which regime changes are certain and which are a matter of opinion.
Sampling parameters. Put Dirichlet priors on the rows of \(A\) and \(B\) and alternate: sample a path given the parameters, then sample the parameters given the path, which is again just counting. The result is a posterior distribution for every parameter, which says how much the tape actually pins each one down.
Where the HMM sits among its relatives
Most sequence models you will meet are an HMM with one assumption changed. Knowing which assumption tells you which of the algorithms above still apply. Each panel opens a short interactive guide to that model.
Mixture model
Each day’s state is drawn independently. An HMM whose transition rows are all equal is exactly a mixture.
Open the guide → Make the state continuousKalman filter
A Gaussian state with linear dynamics. The forward pass becomes the filter and forward–backward becomes the smoother.
Open the guide → Drop linearity and GaussianityParticle filter
A continuous state with arbitrary dynamics. The forward pass has no closed form, so it is approximated with weighted samples.
Open the guide → Model durations explicitlyHidden semi-Markov model
Each stay draws its length from a chosen distribution. Section 11 shows what this buys.
Open the guide →Hidden Markov model
Markov-switching autoregression
Hamilton’s regime-switching model. Each regime has its own autoregressive dynamics. Forward–backward still applies.
Open the guide → Model the labels directlyConditional random field
Models \(P(z\mid x)\) without modelling \(x\), so features can overlap freely. Same trellis, same Viterbi.
Open the guide → Use several chains at onceFactorial HMM
Independent hidden chains jointly drive the observation. Exact inference grows exponentially and is approximated.
Open the guide → Learn a continuous stateRecurrent neural network
The state is a learned vector updated deterministically. Far more expressive, with no exact inference and no readable regimes.
Open the guide →How this goes wrong in practice
- Regimes that are fitted noise. A multi-state HMM fitted to data with no regimes still returns regimes. Switch figure 10 to pure noise and fit three states: the picture looks persuasive. Only the held-out score in figure 11 says there is nothing there.
- The fit can beat the truth. In figure 7 the fitted likelihood often ends above the line for the true parameters. Maximum likelihood fits the sample, noise included, so in-sample likelihood overstates how well you know the model.
- Local optima. Baum–Welch finds the nearest peak. Run many restarts and keep the best, and treat a result that changes between restarts as a warning about the model, not just the optimiser.
- Labels are arbitrary. States must be identified after fitting by what they emit. Averaging parameters across restarts or posterior samples without aligning labels first produces nonsense.
- Smoothed is not tradable. The smoothed belief uses the future. Any backtest that conditions on \(\gamma_t\), or on parameters fitted to the full sample, has look-ahead bias. Real-time decisions get the filtered belief and parameters estimated on the past only.
- The world moves. The model assumes one fixed \(A\) and \(B\) for all time. If the regimes themselves drift, a model fitted on a long history describes a market that no longer exists. Rolling re-estimation helps, and brings label alignment problems with it.
- Degenerate states. With Gaussian emissions a state can collapse onto a single observation with variance near zero and unbounded likelihood. Put a floor on the variances or a prior on them.
| Algorithm | Answers | Recursion | Cost |
|---|---|---|---|
| Forward | \(P(x)\), filtered beliefs | sum over predecessors | \(O(S^2T)\) |
| Viterbi | best single path | max over predecessors, then trace back | \(O(S^2T)\) |
| Forward–backward | \(\gamma\) and \(\xi\), smoothed beliefs | forward message times backward message | \(O(S^2T)\) |
| Baum–Welch | \(\pi\), \(A\), \(B\) | forward–backward, then re-count, repeated | \(O(S^2T)\) per iteration |
| Forward filter, backward sample | draws from the posterior over paths | forward pass, then sample backwards | \(O(S^2T)\) per draw |
Trading on it
Sections 3 to 6 work on a tape that is already complete. A strategy stands at the end of the tape and has to act before the next move prints. Used without care, the same algorithms hand it information from days it has not lived through. There are four ways this happens.
- Smoothed probabilities use the future. Forward–backward (section 4) judges day \(t\) with every later day in hand. Only the filtered probability, \(P(z_t\mid x_{1:t})\), exists at the close of day \(t\). A backtest that trades on smoothed regimes, or on a Viterbi path decoded over the full sample, is trading on tomorrow.
- The decoded path rewrites itself. Run Viterbi again after each new day and earlier labels can change. A chart of regimes drawn today is not the chart you would have seen at the time.
- Parameters fitted on the whole sample know how it ends. Baum–Welch run on all the data learns its transition and emission probabilities from days the strategy has not reached. So does a number of states chosen as in section 10. Refit using data up to the decision date only.
- Labels are arbitrary and can swap between refits. State 1 in this month’s fit may be state 2 in last month’s. Identify a regime by a property, such as the lower mean or the higher volatility, never by its index.
The walk-forward line is the only one of these that could have been run. Compare it with buy and hold before believing the regime model added anything.
Sources and further reading
- Sections 1 to 6 and 12. L. R. Rabiner (1989), “A tutorial on hidden Markov models and selected applications in speech recognition”, Proceedings of the IEEE 77(2), 257–286.
- Section 3. A. J. Viterbi (1967), “Error bounds for convolutional codes and an asymptotically optimum decoding algorithm”, IEEE Transactions on Information Theory 13(2), 260–269.
- Section 6. L. E. Baum, T. Petrie, G. Soules and N. Weiss (1970), “A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains”, Annals of Mathematical Statistics 41(1), 164–171.
- Section 7. A. P. Dempster, N. M. Laird and D. B. Rubin (1977), “Maximum likelihood from incomplete data via the EM algorithm”, Journal of the Royal Statistical Society B 39(1), 1–38.
- Sections 9 and 16. J. D. Hamilton (1989), “A new approach to the economic analysis of nonstationary time series and the business cycle”, Econometrica 57(2), 357–384.
- Section 10. G. Schwarz (1978), “Estimating the dimension of a model”, Annals of Statistics 6(2), 461–464.
- Section 13. S. L. Scott (2002), “Bayesian methods for hidden Markov models: recursive computing in the 21st century”, Journal of the American Statistical Association 97(457), 337–351.
- Section 14. R. E. Kalman (1960), “A new approach to linear filtering and prediction problems”, Journal of Basic Engineering 82, 35–45. Z. Ghahramani and M. I. Jordan (1997), “Factorial hidden Markov models”, Machine Learning 29, 245–273. J. Lafferty, A. McCallum and F. Pereira (2001), “Conditional random fields: probabilistic models for segmenting and labeling sequence data”, Proceedings of ICML.