Markov Chain: What It Is & Applications in Finance 2026

What Is a Markov Chain?

A Markov chain is a sequence of random variables where the probability of moving to the next state depends only on the current state - not on any of the states that came before it. This property is called the Markov property, or memorylessness, and it's the defining feature of the model.

Formally, if X_0, X_1, X_2, ... is a sequence of random variables taking values in a finite or countable set of states S, the process is a Markov chain if:

P(X_{n+1} = j | X_n = i, X_{n-1}, ..., X_0) = P(X_{n+1} = j | X_n = i)

The entire history of the process is irrelevant. All the information you need to predict the future is captured in where the chain sits right now. This sounds like a severe restriction, but it turns out to be an excellent approximation for many real-world systems - and it makes the mathematics tractable.

Markov chains are named after Andrey Markov, who introduced them in 1906 to study the alternation of vowels and consonants in Russian poetry. Since then, they've become one of the most widely used tools in probability theory, with applications spanning finance, physics, biology, natural language processing, and search engine algorithms. In quantitative finance, Markov chain models are used for credit rating transitions, regime detection, options pricing on lattices, and Bayesian inference via MCMC.

---

Transition Matrices

A Markov chain with a finite number of states can be fully described by a transition matrix. Each entry P_{ij} gives the probability of moving from state i to state j in one step.

For a chain with n states, the transition matrix P is an n x n matrix where:

• Every entry is non-negative: P_{ij} >= 0
• Every row sums to 1: the sum of P_{ij} over all j equals 1 for each i

Each row represents a probability distribution over where the chain goes next, given its current state.

Reading a Transition Matrix

Consider a simple two-state Markov chain model with states A and B:

If the chain is currently in state A, there's a 70% chance it stays in A and a 30% chance it moves to B. From state B, there's a 40% chance of jumping to A and a 60% chance of staying in B.

Multi-Step Transitions

One of the most useful properties of the transition matrix: to find the probability of being in state j after k steps starting from state i, you raise the matrix to the kth power. The (i, j) entry of P^k gives you that probability directly. This is straightforward with NumPy and we'll see it in code shortly.

---

A Simple Markov Chain Example: Weather Model

The classic introductory Markov chain example is a weather model. Suppose the weather each day is either Sunny or Rainy, and tomorrow's weather depends only on today's weather with the following transition probabilities:

• If today is Sunny: 80% chance tomorrow is Sunny, 20% chance tomorrow is Rainy
• If today is Rainy: 40% chance tomorrow is Sunny, 60% chance tomorrow is Rainy

The transition matrix is:

P = [[0.8, 0.2], [0.4, 0.6]]

Simulating the Chain in Python

import numpy as np

def simulate_markov_chain(transition_matrix, states, start_state, n_steps):
    """Simulate a discrete-time Markov chain."""
    current = states.index(start_state)
    path = [start_state]

    for _ in range(n_steps):
        current = np.random.choice(
            len(states), p=transition_matrix[current]
        )
        path.append(states[current])

    return path

states = ["Sunny", "Rainy"]
P = np.array([[0.8, 0.2],
              [0.4, 0.6]])

np.random.seed(42)
path = simulate_markov_chain(P, states, "Sunny", 20)
print(" -> ".join(path))

Running this produces a sequence like: Sunny -> Sunny -> Sunny -> Rainy -> Rainy -> Sunny -> ... The chain spends more time in Sunny because both states have a higher probability of transitioning to (or staying in) Sunny than Rainy. We'll formalise this intuition when we look at stationary distributions.

---

Key Properties of Markov Chains

Not all Markov chains behave the same way. Several properties determine the long-run behaviour of a chain and whether useful results like stationary distributions exist.

Irreducibility

A Markov chain is irreducible if every state can be reached from every other state. There's no subset of states that traps the chain permanently. If a chain is reducible - meaning some states are unreachable from others - it may have different long-run behaviours depending on the starting state, which complicates the analysis.

Periodicity

A state has period d if the chain can only return to that state in multiples of d steps. A state with period 1 is called aperiodic. A chain is aperiodic if all its states are aperiodic. Periodicity matters because a periodic chain cycles through states rather than settling into a steady distribution.

A common example of periodicity: imagine a chain on two states where P_{AB} = 1 and P_{BA} = 1. The chain bounces back and forth deterministically - it returns to each state only every 2 steps. This chain has period 2 and will never converge to a stationary distribution in the usual sense.

Ergodicity

A Markov chain is ergodic if it is both irreducible and aperiodic. Ergodic chains have a unique stationary distribution, and regardless of the starting state, the chain converges to this distribution over time. This is the property that makes Markov chains so useful in practice - ergodic chains "forget" their initial conditions.

Absorbing States

An absorbing state is one that, once entered, can never be left. In the transition matrix, the row for an absorbing state has a 1 on the diagonal and 0 everywhere else. Markov chains with absorbing states are called absorbing chains. The main questions for absorbing chains are: what is the probability of eventually being absorbed, and how long does it take on average? Credit default models often use absorbing states to represent the "default" state - once a company defaults, it stays in default.

---

Stationary Distribution

The stationary distribution (also called the steady-state or equilibrium distribution) is a probability distribution pi over the states such that if the chain is distributed according to pi at time n, it's still distributed according to pi at time n+1. Mathematically:

pi * P = pi

where pi is a row vector with entries summing to 1.

For an ergodic Markov chain, the stationary distribution is unique, and the chain converges to it regardless of the starting state. The stationary distribution tells you the long-run fraction of time the chain spends in each state.

Computing the Stationary Distribution

There are two common approaches. The first is to solve the linear system pi * P = pi with the constraint that the entries of pi sum to 1. The second is to raise the transition matrix to a large power - the rows of P^k converge to the stationary distribution as k grows.

import numpy as np

P = np.array([[0.8, 0.2],
              [0.4, 0.6]])

# Method 1: eigenvalue decomposition
eigenvalues, eigenvectors = np.linalg.eig(P.T)
stationary_idx = np.argmin(np.abs(eigenvalues - 1.0))
stationary = np.real(eigenvectors[:, stationary_idx])
stationary = stationary / stationary.sum()
print(f"Stationary distribution (eigenvalue method): {stationary}")

# Method 2: repeated matrix multiplication
P_power = np.linalg.matrix_power(P, 100)
print(f"Stationary distribution (matrix power):      {P_power[0]}")

# Method 3: solve the linear system directly
n = P.shape[0]
A = np.vstack([(P.T - np.eye(n)), np.ones(n)])
b = np.zeros(n + 1)
b[-1] = 1.0
pi = np.linalg.lstsq(A, b, rcond=None)[0]
print(f"Stationary distribution (linear system):     {pi}")

All three methods produce the same answer: pi = [2/3, 1/3], or roughly [0.667, 0.333]. In the long run, the weather chain spends about two-thirds of the time Sunny and one-third Rainy.

---

Markov Chains in Python: A Complete Workflow

Let's put together a more complete example with a three-state chain, simulation, convergence analysis, and visualisation of how the state distribution evolves over time.

import numpy as np

states = ["Bull", "Flat", "Bear"]
P = np.array([
    [0.70, 0.20, 0.10],
    [0.25, 0.50, 0.25],
    [0.15, 0.30, 0.55]
])

assert np.allclose(P.sum(axis=1), 1.0), "Rows must sum to 1"

# Simulate 10,000 steps
np.random.seed(123)
n_steps = 10_000
current = 0
state_counts = np.zeros(3)
history = np.zeros(3)

for step in range(1, n_steps + 1):
    current = np.random.choice(3, p=P[current])
    state_counts[current] += 1

    if step in [10, 100, 1000, 10000]:
        freq = state_counts / step
        print(f"Step {step:>5}: Bull={freq[0]:.3f}  Flat={freq[1]:.3f}  Bear={freq[2]:.3f}")

# Compute stationary distribution analytically
eigenvalues, eigenvectors = np.linalg.eig(P.T)
idx = np.argmin(np.abs(eigenvalues - 1.0))
pi = np.real(eigenvectors[:, idx])
pi = pi / pi.sum()
print(f"\nStationary: Bull={pi[0]:.3f}  Flat={pi[1]:.3f}  Bear={pi[2]:.3f}")

# Multi-step transition probabilities
P_10 = np.linalg.matrix_power(P, 10)
print(f"\n10-step transition matrix:\n{np.round(P_10, 4)}")

As the simulation runs, the empirical frequencies converge to the stationary distribution. After 10 steps, the estimates are noisy. After 10,000, they match the analytical stationary distribution closely. This convergence is guaranteed by the ergodic theorem - one of the most important results in Markov chain theory.

---

Hidden Markov Models

A Hidden Markov Model (HMM) extends the basic Markov chain by adding a layer of uncertainty: you can't directly observe which state the chain is in. Instead, you observe noisy signals that depend on the hidden state.

An HMM has three components:

• Transition matrix (A): the probabilities of moving between hidden states, exactly as in a standard Markov chain
• Emission matrix (B): the probability of observing each possible output given the hidden state
• Initial distribution (pi): the probability of starting in each hidden state

The classic analogy is a casino with an honest die and a loaded die. The dealer switches between them according to a Markov chain (the hidden state), but you can only see the numbers rolled (the observations). From the sequence of rolls, you want to infer which die was being used at each point.

Three Fundamental HMM Problems

1. Evaluation (Forward algorithm): given a model and a sequence of observations, what is the probability of that sequence?
2. Decoding (Viterbi algorithm): given a model and observations, what is the most likely sequence of hidden states?
3. Learning (Baum-Welch algorithm): given observations, what model parameters best explain the data?

HMM Example in Python

import numpy as np

class SimpleHMM:
    """Minimal Hidden Markov Model implementation."""

    def __init__(self, transition, emission, initial):
        self.A = np.array(transition)
        self.B = np.array(emission)
        self.pi = np.array(initial)
        self.n_states = self.A.shape[0]

    def forward(self, observations):
        """Forward algorithm: compute P(observations | model)."""
        T = len(observations)
        alpha = np.zeros((T, self.n_states))
        alpha[0] = self.pi * self.B[:, observations[0]]

        for t in range(1, T):
            for j in range(self.n_states):
                alpha[t, j] = (
                    np.sum(alpha[t - 1] * self.A[:, j])
                    * self.B[j, observations[t]]
                )
        return alpha, np.sum(alpha[-1])

    def viterbi(self, observations):
        """Viterbi algorithm: find most likely state sequence."""
        T = len(observations)
        delta = np.zeros((T, self.n_states))
        psi = np.zeros((T, self.n_states), dtype=int)

        delta[0] = self.pi * self.B[:, observations[0]]

        for t in range(1, T):
            for j in range(self.n_states):
                probs = delta[t - 1] * self.A[:, j]
                psi[t, j] = np.argmax(probs)
                delta[t, j] = np.max(probs) * self.B[j, observations[t]]

        path = np.zeros(T, dtype=int)
        path[-1] = np.argmax(delta[-1])
        for t in range(T - 2, -1, -1):
            path[t] = psi[t + 1, path[t + 1]]

        return path

# Market regime HMM: hidden states are Bull (0) and Bear (1)
# Observations: 0 = positive return, 1 = small move, 2 = negative return
hmm = SimpleHMM(
    transition=[[0.90, 0.10],
                [0.15, 0.85]],
    emission=[[0.6, 0.3, 0.1],
              [0.1, 0.3, 0.6]],
    initial=[0.6, 0.4]
)

observations = [0, 0, 1, 2, 2, 2, 1, 0, 0, 0, 1, 2, 2, 1, 0]
_, likelihood = hmm.forward(observations)
states = hmm.viterbi(observations)

labels = ["Bull", "Bear"]
print(f"Log-likelihood: {np.log(likelihood):.4f}")
print(f"Decoded states:  {[labels[s] for s in states]}")

The Viterbi algorithm finds the most likely sequence of hidden states. Notice how it detects the switch from Bull to Bear when negative returns cluster together, then back to Bull when positive returns return.

---

Applications of Markov Chains in Finance

Markov chain models appear throughout quantitative finance. Here are the most important applications you'll encounter in 2026.

Regime Detection with Hidden Markov Models

Financial markets alternate between distinct regimes - bull and bear markets, high and low volatility periods, risk-on and risk-off environments. HMMs provide a principled statistical framework for identifying these regimes from observed return data.

The hmmlearn library fits Gaussian HMMs to continuous return data, making regime detection straightforward:

import numpy as np

np.random.seed(42)

# Simulate two-regime returns
n = 500
regime = np.zeros(n, dtype=int)
for t in range(1, n):
    if regime[t - 1] == 0:
        regime[t] = 0 if np.random.random() < 0.95 else 1
    else:
        regime[t] = 1 if np.random.random() < 0.90 else 0

returns = np.where(
    regime == 0,
    np.random.normal(0.05, 0.8, n),
    np.random.normal(-0.10, 1.8, n)
)

# Fit a 2-state Gaussian HMM
from hmmlearn.hmm import GaussianHMM

model = GaussianHMM(n_components=2, covariance_type="full", n_iter=200)
model.fit(returns.reshape(-1, 1))

predicted_regimes = model.predict(returns.reshape(-1, 1))

# Identify which fitted state maps to "Bull" (higher mean)
means = model.means_.flatten()
bull_state = np.argmax(means)
accuracy = np.mean(predicted_regimes == (regime if bull_state == 0 else 1 - regime))
print(f"Fitted means:  {means}")
print(f"Fitted vars:   {model.covars_.flatten()}")
print(f"Regime detection accuracy: {accuracy:.1%}")

Regime-switching models are used by systematic hedge funds and asset managers to adjust portfolio allocations dynamically. When the model signals a shift to a bear regime, the strategy can reduce equity exposure, increase hedges, or shift into defensive assets. For a deeper look at the statistical methods behind these approaches, see our dedicated guide.

Credit Rating Transitions

Credit rating agencies (Moody's, S&P, Fitch) assign ratings to corporate bonds - AAA, AA, A, BBB, and so on down to default. The movement of a company's rating over time is naturally modelled as a Markov chain, where the states are rating categories and the transition matrix describes the probability of moving between ratings over a one-year horizon.

Default is an absorbing state - once a company defaults, it stays in default. Banks use these matrices to calculate the probability of default over multi-year horizons (raise the matrix to the appropriate power) and to price credit derivatives. The transition matrix above is simplified but representative of the kind published by rating agencies. In 2026, these matrices underpin regulatory capital calculations under Basel III and internal credit risk models at every major bank.

Options Pricing on Lattices

The binomial tree model for options pricing is a Markov chain in disguise. At each time step, the stock price moves up by a factor u or down by a factor d, with probabilities p and (1-p). The stock price at the next node depends only on the current node - not on the path taken to get there. This is the Markov property applied directly to asset pricing.

The Cox-Ross-Rubinstein binomial model, which converges to the Black-Scholes price as the number of steps increases, is the most widely used lattice method. Trinomial trees extend this to three possible moves per step, giving faster convergence.

Markov Chain Monte Carlo (MCMC)

MCMC is one of the most important applications of Markov chain theory in modern quantitative finance. The idea: construct a Markov chain whose stationary distribution is the probability distribution you want to sample from. Run the chain long enough, and the samples approximate draws from that target distribution.

This is the engine behind Bayesian inference in complex models. When you can't compute a posterior distribution analytically - which is almost always in real-world models with many parameters - MCMC lets you draw samples from it instead.

The two most common MCMC algorithms are:

• Metropolis-Hastings: propose a move, accept or reject it based on how much the move improves (or worsens) the target density
• Gibbs sampling: update one parameter at a time, sampling from its conditional distribution given all others

import numpy as np

def metropolis_hastings(target_log_pdf, initial, n_samples, proposal_std=1.0):
    """Simple Metropolis-Hastings sampler."""
    samples = np.zeros(n_samples)
    current = initial
    current_log_p = target_log_pdf(current)
    accepted = 0

    for i in range(n_samples):
        proposal = current + np.random.normal(0, proposal_std)
        proposal_log_p = target_log_pdf(proposal)

        log_accept_ratio = proposal_log_p - current_log_p
        if np.log(np.random.random()) < log_accept_ratio:
            current = proposal
            current_log_p = proposal_log_p
            accepted += 1

        samples[i] = current

    print(f"Acceptance rate: {accepted / n_samples:.1%}")
    return samples

# Sample from a mixture of two normals
def target(x):
    from scipy.stats import norm
    return np.log(0.3 * norm.pdf(x, -2, 0.8) + 0.7 * norm.pdf(x, 3, 1.2))

np.random.seed(42)
samples = metropolis_hastings(target, initial=0.0, n_samples=50_000, proposal_std=2.0)
burn_in = 5000
samples = samples[burn_in:]

print(f"Sample mean: {samples.mean():.3f}")
print(f"Sample std:  {samples.std():.3f}")

MCMC is used in finance for calibrating stochastic volatility models, estimating credit risk parameters, fitting Bayesian portfolio models, and any situation where the posterior distribution of model parameters doesn't have a nice closed form.

The connection to Markov chains is direct: the Metropolis-Hastings algorithm constructs an ergodic Markov chain whose stationary distribution is the target distribution. The theoretical guarantee that the chain converges to the stationary distribution - the same result we discussed earlier - is what makes MCMC work.

---

Continuous-Time Markov Chains

Everything we've discussed so far involves discrete-time Markov chains - the chain transitions at fixed time steps. Continuous-time Markov chains (CTMCs) allow transitions to occur at any point in time.

In a CTMC, the chain stays in state i for an exponentially distributed random time with rate q_i, then jumps to state j with probability proportional to q_{ij}. The dynamics are described by a generator matrix Q (sometimes called the rate matrix or intensity matrix) rather than a transition matrix.

The relationship between the generator matrix Q and the transition matrix P(t) over a time interval of length t is:

P(t) = exp(Q * t)

where exp denotes the matrix exponential.

Applications in Finance

CTMCs appear naturally in several financial contexts:

• Credit rating migrations: while agencies publish annual transition matrices, the underlying process is continuous. Regulators and risk managers use generator matrices to compute transition probabilities over arbitrary horizons (6 months, 3 years, etc.)
• Poisson processes: the simplest CTMC is a counting process where events (trades, defaults, jumps) arrive at a constant rate. Jump-diffusion models for asset prices combine continuous Brownian motion with a Poisson jump component
• Queue models: order book dynamics can be modelled as a birth-death process, a special case of a CTMC, where buy and sell orders arrive and get filled at random rates

import numpy as np
from scipy.linalg import expm

# Generator matrix for a 3-state credit model (A, BBB, Default)
Q = np.array([
    [-0.05,  0.04,  0.01],
    [ 0.02, -0.10,  0.08],
    [ 0.00,  0.00,  0.00]
])

# Transition probabilities over different horizons
for t in [1, 3, 5, 10]:
    P_t = expm(Q * t)
    print(f"\n{t}-year transition matrix:")
    print(np.round(P_t, 4))

# Probability of default within 5 years, starting from A
P_5 = expm(Q * 5)
print(f"\nP(default within 5 years | start A):   {P_5[0, 2]:.4f}")
print(f"P(default within 5 years | start BBB): {P_5[1, 2]:.4f}")

The matrix exponential gives exact transition probabilities over any time horizon, which is essential for pricing credit-sensitive instruments with non-standard maturities.

---

Markov Chains vs Other Stochastic Processes

Understanding where Markov chains fit relative to other models helps you choose the right tool.

Brownian motion and random walks are continuous-state Markov processes - they satisfy the Markov property, but the state space is the real line rather than a finite set of states. The stock price models used in options pricing are Markov processes, which is why lattice methods (Markov chains on a discrete approximation of the price space) can price them.

---

Frequently Asked Questions

What is the Markov property in simple terms?

The Markov property says that the future depends only on the present, not the past. If you know the current state of a system, knowing its entire history gives you no additional predictive power. In a weather model, if today is Sunny, the probability of rain tomorrow is the same regardless of whether yesterday was sunny or rainy. This memorylessness is what makes Markov chains mathematically tractable - you only need to track the current state, not the full history.

What is a Markov chain example in finance?

Credit rating transitions are the most direct example. A company rated BBB has some probability of being upgraded to A, staying at BBB, being downgraded to BB, or defaulting over the next year. These probabilities form a transition matrix, and the sequence of ratings over time is a Markov chain. Banks use these models daily for credit risk calculations, regulatory capital requirements, and pricing credit derivatives. Default is an absorbing state - once a company enters it, the chain stays there.

What is the difference between a Markov chain and a Markov process?

A Markov chain typically refers to a process with a discrete (finite or countable) state space - the chain jumps between distinct states. A Markov process is the broader category that includes continuous state spaces. Brownian motion, for instance, is a Markov process but not a Markov chain because its state (position) can take any real value. In practice, many people use "Markov chain" and "Markov process" interchangeably when the distinction isn't critical.

How are Hidden Markov Models used in trading?

HMMs are primarily used for regime detection - identifying whether the market is in a bull, bear, or sideways regime based on observed returns. The hidden states represent the unobservable market regime, and the observations are the returns (or other features) you can measure. Once the model identifies the current regime, a trading strategy can adapt - reducing exposure in bear regimes, increasing it in bull regimes, and adjusting volatility estimates accordingly. Several systematic hedge funds have used HMM-based regime models as part of their allocation frameworks since the early 2000s.

What is Markov Chain Monte Carlo (MCMC) used for?

MCMC is used to sample from probability distributions that are difficult or impossible to sample from directly. In finance, this means fitting complex models where the posterior distribution of parameters doesn't have a closed-form expression. Common applications include calibrating stochastic volatility models, estimating credit risk parameters, Bayesian portfolio optimisation, and fitting factor models with many latent variables. The "Markov chain" in MCMC refers to the fact that the sampling algorithm constructs a chain whose stationary distribution matches the target distribution you want to sample from.

Can Markov chains model stock prices directly?

In theory, a Markov chain with discrete price levels could approximate stock price movements, and this is exactly what binomial and trinomial tree models do for options pricing. However, for modelling real stock price dynamics, continuous-state Markov processes (geometric Brownian motion, stochastic volatility models) are far more appropriate. The efficient market hypothesis itself implies that stock prices follow a Markov process - past prices shouldn't help predict future prices beyond what's captured in the current price. Whether markets are truly Markov is debatable, but it's a reasonable starting approximation for many purposes.