Expected Shortfall (CVaR): The Coherent Risk Measure Replacing VaR

What Is Expected Shortfall?

Expected Shortfall (ES), also known as Conditional Value at Risk (CVaR) and occasionally as Average Value at Risk (AVaR) or Tail Value at Risk (TVaR), is a risk measure that answers a slightly different question than Value at Risk (VaR).

VaR asks: "What is the loss I will not exceed with probability (\alpha)?"

Expected Shortfall asks: "Given that I exceed the VaR threshold, what is the expected loss?"

Formally, for a loss random variable (L) and confidence level (\alpha \in (0, 1)) (usually 0.95 or 0.99):

ES_alpha(L) = E[ L | L >= VaR_alpha(L) ]

Equivalently, and more usefully for continuous distributions:

ES_alpha(L) = (1 / (1 - alpha)) * integral_alpha^1 VaR_u(L) du

ES is the average of the VaR values in the tail beyond the confidence level. Where VaR ignores the shape of the tail and reports only the threshold, ES incorporates the magnitude of losses that occur when the threshold is breached.

This distinction matters. Under the Basel Committee's Fundamental Review of the Trading Book (FRTB), banks calculate market-risk capital requirements using ES at the 97.5% level, not VaR (Basel Committee on Banking Supervision, 2019). Insurance regulators, asset managers, and clearing houses have followed similar directions. Understanding ES — its definition, estimation, backtesting, and interpretation — is now a baseline requirement for anyone working in risk. The definitive textbook treatment is McNeil, Frey & Embrechts (2015); the coherence axioms originate in Artzner, Delbaen, Eber & Heath (1999).

---

Why VaR Was Never Enough

Value at Risk became the standard market-risk measure in the 1990s, with RiskMetrics as its most influential incarnation. It has genuine strengths: it is a single, interpretable number in the currency of the portfolio; it aggregates across asset classes; it can be backtested by counting exceedances.

But VaR has three well-documented failings.

It Tells You Nothing About the Tail Beyond the Threshold

Two portfolios can have identical 99% one-day VaR of £10m and radically different tail behaviour: one might have a maximum plausible daily loss of £11m, the other £500m. VaR would rank them equally risky. This is not a hypothetical — the LTCM collapse in 1998 and the 2007-2008 losses in structured-credit portfolios were both instances of tail losses well outside the VaR envelope.

It Is Not Subadditive

A risk measure (\rho) is subadditive if (\rho(A + B) \leq \rho(A) + \rho(B)) for any two portfolios (A) and (B). Economically, this says that combining portfolios should not increase risk — diversification never hurts. VaR can violate this. Two portfolios with modest individual VaRs can have a combined VaR that exceeds their sum, particularly when the loss distributions have discrete jumps or extreme skewness.

Subadditivity is not a mere academic property. Without it, capital allocation across business units can produce perverse incentives: a desk can lower its measured VaR by adding a specific, correlated exposure. Regulators and internal risk teams find this behaviour dangerous.

It Is Not Coherent

Artzner, Delbaen, Eber and Heath (1999) formalised the properties a "good" risk measure should have: monotonicity, subadditivity, positive homogeneity, and translation invariance. Together these define a coherent risk measure. VaR fails subadditivity. Expected Shortfall satisfies all four properties and is therefore coherent (see also McNeil, Frey & Embrechts, 2015, ch. 2).

---

Estimating Expected Shortfall

Three families of estimators dominate practice.

Historical Simulation

The simplest approach uses the empirical distribution of past returns.

1. Compute portfolio P&L over the past (N) days (usually 250–500).
2. Sort the losses from smallest to largest.
3. VaR at level (\alpha) is the (\alpha)-quantile.
4. ES is the average of the losses that exceed VaR.

For a 99% ES with 500 days of data, sort the losses and average the worst 5.

Strengths: no distributional assumptions; captures all observed dependencies and non-linearities.

Weaknesses: limited by sample size — the 99% ES from 250 days is the average of 2.5 observations, which is extremely noisy. Assumes the future looks like the past. Ignores forward-looking information.

Parametric (Variance-Covariance)

Assume returns are normally distributed with mean (\mu) and variance (\sigma^2). Then VaR and ES have closed-form expressions:

VaR_alpha = mu + sigma * z_alpha

ES_alpha = mu + sigma * phi(z_alpha) / (1 - alpha)

where (z_\alpha) is the (\alpha)-quantile of the standard normal and (\phi) is its density. For a zero-mean series and (\alpha = 0.99), (z_\alpha = 2.326), (\phi(z_\alpha) = 0.0267), giving (\text{ES}{0.99} = \sigma \cdot 2.667). Compare with (\text{VaR}{0.99} = \sigma \cdot 2.326): ES is roughly 15% larger than VaR under the normal assumption.

Strengths: simple, closed-form, easy to explain.

Weaknesses: the normal assumption dramatically understates tail risk in real returns. A t-distribution with 4-6 degrees of freedom is a better fit but requires numerical evaluation of ES.

Monte Carlo

Simulate (N) scenarios of portfolio P&L under an explicit model (multi-asset GBM, Heston stochastic volatility, a factor model with fat tails). ES is the sample mean of the worst ((1 - \alpha) N) losses.

Strengths: as flexible as the model. Handles path-dependent positions, non-linear derivatives, credit events, and stress scenarios.

Weaknesses: as reliable as the model. Requires simulation infrastructure and calibration effort. Standard error scales with (\sqrt{N}), so precise tail estimation demands many paths.

Python Implementation

import numpy as np
import pandas as pd
from scipy import stats


def historical_var_es(losses: pd.Series, alpha: float = 0.99) -> dict:
    """Historical VaR and ES from a series of losses (positive = loss)."""
    losses = losses.dropna().sort_values().reset_index(drop=True)
    n = len(losses)
    var_index = int(np.ceil(alpha * n)) - 1
    var = losses.iloc[var_index]
    tail = losses.iloc[var_index:]
    es = tail.mean()
    return {"var": var, "es": es, "n_tail": len(tail)}


def parametric_var_es_normal(mu: float, sigma: float, alpha: float = 0.99) -> dict:
    """Closed-form VaR and ES assuming normal losses."""
    z = stats.norm.ppf(alpha)
    var = mu + sigma * z
    es = mu + sigma * stats.norm.pdf(z) / (1 - alpha)
    return {"var": var, "es": es}


def parametric_var_es_t(mu: float, sigma: float, df: float, alpha: float = 0.99) -> dict:
    """VaR and ES for a scaled Student-t distribution."""
    t_alpha = stats.t.ppf(alpha, df)
    var = mu + sigma * t_alpha
    density = stats.t.pdf(t_alpha, df)
    es = mu + sigma * (density / (1 - alpha)) * (df + t_alpha**2) / (df - 1)
    return {"var": var, "es": es}


# Example: 1,000 daily returns of a portfolio with 15 bp mean and 1.2% vol
np.random.seed(0)
returns = pd.Series(np.random.standard_t(df=5, size=1000) * 0.012 + 0.0015)
losses = -returns

hist = historical_var_es(losses, alpha=0.99)
norm = parametric_var_es_normal(mu=-losses.mean(), sigma=losses.std(), alpha=0.99)
t_fit = parametric_var_es_t(mu=-losses.mean(), sigma=losses.std(), df=5, alpha=0.99)

print(f"Historical VaR 99%:      {hist['var']:.4f}")
print(f"Historical ES 99%:       {hist['es']:.4f}   (avg of worst {hist['n_tail']})")
print()
print(f"Normal parametric VaR:   {norm['var']:.4f}")
print(f"Normal parametric ES:    {norm['es']:.4f}")
print()
print(f"Student-t (df=5) VaR:    {t_fit['var']:.4f}")
print(f"Student-t (df=5) ES:     {t_fit['es']:.4f}")

The normal parametric estimate materially understates both VaR and ES because the true innovations are Student-t; the t estimate is closer to historical.

---

Coherence: The Four Axioms

The formal justification for preferring ES over VaR is that ES is a coherent risk measure in the Artzner-Delbaen-Eber-Heath sense. The four axioms:

1. Monotonicity. If portfolio (A) always loses at least as much as portfolio (B) in every state, then (\rho(A) \geq \rho(B)). Both VaR and ES satisfy this.
2. Subadditivity. (\rho(A + B) \leq \rho(A) + \rho(B)). Diversification cannot increase risk. VaR fails this in general; ES satisfies it.
3. Positive homogeneity. (\rho(\lambda A) = \lambda \rho(A)) for (\lambda > 0). Doubling the position doubles the risk. Both satisfy this.
4. Translation invariance. (\rho(A + c) = \rho(A) - c) for any constant cash (c). Adding risk-free cash reduces measured risk by the amount added. Both satisfy this.

Subadditivity is the deal-breaker for VaR in a regulatory context. A capital framework built on a non-coherent risk measure can produce economically incoherent incentives — legitimising position concentrations that increase system risk. This is a principal reason FRTB moved capital calculations to ES at the 97.5% confidence level.

---

Backtesting Expected Shortfall

Backtesting VaR is easy: count exceedances and compare against expected frequency. Backtesting ES is harder because ES is a conditional expectation, not a quantile.

The Basel Regulatory Approach

Rather than backtest ES directly, Basel III/IV requires banks to backtest VaR at 97.5% and 99% levels and to use the ES value from an internal model, subject to model approval and traffic-light thresholds on VaR exceedances. The reasoning: VaR is easier to backtest, and if a model produces well-calibrated VaR then its ES estimate is likely reasonable as well.

Direct ES Backtests

Recent academic work has produced backtests of ES itself. Two commonly cited procedures:

• Acerbi and Szekely (2014). Multiple test statistics based on the null hypothesis that observed losses in the tail equal predicted ES. Statistical software packages implement several variants.
• Kratz, Lok and McNeil (2018). A joint backtest of ES and VaR built around the "multinomial backtest" of tail counts at several confidence levels.

These tests have moderate statistical power. Because tail observations are rare, direct backtesting of 97.5% ES requires large samples (thousands of days) to distinguish a well-calibrated model from a misspecified one.

A Simple Practitioner Check

At minimum, verify two things:

1. Coverage at VaR. For a 99% one-day model, exceedances should occur roughly 1% of the time. Significantly more indicates a model that is too optimistic in the tails.
2. Average size of exceedances. Compute the average loss on days that exceeded VaR. It should be close to the predicted ES. Substantially larger indicates the model captures where the tail begins but not how bad it gets.

If both checks pass, the model is a reasonable candidate for use. If either fails, revisit the distributional assumptions.

---

ES for Regulatory Capital: FRTB

Basel's Fundamental Review of the Trading Book (FRTB), finalised in January 2019 and phased in over subsequent years, replaced the previous VaR-based capital regime with an internal-models approach based on ES at the 97.5% level over varying liquidity horizons.

Key features:

• 97.5% ES over 10 days replaces 99% VaR over 10 days.
• Liquidity horizons vary by risk factor — from 10 days for liquid FX and rates to 120 days for less liquid credit and equity structural positions. Capital is calculated by scaling from these horizons.
• Stress calibration — the ES calibration uses a period of significant financial stress, similar to the earlier Stressed VaR framework but based on ES rather than VaR.
• Non-modellable risk factors (NMRFs) — factors without sufficient real observations receive a stressed capital add-on outside the ES framework.

The overall effect has been a material increase in market-risk capital requirements at most banks, along with a substantial modelling and data-infrastructure burden.

---

ES Beyond Basel

ES appears in many other risk contexts:

• Insurance. Solvency II uses 99.5% one-year VaR, not ES, but many internal models compute both and life insurers increasingly report on a TVaR (= ES) basis.
• Clearing houses (CCPs). Initial margin models typically calculate risk on an ES basis, often at 99% over 2-5 day close-out periods.
• Asset management. Many long-only managers report ES alongside VaR in risk factsheets. Alternative asset classes with heavy tails (private credit, distressed debt) rely on ES for internal risk limits.
• Portfolio optimisation. Mean-CVaR optimisation, developed by Rockafellar and Uryasev (2000), provides a linear-programming reformulation that produces portfolios optimised on tail risk rather than variance. This has become a standard alternative to mean-variance optimisation for portfolios with skewed or fat-tailed exposures.

---

Practical Recommendations

• Use ES rather than VaR as your primary risk number wherever possible. Report both if the audience is more familiar with VaR.
• Choose the estimation method that matches the portfolio. Historical for linear, well-observed portfolios with enough data. Monte Carlo for portfolios with derivatives, path dependence, or credit events. Parametric only for high-level top-of-house views.
• Do not use the normal distribution for tail estimation on real financial data. Fit a t-distribution or use extreme value theory for the tail.
• Backtest at the VaR level and check that observed tail losses on exceedance days approximate the predicted ES.
• Report ES with confidence intervals when computed from historical or Monte Carlo samples. Tail estimates are noisy.

---

References

• Acerbi, C., & Szekely, B. (2014). Back-testing expected shortfall. Risk Magazine, 27(11), 76–81.
• Artzner, P., Delbaen, F., Eber, J.-M., & Heath, D. (1999). Coherent measures of risk. Mathematical Finance, 9(3), 203–228.
• Basel Committee on Banking Supervision. (2019). Minimum capital requirements for market risk (January 2019 revision). Bank for International Settlements.
• Kratz, M., Lok, Y. H., & McNeil, A. J. (2018). Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall. Journal of Banking & Finance, 88, 393–407.
• McNeil, A. J., Frey, R., & Embrechts, P. (2015). Quantitative Risk Management: Concepts, Techniques and Tools (revised ed.). Princeton University Press.
• Rockafellar, R. T., & Uryasev, S. (2000). Optimization of conditional value-at-risk. Journal of Risk, 2(3), 21–41.

---

Frequently Asked Questions

Is CVaR the same as Expected Shortfall?

Yes, in the continuous-distribution case. The names Expected Shortfall, Conditional Value at Risk, Tail Value at Risk, and Average Value at Risk all refer to the same object: the expected loss conditional on exceeding VaR. Small discrepancies can arise for discrete distributions (where the exact quantile lies between two observed values), and different textbooks handle this edge case differently. In regulatory documents, the term Expected Shortfall is now standard.

Why does Basel use 97.5% instead of 99%?

Two reasons. First, the 97.5% ES of a normal distribution is approximately equal to the 99% VaR of that distribution. Using 97.5% ES therefore does not radically shift the numerical size of reported risk for banks with normal-like distributions. Second, the extra tail observations at 97.5% (versus 99%) make backtesting and calibration statistically more reliable. The trade-off is that 97.5% is less extreme than 99%, which some argue makes it less protective for portfolios with genuinely heavy tails.

How does ES compare in size to VaR?

For a normal distribution, ES at level (\alpha) is approximately (\phi(z_\alpha) / (1 - \alpha)) standard deviations above the mean. At (\alpha = 0.99), this is roughly 2.67 versus 2.33 for VaR — about 15% larger. For heavy-tailed distributions the gap is much bigger: a Student-t with 4 degrees of freedom has ES roughly 1.4x its VaR at the 99% level. The heavier the tail, the more ES diverges from VaR and the more useful the distinction becomes.

Is Expected Shortfall the same as tail conditional expectation?

Yes, for continuous distributions. For distributions with atoms — for example, when losses have a positive probability of exactly equalling VaR — the formal definition of ES uses the tail integral of the quantile function rather than the naive conditional expectation, and the two objects can differ by a small correction. The tail-integral definition is the one that yields coherence.

Can I use ES for portfolio optimisation?

Yes. Rockafellar and Uryasev (2000) showed that CVaR minimisation can be reformulated as a linear program by introducing auxiliary variables. Mean-CVaR optimisation delivers portfolios that are optimised against tail losses rather than variance, which is often more appropriate for portfolios with skew, jump risk, or credit exposure. It is now standard in institutional risk-parity and tail-risk-managed strategies.

Does ES capture liquidity risk?

Not directly. Standard ES treats returns over a fixed horizon and assumes the position can be marked at model or market prices at the end of that horizon. Liquidity risk — the risk that the position cannot be closed at the marked price within the horizon — is handled separately, either through liquidity-adjusted horizons (as in FRTB) or through explicit bid-offer haircuts added on top of the ES number.