modeling-regime-switching-strategies

# Modeling Regime Switching Strategies

## When To Use

- Detecting shifts between market regimes (e.g., risk-on/risk-off, high-vol/low-vol, trending/mean-reverting) to adapt portfolio positioning or signal weights
- Building Hidden Markov Models (HMMs), threshold autoregressive (TAR/STAR) models, or Bayesian change-point detectors for systematic strategy overlays
- Evaluating whether factor exposures, alpha signals, or risk premia behave differently across macro or volatility regimes
- Constructing regime-conditional allocation rules (e.g., shift from momentum to defensive when HMM flags a contraction state)
- Stress-testing strategy performance under historically identified regimes versus live regime estimates

## Inputs To Gather

- **Return series**: Asset or strategy returns at the target frequency (daily, weekly, monthly); confirm total return vs. excess return convention
- **State indicator candidates**: Realized volatility, credit spreads (IG/HY OAS), yield curve slope (2s10s, 3m10y), PMI, leading economic indicators, VIX term structure
- **Number of regimes (K)**: Start with K=2 (expansion/contraction or low-vol/high-vol); test K=3 if a third state (crisis/transition) is economically justified — avoid over-fitting with K>3 unless sample size supports it
- **Model choice**: HMM (Gaussian emission, possibly mixture), Markov-switching VAR, threshold regression (SETAR/LSTAR), or Bayesian online change-point detection (BOCPD)
- **Estimation window**: Full sample for calibration vs. expanding/rolling window for out-of-sample regime assignment
- **Strategy parameters per regime**: Signal weights, leverage caps, hedging ratios, or factor tilts that should shift conditional on detected regime

## Workflow

1. **Select and preprocess state variables**
   - Choose 1–3 observable indicators with economic rationale for regime differentiation (e.g., realized vol + credit spread + yield curve slope)
   - Standardize or z-score inputs; test stationarity (ADF/KPSS) — difference or detrend if needed
   - Align timestamps; handle missing data via last-observation-carried-forward or interpolation with documentation

2. **Specify the regime model**
   - **HMM (Gaussian)**: Define K states; initialize transition matrix A, emission means μ_k, and variances σ²_k; estimate via Baum-Welch (EM algorithm); extract smoothed state probabilities P(S_t = k | Y_{1:T})
   - **Threshold / STAR**: Choose threshold variable and delay parameter; estimate regime-specific coefficients via conditional least squares or grid search over threshold values; test linearity (Hansen test) [VERIFY: confirm asymptotic distribution assumptions for your sample size]
   - **Bayesian change-point (BOCPD)**: Specify hazard function (constant rate λ or time-varying); define predictive model per run-length (e.g., Gaussian with conjugate Normal-Inverse-Gamma prior); compute run-length posterior online
   - For any model, record log-likelihood, BIC/AIC for model comparison across K values

3. **Estimate and diagnose**
   - Run estimation on the calibration sample; check convergence (multiple random restarts for EM to avoid local optima — minimum 20 initializations)
   - Inspect estimated regime parameters: do means and volatilities across states have economic interpretation? (e.g., State 1: μ=+8% ann., σ=12%; State 2: μ=−4% ann., σ=28%)
   - Plot smoothed regime probabilities overlaid on the return series and known macro events (recessions, crises) as a sanity check
   - Check transition matrix persistence: diagonal elements of A should typically be >0.90 for regimes to be tradeable; very transient states may signal noise

4. **Build regime-conditional strategy rules**
   - Map each regime to a strategy configuration: e.g., State 1 (low-vol expansion) → full risk budget, momentum + carry tilts; State 2 (high-vol contraction) → reduced leverage, defensive quality + low-vol tilts
   - Define transition rules: act on smoothed probability threshold (e.g., switch when P(S_t = crisis) > 0.70) vs. Viterbi most-likely-state path
   - Specify implementation lag: regime signal at close of day t → trade at close of t+1 (minimum); account for estimation delay in rolling/expanding window setups
   - Set regime-switch cost assumptions: turnover from rebalancing on state transitions, slippage, and any constraints on switch frequency (e.g., minimum holding period per regime to avoid whipsaw)

5. **Validate out-of-sample**
   - Split sample: calibrate on first 60–70% of history, test regime detection and strategy performance on holdout
   - Compare regime-switching strategy vs. static benchmark (e.g., fixed-weight portfolio, buy-and-hold) on: Sharpe ratio, max drawdown, Calmar ratio, turnover, and regime-detection accuracy (% of months correctly classified relative to NBER dates or a rolling-vol proxy)
   - Run sensitivity on key parameters: number of states K, probability threshold for switching, estimation window length, and indicator choice
   - Perform bootstrap or block-bootstrap of returns within regimes to assess statistical significance of performance differential

6. **Document and deliver**
   - Produce a model specification sheet: state variables, K, estimation method, transition matrix, emission parameters, calibration window
   - Include regime timeline chart (color-coded by state) with annotated macro events
   - Provide strategy performance attribution by regime: contribution of each state to total return, risk, and drawdown
   - List all assumptions, limitations (lookahead bias mitigation, regime-detection lag, in-sample overfitting risk), and [VERIFY] items

## Output

- **Regime model specification**: Transition matrix, emission parameters (means, covariances per state), log-likelihood, BIC
- **Regime probability time series**: Smoothed posterior probabilities for each state at each observation, exportable as CSV or DataFrame
- **Regime timeline visualization**: Chart with shaded regime periods overlaid on cumulative strategy returns
- **Strategy parameter table**: Signal weights, leverage, hedging ratios per regime with transition rules
- **Backtest summary**: Performance metrics (Sharpe, max DD, Calmar, turnover) for regime-switching strategy vs. static benchmark, split by in-sample and out-of-sample periods
- **Sensitivity analysis**: Impact of K, threshold, window length, and indicator selection on detection accuracy and strategy metrics

## Quality Checks

- Regime parameters are economically interpretable — not just statistically distinct but meaningful (e.g., crisis state should correspond to known drawdown periods)
- No lookahead bias: regime classification at time t uses only information available at t (expanding window, not full-sample smoothing applied retroactively for trading decisions)
- Multiple EM restarts confirm convergence to global optimum (log-likelihood stable across top initializations)
- Transition matrix diagonal elements checked for persistence; off-diagonal transitions make economic sense (e.g., direct jump from low-vol to crisis should be rare)
- Out-of-sample regime detection accuracy is meaningfully above random assignment (>60% for 2-state, >45% for 3-state) [VERIFY: thresholds may vary by asset class and frequency]
- Strategy turnover from regime switching is realistic given transaction costs — net-of-cost Sharpe improvement is positive
- All [VERIFY] markers resolved or flagged for human review before deployment