
Intro: Can Pairs Trading Backtesting With Python Survive Costs?
Pairs trading often looks attractive because it appears market-neutral. Instead of betting that the entire market will rise, a trader buys one security and shorts another related security. If the relationship between them normalizes, the spread should produce a return.
That logic is elegant. It is also easy to overestimate.
A simple pairs trading backtesting with Python project can show strong returns before commissions, borrow fees, slippage, financing, taxes, and corporate actions are included. Once those frictions enter the model, a strategy that looked like statistical arbitrage may become a collection of low-margin trades with little economic value.
This stress test examines whether a Python-based mean reversion strategy can survive realistic assumptions. The focus is not on producing the most attractive equity curve. It is on identifying which assumptions matter, where the edge disappears, and how to build a backtest that more closely resembles live execution.
The Real Question Behind a Profitable Backtest
The important question is not, “Did this pair make money historically?”
It is:
Did the relationship generate enough gross profit to compensate for every cost required to trade it?
Consider a strategy that earns an average gross return of 0.40% per trade. That may sound promising. But if opening and closing both legs costs 0.15%, short selling costs another 0.10%, and slippage adds 0.10%, most of the apparent edge has already disappeared.
This is similar to testing a delivery business using revenue while ignoring fuel, wages, insurance, and vehicle maintenance. The business may look profitable on paper, but the cash left after operating expenses tells the real story.
What This Stress Test Measures
A practical test should compare gross and net performance across several dimensions:
- Total return and average return per trade
- Trade count and median holding period
- Win rate and payoff distribution
- Sharpe ratio and maximum drawdown
- Short selling costs and financing
- Bid-ask spreads, slippage, and market impact
- Performance across different market regimes
- Results using correlation versus cointegration
The goal is not to prove that pairs trading always works or never works. The goal is to determine when the strategy has enough margin of safety to remain viable.
Background: What Is Pairs Trading Backtesting With Python?
Pairs trading is a form of Python statistical arbitrage in which two related assets are combined into a spread. The trader buys the relatively cheap asset and shorts the relatively expensive one, expecting the spread to move back toward its historical average.
A common example is testing two energy companies, two banks, or two companies exposed to the same commodity. Similar businesses may respond to common economic forces, but similarity alone does not guarantee a stable tradable relationship.
How Pairs Trading Uses Cointegration and Mean Reversion
A typical workflow is:
- Collect historical, dividend-adjusted prices.
- Estimate a hedge ratio between the assets.
- Construct the spread.
- Test whether the spread is stationary.
- Calculate a rolling mean and standard deviation.
- Enter when the spread moves far from its expected range.
- Exit when it reverts or reaches a stop-loss threshold.
The hedge ratio determines how much of one asset offsets the other. For a simple linear model:
python import statsmodels.api as sm
model = sm.OLS(y, sm.add_constant(x)).fit() hedge_ratio = model.params.iloc[1] spread = y – hedge_ratio * x
The spread is then standardized into a z-score:
python z_score = (spread – spread.rolling(lookback).mean()) / spread.rolling(lookback).std()
A strategy might enter when the absolute z-score exceeds 2 and exit when it returns near zero.
Correlation Versus Cointegration
Correlation measures whether two returns move together over a particular period. Cointegration asks a different question: does a combination of two price series remain stable over time?
Two stocks can have a correlation of 0.90 and still drift permanently apart. For example, both may fall during a market crash and rise during a recovery, producing high return correlation. However, different dividends, product strategies, balance sheets, or regulatory outcomes can cause their long-term price relationship to break.
Correlation is like observing two runners moving in the same direction during one race. Cointegration is closer to checking whether they are connected by an elastic band. The first may be temporary; the second implies a stronger long-term relationship.
Hedge Ratios, Spreads, and Half-Life
The hedge ratio is usually estimated with regression, although the choice of regression direction matters. Ordinary least squares is not perfectly symmetric, so researchers should test whether the relationship is stable under alternative specifications.
The Augmented Dickey-Fuller test is commonly applied to the residual spread. A sufficiently negative test statistic provides evidence that the spread may be stationary. However, statistical significance is not a guarantee of future profitability.
Half-life estimates how quickly the spread tends to revert. A pair with a 10-day half-life may suit a short-term strategy. A pair with a 250-day half-life may tie up capital for too long and face more opportunities for structural change.
Python Statistical Arbitrage Tools for Beginners
Python is useful because the entire research pipeline can be inspected and repeated. pandas handles time series, numpy supports numerical operations, yfinance can retrieve historical market data, and statsmodels provides regression and statistical testing tools.
yfinance for Dividend-Adjusted Price Data
Dividend adjustment is essential when comparing equities. A raw closing price may show only capital appreciation, while an adjusted series attempts to reflect distributions and stock splits.
A basic download might look like this:
python import yfinance as yf
prices = yf.download( [\”SHEL\”, \”BP\”], start=\”2010-01-01\”, end=\”2026-01-01\”, auto_adjust=True )[\”Close\”]
This is useful for research, but it is not a complete institutional data solution. Historical data can contain missing observations, ticker changes, stale prices, and differences in exchange calendars. A production process should validate prices against another source where possible.
statsmodels for Regression and ADF Testing
statsmodels supports ordinary least squares regression and the ADF test:
python from statsmodels.tsa.stattools import adfuller
adf_stat, p_value, *_ = adfuller(spread.dropna())
The test result should be interpreted alongside economic reasoning, rolling stability, and out-of-sample behavior. A low p-value found after testing hundreds of pairs can easily be a product of selection bias.
Trend: Why Backtests Look Better Before Trading Costs
Trading costs are often the difference between a research result and a deployable strategy. In a mean reversion backtest, many trades are small by design. That makes each basis point important.
5 Costs That Can Change a Pairs Trading Result
Backtesting transaction costs
Transaction costs should be charged on both legs and on both entry and exit. A model that charges 10 basis points only once per trade is understating the cost of a four-sided transaction: buy one asset, short the other, close the long, and cover the short.
The assumption should also vary by liquidity. Large-cap stocks may have narrow spreads, while smaller securities can be expensive to trade precisely when the spread signal is strongest.
Short selling costs and stock borrow fees
Short selling costs are frequently omitted. A short position may incur a borrow fee that changes over time. Hard-to-borrow securities can become extremely expensive or unavailable.
This matters because a pair can remain open for weeks. A daily borrow rate that looks negligible can materially reduce returns over a long holding period. The backtest should apply borrow charges to the market value of the short position for each day held.
Bid-ask spreads and market impact
Backtests usually use closing prices, but traders rarely receive the midpoint consistently. A more realistic model buys near the ask and sells near the bid.
Market impact is another issue. If an order is large relative to normal trading volume, the execution price can move against the strategy. The impact is often greatest during volatile divergence events, exactly when pairs strategies try to enter.
Financing, slippage, and taxes
A market-neutral trade still uses capital. Long positions may require financing, while short proceeds may not earn the same rate as the trader’s funding cost. Margin rules can also change.
Slippage should be modeled as a function of volatility, spread width, order size, and liquidity rather than as one fixed number. Taxes and transaction levies can be especially important in some jurisdictions. Even a profitable high-turnover strategy may be unattractive after taxation.
Dividends and corporate actions
Dividends affect both legs. If the trader is short a dividend-paying stock, the short seller generally owes a manufactured dividend payment. Stock splits, special dividends, rights issues, and spin-offs can also change the relationship.
Ignoring these events can create artificial spread movements or false profits.
How Market Regimes Break a Trading Pair
Company-specific shocks and divergence
A disaster, legal dispute, failed acquisition, or accounting issue can permanently change one company’s valuation. The spread may not revert because the original economic relationship no longer exists.
A historical example is an energy pair disrupted by a major operational disaster. Both companies may belong to the same sector, but one can suffer a company-specific decline that the other does not share.
Dividend policy and sector changes
Dividend cuts can alter total-return relationships even when daily prices remain correlated. Sector composition also changes over time. A company may shift toward renewable energy, reduce oil exposure, sell a division, or change its balance-sheet policy.
A pair that worked in one five-year window may therefore be unsuitable in the next.
Insight: A Hands-On Mean Reversion Backtest Stress Test
How to build a realistic mean reversion backtest
Use only information available at each trade
Avoid look-ahead bias. The hedge ratio, mean, standard deviation, and stationarity test must be calculated using data available before the signal. If the model uses the full sample to estimate a relationship, its results are contaminated.
A rolling 252-trading-day window is a reasonable starting point, but it should not be treated as universally correct.
Set rolling windows, entry rules, and exit rules
A test could use:
- A 252-day estimation window
- Entry at a z-score of plus or minus 2
- Exit near zero
- A stop at a z-score of 4
- A maximum holding period of 60 trading days
- No overlapping trades unless explicitly modeled
The maximum holding period is important. Without one, a backtest can hold a broken pair indefinitely and disguise the risk as temporary noise.
Model position sizing and holding period
Position sizing should account for volatility, hedge ratio, liquidity, and portfolio concentration. Equal dollar positions are simple but may not create equal risk.
The strategy should also record holding time. A signal that earns 0.40% after 39 days may be less attractive than it first appears, particularly when annualized financing and opportunity costs are included.
Before-and-after cost comparison
An illustrative test of a Shell-BP-style pair over a long historical period produced a useful warning. Across roughly 49 trades, the median holding period was about 39 trading days. The average trade earned approximately 0.42% before costs but only about 0.17% after assumed trading expenses.
That difference is the central lesson: the strategy did not lose all of its gross edge, but the net edge became modest. A small change in borrow fees, slippage, or taxes could have reversed the result.
Gross returns versus net returns
Gross returns answer whether the signal identifies useful spread movements. Net returns answer whether the movement can be monetized.
Both should be reported separately. If net performance is only slightly positive under optimistic costs, the strategy should be considered fragile.
Trade count, win rate, and median holding time
A high win rate can be misleading if occasional losses are large. The report should include:
- Average and median trade return
- Largest winning and losing trade
- Win rate
- Profit factor
- Trade frequency
- Median and maximum holding period
For example, 49 trades over 16 years may not provide enough observations to establish a reliable edge, even if the final return is positive.
Sharpe ratio, drawdown, and capital efficiency
Sharpe ratio should be calculated from a realistic daily or weekly equity curve, not merely from average trade returns. Maximum drawdown reveals whether the trader could tolerate the strategy during a regime break.
Capital efficiency matters because a market-neutral strategy may require margin and reserve capital. A return that looks attractive on committed capital may be less compelling when calculated on total capital needed to support the positions.
Correlation or cointegration: which pairing method works better?
Correlation is useful as a screening tool, but cointegration is generally more aligned with spread-based mean reversion. A practical process might first filter for economically related securities, then test rolling cointegration and half-life.
However, cointegration should not be treated as a permanent label. It must be reestimated as prices, dividends, business models, and market regimes change.
When a highly correlated pair still fails
Two companies can move together during a broad sell-off and recovery while producing very different long-term returns. This creates high correlation but poor convergence.
The same problem appears when one company cuts its dividend and the other maintains it. Their daily returns may remain similar, but total-return investors are no longer holding equivalent economic exposures.
Forecast: Can Python Statistical Arbitrage Survive Real-World Friction?
What the stress-test results suggest
The evidence suggests that Python statistical arbitrage can survive real-world friction, but only when the gross signal is strong enough and the portfolio is diversified. Small edges are especially vulnerable.
When small edges disappear after costs
If average gross profit per trade is close to total estimated friction, the strategy has little margin for model error. Real execution is rarely better than the backtest. It is usually worse because liquidity deteriorates when volatility rises.
Researchers should run optimistic, base-case, and pessimistic cost scenarios. A strategy that remains profitable only under the optimistic case should not be considered robust.
Why diversification across pairs matters
Individual pair relationships can fail because of earnings surprises, lawsuits, takeovers, dividend changes, or sector transformations. Holding several independent pairs reduces dependence on one spread.
Diversification does not eliminate systemic risk. During a market shock, correlations can rise and liquidity can disappear across many trades. Still, a portfolio of carefully selected pairs is generally more resilient than a single celebrated relationship.
How to improve future pairs trading backtests
Test multiple rolling windows and market regimes
Compare 126-day, 252-day, and longer estimation windows. Separate calm markets, crises, inflationary periods, and high-rate environments. A signal that works only in one period may be regime-dependent.
Reestimate borrow, financing, and slippage assumptions
Use realistic borrow data where available. Vary financing rates and model wider spreads during volatile sessions. Include the possibility that a short position cannot be opened or maintained.
Validate results with out-of-sample data
Split the sample into training, validation, and testing periods. Better still, use walk-forward analysis: estimate the relationship on one window, trade the next period, then roll forward.
This resembles testing a navigation system on roads it has not already memorized. A backtest that succeeds only on familiar historical data is not evidence of generalization.
Call to Action: Build a More Realistic Pairs Trading Backtest
Audit your data, signals, and execution assumptions
Check adjusted prices, trading calendars, missing observations, corporate actions, and ticker changes. Confirm that every feature used by the strategy was available at the time of the trade.
Inspect individual trades rather than relying only on the equity curve. A few unusually favorable fills can distort the result.
Add backtesting transaction costs before trusting returns
Transaction costs should be part of the strategy design, not an afterthought. Model both legs, entry and exit, spread crossing, slippage, and financing.
Then test sensitivity. If adding a few basis points turns a profitable strategy negative, the edge is not yet dependable.
Test short selling costs and alternative instruments
Compare direct equity positions with instruments that may reduce tax or operational friction, such as futures, swaps, or options where appropriate. Each alternative introduces its own risks, including basis risk, expiration, liquidity, and counterparty exposure.
Conclusion: The Real Verdict on Pairs Trading Backtesting With Python
A backtest is only as strong as its cost model
Pairs trading backtesting with Python can reveal useful statistical relationships, but gross returns are not tradeable returns. Borrow fees, financing, spreads, taxes, slippage, and corporate actions can reduce a promising result to a marginal one.
Use mean reversion evidence, not correlation alone
Correlation is a starting filter. Cointegration, hedge-ratio stability, half-life, economic logic, and rolling out-of-sample performance provide stronger evidence that a spread may revert.
Treat Python as a testing tool, not a profit guarantee
Python makes it easier to test assumptions, compare regimes, and expose weaknesses. It does not remove structural breaks or execution risk.
The practical verdict is cautious: Python statistical arbitrage can survive real-world costs when it targets sufficiently liquid pairs, maintains a meaningful gross edge, controls position size, and diversifies across relationships. A backtest that survives pessimistic cost assumptions is far more valuable than one that produces an impressive return before the bill arrives.