The backtest is the go-to test for any automated strategy… and also the easiest one to manipulate. Understanding what a backtest is, and where the overfitting trap lies, is what separates people who know how to judge a robot from people who get carried away by a pretty equity curve.
What is a backtest in trading?
A backtest is a simulation: you take a strategy and apply it to historical market data to see how it would have performed in the past. In MetaTrader 5, for example, the Strategy Tester runs through years of price data and gives you a report with the return, the trades and the risk metrics.
It’s a valuable tool: it gives you a first idea of whether a strategy makes sense before you risk real money. But it has a limit you should never forget.
What a backtest is for, and where its limits are
A backtest helps you rule things out and get your bearings: a strategy that consistently lost money in the past is unlikely to shine in the future. But the reverse isn’t true: an excellent backtest does not guarantee future results. The past is only a sample of what the market can do, and the future always brings conditions that weren’t in that data.
Overfitting: the great backtesting illusion
This is the main danger. Overfitting (or over-optimization) happens when a strategy is tuned so tightly to one specific set of historical data that it stops capturing real market behavior and starts “memorizing” the past. The result is an almost perfect equity curve… on that data, and a failure as soon as the market changes a little.
It’s like a student who memorizes the answers to one specific exam instead of learning the subject: they ace that exam and fail the next one. That’s why, when someone only shows you a flawless backtest with a curve that climbs in a straight line, you should be more suspicious, not less.
Perfect equity curves with barely any dips, plus huge returns in the backtest, are the classic overfitting combination. An honest strategy also shows its bad streaks and its real drawdown.
How to check whether a trading strategy is robust
To tell a solid strategy from a dressed-up one, you use robustness tests. The most important ones:
- Out-of-sample testing: you set aside a stretch of data the strategy never saw while it was being built, and check whether it still works there. If it only works on the data it was built on, that’s a bad sign.
- Walk-forward: you optimize on one period and test on the next, repeating the process over time, which mimics how the strategy would be used in real life.
- Monte Carlo analysis: you slightly alter the conditions (trade order, spreads, etc.) thousands of times to see whether the results hold up or fall apart when small things change.
A strategy that passes these tests isn’t infallible (nothing is), but it gives you far more confidence than one that can only boast a perfect backtest.
Frequently asked questions
Does a good backtest guarantee I’ll make money?
No. A backtest shows how a strategy would have behaved in the past, but past results don’t guarantee future ones. It helps you get your bearings and rule things out; it isn’t a promise of returns.
What exactly is overfitting?
It’s when a strategy is fitted too closely to the specific historical data it was built on, to the point of “memorizing” it. It looks perfect in the past and usually fails in the live market.
How do I know if a strategy is over-optimized?
Be wary of curves that look too perfect and of exaggerated returns. Look for a strategy that shows its real drawdown and has passed tests such as out-of-sample, walk-forward and Monte Carlo.
How we validate AUREUM
We show the full metrics, including the real drawdown, and we put the strategy through out-of-sample, walk-forward and Monte Carlo tests. No magic curves: just the numbers, unfiltered.
See AUREUM and its numbers →