Backtest overfitting: how to tell an edge from a lucky curve

A backtest that looks too good usually is. Not because anyone cheated, but because building a strategy on past data has a built-in pull toward the flattering answer: every rule you add and every parameter you adjust is chosen while looking at the result. This guide covers the ways a backtest flatters an idea, and the checks that catch each one.

What is backtest overfitting?

Backtest overfitting is what happens when a strategy’s rules are tuned until they fit the historical data so closely that they describe its noise rather than a behaviour of the market that repeats. An overfit strategy can look excellent on the data it was tuned on and fail on any other.

The question is not whether the best version looks good. It is how good the best of that many versions would look by chance.

Why does trying many versions inflate the result?

What you can do about it:

• Count your trials. Write down every version you run, not only the one you keep.
• Prefer rules with few moving parts. Every free parameter is one more way to fit the noise.
• Distrust a sharp peak. If a stop at 1.8 times the average range works and 1.7 or 1.9 do not, the 1.8 is fitted. A real effect usually survives a small change.

What is look-ahead bias?

Look-ahead bias is the use of information that did not exist yet at the moment a decision is simulated. It is the most damaging error a backtest can make, because it can turn any idea into a winner, and the hardest to see, because nothing in the result looks wrong.

Its common forms:

• Deciding on a candle that has not closed. A signal computed on a candle’s close cannot be traded at that same candle’s open.
• Levels confirmed after the fact. A swing high is only known once the candles after it have formed; drawn at its own candle, it lets the backtest act on it before anyone could have.
• Knowing the order of events inside a candle. A candle records its open, high, low and close, not which of them came first.

The last one deserves its own section, because every backtest built on candles meets it.

The candle that touches both the stop and the target

When a single candle’s range covers both the stop and the target, the candle alone cannot say which of the two price reached first. A backtest that assumes the target came first books a win that may have been a loss, and over many trades that assumption adds up to an edge that never existed.

There are two honest answers. The first is to look inside the candle: read the same stretch of time on finer candles, one minute at a time, and see which level was crossed first. The second, for the cases the finer candles still cannot settle, is to assume the worst and book the stop.

EdgeLuma does both. Every run reads its fills, stops and targets on the finest candles it holds inside each candle, one minute wherever it has them, and when a candle still touches both levels, the engine books the stop.

How many trades does a result need?

A result over a handful of trades is mostly noise. The expectancy, the average result of one trade, needs a sample before it means anything: over ten or twenty trades it is luck rather than an edge, and around a hundred trades it starts to say something. A long backtest that produces few trades is still a small sample, however many years it covers.

The same is true of every other number. A win rate measured on fifteen trades can change completely with the next five, and a maximum drawdown measured on a short history is the worst thing that happened, not the worst thing that can.

How to check that an edge is real

No single test proves an edge. Together, these make a lucky curve much harder to mistake for one:

1. Hold data back. Split the history: tune on the first part, then run the final rules once, unchanged, on the second. If the result collapses on data the rules never saw, it was fitted.
2. Test across conditions. A rising market, a falling one, a flat one. An edge that only exists in one of them is a bet on that condition.
3. Move the inputs a little. Shift each parameter a step either way. A real effect weakens gently; a fitted one falls off a cliff.
4. Count the costs. A thin edge before costs is often no edge after them, as our guide to fees and slippage shows.
5. Read the trades. If one or two trades make the whole result, the result is those trades.