How Many Sessions Before a Backtest Means Anything

Count trades, not years. A test spanning half a decade of an opening range rule sounds substantial until you notice how few decisions it actually contains. A strategy taking one trade per session gathers roughly as many observations in a year as a fast intraday system gathers in a fortnight. The span is impressive. The sample is thin, and the sample is what decides how much the result is entitled to claim.

The Arithmetic of One Trade a Day

There are around two hundred and fifty sessions in a trading year, and a rule that trades every one of them collects that many observations. Almost no rule does. Add a filter for range width, a filter for direction, a requirement that the break happen before a certain time, and the count falls, sometimes by a third or more. Five years of history can reduce to a few hundred real trades once the filters have done their work.

That is a number small enough to be dominated by a handful of unusually good or unusually bad sessions. Remove the best few trades from a small sample and look at what remains. If the result changes character, the sample was carrying the conclusion on a few shoulders rather than on its own weight.

How Wide the Uncertainty Actually Is

Every measured average sits inside a band of plausible true values, and the band narrows slowly. It narrows with the square root of the sample, which means quadrupling the number of trades halves the uncertainty rather than quartering it. Going from a few hundred trades to a few thousand is years of additional history for a modest improvement in confidence.

The practical consequence is uncomfortable. At the sample sizes an opening range rule realistically produces, a genuinely profitable rule and a break even rule that got lucky can produce equity curves that look alike. Nothing in the curve distinguishes them. Only the width of the band does, and the band is rarely drawn.

Every Split Costs You Twice

The instinct when a result looks good is to check whether it holds up in subsets. Does it work in the first half and the second half? On up days and down days? On each instrument separately? Each of these questions is reasonable and each one divides an already small sample.

A test with a few hundred trades split four ways leaves subsets too small to say anything about. Worse, if you run enough splits, one of them will look poor and one will look excellent through chance alone, and the temptation is then to build a filter around the excellent one. That filter is fitted to noise, and it will be the first thing to fail forward.

Not All Sessions Are Independent

Sample size assumes observations are independent, and consecutive sessions are not. A volatile stretch produces several wide range days in a row, all of which behave similarly. A quiet month produces a run of compressed ranges that also behave similarly. The effective sample is smaller than the trade count suggests, because clusters of correlated sessions carry roughly the information of a single session each.

Testing across several instruments does not solve this either, if those instruments move together. Two index products on the same session are close to one observation, not two. A test that counts them as two is quietly overstating how much evidence it has.

Working With a Short Sample

None of this means a short test is worthless. It means the conclusions drawn from it should be coarse. A short sample can tell you that a rule is catastrophic, because catastrophic shows up quickly. It can tell you that the rule is executable and that the data supports the mechanics. What it cannot do is separate a small edge from no edge, and that is usually the exact question being asked of it.

The honest posture is to state the trade count alongside every result and to treat the count as part of the finding rather than a footnote. A rule with a modest average over a few hundred trades is a hypothesis. It becomes something firmer only as sessions accumulate, and there is no way to accelerate that.