When a backtest is worth believing
There is no threshold that makes a backtest true. What there is, is a list of conditions that each remove one specific way of being fooled, and a result that satisfies most of them deserves cautious weight while a result that satisfies few deserves none.
The conditions, roughly in order of how often they are violated: the sample is large enough that the expectancy is not noise; the rules were fixed before the validation period was examined; the parameters sit on plateaus rather than spikes; realistic costs were applied and the result survives a sensitivity check; the reserved period was used once; the trade order was reshuffled and the drawdown across shuffles is one you could actually sit through; and the result does not depend on a handful of exceptional trades.
Notice that none of these is about the size of the return. A modest, robust, cheap-to-execute edge that survives all seven is worth far more than a spectacular one that survives three, and the spectacular one is more likely to be a beautifully fitted description of a period that has already ended.
The last condition is the one no test can supply: you have to be able to execute it. A method that requires eleven decisions a day, or entries at four in the morning, or sitting through a fourteen-trade losing run, may be perfectly sound and still be untradeable by you. That is not a flaw in the backtest, it is a fact about the pairing of a method and a person, and the honest response is to find that out before funding it rather than after.
And then the final step, which is where this curriculum meets the rest of the app: trade it small, live, and compare the live distribution to the tested one. Not the P&L — the mechanics. Signal frequency, average move, win rate and average R. If those match, the backtest described something real. If they do not, it described your history, and the difference will show up there long before it shows up in the money.
What to take away
- Seven conditions, each removing one specific way of being fooled — none about the size of the return.
- Executability is a real condition and no test can supply it.
- Validate live on mechanics — signal count, average move, R — not on P&L.
Where it goes wrong
- Preferring the spectacular result to the robust one.
- Funding a method you have never sat through a losing run of.
- Comparing live to backtest on P&L, where noise dominates for months.
This chapter, measured against your own trades
In the app the same chapter ends in your figures rather than an example: how often you did the thing it describes, over your last ninety days. You pick one change to make, and Evidence checks afterwards whether it actually changed — from your journal, arithmetic, no opinion involved. Questions you get wrong come back a week later and again a month after that.
Open the free plan →Check that it stuck
Answers shown — in the app these are asked before you see them, and the ones you get wrong come back after a week.