--- title: "Overfitting in backtests · Trado Strategy Lab" description: "Tune a strategy long enough and the past looks perfect. Why overfitting feels like success, and the checks that expose it: holdouts, walk-forward, stability." canonical: "https://algo.trado.trade/learn/overfitting-in-backtests" lastModified: "2026-10-06" --- # Overfitting feels like success You adjust a parameter, the equity curve improves, you adjust another, and it improves again. By the end the strategy looks brilliant. It may simply have memorised the past. Overfitting is the most common way a good-looking backtest fails, and it feels exactly like progress. ## What overfitting is Markets contain a repeating signal and a lot of noise. A strategy with enough adjustable parts can be fitted to the noise of one stretch of history, which will never repeat. The fit looks like skill because the test period is the same one you tuned on. ## Why trying many things makes it worse Test enough random settings on the same data and some will look good by luck alone, the way the best result of a thousand coin-flip sequences looks like a streak. The more combinations you try, the better the best one looks, and the less it means. This is why a strategy that is the winner of a big parameter search deserves more suspicion, not less. ## Warning signs in your own results You can often tell before running any formal test. - The best settings sit at the edge of the range you searched, which suggests the real optimum is somewhere you did not look, or that there is none. - The strategy has many rules and parameters for the amount of history, so it has room to memorise. - A tiny change, such as one bar on a moving-average length, swings the result from great to poor. - It works on the coin you tuned on and loses on a similar coin it never saw. - The equity curve is suspiciously smooth for a market that is not. - You can explain the result only after seeing it, never before. ## How to check for it - **A holdout.** Keep a period the search never saw, and judge the winner only on that. - **Walk-forward testing.** Tune on one window, test on the next, slide forward and repeat, so every result comes from data the settings were not fitted to. - **Neighbourhood stability.** If nudging a parameter slightly destroys the result, you found a spike, not an edge. A broad plateau of good settings is a better sign. - **Parameter importance.** Which settings actually drive the result, and are those the ones you can explain? - **Several coins and periods.** An idea that only works on one coin in one year is a story. - **By market condition.** A strategy that earns everything in one regime and bleeds in the rest is exposed when that regime ends. - **Fewer knobs.** Every free parameter is another way to fit noise. ## What the Lab does about it The optimizer supports grid, random, genetic, TPE, CMA-ES and NSGA-II searches and validates with a holdout and walk-forward windows. Results come with heatmaps, parameter importance and a neighbourhood-stability view. For the whole search it reports Monte Carlo resampling, the probability of backtest overfitting and the deflated Sharpe ratio, which discounts a result for the number of things you tried. On any run, Insights say where the strategy works and where it does not, with regime tables, and the evidence meter says how much to read into it. The verdict will say “No clear edge” rather than flatter a lucky run. See [backtesting](https://algo.trado.trade/backtesting.md). ## Common questions ### Is optimizing a strategy a mistake? No. Searching is how you learn which settings matter. The mistake is treating the best in-sample result as the answer without validating it on data the search never touched. ### What is walk-forward testing? You tune on a window of history, test the tuned settings on the period right after it, then slide both windows forward. Every test result comes from data the settings were not fitted to. ## Next steps - [Join the waitlist](https://algo.trado.trade/waitlist) - [How backtesting works](https://algo.trado.trade/backtesting) ## Related - [Crypto backtesting after every cost](https://algo.trado.trade/backtesting.md): Backtest crypto perpetuals on years of 1-minute data for 900 coins, after fees, funding, slippage and liquidation, with a verdict that admits weak evidence. - [Why crypto backtests lie](https://algo.trado.trade/learn/why-backtests-lie.md): Look-ahead, missing costs, survivorship and ignored liquidation flatter backtests. A checklist to test any backtest, and how the Lab answers each point. - [Learn algo trading with AI](https://algo.trado.trade/learn/learn-algo-trading-with-ai.md): Forums and videos cannot explain your own strategy; an AI that reads your actual runs can. What to ask, a learning loop, and where AI needs checking.