What overfitting is
Markets contain a repeating signal and a lot of noise. A strategy with enough adjustable parts can be fitted to the noise of one stretch of history, which will never repeat. The fit looks like skill because the test period is the same one you tuned on.
Why trying many things makes it worse
Test enough random settings on the same data and some will look good by luck alone, the way the best result of a thousand coin-flip sequences looks like a streak. The more combinations you try, the better the best one looks, and the less it means. This is why a strategy that is the winner of a big parameter search deserves more suspicion, not less.
Warning signs in your own results
You can often tell before running any formal test.
- The best settings sit at the edge of the range you searched, which suggests the real optimum is somewhere you did not look, or that there is none.
- The strategy has many rules and parameters for the amount of history, so it has room to memorise.
- A tiny change, such as one bar on a moving-average length, swings the result from great to poor.
- It works on the coin you tuned on and loses on a similar coin it never saw.
- The equity curve is suspiciously smooth for a market that is not.
- You can explain the result only after seeing it, never before.
How to check for it
- A holdout. Keep a period the search never saw, and judge the winner only on that.
- Walk-forward testing. Tune on one window, test on the next, slide forward and repeat, so every result comes from data the settings were not fitted to.
- Neighbourhood stability. If nudging a parameter slightly destroys the result, you found a spike, not an edge. A broad plateau of good settings is a better sign.
- Parameter importance. Which settings actually drive the result, and are those the ones you can explain?
- Several coins and periods. An idea that only works on one coin in one year is a story.
- By market condition. A strategy that earns everything in one regime and bleeds in the rest is exposed when that regime ends.
- Fewer knobs. Every free parameter is another way to fit noise.
What the Lab does about it
The optimizer supports grid, random, genetic, TPE, CMA-ES and NSGA-II searches and validates with a holdout and walk-forward windows. Results come with heatmaps, parameter importance and a neighbourhood-stability view. For the whole search it reports Monte Carlo resampling, the probability of backtest overfitting and the deflated Sharpe ratio, which discounts a result for the number of things you tried.
On any run, Insights say where the strategy works and where it does not, with regime tables, and the evidence meter says how much to read into it. The verdict will say “No clear edge” rather than flatter a lucky run. See backtesting.