Why most crypto backtests lie

Five ways a backtest lies

None of these needs bad intent. They are the default behaviour of a simple backtester.

• Look-ahead. The strategy trades on information it could not have had: acting on a bar's close at that bar's open, or using an indicator that quietly reads future values. Results look excellent, and no real account can reproduce them.
• Missing costs. Fees on every entry and exit, funding on every position held across a funding time, and slippage when the price moves while your order fills. A strategy that trades often can be profitable before costs and a loser after them.
• Survivorship. Testing only on the coins that are liquid today leaves out the ones that faded. The universe looks healthier than the one you would have faced.
• Ignored liquidation. A leveraged position can be closed by the exchange in the middle of a bar. If the test only looks at closing prices, it never sees the wick that ended the trade.
• One lucky period. A strategy that only ever met one kind of market looks robust until the market changes.

An example with round numbers

Take a hypothetical 20× long entered at 100. With a typical 0.5% maintenance margin it is liquidated near 95.5. Now imagine an hourly bar that dips to 94 and closes at 99. A backtest that only reads closes sees a 1% loss and a position still open. The exchange would have liquidated it, and the loss would have been the whole margin. The numbers are made up to show the mechanism; the mechanism is real on any leveraged perpetual.

Questions to ask any backtest

Before you trust a result from any tool, ask:

• Are fees, funding and slippage charged on every trade, and can I see the numbers used?
• What is the data granularity? Daily or hourly bars hide wicks that 1-minute bars show.
• Is liquidation modelled, for the margin mode I would actually use?
• Is there a period the strategy was never tuned on?
• How many trades are behind the headline number, and does the tool say when that is too few?
• Does it show results by market condition, or only the average?

How the Lab answers each one

• Costs: every trade is priced after the execution profile's fees, slippage and funding.
• Wicks: runs replay 1-minute bars, and exact-exit runs use the trade tape for stop and target fills.
• Liquidation: modelled for isolated and cross margin, with a guard that refuses a stop placed beyond the liquidation price.
• Held-out data: optimizations validate on a holdout and walk forward through time.
• An honest verdict: results are grouped by market regime, the verdict can say “No clear edge” or “Too few trades to tell”, and an evidence meter reads Not enough, Weak, Moderate or Strong. A strategy does not get a green light for paper gains.

Common questions