Why Backtests Lie: Simulating Indicator-Based Exits Is Harder Than It Looks

A backtest produces a number, and numbers are persuasive. That is the whole problem. When a simulation is subtly wrong, it does not crash and it does not warn you. It hands you a slightly better result than reality and lets you act on it.

Simulating a fixed take-profit is easy: the price either reached your level or it did not. Simulating an indicator-based entry or exit is where it gets hard, because now the simulation has to answer a much trickier question: what did the bot actually know at that moment?

Here are three ways we got that question wrong, and what each one taught us.

Key Takeaways

The classic bug is reading a higher-timeframe indicator before its bar has closed. A 4-hour RSI consulted at bar open carries four hours of price action that had not happened yet.
The fix is arithmetic, not heuristics: a value from timeframe T is only legitimately usable once T has elapsed, so shift the lookup back by (trigger timeframe minus simulation timeframe).
Live engines have the same problem in a different costume. An indicator computed on a still-forming candle repaints, meaning a condition can be true and then untrue on the same bar.
A simulator that silently skips what it cannot replay is more permissive than the live bot, and the gap always favours the backtest.
The honest move is to surface the gap in the result rather than hide it. A backtest that tells you where it is lying is worth more than one that looks clean.

1. The look-ahead bug that hides in multi-timeframe strategies

This is the one that is easiest to write and hardest to notice.

Say the simulation steps forward one 5-minute candle at a time, and the strategy has a condition on a 4-hour RSI. At each 5-minute step the engine needs a value for that RSI. The naive implementation looks up the 4-hour bar containing the current moment and reads its value.

That is a time machine.

A 4-hour bar that opened at 12:00 does not have a final RSI value until it closes at 16:00. If the simulation is standing at 12:05 and reads that bar, it is reading a number computed from four hours of price action that, from 12:05's point of view, has not happened yet. The strategy appears to enter right before moves it could not have seen coming, and the equity curve looks excellent.

The fix is not a heuristic. It falls out of writing down when a value is actually knowable. A bar that opens at t on a timeframe of triggerMs is only knowable at t + triggerMs. The simulation makes its decision at the close of the current candle, candleOpenTime + simulationMs. So the value is legitimately usable exactly when:

t + triggerMs <= candleOpenTime + simulationMs
t             <= candleOpenTime - (triggerMs - simulationMs)

Which means the correct lookup is the one you already had, run against a timestamp shifted back by triggerMs - simulationMs.

Two properties of that expression are worth pointing out, because they are what make it safe to apply everywhere:

Same-timeframe conditions get a shift of zero. A 5-minute indicator read at the close of a 5-minute candle is its own close-based value, which was always correct. Only genuinely higher timeframes move.
The shift is derived, not configured. It comes from the candle itself rather than from a setting someone has to remember to set.

The general lesson is one I would apply to any simulation, not just trading: when you are unsure whether you have look-ahead bias, write down the timestamp at which each input becomes knowable and compare it to the timestamp at which you use it. Bias is what remains when you skip that step and rely on intuition instead, and intuition is bad at this because reading "the current 4-hour bar" sounds completely reasonable.

2. The same bug in live code wears a different costume

You would think this problem belongs to backtesting. It does not. The live engine faces it too, and there it is called repainting.

At any moment, the most recent candle is still forming. An indicator computed over a window ending on that candle will change as the candle changes. A condition can be true at 12:01, false at 12:03, and true again at 12:04, all within the same bar, without the market doing anything unusual. If the bot acts on the first of those, it acted on a value that no longer exists.

Our default is to drop the still-forming candle and compute on confirmed bars only, so a signal never repaints. That is also what makes live and backtest agree, since the simulation is by construction working with closed bars.

But "never repaint" is not free. Waiting for confirmation means acting later, and for some strategies that lateness costs more than the noise it avoids. So this is exposed as a per-condition choice: evaluate on the confirmed bar, or evaluate live and accept that the value can move under you.

The design point is the part I would keep: when a trade-off is genuinely a trade-off, do not bury it in a default and pretend it is a fact. Make the safe option the default, then let the user opt into the other one knowingly. What you must not do is let the two modes disagree between live and backtest, because then the user is comparing two different bots.

3. What the simulator cannot replay is more dangerous than what it gets wrong

The first two are correctness bugs. This one is a design problem, and it is the one I think is most underappreciated.

Some conditions cannot be replayed. An external webhook signal is the clearest case: there is no historical record of what an outside system would have sent at 14:20 on some day last March. The simulation simply cannot know.

So what should it do? The tempting answer is to skip that condition and evaluate the rest. That is what makes the result look complete. It is also where the damage happens, and the mechanism is worth spelling out:

If a strategy requires A AND B, and B cannot be simulated, then skipping B leaves A deciding alone. The simulated bot enters on conditions where the live bot would have stayed out. The simulation is systematically more permissive than the real thing.

Notice the direction. The error is not random. Dropping a condition under AND logic can only ever produce more trades than reality, never fewer, so the bias points at a better-looking result every single time. That asymmetry is what makes it dangerous: random noise averages out over many runs, and one-directional bias does not.

We chose to keep running the simulation rather than refuse, because a partial result with a known gap is still useful. But then the gap has to be visible in the output, not buried in a log. The result carries an explicit warning, in the user's own language, with a name that says what happened: "Some Entry Conditions Not Simulated." There is a matching one for exits.

That decision is the part of this system I would defend hardest. A simulator that cannot fail loudly will fail quietly, and quiet failure in something that produces numbers is the worst failure mode there is, because people act on numbers and do not act on stack traces.

What to take from this

If you are building anything that replays history to produce a number:

Every input needs a "knowable at" timestamp. If you cannot state it, you cannot rule out look-ahead bias.
Check the direction of your errors, not just their size. An error that always points the same way compounds; one that points randomly averages out. One-directional optimism is the signature of look-ahead bias.
Make the simulator and the live system share their decision logic wherever you can. Two implementations of "is this condition true" will drift, and you will find out from a user rather than from a test.
Report what you could not do. The credibility of a result comes from its stated limits, not from its cleanliness.

The uncomfortable summary is that a backtest is a claim about a counterfactual, and counterfactuals are easy to get quietly wrong. The engineering goal is not a simulator that never lies. It is one that tells you where it is lying.