Research degrees
Count human choices as well as coded parameters. Narrative filters and discretionary exclusions can overfit without appearing in a settings panel.
Editorially reviewed 24 August 2026
The search process is part of the model
MARGIN 06Overfitting is not limited to an optimizer with thousands of settings. Choosing a pair, timeframe, session, indicator length, screenshot example, exclusion, or exit after seeing outcomes spends degrees of freedom. A clean final rule can therefore carry a large hidden search history and an inflated win rate.
Create a research ledger that records every variant and why it was tried. Use development data for exploration, a validation block for selection, and a final holdout only once. If the holdout fails, preserve the failure; reopening it for tuning converts it into development data.
Count human choices as well as coded parameters. Narrative filters and discretionary exclusions can overfit without appearing in a settings panel.
Selection and evaluation require separate data. Walk-forward or nested validation is safer when repeated model choice is unavoidable.
A defensible rule should degrade gradually under nearby parameters, costs, and dates. A sharp isolated peak is a warning.
Out-of-sample performance judges the entire research process, not only the last formula it produced.
Imagine you are shown 100 photos of dogs and cats and asked to write rules for telling them apart. If you write broad rules — "dogs have longer snouts, cats have pointed ears" — you will do OK on new photos. But if you write rules so specific that they describe only those exact 100 photos — "photo #3 had a brown spot on the left ear, so brown spots mean dog" — you will fail on every new photo. That is overfitting: your rules are so specific to the training data that they do not generalize to anything else.
In forex terms: you backtest a strategy on January 2023 through December 2024. You add a 20 EMA, then an RSI filter, then a session-time filter, then you exclude "that weird week in March." Each adjustment makes the backtest look better — but each tweak is you writing a "brown spot" rule. The strategy now describes what already happened, not what will happen next.
You test EMA 20 — win rate is 48%. Try EMA 50 — 52%. EMA 100 — 55%. EMA 200 with RSI 14 — 62%. You keep adding and tweaking until the number turns green. Every time you test a new parameter and pick the best one, you are overfitting. You ran 10 variations and kept the winner — but by pure chance, one of those 10 will look better than the others even if none of them have a real edge.
A trader on r/Forex tested 6 different moving average periods on EURUSD M15, picked the one that gave 65% wins, and went live. The strategy lost for 3 weeks straight. He had not found the "right" MA period — he had run 6 coin flips and picked the lucky one. If he had tested on out-of-sample data, that same MA period would have shown 48%.
Your backtest shows a -8% drawdown in the second week of March. You look at the chart: "Oh, that was NFP week. My strategy does not trade news. I will exclude that week." Now the drawdown is -4%. Better. Then you exclude a random Monday gap. Now -2.5%. Each exclusion feels justified, but collectively you have removed every scenario where your strategy failed — leaving only the scenarios where it works. You have not improved the strategy. You have hidden its weaknesses from yourself.
You are bar-by-bar replaying EURUSD. You see a bullish engulfing candle forming. You think "this looks like an entry." Then you advance the next candle and it rips +30 pips. You count it as a win. But did you actually enter before seeing the result? If you are replaying with the full chart visible, your brain already knows where price went — and you will unconsciously filter out the setups that failed. Bar-by-bar replay where you cannot see the next candle is the only defense against this. Even then, you must write your entry decision before advancing.
You have 200 backtested trades. You notice that 8 of the 12 biggest losers happened between 8-10 AM London time. You add a filter: "No trades between 8-10 AM London." Now your win rate improves from 55% to 64%. But you just removed 12 data points from a sample of 200 — a 6% change — based on a pattern that could easily be random. With 200 trades, a cluster of 8 losing trades in a specific time window is statistically unremarkable. You filtered noise and called it a rule.
Your final strategy uses: 50 EMA, 200 EMA, RSI 14, ATR 14 stop, session filter (London only), day-of-week filter (no Mondays), and a minimum ADR filter of 60 pips. That is 7 parameters. Each one was tuned individually, and the combination was never tested as a system on unseen data. The probability that all 7 parameters are independently useful — rather than collectively memorizing a specific slice of historical data — is vanishingly small.
The only reliable test: split your historical data into two sets. Use the first 70% to develop your strategy. Do not look at the remaining 30% until your strategy rules are completely locked — written down, no further changes allowed. Then run your backtest on that 30%.
A more rigorous approach used by quant traders: split data into consecutive periods. Optimize your strategy on Period A (Jan-Jun 2023). Test on Period B (Jul-Dec 2023). If Period B passes, keep the rules exactly as-is and test on Period C (Jan-Jun 2024). Repeat. If performance degrades over successive walk-forward windows, you are overfitting. This is tedious — but it catches overfitting that a single out-of-sample split might miss.
FXAbsolute replays candles one at a time so you cannot see what comes next. Out-of-sample test on 5 years of real data. Your win rate will be honest — maybe lower than you want, but real.
▶ Start Free Backtesting →One reproducible testing idea, with its rules, limitations, and review questions made explicit. In your inbox every week.
Measured from 28 million candles
Beginner exploration
Open each answer for a plain-language way to read Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap, test it carefully and decide what to explore next.
This page focuses on “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap”.Find hidden overfitting in parameter searches, manual choices and exclusions using research ledgers, nested samples, stability tests and holdouts.For “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap”, a beginner should identify what the evidence guide measures, assumes or teaches before acting on its conclusion.Treat this page's account of “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap” as a learning reference rather than a prediction, signal or promise of future performance.
For “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap”, confirm the population, sample size, clock, units and calculation behind the headline number.While exploring “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap”, compare the median, spread of outcomes and exceptions instead of reading only the average or best row.Keep your “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap” record honest: use the statistic to frame a test or risk assumption, not to predict the next candle.Before leaving “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap”, recheck the result on a separate period before turning a descriptive pattern into a rule.
Turn one idea from “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap” into a rule with explicit inputs, dates, costs and pass-or-fail conditions.Ask AI to expose missing assumptions in that “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap” test, not to guess the next market move.Use the FXAbsolute AI Backtesting Lab to inspect calculations connected to “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap” and the assumptions behind them.Reproduce any important “Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap” result and reserve unseen data before deciding that an apparent pattern is useful.
Continue your exploration of Backtesting Shows 80% Win Rate, Live Is 50% — The Overfitting Trap with the beginner AI prompt guide, or inspect public calculations in the AI Backtesting Lab.