FinCAD: Mitigating Look-Ahead Bias in LLM Financial Backtesting
A new paper on arXiv identifies a critical flaw in using large language models for financial backtesting: parametric look-ahead bias. Because LLMs are trained on historical data up to a cutoff date, they effectively 'know' past market outcomes, making backtests on pre-cutoff periods unreliable. To address this, the authors propose FinCAD, an inference-time adaptation of Context-Aware Decoding. FinCAD uses an adversarial bias-discovery pipeline to learn a model-specific memory-activating prior prompt, combined with an entity- and date-adaptive rule that scales CAD strength based on per-(entity, date) memorization. This penalizes memorized in-sample dates while decaying to zero out-of-sample. Tested on five 7-14B parameter LLMs and five mega-cap equities, FinCAD reduces in-sample backtest returns by up to -67.1% on memorized dates, while keeping 2025 out-of-sample returns within $8K and Sharpe ratio within 0.10 of baseline, preserving general-purpose capabilities.
Key facts
- Parametric look-ahead bias makes LLM backtesting on historical financial data unreliable.
- FinCAD is an inference-time adaptation of Context-Aware Decoding.
- FinCAD pairs an adversarial bias-discovery pipeline with an entity- and date-adaptive rule.
- Tested on five 7-14B parameter LLMs and five mega-cap equities.
- FinCAD cuts in-sample backtest returns by up to -67.1% on memorized dates.
- Out-of-sample 2025 returns are within $8K of baseline.
- Out-of-sample Sharpe ratio is within 0.10 of baseline.
- General-purpose capabilities are preserved.
Entities
Institutions
- arXiv