Robustness Toolkit

A single equity curve tells you almost nothing. The Lab ships four tools to tell you whether a result is real — use them, that's the point of the Lab.

The four tools

1. Walk-forward — splits history into rolling (or anchored) train/test windows: sweep on train, pick the best combo by your selection metric, run it out-of-sample on test, then stitch the OOS segments into one curve. Mean degradation (OOS versus IS) is the honest number — a strategy that keeps ~70% of its in-sample performance out-of-sample is interesting; one that keeps 10% is fit to noise. The analytics also report WF efficiency and how many of the windows actually produced an OOS segment (OOS windows).
2. Holdout lockbox — reserve the last N% of your date range (Configure's Holdout field). The sweep never sees it. When you've finished iterating, hit Validate on holdout — once. If you iterate against the holdout, it stops being a holdout.
3. Monte Carlo — bootstraps your trade sequence into a fan of equity paths with a risk of ruin estimate. Your backtest is one draw from a distribution; the fan shows you the distribution (final p5/p50/p95, and max-drawdown p50/p95). Size positions off the p5 path, not the median.
4. Deflated Sharpe (DSR) — corrects the Sharpe ratio for how many combos you searched. A 2.0 Sharpe found by testing 5,000 combos is worth far less than one found by testing 10. If DSR is near zero, your leaderboard winner is likely a selection artifact.

The workflow that gets the most out of the Lab

1. Build the simplest version of the idea on one symbol. No parameters.
2. Add parameters and sweep small. Look at the heatmap for plateaus, not peaks.
3. Check DSR before falling in love with a leaderboard row.
4. Walk-forward anything that survives. Degradation is the metric that matters.
5. Monte Carlo the survivor to size it.
6. Validate on holdout exactly once, at the end.
7. Try it in portfolio mode to see how it behaves sharing capital.