Overfit
Indicator families and pairs: best was good, not distinguishable from luck
The idea
Instead of searching millions of combinations, group the 24 indicators into families (momentum, trend, oscillators, risk and so on) and test each family and each pair of families: 28 books.
What happened
| Training years | Sharpe | Worst drop | Invested |
|---|---|---|---|
| Best book (momentum plus risk) | 0.62 | 30% | about 94% |
| SPY buy and hold | 0.41 | 55% | 100% |
Deflated Sharpe 0.943 (bar 0.95). Probability of backtest overfitting 0.47 (bar 0.20). Costs at zero and double barely moved the ranking.
The full results
All 28 books, default costs, 2005–2016 training window. Each book holds the ETFs that score above the median on its indicator family (or the average of two families), rebalances every 21 sessions, and keeps idle cash in T-bills; Sharpe, return and worst drop credit that cash. SPY buy and hold on the same window: Sharpe 0.41, 7.47% a year, 55.2% worst drop.
| Book | Sharpe | Yearly return | Worst drop | Time invested | Sharpe, zero costs | Sharpe, double spread |
|---|---|---|---|---|---|---|
| Pair: momentum + risk Best | 0.62 | 7.56% | 30.1% | 94% | 0.63 | 0.60 |
| Family: risk | 0.61 | 7.05% | 28.8% | 94% | 0.63 | 0.60 |
| Family: momentum | 0.58 | 8.35% | 29.7% | 95% | 0.59 | 0.57 |
| Pair: momentum + liquidity | 0.56 | 7.93% | 34.4% | 93% | 0.58 | 0.54 |
| Family: trend | 0.56 | 8.20% | 34.2% | 95% | 0.57 | 0.55 |
| Pair: trend + risk | 0.56 | 7.02% | 29.4% | 94% | 0.57 | 0.54 |
| Pair: risk + liquidity | 0.56 | 6.92% | 30.3% | 92% | 0.58 | 0.53 |
| Pair: momentum + trend | 0.55 | 8.00% | 31.0% | 95% | 0.56 | 0.54 |
All 28 books, best first
| Book | Sharpe | Yearly return | Worst drop | Time invested | Sharpe, zero costs | Sharpe, double spread |
|---|---|---|---|---|---|---|
| Pair: momentum + risk Best | 0.62 | 7.56% | 30.1% | 94% | 0.63 | 0.60 |
| Family: risk | 0.61 | 7.05% | 28.8% | 94% | 0.63 | 0.60 |
| Family: momentum | 0.58 | 8.35% | 29.7% | 95% | 0.59 | 0.57 |
| Pair: momentum + liquidity | 0.56 | 7.93% | 34.4% | 93% | 0.58 | 0.54 |
| Family: trend | 0.56 | 8.20% | 34.2% | 95% | 0.57 | 0.55 |
| Pair: trend + risk | 0.56 | 7.02% | 29.4% | 94% | 0.57 | 0.54 |
| Pair: risk + liquidity | 0.56 | 6.92% | 30.3% | 92% | 0.58 | 0.53 |
| Pair: momentum + trend | 0.55 | 8.00% | 31.0% | 95% | 0.56 | 0.54 |
| Pair: momentum + oscillator | 0.53 | 8.23% | 35.7% | 93% | 0.55 | 0.52 |
| Pair: momentum + legacy | 0.51 | 7.49% | 33.7% | 93% | 0.53 | 0.49 |
| Pair: reversal + momentum | 0.48 | 7.36% | 36.6% | 93% | 0.50 | 0.47 |
| Pair: reversal + trend | 0.47 | 7.44% | 38.1% | 92% | 0.49 | 0.46 |
| Pair: reversal + liquidity | 0.47 | 7.47% | 40.2% | 91% | 0.49 | 0.45 |
| Pair: trend + legacy | 0.46 | 6.83% | 37.0% | 92% | 0.48 | 0.44 |
| Family: reversal | 0.46 | 7.52% | 43.4% | 90% | 0.48 | 0.44 |
| Pair: liquidity + legacy | 0.45 | 6.85% | 44.8% | 91% | 0.47 | 0.44 |
| Family: liquidity | 0.45 | 6.85% | 37.3% | 90% | 0.47 | 0.43 |
| Pair: oscillator + risk | 0.44 | 6.31% | 36.3% | 91% | 0.46 | 0.42 |
| Pair: trend + liquidity | 0.43 | 6.42% | 38.9% | 93% | 0.45 | 0.42 |
| Pair: reversal + risk | 0.43 | 6.11% | 34.0% | 92% | 0.45 | 0.41 |
| Pair: risk + legacy | 0.43 | 5.79% | 37.1% | 92% | 0.45 | 0.41 |
| Family: legacy | 0.42 | 6.45% | 41.7% | 91% | 0.44 | 0.40 |
| Pair: trend + oscillator | 0.42 | 6.62% | 41.0% | 93% | 0.43 | 0.40 |
| Pair: oscillator + liquidity | 0.42 | 6.71% | 42.0% | 90% | 0.44 | 0.40 |
| Pair: reversal + oscillator | 0.41 | 6.69% | 45.2% | 90% | 0.42 | 0.39 |
| Pair: reversal + legacy | 0.39 | 6.24% | 45.4% | 91% | 0.41 | 0.37 |
| Family: oscillator | 0.38 | 6.32% | 45.2% | 90% | 0.39 | 0.36 |
| Pair: oscillator + legacy | 0.35 | 5.62% | 46.3% | 91% | 0.36 | 0.33 |
Why it failed
The gates are about the whole batch, not the leader: you can't tell the best of 28 apart from the luckiest of 28. The deflated Sharpe was 0.943 against a 0.95 bar, and the probability of overfitting was 0.47 against a 0.20 bar. No later confirmatory years were opened.
Receipts: the plan was written down and committed first. 28 official trials.