Overfit
Wait for the stars to align: 24 setups, 0 passed
The idea
Instead of always being invested, wait for a "confluence" of two or three conditions, hold a fixed 1, 3 or 6 months, and sit in T-bills otherwise.
What we did
- 8 setups defined from indicators we already had (for example: a deep pullback in an ETF still in a long-term uptrend, entered once momentum turns back up; a volatility spike that calms; a breakout after a quiet stretch). 3 horizons (21, 63 and 126 sessions). 24 tests.
- Rule written first: at least 30 events on 15 unique dates; beat random entry dates in the same ETF; a bootstrap confidence interval entirely above zero; false-discovery rate q ≤ 0.10 across all 24.
What happened
0 of 24 passed. The closest:
| Setup | Horizon | Extra return vs random dates | Events | FDR q |
|---|---|---|---|---|
| Deep pullback in an uptrend, momentum turning up | 6 months | +5.9% | 55 (43 dates) | 0.16 |
| Calming volatility | 1 month | +1.3% | 484 | 0.16 |
| Same pullback setup | 3 months | +3.5% | 58 | 0.16 |
Setups that fired often were indistinguishable from random entries in the same ETF. One setup never had 30 events at any horizon.
We also report eight 63-day portfolio books for context. The best Sharpe (0.57, against SPY's 0.41) belonged to a setup that was 83% invested with about no event edge, which is just market exposure.
The full results
All 24 event tests, training years 2005–2016. "Extra return" is the average executable return of the setup's events (buy the next open, sell the hold period later) minus the average for random entry dates in the same ETF. "Interval floor" is the bottom of the one-sided 95% bootstrap interval for that extra return; it needed to be above zero. The false-discovery q-value had to be 0.10 or lower.
| Setup | Hold | Events (dates) | Extra return vs random dates | Interval floor | FDR q | Survived |
|---|---|---|---|---|---|---|
| Deep pullback, 200-day still rising, momentum turning up | 126 sessions | 55 (43) | +5.9% | 1.6% | 0.16 | No |
| VIX spike, then calming | 21 sessions | 484 (154) | +1.3% | 0.3% | 0.16 | No |
| Deep pullback, 200-day still rising, momentum turning up | 63 sessions | 58 (46) | +3.5% | 0.9% | 0.16 | No |
| Quiet-stretch squeeze plus 52-week breakout | 126 sessions | 69 (55) | +2.6% | 0.2% | 0.22 | No |
| Deep pullback, 200-day still rising, momentum turning up | 21 sessions | 81 (67) | +1.4% | −0.2% | 0.35 | No |
| Quiet-stretch squeeze plus 52-week breakout | 63 sessions | 73 (59) | +0.9% | −1.1% | 0.61 | No |
| Quiet-stretch squeeze plus 52-week breakout | 21 sessions | 84 (69) | +0.2% | −0.3% | 0.61 | No |
| Relative strength vs SPY just turned positive | 63 sessions | 381 (341) | +0.2% | −0.6% | 0.61 | No |
All 24 tests, closest first
| Setup | Hold | Events (dates) | Extra return vs random dates | Interval floor | FDR q | Survived |
|---|---|---|---|---|---|---|
| Deep pullback, 200-day still rising, momentum turning up | 126 sessions | 55 (43) | +5.9% | 1.6% | 0.16 | No |
| VIX spike, then calming | 21 sessions | 484 (154) | +1.3% | 0.3% | 0.16 | No |
| Deep pullback, 200-day still rising, momentum turning up | 63 sessions | 58 (46) | +3.5% | 0.9% | 0.16 | No |
| Quiet-stretch squeeze plus 52-week breakout | 126 sessions | 69 (55) | +2.6% | 0.2% | 0.22 | No |
| Deep pullback, 200-day still rising, momentum turning up | 21 sessions | 81 (67) | +1.4% | −0.2% | 0.35 | No |
| Quiet-stretch squeeze plus 52-week breakout | 63 sessions | 73 (59) | +0.9% | −1.1% | 0.61 | No |
| Quiet-stretch squeeze plus 52-week breakout | 21 sessions | 84 (69) | +0.2% | −0.3% | 0.61 | No |
| Relative strength vs SPY just turned positive | 63 sessions | 381 (341) | +0.2% | −0.6% | 0.61 | No |
| VIX spike, then calming | 63 sessions | 251 (108) | +0.4% | −1.6% | 0.61 | No |
| Oversold pullback in an uptrend | 21 sessions | 1,413 (683) | +0.1% | −0.4% | 0.61 | No |
| SPY 200-day regime just switched on | 21 sessions | 294 (27) | +0.3% | −1.0% | 0.61 | No |
| Oversold pullback in an uptrend | 63 sessions | 623 (393) | +0.1% | −0.9% | 0.61 | No |
| Deep 1-year drawdown with an RSI-2 hook | 21 sessions | 191 (101) | +0.1% | −3.8% | 0.61 | No |
| SPY 200-day regime just switched on | 63 sessions | 227 (25) | +0.1% | −2.2% | 0.61 | No |
| Volatility spike, then calming, in an uptrend | 126 sessions | 23 (17) | +0.2% | −4.3% | 0.61 | No |
| Deep 1-year drawdown with an RSI-2 hook | 126 sessions | 63 (42) | +0.5% | −11.0% | 0.61 | No |
| Oversold pullback in an uptrend | 126 sessions | 348 (240) | 0.0% | −1.5% | 0.61 | No |
| Relative strength vs SPY just turned positive | 126 sessions | 266 (238) | +0.0% | −1.6% | 0.61 | No |
| Deep 1-year drawdown with an RSI-2 hook | 63 sessions | 97 (66) | −0.4% | −6.0% | 0.61 | No |
| SPY 200-day regime just switched on | 126 sessions | 172 (22) | −0.7% | −6.8% | 0.61 | No |
| VIX spike, then calming | 126 sessions | 164 (68) | −0.4% | −3.9% | 0.61 | No |
| Relative strength vs SPY just turned positive | 21 sessions | 617 (536) | 0.0% | −0.5% | 0.61 | No |
| Volatility spike, then calming, in an uptrend | 21 sessions | 28 (19) | −0.3% | −3.0% | 0.61 | No |
| Volatility spike, then calming, in an uptrend | 63 sessions | 24 (18) | −1.2% | −3.5% | 0.85 | No |
For context only, the eight 63-day portfolio books. These were reported for reference and carried no pass or fail:
| 63-day book, training years | Sharpe | Yearly return | Worst drop | Time invested |
|---|---|---|---|---|
| Oversold pullback in an uptrend | 0.50 | 6.33% | 25.5% | 81% |
| Volatility spike, then calming, in an uptrend | 0.20 | 2.09% | 9.3% | 15% |
| VIX spike, then calming | 0.25 | 3.58% | 42.7% | 41% |
| Quiet-stretch squeeze plus 52-week breakout | 0.32 | 3.29% | 11.4% | 34% |
| Deep pullback, 200-day still rising, momentum turning up | 0.40 | 4.15% | 12.6% | 22% |
| Relative strength vs SPY just turned positive Highest Sharpe | 0.57 | 7.31% | 23.1% | 83% |
| SPY 200-day regime just switched on | 0.43 | 4.24% | 23.6% | 34% |
| Deep 1-year drawdown with an RSI-2 hook | 0.20 | 3.06% | 50.2% | 29% |
| SPY, fully invested Reference | 0.41 | 7.47% | 55.2% | 100% |
The highest Sharpe belonged to a setup that was in the market about 83% of the time with no event edge, which is mostly market exposure.
Why it failed
With 24 tries on 23 ETFs, an extra 5.9% on 55 events does not stand out from luck. The closest q-value (a false-discovery score) was 0.16, and our bar was 0.10. We did not loosen that bar after seeing the result.
Receipts: the setups were written down and committed before any returns were computed. 24 event trials plus 8 reporting-only books.