What was going to be forecast
Each month, the spread of complaint counts across companies in the CFPB consumer-complaint file: the equal-weighted distribution of log(1 + complaints) among companies with at least one complaint that month, with three pinned credit-bureau names left out. Two optimal-transport methods were pre-registered against simple baselines: an equal-weight W₂ barycenter of the baselines (the average of their quantile functions) and a Wasserstein autoregression of the monthly quantile functions. The plain-language explanation is on the learning page.
The kill rule and the numbers
| Burn-in target months | 2013-01 to 2017-12 |
|---|---|
| Median next-month W₂² (99-point grid, 59 pairs) | 0.0063598632 |
| Median half-sample W₂² (60 months, seed 20261008) | 0.0268769480 |
| Ratio, next-month / half-sample | 0.2366289198 |
| Rule: kill if next-month ≤ half-sample | fired |
| Out-of-sample months scored (2018-01 to 2026-09) | 0 |
The half-sample distance is the noise floor: split each month's companies at random into two halves and measure how far apart their quantile functions are. If consecutive months are closer than that, the month-to-month signal is below the noise and there is nothing for a forecast to beat.
Pre-registration record
| File (tracks/cfpb-complaints-ot/prereg/) | sha256 |
|---|---|
PREREG_cfpb_complaints_ot.md | d3e8ac46de75fdd181a99d3fc3519107ff84d704ae37034390cbf9bd8ac24a46 |
cfpb_ot_learning.html | 9c5d5dc30264673cefe68df15163a6ddae5b89be63cad752843d37801d9f9061 |
test_design_cfpb_ot_v1.json | a5170147070a25163ed3549403ffe48860c749c4ccb4e0243d20aa6befada584 |
The design JSON keeps oos_authorized: false, and that flag is not the switch. Because the burn-in
kill fired, an out-of-sample run of this object would not start even if an approval file were written later.
A different object would need its own pre-registration.