gate.json, secondary.json). The run then stopped in sensitivity S4 (exclude 2020Q1–Q2 from training): KeyError: Period('2020Q1') in WAR-RM, because the masked 2020Q1 origin cross-section was empty. S4 was logged as a trial before it ran and was not scored. sensitivities.json and r1.json were not produced; S1–S3 ran but their results were not saved; 15 approved executions never ran (S4b, S5–S16, FB4 and R1). By Quant's ruling there is no rerun. Trials logged: 24 of the 39 approved.tracks/y9c-trading-ot/prereg/OOS_APPROVAL_v1_1.json (sha256 fe473f36b530…, design commit c38a284); the design JSON keeps oos_authorized: false and its pin 930353641c6a…. Run at commit 5f11ed5. 37 → 39: ERRATA_v1_1b §1 confirmed 37 executions at ce9ce39; its §4 then required the K = 2 FPCA WAR variant and the ridge point forecast with a trial each, giving the 39 pinned in the approval file. §4-B scenarios are deferred with no trial (prereg/DEFERRAL_4B.md). The pinned errata are not edited.Primary verdict: H1 FAIL · H2 FAIL (run INCOMPLETE)
One primary gate: two hypotheses on the 2014Q1–2026Q2 targets, Holm-adjusted across the pair (FWER 0.05). The methods come from the classical-OT guide: §4 barycenters for forecast combination (H1) and §3 density time series in Wasserstein space (H2).
PASS needs all five: Holm p < 0.05; relative CRPS gain G ≥ 3%; pooled 90% coverage in [0.80, 0.97]; G > 0 when any one bank is dropped; and G > 0 in at least 2 of the 3 subperiods (P1 2014–17, P2 2018–21, P3 2022–26Q2). If only the drop-one-bank or subperiod condition fails, the result is PASS-fragile (not adopted). FAIL if G ≤ 0 or the fixed-b upper bound GU is below 3%. Anything else, or fewer than 40 valid target quarters, is INCOMPLETE.
Primary gate, 2014Q1–2026Q2
| Hypothesis | Comparator | G (scaled CRPS) | p (fixed-b, one-sided) | p (Holm) | GU | 90% coverage | Min G, drop one bank | Subperiods with G > 0 | Targets | Verdict |
|---|---|---|---|---|---|---|---|---|---|---|
| H1 · §4 BARY-EW (equal-weight W₂ barycenter, annually refreshed bank set) | B* (epoch) | -4.5% | 0.887 | 1.000 | +1.8% | 0.86 | -6.0% | 1 of 3 | 50 | FAIL |
| H2 · §3 WAR-RM (Wasserstein AR + rank map, cross-section) | B*CS (epoch) | -29.4% | 1.000 | 1.000 | -18.5% | 0.93 | -33.0% | 0 of 3 | 50 | FAIL |
Test C: DM on the per-quarter cross-bank mean loss differential d̄t (positive favours the OT method), Bartlett long-run variance with M = ⌊√T⌋ (7 at T = 50), p-value from 200,000 simulated null statistics (seed 20261006). Scores are scaled by each bank's trailing-16-quarter MAD at the forecast origin. B*, the trimmed members and the widening s are re-selected every Q4 on the trailing 12 target quarters, from walk-forward forecasts only.
Subperiods (pre-registered, descriptive except the G > 0 count)
| Hypothesis | Subperiod | G | p (raw) | 90% coverage | Min G, drop one bank | Targets | Sign |
|---|---|---|---|---|---|---|---|
| H1 | P1 · 2014Q1–2017Q4 | -3.4% | 0.867 | 0.88 | -5.7% | 16 | G ≤ 0 |
| H1 | P2 · 2018Q1–2021Q4 | +0.3% | 0.476 | 0.81 | -1.3% | 16 | G > 0 |
| H1 | P3 · 2022Q1–2026Q2 | -11.9% | 0.871 | 0.88 | -14.1% | 18 | G ≤ 0 |
| H2 | P1 · 2014Q1–2017Q4 | -51.2% | 1.000 | 0.93 | -54.5% | 16 | G ≤ 0 |
| H2 | P2 · 2018Q1–2021Q4 | -21.2% | 0.971 | 0.92 | -28.8% | 16 | G ≤ 0 |
| H2 | P3 · 2022Q1–2026Q2 | -21.2% | 0.956 | 0.93 | -24.4% | 18 | G ≤ 0 |
P3 next to R1 (v1.0 design): not independent evidence
| Hypothesis | P3 G | P3 p (raw, test C) | R1 G | R1 p (v1 test A) | Role |
|---|---|---|---|---|---|
| H1 | -11.9% | 0.871 | — | — | NOT RUN |
| H2 | -21.2% | 0.956 | — | — | NOT RUN |
Break checks (FB1–FB4)
| Hypothesis | Check | Result | Reading |
|---|---|---|---|
| H1 | FB1 fluctuation (rolling 15-quarter mean of d̄t) | stat 2.03 vs cv 3.07 | no reject |
| H1 | FB2 mean shift at 2015Q3 | shift +0.013, p (Holm) 1.000 | no shift |
| H1 | FB2 mean shift at 2016Q3 | shift -0.038, p (Holm) 1.000 | no shift |
| H1 | FB2 mean shift at 2020Q1 | shift -0.019, p (Holm) 1.000 | no shift |
| H1 | FB2 mean shift at 2022Q1 | shift -0.096, p (Holm) 0.617 | no shift |
| H2 | FB1 fluctuation (rolling 15-quarter mean of d̄t) | stat 4.38 vs cv 3.07 | reject |
| H2 | FB2 mean shift at 2015Q3 | shift +0.009, p (Holm) 0.906 | no shift |
| H2 | FB2 mean shift at 2016Q3 | shift +0.127, p (Holm) 0.271 | no shift |
| H2 | FB2 mean shift at 2020Q1 | shift +0.241, p (Holm) 0.028 | shift |
| H2 | FB2 mean shift at 2022Q1 | shift +0.143, p (Holm) 0.467 | no shift |
FB3 (training from 2010Q1) is sensitivity S4b and FB4 (excluding 2020Q1–Q2 targets) is in the sensitivity table below; S10 is the leave-one-year-out check.
Per-bank CRPS gains (H1)
The minimum G after dropping any one bank is in the primary table (column “Min G, drop one bank”).
Figure not rendered: results-page figures are a later rendering step (ERRATA_v1_1b §4); the numbers are in the tables on this page and in the JSON outputs.
Coverage and calibration
| Hypothesis | Method | 50% cov. | 90% cov. | Pinball 0.05 | Pinball 0.5 | Pinball 0.95 | MAE (median) | PIT counts (10 bins) |
|---|---|---|---|---|---|---|---|---|
| H1 | BARY-EW | 0.48 | 0.85 | 0.254 | 0.670 | 0.239 | 1.340 | 93 82 82 91 78 81 83 93 101 117 |
| H1 | B* | 0.35 | 0.67 | 0.227 | 0.615 | 0.308 | 1.230 | 148 70 54 51 69 69 56 80 83 221 |
| H2 | WAR-RM | 0.68 | 0.93 | 0.382 | 0.836 | 0.540 | 1.672 | 129 127 245 320 359 330 313 238 169 153 |
| H2 | B*CS | 0.37 | 0.69 | 0.322 | 0.690 | 0.366 | 1.379 | 471 227 197 174 176 175 164 179 180 440 |
Industry total (T1): forecast vs actual
Figure not rendered: results-page figures are a later rendering step (ERRATA_v1_1b §4); the numbers are in the tables on this page and in the JSON outputs.
Cross-section (T3): WAR-RM density fan
Figure not rendered: results-page figures are a later rendering step (ERRATA_v1_1b §4); the numbers are in the tables on this page and in the JSON outputs.
Secondary family (Holm within the family; cannot change the primary verdict)
| Test | Method | Comparator | G | p (raw, test C) | p (Holm, secondary) | Targets | Role |
|---|---|---|---|---|---|---|---|
| H1b | CRPS-shrunk barycenter weights (κ = n/(n+200)) | B* | +2.8% | 0.107 | 0.967 | 50 | secondary |
| H1c | Equal-weight linear pool (F1 comparator) | B* | +0.3% | 0.461 | 1.000 | 50 | secondary |
| H1d | BARY-EW on the industry total (B0, B1, B3, B4 on the level) | B*T1 | +6.3% | 0.076 | 0.761 | 50 | secondary |
| H1e_h2 | BARY-EW at h = 2 | B*h | -1.4% | 0.633 | 1.000 | 49 | secondary |
| H1e_h3 | BARY-EW at h = 3 | B*h | +3.4% | 0.114 | 0.967 | 48 | secondary |
| H1e_h4 | BARY-EW at h = 4 | B*h | +2.4% | 0.255 | 1.000 | 47 | secondary |
| H2b | WAR-RM mapped to the bank set ($) | B* | -66.0% | 1.000 | 1.000 | 50 | secondary |
| H2c_persistence | Cross-section trimmed W₂² (U19) | persistence Qt | +26.7% | 0.046 | 0.508 | 50 | secondary |
| H2c_climatology | Cross-section trimmed W₂² (U19) | climatology Q̄t | +12.3% | 0.006 | 0.068 | 50 | secondary |
| H2d | Fréchet regression on VIX and rates volatility + rank map | B*CS | -33.6% | 1.000 | 1.000 | 50 | secondary |
| H2e | OT quantile-map recalibration of B* (≥ 100 PITs) | B* | +0.9% | 0.278 | 1.000 | 47 | secondary |
| H2f | Autoregressive OT map (scalar) + rank map | B*CS | -30.2% | 0.999 | 1.000 | 50 | secondary |
Other pre-registered secondary items (outside the Holm family)
| Item | Method | Comparator | Result | Role |
|---|---|---|---|---|
| H2 FPCA | K = 2 FPCA functional WAR + rank map (raw p, no Holm) | B*CS | G -31.9%, p 1.000 | secondary |
| Ridge (pt) | Pooled ridge point forecast, λ by GCV once on burn-in (frozen = 0.02512); scaled MAE | BARY-EW / B* medians | MAE gain +79.5% / +81.2% | secondary metric |
| §4-B | Barycentric scenarios, W₂ scenario reduction, energy score | — | — | DEFERRED |
Sensitivities S1–S16, S4b, FB4 (pre-registered; none can change a primary verdict)
| ID | Variation | H1 G | H2 G | H1d G | Role |
|---|---|---|---|---|---|
| S1 | Bank set: full-sample balanced ($10m, $100m; look-ahead) and the fixed 2013Q4 set | — | — | — | ran; not saved (sensitivities.json not produced) |
| S2 | Modelling scale: revenue / total assets; raw $ | — | — | — | ran; not saved (sensitivities.json not produced) |
| S3 | Cross-section threshold $10m / $50m / $1bn | — | — | — | ran; not saved (sensitivities.json not produced) |
| S4 | Exclude 2020Q1–Q2 from training | — | — | — | CRASHED (KeyError 2020Q1 in WAR-RM); logged, not scored |
| S4b | Training from 2010Q1 | — | — | — | NOT RUN |
| S5 | Drop targets at merger events and the quarter after | — | — | — | NOT RUN |
| S6 | No Q1 dummy in B4/B5; WAR with a Q1 tangent shift | — | — | — | NOT RUN |
| S7 | CRPS on 19 levels (rescore, no re-selection) | — | — | — | NOT RUN |
| S8 | Target ex-CVA/DVA for banks ≥ $100bn (rows before 2011Q1 unadjusted) | — | — | — | NOT RUN |
| S9 | Macro data through the availability date | — | — | — | NOT RUN |
| S10 | Leave one OOS year out | — | — | — | NOT RUN |
| S11 | Mid-year entrant rule (b), industry total | — | — | — | NOT RUN |
| S12 | Rolling 16-quarter estimation windows | — | — | — | NOT RUN |
| S13 | Settings frozen at 2013Q4 | — | — | — | NOT RUN |
| S14 | Frozen burn-in score scale, 2010Q1–2013Q4 (rescore, no re-selection) | — | — | — | NOT RUN |
| S15 | Test A and WPE p-values | — | — | — | NOT RUN |
| S16 | B5 without the Baa–10-year spread | — | — | — | NOT RUN |
| FB4 | Exclude 2020Q1–Q2 targets | — | — | — | NOT RUN |
Design (v1.1)
- Target. Quarterly BHCKA220, de-cumulated from year-to-date (Q1 as reported, no bridging over gaps), thousands of USD. Negative values kept; no logs. A first filing outside Q1 has no quarterly value. Flows are blanked in 16 bank-quarters where year-to-date interest income falls within a year. Two tiered subsidiaries are dropped in 2016Q3–Q4, when their parents also file.
- History. 2009Q1–2026Q2, with expanding estimation windows. 2008 is pre-sample: only 2008Q4 trading assets are used, as the denominator of the 2009Q1 ratio. Burn-in 2009Q1–2013Q4.
- Views. T1 industry total. T2 an annually refreshed ex-ante bank set: at each Q4, banks with |quarterly A220| ≥ $10m in all of the last 16 quarters. T3 banks with lagged trading assets ≥ $100m.
- Baselines. SAA-8, seasonal naive, ratio random walk, SES (α = 0.3), AR(1)+Q1, pooled quantile regression on FRED covariates (VIX, 10-year rate volatility, Nasdaq volatility, Baa–10-year spread), and an empirical-Bayes dynamic panel.
- H1. Equal-weight quantile average (the 1-D W₂ barycenter) of the trimmed baselines, widened by s about the median.
- H2. The forecast cross-section is a point on the W₂ geodesic from the Fréchet mean to the latest cross-section (scalar β). It maps to banks through a Gaussian-copula AR on probit ranks (scalar ρ).
Ex-ante bank-set size by Q4 origin (membership counts only)
| Origin | 2013Q4 | 2014Q4 | 2015Q4 | 2016Q4 | 2017Q4 | 2018Q4 | 2019Q4 | 2020Q4 | 2021Q4 | 2022Q4 | 2023Q4 | 2024Q4 | 2025Q4 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Banks | 12 | 16 | 17 | 18 | 18 | 19 | 17 | 19 | 19 | 21 | 22 | 19 | 21 |
The 2021Q4 set is exactly the v1.0 ex-ante set. At the first freeze (2013Q4), all seven baselines produce forecasts for all 72 burn-in pseudo-out-of-sample cases (12 banks × 6 targets, 2012Q3–2013Q4).
Data: FFIEC NIC FR Y-9C bulk files, 74 quarters 2008Q1–2026Q2. Macro: FRED VIXCLS, DGS10, NASDAQCOM, BAA10Y. Report-quarter backtest on the latest vintage; filing revisions are not modelled.
The v1.0 design (test_design_trading_ot.json, sha256 dffd1323…) stays pinned and byte-identical; it runs only as R1.