Naive / pooled-AR forecast combination

1 of 3 lines pass the pre-registered bar

No multiplicity adjustment across lines or losses; each line has its own pre-registered verdict. Comparisons with pooled AR and seasonal naive, RMSE, squared-error DM, by-year results, the weight path and fallback counts are descriptive only.

DM differential = L(baseline) − L(combination), error = forecast − actual. Positive t favors the combination. CR1 standard errors cluster by target quarter; two-sided p uses Student t(G−1). There is no correction for dependence between quarters. An undefined p is a FAIL.

Net interest income (BHCK4074)

FAIL: combination MAE 118,235 ≥ naive MAE 117,823 thousands USD; absolute-error DM t = -1.545, two-sided p = 0.1408 (bar ≤ 0.05). The verdict uses unrounded values.

Pooled AR naive fallbacks among evaluated cases: 21 (for these, the combination equals naive). Origins using the fewer-than-minimum-quarters weight: 0. Of 900 selected origin–BHC pairs, 6 lack targets and 1 more lack current or seasonal history; 893 cases are scored.

Noninterest income (BHCK4079)

PASS: combination MAE 266,054 < naive MAE 294,201 thousands USD; absolute-error DM t = 2.216, two-sided p = 0.0406 (bar ≤ 0.05). The verdict uses unrounded values.

Pooled AR naive fallbacks among evaluated cases: 4 (for these, the combination equals naive). Origins using the fewer-than-minimum-quarters weight: 0. Of 900 selected origin–BHC pairs, 6 lack targets and 1 more lack current or seasonal history; 893 cases are scored.

Noninterest expense (BHCK4093)

FAIL: combination MAE 223,758 < naive MAE 230,601 thousands USD; absolute-error DM t = 0.527, two-sided p = 0.6047 (bar ≤ 0.05). The verdict uses unrounded values.

Pooled AR naive fallbacks among evaluated cases: 7 (for these, the combination equals naive). Origins using the fewer-than-minimum-quarters weight: 0. Of 900 selected origin–BHC pairs, 6 lack targets and 1 more lack current or seasonal history; 893 cases are scored.

The pre-registered bar

For each line, combo PASSES iff MAE(combo) < MAE(naive) strictly, on unrounded values, AND the full-sample clustered DM (y9c.forecast.clustered_dm; d = |e_naive| − |e_combo|; CR1 by target quarter; t(G−1), G=18) gives a two-sided p ≤ 0.05. An undefined p is a FAIL. There is no multiplicity adjustment. The headline is "k of 3 lines pass". Descriptive only: comparisons vs pooled AR and seasonal naive (MAE + DM), RMSE, squared-error DM, by-year results, the w path by origin and fallback counts. The results page must disclose that (a) pooled AR was chosen as the second component after seeing v1's descriptive tables on this same window, and (b) this is the second method scored on the 2022–2026 window.

Fixed pre-registration · Commit 288d4b85af9526666715ac265aa715aaab059194 · Commit time: 2026-10-03 08:01:24 ET. SHA-256: b16c1a129cc7b9320c40716694c6d39b5a3c6277eaf1511dea7eeeebc70d49e6. Run start: 2026-10-03 08:08:38 ET (recorded as 2026-10-03T12:08:38.522754+00:00 UTC) at git HEAD 5df7998d2a25edc30de9f2e87732e7a040c037d3.

Question and design

Does a walk-forward combination of the two existing Y-9C baselines, naive and pooled AR on log growth, beat naive one quarter ahead on three FR Y-9C income-statement lines for the top-50 BHC panel, on the same cases and window as v1?

Top 50 BHCs by reported BHCK2170 at each origin t, RSSD ID breaks ties; identical to Y-9C (y9c.forecast.design). Targets: 2022Q1–2026Q2; origins: 2021Q4–2026Q1. Per line, the Y-9C case rule from y9c.forecast: bank selected at origin t, target X_t+1 observed, X_t and X_t-3 observed. Identical to v1 (PR #4): the same 893 matched cases per line; all methods are scored on exactly the same cases within a line.

thousands USD; quarterly flows de-cumulated from calendar-year YTD by the shared y9c panel code (y9c.panel.decumulate). Errors receive equal weight per BHC-quarter.

Method (pre-registered text)

• For line ℓ ∈ {BHCK4074 NII, BHCK4079 noninterest income, BHCK4093 noninterest expense}, bank i, origin t: N = X_i,t (naive); P = the pooled AR on log growth from y9c.forecast.baseline_forecasts, imported unchanged and including its naive fallback; C = N + w_ℓ,t·(P − N), with w on the grid {0, 0.05, …, 1.00}.
• w_ℓ,t = argmin over the grid of the sum of |C − X| over pseudo-OOS cases (i, s) with target s+1 ≤ t. N and P for each such case are rebuilt as of origin s, using only data ≤ s, the y9c.forecast.design top-50 selection at s and the shared case rule. Use an expanding window from the earliest origin at which y9c.forecast's pooled AR is defined. Ties go to the smallest w. If the pseudo-OOS set spans fewer than 8 distinct target quarters, w = 0.5. There is one weight per line per origin, shared across banks.
• No tuning, trimming, winsorizing, clipping or case removal. Nonfinite values raise an error.
• Evaluation is identical to v1: targets 2022Q1–2026Q2 (origins 2021Q4–2026Q1), with the same 893 matched cases per line.

Implementation notes (pre-registered)

• C is computed as N + w·(P − N), except that at w = 1 it is set to P (algebraically identical). In floating point this makes w = 0 reproduce naive and w = 1 reproduce pooled AR exactly, and a case with P = N (including pooled AR's naive fallback) has C = N at every w, so it adds the same loss at every grid point.
• At origin t the panel is truncated to quarters ≤ t before the weight is fit. Pseudo-OOS N and P at each origin s come from y9c.forecast.baseline_forecasts(y9c.forecast.design(truncated panel), s): selection and pooled-AR coefficients at s use only rows dated ≤ s; the target X_s+1 ≤ t is used only as the actual for the shared case rule and the loss.
• The earliest pseudo-OOS origin is the earliest quarter at which y9c.forecast.baseline_forecasts does not raise its pooled-AR burn-in or rank error; the window then expands to origin t − 1.
• Evaluation-window N, P and seasonal naive are y9c.forecast.baseline_forecasts on the full panel at origins 2021Q4–2026Q1, exactly as in v1; combo is formed from those with the origin-t weight.

Imported from y9c.forecast (not reimplemented): naive Xhat_t+1 = X_t; seasonal naive Xhat_t+1 = X_t-3; pooled AR on log growth g(s+1) = a + b1 g(s) + b4 g(s-3), expanding pooled OLS on historical top-50 rows with s+1 <= t, naive fallback when log inputs are missing or non-positive.

Weight path · Descriptive only

w is the weight on pooled AR (w = 0 is naive, w = 1 is pooled AR). Each origin's weight is fit only on pseudo-out-of-sample cases whose targets were already observed at that origin.

Per-year errors and DM · Descriptive only

Per-year breakdowns have few target-quarter clusters each and do not change the full-sample verdicts.

Net interest income · Descriptive only

Per-year DM vs the combination · Descriptive only

Noninterest income · Descriptive only

Per-year DM vs the combination · Descriptive only

Noninterest expense · Descriptive only

Per-year DM vs the combination · Descriptive only

Figures

Data vintage

Coverage: 2018Q1–2026Q2, 34 quarters. Build time (UTC): 2026-09-26T21:15:19.490629+00:00. Source: FFIEC NIC Financial Data Download.

Tables and reproduction

overall_errors CSV · overall_errors JSON · by_year_errors CSV · by_year_errors JSON · dm_tests CSV · dm_tests JSON · weights CSV · weights JSON · verdicts JSON · run_metadata JSON · test_design_combo JSON · vintage JSON

make statement-forecast-combo-test
make statement-forecast-combo

The shared y9c package supplies de-cumulation, the naive, seasonal-naive and pooled-AR baselines, metrics and the clustered DM test; the combination imports them unchanged. Only aggregate tables and figures are exported.

Limitations

Report-quarter backtest on currently downloaded NIC filings, as in Y-9C and v1: period-t filings are treated as available at t; filing lags and later revisions are not modeled.

RSSD identities are used as reported, without pro-forma merger adjustments; acquisition jumps remain. Missing predecessor quarters are never bridged or imputed. Dollar losses emphasize large institutions. The weight is a single scalar per line shared across banks. Research only.

Citations

• Pesaran, M. H., Pick, A., & Timmermann, A. (2026). Forecasting with panel data: Estimation uncertainty versus parameter heterogeneity. Quantitative Economics, 17(2), 342–393. doi:10.3982/QE2589. Working paper: CEPR Discussion Paper DP17123 (2022). https://doi.org/10.3982/QE2589
• Wang, X., Hyndman, R. J., Li, F., & Kang, Y. (2023). Forecast combinations: An over 50-year review. International Journal of Forecasting, 39(4), 1518–1547. doi:10.1016/j.ijforecast.2022.11.005. https://doi.org/10.1016/j.ijforecast.2022.11.005