Teaching: Naive / Pooled-AR Forecast Combination

1. Why this method, and why now

The first method (Liu, Moon & Schorfheide empirical-Bayes panel) beat naive on 0 of 3 lines. The diagnosis below says the failure came from the model's structure (mean-reverting asset ratios in levels) rather than from the empirical-Bayes step. Naive is hard to beat because bank income lines are very persistent over one quarter.

One cheap, transparent forecaster in the repo already does better than naive on two lines by MAE: the shared pooled AR on log growth baseline. It does worse on NII. That is exactly the setting forecast-combination theory covers. When two forecasts each have useful information and their errors are not perfectly correlated, a convex combination with a well-estimated weight can beat both. A weight learned on past data also shrinks toward whichever component has actually done better.

Pesaran, Pick & Timmermann (2022 working paper; Quantitative Economics 2026) is the recent anchor paper for panel data. It compares individual, pooled, fixed-effects and empirical-Bayes panel forecasts, proposes optimal combination weights, and finds that empirical-Bayes and combination methods do best overall and rarely give the worst forecast for any single series. Our v1 already tested the empirical-Bayes branch. This v2 tests the combination branch.

Diagnosis of v1 (from the published tables)

Line	Naive MAE	Pooled AR MAE	EB panel MAE	EB vs naive DM p (abs)
NII (BHCK4074)	117,823	134,049	150,011	0.0026
Noninterest income (BHCK4079)	294,201	262,576	394,099	0.0002
Noninterest expense (BHCK4093)	230,601	222,663	277,329	0.0158

Thousands USD, 893 matched BHC-quarters per line, targets 2022Q1–2026Q2. Source: results/tables/overall_errors.csv, verdicts.json.

Mean reversion in the wrong place. EB models 400·X/A(t−1) in levels over a 12-quarter window. Its estimated persistence was low (NII median ρ ≈ 0.63; noninterest-income ρ fell to about 0.15 or lower while 2023Q1 was in the window), so forecasts were pulled toward each bank's window mean while flows trended through the 2022–2024 rate cycle. Naive never mean-reverts.
Acquisition spikes. First Citizens' 2023Q1 bargain-purchase gain alone accounts for 24.4M of the 99.9M noninterest-income gap, because it inflated σ² and pushed ρ down.
Merger quarters were hit twice by lagged-asset scaling: 8.4M / 25.6M / 34.8M of the gaps.
The Tweedie shrinkage itself cost little: about 1M / 17M / 6M. EB was not the problem.
Growth-space forecasting with a seasonal lag already helps on two lines. Pooled AR has lower MAE than naive on noninterest income and expense. On NII it is worse, and Y-9C's published DM for pooled AR vs naive on NII favors naive (p = 0.025). No naive-vs-pooled-AR DM is published for the two noninterest lines, so those two MAE gaps are descriptive only.

2. What the papers found

Pesaran, Pick & Timmermann (anchor)

Setup: linear dynamic panels with heterogeneous slopes, weakly exogenous regressors and possibly correlated heterogeneity. They compare forecasts from individual, pooled, fixed-effects and empirical-Bayes estimates, and combinations of individual and pooled forecasts.
Key claim: whether pooling helps depends on a trade-off. Pooling removes estimation noise but adds bias from heterogeneity. The balance depends on how heterogeneous the parameters are, whether that heterogeneity is correlated with the regressors, the model's fit, and N and T. With short T (ours is short), estimation noise dominates and pooled or shrunk forecasts tend to win.
Result: in simulations and in applications to house prices and CPI inflation, empirical-Bayes and forecast-combination methods perform best overall and rarely produce the least accurate forecast for any individual series.
Limitations for us: their optimal weights target squared error under their model. We score absolute error, and our components include a model-free random walk. So we borrow the principle (estimate a combination weight, pool it across units because T is short) rather than their closed-form weight.

Wang, Hyndman, Li & Kang (supporting review)

This 50-year review of forecast combination documents the long-standing finding that combinations often beat the best single component, and that simple or heavily regularized weights are hard to beat. Elaborate estimated weights often lose to equal weights because the weights are estimated with error (the "forecast combination puzzle").
Our takeaway: use one scalar weight per line, on a coarse grid, estimated on all banks pooled. Do not use bank-specific or time-varying weights.

3. The math

For line ℓ, bank i, origin t (all inputs dated ≤ t):

N_i,t+1 = X_i,t (naive) P_i,t+1 = X_i,t · exp( a + b1·g_i,t + b4·g_i,t−3 ) (pooled AR on log growth, y9c.forecast, unchanged) C_i,t+1 = N_i,t+1 + w_ℓ,t · (P_i,t+1 − N_i,t+1), w_ℓ,t ∈ {0, 0.05, …, 1}

The weight is fit only on pseudo-out-of-sample cases whose targets are already observed at t. For every earlier origin s with s+1 ≤ t, we rebuild N and P exactly as they would have been produced at s, using data ≤ s, the top-50 design at s and the same case rule. Then:

w_ℓ,t = argmin_w Σ_(i,s : s+1 \leq t) | N_i,s+1 + w\cdot(P_i,s+1 - N_i,s+1) - X_i,s+1 | ties \to the smallest w (toward naive) if the pseudo-OOS set spans < 8 distinct target quarters \to w = 0.5

Intuition. At w = 0 the method is naive and at w = 1 it is pooled AR. Because absolute error is convex, the combined MAE at any w is at most the weighted average of the two component MAEs, and it is strictly lower when the two error sets often have opposite signs. The weight is fit on the same loss that is scored (MAE), is shared by all ~50 banks because 12-quarter bank histories are too short, and is refit as each quarter's outcome arrives.

What this fixes structurally. Nothing is scaled by assets, so merger quarters are not hit twice. Nothing reverts to a 12-quarter level mean. The seasonal lag in P handles quarter-of-year patterns that naive ignores. Spikes are not trimmed. If they make P unreliable, the learned weight moves toward naive.

4. What we will do on this data

Panel, lines, universe and cases are identical to v1: top 50 BHCs by BHCK2170 at each origin (RSSD tie-break), BHCK4074 / 4079 / 4093 de-cumulated by y9c.panel, targets 2022Q1–2026Q2 (origins 2021Q4–2026Q1, 18 target quarters), the shared case rule, and the same 893 matched cases per line.
Components are imported, not re-implemented: naive and pooled AR come from y9c.forecast.baseline_forecasts, including pooled AR's naive fallback when log inputs are missing or non-positive. A fallback case has P = N, so C = N.
Pseudo-OOS history: pooled-AR forecasts are generated at every origin from the earliest one where y9c.forecast can fit it (the data start in 2018Q1) up to t−1. This uses only the shared code path and expanding pooled OLS on rows with s+1 ≤ origin. Pre-2022 history includes 2020; it is not dropped.
Leakage rules: the panel is hard-truncated to quarters ≤ t before anything is computed. The weight set includes only cases with target ≤ t. Selection never looks at t+1. No step uses the 2022–2026 evaluation outcomes except through this expanding, pre-origin rule.
No tuning: the grid, tie rule, minimum-history rule, loss and components are fixed here. There is no trimming, winsorizing or case removal. Nonfinite forecasts raise an error.
Reported descriptively: the w path by origin and line, by-year errors, RMSE, squared-error DM, the comparison against pooled AR and seasonal naive, and the fallback counts.

Honest caveat on selection. We picked pooled AR as the second component after seeing v1's descriptive tables, which cover the same 2022–2026 window. The weight itself is learned walk-forward, but choosing the component was informed by the test window. This is the second method scored on this window, so a pass here is weaker evidence than a pass on fresh data. The results page must state this. A frozen-spec confirmation on quarters after 2026Q2, once filed, is the clean follow-up. It is not part of this run.

5. Pass bar (fixed before the run)

This is the v1 bar unchanged, so the two methods can be compared directly.

Per line, the combination PASSES iff MAE(combo) < MAE(naive) strictly, on unrounded values, and the full-sample clustered Diebold–Mariano test on absolute error (naive vs combo; d = |e_naive| − |e_combo|; CR1 by target quarter; Student t(G−1), G = 18) gives a two-sided p ≤ 0.05.
An undefined p is a FAIL. There is no multiplicity adjustment, matching v1. The headline is "k of 3 lines pass."
Descriptive only, never a gate: combo vs pooled AR (MAE and DM), combo vs seasonal naive, RMSE, squared-error DM, per-year results and the weight path.
The real data are run once and the result is published whatever it is, including 0 of 3.

Prior expectation (not a prediction): the best chance is noninterest income, where pooled AR beat naive by the largest margin. Expense is possible. NII is unlikely, because pooled AR lost to naive there and the learned weight should stay near 0.

6. How we will know it worked

The pass bar above, per line.
Sanity checks: the combo at w = 0 reproduces naive and at w = 1 reproduces pooled AR exactly. The weight path is finite and on the grid. The v1 and Y-9C published files are byte-identical to main.
Leakage tests: perturbing or deleting all data dated ≥ t leaves w_ℓ,t and every forecast for targets ≤ t unchanged, while target t+1 does move (a non-vacuity check).

7. Citations

Pesaran, M. H., Pick, A., & Timmermann, A. (2026). Forecasting with panel data: Estimation uncertainty versus parameter heterogeneity. Quantitative Economics, 17(2), 342–393. doi:10.3982/QE2589. Working paper: CEPR Discussion Paper DP17123 (2022), cepr.org/publications/dp17123.
Wang, X., Hyndman, R. J., Li, F., & Kang, Y. (2023). Forecast combinations: An over 50-year review. International Journal of Forecasting, 39(4), 1518–1547. doi:10.1016/j.ijforecast.2022.11.005; preprint arXiv:2205.04216.
Context (v1 method): Liu, L., Moon, H. R., & Schorfheide, F. (2020). Forecasting with dynamic panel data models. Econometrica, 88(1), 171–201. NBER w25102.