TRACE, beside explainable boosting

What an explainable boosting machine is

An explainable boosting machine (EBM) predicts a single number, usually a mean, as a sum of small functions of one feature at a time:

g(x) = g0 + f1(x1) + f2(x2) + … + a few pairwise terms.

Each fj is a shape function. In the usual implementation the feature is cut into bins, and the fit cycles through the features: hold the rest of the sum fixed, update this feature's bins from the residual gradient, then move on. Many rounds, a small learning rate. Because the pieces are added, you can plot each fj on its own and read off what that feature does. That is the glass box.

Binning is what makes this cheap. A column becomes a bin code, and the update is a histogram of gradients. The cost of a round grows with the number of rows, not with the square of the number of rows.

Pairs are optional. Looking at every pair in full is expensive, so GA2M / FAST first screens candidate pairs from the residuals and only then fits the ones that look useful (Lou, Caruana, Gehrke and Hooker, Accurate Intelligible Models with Pairwise Interactions, KDD 2013). InterpretML packages this whole pattern: cyclic boosting, binning, the FAST-style interaction pass, and the plots (Nori, Jenkins, Koch and Caruana, InterpretML: A Unified Framework for Machine Learning Interpretability, arXiv:1909.09223, 2019).

Where EBMs are strong

• They are built for the thing most regression losses ask for: a conditional mean (or a similar one-number summary). On mean squared error they are often the right baseline, and they can beat a distributional model that spends its capacity on scale and tails.
• The shape plots are the explanation. There is no second model that “explains” the first.
• The public implementation in InterpretML is careful, fast, and widely exercised. TRACE is a research prototype. When the comparison below says EBM wins, that is the result.

What a mean model does not show is a change in width or in the tail when the centre stays put. That is the gap TRACE is aimed at.

Mean versus the whole distribution

The fair distributional baseline is not “EBM has no intervals”. Fit the EBM, take residuals on a calibration split, and add those residual quantiles to the prediction. The band then has a width, but the width does not depend on x, and it does not have a separate tail shape per feature. TRACE's scale channel is allowed to depend on x. Its shape channel is allowed to bend the tails.

TRACE's grid pinball loss does not promise 80% or 90% coverage, so the raw row can sit above those levels. The calibrated row is a split-conformal offset and is scored on its own. The residual-quantile EBM is built to hit a coverage target when the residuals are exchangeable. Both coverages are in the table.

Raw TRACE coverage and calibrated TRACE coverage are different rows. EBM plus conformal residuals still wins on S1 period 2, MSE of the mean; on vendor spend, CRPS and MSE of the mean; on n = 2e4, CRPS and MSE of the mean. A re-run of this same comparison after the v2 code matches the table to the digits shown. Full tables are in docs/TRACE_SIMS.md.

v2 hybrid, scored and not adopted

The v2 mean channel is a cyclic squared-loss booster, and the other channels are a CRPS fit of the residual. On 20 fresh seeds the first run does not match the additive booster. Seed-mean MSE is 1.14×, 4.76× and 3.81× that booster. The bar was 1.01× on every case. The gap is overfitting: one level was written into every pool on every visit, and early stopping kept that checkpoint. Closing the validation leak in the bin edges, and a second pre-registered attempt that writes one two-leaf split per visit, both still miss the bar. The hybrid stays available. It is not the default. The picture and the other two v2 options are on the theory page.

What a mean model cannot return

An explainable boosting machine fits one number per row. The two jobs below are not an accuracy contest with that machine. It has no composition/structure split, and it has no row that is itself a sample from a distribution.

• Shift between two periods, same conditional law, different covariate law. Total W2² 0.0672, composition 0.0850, structure 0.0014, cross -0.0191. Structure is the small term, which is what a covariate-only shift should look like. The split is order-dependent. T9 checks that the three terms add back to the total.
• Distribution-valued rows. Each row is an exponential sample whose scale is linear in a discrete covariate, and the Hosking fit's λ₂ RMSE on that draw is 0.0087. Collapsing each row to its mean and fitting an EBM would answer a different question.

There is no winner rule on the vendor-spend draw. It is a simulated table of units, price, deliveries, distance, quality and season, with a planted location, a planted scale and a small tail. The metrics sit next to S1–S4 in the simulation note.

How TRACE fits

The model, written on the quantile function:

Q(u | x) = Q0(u) + Σj fj(xj, u) + Σpairs fjk(xj, xk, u)

then a monotone projection in u. Q0 starts as the lower pinball endpoint of the training outcomes (the order statistic at ceil(u n) - 1) and then absorbs the shared channel correction. Each feature function uses a short basis in the level u.

1. Cut every feature into histogram bins. A missing value gets its own bin. Codes are one byte when there are at most 255 bins.
2. Each round, compute the pinball subgradient of the current quantile fit once, on a fixed grid (19 levels by default). Average pinball on that grid is the training loss, and twice that average is the reported CRPS approximation.
3. Update every feature from that same gradient. Do not refresh the gradient between features. Refreshing it would make the answer depend on which column was visited first, which is the failure TRACE is written to avoid.
4. Add the updates in a canonical order of the raw columns, so a permutation of the columns is the same sum.
5. Center each feature update, then add the average of the removed channel means back once, into the shared quantile level. A common scale correction is kept. It is not added once per feature.
6. Stop early on a held-out slice, but not before min_rounds. With more than one bag and a bag fraction below 1, that slice is the out-of-bag rows. Drop a feature only after those rounds, and keep it when either the full pinball gain or the shape channel's tail-weighted gain clears the threshold. Several bags keep a feature selected in at least 70% of them.
7. Screen pairs by a residual-gradient energy on a two-dimensional histogram, fit them, then center each pair on both margins and move those margins into the main effects. Project each predicted curve so it increases in u. An optional calibration split adds a conformal offset at each level and projects again.

The projection is what makes the delivered quantiles monotone. It is also what blurs a shape interaction: a bend that would have decreased in u gets flattened, and the pure interaction contrast shrinks. A pure location interaction, constant in u, is left alone. The measured ratios are in the simulation note.

A round is one pass over the rows for the gradient and one pass per feature for the histogram. Nothing in that path builds an n × n matrix. The design note has the pseudocode and the complexity table: TRACE_DESIGN.md.

Checks and oddness

The booster will happily use a column that is a copy of the outcome, a copy of another column, or a column that only lines up with the outcome inside a random split. The checks are part of the method, and they are ordinary statistics with fixed seeds, not a second model.

• Target proxy. Correlation of each column with y, and with y shifted by a row. A column that leads y (today's feature is tomorrow's outcome) shows up at a positive shift.
• Duplicate. Bit-identical columns, and pairs that are almost collinear.
• Placebo target. Shuffle y and look at the largest correlation any feature manages. A real proxy sits far above that maximum.
• Adversarial validation. Wasserstein distance, per feature, between the training window and the test window. A time-only column moves a lot under a time split and little under a random split.
• Time-respecting importance. Fit on the past, shuffle a column on the future, and see whether the pinball loss actually gets worse. A random split can still reward a column that will not travel forward.

Oddness

Hold out a calibration set. Score every row by its mean pinball loss under the fitted quantiles. The conformal p-value of a new score s is (1 + number of calibration scores ≥ s) / (n_cal + 1). Under exchangeability, those p-values stay above α except for about a fraction α of ordinary rows. A planted spike in y should land in the tail.

The planted designs, the thresholds that were written down first, and the numbers that came back are in TRACE_SIMS.md.