Learn home · TRACE · TRACE theory · research, unvalidated
Explaining a shift
Two periods, one outcome. Part of the change is who shows up. Part is that the same covariates now map to a different distribution. This page splits those two parts, channel by channel, without letting the order of the split decide the answer.
The distance between two distributions
A period is a list of rows. Each row has covariates \(x\) and one outcome \(y\). Period A and period B need not have the same rows. The question is how the outcome distribution moved, and why.
For a one-dimensional outcome the squared Wasserstein-2 distance is ordinary squared distance between quantile functions. If \(Q_A(u)\) is the level-\(u\) quantile of period A, and \(Q_B\) the same for period B, then
\[ W_2^2(A,B) = \int_0^1 \bigl(Q_B(u) - Q_A(u)\bigr)^2 \, du. \]A location shift of size \(d\) contributes \(d^2\). A change in width contributes too, and so does a change in the tails. The integral adds them.
Channels
Quantile functions are curves. The shifted Legendre polynomials \(P^*_k\) on \([0,1]\) are a perpendicular set of curves, so a quantile function has coordinates
\[ \lambda_{k+1} = \int_0^1 Q(u)\, P^*_k(u)\, du, \qquad c_k = (2k+1)\,\lambda_{k+1}. \]\(Q = \sum_k c_k P^*_k\) when the curve lives in that span. Channel 0 is the mean. Higher channels carry scale and shape. Because the polynomials are perpendicular,
\[ W_2^2 = \sum_k \frac{(\Delta c_k)^2}{2k+1} = \sum_k (2k+1)\,(\Delta\lambda_{k+1})^2 \]inside the span. Anything the span cannot see is an extra remainder, the unresolved channel. It is reported next to the sum, so the total stays exact. TRACE stores the orthonormal coefficients \(a_k = c_k / \sqrt{2k+1}\). The same identity in those units is \(W_2^2 = \sum_k (\Delta a_k)^2\).
Composition and structure
Hold the conditional law \(Q(u\mid x)\) fixed and change only the mix of covariates. That is composition: different rows walked in, the map from \(x\) to the outcome stayed put. Hold the covariate mix fixed and change the map. That is structure.
Neither piece is observed alone. Period A gives the pair (structure A, covariates A). Period B gives (structure B, covariates B). The two counterfactuals fill the square:
- \(C\): structure B, covariates A.
- \(C'\): structure A, covariates B.
On one channel the four numbers are \(\lambda^A\), \(\lambda^{C'}\), \(\lambda^C\), \(\lambda^B\). Moving covariates first and structure second does not have to match the other order. The cross term \(\lambda^B - \lambda^C - \lambda^{C'} + \lambda^A\) is that disagreement. It is real. It is split in half rather than assigned to whichever switch was written down first.
Two averages, and the gap
There are two honest answers to “the distribution of a period”.
The marginal law mixes the rows and then takes quantiles: \(F_t(y) = \int F_t(y\mid x)\, dF_t(x)\). No model is required. Only the mean is linear in that mixture. Higher L-moments of a mixture are not the mixture of the L-moments, so a per-feature split of those channels is not an identity. The channel split still is.
The barycenter averages the quantile functions themselves: \(\bar Q_t(u) = \mathbb{E}_{x\sim F_t}[Q_t(u\mid x)]\). In one dimension that is the Wasserstein barycenter of the conditional laws. Coefficients pass through the expectation, \(\bar c_{t,k} = \mathbb{E}[c_{t,k}(x)]\). TRACE writes each channel as an intercept plus a sum of functions of one feature,
\[ c_{t,k}(x) = b_{t,k} + \sum_j f_{t,kj}(x_j), \]so the average splits by feature with nothing left over.
Averaging both orders
For one channel, label the empty coalition by \(\lambda^A\), the composition-only coalition by \(\lambda^{C'}\), the structure-only coalition by \(\lambda^C\), and both by \(\lambda^B\). The Shapley value averages the two orders:
\begin{align*} \mathrm{comp} &= \tfrac12\bigl[(\lambda^B - \lambda^C) + (\lambda^{C'} - \lambda^A)\bigr], \\ \mathrm{struct} &= \tfrac12\bigl[(\lambda^C - \lambda^A) + (\lambda^B - \lambda^{C'})\bigr]. \end{align*}Add them and the counterfactuals cancel: \(\mathrm{comp} + \mathrm{struct} = \lambda^B - \lambda^A\). Swap which move is written first and the same two numbers come back. The interaction is reported on its own as well, even though each player has already received half of it.
A share for each feature
On the barycenter the players are the feature-level moves. Write \(\bar g^{\,s}_{t}(j)\) for the average of feature \(j\)'s function when the structure is period \(t\) and the covariates are drawn from period \(s\). Then
\begin{align*} \mathrm{comp}_{kj} &= \tfrac12\bigl[(\bar g^{B}_{A} - \bar g^{A}_{A}) + (\bar g^{B}_{B} - \bar g^{A}_{B})\bigr], \\ \mathrm{struct}_{kj} &= \tfrac12\bigl[(\bar g^{A}_{B} - \bar g^{A}_{A}) + (\bar g^{B}_{B} - \bar g^{A}_{B})\bigr], \end{align*}plus one intercept player \(\Delta b_k\). That is \(2p+1\) numbers, and they add up to \(\Delta \bar c_k\).
The squared distance is a square of a sum. The fair share of a player with coefficient \(a_i\), when \(\sum_i a_i = \Delta c_k\), is
\[ \varphi_i = \frac{a_i \sum_i a_i}{2k+1}. \]Sum the shares and you get exactly that channel's \(W_2^2\). A share can be negative: the player pulled against the net move. The negative number is kept. Clipping it would break the sum.
In orthonormal coefficients the same rule is \(\varphi_i = \alpha_i \sum_i \alpha_i\), because the \(\sqrt{2k+1}\) factors cancel. The marginal-law split uses the same formula with two players, composition and structure, after converting \(\Delta\lambda\) to the \(c\) scale.
Reweighting, cross-fit, and trimming
The barycenter counterfactual does not need weights. Average structure B's curve on period A's rows. That is \(\bar c_C = \mathbb{E}_A[c_B(x)]\).
The marginal counterfactual does. Bayes turns a period classifier into a density ratio. With \(T\) the period label,
\[ \psi(x) = \frac{P(A\mid x)}{P(B\mid x)}\cdot\frac{P(B)}{P(A)}. \]Reweight period B's rows by \(\psi\) and the weighted sample has the law of \(C\): structure B, covariates A. Reweight period A by \(1/\psi\) and you get \(C'\). A logistic regression on the raw covariates is the classifier used here. With equal sample sizes the prior ratio is 1, so \(\psi(x) = \exp(x^\top\beta)\) once the intercept is included in \(\beta\).
Each row's weight comes from a model that did not train on that row. The rows are cut into folds, stratified by period, and the fold of a row is scored by the fit on the other folds. The weights are then scaled to average 1.
Extreme weights are capped at a quantile chosen before looking at the result. The mass removed by the cap, as a fraction of the weight sum, is reported. Overlap is an assumption, not a conclusion: if period A's covariates sit outside period B's, no finite weight rebuilds \(C\).
Bands and a noise floor
Rows are resampled inside each period, and the whole calculation is repeated, including the cross-fit and the TRACE fit. The 2.5 and 97.5 percentiles of those replicates are a pointwise band. A uniform band across a list of components uses the largest studentized residual in each replicate, takes the 95th percentile of that maximum, and inflates every component's standard error by that one critical value. A component is marked resolved when the uniform interval misses zero.
A squared distance is a poor thing to test against zero with a bootstrap: noise biases it upward. The noise floor is a permutation. Shuffle the period labels, keep the counts, and recompute. Under exchangeability the shuffled statistics describe the null. The 95th percentile of the shuffled total \(W_2^2\) is the floor. The observed total is compared with that floor, not with zero.
Arrivals, departures, and mass
Some rows are in A only (exits). Some are in B only (entries). The rest are stayers. Let \(e_A\) be the share of A that leaves and \(e_B\) the share of B that is new. The barycenter is linear in the row weights, so the change in average coefficients splits exactly:
\begin{align*} \mathrm{stay} &= \tfrac12\bigl[(1-e_A)+(1-e_B)\bigr]\,(\bar c_B^{S} - \bar c_A^{S}), \\ \mathrm{mix} &= \tfrac12(\bar c_A^{S} + \bar c_B^{S})\,(e_A - e_B), \\ \mathrm{entry} &= e_B\,\bar c^{N}, \\ \mathrm{exit} &= -e_A\,\bar c^{X}. \end{align*}These four are players in the same square-of-a-sum game. If the stayers did not change, the stay term is zero and the move is mix plus the arrivals and departures.
What this does not say
The split is a fair division of a squared distance, not a causal claim. “Composition” means the covariate mix under a stated conditional structure, and it needs overlap. The marginal law and the barycenter answer different questions; a small gap means they happened to agree, not that one of them was checked against the other and discarded. Bootstrap coverage is an asymptotic statement. A negative share is a direction, not a defect.
The algebra and the estimator are in SHIFT_V2.md. The numeric identities are docs/theory/verify/shift_v2_checks.py.