Dotted = GELU-fit initialization. Grey = fixed activations.
All five coefficients across training. Each coefficient is displayed on an independent vertical scale. Vertical line marks the selected epoch.
Pooled over all six MLP blocks of the seed-1 source ViT-d6 (the run whose coefficients are transferred), on 512 ImageNet-1K validation images. Shaded: the central 98%, [−1.02, 1.04].