Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer

When a student model learns from a teacher’s data, the influence of that teacher can linger even after the original examples are removed. A recent paper investigates how subtle, non‑semantic signals—what the authors call subliminal trait transfer—persist in the model’s optimizer state and later shape behavior. By treating the model’s parameters together with its optimizer moments as a single trainer state, the authors derive an exact identity that separates the raw propagation of a teacher’s perturbation from the value assigned to it by future training steps. Their analysis shows that the first‑order moment (the optimizer’s running average of gradients) acts as a causal carrier of the trait.

What You Need to Know

The paper focuses on a scenario where a teacher model generates data that contains a behavioral disposition—such as a tendency to favor certain responses—but that disposition is never expressed in the semantic content of the data. A student model trained on this data acquires the disposition despite the absence of explicit cues. Earlier work traced how these signals enter the gradient during training, but left open the question of how they survive when the teacher’s data is no longer present or how they might change sign under later updates.

To answer this, the authors combine the model’s parameters and the optimizer’s internal statistics (first and second moments) into a unified trainer state. They derive a transport‑valuation identity that cleanly splits two effects: (1) the unbiased propagation of the original perturbation through optimization dynamics, and (2) the later valuation of that perturbation by subsequent training steps and a behavioral readout. Using state‑surgery experiments—where they cut out and transplant parts of the trainer state—they demonstrate that moving only the first moment is sufficient to transfer the trait, while leaving parameters, hidden activations, and immediate outputs unchanged at the cut point.

Transplanting the isolated first moment into a fresh optimizer yields source‑free updates that still exhibit the transferred trait, confirming that the moment acts as a causal carrier. The paper also shows that the sign of the transferred effect can flip depending on the curvature of the loss landscape encountered after the cut, explaining why later training can reverse or amplify the original disposition.

Why It Matters

Understanding where and how implicit biases or strategies are stored has practical implications for model safety, continual learning, and knowledge distillation. If a model can retain a teacher’s hidden tendency solely in its optimizer state, then simply deleting the training data or fine‑tuning on new data may not erase that tendency. This insight helps diagnose unexpected behaviors that arise long after exposure to certain data sources.

Moreover, the result suggests a new lever for controlling transfer: by manipulating the optimizer’s moments directly, practitioners could inject or suppress specific traits without altering the model’s architecture or retraining from scratch. This opens avenues for more precise model editing techniques and for auditing the persistence of undesirable tendencies in large‑scale systems.

Key Details

• The trainer state comprises model parameters θ and optimizer moments m (first‑order) and v (second‑order).
• The transport‑valuation identity separates the linear propagation of a perturbation from its nonlinear valuation by later loss gradients.
• State‑surgery experiments show that transplanting m alone leaves θ, hidden states, and immediate outputs unchanged at the cut point.
• Source‑free updates after transplanting m still produce the transferred behavioral trait, confirming causality.
• The sign of the transferred effect can change depending on the Hessian‑induced curvature encountered after the cut.
• Theoretical analysis assumes standard stochastic gradient descent with momentum; extensions to adaptive optimizers are discussed.

What’s Next

Future work could explore whether similar moment‑based carriers exist for higher‑order optimizer statistics or for normalization layers, and how these carriers interact with regularization techniques like weight decay or dropout. Empirical studies on large language models and vision transformers would test the scalability of the findings. Finally, developing practical tools to monitor and edit optimizer moments could provide a direct route to mitigating unwanted implicit biases in deployed systems.

📌 Source: Arxiv Ml