.svg){kind=spacer}
Details
- Anthropic researchers unveil Model Spec Midtraining (MSM), a new training phase between pretraining and alignment fine-tuning (AFT) to improve how AI models generalize desired behaviors.
- MSM trains models on synthetic documents discussing the Model Spec or constitution, teaching the 'what and why' before AFT demonstrates aligned actions.
- Toy example: Identical AFT data on cheese preferences leads to different generalizations—pro-America values or affordability—based on MSM spec used.
- Realistic application: MSM on a harmlessness spec drastically reduces unsafe actions in agentic settings compared to AFT alone.
- Empirical tests show value explanations or detailed subrules in specs outperform basic rules, reducing issues like policy misuse (e.g., 20% to 2% in one case).
- MSM reduces need for chain-of-thought supervision and enables studying optimal specs for alignment.
Impact
Anthropic's MSM builds on midtraining trends seen in OpenAI's alignment experiments and IBM's reasoning studies, positioning it as a key technique to bridge pretraining and post-training distributions. By enabling models to internalize specs' motivations, it pressures rivals like OpenAI to refine their Model Spec compliance methods, potentially narrowing gaps in agentic safety where standard AFT falls short. This could accelerate scalable oversight in frontier models, aligning with calls for robust generalization amid rising AI autonomy risks, without relying on costly RLHF variants.
