What is alignment in model training?

• A. Adjusting a model to match user preferences, such as ensuring polite responses in customer service applications
• B. Matching model size to hardware
• C. Aligning tokens to a fixed vocabulary
• D. Aligning training steps with a clock

Answer: (A)

Alignment in model training means shaping how the model behaves so its outputs reflect human goals and preferences. It’s about making the model act in ways people find safe, useful, and appropriate, not just maximizing raw capabilities. For example, adjusting a model to be polite and helpful in customer service ensures interactions feel respectful and constructive, aligning the model’s responses with user expectations and organizational guidelines. This often involves feedback from people and reward-based fine-tuning to steer behavior toward desirable outcomes.

Other options describe things like fitting the model to hardware, which affects efficiency, not how its answers align with human preferences; token vocabulary setup, which is a linguistic/engineering detail; or training schedule, which is a logistical aspect. These influence how the model runs, not how it behaves in line with human intent.