Which statement best describes base models in terms of their primary training objective?

• A. They are trained to predict the next token.
• B. They are trained to classify images.
• C. They are trained to generate poetry only.
• D. They are trained to translate languages exclusively.

Answer: (A)

Base models are trained to predict the next token in a sequence, an autoregressive language modeling objective. This means during training the model looks at a stream of text and learns to assign high probability to the actual next word or subword given all the previous ones. By repeatedly predicting the next token and updating its weights to maximize the likelihood of the observed data, the model learns grammar, facts, reasoning patterns, and how humans typically structure language. This setup is what enables the model to generate coherent text by sampling tokens sequentially and continues to be the defining aim of base language models.

Other tasks like image classification, translation, or generating poetry are achieved with different objectives or architectures. Image classification focuses on labeling an image’s content, translation requires mapping from one language to another with a source and target sequence, and poetry generation isn’t restricted to predicting the next token in a general sense. The primary training signal for base language models, however, is next-token prediction, which is why that statement best describes them.