How are LLMs typically trained?

• A. They are trained on large datasets to predict the next word using self-supervised learning and fine-tuned for specific tasks.
• B. They are trained by unsupervised clustering without words.
• C. They are trained only with labeled data for each token.
• D. They are trained by human handwriting analysis.

Answer: (C)

Training large language models hinges on learning from vast amounts of text using a self-supervised objective. The model is exposed to huge corpora and learns to predict the next word in a sequence, which provides supervision without any manual labeling. This pretraining teaches the model rich language patterns, structure, and knowledge. After this broad understanding is built, the model is fine-tuned or adapted to specific tasks using task-specific data, often with labels, so it can perform well on things like translation, summarization, or QA. This combination—large-scale self-supervised pretraining to model language, followed by task-focused fine-tuning—is what makes LLMs powerful and flexible.

Unsupervised clustering without words wouldn’t teach the model to generate coherent language or understand textual sequences. Requiring labeled data for every token would be impractical at the scale of these models. Handwriting analysis is not relevant to training models on digital text.