Why are LLMs considered autoregressive?

• A. They generate tokens in parallel from the input.
• B. They rely solely on supervised labels for each token.
• C. They do not use previous context.
• D. They use previous outputs as inputs for the next token generation loop.

Answer: (D)

Autoregressive generation means the model uses what it has already produced to determine what comes next. In large language models, you start with some input, predict the next token, add that token to the input, and then predict the following token, repeating this loop. This creates a sequence where each token depends on all previous tokens, forming the conditional probabilities P(token_t | token_1, ..., token_{t-1}).

That’s why the best description is using previous outputs as inputs for the next token generation loop—the model’s current choice directly relies on what it produced earlier. The other ideas don’t capture this sequential dependency: generating tokens in parallel would break the step-by-step use of past outputs; relying only on token-level supervision or not using past context would not describe the iterative, context-dependent generation process that defines autoregression.