What is an example of a decoder-based transformer?

• A. BERT from Google.
• B. Llama from Meta.
• C. BART from Facebook.
• D. T5 from Google.

Answer: (B)

Decoder-based transformers are the autoregressive models that generate text token by token, using masked self-attention so each position only sees previous tokens. Llama from Meta is built as a decoder-only model, meaning it consists of a stack of layers that generate output directly from past tokens without a separate encoding stage. This makes it a classic example of a decoder-based transformer.

BERT from Google is an encoder-only model designed for understanding tasks, not generation, so it doesn’t fit the decoder-only pattern. BART from Facebook and T5 from Google use encoder-decoder architectures, with an input processed by an encoder and output produced by a separate decoder, which differentiates them from a decoder-only setup.