In an LLM conversation, how does attention contribute to processing long input sequences?

• A. It weighs tokens differently based on their relevance across the sequence.
• B. It converts all tokens to numeric IDs without context.
• C. It discards all tokens except the first.
• D. It duplicates tokens to increase sequence length.

Answer: (A)

Attention lets each token blend information from every other token by computing how relevant each one is to it and then weighing those contributions accordingly. This means tokens don’t just pass through in order; instead, the model forms a representation based on a weighted sum of all tokens, with the weights reflecting context-specific importance. When input sequences are long, this global weighting lets distant parts of the text influence each token’s meaning, capturing long-range dependencies in a single step rather than relying on sequential processing alone. The weights are learned, so the model can focus on the most relevant parts of the sequence for the task at hand.