AI and LLMs · Junior · Question 73 of 75
Why do long answers take noticeably longer than long prompts?
- 1Input tokens are always slower to process than output ones
- 2Both are processed instantly, whatever their length
- 3Output is generated token by token, so long answers are slow
Show the answer
Correct answer: Output is generated token by token, so long answers are slow
Explanation
The prompt is processed in parallel, but each output token depends on the previous one. Asking for concise answers is one of the easiest latency wins.