AI and LLMs · Senior · Question 58 of 75
What's a sensible latency budget approach for a chat assistant with tools?
- 1Hide all progress until the full answer is ready
- 2Ignore latency, since users expect AI to be slow
- 3Targets per step, and stream progress to users
Show the answer
Correct answer: Targets per step, and stream progress to users
Explanation
Showing "searching orders..." while tools run, plus timeouts per tool, keeps the experience responsive.