AI and LLMs · Senior · Question 11 of 75
When does a batch API make sense for LLM workloads?
- 1For live chat, where users need answers in real time
- 2For non-urgent bulk jobs where lower cost beats latency
- 3For any request that contains personal data at all
Show the answer
Correct answer: For non-urgent bulk jobs where lower cost beats latency
Explanation
Classifying a million tickets overnight at a discount suits batch. Interactive features need synchronous calls.