AI and LLMs · Middle · Question 32 of 75
What is a semantic cache for LLM calls, and its main risk?
- 1Caching tokens on the client; it uses too much disk space
- 2Reusing answers for similar queries; it may return wrong cached answers
- 3Storing model weights in RAM; it slows the server down
Show the answer
Correct answer: Reusing answers for similar queries; it may return wrong cached answers
Explanation
"Cancel my order" and "Can I cancel an order?" are similar but may need different answers. Thresholds and scoping are critical.