Does response caching save me money?
It saves on provider tokens, not on ModelRiver requests. A cache hit is answered by ModelRiver in under 10ms without calling the provider, so you pay no tokens for it, but it still counts as a request toward your plan.
When is a response served from the cache?#
Only when the new request matches an earlier one exactly: the same prompts, model, temperature, top_p and attachments, within the cache window you choose (for example 15 minutes, 1 hour or 1 day).
When is caching worth turning on?#
When many users send the same thing, for example:
- FAQ-style questions with fixed wording
- Classifying or summarizing the same content more than once
- Retries from your own app after a timeout
For chat that's different every time, caching rarely hits and adds nothing.
What isn't cached?#
- Streaming requests
- Requests in a conversation session
- Playground runs
How do I turn it on or clear it?#
Open the workflow, go to Caching (optional), pick a window and publish. To clear cached responses, for example after changing your prompt, use Clear in the same place. In Request Logs, the Cache filter shows which requests were served from the cache.