AWS says caching repeated context in Amazon Bedrock reduces time-to-first-token and input costs on cache hits.

AWS published guidance on prompt caching in Amazon Bedrock, a feature that stores partially processed input so later requests with the same prefix skip redundant computation. The company reports cache hits cut input token costs by up to 90 percent and reduce time-to-first-token without altering the model or prompt quality.

The post walks through six caching patterns using the Converse API, moving from message content caching for long documents to system prompt caching, tool definition caching for agentic workflows, mixed TTL tiers, per-tenant isolation, and LangChain integration. Amazon says cache entries remain scoped to individual AWS accounts and AWS Regions, and each cache checkpoint uses token thresholds.

Teams that resend long contracts, policy documents, or tool schemas alongside many separate questions gain the most, because caching removes repeated processing rather than shortening the context. Response caching alone handles identical queries and offers no help when the same context pairs with different questions.

Operators should evaluate cache hit rates, token thresholds, and TTL settings before rolling the feature across workloads, since cache scope and account boundaries shape the savings. Multi-tenant products need per-tenant cache separation, and LangChain users can adopt the same pattern inside existing chains.

What matters

• Amazon Bedrock now caches system prompts, documents, and tool definitions at the infrastructure level.
• Builders sending the same context with many questions avoid paying full price for tokens already processed.
• Cache entries stay scoped to individual AWS accounts and Regions, so multi-tenant teams must plan isolation.

Why it matters

Cache entries stay scoped to individual AWS accounts and Regions, so multi-tenant teams must plan isolation.

This GenAI News article was prepared in original wording using reporting and materials published by AWS Machine Learning Blog. Source reference: https://aws.amazon.com/blogs/machine-learning/optimizing-cost-and-latency-with-amazon-bedrock-prompt-caching/.

Drafted by the GenAI News review pipeline.