EdTech Tracker

← Coding lens

prompt caching

technique judged 98% likely a named technique or architecture

First seen
2024-10-01
Independent sources
11
Articles
32
Last 4 weeks vs prior
0 vs 15

See how it has spread across media tiers

Mentions over time

Articles per week (by publish date) using the term, in any spelling.
05.5112024-W402026-W332026-W41
Table view
WeekArticlesSources
2024-W4011
2026-W0111
2026-W0921
2026-W1161
2026-W2411
2026-W2511
2026-W3322
2026-W3711
2026-W3822
2026-W3911
2026-W40112
2026-W4131

In their words

Prompt caching on Amazon Bedrock can stop working without a single error.

Prompt caching.

Several replies to the original pointed at a third problem: every filtered history is a prompt cache miss.

⚡ Laptop: with the retrieval cache and prompt caching working together on questions that share paragraphs, the model started answering about 10x sooner.

Both had input_tokens and both reported prompt-cache activity.

On DeepSeek V4 Pro the saving was about 2% per session, smaller than ordinary run-to-run variation, because prompt caching had already made the repeated output nearly free.

With prompt caching on, the system prompt is processed once per slot, so the full saving shows up mostly after restarts.

Prompt caching sounds like an obvious win on paper.