Claude Fable 5.1 & GPT-6 Astra packages are live

Caching

Free

Caching layers, invalidation strategies, stampede protection and the correctness rules that keep a cache from serving one user another user's data.

218 lines9.1 KB Sarvam Ai Performance
targetModels
Sarvam-105BSarvam-30BSarvam FamilyFuture Sarvam Models
name
caching
category
Performance
description
Caching layers, invalidation strategies, stampede protection and the correctness rules that keep a cache from serving one user another user's data.
license
MIT
author
Agent.md maintainers
last-verified
reviewed-by
unreviewed
<!-- Generated from models/_canonical by scripts/build-model-variants.js. Edit the canonical source, not this file. Behavioural profile for Sarvam: scripts/model-profiles.json -->

#Locale

Examples use Indian conventions: ₹ amounts, IST, dd/mm/yyyy, Aadhaar and DPDP Act where a standard mentions identity or privacy law. Keep them when you copy an example.


#Purpose

Rules for caching. A cache trades freshness for speed, and the trade is only acceptable when you can state how stale data is allowed to be and how it is invalidated.

The rule that precedes all others: cache after the operation is correct and indexed, never instead. A cache in front of an unindexed query hides the problem until the cache misses, which is exactly when load is highest. → Database/query-optimization


#Know which layer you are in

LayerScopeInvalidationTypical TTL
BrowserOne userImpossible once sentImmutable assets only
CDNAll usersPurge by URL or tagMinutes to a year
Reverse proxyAll usersPurgeSeconds to minutes
Application (Redis)All instancesDelete by keySeconds to hours
In-processOne instanceRestartSeconds
DatabasePlan cache, shared_buffersAutomatic

Each layer has a different invalidation cost. A browser cache cannot be invalidated — once max-age=86400 is sent, that client will not ask again for a day. This is why HTML is short-lived and only content-hashed assets are cached hard:

arduino
Cache-Control: public, max-age=31536000, immutable       # /assets/app.a1b2c3.js
Cache-Control: public, max-age=0, s-maxage=300, stale-while-revalidate=86400   # HTML
Cache-Control: private, no-store                         # authenticated responses

max-age is the browser; s-maxage is the shared cache. Confusing them is how a deploy takes a day to reach users.


#Never cache a personalised response in a shared cache

This is the highest-impact caching bug in existence: one user's page served to another.

  • Authenticated responses get Cache-Control: private, no-store.
  • If a response varies by user, either do not cache it in a shared layer, or make the user part of the cache key — not a Vary header, which fragments and is easy to get wrong.
  • Vary: Cookie disables the cache in practice (every user has a different cookie) while looking like it works.
  • Audit: does any cache key omit the tenant or user for a response that depends on them? → API/api-security

#Invalidation: decide before you cache

Three strategies, and every cached value uses one of them explicitly:

StrategyCorrectnessComplexity
TTL onlyStale for up to the TTLLowest — start here
Write-through / delete-on-writeFresh immediatelyMust find every write path
Tag or event-basedFresh, handles fan-outHighest
ts
// Cache-aside with explicit invalidation on write
async function getOrder(id: string) {
  const hit = await redis.get(`order:${id}`);
  if (hit) return JSON.parse(hit);
  const order = await db.order.findUnique({ where: { id } });
  await redis.set(`order:${id}`, JSON.stringify(order), { EX: 300 });
  return order;
}

async function updateOrder(id: string, data: Patch) {
  const order = await db.order.update({ where: { id }, data });
  await redis.del(`order:${id}`);          // delete, do not update
  await redis.del(`orders:list:${order.tenantId}`);   // derived views too
  return order;
}

Delete rather than update the cache on write: writing the new value races with concurrent readers and can leave the cache permanently ahead of or behind the database.

A cache with no stated invalidation mechanism is a bug with a delay fuse. Write the mechanism down next to the TTL.


#Stampede, and the failure that follows a success

When a hot key expires, every concurrent request misses at once and hits the origin together. The database that was comfortable at 5% miss rate is not comfortable at 100%.

Three defences, in increasing order of effectiveness:

ts
// 1. Jittered TTL — hot keys do not expire in lockstep
await redis.set(key, value, { EX: 300 + Math.floor(Math.random() * 60) });

// 2. Single-flight — one recompute, others wait or serve stale
const lock = await redis.set(`lock:${key}`, id, { NX: true, PX: 10_000 });
if (!lock) return staleValue ?? await waitForRecompute(key);

// 3. Stale-while-revalidate — always serve, refresh in the background
if (entry.expiresAt < now) refreshInBackground(key);
return entry.value;

Stale-while-revalidate is usually the right answer for read-heavy data: users never wait, and the origin sees one request per key per refresh interval.

Also plan for the cache being down. Decide explicitly whether a Redis outage means serve-from-origin (fail open, origin may collapse) or fail requests. Write the decision down and rate-limit the origin accordingly. → Database/redis


#Cache the right things

Good candidatePoor candidate
Expensive aggregation, rarely changingA primary-key lookup already sub-millisecond
Third-party API responsesData that must be exact (balances, inventory)
Rendered fragments, config, feature flagsAnything personalised, in a shared cache
Session and permission lookups (short TTL)Values whose staleness has legal meaning

Caching a fast query adds a network hop, a serialisation cost and an invalidation bug for no gain. Measure the operation first.

Name keys so a whole class can be found, measured and expired:

ruby
<entity>:<version>:<scope>:<id>[:<variant>]

order:v2:tenant_8fd2:ord_4a1b          # single entity
orders:v2:tenant_8fd2:status=paid:p1   # a derived list view
flags:v1:global                        # configuration

The v2 segment is a schema version: bumping it invalidates every key of that shape at once, which is the only practical way to deploy a changed value format without serving old shapes to new code.

Monitor hit ratio, latency saved, eviction rate and memory. A cache_hit_ratio below ~80% usually means the key design is wrong, not that caching does not apply — typically a key containing a requestId, a timestamp, or a full URL with tracking parameters — something that varies more than the value does.


#Anti-patterns

Anti-patternWhy it failsFix
Caching to hide a slow queryBreaks under the load that caused itIndex first
No stated invalidation mechanismPermanently stale dataDecide before caching
Caching personalised data in a shared layerOne user's data served to anotherprivate, no-store, or key by user
Vary: CookieCache fragmented per user; effectively disabledKey explicitly
Updating the cache on writeRaces with readers; driftsDelete the key
Forgetting derived views on invalidationList stale while the item is freshInvalidate every affected key
Uniform TTLs on hot keysSynchronised stampedeJitter
No single-flight on recomputeEvery miss hits the origin at onceLock, or stale-while-revalidate
Long browser max-age on HTMLA deploy takes a day to reach usersShort s-maxage, no browser cache
Caching a sub-millisecond lookupAdds a hop and an invalidation bugDo not cache it
Caching exact-value dataUsers see wrong balancesRead through
No plan for cache unavailabilityOrigin collapses on a Redis restartExplicit fail-open/closed
Unbounded cache growthMemory exhaustion, then random evictionmaxmemory plus a policy
No hit-ratio monitoringThe cache may be doing nothingTrack and alert
Cache key missing the tenantCross-tenant data exposureInclude every varying dimension

#Checklist

  • Verify: The underlying operation is correct and indexed before any cache is added
  • Verify: Each cached value names its layer, TTL and invalidation mechanism
  • Verify: Authenticated and personalised responses are never in a shared cache
  • Verify: Cache keys include every dimension the value varies on, including tenant
  • Verify: max-age and s-maxage are set deliberately and differ where appropriate
  • Verify: Immutable assets are content-hashed and cached for a year
  • Verify: Writes delete cache entries rather than updating them
  • Verify: Derived and list views are invalidated alongside item keys
  • Verify: Hot-key TTLs are jittered
  • Verify: Recomputation is single-flighted, or stale-while-revalidate is used
  • Verify: Behaviour when the cache is unavailable is explicit and documented
  • Verify: Origin load is bounded even at a 100% miss rate
  • Verify: Memory limits and an eviction policy are configured
  • Verify: Hit ratio, eviction rate and memory use are monitored