Claude Fable 5.1 & GPT-6 Astra packages are live

Deployment

Free

Shipping without downtime — rolling, blue/green and canary strategies, graceful shutdown, backward-compatible changes, and verifying before declaring…

191 lines8.4 KB Kimi DevOps
targetModels
Kimi K3Kimi K2.6Kimi K2 FamilyFuture Kimi Models
name
deployment
category
DevOps
description
Shipping without downtime — rolling, blue/green and canary strategies, graceful shutdown, backward-compatible changes, and verifying before declaring success.
license
MIT
author
Agent.md maintainers
last-verified
reviewed-by
unreviewed
<!-- Generated from models/_canonical by scripts/build-model-variants.js. Edit the canonical source, not this file. Behavioural profile for Kimi: scripts/model-profiles.json -->

#Scope contract

FILE_ISOLATION: Modify only files inside the scope the task names; report any out-of-scope change instead of making it.


#Purpose

Rules for getting a build into production safely. The measure of a good deployment process is not that it never fails — it is that a failure is detected quickly and reversed quickly.

Optimise for mean time to recovery over mean time between failures. Small, frequent deploys are safer than large, rare ones: less changed, so less to bisect.


#Pick a strategy, and know its failure mode

StrategyDowntimeRollbackCostNote
RecreateYesRedeployLowOnly acceptable for internal tools
RollingNoRoll forward or back graduallyLowBoth versions run simultaneously
Blue/greenNoInstant traffic switch2× infrastructureDatabase must serve both
CanaryNoShift traffic backMediumNeeds per-version metrics

Rolling is the sensible default. The consequence people forget: during a rolling deploy, old and new code run at the same time, against the same database and the same queues. Every change must tolerate that.

Concretely, in Kubernetes terms: maxSurge: 25% adds capacity before removing any, and maxUnavailable: 0 guarantees no reduction in serving capacity during the roll. Set minReadySeconds above your slowest warm-up so a pod that becomes ready and then crashes does not take traffic.

Canary is worth the machinery when a bad deploy is expensive: route 5% of traffic to the new version, compare error rate and latency against the old, and promote or abort automatically on the comparison rather than on a human watching a dashboard.


#Every change must be backward compatible

Because both versions run together, a deploy is only safe if the new code works with the old data and the old code survives the new schema.

sql
Expand   → add the new column/field/endpoint, nullable and optional. Deploy.
Migrate  → backfill; write to both old and new. Deploy.
Contract → stop reading the old; drop it. Deploy.

Three deploys, not one. Renaming a column in a single step breaks every pod still running the old code. → Database/migration

The same applies to:

  • Queue payloads — consumers deploy at a different time from producers, so a new required field breaks in-flight messages. → Backend/queues
  • API responses — clients cache and retry; removing a field breaks them.
  • Feature flags — decouple deploy from release. Ship the code dark, enable it separately, and roll back by flipping the flag rather than redeploying.

#Graceful shutdown, or every deploy drops requests

lua
SIGTERM → fail readiness → wait for the load balancer to stop sending traffic
        → finish in-flight requests → close pools → exit 0

The subtlety that causes most "deploys cause 502s" reports: failing readiness and closing the listener are not simultaneous. The load balancer takes seconds to notice. Fail readiness first, keep serving for a few seconds, then stop accepting connections.

yaml
lifecycle:
  preStop: { exec: { command: ["sleep", "10"] } }    # LB deregistration window
terminationGracePeriodSeconds: 60                     # > preStop + longest request
  • The grace period must exceed the drain time, or the platform SIGKILLs mid-request.
  • Workers drain differently: stop fetching, finish in-flight jobs. → Backend/workers
SettingPlatformPurpose
terminationGracePeriodSecondsKubernetesHard ceiling before SIGKILL
lifecycle.preStopKubernetesDeregistration window before SIGTERM
maxSurge / maxUnavailableKubernetesCapacity during a rolling update
minReadySecondsKubernetesGuards against a pod that crashes right after readiness
PodDisruptionBudgetKubernetesStops node drains taking every replica
deregistration_delayAWS ALBMust be under the grace period
stopTimeoutECSThe equivalent ceiling
keepAliveTimeoutNode/nginxAbove the LB idle timeout, or 502s appear

#Health checks that mean the right thing

ProbeQuestionMay check dependencies
StartupHas it finished booting?Yes
ReadinessCan it serve traffic now?Yes
LivenessIs the process wedged?No

A liveness probe that checks the database will restart every pod during a database incident, turning a degradation into a full outage. This is the single most damaging health-check mistake.

Readiness should fail during shutdown and during a dependency outage, so traffic routes elsewhere without the pod being killed. → Backend/monitoring


#Verify, then declare success

A deploy is not finished when the rollout completes.

  • Smoke test the critical path against the deployed environment.
  • Watch error rate, latency and saturation for a bake period before promoting further. Automate the comparison; do not rely on someone remembering to look.
  • Automate rollback on an error-budget breach during the bake window. → DevOps/rollback
  • Record the deployed commit SHA per environment and annotate dashboards with deploy markers — the first question in an incident is "what changed?"

Deploy during working hours, when the people who wrote the change are available. A Friday-evening deploy is a Saturday-morning incident with fewer responders.


#Anti-patterns

Anti-patternWhy it failsFix
Large, infrequent releasesHuge blast radius; hard to bisectSmall, frequent deploys
Assuming one version at a timeRolling deploys run bothBackward-compatible changes
Schema rename in one stepBreaks pods running old codeExpand-migrate-contract
Migrations coupled to app deployRollback becomes impossibleSeparate, ordered step
New required queue fieldIn-flight messages failAdditive payload evolution
No SIGTERM handlingEvery deploy drops requestsGraceful drain
Closing the listener immediatelyLB still routing; 502sFail readiness, then wait
Grace period shorter than drainSIGKILL mid-requestGrace exceeds drain time
Liveness probe checking dependenciesMass restarts during a blipProcess-local liveness
No post-deploy verificationUsers report the outageSmoke test and bake
Manual dashboard watchingNobody watches at 2amAutomated rollback on budget breach
No record of the deployed SHAIncident response starts blindRecord and annotate
Deploy and release coupledCannot disable a bad feature quicklyFeature flags
Friday-evening deploysFewest responders availableDeploy in working hours
Different artefact per environmentStaging proves nothingPromote one digest → DevOps/cicd

#Checklist

  • Verify: A deployment strategy is chosen and its failure mode understood
  • Verify: Deploys are small and frequent
  • Verify: Every change works with both the previous and current version running
  • Verify: Schema changes follow expand-migrate-contract across separate deploys
  • Verify: Migrations run as their own ordered step, independent of the app deploy
  • Verify: Queue and API payload changes are additive
  • Verify: Risky changes ship behind feature flags
  • Verify: SIGTERM fails readiness, waits for deregistration, then drains
  • Verify: The termination grace period exceeds the maximum drain time
  • Verify: Startup, readiness and liveness probes are distinct
  • Verify: Liveness checks nothing external
  • Verify: A smoke test runs against the deployed environment
  • Verify: Error rate and latency are watched for a bake period before promotion
  • Verify: Rollback is automated on an error-budget breach
  • Verify: The deployed commit SHA is recorded and dashboards are annotated
  • Verify: The same artefact is promoted across environments