<!-- Generated from models/_canonical by scripts/build-model-variants.js. Edit the canonical source, not this file. Behavioural profile for DeepSeek: scripts/model-profiles.json -->

#Task boundary

1. Implement exactly the task as stated. Do not add abstractions, options, config, or files the task did not name.
2. Comments, identifiers, commit messages and log strings are English only.
3. Stop when the checklist at the end passes. Do not refactor or "improve" surrounding code.
4. Every checklist item below is backed by an assertion in a test or by pasted command output, never by a sentence.

---

#Purpose

Rules for instrumenting a service so that you can tell whether it is healthy, and find out why when it is not.

Two different questions, two different tools:

1. Is it broken? Metrics and alerts. Low cardinality, cheap, always on.
2. Why is it broken? Traces and logs. High cardinality, expensive, sampled.

Instrumenting for the first and hoping it answers the second is the most common mistake. → Backend/logging

---

#Measure the four golden signals

ts
const httpDuration = new Histogram({
  name: "http_request_duration_seconds",
  help: "Request duration",
  labelNames: ["method", "route", "status"],   // route TEMPLATE, bounded set
  buckets: [0.01, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10],
});

Measure a histogram, not an average. An average hides the tail entirely: a p50 of 40 ms and a p99 of 9 s average out to something that looks fine while 1% of users cannot use the product.

Buckets must be chosen for your SLO — you cannot compute a p99 accurately if all your data lands in one bucket.

Saturation is the leading indicator. A database connection pool at 95% utilisation is an outage in a few minutes; latency has not moved yet.

---

#Control cardinality

A time series exists for every unique label combination. Cardinality is multiplicative, and it is what makes monitoring bills explode and queries time out.

ts
// 1 series per route — correct
{ route: "/orders/:id" }

// 1 series per order — millions of series, and the collector falls over
{ route: `/orders/${id}` }

Never use as a metric label: user id, order id, email, session id, full URL path, raw error message, or a timestamp. Those are log fields and trace attributes, where high cardinality is the point.

Keep total series per service in the thousands, not millions. Audit label sets in review — a new label multiplies every existing series.

---

#Instrument what the business cares about

Infrastructure metrics tell you a host is unhealthy. Business metrics tell you the product is broken, which is the thing users notice.

1. payments_captured_total, signups_completed_total, orders_created_total
2. Queue depth and oldest-message age per queue → Backend/queues
3. Job success/failure counts and durations by job type
4. Dependency call rate, error rate and latency, per dependency
5. Cache hit ratio

A deploy that breaks checkout while every host stays green is a routine outage. The business metric is what catches it.

---

#Traces

Distributed tracing is the only practical way to answer "where did the 4 seconds go" across services.

1. Use OpenTelemetry. Vendor-neutral, and every backend consumes it.
2. Auto-instrument HTTP, database and queue clients; add manual spans for significant internal work.
3. Propagate W3C traceparent across every service and queue boundary — a trace that stops at the first hop is nearly useless.
4. Put high-cardinality identifiers on span attributes (user.id, order.id), which is exactly where they belong.
5. Sample tail-based where available: keep 100% of errors and slow requests, a small fraction of fast successes.
6. Record the trace_id in logs so the two link.

ts
// OpenTelemetry: auto-instrument the edges, add spans for real work
const sdk = new NodeSDK({
  resource: resourceFromAttributes({
    [ATTR_SERVICE_NAME]: "orders-api",
    [ATTR_SERVICE_VERSION]: process.env.GIT_SHA,
    [ATTR_DEPLOYMENT_ENVIRONMENT_NAME]: process.env.ENV,
  }),
  traceExporter: new OTLPTraceExporter({ url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT }),
  instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();

await tracer.startActiveSpan("order.price", async (span) => {
  span.setAttribute("order.id", orderId);        // high cardinality belongs here
  span.setAttribute("tenant.id", tenantId);
  try { return await price(order); }
  catch (err) { span.recordException(err); span.setStatus({ code: SpanStatusCode.ERROR }); throw err; }
  finally { span.end(); }
});

---

#SLOs and alerts

Define the objective before the alert. An alert with no SLO behind it is a threshold someone guessed.

yaml
# 99.9% of checkout requests succeed within 500ms over 28 days.
# Error budget: 0.1% ≈ 40 minutes per month.

Alert on burn rate, not on instantaneous thresholds:

This is what removes the 3am page for a two-minute blip while still catching a slow burn that a static threshold never trips.

Every paging alert must satisfy all four:

1. A user is affected — not a proxy metric.
2. It is actionable — there is something the responder can do now.
3. It is urgent — it cannot wait until morning.
4. It links to a runbook with the first three diagnostic steps.

Anything failing one of these is a ticket or a dashboard, not a page. Review alerts monthly and delete the ones nobody acted on — an alert nobody trusts makes every other alert weaker. → DevOps/monitoring

---

#Health checks

Separate the two, because they mean different things to the orchestrator:

1. Liveness — is the process wedged? Must not check dependencies. A liveness check that fails when the database is down makes Kubernetes restart every pod during a database incident, turning a degradation into a full outage.
2. Readiness — can it serve traffic now? May check the database and required dependencies, and should fail during startup and graceful shutdown.

---

#Anti-patterns

---

#Checklist

• Latency, traffic, errors and saturation are instrumented for every service
• Latency is a histogram with buckets chosen for the SLO
• Metric labels are bounded; no ids or raw paths as labels
• Business-level metrics exist alongside infrastructure metrics
• Dependency call rate, errors and latency are tracked per dependency
• Queue depth and oldest-message age are monitored
• OpenTelemetry traces cover every service, with context propagated
• High-cardinality identifiers are span attributes, not metric labels
• Sampling keeps all errors and slow requests
• Logs carry the trace_id
• An SLO with an explicit error budget is defined for each critical journey
• Alerts fire on multi-window burn rate, not static thresholds
• An alert fires on unexpected absence of traffic
• Every paging alert is user-affecting, actionable, urgent and has a runbook
• Alerts are reviewed monthly and unused ones deleted
• Liveness and readiness probes are separate; liveness checks no dependencies