Mark Phase 5 done

RED per service, the service map and exemplar click-through are verified
live. Build step 3's second metrics path and step 6's TraceQL test
migration are not done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/roadmap/phase-5.md

@@ -2,9 +2,51 @@
 
 *Part of `docs/roadmap/` — see [`intro.md`](intro.md) for platform/background context and [`summary.md`](summary.md) for how every phase fits together. Prev: [`phase-4.md`](phase-4.md) · Next: [`phase-6.md`](phase-6.md)*
 
-**Status:** In progress
+**Status:** Done
 
-**Message:** Started 2026-10-13.
+**Message:** Done 2026-10-25. All three Done-when criteria verified live: RED per
+service and the service map (7 edges, including the two stream edges
+gateway → notifier and gateway → audit) from Tempo's span metrics, and a Grafana
+exemplar click that opens the trace.
+
+- **Build steps done:** 1, 2, 3 (first path only), 4 and 5. Not done: 3's second
+  path and 6 (see below).
+- **Metrics path:** each service exposes OTel metrics through
+  `PrometheusMetricReader` on its own `:9464` (`common/telemetry.py`), not an OTLP
+  exporter, because Alloy 404s OTLP metrics. Prometheus's `ledgerline` job scrapes
+  them through czy's Traefik at `https://ledgerline.vione.cloud/metrics/<service>`
+  behind basic auth.
+- **Custom metrics:** the five from Build step 2, in `services/*/app.py` and
+  `common/metrics.py`. `ledgerline.db.pool.in_use` is labelled `pool`
+  (`normal`/`starved`/`default`), not `service`; the scrape adds `service`.
+  `currency` outside EUR/USD/GBP is recorded as `other`.
+- **Cardinality experiment (Build step 4, done):** `account_id` on the transaction counter turned 75
+  transactions into 68 series across 60 accounts, and `account_id` entered the
+  TSDB Status top-10. After the revert (`d54ea2b`) the series went stale, but head
+  series stayed up until compaction.
+- **Hand-written vs span-metrics histogram** (same 75 requests): counts equal,
+  `_sum` 400.85 s vs 401.39 s (the span adds ~7 ms per request of middleware and
+  serialization). p95 was 9.5 s vs 14.6 s. The gap comes only from Tempo's wide
+  8.192–16.384 s bucket.
+- **Exemplar fix:** `trace_id_label_name: trace_id` in `k8s/tempo/tempo.yml`.
+  Tempo's code default is `traceID`; its docs wrongly say `trace_id`.
+- **Not done:** Build step 3's second metrics path and its write-up (a Phase 10
+  artefact), and Build step 6 (migrating the call-counting tests to TraceQL).
+- **Carry forward:**
+  - RED must count `SPAN_KIND_SERVER|SPAN_KIND_CONSUMER`. Notifier and audit only
+    have CONSUMER spans.
+  - An error-ratio query needs `(errors or total * 0) / total`, or services with
+    zero errors drop out.
+  - For gateway's latency tail, use `ledgerline_transaction_duration_seconds`.
+    The span-metrics p95/p99 there is an interpolation artifact.
+  - The Python SDK has no cardinality limit, and the Python Prometheus exporter
+    emits no exemplars, so only Tempo's span metrics carry them.
+  - Built-in HTTP histograms are most of the app's series. The body-size ones
+    (~⅓) can be dropped with a View or `metric_relabel_configs` if needed.
+  - Exemplars stored before 2026-10-24 19:10 UTC are labelled `traceID` and don't
+    link. Exemplar trace IDs drop leading zeros.
+  - Tempo stops sending idle service-graph and span-metric series after a few
+    minutes, so "now" queries come back empty without traffic.
 
 **Time-box:** 2 evenings.
 

docs/roadmap/summary.md

@@ -20,7 +20,7 @@ warnings), **Time-box**, **Concepts**, **Resources**, **Build**, **Done when**.
 | 2 | [`phase-2.md`](phase-2.md) | Done | Concepts + first trace | 1 evening | One trace in Tempo |
 | 3 | [`phase-3.md`](phase-3.md) | Done | Distributed tracing | 2–3 evenings | Connected trace across 7 services incl. the async hop |
 | 4 | [`phase-4.md`](phase-4.md) | Done | Structured logs | 1–2 evenings | Two-way trace↔log correlation |
-| 5 | [`phase-5.md`](phase-5.md) | In progress | Metrics | 2 evenings | RED per service, service map, exemplars |
+| 5 | [`phase-5.md`](phase-5.md) | Done | Metrics | 2 evenings | RED per service, service map, exemplars |
 | 6 | [`phase-6.md`](phase-6.md) | Not started | Dashboards as code | 2 evenings | `kubectl apply -k k8s/` restores all dashboards |
 | 7 | [`phase-7.md`](phase-7.md) | Not started | k6 + percentiles | 1–2 evenings | Baseline vs degraded load profiles |
 | 8 | [`phase-8.md`](phase-8.md) | Not started | SLO / error budget / RTO / RPO | 1–2 evenings | `docs/SLO.md` + burn-rate alerts |
@@ -52,7 +52,8 @@ warnings), **Time-box**, **Concepts**, **Resources**, **Build**, **Done when**.
 - **5 — Metrics.** Instrument RED metrics per service, understand cardinality
   limits, and route app metrics to Prometheus given Alloy rejects OTLP metrics
   (404 on `/v1/metrics`). Tempo's metrics-generator already produces span metrics and the
-  service graph — don't duplicate it in the collector.
+  service graph — don't duplicate it in the collector. **Done** — see
+  `phase-5.md` (second metrics path and the TraceQL test migration not done).
 - **6 — Dashboards and alerts as code.** Generate dashboard JSON with the
   Grafana Foundation SDK and ship it with the platform (`k8s/grafana/`), so
   "destroy and recreate in seconds" is a real, repeatable command.