Movses AslanyanandClaude Opus 5.5
7719ef5fcf
k8s/ is now a faithful port of the mentor's Ansible Grafana stack (Prometheus, Alertmanager, Loki, Tempo, Alloy, Grafana, node-exporter) for the ledgerline-observability namespace on falken: same versions and configs, one *.vione.cloud ingress per tool behind Traefik basic auth, secrets generated by create-secrets.sh into gitignored files. Two fixes over the Ansible role, both verified live: Tempo's metrics-generator needs traces_storage or it runs no processors, and Prometheus needs exemplar-storage to keep exemplars. Ledgerline stays on czy; an Alloy agent in the Traefik override ships its container logs to Loki. Roadmap and CLAUDE.md now point at k8s/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5.2 KiB
5.2 KiB
Roadmap summary — all phases at a glance
Part of docs/roadmap/ — the one-page overview. Read intro.md
first for platform/background context; read the individual phase-N.md file
for the phase you're actually working on. This file exists so an agent (or
you) can check overall status and sequencing without loading every phase's
full detail.
Twelve phases (0 through 11 — Phase 0 has two halves, 0a and 0b). Each phase file follows the same shape: Status, Message (what's done, notes, warnings), Time-box, Concepts, Resources, Build, Done when.
Status and focus
| Phase | File | Status | Focus | Time-box | Main output |
|---|---|---|---|---|---|
| 0a | phase-0.md |
Done | Integration test suite (agent, tests only) | 1 evening | Red suite + RED-BASELINE.md |
| 0b | phase-0.md |
Done | Base project implemented to green (second agent) | 1–2 evenings | Ledgerline running, list of frustrations |
| 1 | phase-1.md |
Done | Deployment: Compose + Traefik on a server | 1 evening | https://ledgerline.vione.cloud |
| 2 | phase-2.md |
Not started | Concepts + first trace | 1 evening | One trace in Tempo |
| 3 | phase-3.md |
Not started | Distributed tracing | 2–3 evenings | Connected trace across 7 services incl. the async hop |
| 4 | phase-4.md |
Not started | Structured logs | 1–2 evenings | Two-way trace↔log correlation |
| 5 | phase-5.md |
Not started | Metrics | 2 evenings | RED per service, service map, exemplars |
| 6 | phase-6.md |
Not started | Dashboards as code | 2 evenings | kubectl apply -k k8s/ restores all dashboards |
| 7 | phase-7.md |
Not started | k6 + percentiles | 1–2 evenings | Baseline vs degraded load profiles |
| 8 | phase-8.md |
Not started | SLO / error budget / RTO / RPO | 1–2 evenings | docs/SLO.md + burn-rate alerts |
| 9 | phase-9.md |
Not started | Bottleneck hunting | 2 evenings | docs/INVESTIGATION.md |
| 10 | phase-10.md |
Not started | Filebeat / Logstash / OTel judgment | 1–2 evenings | docs/PIPELINE-CHOICES.md |
| 11 | phase-11.md |
Not started | Delivery | 1 evening | README + rehearsed demo |
What each phase is, in one line
- 0 — Build the base project, test-first. Write the black-box test suite,
then implement the seven-service system against it as a frozen contract.
Done — see
phase-0.mdfor the full implementation report. - 1 — Deployment. Put Ledgerline on a server with Docker Compose behind
Traefik and a Let's Encrypt certificate, served at a real domain. Plumbing
only — no telemetry, no app changes. Done — see
phase-1.md. - 2 — The model, and one signal end to end. Learn the OTel API/SDK/Collector
split and the M.E.L.T. vocabulary; deploy the real Grafana-stack platform
(the mentor's Ansible stack, ported to k3s as
k8s/— already deployed onfalken) and get one auto-instrumented trace into Tempo. - 3 — Distributed tracing. The heart of the task: auto- and manual-instrument all seven services, and hand-propagate trace context across the Redis Stream hop, which carries none by default.
- 4 — Structured logs, correlated to traces. Replace prose logging with
structured events carrying a lowercase
trace_id, so Loki and Tempo link in both directions. - 5 — Metrics. Instrument RED metrics per service, understand cardinality limits, and route app metrics to Prometheus given Alloy drops OTLP metrics silently. Tempo's metrics-generator already produces span metrics and the service graph — don't duplicate it in the collector.
- 6 — Dashboards and alerts as code. Generate dashboard JSON with the
Grafana Foundation SDK and ship it with the platform (
k8s/grafana/), so "destroy and recreate in seconds" is a real, repeatable command. - 7 — Load testing with k6, and percentiles. Learn why percentiles can't be averaged, what coordinated omission is, and drive Ledgerline with arrival-rate executors to get a clean before/after load profile.
- 8 — SLI, SLO, SLA, error budgets, RTO, RPO. Define Ledgerline's SLIs and SLOs from the Phase 7 baseline (not from thin air), write an error-budget policy, and wire multi-burn-rate alerts.
- 9 — Bottleneck hunting (the capstone). Given a random, undisclosed set of chaos flags, find the pathologies from telemetry alone using a top-down, hypothesis-driven method, and write it up as an incident doc.
- 10 — Tool-selection judgment. Reason through Filebeat vs Logstash vs OpenTelemetry on six concrete axes, using the platform's real second stack (Filebeat → Logstash → Elasticsearch → Kibana) as the counterpoint.
- 11 — Package the delivery. README, a rehearsed five-minute demo, the
written deliverables (
SLO.md,PIPELINE-CHOICES.md,INVESTIGATION.md,CONCEPTS.md), and an honest "what I'd do differently in production" section.
If you have to cut scope
Cut Phase 10's build (keep the writing) and Phase 7's soak test first. Do not cut Phase 9 — it's the only phase that proves you learned anything.