Port the observability platform to k3s and ship czy logs to it
k8s/ is now a faithful port of the mentor's Ansible Grafana stack (Prometheus, Alertmanager, Loki, Tempo, Alloy, Grafana, node-exporter) for the ledgerline-observability namespace on falken: same versions and configs, one *.vione.cloud ingress per tool behind Traefik basic auth, secrets generated by create-secrets.sh into gitignored files. Two fixes over the Ansible role, both verified live: Tempo's metrics-generator needs traces_storage or it runs no processors, and Prometheus needs exemplar-storage to keep exemplars. Ledgerline stays on czy; an Alloy agent in the Traefik override ships its container logs to Loki. Roadmap and CLAUDE.md now point at k8s/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
34a11da848
commit
7719ef5fcf
@@ -21,7 +21,7 @@ warnings), **Time-box**, **Concepts**, **Resources**, **Build**, **Done when**.
|
||||
| 3 | [`phase-3.md`](phase-3.md) | Not started | Distributed tracing | 2–3 evenings | Connected trace across 7 services incl. the async hop |
|
||||
| 4 | [`phase-4.md`](phase-4.md) | Not started | Structured logs | 1–2 evenings | Two-way trace↔log correlation |
|
||||
| 5 | [`phase-5.md`](phase-5.md) | Not started | Metrics | 2 evenings | RED per service, service map, exemplars |
|
||||
| 6 | [`phase-6.md`](phase-6.md) | Not started | Dashboards as code | 2 evenings | `make obs-reset` |
|
||||
| 6 | [`phase-6.md`](phase-6.md) | Not started | Dashboards as code | 2 evenings | `kubectl apply -k k8s/` restores all dashboards |
|
||||
| 7 | [`phase-7.md`](phase-7.md) | Not started | k6 + percentiles | 1–2 evenings | Baseline vs degraded load profiles |
|
||||
| 8 | [`phase-8.md`](phase-8.md) | Not started | SLO / error budget / RTO / RPO | 1–2 evenings | `docs/SLO.md` + burn-rate alerts |
|
||||
| 9 | [`phase-9.md`](phase-9.md) | Not started | Bottleneck hunting | 2 evenings | `docs/INVESTIGATION.md` |
|
||||
@@ -37,8 +37,9 @@ warnings), **Time-box**, **Concepts**, **Resources**, **Build**, **Done when**.
|
||||
Traefik and a Let's Encrypt certificate, served at a real domain. Plumbing
|
||||
only — no telemetry, no app changes. **Done** — see `phase-1.md`.
|
||||
- **2 — The model, and one signal end to end.** Learn the OTel API/SDK/Collector
|
||||
split and the M.E.L.T. vocabulary; deploy the real Grafana-stack platform via
|
||||
Ansible and get one auto-instrumented trace into Tempo.
|
||||
split and the M.E.L.T. vocabulary; deploy the real Grafana-stack platform
|
||||
(the mentor's Ansible stack, ported to k3s as `k8s/` — already deployed on
|
||||
`falken`) and get one auto-instrumented trace into Tempo.
|
||||
- **3 — Distributed tracing.** The heart of the task: auto- and
|
||||
manual-instrument all seven services, and hand-propagate trace context
|
||||
across the Redis Stream hop, which carries none by default.
|
||||
@@ -50,8 +51,8 @@ warnings), **Time-box**, **Concepts**, **Resources**, **Build**, **Done when**.
|
||||
silently. Tempo's metrics-generator already produces span metrics and the
|
||||
service graph — don't duplicate it in the collector.
|
||||
- **6 — Dashboards and alerts as code.** Generate dashboard JSON with the
|
||||
Grafana Foundation SDK and ship it through the Ansible role, so "destroy
|
||||
and recreate in seconds" is a real, repeatable command.
|
||||
Grafana Foundation SDK and ship it with the platform (`k8s/grafana/`), so
|
||||
"destroy and recreate in seconds" is a real, repeatable command.
|
||||
- **7 — Load testing with k6, and percentiles.** Learn why percentiles can't
|
||||
be averaged, what coordinated omission is, and drive Ledgerline with
|
||||
arrival-rate executors to get a clean before/after load profile.
|
||||
|
||||
Reference in New Issue
Block a user