mohsen_hassani/LedgerLine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add dashboards and alerts as code, mark Phase 6 done

Browse Source 
Dashboards: observability/dashboards/generate.py (Grafana Foundation SDK)
writes k8s/grafana/dashboards/*.json -- a RED dashboard per HTTP service
and a system overview -- shipped through a file provider and
configMapGenerators, so `kubectl apply -k k8s/` rebuilds them. Verified by
deleting Grafana and its volume and re-applying. Adds tracesToMetrics to
the Tempo datasource.

Alerts: Prometheus rule files delivering to Alertmanager, under a
labels/annotations contract enforced by observability/alerts/check_contract.py,
with runbooks in docs/runbooks/.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Movses Aslanyan
2026-10-25 20:01:52 +03:30
co-authored by Claude Sonnet 5.5
parent b50f8c3b24
commit fef735c922
19 changed files with 1314 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/roadmap/summary.md
+4 -2
View File
@@ -21,7 +21,7 @@ warnings), **Time-box**, **Concepts**, **Resources**, **Build**, **Done when**.
| 3 | [`phase-3.md`](phase-3.md) | Done | Distributed tracing | 2–3 evenings | Connected trace across 7 services incl. the async hop |
| 4 | [`phase-4.md`](phase-4.md) | Done | Structured logs | 1–2 evenings | Two-way trace↔log correlation |
| 5 | [`phase-5.md`](phase-5.md) | Done | Metrics | 2 evenings | RED per service, service map, exemplars |
| 6 | [`phase-6.md`](phase-6.md) | Not started | Dashboards as code | 2 evenings | `kubectl apply -k k8s/` restores all dashboards |
| 6 | [`phase-6.md`](phase-6.md) | Done | Dashboards as code | 2 evenings | `kubectl apply -k k8s/` restores all dashboards |
| 7 | [`phase-7.md`](phase-7.md) | Not started | k6 + percentiles | 1–2 evenings | Baseline vs degraded load profiles |
| 8 | [`phase-8.md`](phase-8.md) | Not started | SLO / error budget / RTO / RPO | 1–2 evenings | `docs/SLO.md` + burn-rate alerts |
| 9 | [`phase-9.md`](phase-9.md) | Not started | Bottleneck hunting | 2 evenings | `docs/INVESTIGATION.md` |
@@ -56,7 +56,9 @@ warnings), **Time-box**, **Concepts**, **Resources**, **Build**, **Done when**.
`phase-5.md` (second metrics path and the TraceQL test migration not done).
- **6 — Dashboards and alerts as code.** Generate dashboard JSON with the
Grafana Foundation SDK and ship it with the platform (`k8s/grafana/`), so
"destroy and recreate in seconds" is a real, repeatable command.
"destroy and recreate in seconds" is a real, repeatable command. **Done** —
see `phase-6.md` (alerts as Prometheus rules under a contract; the
`tracesToLogsV2` question and a real alert firing not done).
- **7 — Load testing with k6, and percentiles.** Learn why percentiles can't
be averaged, what coordinated omission is, and drive Ledgerline with
arrival-rate executors to get a clean before/after load profile.