Dashboards: observability/dashboards/generate.py (Grafana Foundation SDK)
writes k8s/grafana/dashboards/*.json -- a RED dashboard per HTTP service
and a system overview -- shipped through a file provider and
configMapGenerators, so `kubectl apply -k k8s/` rebuilds them. Verified by
deleting Grafana and its volume and re-applying. Adds tracesToMetrics to
the Tempo datasource.
Alerts: Prometheus rule files delivering to Alertmanager, under a
labels/annotations contract enforced by observability/alerts/check_contract.py,
with runbooks in docs/runbooks/.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
k8s/ is now a faithful port of the mentor's Ansible Grafana stack
(Prometheus, Alertmanager, Loki, Tempo, Alloy, Grafana, node-exporter)
for the ledgerline-observability namespace on falken: same versions and
configs, one *.vione.cloud ingress per tool behind Traefik basic auth,
secrets generated by create-secrets.sh into gitignored files.
Two fixes over the Ansible role, both verified live: Tempo's
metrics-generator needs traces_storage or it runs no processors, and
Prometheus needs exemplar-storage to keep exemplars.
Ledgerline stays on czy; an Alloy agent in the Traefik override ships
its container logs to Loki. Roadmap and CLAUDE.md now point at k8s/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Movses AslanyanandClaude Sonnet 5.5