mohsen_hassani/LedgerLine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add deployment phase and Traefik compose override

Browse Source 
Insert a new Phase 1 (deployment: Compose + Traefik on a server, marked
Done) between the base project and the OTel concepts phase, and shift
the old phases 1-10 to 2-11. Update summary, intro, README and CLAUDE.md
references (spoiler rules now target Phase 9, the bottleneck hunt).

Add docker-compose.traefik.yml: an override that fronts the gateway with
Traefik and Let's Encrypt and removes all other published host ports.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Movses Aslanyan
2026-10-09 15:38:29 +03:30
co-authored by Claude Sonnet 5.5
parent b2112e55e1
commit 5e514ecf25
17 changed files with 1080 additions and 937 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/roadmap/intro.md
+39 -38
View File
@@ -1,7 +1,7 @@
# Observability & APM — Delivery Roadmap for Babak's Task
*Part of `docs/roadmap/` — platform background and reading guidance shared by
every phase. Per-phase detail lives in `phase-0.md` through `phase-10.md`;
every phase. Per-phase detail lives in `phase-0.md` through `phase-11.md`;
[`summary.md`](summary.md) gives the one-page overview and status of all of
them.*
@@ -90,16 +90,16 @@ learn OTel first, every backend becomes "point OTLP at a different URL."
One exception: learn **Prometheus's data model** (labels, series, cardinality) fairly
early, in parallel with OTel metrics — because it *constrains what you're allowed to
emit*. PromQL, LogQL and TraceQL are query languages; learn them when you need them
(Phases 4–5), not up front.
(Phases 5–6), not up front.
Filebeat and Logstash are a **different world entirely** (Elastic's). They are not
part of the Grafana stack and you do not need them for this task. Phase 9 covers
part of the Grafana stack and you do not need them for this task. Phase 10 covers
them as a judgment question, which is exactly how Babak framed them.
### Verdict on the OneUptime article you found
**Yes, it's valuable — but not yet.** It's a *configuration cheat-sheet*, not a
teaching resource. Read it in **Phase 4/5**, not now; it assumes you already
teaching resource. Read it in **Phase 5/6**, not now; it assumes you already
understand spans, span metrics and PromQL.
What's genuinely good in it, and rare to find in one place:
@@ -133,7 +133,7 @@ Two caveats:
This changes *where* your telemetry goes and *how the pipeline is configured*. It
does not change the learning arc, the app, or the test suite. Read this before
Phase 1; it rewrites parts of Phases 1, 3, 4, 5, 7 and 9.
Phase 2; it rewrites parts of Phases 2, 4, 5, 6, 8 and 10.
### What the repo is
@@ -199,7 +199,7 @@ Prometheus`. The shipped config does not do that:
Neither is wrong; the diagram shows the intended architecture and the code
implements a simpler one. Closing that gap — adding `prometheus.scrape` +
`prometheus.remote_write` to Alloy so metrics really do flow through the collector —
is a genuinely good Phase 4 exercise, and **noticing the discrepancy at all is worth
is a genuinely good Phase 5 exercise, and **noticing the discrepancy at all is worth
mentioning to Babak.** Spotting that a diagram and its implementation have drifted is
exactly the skill this task is meant to build.
@@ -264,7 +264,7 @@ Tempo and remote-written into Prometheus (which runs with
connectors to Alloy** — you'd be generating the same series twice under different
names. The metric names differ from the OneUptime article's; go find the actual
names in Prometheus before writing any PromQL. (`local-blocks` also gives you
TraceQL metrics queries in Tempo — a bonus worth exploring in Phase 8.)
TraceQL metrics queries in Tempo — a bonus worth exploring in Phase 9.)
**3. Log→trace correlation depends on an exact, case-sensitive regex.** The Loki
datasource is provisioned with:
@@ -295,16 +295,17 @@ your deliverable, and the natural home for it is *inside this Ansible role*.
| Phase | Impact |
|---|---|
| 0 / 0a | **None.** App and test suite are unaffected. |
| 1 | Replaced. Deploy the real stack with Ansible instead of running a single container. |
| 2 | Endpoint changes to Alloy; tracing work itself is identical. |
| 3 | Rewritten. OTLP logs go nowhere; stdout scraping is the path. Lowercase `trace_id` is mandatory. |
| 4 | Rewritten. Tempo generates span metrics; app metrics need a scrape endpoint or an Alloy extension. |
| 5 | **Upgraded.** Dashboards-as-code becomes an Ansible deliverable — a better answer than a Makefile target. |
| 6 | None. |
| 7 | Choice added: Prometheus rules + Alertmanager (the platform's grain) vs Grafana unified alerting. |
| 8 | None, plus TraceQL metrics from `local-blocks`. |
| 9 | **Massively upgraded.** You now have a real Filebeat → Logstash → Elasticsearch pipeline to run, not a thought experiment. |
| 10 | Demo now includes `ansible-playbook` rebuilding the platform. |
| 1 | **New.** Deploy Ledgerline itself to a server with Docker Compose behind Traefik. Plumbing only — no telemetry, no app changes. |
| 2 | Replaced. Deploy the real stack with Ansible instead of running a single container. |
| 3 | Endpoint changes to Alloy; tracing work itself is identical. |
| 4 | Rewritten. OTLP logs go nowhere; stdout scraping is the path. Lowercase `trace_id` is mandatory. |
| 5 | Rewritten. Tempo generates span metrics; app metrics need a scrape endpoint or an Alloy extension. |
| 6 | **Upgraded.** Dashboards-as-code becomes an Ansible deliverable — a better answer than a Makefile target. |
| 7 | None. |
| 8 | Choice added: Prometheus rules + Alertmanager (the platform's grain) vs Grafana unified alerting. |
| 9 | None, plus TraceQL metrics from `local-blocks`. |
| 10 | **Massively upgraded.** You now have a real Filebeat → Logstash → Elasticsearch pipeline to run, not a thought experiment. |
| 11 | Demo now includes `ansible-playbook` rebuilding the platform. |
### Where to run it
@@ -326,7 +327,7 @@ Is this repo a **platform you must consume as-is**, or a **starting point you ma
modify**? It's in your own Gitea and nothing marks it read-only, so this roadmap
assumes you own it and will edit `config.alloy.j2`, add a dashboards provider, and
extend `group_vars`. If Babak intends it as a fixed platform, several
recommendations below change — ask before Phase 3.
recommendations below change — ask before Phase 4.
---
@@ -340,8 +341,8 @@ subtitle-to-notes workflow.
| Resource | Format | Length | Cost | Best for |
|---|---|---|---|---|
| *Observability with Grafana, Prometheus, Loki, Alloy and Tempo* (Aref Karimi, Udemy) | Video | ~10 h, 132 lectures | Paid (often ~€15 on sale) | The **Grafana-stack backbone**. Long-running bestseller, updated Jan 2026, covers Prometheus, PromQL, Grafana dashboards, Loki, Tempo, Alloy, Mimir, and OTel. Best fit for Phases 3–5. Comes with browser-based Killercoda playgrounds. |
| *OpenTelemetry for Observability: The Complete Course* (Udemy) | Video | ~7.5 h, 113 lectures | Paid | The **OTel backbone**. Instrumentation → context propagation → Prometheus/Loki/Tempo/Grafana → Kubernetes. Closest single course to Babak's exact task. Best fit for Phases 2–4. |
| *Observability with Grafana, Prometheus, Loki, Alloy and Tempo* (Aref Karimi, Udemy) | Video | ~10 h, 132 lectures | Paid (often ~€15 on sale) | The **Grafana-stack backbone**. Long-running bestseller, updated Jan 2026, covers Prometheus, PromQL, Grafana dashboards, Loki, Tempo, Alloy, Mimir, and OTel. Best fit for Phases 4–6. Comes with browser-based Killercoda playgrounds. |
| *OpenTelemetry for Observability: The Complete Course* (Udemy) | Video | ~7.5 h, 113 lectures | Paid | The **OTel backbone**. Instrumentation → context propagation → Prometheus/Loki/Tempo/Grafana → Kubernetes. Closest single course to Babak's exact task. Best fit for Phases 3–5. |
If budget allows, buy both — they overlap by maybe 30% and the second half of each
is different. If only one: take the **OpenTelemetry** one, because instrumentation
@@ -351,27 +352,27 @@ is the part you have to *write*, and Grafana's own docs are excellent for the re
| Resource | Format | Length | Use in |
|---|---|---|---|
| freeCodeCamp — *OpenTelemetry Course* (Ania Kubów) | YouTube | ~1 h | Phase 1. Concepts, M.E.L.T., context propagation. **Note: the code is Node.js**, not Python — watch it for the model, not the syntax. |
| Jessica Garson — *Introduction to OpenTelemetry with Python* (PyTexas 2025) | YouTube talk | ~30 min | Phase 2. The best Python-specific overview, with a companion repo. |
| Jessica Garson — *Introduction to OpenTelemetry with Django* (DjangoCon US 2024) | YouTube talk | ~30 min | Phase 2. Auto- vs manual-instrumentation side by side. |
| freeCodeCamp — *OpenTelemetry Course* (Ania Kubów) | YouTube | ~1 h | Phase 2. Concepts, M.E.L.T., context propagation. **Note: the code is Node.js**, not Python — watch it for the model, not the syntax. |
| Jessica Garson — *Introduction to OpenTelemetry with Python* (PyTexas 2025) | YouTube talk | ~30 min | Phase 3. The best Python-specific overview, with a companion repo. |
| Jessica Garson — *Introduction to OpenTelemetry with Django* (DjangoCon US 2024) | YouTube talk | ~30 min | Phase 3. Auto- vs manual-instrumentation side by side. |
| **OpenTelemetry Demo ("Astronomy Shop")** — `github.com/open-telemetry/opentelemetry-demo`, and Grafana's fork `grafana/lgtm-otel-demo` | Running code | — | **Every phase.** This is your "how professionals do it" reference. ~25 containers, polyglot, real auto+manual instrumentation, a Collector config you can steal from, and a feature-flag service for injecting failures. Run it once early, then keep it open in a second terminal as a lookup. |
| `grafana/docker-otel-lgtm` Docker image | Tool | — | Phase 1. Grafana + Prometheus + Tempo + Loki + Pyroscope + Collector in **one container** with OTel defaults pre-wired. Removes all backend setup friction so you can focus on instrumentation. |
| Official OpenTelemetry docs — Concepts, Python instrumentation, Semantic Conventions, Sampling | Docs | — | Phases 1–4. The neutral source of truth. |
| Grafana docs — *Observability as code* / Foundation SDK; k6 docs | Docs | — | Phases 5–6 |
| **Grafana Alloy docs** — components reference, especially `otelcol.receiver.otlp`, `otelcol.exporter.*`, `loki.source.docker`, `prometheus.remote_write` | Docs | — | Phases 1–4. **Required** — your platform uses Alloy, not the vanilla Collector, and its config language is component-based, not YAML pipelines. |
| **Grafana Tempo docs** — *Metrics-generator* and *TraceQL* | Docs | — | Phases 4, 8. This is where your RED metrics and service graph actually come from. |
| Elastic docs — Filebeat inputs/modules, Logstash grok filter | Docs | — | Phase 9, now hands-on |
| Google SRE Workbook — *Implementing SLOs*, *Alerting on SLOs*, *Error Budget Policy* | Free book chapters | ~2 h total | Phase 7 |
| Google SRE — *The Art of SLOs* workshop (slides + participant & facilitator handbooks, all public PDFs) | Slides/PDF + exercises | ~2 h | Phase 7. Self-run the exercises with Ledgerline as the subject service. |
| Coursera — *Measuring and Managing Reliability* (Google Cloud) | Video course | ~1 week | Phase 7, optional. Self-paced SLI/SLO/error-budget deep dive; audit for free. |
| Brendan Gregg — the **USE method**; Tom Wilkie — the **RED method** | Articles | ~20 min each | Phase 8 |
| Cloudflare — *How Cloudflare runs Prometheus at scale* | Article | ~20 min | Phase 4. The clearest explanation anywhere of what cardinality actually costs. |
| SigNoz — *OpenTelemetry FastAPI Instrumentation*; Uptrace — *OpenTelemetry FastAPI* | Articles | ~20 min each | Phase 2 |
| Better Stack — *Filebeat vs Logstash*; SigNoz — *OpenTelemetry vs ELK* | Articles | ~30 min | Phase 9 |
| `grafana/docker-otel-lgtm` Docker image | Tool | — | Phase 2. Grafana + Prometheus + Tempo + Loki + Pyroscope + Collector in **one container** with OTel defaults pre-wired. Removes all backend setup friction so you can focus on instrumentation. |
| Official OpenTelemetry docs — Concepts, Python instrumentation, Semantic Conventions, Sampling | Docs | — | Phases 2–5. The neutral source of truth. |
| Grafana docs — *Observability as code* / Foundation SDK; k6 docs | Docs | — | Phases 6–7 |
| **Grafana Alloy docs** — components reference, especially `otelcol.receiver.otlp`, `otelcol.exporter.*`, `loki.source.docker`, `prometheus.remote_write` | Docs | — | Phases 2–5. **Required** — your platform uses Alloy, not the vanilla Collector, and its config language is component-based, not YAML pipelines. |
| **Grafana Tempo docs** — *Metrics-generator* and *TraceQL* | Docs | — | Phases 5, 9. This is where your RED metrics and service graph actually come from. |
| Elastic docs — Filebeat inputs/modules, Logstash grok filter | Docs | — | Phase 10, now hands-on |
| Google SRE Workbook — *Implementing SLOs*, *Alerting on SLOs*, *Error Budget Policy* | Free book chapters | ~2 h total | Phase 8 |
| Google SRE — *The Art of SLOs* workshop (slides + participant & facilitator handbooks, all public PDFs) | Slides/PDF + exercises | ~2 h | Phase 8. Self-run the exercises with Ledgerline as the subject service. |
| Coursera — *Measuring and Managing Reliability* (Google Cloud) | Video course | ~1 week | Phase 8, optional. Self-paced SLI/SLO/error-budget deep dive; audit for free. |
| Brendan Gregg — the **USE method**; Tom Wilkie — the **RED method** | Articles | ~20 min each | Phase 9 |
| Cloudflare — *How Cloudflare runs Prometheus at scale* | Article | ~20 min | Phase 5. The clearest explanation anywhere of what cardinality actually costs. |
| SigNoz — *OpenTelemetry FastAPI Instrumentation*; Uptrace — *OpenTelemetry FastAPI* | Articles | ~20 min each | Phase 3 |
| Better Stack — *Filebeat vs Logstash*; SigNoz — *OpenTelemetry vs ELK* | Articles | ~30 min | Phase 10 |
**Where to run it:** Docker Compose on your laptop for Phases 0–8. Don't put this on
**Where to run it:** Docker Compose on your laptop for Phases 0–9. Don't put this on
your k8s cluster until everything works locally — debugging instrumentation and
debugging Kubernetes at the same time is how these projects die. Porting it to the
cluster is a good Phase 10+ follow-up, and it's the natural bridge to the DevOps half
cluster is a good Phase 11+ follow-up, and it's the natural bridge to the DevOps half
Babak has planned for you.