Mark Phases 3 and 4 done, and start Phase 5

Phase 4 records what was built, the two skipped Build steps (OTLP-logs
experiment, LogQL test migration) and what carries forward. Phase 3
records its outcome, with the chaos-nightmare and blocking_event_loop
criteria attested by the user.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Movses Aslanyan
2026-10-13 21:32:43 +03:30
co-authored by Claude Opus 5.5
parent 2c18962243
commit 44d7801617
4 changed files with 60 additions and 15 deletions
+37 -3
View File
@@ -2,10 +2,44 @@
*Part of `docs/roadmap/` — see [`intro.md`](intro.md) for platform/background context and [`summary.md`](summary.md) for how every phase fits together. Prev: [`phase-3.md`](phase-3.md) · Next: [`phase-5.md`](phase-5.md)*
**Status:** Not started
**Status:** Done
**Message:** Depends on Phase 3 (trace IDs must exist before log correlation
means anything).
**Message:** Done 2026-10-13. Both directions work in two clicks: a Loki line's
`TraceID` derived field opens the trace, and a Tempo span's logs link returns that
request's lines from every service that logged, the audit/notifier consumers
included (verified live on trace `a4f034b1d66a69b357aef66a647afcef`).
- `common/logging_config.py`: structlog → one JSON object per line on stdout, with
stdlib and uvicorn records going through the same processor chain. Processors
add level, logger, ISO-8601 UTC timestamp, `service`, lowercase `trace_id`/`span_id`
(only inside a valid span) and field-name redaction. `uvicorn.access` is left
alone on purpose, because re-enabling its propagation undoes `--no-access-log`.
- Every call site is now a past-tense event with shared field names (`account_id`,
`transaction_id`, `amount_minor`, `risk_score`, `status_code`, `consumer_group`,
`message_id`, `chaos_flag`). New events: `transaction_failed` (the gateway's 502
path used to log nothing), `transaction_scored`, `holder_screened`,
`stream_message_processed`. `holder_name` is no longer logged at all.
- The czy agent sets `service_name` from the Compose service name. Before that
change, Loki derived it from `container` (`ledgerline-sanctions-1`), so Grafana's
generated `{service_name="sanctions"}` matched nothing. Streams written before
2026-10-13 17:52 UTC keep the old value and don't link.
- **Skipped:** Build step 5 (the OTLP-logs silent-drop experiment; Alloy still has no
`logs` output) and the second Build step 6 (migrating `count_log_lines` to LogQL). The tests run
against the local stack, whose logs never reach Loki, so the migration first
needs a local log path into Loki.
- **Carry forward:**
- The pathology tests still count `docker compose logs` lines matching
`(?i)lookup` (sanctions) and `(?i)healthz` (downstreams). Those words must stay
in exactly `lookup_received` and `healthz_probe_received`.
- `chaos_engaged` lines stay at INFO by choice, so they're visible in Loki during
Phase 9.
- httpx's `HTTP Request:` INFO lines are kept; most of gateway's volume is the
frontend health poll.
- The deploy hot-reloads bind-mounted code into the old image before
`--build`, so a commit that adds a dependency crash-loops the services for
about a minute.
- `alloy-agent`'s config is a single-file bind mount: after `git pull` it needs
`docker compose ... restart alloy-agent` to see the new file.
**Time-box:** 1–2 evenings. The easiest phase, and the biggest quality-of-life win.