sanctions, notifier, ledger, insights and audit now call setup_telemetry
and instrument their engines with one SQLAlchemyInstrumentor call per
process (a second instrument() call is ignored, leaving that engine with
connect spans only). ledger and insights pass both their normal and
starved engines. The FastAPI services also get the server span, with the
same /healthz and receive/send exclusions as sanctions.
Compose: OTEL_SEMCONV_STABILITY_OPT_IN=http,database for the stable
attribute names, the OTLP endpoint moves to the shared env, and the four
newly instrumented services get the auth header from .env.observability.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FastAPIInstrumentor.instrument_app gives sanctions its SERVER span back,
skipping /healthz (the Docker healthcheck flooded Tempo search) and the
ASGI receive/send INTERNAL spans. Drop the OTEL_* env vars only the
zero-code configurator read, and the unused SERVICE_NAME.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
common/telemetry.py builds the Resource, TracerProvider, BatchSpanProcessor
and OTLP/HTTP exporter by hand; sanctions calls setup_telemetry() at
import instead of running under opentelemetry-instrument. No instrumentor
is enabled yet, so sanctions emits no spans for now.
service.version comes from OTEL_RESOURCE_ATTRIBUTES: deploy.sh exports
DEPLOY_HASH (the server's HEAD after the pull) and compose falls back to
"dev". deployment.environment.name=prod is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 2 traced sanctions through the zero-code wrapper, so there is no
common/telemetry.py yet; note that build step 1 writes it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First auto-instrumented sanctions trace verified in Tempo. Records what
carries forward: /healthz trace noise, the prod environment label in the
base compose file, and the red G10 guard test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add opentelemetry-distro/exporter-otlp (pinned pair) and run
opentelemetry-bootstrap at build time; wrap the sanctions command with
opentelemetry-instrument and configure the SDK via OTEL_* env (traces
only, OTLP/HTTP to otlp.vione.cloud). create-secrets.sh now writes the
base64 OTLP auth header into .env.observability.
The G10 guard test (no observability libraries in requirements) is
expected to fail from here until the guard suite is retired in Phase 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Alloy only registers an OTLP/HTTP route for a signal with an output, so
POST /v1/metrics and /v1/logs return 404 rather than being accepted and
silently dropped (verified live against otlp.vione.cloud). Fix the claim
everywhere it appeared. Also mark Phase 2 in progress.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
k8s/ is now a faithful port of the mentor's Ansible Grafana stack
(Prometheus, Alertmanager, Loki, Tempo, Alloy, Grafana, node-exporter)
for the ledgerline-observability namespace on falken: same versions and
configs, one *.vione.cloud ingress per tool behind Traefik basic auth,
secrets generated by create-secrets.sh into gitignored files.
Two fixes over the Ansible role, both verified live: Tempo's
metrics-generator needs traces_storage or it runs no processors, and
Prometheus needs exemplar-storage to keep exemplars.
Ledgerline stays on czy; an Alloy agent in the Traefik override ships
its container logs to Loki. Roadmap and CLAUDE.md now point at k8s/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
phase-1: deploy via deploy.sh, correct the container count, and note
that stripped host ports leave no path for the Phase 5 scrape.
phase-2: explain why Grafana is not on the Ledgerline server and why
Cloudflare's proxy never forwards :3000 or the other platform ports.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Wrap Submit and Load burst in one full-width .row and Result, Chaos and
Service health in a second. Each .row splits its width evenly between its
panels (gap subtracted first) and stacks below 900px.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Put Submit and Load burst side by side on the first row at half width,
and Result, Chaos and Service health on the second row at a third each.
Uses a 6-column grid so the gap is accounted for exactly; stacks to one
column below 900px.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Insert a new Phase 1 (deployment: Compose + Traefik on a server, marked
Done) between the base project and the OTel concepts phase, and shift
the old phases 1-10 to 2-11. Update summary, intro, README and CLAUDE.md
references (spoiler rules now target Phase 9, the bottleneck hunt).
Add docker-compose.traefik.yml: an override that fronts the gateway with
Traefik and Let's Encrypt and removes all other published host ports.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Replace the hardcoded "Phase 0a" line with an instruction to read
docs/roadmap/summary.md first, and add a verify-report-confirm rule
for marking a phase Done in the roadmap files.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
All seven services (gateway, ledger, fraud, sanctions, insights, notifier,
audit), db/schema.sql plus optional_indexes.sql, scripts/seed.py and
smoke.py, the gateway-served frontend, and docker-compose.yml. Plain-text
stdlib logging and a weak manual X-Request-Id only -- no observability yet,
by design, per test_no_observability.py. Implements all 13 named chaos-flag
pathologies from docs/PATHOLOGIES.md behind Redis-backed runtime flags.
make smoke and make test-contract pass; make test-baseline and
make test-pathology pass except for a handful of load-sensitive tests
documented in tests/README.md rather than papered over.
81 tests across 11 files plus a 5-module harness (load, pg, streams,
containers, payloads), covering all eleven Phase 0 goals and all 13 named
chaos-flag pathologies. Talks to the system only over HTTP, Postgres, Redis,
and the Docker CLI -- no imports of application code. RED-BASELINE.md records
the suite failing red against the empty repo before any implementation
existed; README.md documents the request-contract fields invented to fill
gaps in the spec, measurement methodology, and known-flaky tests.
This suite is a frozen contract from this point forward per CLAUDE.md.
Deploys the Grafana stack (Prometheus/Alertmanager/Loki/Tempo/Grafana with
Alloy as collector) and the ELK stack (Filebeat/Logstash/Elasticsearch/
Kibana) via Docker Compose on a target host. External to the application
repo, pre-built by the mentor.
Roadmap, phase specs, pathology catalog, and CLAUDE.md establishing
Ledgerline as an observability learning project built on a deliberately
imperfect seven-service payments backend.
