Where an error is caught and logged under a span that would otherwise
stay clean, also record_exception and set ERROR on that span: the
CONSUMER span when a stream message fails, fraud.score_transaction when
the sanctions call times out or fails, and the SERVER span of every
/readyz check (a 503 already made it ERROR, but without the exception).
Left alone where instrumentation already records it: ledger's app-wide
exception handler (FastAPI SERVER span), caught httpx errors (CLIENT
span), and the stream publish failure (PRODUCER span). The consumers'
failed reads run outside any span, so they stay log-only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Manual INTERNAL spans for the business steps the auto-instrumentation
can't name: fraud.score_transaction, sanctions.lookup_watchlist,
ledger.settle and insights.aggregate_120d, with ledgerline.* attributes
(account, amount, risk score, match outcome). holder_name stays off the
spans. Each span starts after its handler's chaos flag reads and
injection points, and no chaos-derived values become attributes.
The gateway's /api/health/all fan-out now runs under
suppress_instrumentation: with the route excluded, its five downstream
/healthz calls were each a parentless root trace.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sanctions, notifier, ledger, insights and audit now call setup_telemetry
and instrument their engines with one SQLAlchemyInstrumentor call per
process (a second instrument() call is ignored, leaving that engine with
connect spans only). ledger and insights pass both their normal and
starved engines. The FastAPI services also get the server span, with the
same /healthz and receive/send exclusions as sanctions.
Compose: OTEL_SEMCONV_STABILITY_OPT_IN=http,database for the stable
attribute names, the OTLP endpoint moves to the shared env, and the four
newly instrumented services get the auth header from .env.observability.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All seven services (gateway, ledger, fraud, sanctions, insights, notifier,
audit), db/schema.sql plus optional_indexes.sql, scripts/seed.py and
smoke.py, the gateway-served frontend, and docker-compose.yml. Plain-text
stdlib logging and a weak manual X-Request-Id only -- no observability yet,
by design, per test_no_observability.py. Implements all 13 named chaos-flag
pathologies from docs/PATHOLOGIES.md behind Redis-backed runtime flags.
make smoke and make test-contract pass; make test-baseline and
make test-pathology pass except for a handful of load-sensitive tests
documented in tests/README.md rather than papered over.
Movses AslanyanandClaude Opus 5.5