Manual INTERNAL spans for the business steps the auto-instrumentation
can't name: fraud.score_transaction, sanctions.lookup_watchlist,
ledger.settle and insights.aggregate_120d, with ledgerline.* attributes
(account, amount, risk score, match outcome). holder_name stays off the
spans. Each span starts after its handler's chaos flag reads and
injection points, and no chaos-derived values become attributes.
The gateway's /api/health/all fan-out now runs under
suppress_instrumentation: with the route excluded, its five downstream
/healthz calls were each a parentless root trace.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Redis Streams have no headers, so the gateway injects traceparent into
the message fields inside a PRODUCER span. The notifier and audit
consumers extract it and open one CONSUMER span per message, so their
DB and XACK spans join the transaction's trace. The idle XREADGROUP poll
runs under suppress_instrumentation (it was a ~2s root trace every loop),
and audit strips the trace fields so its stored payload is unchanged.
The gateway also stops tracing its frontend's /api/health/ polling.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setup_telemetry now instruments httpx and redis process-wide. Both patch
the client classes, so clients created at import time (gateway's
_proxy_client, common.redis_client) are covered too. httpx CLIENT spans
inject traceparent, so the downstream FastAPI SERVER spans join the
caller's trace. gateway and fraud get setup_telemetry, the FastAPI span,
and the OTLP auth header. All seven services now export traces.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sanctions, notifier, ledger, insights and audit now call setup_telemetry
and instrument their engines with one SQLAlchemyInstrumentor call per
process (a second instrument() call is ignored, leaving that engine with
connect spans only). ledger and insights pass both their normal and
starved engines. The FastAPI services also get the server span, with the
same /healthz and receive/send exclusions as sanctions.
Compose: OTEL_SEMCONV_STABILITY_OPT_IN=http,database for the stable
attribute names, the OTLP endpoint moves to the shared env, and the four
newly instrumented services get the auth header from .env.observability.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FastAPIInstrumentor.instrument_app gives sanctions its SERVER span back,
skipping /healthz (the Docker healthcheck flooded Tempo search) and the
ASGI receive/send INTERNAL spans. Drop the OTEL_* env vars only the
zero-code configurator read, and the unused SERVICE_NAME.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
common/telemetry.py builds the Resource, TracerProvider, BatchSpanProcessor
and OTLP/HTTP exporter by hand; sanctions calls setup_telemetry() at
import instead of running under opentelemetry-instrument. No instrumentor
is enabled yet, so sanctions emits no spans for now.
service.version comes from OTEL_RESOURCE_ATTRIBUTES: deploy.sh exports
DEPLOY_HASH (the server's HEAD after the pull) and compose falls back to
"dev". deployment.environment.name=prod is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Wrap Submit and Load burst in one full-width .row and Result, Chaos and
Service health in a second. Each .row splits its width evenly between its
panels (gap subtracted first) and stacks below 900px.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Put Submit and Load burst side by side on the first row at half width,
and Result, Chaos and Service health on the second row at a third each.
Uses a 6-column grid so the gap is accounted for exactly; stacks to one
column below 900px.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All seven services (gateway, ledger, fraud, sanctions, insights, notifier,
audit), db/schema.sql plus optional_indexes.sql, scripts/seed.py and
smoke.py, the gateway-served frontend, and docker-compose.yml. Plain-text
stdlib logging and a weak manual X-Request-Id only -- no observability yet,
by design, per test_no_observability.py. Implements all 13 named chaos-flag
pathologies from docs/PATHOLOGIES.md behind Redis-backed runtime flags.
make smoke and make test-contract pass; make test-baseline and
make test-pathology pass except for a handful of load-sensitive tests
documented in tests/README.md rather than papered over.
Movses AslanyanandClaude Opus 5.5