Record swallowed exceptions on their spans

Where an error is caught and logged under a span that would otherwise
stay clean, also record_exception and set ERROR on that span: the
CONSUMER span when a stream message fails, fraud.score_transaction when
the sanctions call times out or fails, and the SERVER span of every
/readyz check (a 503 already made it ERROR, but without the exception).

Left alone where instrumentation already records it: ledger's app-wide
exception handler (FastAPI SERVER span), caught httpx errors (CLIENT
span), and the stream publish failure (PRODUCER span). The consumers'
failed reads run outside any span, so they stay log-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Movses Aslanyan
2026-10-13 19:48:27 +03:30
co-authored by Claude Opus 5.5
parent d52a19965a
commit 1ab425dc44
7 changed files with 37 additions and 7 deletions
+5
View File
@@ -4,8 +4,10 @@ import asyncio
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from opentelemetry import trace
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor
from opentelemetry.trace import StatusCode
from common.db import engine
from common.logging_config import configure_logging
@@ -55,6 +57,9 @@ async def readyz():
await conn.exec_driver_sql("SELECT 1")
await redis_client.ping()
except Exception as exc:
span = trace.get_current_span()
span.record_exception(exc)
span.set_status(StatusCode.ERROR, str(exc))
logger.exception(f"readyz check failed: {exc}")
return JSONResponse(status_code=503, content={"status": "unavailable"})
return {"status": "ok"}