Record swallowed exceptions on their spans

Where an error is caught and logged under a span that would otherwise
stay clean, also record_exception and set ERROR on that span: the
CONSUMER span when a stream message fails, fraud.score_transaction when
the sanctions call times out or fails, and the SERVER span of every
/readyz check (a 503 already made it ERROR, but without the exception).

Left alone where instrumentation already records it: ledger's app-wide
exception handler (FastAPI SERVER span), caught httpx errors (CLIENT
span), and the stream publish failure (PRODUCER span). The consumers'
failed reads run outside any span, so they stay log-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Movses Aslanyan
2026-10-13 19:48:27 +03:30
co-authored by Claude Opus 5.5
parent d52a19965a
commit 1ab425dc44
7 changed files with 37 additions and 7 deletions
+4
View File
@@ -2,6 +2,7 @@ from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel, field_validator
from opentelemetry import trace
from opentelemetry.trace import StatusCode
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor
@@ -75,6 +76,9 @@ async def readyz():
async with normal_engine.connect() as conn:
await conn.exec_driver_sql("SELECT 1")
except Exception as exc:
span = trace.get_current_span()
span.record_exception(exc)
span.set_status(StatusCode.ERROR, str(exc))
logger.exception(f"readyz check failed: {exc}")
return JSONResponse(status_code=503, content={"status": "unavailable"})
return {"status": "ok"}