Add project documentation and Claude instructions

Roadmap, phase specs, pathology catalog, and CLAUDE.md establishing
Ledgerline as an observability learning project built on a deliberately
imperfect seven-service payments backend.

docs/PATHOLOGIES.md

@@ -0,0 +1,156 @@
+# Pathology catalogue
+
+Thirteen deliberate pathologies, toggled at runtime via the `chaos:flags` Redis
+hash (never a restart). Each lives in a `chaos.py` module inside its owning
+service. All flags default off.
+
+> Spoiler note (see root `CLAUDE.md`): this file is a complete answer key for
+> the Phase 8 bottleneck hunt. Don't read it, and don't have it read to you,
+> until that phase is done.
+
+---
+
+## slow_sanctions (`sanctions`)
+
+**Mechanism:** `services/sanctions/chaos.py`. On every `/internal/lookup`
+call, sleeps for a log-normal delay (median ≈ 40ms, p99 ≈ 1.8s). 1.5% of
+calls instead sleep a flat 5s -- longer than the 3s downstream timeout every
+caller uses (`common/http.py`), so these surface as timeouts one hop up, in
+`fraud`.
+
+**Symptom:** heavy-tailed latency on `POST /api/transactions`; most requests
+look normal, a fat p95/p99 tail; occasional 504s from `fraud`/`gateway` with
+no corresponding 5xx logged in `sanctions` itself (it never errors -- it's
+just slow).
+
+## sanctions_errors (`sanctions`)
+
+**Mechanism:** `services/sanctions/chaos.py`. 3% of `/internal/lookup` calls
+return HTTP 503 outright, chosen independently per call via `random.random()`.
+
+**Symptom:** a small, steady rate of 502/504 responses from `gateway`, fully
+uncorrelated with load or account. `sanctions` logs a "lookup" line for every
+attempt regardless, so its request count won't match its success count.
+
+## n_plus_one (`insights`)
+
+**Mechanism:** `services/insights/analytics.py::_n_plus_one_summary`. Instead
+of one `LEFT JOIN` against `merchant_categories`, fetches up to 500 raw
+transaction rows and issues one additional `SELECT ... FROM merchant_categories
+WHERE id = :id` per row.
+
+**Symptom:** `GET /api/accounts/{id}/insights` for a high-volume (hot) account
+issues 50-500+ SQL statements instead of 1-2, and its latency grows with the
+account's transaction count rather than staying flat.
+
+## pool_starvation (`ledger`)
+
+**Mechanism:** `services/ledger/db.py`. Two engines are built at startup: a
+normal one (`pool_size=20, max_overflow=10`) and a starved one (`pool_size=2,
+max_overflow=0, pool_timeout=10`). Every `ledger` request picks which
+already-running pool to borrow a connection from based on the current flag --
+this is what makes the toggle live without a restart, since SQLAlchemy pools
+can't be resized in place.
+
+**Symptom:** throughput plateaus as concurrency increases past ~2 in-flight
+requests; latency and queueing rise sharply under concurrent load while a
+single request in isolation still looks fine.
+
+## row_lock_contention (`ledger`)
+
+**Mechanism:** `services/ledger/repository.py::settle_transaction`. Only for
+accounts with `is_hot = true`: takes `SELECT balance_minor FROM accounts ...
+FOR UPDATE` and holds the transaction open for an extra 150ms before
+committing the balance update.
+
+**Symptom:** concurrent transactions against a hot account show p95 latency
+several times higher than the same concurrency spread across many cold
+accounts; cold-account traffic is unaffected.
+
+## blocking_event_loop (`fraud`)
+
+**Mechanism:** `services/fraud/chaos.py::synchronous_cpu_work`. Runs ~120ms of
+synchronous SHA-256 hashing directly inside the `async def` handler for
+`/internal/score` -- no `run_in_executor`/`to_thread`.
+
+**Symptom:** concurrent requests to `fraud` (and anything waiting on it)
+serialise: wall-clock time for N concurrent requests approaches N × single-request
+time instead of staying close to single-request time, even though nothing is
+CPU-bound at the infrastructure level.
+
+## retry_storm (`gateway`)
+
+**Mechanism:** `services/gateway/orchestrator.py::_score_with_retry`. On any
+non-2xx from `fraud`, retries immediately up to 3 more times -- no backoff,
+no jitter, no circuit breaker.
+
+**Symptom:** when `sanctions` is also failing or slow, the number of
+`sanctions` "lookup" log lines per transaction multiplies (up to 4x), and
+downstream load spikes exactly when the downstream is already unhealthy --
+the opposite of what you want.
+
+## no_batching (`gateway`)
+
+**Mechanism:** `services/gateway/orchestrator.py::create_transaction`. Splits
+the transaction into 1-5 synthetic "line items" and calls `fraud` once per
+item, sequentially, instead of once for the whole transaction.
+
+**Symptom:** `sanctions` lookups per transaction multiply (roughly 2-5x) and
+end-to-end latency for `POST /api/transactions` grows proportionally, even
+though sanctions/fraud themselves are healthy.
+
+## unbounded_cache (`insights`)
+
+**Mechanism:** `services/insights/chaos.py` / `app.py`. Instead of the normal
+Redis cache (keyed by account + current chaos-flag fingerprint, TTL below),
+memoizes summaries in a module-level dict keyed by `(account_id, fingerprint,
+time.time_ns())` -- a key that is different on every single call. It never
+hits, and it never evicts.
+
+**Symptom:** `insights`' resident memory grows monotonically under repeated
+identical-looking requests and never plateaus or drops; cache hit rate (if you
+were watching Redis) would be zero -- the "cache" provides no speedup, only
+a leak.
+
+## slow_consumer (`notifier`)
+
+**Mechanism:** `services/notifier/consumer.py`. Sleeps 250ms per stream
+message before acking, inside the `notifier` consumer-group loop.
+
+**Symptom:** the `notifier` consumer-group lag on `stream:transactions` grows
+under sustained load and drains slowly afterward, while the `audit` group
+(same stream, independent group) keeps up fine -- isolating the problem to
+one specific consumer, not the stream or producer.
+
+## cold_cache (`insights`)
+
+**Mechanism:** `services/insights/app.py`. The Redis summary cache TTL drops
+from 900s to 60s.
+
+**Symptom:** repeated identical requests for the same account show periodic
+latency spikes roughly every 60s (a full recompute) instead of one initial
+spike followed by uniformly fast cache hits.
+
+## poison_data (`ledger`)
+
+**Mechanism:** `services/ledger/app.py` / `repository.py::find_null_currency_row`.
+Every `POST /internal/transactions` checks whether the account has any
+existing transaction with a `NULL` currency (one of the seeded malformed row
+varieties, §8). When the flag is on, that row's `currency` is accessed with no
+null guard (`.upper()` on `None`), which raises and surfaces as an unhandled
+HTTP 500. When off, the same check is guarded and does nothing.
+
+**Symptom:** a specific, small set of accounts reliably 500 on every
+transaction attempt while the flag is on, and behave completely normally
+while it's off -- input data that was always there, suddenly fatal.
+
+## chatty_health (`gateway`)
+
+**Mechanism:** `services/gateway/orchestrator.py::_run_chatty_health_probe`.
+Before doing any real work, `POST /api/transactions` calls `GET /healthz`
+sequentially on all five downstreams, tagged with an `X-Ledgerline-Probe`
+header so the extra hits are distinguishable from routine health polling.
+
+**Symptom:** every transaction request produces five extra, sequential
+health-check round trips before the real work even starts, inflating latency
+by a fixed per-request tax that has nothing to do with load or data size.