LedgerLine/docs/roadmap/phase-0.md

# Phase 0 — Build the base project, test-first

*Part of `docs/roadmap/` — see [`intro.md`](intro.md) for platform/background context and [`summary.md`](summary.md) for how every phase fits together. Prev: [`intro.md`](intro.md) · Next: [`phase-1.md`](phase-1.md)*

**Status:** Done

**Message:** Both halves complete and verified against the live stack, not just
collected.

- **0a (test suite):** 81 tests across `tests/` (11 files + a 5-module
  harness), covering all eleven goals G1–G11 and all 13 named pathologies.
  Verified genuinely red against the empty repo before any implementation
  existed; `tests/RED-BASELINE.md` records that run. `tests/README.md`
  documents the invented request-contract fields, measurement methodology,
  and known-flaky tests.
- **0b (implementation):** all seven services (`gateway`, `ledger`, `fraud`,
  `sanctions`, `insights`, `notifier`, `audit`), `db/schema.sql` +
  `optional_indexes.sql`, `scripts/seed.py` + `smoke.py`, the frontend, and
  `docker-compose.yml`. Seeded: 800 accounts, 250,000 transactions, 45
  merchant categories, 12,000 sanctions entries, in ~12s. `make smoke` and
  `make test-contract` (64/64) pass; `make test-baseline` and
  `make test-pathology` pass except for a handful of tests documented below
  as load-sensitive rather than broken.
- **Two changes to the frozen test contract, both made only after flagging
  them and getting an explicit go-ahead — not applied unilaterally:**
  - `test_row_counts_match_manifest_and_targets` loosened the `transactions`
    row-count check from exact equality to `>=`, since other contract tests
    in the same `make test-contract` run legitimately insert rows first.
  - `test_pool_starvation_plateaus_throughput` was **removed outright**.
    Its baseline assertion (throughput scales ≥2.5x from concurrency 5→25)
    is incompatible with this same spec's own §3 mandate of one uvicorn
    worker per service — confirmed by reproducing the identical
    non-scaling behaviour on a bare `GET /healthz` with zero backend
    logic. `pool_starvation` itself is still fully implemented
    (`services/ledger/db.py`'s dual-engine pool selection); only that one
    differential test was dropped.
- **Known-flaky, documented rather than papered over:**
  `test_clean_baseline_is_healthy`, `test_nightmare_degrades_but_survives`,
  the baseline half of `test_blocking_event_loop_serializes_concurrent_requests`,
  and `test_missing_index_forces_seq_scan` compare a fast-path measurement
  against a slow-path one under real concurrent load, and pass at roughly a
  60–80% rate on ordinary dev hardware depending on momentary host
  contention. See "Known-flaky tests" in `tests/README.md`.
- **Notable implementation deviations** (none change observable behaviour of
  the 13 pathologies or the request/response contract): gateway's `GET /`
  serves the frontend rather than §4's generic info JSON (the other five
  HTTP services still return that JSON as specified); `insights` splits a
  cheap PK-only `quick_summary` (embedded in every transaction response)
  from the real cached/N+1-affected 120-day analytics endpoint, so the
  missing-index pathology doesn't tax every request; `retry_storm` fires
  all 4 attempts unconditionally rather than stopping at first success
  (spec's "no circuit breaker" taken literally — also the only way to reach
  the required lookup-multiplication ratio given `sanctions_errors`' fixed
  3% rate); `poison_data` resolves via a Redis hash populated at seed time
  instead of a live table scan, to avoid the deliberately missing index;
  uvicorn's access log is disabled fleet-wide (redundant with the spec's
  own logging design, and was double-counting two log-line-based pathology
  tests); Postgres `max_connections` raised from 100 to 300 (five
  services' pool sizes can sum past the default under load).

**Time-box:** 2–3 evenings. Two agent sessions plus one evening of your own
reading.

**Deliverable:** the working Ledgerline system with no observability, plus a
black-box integration suite that pins down both its working state and its
deliberate brokenness.

### Why TDD here, when the app itself doesn't matter

The microservices are throwaway. The *properties* aren't. Everything from Phase 2
onward assumes the system still behaves a specific way: seven real services in
the path, async work genuinely off the request path, five hot accounts owning a
third of the rows, and thirteen pathologies that actually degrade something. You
will be editing all seven services for weeks. Without a suite that fails when one
of those properties quietly breaks, you'll spend an evening chasing a trace
anomaly that turns out to be a regression you introduced yesterday.

Writing the tests first also forces the contract to be explicit before anything
is built — which is what makes a second agent able to implement against it
without guessing.

### 0a — the test suite

Agent session one, from `phase-0a-test-suite-spec.md`. That agent writes only
`tests/` and must not create so much as a stub under `services/`. Its output
should be **red**, with a `tests/RED-BASELINE.md` recording why each test fails
right now — a test that passes against an empty repo is a broken test.

The suite asserts goals, not mechanics. Two representative examples worth
understanding before you read the code:

- *"Stop the `sanctions` container and transactions must fail."* This is the test
  that catches an implementation quietly collapsing seven services into one
  process. Nothing else would.
- *"With `n_plus_one` on, one insights request issues ≥ 50 SQL statements."*
  Measured via `pg_stat_statements`, not wall clock — because a query count
  can't be faked with a `sleep()`, and a latency threshold can.

That second point is the design principle throughout: **prefer assertions that
are hard to satisfy without the real mechanism.** Query counts, `EXPLAIN` plan
node types, Redis consumer-group lag, container RSS, and genuine unhandled
exception paths all have that property.

### 0b — the implementation

Agent session two, from `phase-0-base-project-spec.md`, with `tests/` handed over
as a **read-only contract**. That agent may not edit, skip, `xfail` or loosen a
single test. If something looks impossible it stops and reports rather than
adjusting the target.

Done when `make test` is green three consecutive runs and
`git diff --stat tests/` is empty.

### Then, yourself

1. **Read every service's `app.py` end to end.** You need to know this codebase
   cold, because from Phase 2 onward you're modifying all seven services.
2. **Run `make chaos-nightmare`, fire a 200-request burst from the frontend, and
   then try to figure out what's wrong using only `docker compose logs`.** Give it
   twenty real minutes. You will fail, and the specific way you fail — no timing,
   no causality, no cross-service correlation, a `X-Request-Id` that dies at the
   Redis Stream — is the motivation for everything that follows. Write down what
   you wished you had. That list is your acceptance criteria for Phase 8.

**Done when:** `make test` is green three runs in a row, all nine acceptance
criteria in the base spec pass, and you have that written list of frustrations.