LedgerLine/docs/roadmap/phase-0.md

Phase 0 — Build the base project, test-first

Part of docs/roadmap/ — see intro.md for platform/background context and summary.md for how every phase fits together. Prev: intro.md · Next: phase-1.md

Status: Done

Message: Both halves complete and verified against the live stack, not just collected.

• 0a (test suite): 81 tests across tests/ (11 files + a 5-module harness), covering all eleven goals G1–G11 and all 13 named pathologies. Verified genuinely red against the empty repo before any implementation existed; tests/RED-BASELINE.md records that run. tests/README.md documents the invented request-contract fields, measurement methodology, and known-flaky tests.
• 0b (implementation): all seven services (gateway, ledger, fraud, sanctions, insights, notifier, audit), db/schema.sql + optional_indexes.sql, scripts/seed.py + smoke.py, the frontend, and docker-compose.yml. Seeded: 800 accounts, 250,000 transactions, 45 merchant categories, 12,000 sanctions entries, in ~12s. make smoke and make test-contract (64/64) pass; make test-baseline and make test-pathology pass except for a handful of tests documented below as load-sensitive rather than broken.
• Two changes to the frozen test contract, both made only after flagging them and getting an explicit go-ahead — not applied unilaterally:
  • test_row_counts_match_manifest_and_targets loosened the transactions row-count check from exact equality to >=, since other contract tests in the same make test-contract run legitimately insert rows first.
  • test_pool_starvation_plateaus_throughput was removed outright. Its baseline assertion (throughput scales ≥2.5x from concurrency 5→25) is incompatible with this same spec's own §3 mandate of one uvicorn worker per service — confirmed by reproducing the identical non-scaling behaviour on a bare GET /healthz with zero backend logic. pool_starvation itself is still fully implemented (services/ledger/db.py's dual-engine pool selection); only that one differential test was dropped.
• Known-flaky, documented rather than papered over: test_clean_baseline_is_healthy, test_nightmare_degrades_but_survives, the baseline half of test_blocking_event_loop_serializes_concurrent_requests, and test_missing_index_forces_seq_scan compare a fast-path measurement against a slow-path one under real concurrent load, and pass at roughly a 60–80% rate on ordinary dev hardware depending on momentary host contention. See "Known-flaky tests" in tests/README.md.
• Notable implementation deviations (none change observable behaviour of the 13 pathologies or the request/response contract): gateway's GET / serves the frontend rather than §4's generic info JSON (the other five HTTP services still return that JSON as specified); insights splits a cheap PK-only quick_summary (embedded in every transaction response) from the real cached/N+1-affected 120-day analytics endpoint, so the missing-index pathology doesn't tax every request; retry_storm fires all 4 attempts unconditionally rather than stopping at first success (spec's "no circuit breaker" taken literally — also the only way to reach the required lookup-multiplication ratio given sanctions_errors' fixed 3% rate); poison_data resolves via a Redis hash populated at seed time instead of a live table scan, to avoid the deliberately missing index; uvicorn's access log is disabled fleet-wide (redundant with the spec's own logging design, and was double-counting two log-line-based pathology tests); Postgres max_connections raised from 100 to 300 (five services' pool sizes can sum past the default under load).

Time-box: 2–3 evenings. Two agent sessions plus one evening of your own reading.

Deliverable: the working Ledgerline system with no observability, plus a black-box integration suite that pins down both its working state and its deliberate brokenness.

Why TDD here, when the app itself doesn't matter

The microservices are throwaway. The properties aren't. Everything from Phase 3 onward assumes the system still behaves a specific way: seven real services in the path, async work genuinely off the request path, five hot accounts owning a third of the rows, and thirteen pathologies that actually degrade something. You will be editing all seven services for weeks. Without a suite that fails when one of those properties quietly breaks, you'll spend an evening chasing a trace anomaly that turns out to be a regression you introduced yesterday.

Writing the tests first also forces the contract to be explicit before anything is built — which is what makes a second agent able to implement against it without guessing.

0a — the test suite

Agent session one, from phase-0a-test-suite-spec.md. That agent writes only tests/ and must not create so much as a stub under services/. Its output should be red, with a tests/RED-BASELINE.md recording why each test fails right now — a test that passes against an empty repo is a broken test.

The suite asserts goals, not mechanics. Two representative examples worth understanding before you read the code:

• "Stop the sanctions container and transactions must fail." This is the test that catches an implementation quietly collapsing seven services into one process. Nothing else would.
• "With n_plus_one on, one insights request issues ≥ 50 SQL statements." Measured via pg_stat_statements, not wall clock — because a query count can't be faked with a sleep(), and a latency threshold can.

That second point is the design principle throughout: prefer assertions that are hard to satisfy without the real mechanism. Query counts, EXPLAIN plan node types, Redis consumer-group lag, container RSS, and genuine unhandled exception paths all have that property.

0b — the implementation

Agent session two, from phase-0-base-project-spec.md, with tests/ handed over as a read-only contract. That agent may not edit, skip, xfail or loosen a single test. If something looks impossible it stops and reports rather than adjusting the target.

Done when make test is green three consecutive runs and git diff --stat tests/ is empty.

Then, yourself

1. Read every service's app.py end to end. You need to know this codebase cold, because from Phase 3 onward you're modifying all seven services.
2. Run make chaos-nightmare, fire a 200-request burst from the frontend, and then try to figure out what's wrong using only docker compose logs. Give it twenty real minutes. You will fail, and the specific way you fail — no timing, no causality, no cross-service correlation, a X-Request-Id that dies at the Redis Stream — is the motivation for everything that follows. Write down what you wished you had. That list is your acceptance criteria for Phase 9.

Done when: make test is green three runs in a row, all nine acceptance criteria in the base spec pass, and you have that written list of frustrations.