Movses Aslanyan
ccf1783779
81 tests across 11 files plus a 5-module harness (load, pg, streams, containers, payloads), covering all eleven Phase 0 goals and all 13 named chaos-flag pathologies. Talks to the system only over HTTP, Postgres, Redis, and the Docker CLI -- no imports of application code. RED-BASELINE.md records the suite failing red against the empty repo before any implementation existed; README.md documents the request-contract fields invented to fill gaps in the spec, measurement methodology, and known-flaky tests. This suite is a frozen contract from this point forward per CLAUDE.md.
63 lines
2.8 KiB
Markdown
63 lines
2.8 KiB
Markdown
# RED-BASELINE
|
|
|
|
Recorded by running the suite, as written, against the actual state of the
|
|
repo at the end of Phase 0a: no `services/`, `common/`, `db/`, `scripts/`, or
|
|
`docker-compose.yml` exist yet, and no containers are running. This is the
|
|
required proof that nothing in the suite passes vacuously.
|
|
|
|
Commands run (Python 3.12 venv, `pip install -r requirements-test.txt`):
|
|
|
|
```
|
|
pytest --collect-only -q # 82 tests collected, 0 errors
|
|
pytest -m "contract or seed or guard" -q # 64 errors, 18 deselected
|
|
pytest -m "pathology or slow or baseline" -q # 18 errors, 64 deselected
|
|
```
|
|
|
|
## Why every test currently fails
|
|
|
|
**All 82 tests fail at fixture setup with the same root cause: connection
|
|
refused.** The session-scoped, autouse `stack_up` fixture in `conftest.py`
|
|
polls `GET /healthz` on all six HTTP services for 30s, gets nothing back
|
|
(there is no gateway, no ledger, no fraud, no sanctions, no insights, no
|
|
notifier -- none of `services/` exists), and calls `pytest.fail()` with:
|
|
|
|
```
|
|
Stack is not up: fraud, gateway, insights, ledger, notifier, sanctions did not
|
|
answer /healthz within 30s. Run `make up && make seed` first.
|
|
```
|
|
|
|
Every other test in the suite depends on `stack_up` (directly, or
|
|
transitively through `api`, `chaos`, `manifest` -> `seeded`, or `dsn` used
|
|
against a nonexistent Postgres). pytest caches the session-scoped failure, so
|
|
after the first 30s timeout every remaining test errors immediately at setup
|
|
with the same message, rather than re-polling.
|
|
|
|
This is deliberately the *first* thing every test checks. It is a stronger
|
|
and more honest RED signal than a wall of individual connection-refused
|
|
tracebacks from `httpx`/`psycopg`/`redis`, and it's exactly the failure mode a
|
|
human would hit running this suite manually against nothing.
|
|
|
|
## The one test that would fail on its own merits even with a live stack
|
|
|
|
`tests/test_no_observability.py::test_application_code_exists` is a sanity
|
|
guard, not a real assertion about the system: it exists specifically so
|
|
`test_no_file_imports_observability_libraries` (which walks `services/` and
|
|
`common/` looking for forbidden imports) can't pass vacuously by finding zero
|
|
`.py` files. Right now it fails at the `stack_up` setup step like everything
|
|
else, but if `stack_up` is bypassed (e.g. by testing this file's guard logic
|
|
in isolation against the current empty repo), it fails on its own assertion:
|
|
|
|
```
|
|
no .py files found under services/ or common/ -- either the app hasn't been
|
|
implemented yet, or the guard tests below would pass vacuously
|
|
```
|
|
|
|
That confirms the vacuous-pass guard itself is red for the right reason too.
|
|
|
|
## What "green" requires
|
|
|
|
`make up && make seed` need `docker-compose.yml`, `db/schema.sql`,
|
|
`scripts/seed.py`, and all seven `services/*` -- i.e. all of Phase 0b, built
|
|
from `docs/phase-0-base-project-spec.md` against this suite as a read-only
|
|
contract.
|