Movses Aslanyan
ccf1783779
81 tests across 11 files plus a 5-module harness (load, pg, streams, containers, payloads), covering all eleven Phase 0 goals and all 13 named chaos-flag pathologies. Talks to the system only over HTTP, Postgres, Redis, and the Docker CLI -- no imports of application code. RED-BASELINE.md records the suite failing red against the empty repo before any implementation existed; README.md documents the request-contract fields invented to fill gaps in the spec, measurement methodology, and known-flaky tests. This suite is a frozen contract from this point forward per CLAUDE.md.
2.8 KiB
RED-BASELINE
Recorded by running the suite, as written, against the actual state of the
repo at the end of Phase 0a: no services/, common/, db/, scripts/, or
docker-compose.yml exist yet, and no containers are running. This is the
required proof that nothing in the suite passes vacuously.
Commands run (Python 3.12 venv, pip install -r requirements-test.txt):
pytest --collect-only -q # 82 tests collected, 0 errors
pytest -m "contract or seed or guard" -q # 64 errors, 18 deselected
pytest -m "pathology or slow or baseline" -q # 18 errors, 64 deselected
Why every test currently fails
All 82 tests fail at fixture setup with the same root cause: connection
refused. The session-scoped, autouse stack_up fixture in conftest.py
polls GET /healthz on all six HTTP services for 30s, gets nothing back
(there is no gateway, no ledger, no fraud, no sanctions, no insights, no
notifier -- none of services/ exists), and calls pytest.fail() with:
Stack is not up: fraud, gateway, insights, ledger, notifier, sanctions did not
answer /healthz within 30s. Run `make up && make seed` first.
Every other test in the suite depends on stack_up (directly, or
transitively through api, chaos, manifest -> seeded, or dsn used
against a nonexistent Postgres). pytest caches the session-scoped failure, so
after the first 30s timeout every remaining test errors immediately at setup
with the same message, rather than re-polling.
This is deliberately the first thing every test checks. It is a stronger
and more honest RED signal than a wall of individual connection-refused
tracebacks from httpx/psycopg/redis, and it's exactly the failure mode a
human would hit running this suite manually against nothing.
The one test that would fail on its own merits even with a live stack
tests/test_no_observability.py::test_application_code_exists is a sanity
guard, not a real assertion about the system: it exists specifically so
test_no_file_imports_observability_libraries (which walks services/ and
common/ looking for forbidden imports) can't pass vacuously by finding zero
.py files. Right now it fails at the stack_up setup step like everything
else, but if stack_up is bypassed (e.g. by testing this file's guard logic
in isolation against the current empty repo), it fails on its own assertion:
no .py files found under services/ or common/ -- either the app hasn't been
implemented yet, or the guard tests below would pass vacuously
That confirms the vacuous-pass guard itself is red for the right reason too.
What "green" requires
make up && make seed need docker-compose.yml, db/schema.sql,
scripts/seed.py, and all seven services/* -- i.e. all of Phase 0b, built
from docs/phase-0-base-project-spec.md against this suite as a read-only
contract.