A flaky gate is worse than no gate: seed the suite

The pipeline caught two flaky tests of mine that sixty local runs had not.
Both were bad tests rather than bad code, and both were mine from this
session:

- the eater test counted SURVIVORS, not arrivals. An eater crosses the whole
  court in ~24s and despawns, so a snapshot at 40s can legitimately show an
  empty court — that read as "the eaters never came". It tracks arrivals over
  the window now, and asserts the cap as a running peak.
- the auto-aim test never actually aimed at the body it claimed to: fy was
  pinned at -0.95 and fx was a scaled x offset, so the ray leaned up-court and
  the called body could genuinely be the best-aligned thing in the cone. That
  is the magnet working. It now normalises a vector straight at the wrong
  body, picks the one furthest in x from the called one so the two are
  separable, and asserts the magnet takes what you pointed at.

Measured 8 failures in 40 runs before, 0 in 60 after.

And the durable fix: Math.random is seeded at the top of test.js. This suite
is a Dockerfile build stage, so a red run blocks the deploy, which makes a
1-in-5 flake worse than no gate — it trains you to re-run it. The game leans
on randomness everywhere, so the stream is made reproducible rather than the
randomness stripped out. SEED=1234 node test.js re-rolls it; 25 seeds pass.
The cost is noted in PROJECT.md: under one seed a guarded test can silently
take its early-out forever, so I checked all four guards still run their
bodies.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
wuwiandClaude Opus 5 committed 2026-08-11 20:29:34 +02:00
1 parent 5b2f1d55ce
commit d34683bc68
2 files changed
+63 -10

No files matched your search

+10
View File
@@ -6,6 +6,16 @@ everything they specified is shipped, and what still matters from them is record
long history, `~/.claude/projects/.../memory/splatter-tennis-history.md` is auto-loaded; for the
detail of any individual change, read the git log.
**THE SUITE IS DETERMINISTIC ON PURPOSE (`Math.random` is seeded at the top of `test.js`).** It is a
Dockerfile build stage, so a red run blocks the deploy — which makes a test that fails one run in
five *worse than no gate*, because it trains you to re-run it. Two drill tests did exactly that and
the pipeline caught what sixty local runs had not: one asserted on a random batch composition, the
other on a SNAPSHOT of a population that legitimately empties (an eater crosses the whole court in
~24 s and despawns, so "no eaters on court at 40 s" is normal, not a failure). **Fuzz with
`SEED=1234 node test.js` whenever you touch anything probabilistic** — and beware the cost of one
fixed seed: a guarded test can silently take its early-out every run and quietly stop testing
anything, so if you add a `return` for an awkward deal, check the body still runs.
**`node test.js` must pass before any commit** — it drives the real game modules headless and pins
the state-machine contracts that have historically produced the show-stoppers. The Dockerfile runs
it as a build stage, so a red suite cannot deploy — but finding out from Coolify is the slow way.