A flaky gate is worse than no gate: seed the suite
The pipeline caught two flaky tests of mine that sixty local runs had not. Both were bad tests rather than bad code, and both were mine from this session: - the eater test counted SURVIVORS, not arrivals. An eater crosses the whole court in ~24s and despawns, so a snapshot at 40s can legitimately show an empty court — that read as "the eaters never came". It tracks arrivals over the window now, and asserts the cap as a running peak. - the auto-aim test never actually aimed at the body it claimed to: fy was pinned at -0.95 and fx was a scaled x offset, so the ray leaned up-court and the called body could genuinely be the best-aligned thing in the cone. That is the magnet working. It now normalises a vector straight at the wrong body, picks the one furthest in x from the called one so the two are separable, and asserts the magnet takes what you pointed at. Measured 8 failures in 40 runs before, 0 in 60 after. And the durable fix: Math.random is seeded at the top of test.js. This suite is a Dockerfile build stage, so a red run blocks the deploy, which makes a 1-in-5 flake worse than no gate — it trains you to re-run it. The game leans on randomness everywhere, so the stream is made reproducible rather than the randomness stripped out. SEED=1234 node test.js re-rolls it; 25 seeds pass. The cost is noted in PROJECT.md: under one seed a guarded test can silently take its early-out forever, so I checked all four guards still run their bodies. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5b2f1d55ce
commit
d34683bc68
2 files changed
+63
-10
No files matched your search
+10
@@ -6,6 +6,16 @@ everything they specified is shipped, and what still matters from them is record
|
||||
long history, `~/.claude/projects/.../memory/splatter-tennis-history.md` is auto-loaded; for the
|
||||
detail of any individual change, read the git log.
|
||||
|
||||
**THE SUITE IS DETERMINISTIC ON PURPOSE (`Math.random` is seeded at the top of `test.js`).** It is a
|
||||
Dockerfile build stage, so a red run blocks the deploy — which makes a test that fails one run in
|
||||
five *worse than no gate*, because it trains you to re-run it. Two drill tests did exactly that and
|
||||
the pipeline caught what sixty local runs had not: one asserted on a random batch composition, the
|
||||
other on a SNAPSHOT of a population that legitimately empties (an eater crosses the whole court in
|
||||
~24 s and despawns, so "no eaters on court at 40 s" is normal, not a failure). **Fuzz with
|
||||
`SEED=1234 node test.js` whenever you touch anything probabilistic** — and beware the cost of one
|
||||
fixed seed: a guarded test can silently take its early-out every run and quietly stop testing
|
||||
anything, so if you add a `return` for an awkward deal, check the body still runs.
|
||||
|
||||
**`node test.js` must pass before any commit** — it drives the real game modules headless and pins
|
||||
the state-machine contracts that have historically produced the show-stoppers. The Dockerfile runs
|
||||
it as a build stage, so a red suite cannot deploy — but finding out from Coolify is the slow way.
|
||||
|
||||
Reference in new issue
Block a user