diff --git a/PROJECT.md b/PROJECT.md index c0d6252..e67d9ab 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -431,11 +431,18 @@ Measured, 60 seeds × 150 s of drill each: before, **7 seeds produced overlaps, is still the cap of 7, so the fan is wider, not thinner. Suite green on 20260811 / 1234 / 777 / 99 / 424242 / 8, and on 299 of seeds 1..300 — the one straggler is unrelated and still open (see below). -**OPEN, found by the same fuzz:** `SEED=80` fails *"opponent one returns a slid serve: 5/10"*. That -assertion wants 6 of 10 sliced serves returned and the AI's real rate sits right on the threshold, so -it is a coin-flip gate of exactly the kind this file warns about. Untouched here because the honest -fix is a decision about the opponent, not the test: either widen `n` so the mean settles, or accept -5/10 as the real return rate and say so. +**The one straggler the same fuzz found, and the arithmetic to copy for any rate gate here.** +`SEED=80` failed *"opponent one returns a slid serve: 5/10"*. The receiver is innocent: measured over +200 trials a seed across eight seeds, the real return rate is **0.85..0.92**. The gate was simply too +small a sample — asking 6 of 10 at p ≈ 0.88 fails with probability ~0.004, one seed in 230, and 300 +seeds duly produced one. **n is now 40 at the same 60% bar**, which puts the threshold five standard +deviations below the true rate while still failing instantly if the receiver regresses toward the +0-in-24 the test was born from. It costs nothing — a returned serve breaks the frame loop early, so +the whole suite still runs in about a tenth of a second. Green on seeds 1..400. + +The general lesson: **a pass/fail gate on a random rate needs its sample size derived from the rate, +not guessed.** If the true rate is p and the bar is b, n has to put b several standard deviations +(`sqrt(p(1-p)/n)`) below p, or the gate is a slow coin flip that trains you to re-run the suite. ## Session log: THE NETPLAY LOBBY — a link to send, a code to type, a name to be (2026-08-15) diff --git a/test.js b/test.js index 073ee0e..bbd4b6e 100644 --- a/test.js +++ b/test.js @@ -1108,8 +1108,14 @@ test('a served ball with a slide on it is no longer a free kill', () => { } return false; }; + // n = 40, NOT 10, and this is the arithmetic to copy for any other rate gate here. The real return + // rate is ~0.88 (measured, 200 trials a seed across eight seeds: 0.85..0.92), so a 10-sample gate + // asking for 6 fails whenever the tail lands — p ~= 0.004, which is one seed in 230 and duly showed + // up as SEED=80. At 40 the same 60% bar sits five standard deviations below the true rate: the gate + // still fails instantly if the receiver regresses toward the 0-in-24 this test was born from, and + // it costs nothing (a returned serve breaks the frame loop early). let got = 0; - const n = 10; + const n = 40; for (let i = 0; i < n; i++) if (serve(-0.5)) got++; assert.ok(got >= n * 0.6, `opponent one returns a slid serve: ${got}/${n} (was 0)`); });