A rate gate needs its sample size derived from the rate

The slid-serve test asked for 6 of 10 sliced serves returned. Measured over 200
trials a seed across eight seeds, the receiver's real rate is 0.85..0.92 — so
the assertion was sound and the SAMPLE was too small: at p ~= 0.88, 10 trials
land at or below 5 with probability ~0.004, one seed in 230, and SEED=80 was it.

n is now 40 at the same 60% bar, which puts the threshold five standard
deviations below the true rate while still failing instantly if the receiver
regresses toward the 0-in-24 the test was born from. It is free: a returned
serve breaks the frame loop early, and the suite still runs in ~0.11 s.

Green on seeds 1..400.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
wuwiandClaude Opus 5 committed 2026-08-16 12:15:39 +02:00
1 parent bdc3f23f92
commit 131c0fa114
2 files changed
+19 -6

No files matched your search

+12 -5
View File
@@ -431,11 +431,18 @@ Measured, 60 seeds × 150 s of drill each: before, **7 seeds produced overlaps,
is still the cap of 7, so the fan is wider, not thinner. Suite green on 20260811 / 1234 / 777 / 99 /
424242 / 8, and on 299 of seeds 1..300 — the one straggler is unrelated and still open (see below).
**OPEN, found by the same fuzz:** `SEED=80` fails *"opponent one returns a slid serve: 5/10"*. That
assertion wants 6 of 10 sliced serves returned and the AI's real rate sits right on the threshold, so
it is a coin-flip gate of exactly the kind this file warns about. Untouched here because the honest
fix is a decision about the opponent, not the test: either widen `n` so the mean settles, or accept
5/10 as the real return rate and say so.
**The one straggler the same fuzz found, and the arithmetic to copy for any rate gate here.**
`SEED=80` failed *"opponent one returns a slid serve: 5/10"*. The receiver is innocent: measured over
200 trials a seed across eight seeds, the real return rate is **0.85..0.92**. The gate was simply too
small a sample — asking 6 of 10 at p ≈ 0.88 fails with probability ~0.004, one seed in 230, and 300
seeds duly produced one. **n is now 40 at the same 60% bar**, which puts the threshold five standard
deviations below the true rate while still failing instantly if the receiver regresses toward the
0-in-24 the test was born from. It costs nothing — a returned serve breaks the frame loop early, so
the whole suite still runs in about a tenth of a second. Green on seeds 1..400.
The general lesson: **a pass/fail gate on a random rate needs its sample size derived from the rate,
not guessed.** If the true rate is p and the bar is b, n has to put b several standard deviations
(`sqrt(p(1-p)/n)`) below p, or the gate is a slow coin flip that trains you to re-run the suite.
## Session log: THE NETPLAY LOBBY — a link to send, a code to type, a name to be (2026-08-15)
+7 -1
View File
@@ -1108,8 +1108,14 @@ test('a served ball with a slide on it is no longer a free kill', () => {
}
return false;
};
// n = 40, NOT 10, and this is the arithmetic to copy for any other rate gate here. The real return
// rate is ~0.88 (measured, 200 trials a seed across eight seeds: 0.85..0.92), so a 10-sample gate
// asking for 6 fails whenever the tail lands — p ~= 0.004, which is one seed in 230 and duly showed
// up as SEED=80. At 40 the same 60% bar sits five standard deviations below the true rate: the gate
// still fails instantly if the receiver regresses toward the 0-in-24 this test was born from, and
// it costs nothing (a returned serve breaks the frame loop early).
let got = 0;
const n = 10;
const n = 40;
for (let i = 0; i < n; i++) if (serve(-0.5)) got++;
assert.ok(got >= n * 0.6, `opponent one returns a slid serve: ${got}/${n} (was 0)`);
});