A rate gate needs its sample size derived from the rate
The slid-serve test asked for 6 of 10 sliced serves returned. Measured over 200 trials a seed across eight seeds, the receiver's real rate is 0.85..0.92 — so the assertion was sound and the SAMPLE was too small: at p ~= 0.88, 10 trials land at or below 5 with probability ~0.004, one seed in 230, and SEED=80 was it. n is now 40 at the same 60% bar, which puts the threshold five standard deviations below the true rate while still failing instantly if the receiver regresses toward the 0-in-24 the test was born from. It is free: a returned serve breaks the frame loop early, and the suite still runs in ~0.11 s. Green on seeds 1..400. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
bdc3f23f92
commit
131c0fa114
2 files changed
+19
-6
No files matched your search
+12
-5
@@ -431,11 +431,18 @@ Measured, 60 seeds × 150 s of drill each: before, **7 seeds produced overlaps,
|
|||||||
is still the cap of 7, so the fan is wider, not thinner. Suite green on 20260811 / 1234 / 777 / 99 /
|
is still the cap of 7, so the fan is wider, not thinner. Suite green on 20260811 / 1234 / 777 / 99 /
|
||||||
424242 / 8, and on 299 of seeds 1..300 — the one straggler is unrelated and still open (see below).
|
424242 / 8, and on 299 of seeds 1..300 — the one straggler is unrelated and still open (see below).
|
||||||
|
|
||||||
**OPEN, found by the same fuzz:** `SEED=80` fails *"opponent one returns a slid serve: 5/10"*. That
|
**The one straggler the same fuzz found, and the arithmetic to copy for any rate gate here.**
|
||||||
assertion wants 6 of 10 sliced serves returned and the AI's real rate sits right on the threshold, so
|
`SEED=80` failed *"opponent one returns a slid serve: 5/10"*. The receiver is innocent: measured over
|
||||||
it is a coin-flip gate of exactly the kind this file warns about. Untouched here because the honest
|
200 trials a seed across eight seeds, the real return rate is **0.85..0.92**. The gate was simply too
|
||||||
fix is a decision about the opponent, not the test: either widen `n` so the mean settles, or accept
|
small a sample — asking 6 of 10 at p ≈ 0.88 fails with probability ~0.004, one seed in 230, and 300
|
||||||
5/10 as the real return rate and say so.
|
seeds duly produced one. **n is now 40 at the same 60% bar**, which puts the threshold five standard
|
||||||
|
deviations below the true rate while still failing instantly if the receiver regresses toward the
|
||||||
|
0-in-24 the test was born from. It costs nothing — a returned serve breaks the frame loop early, so
|
||||||
|
the whole suite still runs in about a tenth of a second. Green on seeds 1..400.
|
||||||
|
|
||||||
|
The general lesson: **a pass/fail gate on a random rate needs its sample size derived from the rate,
|
||||||
|
not guessed.** If the true rate is p and the bar is b, n has to put b several standard deviations
|
||||||
|
(`sqrt(p(1-p)/n)`) below p, or the gate is a slow coin flip that trains you to re-run the suite.
|
||||||
|
|
||||||
## Session log: THE NETPLAY LOBBY — a link to send, a code to type, a name to be (2026-08-15)
|
## Session log: THE NETPLAY LOBBY — a link to send, a code to type, a name to be (2026-08-15)
|
||||||
|
|
||||||
|
|||||||
@@ -1108,8 +1108,14 @@ test('a served ball with a slide on it is no longer a free kill', () => {
|
|||||||
}
|
}
|
||||||
return false;
|
return false;
|
||||||
};
|
};
|
||||||
|
// n = 40, NOT 10, and this is the arithmetic to copy for any other rate gate here. The real return
|
||||||
|
// rate is ~0.88 (measured, 200 trials a seed across eight seeds: 0.85..0.92), so a 10-sample gate
|
||||||
|
// asking for 6 fails whenever the tail lands — p ~= 0.004, which is one seed in 230 and duly showed
|
||||||
|
// up as SEED=80. At 40 the same 60% bar sits five standard deviations below the true rate: the gate
|
||||||
|
// still fails instantly if the receiver regresses toward the 0-in-24 this test was born from, and
|
||||||
|
// it costs nothing (a returned serve breaks the frame loop early).
|
||||||
let got = 0;
|
let got = 0;
|
||||||
const n = 10;
|
const n = 40;
|
||||||
for (let i = 0; i < n; i++) if (serve(-0.5)) got++;
|
for (let i = 0; i < n; i++) if (serve(-0.5)) got++;
|
||||||
assert.ok(got >= n * 0.6, `opponent one returns a slid serve: ${got}/${n} (was 0)`);
|
assert.ok(got >= n * 0.6, `opponent one returns a slid serve: ${got}/${n} (was 0)`);
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in new issue
Block a user