Files
Codeman/test
Codeman maintainer 5c45d434ac test(qr-auth): replace flaky max-deviation bias check with chi-square
The short-code distribution test asserted that no base62 character
deviated more than 15% from its expected count. That statistic is the
maximum over 62 correlated near-normal cells, so its tail is fat: at
n=36000 the per-cell relative SD is ~4.1%, which puts the 15% bound at
|z| ~ 3.65, and taken as a max over 62 cells it fires on a perfectly
uniform generator about 1.6% of the time. Measured directly: 48 spurious
failures in 3000 simulated runs. It had been rerun-to-green repeatedly
and most recently red-herringed a PR review.

Chi-square is the correct test for "is this multinomial uniform", and
unlike 0.15 its threshold is derivable. df=61, Wilson-Hilferty puts the
p=1e-6 critical value at ~129, so the bound is 130.

Power is unchanged. Removing rejection sampling from generateShortCode
reintroduces modulo bias (256 % 62 = 8, so eight characters draw five
chances per 256 instead of four) and was verified against the real code
in an isolated worktree: chi-square 243.06 against the 130 limit. The
threshold sits in a wide empty gap, 3000 clean runs peaked at 104 while
200 biased runs bottomed out at 174.5.

Also iterate the alphabet explicitly rather than the observed keys, so a
character that never appears counts as zero instead of being skipped.

Verified: 30/30 consecutive runs of the real test pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 04:15:30 +02:00
..
2026-07-20 17:35:40 +02:00
2026-07-20 17:35:40 +02:00
2026-06-19 17:58:06 -04:00
2026-06-19 17:58:06 -04:00
2026-07-20 02:36:02 +02:00
2026-07-20 02:36:02 +02:00
2026-06-19 13:10:17 -04:00