chore(test): make npm test the CI gate and give each excluded suite a runner

`npm test` ran config/vitest.config.ts, which includes the browser, visual and
perf suites. On any machine without chromium, a free port and per-machine PNG
baselines that fails ~87 tests on a clean master, so the repo's most obvious
command could not be used as a pass/fail signal. The workaround had spread into
four docs as "never run bare `npm test`" warnings.

`npm test` now runs config/vitest.ci.config.ts — byte-for-byte what CI runs — so
local green means CI green. Verified: 264 files, 5248 tests, exit 0.

The suites it leaves out are not abandoned; each has a command:

  test:browser  5 Playwright files (chromium + a live server; codex-predictive-echo
                also needs a real codex binary)
  test:mobile   unchanged — the above plus per-machine PNG baselines
  test:perf     2 wall-clock benchmarks; need an otherwise idle machine
  test:all      the old everything-behaviour, kept reachable

test:ci is untouched (CI still calls it). test:watch and test:coverage follow
test onto the gate's config.

The more important half is the hole this closes. The exclusion list lived as
literals in one config and pointed one way only: a file excluded from CI and
added to no runner would be tested by NOTHING, silently, with every command
still green — vitest counts "no files matched a filter" as success. That is the
same shape as the #279/#280 blind spot already documented in CLAUDE.md.

So the globs moved to config/test-suites.ts, one array per REASON a suite cannot
run in CI, and all three configs derive from it. test/test-suite-partition.test.ts
then checks the arithmetic against the files on disk: it fails if any test file
is reachable by no runner, or by two. Confirmed it fires by orphaning a file and
watching it name it. The partition is exact today:

  gate 264 + browser 5 + perf 2 + mobile 9 = 280 = every *.test.ts in the repo

⚠️ One sharp edge, deliberate and documented: a file filter must match its
runner. `npm test -- test/mobile/keyboard.test.ts` now matches nothing and exits
GREEN having run zero tests, because the gate's config excludes that path.
CLAUDE.md recommended exactly that command in the on-screen-keyboard note; that
line now says `npm run test:mobile -- <file>`, and the Testing section calls out
the trap, since a green run of zero tests is worse than a red one.

Docs synced: CLAUDE.md, AGENTS.md, .github/CONTRIBUTING.md, README.md,
README.zh-CN.md, and two ci.yml comments that claimed only test/mobile/** was
excluded — it is three suites, and 5 Playwright files rather than 3.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer
2026-08-18 19:55:23 +02:00
parent 5080390e2c
commit 947ff6f6fa
14 changed files with 293 additions and 39 deletions
+12 -3
View File
@@ -37,11 +37,20 @@ npm run check:frontend-syntax # syntax-checks the plain-JS frontend modules
### Tests
```bash
npm test -- test/<file>.test.ts # one file (the normal way)
npm run test:ci # the full CI sweep
npm test # the gate — exactly what CI runs
npm test -- test/<file>.test.ts # one file
```
**Never run bare `npm test`.** The default config includes browser-driven Playwright suites that need a live server, Chromium, and environment-specific baselines; they will hang or fail on a normal machine. `test:ci` is the honest "run everything" command, it is exactly what CI runs.
`npm test` is the same suite CI runs, so a green run locally means a green run there. It leaves out three suites that cannot pass on an arbitrary machine, each with its own command:
```bash
npm run test:browser # Playwright + chromium (+ a live server; codex-predictive-echo needs a real codex binary)
npm run test:mobile # the above plus environment-specific PNG baselines
npm run test:perf # wall-clock benchmarks — run on an otherwise idle machine
npm run test:all # literally everything, environmental failures included
```
Expect `test:browser`/`test:mobile`/`test:perf` to fail where the machine cannot provide what they need; read that as "not runnable here", not as a regression. `config/test-suites.ts` holds the globs, and both configs derive from it, so the exclusions and those runners cannot drift apart.
If you add a test that binds a port, pick a unique one at 3150 or above (search the repo for `const PORT =` first). Never 3000.
+11 -4
View File
@@ -87,7 +87,9 @@ jobs:
fi
- name: Run unit & integration tests
# Excludes the browser-driven mobile suite (test/mobile/**); see config/vitest.ci.config.ts.
# Excludes the suites that need chromium, per-machine PNG baselines or a
# quiet machine — see config/test-suites.ts for the list and the reason
# behind each entry. Identical to what `npm test` runs locally.
# Safe in CI: TmuxManager no-ops all shell commands under VITEST (test/setup.ts).
run: npm run test:ci
@@ -99,6 +101,11 @@ jobs:
run: npx vitest run
working-directory: packages/xterm-zerolag-input
# Note: The browser-driven mobile suite (test/mobile/**) is excluded from CI —
# it needs a live server + chromium + environment-specific PNG baselines.
# Run it locally/manually. All other tests run via the `test` job above.
# Note: three suites are excluded from CI, each with its own local runner:
# npm run test:browser Playwright + chromium (+ a live server, and a real
# codex binary for codex-predictive-echo)
# npm run test:mobile the above plus environment-specific PNG baselines
# npm run test:perf wall-clock benchmarks; need an otherwise idle machine
# config/test-suites.ts holds the globs; the configs derive from it so the
# exclusions here and those runners cannot drift apart. Everything else runs in
# the `test` job above, which is the same thing `npm test` runs.