mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-09-30 12:39:42 +02:00
Codeman maintainer
947ff6f6fa
`npm test` ran config/vitest.config.ts, which includes the browser, visual and
perf suites. On any machine without chromium, a free port and per-machine PNG
baselines that fails ~87 tests on a clean master, so the repo's most obvious
command could not be used as a pass/fail signal. The workaround had spread into
four docs as "never run bare `npm test`" warnings.
`npm test` now runs config/vitest.ci.config.ts — byte-for-byte what CI runs — so
local green means CI green. Verified: 264 files, 5248 tests, exit 0.
The suites it leaves out are not abandoned; each has a command:
test:browser 5 Playwright files (chromium + a live server; codex-predictive-echo
also needs a real codex binary)
test:mobile unchanged — the above plus per-machine PNG baselines
test:perf 2 wall-clock benchmarks; need an otherwise idle machine
test:all the old everything-behaviour, kept reachable
test:ci is untouched (CI still calls it). test:watch and test:coverage follow
test onto the gate's config.
The more important half is the hole this closes. The exclusion list lived as
literals in one config and pointed one way only: a file excluded from CI and
added to no runner would be tested by NOTHING, silently, with every command
still green — vitest counts "no files matched a filter" as success. That is the
same shape as the #279/#280 blind spot already documented in CLAUDE.md.
So the globs moved to config/test-suites.ts, one array per REASON a suite cannot
run in CI, and all three configs derive from it. test/test-suite-partition.test.ts
then checks the arithmetic against the files on disk: it fails if any test file
is reachable by no runner, or by two. Confirmed it fires by orphaning a file and
watching it name it. The partition is exact today:
gate 264 + browser 5 + perf 2 + mobile 9 = 280 = every *.test.ts in the repo
⚠️ One sharp edge, deliberate and documented: a file filter must match its
runner. `npm test -- test/mobile/keyboard.test.ts` now matches nothing and exits
GREEN having run zero tests, because the gate's config excludes that path.
CLAUDE.md recommended exactly that command in the on-screen-keyboard note; that
line now says `npm run test:mobile -- <file>`, and the Testing section calls out
the trap, since a green run of zero tests is worse than a red one.
Docs synced: CLAUDE.md, AGENTS.md, .github/CONTRIBUTING.md, README.md,
README.zh-CN.md, and two ci.yml comments that claimed only test/mobile/** was
excluded — it is three suites, and 5 Playwright files rather than 3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
46 lines
1.9 KiB
TypeScript
46 lines
1.9 KiB
TypeScript
/**
|
|
* The test suites that `npm test` deliberately does NOT run, in one place.
|
|
*
|
|
* Why this file exists: the exclusion list used to live only in
|
|
* config/vitest.ci.config.ts, as literals. Anything excluded there was
|
|
* therefore reachable only by running the everything-config by hand and reading
|
|
* past its failures — and a newly excluded file was reachable by nothing at
|
|
* all, silently, because nothing pointed at it. Both configs now derive their
|
|
* globs from the arrays below, so adding a suite here puts it in exactly one
|
|
* runner and takes it out of exactly one gate.
|
|
*
|
|
* Adding a new test that cannot run in CI: put its glob in the array that
|
|
* describes WHY it cannot, not in whichever one is shortest.
|
|
*/
|
|
|
|
/**
|
|
* Playwright-driven: needs chromium and, in most cases, a live Codeman server
|
|
* on a real port. Deterministic where the environment provides both, which is
|
|
* why these are a runnable suite (`npm run test:browser`) rather than skipped.
|
|
*/
|
|
export const BROWSER_TEST_GLOBS = [
|
|
'test/inline-rename.test.ts',
|
|
'test/opencode-resize.test.ts',
|
|
'test/webgl-fallback.test.ts',
|
|
'test/terminal-copy-shortcut.test.ts',
|
|
'test/codex-predictive-echo.test.ts', // also needs a real codex binary
|
|
];
|
|
|
|
/**
|
|
* Wall-clock benchmarks. They assert on durations, so a loaded shared runner
|
|
* fails them for reasons that have nothing to do with the diff under test.
|
|
*/
|
|
export const PERF_TEST_GLOBS = ['test/perf-*.test.ts'];
|
|
|
|
/**
|
|
* Browser + visual regression: chromium AND environment-specific PNG baselines
|
|
* that are generated per machine. Has its own config
|
|
* (test/mobile/vitest.config.ts) because it needs serial execution, a longer
|
|
* timeout and the `pretest:mobile` vendor step — run it with
|
|
* `npm run test:mobile`, not through the configs here.
|
|
*/
|
|
export const MOBILE_TEST_GLOBS = ['test/mobile/**'];
|
|
|
|
/** Everything `npm test` skips. */
|
|
export const NON_CI_TEST_GLOBS = [...MOBILE_TEST_GLOBS, ...PERF_TEST_GLOBS, ...BROWSER_TEST_GLOBS];
|