From 2810d4491f2b9c8873f5df3b54d071addd6b361e Mon Sep 17 00:00:00 2001 From: Benjamin Diedrichsen Date: Tue, 1 Sep 2026 15:28:12 +0200 Subject: [PATCH] [fix] release: stop the snapshot job starving the release job MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pushing a release pushes the branch and then the tags seconds apart. publish-snapshot.yml keys its concurrency group on the branch and release.yml keys its own on the tag, so the two never gate each other — on a single runner they race for it and the branch push always wins. On 1.0.1 the snapshot job wedged extracting a layer of the runner image, the release job never started, release.mjs gave up after its 20-minute wait, and two of three tags were left unpushed. The report still printed a bold "Done" above an empty shipped list, so it read as a success. - publish-snapshot.yml skips commits whose message starts with "release:". A snapshot of a release commit is the same tree the tag is about to publish properly, so skipping costs nothing and removes the race. - waitForRelease() offers to keep waiting instead of giving up. No timeout value survives a wedged runner, so the real choice is between asking and making the operator finish the release by hand. --yes and a non-interactive run still give up; the latter matters because confirm() answers with its default without a terminal, which would extend the deadline forever. - The final header says "Blocked" when it is, and labels the packages that did ship before the blockage. - --wait-timeout defaults to 2400s rather than 1200s. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ --- .gitea/workflows/publish-snapshot.yml | 11 +++++++ CLAUDE.md | 30 +++++++++++++---- scripts/release.mjs | 46 ++++++++++++++++++++++----- 3 files changed, 73 insertions(+), 14 deletions(-) diff --git a/.gitea/workflows/publish-snapshot.yml b/.gitea/workflows/publish-snapshot.yml index 657d748..cdda388 100644 --- a/.gitea/workflows/publish-snapshot.yml +++ b/.gitea/workflows/publish-snapshot.yml @@ -20,6 +20,17 @@ concurrency: jobs: snapshot: + # Not on a release commit. `scripts/release.mjs` pushes the branch and then + # the tags seconds apart, and this workflow's concurrency group is keyed on + # the branch while release.yml's is keyed on the tag — so the two never gate + # each other, they race for the runner, and the branch push always gets + # there first. Measured: a snapshot job that wedged pulling the runner image + # held the runner long enough for release.mjs to give up waiting on npmjs, + # leaving two of three tags unpushed. + # + # Skipping costs nothing. A snapshot of a release commit is the same tree + # the tag is about to publish properly, under a version nobody installs. + if: ${{ !startsWith(github.event.head_commit.message, 'release:') }} runs-on: ubuntu-latest env: diff --git a/CLAUDE.md b/CLAUDE.md index e3b1504..644177a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -342,12 +342,15 @@ regardless of `--tag`; on Gitea it did not exist at all. Note that `npm view ` against a registry with no `latest` tag prints nothing and exits **0**, which is why this looked like a working lookup. (`npm view @` does exit 1 for a missing version, so the workflows' idempotency guards are -fine.) All four packages were reset to `0.5.0`; `1.0.0-alpha5` stays the -numerically highest version on npmjs, so install with an explicit `@latest`. +fine.) All four packages were reset to `0.5.0`, and `nopy`, `nopy-cubes` and +`nopy-cubes-core` have since gone out as `1.0.1` — the first release where +`latest` actually moved on both registries. `keyman` is still `0.7.0` and on +neither. - Push to `main` → `publish-snapshot.yml` publishes every package to the Gitea registry as `-main..g` under the `main` dist-tag. The - version is set on the runner with `npm pkg set` and never committed. + version is set on the runner with `npm pkg set` and never committed. It skips + commits whose message starts with `release:` — see *Runner contention* below. - `git tag -v` (e.g. `nopy-v1.2.0` — the directory under `packages/`, not the npm name) → `release.yml` publishes to Gitea *and* npmjs. The tag chooses the package, `package.json` supplies the version, and the run @@ -400,12 +403,27 @@ gets `-vvv --debug`. Treat `docs/REFACTORING.md` as a plan, not a record. The publish lane has now run against the Gitea registry: all four packages are there under `@main`, and `pnpm run try:snapshot` installs them into a throwaway -project with npm and runs the binary. The npmjs lane has only ever published -`@bitsquare/nopy`; `keyman`, `nopy-cubes` and `nopy-cubes-core` have never been -released there. That used to be caught by the *check linked deps are released* +project with npm and runs the binary. The npmjs lane has published `nopy`, +`nopy-cubes` and `nopy-cubes-core` at `1.0.1`; `keyman` has never been released +there. Ordering used to be enforced by the *check linked deps are released* guard in `release.yml`; now it is `pnpm run release` that holds `nopy`'s tag back until `nopy-cubes` answers on npmjs. +### Runner contention + +`scripts/release.mjs` pushes the branch and then the tags seconds apart. +`publish-snapshot.yml` keys its concurrency group on the branch and `release.yml` +keys its own on the tag, so the two workflows never gate each other — on a +single runner they simply race for it, and the branch push always wins. The +1.0.1 release is what surfaced this: the snapshot job wedged extracting a layer +of `runner-images:ubuntu-latest`, the release job never started, `release.mjs` +gave up after its 20-minute wait, and two of the three tags were left unpushed +while the report still printed a bold **Done**. Three things changed as a +result — the snapshot job skips `release:` commits, the wait offers to keep +waiting rather than giving up (no timeout survives a wedged runner), and the +final header says **Blocked** when it is. The tags were pushed by hand +afterwards; all three packages are on npmjs. + Nothing checks that a bundle and the CLI reading it are compatible versions; `nopy.engines` was considered and deferred. `docs/CUBE-PACKAGES.md` is where all of this came from and is now a record of what was built, including what differed diff --git a/scripts/release.mjs b/scripts/release.mjs index 3d72ec6..7e1f855 100644 --- a/scripts/release.mjs +++ b/scripts/release.mjs @@ -165,10 +165,21 @@ async function versionExists(name, version) { } } -/** Polls until `name@version` resolves on npmjs, or the deadline passes. */ -async function waitForRelease(name, version, timeoutMs) { +/** + * Polls until `name@version` resolves on npmjs, offering to keep waiting when + * the deadline passes. + * + * The offer is the point. A timeout here is far more often a runner that has + * not started the job than a release that failed, and no timeout value survives + * a runner that has wedged — so the choice is between asking and making the + * operator finish the release by hand. `--yes` and a non-interactive run give + * up instead, the latter because {@link confirm} answers with its default when + * there is no terminal, which here would extend the deadline forever. + */ +async function waitForRelease(name, version, timeoutMs, mayExtend) { const started = Date.now(); const label = `${name}@${version}`; + let deadline = started + timeoutMs; process.stdout.write(` waiting for ${label} on npmjs `); for (;;) { @@ -177,9 +188,16 @@ async function waitForRelease(name, version, timeoutMs) { console.log(chalk.green(` published after ${seconds}s`)); return true; } - if (Date.now() - started > timeoutMs) { + if (Date.now() > deadline) { console.log(chalk.red(' timed out')); - return false; + if (!mayExtend || !process.stdin.isTTY) return false; + console.log( + chalk.dim(' A queued or wedged runner looks exactly like this — check the run.') + ); + if (!(await confirm(`Keep waiting for ${label}?`, true))) return false; + deadline = Date.now() + timeoutMs; + process.stdout.write(` waiting for ${label} on npmjs `); + continue; } process.stdout.write('.'); await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS)); @@ -664,7 +682,7 @@ async function release(options) { console.log(` push ${options.remote} ${branch}, then tags in the order above`); console.log( options.wait - ? ` wait for each version on npmjs (up to ${options.waitTimeout}s each)\n` + ? ` wait for each version on npmjs (asks to continue after ${options.waitTimeout}s)\n` : ' wait no — tags are pushed back to back\n' ); @@ -751,7 +769,8 @@ async function release(options) { const landed = await waitForRelease( entry.pkg.manifest.name, entry.version, - options.waitTimeout * 1000 + options.waitTimeout * 1000, + !options.yes ); if (!landed) { pending.push(entry); @@ -772,7 +791,13 @@ async function release(options) { const blocked = pending[0]; const shipped = blocked ? plan.slice(0, plan.indexOf(blocked)) : plan; - console.log(chalk.bold('\n Done\n')); + // The header has to know whether this worked. It was an unconditional 'Done' + // over a `shipped` list that is *empty* when the package that blocked is the + // first one — a success banner above a release that shipped nothing, with the + // diagnosis a screen further down. It read as success and was believed. + console.log(chalk.bold(blocked ? '\n Blocked\n' : '\n Done\n')); + + if (blocked && shipped.length > 0) console.log(chalk.dim(' Released before the blockage:\n')); for (const entry of shipped) { console.log(` ${entry.pkg.manifest.name}@${entry.version} (${distTag(entry.version)})`); console.log(chalk.dim(` npm install -g ${entry.pkg.manifest.name}@${entry.version}`)); @@ -817,7 +842,12 @@ program .option('--no-verify', 'Skip the lint/typecheck/test/build gate') .option('--no-changelog', 'Do not prompt for release notes') .option('--no-wait', 'Do not poll npmjs between tag pushes') - .option('--wait-timeout ', 'How long to wait for each version', Number, 1200) + .option( + '--wait-timeout ', + 'How long to wait before asking to keep waiting', + Number, + 2400 + ) .addHelpText( 'after', `