Files
ansiblings/README.PUBLISH.md
T
Benjamin Diedrichsen 2c1d27b4fb
Publish snapshot / snapshot (push) Failing after 37s
workflow update
2026-07-27 16:21:00 +02:00

18 KiB

Publishing

Everything about how the packages in this repository are verified, versioned and shipped. If you only want to cut a release, jump to Cutting a release.

What ships

Directory Package Binary
packages/nopy @bitstack/nopy nopy
packages/keyman @bitstack/keyman keyman

Both are ESM, both declare engines.node >= 22, and both expose a single executable through bin, so npm install -g puts nopy / keyman on the PATH. cubes/ is not a package and is never published.

The tarball contents are pinned by files: ["dist", "README.md", "LICENSE"] — sources and tests are not shipped. publishConfig.access: "public" is what makes a scoped package publishable to npmjs without an extra flag; the workflows pass --access public anyway.

Versions and changelogs are maintained by hand. Nothing in CI commits a version bump, opens a release PR, or pushes a tag. A release happens because you tagged a commit whose manifest already carries the version you want.

The three workflows

All three live in .gitea/workflows and run on the ubuntu-latest runner label.

Workflow Trigger Publishes
ci.yml pull requests, non-main pushes nothing
publish-snapshot.yml pushes to main both packages → Gitea, tag main
release.yml tags matching *-v* one package → Gitea and npmjs

ci.yml explicitly excludes main and all tags (branches-ignore + tags-ignore). That is not an oversight: both publish workflows carry the full gate themselves, so excluding them here means no commit is ever verified twice, while nothing is ever published from an unverified tree.

Concurrency: ci.yml cancels superseded runs for the same ref (cancel-in-progress: true); the two publish workflows never cancel, because a half-finished publish is worse than a slow queue.

ci.yml

checkout → pnpm → node → pnpm store cache → install
        → lint:ci → typecheck → test:coverage → coverage summary
        → build → npm pack --dry-run (per package) → upload coverage

The npm pack --dry-run --ignore-scripts step prints the exact file list that would be published. It is there to catch a files or bin entry pointing at something the build no longer produces — a failure that would otherwise only surface after the version is already on a registry and immutable.

Coverage HTML/JSON reports are uploaded as a coverage artifact with a 7-day retention. Both the artifact upload and the store cache are continue-on-error: true, so a runner without a cache or artifact server gets slower CI rather than broken CI.

publish-snapshot.yml

checkout → pnpm → node → cache → install
        → lint:ci → typecheck → test:coverage → coverage summary → build
        → write .npmrc → publish both packages → delete .npmrc

One job, no needs: barrier, so install and build happen exactly once and nothing has to be passed between jobs as an artifact.

release.yml

checkout → resolve tag → check secrets
        → pnpm → node → cache → install
        → lint:ci → typecheck → test:coverage → build
        → publish to Gitea → publish to npmjs → delete .npmrc
        → create the Gitea release → step summary

Tag resolution and the secret check run before anything is installed or built, so a malformed tag or a missing token fails in seconds instead of after the whole gate.

The verification gate

The same three commands guard every path into a registry:

Step Command
Lint pnpm run lint:ci
Typecheck pnpm run typecheck
Test + coverage pnpm run test:coverage

Coverage is a hard failure below 85 % branches (and 85 % functions, 80 % lines, 80 % statements). The thresholds live in coverage.thresholds in each package's vitest.config.ts, not in a CI-only flag, which means pnpm run test:coverage fails identically on a laptop, in the pre-push hook, and on the runner. Barrel files and CLI argv wiring are excluded from measurement; everything with behaviour in it is not.

The pre-push hook installed by simple-git-hooks runs exactly this gate, so a push that survives locally will not surprise you on the runner. Bypass with SKIP_SIMPLE_GIT_HOOKS=1 when you must — CI will still catch it.

pnpm run coverage:summary renders the per-package json-summary reports as a Markdown table. In CI it appends to $GITHUB_STEP_SUMMARY so the numbers show up on the run page without opening the log. It is reporting only — if: always() and continue-on-error: true — and can never be the reason a run goes red.

Versions and dist-tags

Source Version Registry dist-tag
push to main 1.0.0-main.42.g736c012 Gitea main
tag nopy-v1.2.0 1.2.0 Gitea, npmjs latest
tag nopy-v1.2.0-rc.1 1.2.0-rc.1 Gitea, npmjs next

The rule for the dist-tag is mechanical: a version containing a prerelease part (anything with a - in it) goes out as next and is marked as a prerelease on the Gitea release; anything else goes out as latest. There is no way to publish a prerelease over latest by accident.

Snapshots

Every commit that lands on main publishes both packages to the Gitea registry, versioned as:

<manifest version>-main.<run number>.g<short sha>

for example 1.0.0-main.42.g736c012. The g prefix keeps the identifier valid semver even when the abbreviated sha happens to be all digits. The run number is monotonic, so every push produces a version that has never existed before.

pnpm add @bitstack/nopy@main

Snapshots never reach npmjs and never move latest. The version is written into the manifest on the runner with npm pkg set immediately before publishing; that edit is discarded with the workspace and is never committed.

Cutting a release

  1. Bump version in packages/<pkg>/package.json.

  2. Add a changelog entry (see below).

  3. Commit, merge to main, and let the snapshot workflow go green.

  4. Tag that commit and push the tag:

    git tag nopy-v1.2.0
    git push origin nopy-v1.2.0
    

The tag name is <directory>-v<version> — the directory under packages/, not the npm name. nopy-v1.2.0, not @bitstack/nopy-v1.2.0.

The tag decides which package ships; package.json decides the version. The workflow re-reads the manifest and refuses to continue if the two disagree:

::error::Tag 'nopy-v1.2.0' asks for 1.2.0, but packages/nopy/package.json
declares 1.1.0. Bump the manifest and re-tag.

An unknown package name fails the same way. Both checks run before install, so a typo costs you seconds.

Tagging one package does not touch the other. Two packages at the same version is a coincidence, not a requirement.

What a successful run leaves behind:

  • @bitstack/<pkg>@<version> on the Gitea registry
  • the same tarball on npmjs, public, under latest or next
  • a Gitea release on the tag, with notes and an install snippet
  • a step summary with both install commands

Changelogs and release notes

Neither package has a CHANGELOG.md yet. Without one, the Gitea release body is just the install snippet — nothing fails.

When you add one, release.yml extracts the section for the version being released. The parser is deliberately dumb: it looks for the first ## heading whose text contains the version string, and takes every line until the next ## heading. Any of these work:

## 1.2.0

## [1.2.0] - 2026-07-27

## v1.2.0 — Faster inventory parsing

Version strings that are prefixes of each other are the one thing to watch: with both ## 1.2.0 and ## 1.2.0-rc.1 in the file, releasing 1.2.0 matches whichever heading comes first. Keep the newest at the top, as usual, and this resolves itself.

Secrets

Configure under Settings → Actions → Secrets, on the repository or on the organisation to share across repos.

Secret Required Purpose
NPM_TOKEN yes npmjs granular token, read-and-write on @bitstack/*
MYGITEA_NPM_TOKEN yes Gitea PAT with write:package

GITEA_TOKEN is injected into every run by Gitea itself, and the workflows fall back to it via ${{ secrets.MYGITEA_NPM_TOKEN || secrets.GITEA_TOKEN }}. On gitea.bitsquare.dev that fallback is not sufficient: the automatic token is a short-lived task token scoped to git and repo API calls, and the package registry rejects it with E401 Incorrect or missing password at the publish step, with the gate already green. MYGITEA_NPM_TOKEN is therefore required in practice. Create it under Settings → Applications → Access Tokens with the package scope set to read-and-write; its owner needs package-write on the BitSquare organisation, since the registry path is org-owned.

For npmjs, create a granular access token scoped to @bitstack/* with read-and-write permission, and set 2FA to not-required so it works unattended. npm warns against that combination and points at Trusted Publishing instead — but Trusted Publishing federates only GitHub Actions and GitLab CI/CD over OIDC, and Gitea is not a provider it accepts. A token is the only route from this runner. Scoping the token to @bitstack/* is what keeps the exposure small: a leak lets someone publish to that scope, not touch the account.

npm caps granular token lifetime at 90 days, so NPM_TOKEN needs rotating roughly quarterly. The failure mode is gentle — an expired token trips the secret check in release.yml within seconds, before anything is installed or built. If it expires between the two registries, re-running after rotation is safe; the npm view guard skips whatever already published.

release.yml refuses to start if either token is missing, and says which one.

npm --provenance is deliberately not used. It requires GitHub Actions OIDC, which Gitea has no equivalent for; passing the flag would only fail the publish. Releases are therefore unsigned in the provenance sense — the audit trail is the tag, the run log, and the Gitea release.

Registry authentication in the workflows

release.yml has to talk to two different registries about the same @bitstack scope inside one job. It does that without ever mutating ~/.npmrc:

  • each publish step writes its own credentials file, created with install -m 600 /dev/null (which both sets the mode and truncates whatever was there before);
  • npm_config_userconfig points npm at that file for the duration of the step;
  • the npmjs step therefore starts from a file that no longer contains the Gitea token;
  • a final if: always() step deletes it, so it does not survive into a later step or a cached workspace.

.npmrc, .npmrc-* and release.json are in .gitignore, so a credentials file written into the workspace can never be committed by accident.

Installing the packages

From npmjs — public, no configuration:

npm install -g @bitstack/nopy @bitstack/keyman

From the Gitea registry, which holds every snapshot plus a mirror of every release. Per-project, in the repo's .npmrc:

@bitstack:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/

Globally with credentials, in ~/.npmrc:

@bitstack:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
//gitea.bitsquare.dev/api/packages/BitSquare/npm/:_authToken=<your gitea token>

Reads only need a token if the repository is private. The registry URL is derived in the workflows as {server_url}/api/packages/{owner}/npm/, so it follows the instance and organisation automatically.

To track snapshots in another project:

pnpm add @bitstack/nopy@main

Design decisions

Every publish is idempotent. Each step asks the registry whether that exact version already exists (npm view <name>@<version>) and skips if it does. A release that publishes to Gitea and then fails on npmjs can simply be re-run: the Gitea publish is skipped, the npmjs publish proceeds. This matters because Gitea refuses to overwrite an existing version — without the check, a re-run would fail on the first registry and never reach the second.

The build is explicit, publishes are --ignore-scripts. prepack exists for humans running npm pack locally; in CI the build has already run as its own step, and repeating it inside npm publish would only cost time and add a way for a lifecycle script to change what ships after the gate looked at it.

One job per workflow. No artifact hand-off, no second install, no risk of publishing a tree that a different job built.

Caching is best-effort. actions/cache@v4 is wrapped in continue-on-error: true, and setup-node's built-in cache: is not used, so a Gitea runner without a cache backend still works.

actions/upload-artifact@v3, not v4 — v4 depends on a backend API that many Gitea runner setups do not implement.

pnpm and Node versions come from the repo. pnpm/action-setup@v4 reads packageManager from the root manifest; setup-node reads .nvmrc. There is no version pinned in the workflow files to drift out of sync.

Checking things locally

Reproduce the CI gate exactly:

pnpm install --frozen-lockfile
pnpm run lint:ci && pnpm run typecheck && pnpm run test:coverage

See what would actually be in the tarball:

pnpm run build
cd packages/nopy && npm pack --dry-run --ignore-scripts

Try the binary as an end user would get it, without publishing:

cd packages/nopy && pnpm run link:local   # build + npm link
nopy --help
npm unlink -g @bitstack/nopy

Check that a version is not already taken before you tag:

npm view @bitstack/nopy@1.2.0 version                    # npmjs
npm view @bitstack/nopy@1.2.0 version \
  --registry https://gitea.bitsquare.dev/api/packages/BitSquare/npm/

Troubleshooting

Symptom Cause and fix
Tag ... asks for X, but package.json declares Y The manifest was not bumped, or the tag is on the wrong commit. Fix the manifest, re-tag.
Tag ... names package 'foo', but packages/foo/package.json does not exist Tag prefix must be the directory name under packages/.
NPM_TOKEN secret is not set Add the secret; the run stops before installing anything.
401/403 from the Gitea registry The automatic GITEA_TOKEN lacks write:package. Add a MYGITEA_NPM_TOKEN PAT — no edit needed.
E409 Conflict / version already exists Only reachable if a version was published outside the workflow; the npm view guard covers re-runs.
Coverage step fails, thresholds look met Thresholds are per package. Read which package failed — the summary table shows both.
npm pack --dry-run step fails A files or bin path no longer exists after the build. Fix before it reaches a registry.
Snapshot workflow green, nothing installable Snapshots are only on Gitea and only under @main. Point the scope at the Gitea registry.
The release workflow did not trigger The tag must match *-v* and must be pushed (git push origin <tag>), not just created.

Recovering from a bad publish

On npmjs, a version is permanent. Do not try to re-use it — publish a patch. Meanwhile:

npm dist-tag add @bitstack/nopy@1.1.9 latest    # point users back
npm deprecate @bitstack/nopy@1.2.0 "Broken build, use 1.2.1"

npm unpublish is only possible within 72 hours and burns the version number forever; a deprecation with a working latest is almost always the better move.

On Gitea, delete the version under Packages → @bitstack/… → Settings before that exact version can be published again.

A bad tag can be moved, but only before the release workflow has published anything:

git push --delete origin nopy-v1.2.0
git tag -d nopy-v1.2.0

Once a tarball is on npmjs, the tag is a historical record — leave it and roll forward.

A bad snapshot needs no action at all: the next push to main produces a new run number and therefore a new version, and @main moves to it.