Author SHA1 Message Date
Benjamin Diedrichsen 37566873cc release: @bitsquare/keyman@0.7.3, @bitsquare/nopy-cubes@1.0.2, @bitsquare/nopy@1.0.2, @bitsquare/nopy-cubes-core@1.0.2
Release / release (push) Successful in 1m21s
Publish snapshot / snapshot (push) Skipped
2026-09-02 14:03:38 +02:00
Benjamin Diedrichsen 70c3d1e36e remove cockpit cube
Publish snapshot / snapshot (push) Successful in 1m6s
2026-09-02 14:00:44 +02:00
Benjamin DiedrichsenandClaude Fable 5 8973ff7113 nopy: add create-cube command scaffolding a cube from bundled templates
Publish snapshot / snapshot (push) Successful in 1m17s
Gathers id, name and directory from flags or prompts (only what the flags
do not supply), then writes manifest.mjs + deploy.py from templates under
src/templates/cube. Templates are named *.example.* so the template
directory itself can never match the loader's manifest+deploy pair rule.

The scaffold refuses a directory that already holds cube files by the
loader's own patterns, checks the id against the loaded cube set (best
effort, exempting the target directory so --force re-scaffolds work), and
warns when the target lands outside every configured cube directory.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApuW1MMGK2pUQ9mTVRTxc5
2026-09-02 13:57:51 +02:00
Benjamin DiedrichsenandClaude Opus 5 568d4c83ff keyman: replace removed inquirer prompt type 'list' with 'select'
inquirer v10 removed the legacy 'list' prompt in favour of 'select', so
every list-style prompt — starting with the main menu — died with
"Prompt type \"list\" is not registered" on any real run against the
declared ^14 dependency. The tests never saw it because they all mock
inquirer.prompt, which accepts any type string. Choice shapes and the
'default' option are unchanged; 'select' takes them as-is.

Bump to 0.7.2 to ship the fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ce5atB2tXDXyz2jd9s1bqE
2026-09-02 13:50:59 +02:00
Benjamin DiedrichsenandClaude Fable 5 643d7379ba cubes: user:add gets optional PUBKEY, space-separated GROUPS, exists guard
PUBKEY defaults to empty now — empty means no key is authorised, and some
users need none. GROUPS was always split on whitespace by deploy.py, so the
comma-separated prompt label and README were documenting a bug; both now say
space-separated. The deploy script checks the Users fact up front and noops
when the user exists, since rerunning reset the password and overwrote
~/.config/fish.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ce5atB2tXDXyz2jd9s1bqE
2026-09-02 13:36:51 +02:00
Benjamin DiedrichsenandClaude Fable 5 f1cc9effa0 nopy: add init command with bundled NOPY.LLM.md guide
`nopy init` writes a starter .nopyrc.json and NOPY.LLM.md — an LLM-facing
usage guide covering cubes, config, variables, sessions, and pyinfra — into
the working directory. Existing files are skipped unless --force. The guide
ships as dist/templates/NOPY.LLM.md, resolved relative to the module so it
works from source and from an installed package alike.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ce5atB2tXDXyz2jd9s1bqE
2026-09-02 13:06:42 +02:00
Benjamin Diedrichsen 4fa69cce0d release: @bitsquare/keyman@0.7.1
Publish snapshot / snapshot (push) Skipped
Release / release (push) Successful in 1m24s
2026-09-02 08:21:36 +02:00
Benjamin DiedrichsenandClaude Opus 5 2810d4491f [fix] release: stop the snapshot job starving the release job
Publish snapshot / snapshot (push) Successful in 1m5s
Pushing a release pushes the branch and then the tags seconds apart.
publish-snapshot.yml keys its concurrency group on the branch and
release.yml keys its own on the tag, so the two never gate each other —
on a single runner they race for it and the branch push always wins.

On 1.0.1 the snapshot job wedged extracting a layer of the runner image,
the release job never started, release.mjs gave up after its 20-minute
wait, and two of three tags were left unpushed. The report still printed
a bold "Done" above an empty shipped list, so it read as a success.

- publish-snapshot.yml skips commits whose message starts with "release:".
  A snapshot of a release commit is the same tree the tag is about to
  publish properly, so skipping costs nothing and removes the race.
- waitForRelease() offers to keep waiting instead of giving up. No
  timeout value survives a wedged runner, so the real choice is between
  asking and making the operator finish the release by hand. --yes and a
  non-interactive run still give up; the latter matters because confirm()
  answers with its default without a terminal, which would extend the
  deadline forever.
- The final header says "Blocked" when it is, and labels the packages
  that did ship before the blockage.
- --wait-timeout defaults to 2400s rather than 1200s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ
2026-09-01 15:28:12 +02:00
Benjamin Diedrichsen eac44b637e release: @bitsquare/nopy-cubes@1.0.1, @bitsquare/nopy@1.0.1, @bitsquare/nopy-cubes-core@1.0.1
Publish snapshot / snapshot (push) Canceled after 37m36s
Release / release (push) Successful in 1m7s
2026-09-01 13:05:46 +02:00
Benjamin Diedrichsen a28b75f522 bump versions 2026-09-01 13:03:39 +02:00
Benjamin Diedrichsen 41f4e49aa6 add rules to claud md 2026-09-01 13:01:13 +02:00
Benjamin DiedrichsenandClaude Opus 5 05f2d6aa56 [docs] nopy: record the seven findings this branch closed
The documentation half of the same work: `DOCS-AUDIT.md` marks §1.3, §1.5,
§2.3, §4.2 (all three points), §5.1, §6.1, §6.2 and §6.5 closed, each keeping
its original text as the record with what closed it quoted underneath, and the
"suggested order of attack" is rewritten to what is actually left — §5.2, §5.3,
the two missing cube READMEs, and the two findings (§2.7, §4.4) that are stated
accurately in `docs/API.md` while the code still behaves as they describe.

`docs/API.md` drops the two entries from its *Known gaps* list that are no
longer gaps, documents the argv and the absent shell, describes the resolution
stack and the error it raises, and inverts the `.default()`/`.describe()`
warning: the order used to matter and no longer does, which is worth saying
outright since the old advice is in the reader's memory and in 15 manifests.

The README's "topological sorting" becomes "in dependency order, with cycle
detection" — the sort never existed, but until this branch neither did the
thing a sort would have been for — and `--no-history` is spelled
`--no-save-history` wherever it appears.

One line of code rides along, because it is what a `docs/API.md` note has been
asking for: `CubePackageRef` is re-exported from `src/index.ts`, so importing
`NopyConfig` from `@bitsquare/nopy` no longer gives you a type whose own
members you cannot name. The note in `docs/API.md` saying it is missing goes
with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ
2026-09-01 12:49:30 +02:00
Benjamin DiedrichsenandClaude Opus 5 b5702e423a [fix] cubes: service:autostart reads its data, and its README describes it
Closes DOCS-AUDIT §6.1 and §5.1.

**The script could not run.** It read `APP` off `host.data` and then used
`SERVICE_NAME` and `AUTOSTART` as if they were in scope, so the very first
statement — `if AUTOSTART:` — raised `NameError`; `server` was used in the else
branch but never imported. Three lines: import `server` alongside `systemd`,
read the two names next to `APP`. The logic underneath was always right.
`python3 -m py_compile` passes.

**The README documented a different cube.** It was titled "TypeStack Install
Cube" and described cloning a git repository, `yarn install`, `yarn build`,
`docker compose up -d` and PM2 — none of which this cube does, and it listed
parameters (`USER`, `REPO`, `ENV`, `NODE_PATH`) the manifest does not have,
carrying someone's private repository URL and username as defaults.

Rewritten from the manifest and the now-working script: the three parameters
that exist, and the thing the old text obscured by describing a deploy
pipeline — this cube does not create the unit file, it enables and starts one
that is already installed. `SERVICE_NAME` is documented as what it is, a label
that never reaches systemd, so getting it wrong is cosmetic rather than a cube
managing the wrong unit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ
2026-09-01 12:48:55 +02:00
Benjamin DiedrichsenandClaude Opus 5 89450cb7bc [fix] nopy: --no-save-history, so -H keeps its id
Closes DOCS-AUDIT §6.2.

Commander derives an option's destination from its long flag with `no-`
stripped, so `--no-history` wrote to the same `options.history` that
`-H, --history <id>` reads. `nopy install -H abc --no-history` set it to
`false`, the id was discarded without a word, and the run fell through to a
full interactive session instead of replaying anything.

The two cannot share a destination, so one spelling had to change, and it is
the boolean that moved: `-H <id>` is what the help text, the README and
`docs/API.md` all use, and "save history" is what the flag actually suppresses
— next to `-s, --save-session`. The old spelling now fails loudly instead of
silently.

Verified by running the CLI, since `nopy.cli.ts` is argv wiring and excluded
from coverage:

    install --no-history                          -> error: unknown option '--no-history'
    install -H nonexistent-id --no-save-history   -> Session not found: nonexistent-id

The second line is the finding: the id used to be destroyed there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ
2026-09-01 12:48:44 +02:00
Benjamin DiedrichsenandClaude Opus 5 09554b6785 [fix] nopy: find the prompt label through zod's wrappers
Closes DOCS-AUDIT §2.3.

The cube contract says each schema field is `.describe()`d and that the
description is the prompt label. Whether it was depended on the order the
manifest happened to chain in: zod 4 keys a description to the schema
*instance* and `.default()` returns a new `ZodDefault` around the described
type rather than mutating it, so the wrapper carries no description of its own
and the prompt, reading the outer node, fell back to the bare key.

    z.boolean().describe('Update package cache').default(false)  ->  'UPDATE'
    z.boolean().default(false).describe('Update package cache')  ->  the sentence

15 of the 22 core cubes are written the first way, so most prompts showed a key.
`promptLabel()` walks down through `default` / `optional` / `nullable` looking
for a description, which makes the two orders equivalent — the answer that
cannot regress, where re-ordering every manifest and hoping the next one written
gets it right can. It discriminates on `zodKind`, not `instanceof`, for the
reason recorded on that helper: a manifest built by a different zod copy fails
every `instanceof` in the module.

The mocked test asserts all four shapes, including a doubly-wrapped
`describe().optional().default()` and a field with no description at all. The
pty test is the one that carries the weight: its probe schema is written in the
losing order, and it now waits for `First value` on a real enquirer render, so
removing the unwrapping fails a test that talks to an actual terminal rather
than to a mocked `Form`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ
2026-09-01 12:48:16 +02:00
Benjamin DiedrichsenandClaude Opus 5 bb8b1bfa5c [fix] nopy: detect a dependency cycle instead of overflowing the stack
Closes DOCS-AUDIT §6.5, and the substantive half of §1.5. `docs/API.md` has
documented a circular-dependency error since it was written; nothing raised it.
Two mutually dependent cubes recursed until V8 gave up, and a `RangeError`
names no cube — it reads as a nopy crash rather than as a manifest that says
something impossible.

`BuildContext` now carries a resolution stack: `resolveCube` pushes its
(cube, host) pair, delegates the body to `visitCube`, and pops in a `finally`.
A pair re-entered while it is still on the stack raises a `NopyUsageError`
naming the whole path — `Circular dependency on host1: a → b → c → a`. The
whole path, not just the repeated cube, because dependencies are declared
dynamically and a hook may `exec` anything at all, so the edge that closed the
loop is rarely the one you would guess from the two ends.

It has to be a structure of its own. `resolvedCubes` is written by
`buildDeployCall`, which runs *after* the descent, so a cycle never reaches it;
and it cannot be widened into a "seen" set, because re-entering a *finished*
cube with different `param` overrides is exactly what a dependency or a hook is
for. That distinction is what the diamond test pins: `shared` is entered twice
under `left` and `right` and must still resolve, while `a → b → a` must not.

There is still no topological sort and there does not need to be — emission is
post-order, so the order already is a topological one. Cycle detection was the
one thing a sort would have given that the recursion did not.

Six tests: self-dependency, a three-cube loop, the loop reported as usage
rather than as a stack overflow, a loop closed by a hook's `exec` rather than a
`dependencies()` entry, the diamond, and the same cube on two hosts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ
2026-09-01 12:48:06 +02:00
Benjamin DiedrichsenandClaude Opus 5 4daf27a3cd [fix] nopy: spawn the pyinfra argv, never a shell string
Closes DOCS-AUDIT §4.2 (point 3, the last one open) and §1.3.

`executeCall` joined `DeployCall.command` and ran it through
`execa({shell: true})`, which made every value on that command line shell
syntax. The finding framed it as a quoting problem "in the password"; it was
wider than that. `--data` values were interpolated inside double quotes, so a
backtick or a `$(…)` in *any* variable value was command substitution and a `;`
ended the command and began another.

`buildDeployCall` now emits a true argv — one element per argument, nothing
pre-quoted — and the executor spawns `execa(command[0], command.slice(1))` with
no `shell` option at all. pyinfra is still found on PATH and stdio stays
inherited, so live output is unchanged.

`maskCommand()` walks the argv by position instead of pattern-matching a joined
string, which closes a leak of its own: it used to bound a secret's value on the
closing `"` the builder had written two modules away, so a value containing a
`"` leaked its own tail. It is now the only thing that turns the command back
into a string, for display, and it shell-quotes as it goes so `--print-only`
output stays pasteable.

Also in `buildDeployCall`: `logConfigToFlags()` finally has a caller (§1.3). It
was exported and unit-tested with nothing consuming it, so `log.verbosity` and
`log.debug` in `.nopyrc.json` did nothing at all. The flags are prefixed onto
the argv right after `-y`. Consequence worth knowing rather than discovering:
`packages/nopy/.nopyrc.json` has always asked for `"verbosity": "trace",
"debug": true`, so a run from that directory now really does get `-vvv --debug`.

The tests move with it — the mock is `execa(file, args, opts)` with no factory
to unwrap, and the new cases are the ones that would have caught this: an argv
element holding `$(id); rm -rf /` stays one element, a secret whose value
contains a quote is masked whole, and `execa` is asserted never to be asked for
a shell.

What remains is not fixable here: the value still reaches pyinfra on its command
line, so it is visible in `ps`. That is pyinfra's `--data` interface.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ
2026-09-01 12:47:50 +02:00
Benjamin DiedrichsenandClaude Opus 5 2019626618 [feat] release: interactive release client, drop the CI linked-deps guard
`pnpm run release` (scripts/release.mjs, zx + enquirer + commander) replaces
the hand sequence of bump, changelog, gate, tag, push. It picks packages from a
list annotated with what npmjs already has, computes versions from the manifest,
collects notes in $EDITOR seeded with the commits since the package's last tag,
and prepends them to CHANGELOG.md in the format release.yml's parser expects.

The gate (lint:ci -> typecheck -> test:coverage -> build -> verify-pack) runs
against the bumped tree *before* the commit, so a failure leaves nothing to
unpick -- it offers to restore instead. Tags go out dependency-first, and each
version is polled on npmjs before the next tag is pushed.

That polling is what lets release.yml lose its `check linked deps are released`
step: the ordering is now enforced before CI ever sees a tag, rather than after.
linked-deps.mjs stays as a hand-check. The accepted cost is that a tag pushed
some other way is no longer caught.

Three things found by running it rather than reading it:

- Tags are annotated (`-a -m`). A lightweight tag is rejected outright under
  tag.forceSignAnnotated, which is set on the machine this was written on.
- pnpm 11 forwards the `--` in `pnpm run release -- --dry-run` literally, and
  commander reads a bare `--` as "the rest are positionals". The script takes no
  positionals, so it strips it and both spellings work.
- Prompts refuse with a message naming the flag that avoids them when stdin is
  not a TTY, instead of hanging as an unsettled top-level await.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCzYTAm9QUhvLNr2EpdagJ
2026-09-01 12:31:17 +02:00
Benjamin Diedrichsen 0aa0be5542 hardening and bugfixing prior to stable release 2026-07-31 18:21:43 +02:00
Benjamin Diedrichsen ac7ea07e3c improving docker support with various fixes to support image building 2026-07-30 20:51:22 +02:00
Benjamin DiedrichsenandClaude Opus 5 da84523a6d [fix] keyman: a permission-based test the CI runner is root for
Publish snapshot / snapshot (push) Successful in 1m4s
The snapshot run for 0.7.0 failed on this one test and published nothing.
`scanPrivateKeys` classifies a file it cannot open as not-a-key, and the test
made the file unopenable with `chmod 0o000` — which stops nobody with uid 0,
and Gitea's act_runner is a container running as root. So the file was read,
recognised as a private key not named id_*, and reported as skipped.

A dangling symlink instead: ENOENT is not a permission anyone can override,
and it is a realistic ~/.ssh inhabitant. Verified by running the gate in a
node:22 container as root, where the whole workspace is now green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 20:32:36 +02:00
Benjamin DiedrichsenandClaude Opus 5 ab4bc08e50 [chore] keyman 0.7.0
Publish snapshot / snapshot (push) Failing after 1m1s
Two minors over 0.5.0, matching what PLAN.md proposed: 0.6.0 for the
behaviour changes through Phase 4 (recipient verification, 0600 plaintext,
passphrase never handled, overwrite confirmations) and 0.7.0 for the vault
layout finally honouring keysDir/tmpDir everywhere — which is a migration
for anyone on custom names, documented in the README.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 20:17:40 +02:00
Benjamin DiedrichsenandClaude Opus 5 7862fab809 [docs] keyman: rewrite the README, close the audit
Phase 9 of packages/keyman/docs/PLAN.md; closes AUDIT §5.2, §5.3, §5.4, §5.5,
and the §4 one-liners no phase had claimed (§4.1–§4.4, §3.7).

The README is the only document that ships (package.json files: dist,
README.md, LICENSE), and it described four of nine menu entries, invented key
rotation, told the user to run ssh-keygen by hand, asked them to write a
.gitignore keyman now writes, and mentioned none of the command line. It is
rewritten against the code: every operation, the rotate/retire sequence, the
id_ prefix and what happens to keys without it, installation with the scope
mapping (never a bare --registry, which would send 55 transitive dependencies
to a registry that has never heard of them), the configuration semantics
including which relative path resolves against what, and the Phase 5 migration
for a split vault.

The CLI section is helpText() verbatim, with tests/readme.test.ts asserting the
two are identical and that every menu label appears — so a flag or an operation
added later fails the gate instead of shipping undocumented. That is the part
that keeps this from drifting again.

Also: index.ts loses the bin's shebang (it is only ever imported), exports the
config types so a consumer can name what loadConfig returns, and re-exports the
update module wholesale rather than half of it by name — verified by importing
the built dist/index.js and reading its keys. The narrow surface is now a
comment stating the rule rather than an accident.

AUDIT.md marks all 30 findings closed except the second half of §1.8, keeping
each finding's text as the record with what closed it quoted underneath, the
way DOCS-AUDIT.md does. PLAN.md gains a status section naming the three
deviations. Root CLAUDE.md records the keyman architecture as it now is,
including the deliberate `resolution` divergence from nopy.

DOCS-AUDIT.md §2.10, §6.4 and the §7 keyman-config entry are amended in the
working tree but left unstaged, since that file carries unrelated WIP.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 17:57:00 +02:00
Benjamin DiedrichsenandClaude Opus 5 3436f3cbe2 [feat] keyman: key rotation, in two halves
Phase 10 of docs/PLAN.md; closes AUDIT §3.6, the README's oldest lie
("Support for key rotation", with no occurrence of "rotat" in src/).

Rotation only ever adds. `rotateKey` generates a replacement under the next
name in the series — prod → prod-2 → prod-3 — and encrypts it *alongside*
the key it replaces, so both are in the vault at once. `retireKey` is a
separate operation, and the only one in keyman that destroys an encrypted
key. The gap between the two is where the new public key gets deployed and
tested: a rotation that replaces the key in one step locks you out of the
host you were rotating for, because the replacement is not on it yet and
the only copy of the one that is has gone.

The name has to change — the vault layout derives the directory from it, so
a replacement also called `prod` *is* the `prod` entry. `nextRotationName`
skips any version already taken in the vault, in tmp or in .ssh, so it
never asks ssh-keygen to overwrite a private key in use. Retirement warns
when nothing in the vault supersedes the key and then makes the user type
its name, since that deletion is unrecoverable.

Three things extracted rather than copied: `listVaultKeys` (vault.ts) now
backs decrypt, rotate and retire; `createKeyPair` and `promptKeyOptions`
(generate.ts) are shared with rotation, which also carries the old key's
comment over as the default. Verified against the real binaries that a
hyphen-suffixed name survives ssh-keygen and age, that the vault entry
round-trips byte-identically, and that ssh-keygen writes the replacement
0600 without help.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:53:11 +02:00
Benjamin Diedrichsen 0993a4d3bb [keyman] portability, and stop keys from being silently invisible
Four things that each made keyman quietly less useful than it looked.

**Clipboard.** `pbcopy` was spawned unconditionally, with a comment admitting
it. Copy is now a list of commands per platform — pbcopy, clip, and wl-copy /
xclip / xsel tried in order on everything else, because there is no single
answer under Linux and trying them beats detecting the session type. Only an
absent tool advances to the next candidate: one that ran and refused has an
opinion. And if nothing is installed the key is printed, since "give me this
public key" is answerable without a clipboard and used to be a dead end
everywhere but macOS. Verified the round trip through real pbcopy/pbpaste.

**Home directories.** `/home/<user>` was hardcoded — wrong on the platform
this was written on. A named user is now looked for beside the current user's
home first, which is right wherever homes live together whatever that
directory is called, then in /home and /Users, and the failure names every
path tried instead of feeding a nonexistent one to readdir. For the current
user, `HOME` still wins, with `os.userInfo()` behind it: `process.env.HOME ||
''` made an unset HOME fatal, which it is not in a cron job or a container.

**Keys that are not named id_*.** A key called `deploy_ed25519` was absent
from every menu with nothing said. It still is — the vault stores
`<name minus id_>/id_<name>.age` and decrypt rebuilds the filename from the
directory, so relaxing discovery means changing the on-disk layout, which the
plan sizes as its largest single item and is not folded in here. What it does
do is say so: any file whose first line carries a private key header and whose
name lacks the prefix is now reported, per directory, with the reason. A
bounded 64-byte read, because classifying a key is no reason to load one.

**Plaintext hygiene.** A "Clear decrypted keys" entry, defaulting to no and
listing what it would delete first, and a vault `.gitignore` written on first
run covering the age identity and the tmp directory — which the README asked
the user to do by hand. Never overwritten, and silent about a configured
directory that sits outside the vault, since a .gitignore cannot speak for a
path above itself and pretending otherwise reads as protection that is absent.
2026-07-30 15:42:35 +02:00
Benjamin Diedrichsen 270cbe628a [keyman] warn on unknown config keys, report which files were read, drop the inert merge machinery
Three things about .keymanrc.json.

`z.object` strips a key it does not know, so `{"vaultroot": "…"}` was
indistinguishable from an empty file: the vault stayed at the default and
nothing said why. Now warned per file, listing the known keys, because for a
casing slip naming the alternatives is most of the help. Warned rather than
fatal — this module degrades to defaults throughout — and warned inside the
per-file loop, the only place the filename exists: z.strictObject on the
merged result cannot say which file said it. The known-key list is derived
from the schema shape, so it cannot drift.

`--print-config` now includes `configFiles`, in merge order. That was the one
question it could not answer, and it existed only as unstructured stderr from
loadConfig — the wrong half of the output for it. Assembled in
describeConfig() rather than in cli.ts, which is excluded from coverage.

And the `resolution` machinery is gone: roughly 45 lines that could not change
an outcome, because every schema property is a string and both strategies
return the child's value for primitives. Its one test passed either way.
mergeConfigs is now a spread. The divergence from nopy, where the same
machinery is load-bearing, is recorded in the comment above it.
2026-07-30 15:15:40 +02:00
Benjamin Diedrichsen 764f890900 [keyman] keep the passphrase off argv, and recover a missing .pub
Generate prompted for the passphrase itself and passed it as `-N <value>`,
so it sat in this process's argv — readable by any user on the box through
`ps` for the length of the spawn — and in keyman's memory before that.
Verified that omitting `-N` makes ssh-keygen prompt *and* confirm, so the
prompt and the flag are both gone and the spawn inherits stdio. keyman no
longer learns the passphrase, which is strictly better than handling it more
carefully, and it deletes code.

The other half is the missing `.pub`. The selection list is built from
private keys, so an orphan is offered like any other, and copyFileSync
discovered the absent sibling only *after* age had written the encrypted
key: a vault entry with no public key, and an exception that took the rest
of the batch with it. It is now derived with `ssh-keygen -y -f`, before the
vault directory is created. Verified against real binaries that the derived
key matches the original byte for byte, that an encrypted key prompts (on
stderr — hence stdout piped, stdin and stderr inherited), and that a refused
derivation degrades to storing the private key alone rather than failing.

encrypt's loop now isolates per key and reports which ones did not make it,
except for ToolNotFoundError: age missing is not a per-key problem and nine
more identical errors help nobody.

storeInVault is the shared write path both callers had a copy of. It also
undoes its own mess: age has to write into a directory that already exists,
so a failure could leave an empty directory or a truncated .age — which list
counts as a vault entry and decrypt offers. The .age is removed because we
named it, the directory only while empty, since one holding an earlier key
is not ours to delete.
2026-07-30 15:01:22 +02:00
Benjamin Diedrichsen da9df57e11 [keyman] thread the configured keys and tmp directories through encrypt/decrypt
encryptKeys and decryptKeys each took `vaultDir` and rebuilt `<vault>/keys`
and `<vault>/tmp` from it, so `keysDir` and `tmpDir` in .keymanrc.json were
honoured by main and list and silently ignored by the two operations that
write. main was also passing vaultRoot where encrypt expected the keys
directory, which put encrypted keys one level above where list looks for
them: with any config at all, a key encrypted a second ago was invisible.

Both now take keysDir and tmpDir explicitly. The decrypt location prompt
names the real directories instead of the hardcoded `vault/tmp` and
`~/.ssh`, which meant its labels were also its values — hence LOCAL_MODE.

tests/vault-layout.test.ts is the regression: encrypt then list, driven
through keyman() with only age and the prompts mocked, against a config
using keysDir `encrypted` and tmpDir `plain`. Every unit suite passed
through this bug because each was told which directory to use; the seam
between them was untested. Verified it fails when main is reverted to pass
vaultRoot.
2026-07-30 14:45:27 +02:00
Benjamin DiedrichsenandClaude Opus 5 653d348ecc [keyman] phase 4: decrypt stops destroying keys and stops the 0644 window
Verified before the fix: `age -d -o <existing>` overwrites without a word
("PRECIOUS EXISTING KEY" became "secret"), and the old `cp` for the public
key did the same. Decrypting a vault entry on top of a newer working key
in ~/.ssh destroyed it with no prompt, no backup and no mention. It is the
only finding in the audit that loses data the user never asked to touch.

Every collision — private and public, both output modes — is now settled
before anything is written, so the questions are asked about files that
still exist. Default is to keep what is there.

cp and chmod are gone. Three spawns per key become one, it works where
those binaries do not, and the chmod happens in-process immediately after
age returns: age creates its output 0644 regardless of umask, so a
plaintext private key was world-readable for the length of two spawns and
stayed 0644 whenever the chmod itself failed. ~/.ssh is created 0700 when
absent rather than assumed.

decrypt.test.ts stops asserting on which binaries were spawned. The age
stand-in now writes its -o file at 0644 the way age does, and the tests
assert the bytes and the mode on disk — the outcome rather than the
mechanism.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 14:39:30 +02:00
Benjamin DiedrichsenandClaude Opus 5 77bd43818f [keyman] phase 3: derive the age recipient, and survive not having one
main.ts asserted the recipient non-null twice — extractAgePublicKey(...)!
— and the type already said null was possible. With no age.key the vault
encrypted to the string "null": execa stringifies it, age exits 1, and on
the generate path that happens *after* ssh-keygen has written a plaintext
private key into tmpDir, so the user is told the operation failed and left
with a key on disk. Now the recipient is resolved once, remembered on
success, and a null prints the remedy (age-keygen -o <path>) and returns
to the menu. list, copy and decrypt still work without one.

extractAgePublicKey now derives the public key with `age-keygen -y`
instead of scraping the `# public key:` comment. The comment is ordinary
text nothing re-checks; verified that rewriting it does not change what
-y reports, so a stale or forged comment silently encrypted the vault to
a recipient nobody holds the private half of.

The comment survives as a fallback for a machine with no age-keygen,
behind a warning that it is unverified — but not when age-keygen runs and
refuses the file. That means age cannot read the identity, and trusting
the comment there would encrypt to a recipient the vault could never
decrypt with.

runTool throws ToolNotFoundError for ENOENT so the two cases can be told
apart. Its own tests move to tool.test.ts, which keeps real processes;
utils.test.ts mocks execa, since the gate cannot require age installed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 14:37:26 +02:00
Benjamin DiedrichsenandClaude Opus 5 11c323b715 [keyman] phase 2: guard the directories nothing creates
encrypt read ~/.ssh and the tmp directory, and decrypt read <vault>/keys,
with no existsSync between them. main.ts created vaultRoot and tmpDir but
never keysDir, so decrypt on a fresh vault threw ENOENT instead of
printing the "no encrypted keys" message it already had — the message was
unreachable until something else created the directory.

Both functions now fall through to their warning. main.ts creates all
three directories, 0700: the vault holds the age identity and tmp holds
plaintext private keys.

age spawns go through runTool, which separates "not installed" (ENOENT,
whose message is `spawn age ENOENT`) from "age refused" (whose reason is
on stderr and nowhere in the thrown message). Tested against real
processes, not a mocked execa — the shape of the failure is the point.

list.ts kept statSync rather than switching to withFileTypes as planned:
withFileTypes reports a symlinked key directory as a link and would have
silently dropped it. `throwIfNoEntry: false` fixes the dangling-symlink
throw and keeps following the good ones. Both cases now have a test.

Also deletes the three debug logs (encrypt.ts printed both key arrays,
decrypt.ts printed every candidate path from inside a filter).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 14:33:26 +02:00
Benjamin DiedrichsenandClaude Opus 5 8fa0cfa271 [keyman] audit + remediation plan, and phase 1: CLI error boundary
docs/AUDIT.md and docs/PLAN.md record the review and the ten phases it
turns into. This commit is phase 1.

keyman.cli.ts fell through to an interactive session for --help, ignored
unknown flags, and called keyman() unawaited — so Ctrl-C at any prompt,
and any rejection inside the menu loop, became an unhandled-rejection
stack trace. flagValue() also read `--channel --force` as the channel
"--force", which reached the dist-tag lookup as a key that cannot exist
and reported an unreachable registry.

New keyman.args.ts owns the parse: both --flag value and --flag=value, a
UsageError for an unknown flag or command, --channel validated against
the three real channels, and self-update-only flags rejected rather than
silently ignored. It is a separate module because cli.ts is excluded from
coverage and these are rules, not wiring. --help short-circuits before
tokenising, so it answers a line the parser would otherwise reject.

Usage errors exit 2; ExitPromptError is caught by name (@inquirer/core is
transitive here and does not resolve) and prints Goodbye.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 14:17:52 +02:00
Benjamin Diedrichsen 75983ab3b1 make stable vagrant machine host key for better dx on vagrant vm spawning 2026-07-29 14:25:34 +02:00
112 changed files with 11513 additions and 1522 deletions
+11
View File
@@ -20,6 +20,17 @@ concurrency:
jobs:
snapshot:
# Not on a release commit. `scripts/release.mjs` pushes the branch and then
# the tags seconds apart, and this workflow's concurrency group is keyed on
# the branch while release.yml's is keyed on the tag — so the two never gate
# each other, they race for the runner, and the branch push always gets
# there first. Measured: a snapshot job that wedged pulling the runner image
# held the runner long enough for release.mjs to give up waiting on npmjs,
# leaving two of three tags unpushed.
#
# Skipping costs nothing. A snapshot of a release commit is the same tree
# the tag is about to publish properly, under a version nobody installs.
if: ${{ !startsWith(github.event.head_commit.message, 'release:') }}
runs-on: ubuntu-latest
env:
+8 -26
View File
@@ -125,32 +125,14 @@ jobs:
- name: Install
run: pnpm install --frozen-lockfile
- name: Check the linked workspace packages are already released
env:
NAME: ${{ steps.target.outputs.name }}
DIR: ${{ steps.target.outputs.dir }}
run: |
set -euo pipefail
# `pnpm publish` turns `workspace:*` into the version the linked
# package declares at this commit. If that version is not on the
# registry yet, the release installs to a broken tree — and npmjs
# only lets you unpublish for 72 hours. Release the dependency first:
# nopy-cubes, then nopy, then any bundle.
#
# npmjs only: it is the irreversible one, and it needs no credentials
# to read, which this step does not have yet.
missing=0
for spec in $(node scripts/linked-deps.mjs "$DIR" | tr ' ' '@'); do
# Scoped, not `--registry`: `@scope:registry` outranks it, so a bare
# flag can be silently overridden by any project-level .npmrc.
if npm view "$spec" version --@bitsquare:registry="$NPMJS_REGISTRY" >/dev/null 2>&1; then
echo "${spec} is published"
else
echo "::error::${NAME} depends on ${spec}, which is not on npmjs. Release it first."
missing=1
fi
done
exit "$missing"
# There used to be a *check linked deps are released* step here, refusing
# to publish a package whose `workspace:` dependency was not yet on npmjs.
# It was removed: `scripts/release.mjs` is what creates release tags now,
# and it already pushes them dependency-first and waits for each version to
# resolve on npmjs before pushing the next — so the ordering is enforced
# before CI ever sees a tag, rather than after. `node scripts/linked-deps.mjs
# <dir>` still prints what a package would bake in, if you want to check by
# hand. A tag pushed some other way is no longer caught.
- name: Lint
run: pnpm run lint:ci
+6
View File
@@ -1,6 +1,12 @@
.vault
.vagrant
# Persistent SSH host key for the dev VM — a real private key, and machine-local
# anyway (see the Vagrantfile).
.vagrant-hostkeys
.python-version
# The pty drivers under scripts/ import each other, so running one leaves a
# bytecode cache next to them.
__pycache__/
node_modules
cache
+102 -29
View File
@@ -2,6 +2,10 @@
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Rules
- Do not create branches without being asked
## What this repo is
A pnpm workspace holding two independently published CLIs, the authoring package
@@ -142,13 +146,27 @@ One pass per invocation, `nopy.main.ts` orchestrating:
variables (prompt, or read them back from the session on replay) → run
`before` hooks → resolve `manifest.dependencies(vars)` (dynamic: it receives
the *collected* variables) → emit the deploy call → run `after` hooks. There
is no separate topological sort; ordering falls out of the recursion, and a
`${cubeId}:${host}` set makes emission idempotent. Hooks get a `HookContext`
is no separate topological sort; emission is post-order, so the ordering *is*
topological without an algorithm computing it, and a `${cubeId}:${host}` set
makes emission idempotent. Hooks get a `HookContext`
whose `exec(id, vars)` re-enters `resolveCube`, so a hook can pull in a cube
that is not a declared dependency.
6. **`nopy.executor.ts`** — runs the built `pyinfra <host> -y --data K=V ... --chdir <cubeDir> <script>`
Cycles are caught by a separate **resolution stack** — a (cube, host) pair
re-entered while still resolving raises with the whole path named. It has to
be separate from `resolvedCubes`, which is written *after* the descent and so
never sees a cycle at all, and which cannot be widened into a "seen" set
because re-entering a finished cube with different `param` overrides is
exactly what a dependency or a hook is for.
6. **`nopy.executor.ts`** — runs the built
`pyinfra <host> -y [-vv] [--debug] --data K=V ... --chdir <cubeDir> <script>`
commands through execa with inherited stdio, sequentially, stopping at the
first failure unless `continueOnError`.
first failure unless `continueOnError`. `DeployCall.command` is a true argv
and is spawned **without a shell**: it used to be joined into one string and
run through `execa({shell: true})`, which made every `--data` value shell
syntax — a password or a variable holding `;` or `$(…)` was executed. The only
thing that joins it back into a string is `maskCommand()`, for display, which
shell-quotes as it goes so `--print-only` output stays pasteable. The
verbosity/debug flags come from `config.log` through `logConfigToFlags()`.
### Variables
@@ -214,13 +232,36 @@ out and watching them stay green.
## keyman architecture
Much smaller: `keyman.cli.ts` (argv, plus a `--print-config` escape hatch) →
`keyman.main.ts`, an inquirer menu loop dispatching to one module per operation
(`list`/`copy`/`generate`/`encrypt`/`decrypt`). `keyman.config.ts` mirrors nopy's
upward-traversal + `resolution` merge for `.keymanrc.json`, but validates the
result with Zod and falls back to defaults instead of throwing. `VAULT_ROOT` in
the environment beats the config file. Encryption shells out to `age` /
`age-keygen` / `ssh-keygen`, which must be on `PATH`.
Much smaller: `keyman.cli.ts` (wiring only — argv parsing lives in
`keyman.args.ts`, which is covered, and the CLI is the error boundary that turns a
`UsageError` into one line instead of a stack trace) → `keyman.main.ts`, an
inquirer menu loop dispatching to one module per operation
(`list`/`copy`/`generate`/`encrypt`/`decrypt`/`rotate`/`retire`/`clear`).
`keyman.config.ts` mirrors nopy's upward traversal for `.keymanrc.json` but not
its `resolution` merge: every keyman property is a string, so a child simply wins
and the strategies could not change an outcome — see `docs/AUDIT.md` §3.3. It
validates with Zod, falls back to defaults instead of throwing, and warns about a
key it does not know rather than letting Zod strip it silently. `VAULT_ROOT` in
the environment beats the config file. It shells out to `age`, `age-keygen`
(`-y`, to derive the recipient from the identity rather than trusting the
`# public key:` comment) and `ssh-keygen` (`-y`, to recover a missing `.pub`),
which must be on `PATH`; `runTool` tells a missing binary apart from a refusing
one. Nothing shells out to `cp` or `chmod` any more — `decrypt` copies and
chmods in-process, because the old spawn left a private key at age's 0644 for the
length of two processes.
The write path is one function, `storeInVault` (`keyman.vault.ts`), shared by
`encrypt`, `generate` and `rotate`; `listVaultKeys` is the one reader of the
`<keysDir>/<name>/id_<name>.age` layout. Rotation is deliberately two operations
(`rotate` adds a replacement under the next name in the series, `retire` deletes
the superseded key), because a rotation that replaces the key in place locks you
out of the host it was for. keyman never handles a passphrase: `ssh-keygen`
prompts for it with stdio inherited, since `-N <value>` put it in argv where `ps`
could read it.
`docs/AUDIT.md` is a full audit of the package with each finding marked closed as
it landed, and `docs/PLAN.md` the ten phases that closed them. Both are records
now, not plans.
### Updating
@@ -254,6 +295,14 @@ another edge to the publish order. Extract it if a third CLI appears.
Tag-driven, one package at a time; see `README.PUBLISH.md`.
`pnpm run release` (`scripts/release.mjs`, zx + enquirer + commander) is the
front door: pick packages, pick versions, write notes into `CHANGELOG.md`, run
the gate **against the bumped tree before committing** so a failure leaves
nothing to unpick, then commit, tag and push. Tags go out dependency-first and it
polls npmjs for each version before pushing the next — which is what replaced the
CI-side linked-deps guard. Tags are annotated (`-a -m`), not lightweight: a
lightweight tag is rejected outright under `tag.forceSignAnnotated`.
### Registry resolution
The repo commits a root `.npmrc` mapping `@bitsquare:registry` to the Gitea
@@ -293,12 +342,15 @@ regardless of `--tag`; on Gitea it did not exist at all. Note that `npm view
<name>` against a registry with no `latest` tag prints nothing and exits **0**,
which is why this looked like a working lookup. (`npm view <name>@<version>`
does exit 1 for a missing version, so the workflows' idempotency guards are
fine.) All four packages were reset to `0.5.0`; `1.0.0-alpha5` stays the
numerically highest version on npmjs, so install with an explicit `@latest`.
fine.) All four packages were reset to `0.5.0`, and `nopy`, `nopy-cubes` and
`nopy-cubes-core` have since gone out as `1.0.1` — the first release where
`latest` actually moved on both registries. `keyman` is still `0.7.0` and on
neither.
- Push to `main` → `publish-snapshot.yml` publishes every package to the Gitea
registry as `<version>-main.<run>.g<sha>` under the `main` dist-tag. The
version is set on the runner with `npm pkg set` and never committed.
version is set on the runner with `npm pkg set` and never committed. It skips
commits whose message starts with `release:` — see *Runner contention* below.
- `git tag <dir>-v<version>` (e.g. `nopy-v1.2.0` — the directory under
`packages/`, not the npm name) → `release.yml` publishes to Gitea *and* npmjs.
The tag chooses the package, `package.json` supplies the version, and the run
@@ -334,22 +386,43 @@ Three things the `workspace:*` links added, all of them non-obvious:
`scripts/publish-order.mjs` topologically sorts over the `workspace:` edges;
the snapshot workflow stamps *every* version first and only then publishes in
that order, because `pnpm publish` reads the linked package's version at pack
time. `release.yml` additionally refuses to ship a package whose linked
dependency is not yet on npmjs (`scripts/linked-deps.mjs`) — npmjs is the
registry you cannot take a mistake back from.
time. `release.yml` used to additionally refuse to ship a package whose linked
dependency was not yet on npmjs (`scripts/linked-deps.mjs`); that step is gone,
and `scripts/release.mjs` enforces the same ordering earlier instead — it
pushes tags dependency-first and polls npmjs for each version before pushing
the next. `linked-deps.mjs` survives as a hand-check. A tag pushed some other
way is no longer caught, which is the accepted cost.
## Known drift
`logConfigToFlags()` is exported and tested but nothing feeds its output into the
built pyinfra command, so `log.verbosity` / `log.debug` in `.nopyrc.json`
currently have no effect. Treat `docs/REFACTORING.md` as a plan, not a record.
`logConfigToFlags()` is now consumed by `buildDeployCall`, so `log.verbosity` /
`log.debug` in `.nopyrc.json` finally do what the README says. Note the
consequence: `packages/nopy/.nopyrc.json` has always asked for
`"verbosity": "trace", "debug": true`, and a run from that directory now actually
gets `-vvv --debug`. Treat `docs/REFACTORING.md` as a plan, not a record.
The publish lane has now run against the Gitea registry: all four packages are
there under `@main`, and `pnpm run try:snapshot` installs them into a throwaway
project with npm and runs the binary. The npmjs lane has only ever published
`@bitsquare/nopy`; `keyman`, `nopy-cubes` and `nopy-cubes-core` have never been
released there, so the *check linked deps are released* guard in `release.yml`
will stop the first `nopy` release until `nopy-cubes` ships.
project with npm and runs the binary. The npmjs lane has published `nopy`,
`nopy-cubes` and `nopy-cubes-core` at `1.0.1`; `keyman` has never been released
there. Ordering used to be enforced by the *check linked deps are released*
guard in `release.yml`; now it is `pnpm run release` that holds `nopy`'s tag back
until `nopy-cubes` answers on npmjs.
### Runner contention
`scripts/release.mjs` pushes the branch and then the tags seconds apart.
`publish-snapshot.yml` keys its concurrency group on the branch and `release.yml`
keys its own on the tag, so the two workflows never gate each other — on a
single runner they simply race for it, and the branch push always wins. The
1.0.1 release is what surfaced this: the snapshot job wedged extracting a layer
of `runner-images:ubuntu-latest`, the release job never started, `release.mjs`
gave up after its 20-minute wait, and two of the three tags were left unpushed
while the report still printed a bold **Done**. Three things changed as a
result — the snapshot job skips `release:` commits, the wait offers to keep
waiting rather than giving up (no timeout survives a wedged runner), and the
final header says **Blocked** when it is. The tags were pushed by hand
afterwards; all three packages are on npmjs.
Nothing checks that a bundle and the CLI reading it are compatible versions;
`nopy.engines` was considered and deferred. `docs/CUBE-PACKAGES.md` is where all
@@ -360,9 +433,9 @@ from the plan.
`src/index.ts` plus the authoring package; its *Known gaps* section is the short
list of behaviour that surprises a reader (`--json` printing nothing on success,
`DeployCall.dependencies` always empty, `ExecutionResult.stdout` never populated,
no cycle detection, and `self-update` reporting an empty dist-tag as an
unreachable registry). `CubePackageRef` is referenced by the exported
`NopyConfig` but is not itself re-exported, so a consumer cannot name the type —
one line, not yet fixed. `DOCS-AUDIT.md` tracks the drift in the remaining
and `self-update` reporting an empty dist-tag as an unreachable registry).
`DOCS-AUDIT.md` tracks the drift in the remaining
documents; §2.9 (the nopy README shipping yarn-workspace instructions to npmjs)
is closed, so the keyman README (§2.10) is now the worst of them.
and §2.10 (the keyman README describing four of nine operations and inventing a
tenth) are both closed. The keyman README now quotes `helpText()` verbatim and a
test fails if the two diverge, which is the shape worth copying for nopy.
+349 -54
View File
@@ -15,21 +15,48 @@ Verified against the working tree at commit `fcc1817`. Line numbers are from tha
state.
Findings closed since are marked **✅ … fixed** and keep their original text as
the record of what was wrong. So far: §1.1 (`--use-defaults`), §2.2
(`getDefaults()`), §2.1 (precedence — the second half closed differently than
proposed), §3 in full (`docs/API.md`, regenerated), §4.2 (password on stdout —
points 1 and 2 of 3), §4.3 (what a session records), §2.9 (the nopy README's
yarn install instructions), and one bullet of §6.4.
the record of what was wrong. So far: §1.1 (`--use-defaults`), §1.3 (`log.*`),
§1.5 (topological order, both halves), §2.2 (`getDefaults()`), §2.1 (precedence —
the second half closed differently than proposed), §2.3 (the prompt label lost to
`.default()`), §3 in full (`docs/API.md`, regenerated), §4.2 (password on stdout,
all three points), §4.3 (what a session records), §2.9 (the nopy README's
yarn install instructions), §2.4 (`version` / `timestamp`, implemented rather
than deleted), §2.5 (`listSessions`' filename filter), §4.5 (`-s` on a replay,
plus `-l` and history), §5.1 and §6.1 (`service/autostart`, the README and the
script), §6.2 (`-H <id>` versus `--no-history`), §6.5 (cycle detection), §6.7 (a
secret written to the session in plaintext), and one bullet of §6.4.
Closing §3 also settled the documentation half of several findings elsewhere
without touching their underlying cause: §1.2, §1.3, §1.5, §2.3, §2.7, §4.4 and
§6.5 are each now stated accurately in `docs/API.md`, but the code still behaves
as those findings describe and they stay open.
without touching their underlying cause. Two of those are still in that state:
§2.7 and §4.4 are stated accurately in `docs/API.md`, but the code still behaves
as they describe and they stay open. The other four — §1.3, §1.5, §2.3 and §6.5 —
have since been closed in the code as well. §1.2 was closed outright by removing
the flag.
## Where the drift is
A field run against a fresh VM sorted the findings for us, and they cluster on
one seam. **Everything a human reads on screen matched the documentation.
Everything machine-facing had drifted** — `--json`, the session format, `-s` on a
replay, `-l` and history, `runtime:nodevm`'s parameters, the bundle's install
command.
That is not random rot. The interactive surface is maintained by daily use: a
wrong prompt label is noticed the next time someone runs the thing. The scripting
surface was documented from intent and then never exercised, so nothing pushed
back when it changed or was never built.
Corrections come in the two shapes that distinction implies. `--json` was
documented from intent and never built, so it was removed (§1.2). The rest was
built and then drifted, so it was fixed. Keeping it fixed means what remains of
the scripting surface — `--print-only`, sessions, history — needs tests that
assert on **stdout**, not prose.
---
## Contents
- [Where the drift is](#where-the-drift-is)
- [1. Documented features that do not exist](#1-documented-features-that-do-not-exist)
- [2. Documented behaviour that differs from the code](#2-documented-behaviour-that-differs-from-the-code)
- [3. ✅ `docs/API.md` — systematic drift — fixed](#3--docsapimd--systematic-drift--fixed)
@@ -82,7 +109,7 @@ nopy.cli.ts:95 useDefaults: options... # passed
cubes/dependencies.ts:35 useDefaults?: boolean; # declared — and that is all
```
### 1.2 🔴 `-j, --json` produces no output on success
### 1.2 🔴 `-j, --json` produces no output on success — **closed by removal**
| | |
|---|---|
@@ -104,7 +131,30 @@ Related: `--dry-run --json` prints the **text** plan, not JSON.
argument (`nopy.executor.ts:172`), even though the function supports it
(`nopy.executor.ts:110`).
### 1.3 🟠 `log.verbosity` and `log.debug` have no effect
**Closed by deleting the flag, not by implementing it.** `executeDeployCalls`
runs pyinfra with *inherited* stdio, so during a run nopy does not own its own
stdout — pyinfra does. A JSON blob appended after an unbounded amount of another
process's output is not machine-readable by any definition a caller could rely
on; making it so means capturing pyinfra's output and giving up live progress.
Same root cause as `ExecutionResult.stdout` never being populated. What replaced
it is a promise a test can hold to: **stdout carries the deploy commands and
pyinfra's own output, everything nopy says about itself goes to stderr**, and the
exit code is the verdict. `nopy history --json` is a different flag, it works,
and it stays.
### 1.3 ✅ `log.verbosity` and `log.debug` have no effect — **fixed**
> **Closed by implementing it, not by deleting the documentation.** The two
> tables in the README were accurate about pyinfra's flags and about the mapping
> `logConfigToFlags()` already computed — the only missing step was one call.
> `buildDeployCall` now prefixes `logConfigToFlags(this.config.log)` onto the
> pyinfra argv, right after `-y`. Verified against a real `pyinfra` binary, which
> accepts `-vv --debug` in that position.
>
> Worth knowing rather than discovering: `packages/nopy/.nopyrc.json` asks for
> `"verbosity": "trace", "debug": true`, and it now gets `-vvv --debug`. The
> value was left alone — it says what its author meant, and it never did anything
> until now.
Pre-existing known drift, recorded in `CLAUDE.md`, but the README still presents
it as a working feature — two tables, a recommendation paragraph, and a slot in
@@ -124,7 +174,25 @@ This is live in the repo's own config: `packages/nopy/.nopyrc.json` sets
the current interface at `cubes/types.ts:43-56`, which has `id`, `name`,
`schema`, `dependencies`, `before`, `after` and nothing else.
### 1.5 🟠 Topological sorting
### 1.5 ✅ Topological sorting — **closed, both halves**
> **The vocabulary complaint was answered by §6.9; the substantive one is now
> fixed in code.** There is still no sort pass, and there does not need to be:
> emission is post-order, so the output *is* a topological order of the graph
> (§6.9 records the measurement and two tests pin it). What the finding was right
> about is that a sort detects cycles and this did not.
>
> `BuildContext` now carries a **resolution stack** alongside `resolvedCubes`: a
> (cube, host) pair re-entered while it is still resolving raises a
> `NopyUsageError` naming the whole path — `Circular dependency on host1:
> a → b → c → a`. It has to be a separate structure. `resolvedCubes` is written
> *after* the descent, so a cycle never reaches it, and it cannot be widened into
> a "seen" set because re-entering a *finished* cube with different `param`
> overrides is precisely what a dependency or a hook is for. Six tests cover it,
> including a loop closed by a hook's `exec` rather than a `dependencies()`
> entry, and a diamond that must still be allowed.
>
> The README claims are reworded: "in dependency order, with cycle detection".
`README.md:11` ("Dependency resolution with **topological sorting**"),
`README.md:25` ("Topologically sorts cubes based on dependencies") and
@@ -242,7 +310,20 @@ Three cubes in this repo are in that state today:
The failure is silent — no error, no warning, just a pyinfra run with an empty
data set.
### 2.3 🔴 `.describe()` before `.default()` loses the prompt label
### 2.3 ✅ `.describe()` before `.default()` loses the prompt label — **fixed**
> **Closed the one-line way, not by re-ordering 15 manifests.** `nopy.prompts.ts`
> has a `promptLabel()` that walks down through `default` / `optional` /
> `nullable` wrappers looking for a description, so both chaining orders now give
> the sentence and neither can regress. Discriminates on `zodKind`, not
> `instanceof`, for the reason recorded on that function.
>
> Proven twice over. The mocked test asserts all four shapes —
> `describe().default()`, `default().describe()`, a doubly-wrapped
> `describe().optional().default()`, and a field with no description at all. The
> **pty** test is the one that matters: its probe schema is written in the losing
> order and it now waits for `First value` on a real enquirer render, so removing
> the unwrapping fails a test that talks to an actual terminal.
`CLAUDE.md` and the cube contract state that each schema field is `.describe()`d
and "the description is the prompt label". `nopy.prompts.ts:184-185` reads it as:
@@ -275,7 +356,19 @@ the two documents disagree, and neither mentions that it matters.
`net:tailscale` (all 4 fields), `runtime:nodevm` (all 4), `user:add` (all 4),
`ssh:keygen` (all 4) and `admin:locale` (all 4).
### 2.4 🟠 Session files claim `version` and `timestamp` fields
### 2.4 ✅ Session files claim `version` and `timestamp` fields — **fixed**
Closed by implementing them rather than deleting the claim. `createSession`
stamps `version: '1.0.0'` (exported as `SESSION_VERSION`) and an ISO
`timestamp`; `nopy()` fills in a default `name` at save time, using the same
`describeSession()` the history list uses — one implementation, so the two
cannot drift. `loadSession` still requires only `cubes` and `auth`, so every
session written before this, and every hand-written one, keeps loading; an
unrecognised `version` is a warning on stderr, never a refusal. The interface
and both documents now mark the three fields optional, which is what they are.
The original finding follows.
`README.md:179-181` shows a session with `"version": "1.0.0"` and
`"timestamp": "2025-10-13T10:30:00Z"`, and `docs/SESSION_FORMAT.md:305-306`
@@ -293,7 +386,14 @@ Consequence: a `version` field implies a compatibility check that does not exist
Nothing reads it, so an incompatible old session fails later and more obscurely
than a version check would.
### 2.5 🟠 Session filename convention does not match `listSessions()`
### 2.5 ✅ Session filename convention does not match `listSessions()` — **fixed**
`listSessions` now matches `*.nopysession.json` and `*.nopysession.mjs` as well
as the two shorter suffixes — `saveSession` writes whatever path it is handed,
so files under the old name exist and there was no reason to stop finding them.
The original finding follows.
The READMEs consistently use `*.nopysession.json` (`README.md:223`, `330`, `338`;
`docs/DOCKER.md:54`; the shipped `example.nopysession.json`).
@@ -310,7 +410,15 @@ file naming the README recommends is invisible to the function documented at
`docs/API.md:430`. `loadSession` is unaffected (it switches on `.json`/`.mjs`),
so this only bites the listing API.
### 2.6 🟠 `docs/DOCKER.md` container name contradicts the file it points at
### 2.6 ✅ `docs/DOCKER.md` container name contradicts the file it points at — **fixed**
> **Resolved.** `example.nopysession.json` now targets
> `@docker/nopy-test-container`, matching the guide, `packages/nopy/.nopyrc.json`
> and the session example in the nopy README. `nopy-test-ubuntu` was the outlier.
> Worth knowing why it silently "worked": an identifier with no matching
> container is read as an *image*, so the run built and committed a throwaway
> image instead of failing — the mode `docs/DOCKER.md` now documents at the end.
> The finding below is kept as the record of what was wrong.
`docs/DOCKER.md:35` and `:45`:
@@ -372,7 +480,18 @@ this is what a reader sees on npmjs.com — build-from-monorepo instructions
instead of `npm install -g @bitsquare/nopy`, which is what the root README and
`README.PUBLISH.md:314` correctly tell people to run.
### 2.10 🟠 keyman README: two operations missing, one operation invented
### 2.10 ✅ keyman README: two operations missing, one operation invented — **fixed**
> **Resolved.** The README was rewritten against the code (`packages/keyman/docs/PLAN.md`
> Phase 9). All nine menu entries are documented, and a test asserts it contains
> every label `keyman.main.ts` offers, so a tenth cannot arrive undocumented.
> Encrypt is described as the union of `~/.ssh` and the tmp directory, which is
> what it does. The Quick Start now points at the Generate operation instead of
> `ssh-keygen`. Rotation stopped being an invention in Phase 10: it exists, in two
> halves (`keyman.rotate.ts`), and the README documents the sequence. The whole CLI
> surface is there too — `helpText()` quoted verbatim, with a test that fails if the
> two diverge — which was the other half of this, tracked as
> `packages/keyman/docs/AUDIT.md` §5.3. The finding below is kept as the record.
`packages/keyman/README.md:90-96` lists four menu entries: List, Encrypt,
Decrypt, Quit. The menu (`keyman.main.ts:54-61`) has six:
@@ -403,7 +522,9 @@ Root `README.md:57-58` describes "a hard **85 % branch** floor". Both
the root README is the odd one out, and it is the file a new contributor reads
first.
### 2.12 🟡 `docs/DOCKER.md` relative link is broken
### 2.12 ✅ `docs/DOCKER.md` relative link is broken — **fixed**
> **Resolved.** The link is now `../README.md`.
`docs/DOCKER.md:8` links `[README.md](./README.md)`, which resolves to
`packages/nopy/docs/README.md` — nonexistent. It should be `../README.md`.
@@ -422,8 +543,7 @@ first.
>
> Three things were deliberately added rather than merely corrected. A
> **Known gaps** section states the behaviour a reader would otherwise take on
> trust — `logConfigToFlags` being unconsumed (§1.3), `--json` printing nothing
> on success (§1.2), the absent cycle detection (§1.5, §6.5), `DeployCall.dependencies`
> trust — `logConfigToFlags` being unconsumed (§1.3), the absent cycle detection (§1.5, §6.5), `DeployCall.dependencies`
> always being `[]`, `ExecutionResult.stdout`/`stderr` never being populated, and
> hook variables not being schema-validated (§2.7). The `.describe()`/`.default()`
> ordering hazard (§2.3) is called out where the manifest example lives, with the
@@ -539,9 +659,30 @@ Because `loadCubes` turns each failure into an `errors` entry and `nopy.main.ts:
aborts when `errors.length > 0`, a fresh clone cannot run a single cube. Neither
README mentions a setup step.
### 4.2 🟠 The SSH password is printed in plaintext — **mostly fixed**
### 4.2 ✅ The SSH password is printed in plaintext — **fixed as far as it can be**
> **Points 1 and 2 resolved; point 3 stands.** `maskCommand()`
> **Point 3 is now closed too, and it was worse than the finding said.** The
> command is no longer a shell string. `buildDeployCall` emits a true argv — one
> element per argument, nothing pre-quoted — and `executeCall` spawns it as
> `execa(command[0], command.slice(1))` with no `shell` option at all.
>
> The finding called the quoting a vulnerability "in the password". It was not
> limited to the password: with `shell: true` the whole joined string was parsed
> by `sh`, and `--data` values were interpolated inside double quotes, so a `$(…)`
> or a backtick in *any* variable value was command substitution. Verified both
> ways against a real pyinfra: `--data 'MOTD=$(id); rm -rf /'` now arrives at
> `host.data.MOTD` verbatim.
>
> `maskCommand()` was rewritten to walk the argv by position rather than to
> pattern-match a joined string, which also fixes a leak the old version had — it
> bounded a secret's value on the closing `"` the builder had written, so a value
> containing a `"` leaked its own tail. It shell-quotes as it joins, so
> `--print-only` output is still pasteable.
>
> What remains is not fixable here: the value still reaches pyinfra on its
> command line and so is visible in `ps`. That is pyinfra's `--data` interface.
> **Points 1 and 2 resolved earlier.** `maskCommand()`
> (`nopy.executor.ts`) rewrites the SSH `--password` and every `--data` value the
> manifest declared a secret, and it is applied at all three places the command
> string is printed: the debug log, the dry-run plan, and `--print-only`. The
@@ -549,10 +690,8 @@ README mentions a setup step.
> says which keys are sensitive, so `TOKEN`, `PSK` and `AUTH_KEY` are covered
> too, and it no longer matters that a key merely *looks* like a password.
>
> Point 3 is unchanged and now documented instead: the value still reaches
> pyinfra on its command line, so it is visible in `ps`. That is inherent to
> pyinfra's `--data` interface, not something nopy can mask. The shell-quoting
> concern in the same point is also still open. See `docs/REFACTORING.md` item 7.
> (At the time: point 3 unchanged, documented rather than fixed. See
> `docs/REFACTORING.md` item 7.)
Not stated in any document, and it sits directly against the security notes at
`README.md:217` and `:325` (which are narrowly about *storage*, and are correct
@@ -612,7 +751,22 @@ Worth documenting alongside `--dry-run`, since the difference is not obvious:
`--print-only` returns a `NopyResult` with `successful: 0` and skips execution
entirely, while `--dry-run` goes through the executor.
### 4.5 🟡 `--save-session` is ignored during a replay
### 4.5 ✅ `--save-session` is ignored during a replay — **fixed**
The guard is gone: the resolved cube set is exactly what the user asked to
capture, and a session written from a replay is no less valid than one written
from a fresh run. The README's "Recording a Session" examples now include the
replay form.
Fixed alongside it, from the same field run: a `--load-session` run was excluded
from history along with `-R`/`-H`, which was right for the latter two and wrong
for the first — a session file has never been in history, so `nopy history`
reported nothing afterwards and `-R` had nothing to repeat. `WorkflowResult`
now carries `replaySource: 'file' | 'history' | undefined` instead of a boolean,
which is the distinction the boolean could not express.
The original finding follows.
`nopy.main.ts:191` guards with `saveSessionPath && !workflow.isReplay`, so
`nopy install -R -s out.json` writes nothing and says nothing. The
@@ -641,7 +795,15 @@ a project without `KEY_DIR` in their config gets `None`.
Two cubes have **no README at all**: `cubes/admin/hostname` and `cubes/git/clone`
(20 of 22 have one).
### 5.1 🔴 `cubes/service/autostart/README.md` documents a different cube
### 5.1 ✅ `cubes/service/autostart/README.md` documents a different cube — **fixed**
> **Rewritten from the manifest and the (now working, see §6.1) deploy script.**
> Three parameters, `APP` / `SERVICE_NAME` / `AUTOSTART`, each described as what
> it actually does — including that `SERVICE_NAME` never reaches systemd and is
> a label only, so getting it wrong is cosmetic rather than a cube that manages
> the wrong unit. The new text also states the thing the old one obscured by
> describing a deploy pipeline: this cube does **not** create the unit file, it
> only enables and starts one that already exists.
The file is titled **"TypeStack Install Cube"** and describes cloning a git
repository, `yarn install`, `yarn build`, `docker compose up -d`, and PM2 process
@@ -705,7 +867,14 @@ have empty schemas.)
Not documentation issues, but found while checking the docs and worth recording.
### 6.1 🔴 `cubes/service/autostart/deploy.py` cannot run
### 6.1 ✅ `cubes/service/autostart/deploy.py` cannot run — **fixed**
> **Three lines.** `server` is imported alongside `systemd`, and `SERVICE_NAME`
> and `AUTOSTART` are read off `host.data` next to `APP`. The logic underneath
> was always right; nothing else changed. `python3 -m py_compile` passes.
>
> The "fails twice over" clause is stale: §2.2 closed with `-D`, so the cube does
> get its `--data` now.
```python
from pyinfra.operations import systemd # `server` is never imported
@@ -722,7 +891,31 @@ if AUTOSTART: # NameError
`host.data`; `server` is used but not imported. The script raises `NameError` on
the `if`. Per §2.2 this cube also gets no `--data` at all, so it fails twice over.
### 6.2 🔴 `-H <id>` and `--no-history` share one destination
### 6.2 ✅ `-H <id>` and `--no-history` share one destination — **fixed**
> **The boolean was renamed, not the replay flag.** `--no-history` is now
> `--no-save-history`, writing to `options.saveHistory`; `-H, --history <id>`
> keeps `options.history` and every documented invocation of it is unchanged.
> Renaming the boolean is the right way round twice over: `-H <id>` is what the
> help text, the README and `docs/API.md` all use, and "save history" is what the
> flag actually suppresses, next to `-s, --save-session`.
>
> Commander cannot be told to use a different destination — `attributeName()` is
> derived from the long flag with `no-` stripped — so separating the two meant
> changing one of the two spellings. The old spelling now fails loudly rather
> than silently, which is the point: `nopy install --no-history` prints
> `error: unknown option '--no-history'`.
>
> Verified by running the CLI, since `nopy.cli.ts` is argv wiring and excluded
> from coverage:
>
> ```
> install -H nonexistent-id -> Session not found: nonexistent-id
> install -H nonexistent-id --no-save-history -> Session not found: nonexistent-id
> install --no-history -> error: unknown option '--no-history'
> ```
>
> The middle line is the finding: the id used to be destroyed there.
Both options write to `options.history` (`nopy.cli.ts:57` and `:64`). Verified
with Commander:
@@ -752,13 +945,19 @@ the three has to give.
with §4.2 this is a second path by which secrets reach stdout.~~ **Removed**
alongside the `--use-defaults` work; it would have made an unattended run
unreadable. The two other paths in §4.2 are untouched.
- `keyman.encrypt.ts:19-20` — `console.log(tmpKeys); console.log(sshKeys);`
before the prompt.
- ~~`keyman.encrypt.ts:19-20` — `console.log(tmpKeys); console.log(sshKeys);`
before the prompt.~~ **Removed** in Phase 2 of the keyman remediation, along
with a third one nobody had noticed: a `console.log` *inside* a `filter`
callback in `keyman.decrypt.ts`, printing a line per vault directory.
### 6.5 🟡 No cycle detection
### 6.5 ✅ No cycle detection — **fixed**
Covered under §1.5. `docs/API.md:160` documents the error; there is no code that
raises it. Mutually dependent cubes recurse until the stack overflows.
Covered under §1.5, and closed there: the resolution stack raises a
`NopyUsageError` naming the whole path. `docs/API.md` documents the error again,
and this time something raises it.
~~`docs/API.md:160` documents the error; there is no code that raises it.
Mutually dependent cubes recurse until the stack overflows.~~
### 6.6 🟠 `ssh:keygen` depends on `user:add` but shares nothing with it
@@ -783,6 +982,88 @@ same shape as the `PASSWORD` default that was removed. It is recorded in the
session, so replays are stable, but each fresh `-D` run still creates a
differently-named account.
### 6.7 ✅ A declared secret was written to the session file in plaintext — **fixed**
Found in the acceptance run, not by reading. `README.md` promises that a session
holds no secret: "Passwords are never stored in session files. This covers both
the SSH password ... and any schema key a cube's manifest lists under `secrets`."
The `variables` block honoured that — `persistable()` leaves a declared secret
out entirely. The `env` block, one key higher in the same file, was a verbatim
copy of `.nopyrc.json`'s, so a credential declared there was written to the
session **and** to `.nopy.history.json` in plaintext.
Two of the fixes above widened the blast radius before it was noticed: §4.5 made
`--save-session` work on a replay, and §3.2 started recording `--load-session`
runs to history. Both write more files than before.
`Variables.persistableEnv()` applies the same rule to `env` that `persistable()`
applies to `variables`, and `nopy()` uses it instead of `config.env`. Verified in
the field: with `SSH_PASSWORD` declared under `secrets` and set in `env`, neither
the written session nor the history file contains the value.
### 6.8 ✅ `--print-only` was recorded in history — **fixed**
Also found in the acceptance run. `--dry-run` is excluded from history because it
deploys nothing; `--print-only`, which also deploys nothing, was not. Four
interactive runs against the VM produced four history entries, two of them from
`-P` passes that had only printed a command — and since `-R` repeats the head of
the list, the safe look-before-you-leap flag displaced the last real deployment
as the thing a bare `-R` would re-run.
One condition, `!printOnly`, alongside the `!dryRun` it belongs with. The
`README` list of "a run is *not* recorded when" and `docs/API.md` say so now.
### 6.9 ✅ Deploy order is the dependency tree; `CubeSelection` decides only the ties — **write-up corrected**
Found in the acceptance run, and the first write-up of it here was wrong. It
claimed a fix "has to decide what the right order even is — the order they were
picked in, or a topological one over `dependencies()`". Neither: the order is the
dependency tree, and that is already what nopy does. `resolveCube` resolves
`dependencies()` before emitting the cube itself, so emission is DFS post-order —
a topological order by construction. The recursion *is* the sort, which is what
`docs/API.md` means by "no separate topological sort", and `nopy.main.ts` walking
`selectedCubes` cannot break it: a cube listed ahead of its own dependency still
drags that dependency in first, and the second visit is deduped by `callKey`
rather than re-emitted at the tail. Pinned by *deploy order across several
selected cubes* in `tests/cubes.dependencies.test.ts`, both ways round.
This does not reopen §1.5, which stands: the *output* is a topological order but
there is no sort *algorithm*, and the price of that is still no cycle detection —
two mutually dependent cubes recurse until the stack overflows.
What list order does decide is where a cube with **no** edge lands, and that is
the whole of the real finding. `CubeSelection` returns enquirer's `selected`,
which is `choices.filter(enabled)` — display order, sorted by cube id, not the
order you ticked them. So picking `user:add` and `runtime:nodevm` yields
`['runtime:nodevm', 'user:add']`, and nothing reorders them because
`runtime:nodevm` declares `dependencies: () => []`. The acceptance run split them
into two invocations.
That missing edge is deliberate and stays missing: `user:add` *creates* a user,
so declaring it would make installing Node into an existing account silently
provision a new one. The prerequisite `SHELL=fish` really has is "fish and Oh My
Fish exist for `USER`", which no cube offers on its own — `user:add` only
provides it in passing. §5.3's `DeployError` is the answer for that, and it fires
before anything is changed. Ordering cannot fix an edge nobody can honestly
declare.
One residue, verified and left alone: an `after` hook's `exec(id)` runs after its
own cube is emitted, so it expresses "B after A" — but if B is also selected and
listed first, B is emitted first and the intent inverts. `after` hooks are not
the dependency graph and no cube in the bundle relies on this.
### 6.10 ✅ `runtime:nodevm` installed apt packages without refreshing the index — **fixed**
The same defect as §5.3 one operation earlier, and it only surfaced once §5.3 was
fixed and the cube could be run on a box where nothing else had. `apt.packages`
was called without `update`, alone among the six cubes in the bundle that install
packages. On a fresh `bento/ubuntu-24.04` the shipped index names .deb versions
the mirror has already superseded, so the fetch 404s and pyinfra reports
`executed 0 commands` before nvm is ever reached.
It passed on the first VM only because `user:add` had run there and pulled in
`apt:essentials`, which does pass `update`. Fixed with `update=True` on the call.
---
## 7. Checked and accurate
@@ -813,35 +1094,49 @@ Recording what was verified and found correct, so a future pass need not redo it
scan, dotted/`node_modules` skipping, the prefixed `*.manifest.mjs` fallback,
and the three-step id resolution match `cubes/loader.ts` exactly.
- **pyinfra `--data` type coercion** (`README.md:101`) — correct.
- **keyman config** — priority (`VAULT_ROOT` > file > defaults), the four default
values, and the vault layout match `keyman.config.ts` and `keyman.encrypt.ts`.
- **keyman config** — priority (`VAULT_ROOT` > file > defaults) and the four
default values match `keyman.config.ts`. The third clause of this entry used to
read "and the vault layout match[es] … `keyman.encrypt.ts`", which was true only
because `encrypt.ts` hardcoded `keys` and ignored the config — checking a
documented layout against the file that ignores the configuration is what kept
that defect invisible here. Both are honest now: the layout is configurable and
`encrypt` reads the configuration (`packages/keyman/docs/AUDIT.md` §1.1, §5.1).
---
## Suggested order of attack
**1 — ~~Decide on the three phantom features.~~ Two left.** §1.1 (`-D`) is
**done** — implemented, tested, and verified against every cube in `cubes/`.
That closed §2.2 and half of §2.1 with it, since neither could be left standing
under a run that never prompts. §1.2 (`--json`) and §1.3 (`log.*`) are still
"documented, wired up, never read": each is a small implementation or a small
deletion, but neither can stay documented as working.
**1 — ~~Decide on the three phantom features.~~ Done, three different ways.**
§1.1 (`-D`) was implemented, tested, and verified against every cube in `cubes/`;
that closed §2.2 and half of §2.1 with it, since neither could be left standing
under a run that never prompts. §1.2 (`--json`) was removed rather than
implemented, for the reason recorded there. §1.3 (`log.*`) was implemented —
`logConfigToFlags()` finally has a caller, in `buildDeployCall`.
**4 — Decide the `.describe()`/`.default()` ordering (§2.3).** Either read
through the `ZodDefault` wrapper in `nopy.prompts.ts`, or fix the ordering in all
14 manifests and the README example. The first is one line and cannot regress.
**4 — ~~Decide the `.describe()`/`.default()` ordering (§2.3).~~ Done, by
reading through the wrapper.** `promptLabel()` in `nopy.prompts.ts` walks
`default`/`optional`/`nullable` down to the described schema, so both orders
work and the 15 manifests that had it "wrong" needed no edit. The alternative —
reordering every manifest — would have left the next one free to regress.
**5 — ~~Regenerate `docs/API.md` (§3).~~ Done.** Rewritten against the source
rather than patched, and extended to the exports that never had an entry
(variables, history, prompts, the authoring package). One new finding came out of
it: `CubePackageRef` is not re-exported from `src/index.ts` although `NopyConfig`
refers to it — a one-line fix, left for whoever next touches the export list.
it — `CubePackageRef` was not re-exported from `src/index.ts` although
`NopyConfig` refers to it — and that one line has since been added.
**6 — Cube docs (§5) and the two missing READMEs.** `service/autostart` is the
worst — its README belongs to a different cube, and its `deploy.py` does not run
at all (§6.1).
**6 — Cube docs (§5).** `service/autostart` was the worst and is **done**: its
`deploy.py` now reads its three variables off `host.data` instead of raising
`NameError` (§6.1), and its README describes that cube rather than a different
one (§5.1). Still open: §5.2 (four wrong parameters in
`network/wifi/access-point`), §5.3 (two cubes claiming to have no parameters) and
the missing READMEs.
**7 — Secrets on stdout (§4.2, §6.4).** The `console.log` in `Variables.assign`
is gone. Still open: mask the password in the executor's debug line and in the
dry-run plan, and pass `--user`/`--password` as argv rather than interpolating
into a shell string.
**7 — ~~Secrets on stdout (§4.2, §6.4).~~ Done as far as it can be.** The
`console.log` in `Variables.assign` is gone; the password is masked in the
executor's debug line and in the dry-run plan; and the whole command is argv now,
run without a shell, so nothing is interpolated into a string any shell will
re-parse. What is left is inherent: pyinfra takes `--data` on its own argv, so
the value is visible in `ps` on the machine running the deploy for the length of
the run. Fixing that means a change on pyinfra's side, not this one's. §6.4's
remaining bullet is unrelated debug output.
+742
View File
@@ -0,0 +1,742 @@
# Field-report implementation plan
Turns the findings of the wild-run field report into work. Ordered by severity,
then by whether a fix unblocks a later one. Every phase is independently
shippable and ends at the existing gate (`lint:ci` → `typecheck` →
`test:coverage`).
Findings the field run confirmed that `DOCS-AUDIT.md` already tracks keep their
audit number, so the two documents stay in step: closing an item here closes it
there.
## Contents
- [0. Retractions](#0-retractions) — two findings were harness artefacts
- [1. The secret leak](#1-the-secret-leak) — `env` broadcasts a credential in the clear
- [2. Remove `--json`](#2-remove---json) — audit §1.2, closed by deletion
- [3. Replay and session correctness](#3-replay-and-session-correctness) — audit §2.4, §2.5, §4.5
- [4. The first five minutes](#4-the-first-five-minutes)
- [5. Cube defects](#5-cube-defects) — audit §5.3
- [6. Documentation sweep](#6-documentation-sweep)
- [7. Harness fix and acceptance run](#7-harness-fix-and-acceptance-run)
---
## 0. Retractions
Two findings in the field report were caused by the PTY driver I used to script
the TUI, not by nopy. The driver called `pty.fork()` and never issued
`TIOCSWINSZ`, so the child saw a **0×0 terminal**.
`enquirer`'s `utils.height` (`lib/utils.js:80-86`) computes a sane fallback and
then throws it away:
```js
let rows = (stream && stream.rows) ? stream.rows : fallback; // fallback = 25
if (stream && typeof stream.getWindowSize === 'function') {
rows = stream.getWindowSize()[1]; // ← unconditional
}
```
A TTY always has `getWindowSize`, so `height` becomes `0`, and
`ArrayPrompt.limit` (`lib/types/array.js:604`) returns `Math.min(limit, 0)`.
`visible` is then empty for every array prompt.
Re-run with a 50×200 window, both work correctly:
| Field report | Actual |
| --- | --- |
| §3.6 multi-field forms never render their fields | All four fields render, accept input, and submit: `RESULT {"USER":"X","PASSWORD":"changeme","GROUPS":"","PUBKEY":""}` |
| §3.14 cube filter says "No matching choices" while matching fine | Filter renders correctly, highlights the matched substring, and returns `["user:add"]` |
**What survives, and it is worth fixing.** nopy has no defence against a
terminal that reports a degenerate size: the form silently submits `{}`, and the
run proceeds with every variable absent. That is [§4.4](#44-survive-a-terminal-that-reports-no-size)
and [§4.5](#45-never-deploy-a-cube-with-a-missing-required-variable). Field
report §3.7 (a required key dropped from the command) was reached through the
0×0 form, but the hole it exposed is real and independent: nothing on the
interactive path checks that a cube's required variables were actually filled.
Everything else in the field report stands.
---
## 1. The secret leak
Highest severity: following the documentation as written prints a credential in
plaintext, and the workaround it is prescribed for does not work either.
### 1.1 A declared secret must never be broadcast to cubes that do not declare it
**What happens.** `Variables.bucket()` (`nopy.common.ts:193-203`) seeds *every*
key of config `env` onto *every* cube as an `env`-origin assignment, and
`isSecret` (`nopy.common.ts:135-137`) is keyed per cube. So with `PASSWORD` under
`env`, a dry run prints:
```
Step 1: apt:essentials … --data "PASSWORD=wildpass123" ← unmasked
Step 2: user:add … --data "PASSWORD=********" ← masked
Step 3: runtime:nodevm … --data "PASSWORD=wildpass123" ← unmasked
```
**Why the obvious fix is wrong.** "Seed `env` only onto cubes whose schema
declares the key" breaks a shipped cube: `ssh/keyman/deploy.py:28` reads
`host.data.get('KEY_DIR')`, a key its manifest does not declare and that exists
only in `.nopyrc.json` `env` (`packages/nopy/.nopyrc.json:5`). Broadcast is
load-bearing.
**Fix.** Narrow the rule to secrets only — broadcast stays, secrets stop
travelling:
1. In `nopy.main.ts`, after `loadCubes()`, collect the union of every loaded
manifest's `secrets` and hand it to `Variables`:
```ts
const declaredSecrets = new Set(Object.values(cubes).flatMap((c) => c.secrets));
const variables = new Variables(config.env, declaredSecrets);
```
Deterministic and ordering-free: it is computed before the first
`resolveCube`, so it does not depend on which cube resolves first.
2. In `bucket()`, skip seeding an `env` key that is in `declaredSecrets` unless
the cube itself declares that key in its schema. `Variables` needs the cube's
schema keys for this — add `declareSchema(cube, keys)`, called from
`BuildContext.resolveCube` immediately after `declareSecrets`
(`cubes/dependencies.ts:121`), before any assignment creates the bucket.
3. Mark globally, mask globally: a key in `declaredSecrets` is `redacted` on
whichever cube it does land on, even if that cube's own manifest forgot to
list it. Cheap defence against a manifest that declares `PASSWORD` in `schema`
and omits it from `secrets`.
4. New optional `.nopyrc.json` key, for an `env` secret no manifest declares
(an API token a hook uses, say):
```json
{ "secrets": ["DEPLOY_TOKEN"], "env": { "DEPLOY_TOKEN": "…" } }
```
Unions into `declaredSecrets`. Validate it in `nopy.config.ts` alongside the
other properties.
**Verify.** New test in `tests/common.test.ts`: `env` carrying a key that cube A
declares secret and cube B does not → B's `get()` does not contain the key; A's
does and is redacted. New test in `tests/executor.test.ts`: the printed plan for
B contains no occurrence of the value.
### 1.2 `env` must satisfy the `--use-defaults` gap check
**What happens.** `fillSessionGaps` (`cubes/dependencies.ts:79-89`) builds
`gaps` as `missingRequired ∪ cube.secrets` — *unconditionally* including every
secret, regardless of whether anything supplied a value. Under `-D` it throws,
and the message tells you to do the thing you have already done:
```
Error: Cube "user:add" cannot be replayed with --use-defaults: PASSWORD would
have to be entered. … set the values under "env" in .nopyrc.json.
```
The value **was** read — without `-D` the prompt came pre-filled from `env`.
**Fix.** Under `useDefaults`, a gap is satisfied when something outside the
session supplied it deliberately:
```ts
const unsatisfied = gaps.filter((key) => {
const origin = this.variables.of(cube.id, key)?.origin;
return origin !== 'env' && origin !== 'param';
});
if (unsatisfied.length > 0) throw new Error(…);
```
`default` is deliberately **not** accepted for a secret. On a replay the
recorded value is gone by design, so falling through to a manifest default would
deploy a different credential than the run being replayed — silently. The
message says so, instead of repeating advice that already failed:
> `Cube "user:add" cannot be replayed with --use-defaults: PASSWORD is a secret
> and secrets are never recorded in a session. Set it under "env" in
> .nopyrc.json (a schema .default() is not accepted for a secret), pass it from
> a dependency, or replay without --use-defaults.`
**Verify.** `tests/cubes.dependencies.test.ts`: `-D` replay with the secret under
`env` succeeds and the value reaches the deploy call; with only a schema
`.default()` it throws and the message names the key. Both are new cases.
### 1.3 Documentation
`README.md:291` currently prescribes exactly the leak. After 1.1 and 1.2 the
advice becomes true; add one sentence under *Secrets* stating the new rule — a
declared secret in `env` reaches only the cubes that declare it — so the
interaction between the two features is written down once, in the place a reader
of either lands.
---
## 2. Remove `--json`
Audit §1.2, independently confirmed: `nopy install -R --json > j.out` →
`json.load()` raises; stdout carries seven ANSI-coloured log lines and no JSON.
**Closed by deletion, not by implementation.**
**Why removal is the right call and not just the cheap one.** `executeDeployCalls`
runs pyinfra through execa with *inherited* stdio (`nopy.executor.ts:117`), so
during a real run nopy does not own its own stdout — pyinfra does, and writes an
unbounded amount to it. A JSON blob appended after that is not machine-readable
by any definition a caller could rely on; making it so means capturing pyinfra's
output and giving up live progress, which is a real feature traded for a
speculative one. That is the same root cause as the documented gap that
`ExecutionResult.stdout` is never populated. The CI case the flag was for is
already covered: `--print-only` for the plan and the exit code for the verdict
(`nopy.cli.ts:138-140` exits 1 on any failure). Nothing can depend on the current
behaviour, because there is no current behaviour.
### 2.1 The flag, from the `install` command
- `nopy.cli.ts:91` — drop the `.option('-j, --json', …)` line.
- `:133` — drop `jsonOutput: options.json` from the `nopy()` call.
- `:147-160` — the `if (options.json)` error branch collapses to the single
`console.error`. This is the same statement [§4.3](#43-routine-errors-print-a-raw-node-stack-trace)
rewrites, so whichever phase lands first does both; the other just reads it.
### 2.2 `jsonOutput`, from the library
- `NopyOptions.jsonOutput` (`nopy.main.ts:110`) and its destructure (`:141`).
This is a **breaking change to an exported interface** — `docs/API.md:273`
documents it. It is a `0.x` minor bump, and an unknown property is a type error
rather than a silent behaviour change, so a consumer finds out at compile time.
- `:148` — the banner guard becomes `if (!replaySession && !loadSessionPath)`.
- `:158` — the JSON error dump goes; `log.error` on `:156-157` already reported
the same errors.
- `:227` — the `onProgress` callback loses its guard and always logs.
- `outputExecutionPlan(calls, asJson?)` (`nopy.executor.ts:148-158`) — drop the
parameter and the dead JSON branch. Exported and documented (`docs/API.md:588`);
nothing in `src/` passes the second argument, only a test does.
### 2.3 stdout hygiene — the one fix that survives, and now matters more
With `--json` gone, `--print-only` is the machine-readable surface, so it has to
be clean. Two writers currently pollute it, and `jsonOutput` was the only thing
holding either back:
- `configureLogtape`'s console sink uses `console.log` (`nopy.main.ts:34`),
against `README.md:409`, which promises stderr. Switch to `console.error`.
- `printActiveConfig` ends in `console.log` (`nopy.main.ts:95`) and is suppressed
today only by `jsonOutput` and by replay. Same switch.
Write the rule down once, in the README: **stdout carries the deploy commands and
pyinfra's own output; everything nopy says about itself goes to stderr.** That is
a promise a test can hold to, which the old `--json` claim never was.
Ripple worth knowing before starting: `tests/main.test.ts` spies on `console.log`
(`logSpy`) throughout, so moving logtape to `console.error` means moving those
spies. Mechanical, but it touches most of the file.
### 2.4 Documentation — most of the work
| File | Change |
| --- | --- |
| `packages/nopy/README.md:537-544` | Delete the *JSON output (for CI/CD)* block. Replace with the CI recipe that works: `--print-only` for the plan, exit code `1` for the verdict, `--continue-on-error` when you want every failure in one run. |
| `packages/nopy/README.md:409` | The stderr promise stays and is now load-bearing; reword its reason from `--json` to `--print-only` and piped stdout. |
| `docs/API.md:273` | Remove the `jsonOutput` row from the `NopyOptions` table. |
| `docs/API.md:588-596` | `outputExecutionPlan(calls, asJson?)` → `outputExecutionPlan(calls)`; the note that `--dry-run --json` prints the text plan goes with it. |
| `docs/API.md:1034` | Reword the stderr note the same way as `README.md:409`. |
| `docs/API.md:1170-1173` | *Known gaps*: the `--json` entry disappears — that is the point. `ExecutionResult.stdout` is never populated **stays**, and gains the reason (stdio is inherited), since that is now the honest answer to "how do I capture output?". |
| `docs/CUBE-PACKAGES.md:317` | Future-work line proposes surfacing a cube's source "in the interactive picker and in `--json` output"; drop the second half. |
| `nopy.exit.ts:77` | Comment cites `--json` and `--print-only` as the reason for the exit discipline; leave the discipline, drop the `--json` half. |
| `DOCS-AUDIT.md:85` | Mark §1.2 closed **by removal** and say so in one line — a reader of that document should not go looking for the fix. Also touch its back-references at `:446` and `:854`. |
### 2.5 `nopy history --json` is a different flag — keep it
`nopy.cli.ts:169-177` is a second, unrelated `-j, --json`, on the `history`
command, and it works: `JSON.stringify(listHistory())`. It was never part of
audit §1.2 — the field run used it successfully. Keep it. Nothing else writes to
stdout during `history`, so it has none of the problem above, it is three lines,
and it is how a script finds the id to pass to `-H`.
If the intent is that nopy has no JSON surface at all, removing it is
`nopy.cli.ts:169` plus `:173-177`, and `README.md:541` and `:573`. Flagging it
rather than deciding it: this one is a working feature, so deleting it is a
different kind of change from deleting one that never worked.
### 2.6 Tests
Delete, rather than adapt — they assert behaviour that no longer exists:
- `tests/main.test.ts:162-168` — *emits the errors as JSON when jsonOutput is set*
- `tests/main.test.ts:224-227` — *is suppressed for JSON output* (the sibling
`replaySession` / `loadSession` suppression cases stay and still cover `:148`)
- `tests/main.test.ts:366-373` — *stays silent on progress when jsonOutput is set*
- `tests/executor.test.ts:129` — the `outputExecutionPlan(calls, true)` case
Add one that holds the new rule: run with `printOnly` and assert `console.log`
received the command block and **nothing else** — banner and progress lines on
`console.error`. Deleting a covered branch moves coverage up, not down, so the
gate is not at risk here.
---
## 3. Replay and session correctness
### 3.1 `--save-session` no-ops on a replay (audit §4.5)
`nopy.main.ts:199` guards with `!workflow.isReplay`, so
`nopy install -R -s out.json` exits 0 and writes nothing. Drop the guard: the
resolved cube set is exactly what the user asked to capture, and a replay's
session is no less valid than a fresh run's.
### 3.2 A `--load-session` replay is not recorded
`nopy.main.ts:203` excludes every replay from history. For `-R` and `-H` that is
right and documented (`README.md:494`) — repeating must not push the original
out of the list. For `-l` it is wrong: the run is not already in history, so
after deploying from a session file `nopy history` says *"No sessions in
history"* and `-R` has nothing to repeat. That is what happened in the field run.
Record `-l` runs; keep `-R`/`-H` non-recording. `WorkflowResult` needs to
distinguish them — replace the boolean `isReplay` with
`replaySource: 'file' | 'history' | undefined`, or add a second flag. Then fix
`README.md:496-500`, whose explicit *"a run is not recorded when"* list omits
replays entirely and so contradicts `:494`.
### 3.3 The written session does not match the documented format (audit §2.4)
Documented (`README.md:224-254`, `docs/SESSION_FORMAT.md`) versus written:
| Field | Documented | Written |
| --- | --- | --- |
| `version` | `"1.0.0"` | absent |
| `name` | `"My Deployment Session"` | absent |
| `timestamp` | ISO 8601 | absent |
| `auth.method` | `"ssh-key"` | `"ssh"` |
| `auth.username` | `"root"` | absent |
This is what you consult in order to hand-write a session, which is what the
field run had to do.
Implement rather than delete — all three fields are cheap and two are useful:
- `createSession` (`nopy.session.ts:183-197`) stamps `version: '1.0.0'` and
`timestamp: new Date().toISOString()`, and derives a default `name` the way
`generateEntryName` already does for history (`nopy.history.ts:84-102`).
- `loadSession` (`:130-158`) keeps accepting sessions without them — every
existing file and every hand-written one must stay loadable. Warn on a
`version` it does not know; do not fail.
- `auth.method: 'ssh'` is real, not a bug: `runInteractiveWorkflow:64-67` uses it
for `@vagrant/` and `@docker/` hosts, where the connector owns authentication.
It is simply undocumented. Document the third value and when it appears.
### 3.4 `listSessions` does not match the documented filename (audit §2.5)
Docs say `.nopysession.json`; `listSessions` (`nopy.session.ts:166-175`) filters
for `.session.json` / `.session.mjs`, which `wild.nopysession.json` does not
match. Widen the filter to `.nopysession.json` / `.nopysession.mjs` and keep the
old suffixes.
---
## 4. The first five minutes
The four roughest edges a new user meets all sit before anything that works
well.
### 4.1 The documented install command 404s
`packages/nopy-cubes-core/README.md:9`, `packages/nopy/README.md:317` and `:344`
all open with:
```sh
pnpm add -D @bitsquare/nopy-cubes-core
```
```
[ERR_PNPM_FETCH_404] GET https://registry.npmjs.org/@bitsquare%2Fnopy-cubes-core: Not Found
```
The bundle has never been published to npmjs, and an *untagged* Gitea install
resolves to nothing because Gitea publishes no `latest` tag. What rescued the
field run was pnpm's own error listing `main: 0.5.0-main.17.gda84523`.
Replace both snippets with the form that works, and say why:
```sh
pnpm add -D @bitsquare/nopy-cubes-core@main \
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
```
`nopy-cubes-core`'s README does not mention Gitea at all; nopy's mentions it only
in a *Channels* section framed around installing the CLI. Both need the tag
requirement stated where the install command is, not two sections away. Revisit
when `release.yml` first ships the bundle to npmjs — the guard in that workflow
blocks the first `nopy` release until it does.
### 4.2 pyinfra is an unstated prerequisite
Nothing in the README says pyinfra must be installed separately and on `PATH`;
`nopy.executor.ts:117` spawns it directly. The field run only worked because it
happened to be there. Add a *Requirements* block next to the install command:
Node ≥ 22, `pyinfra` on `PATH` (`pipx install pyinfra`), plus whatever the chosen
connector needs (`vagrant`, `docker`). Optionally probe for it once at startup
and fail with one line instead of a spawn error.
### 4.3 Routine errors print a raw Node stack trace
`nopy.cli.ts:159` passes the error object as a third argument:
```ts
console.error('Error:', error instanceof Error ? error.message : error, error);
```
so the message prints, then the whole error prints again with frames into
`dist/`. Running outside a project — the most likely first-run mistake — yields:
```
Error: No .nopyrc.json found. Create one in your project directory or any parent directory.
at loadConfig (…/dist/nopy.config.js:187:15)
at Command.<anonymous> (…/dist/nopy.cli.js:74:24)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
```
Drop the third argument; print the stack only under `NOPY_DEBUG`. Adopt keyman's
shape (`keyman.cli.ts` is the error boundary that turns a `UsageError` into one
line) so the two CLIs stay in step: a `NopyUsageError` for the errors that are
the user's to fix — no config, no cubes, missing required variable, unknown
session — and a stack for everything else.
### 4.4 Survive a terminal that reports no size
Per [§0](#0-retractions): with `stdout.rows === 0`, every enquirer array prompt
renders "No matching choices", the form submits `{}`, and nopy deploys with every
variable defaulted. Reachable outside a test harness — some CI pseudo-terminals,
`script -q`, and editor terminals during startup all report 0 rows.
Passing an explicit `limit` does **not** help (measured): enquirer clamps it with
`Math.min(limit, this.height)`. But `height` itself has an escape hatch one line
above the bug — `prompt.js:396`:
```js
get height() { return this.options.rows || utils.height(this.stdout, 25); }
```
`options.rows` short-circuits the broken function entirely, so the fix is to pass
a floored size rather than to fake a stdout:
```ts
const MIN_ROWS = 24, MIN_COLS = 80;
const terminalSize = (out = process.stdout) => ({
rows: Math.max(out.rows || 0, MIN_ROWS),
columns: Math.max(out.columns || 0, MIN_COLS),
});
```
Measured, 2×2:
| PTY | without | with |
| --- | --- | --- |
| 0×0 | `RESULT {}` | `RESULT {"USER":"X","PASSWORD":"changeme","GROUPS":"","PUBKEY":""}` |
| 50×200 | full result | full result (`rows` passes through as 50) |
Apply to both enquirer call sites — `CubeSelection` (`nopy.prompts.ts:61-68`) and
`VariableAssignment` (`:238-243`) — and derive `pageSize` (`:55-56`) from the same
helper, where `process.stdout.rows || 24` already fails for `0` only to be clamped
away again.
*(An earlier draft of this section proposed a `Proxy` over `process.stdout`
reporting the floor. It works — also measured — but it fakes a stream object to
reach a value the prompt will take directly. `options.rows` is the same fix
without the impersonation.)*
enquirer 2.4.1 is the last release (2023) and this is its bug. Worth a comment at
the call site so nobody "simplifies" the sizes away later.
### 4.5 Never deploy a cube with a missing required variable
Field report §3.7. `README.md:99` guarantees *"Every key defined in the manifest
`schema` is guaranteed to be present on `host.data`"*, and the interactive path
does not enforce it: `resolveCube` calls `VariableAssignment`
(`cubes/dependencies.ts:138`) and goes straight to `buildDeployCall`.
`assertVariablesComplete` exists and runs **only** under `useDefaults` (`:136`).
`buildDeployCall` then emits `--data` for whatever variables exist
(`:186-189`), so a key nothing ever assigned is absent from the command
entirely and the deploy script reads `None`.
Two ways in, both real: a form that submits nothing (§4.4), and a form the user
cancels — `VariableAssignment`'s `catch {}` (`nopy.prompts.ts:252-254`) swallows
cancellation and returns as though it succeeded.
- Call the completeness check on the interactive path too, with a message that
fits: `Cube "user:add" is missing PUBKEY. It has no default value and nothing
supplied one.` (The replay path already does this at
`cubes/dependencies.ts:94-100`.)
- Distinguish cancel from error in `VariableAssignment` and route a cancel
through `nopy.exit.ts` like the other prompts, instead of continuing with a
half-filled cube.
**Verify.** `tests/cubes.dependencies.test.ts`: a cube with a required
no-default key, with the form stubbed to return `{}` → resolution throws and
names the key. This test fails today.
### 4.6 `self-update` prints a command that cannot work
From a project with no scope mapping in `.npmrc`:
```
Channel: main
Registry: https://registry.npmjs.org/
Available: unknown
Would run: npm install --global @bitsquare/nopy@main
```
`main` snapshots exist only on Gitea, and `buildSelfUpdateCommand`
(`nopy.update.ts:378-381`) deliberately omits the registry flag when the registry
*is* npmjs — correct in general, wrong for this combination. The channel is
derived from the running version, so nopy already knows the command is
unrunnable.
Detect `channel === 'main' && registry === NPMJS_REGISTRY` in the CLI action and
refuse with a line that fixes it:
> `You are running a main snapshot, which is published to Gitea only, but
> @bitsquare resolves to npmjs. Re-run with --registry <url>, or set it once:
> npm config set @bitsquare:registry <url>`
**Verify.** `tests/update.test.ts` already covers `buildSelfUpdateCommand`'s
registry logic; add the combination case.
---
## 5. Cube defects
### 5.1 `GLOBAL_PACKAGES` is accepted and then ignored
`runtime/nodevm/deploy.py:9` reads `GLOBAL_PACKAGES` off `host.data` and never
uses it; `:49` hardcodes the list. The field run passed
`GLOBAL_PACKAGES=npm-check-updates`, watched it appear in the plan and on the
command line, and found it absent from `npm ls -g`.
```python
"npm install -g pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx"
```
Fix both halves so behaviour does not change for anyone who never set the
variable — use the parameter in the deploy, and make the manifest default the
list that is hardcoded today:
```python
f"npm install -g {GLOBAL_PACKAGES}"
```
```js
GLOBAL_PACKAGES: z.string()
.describe('Space-separated list of global npm packages to install')
.default('pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx'),
```
The current default (`npm-check-updates`) is not what the cube installs, so
today's default is wrong in both directions.
### 5.2 `runtime/nodevm/README.md` describes a different cube (audit §5.3)
| README says | Manifest / `deploy.py` |
| --- | --- |
| "This cube currently has no configurable parameters" (`:44`) | `VERSION`, `USER`, `ALIAS`, `GLOBAL_PACKAGES`, and `SHELL` after [§5.3](#53-runtimenodevm-has-an-undeclared-shell-dependency--add-a-shell-parameter) |
| "official NodeSource setup script" (`:22`) | `nvm` — `deploy.py:34-37` |
| "Installs the latest LTS version" (`:23`) | whatever `VERSION` says, default `v22.20.0` |
| "npm@11.1.0" in the global list (`:33`) | not installed |
| "Node.js is installed system-wide" (`:80`) | per-user under `~/.nvm` for `USER` |
Rewrite against the manifest. It is the only file that would tell a reader
`VERSION` or `USER` exist. `runtime/docker/README.md` makes the same
"no configurable parameters" claim with a `DISTRO` field — same fix, same commit.
### 5.3 `runtime:nodevm` has an undeclared shell dependency — add a `SHELL` parameter
`dependencies: () => []`, but `deploy.py` runs `omf install nvm` (`:35`) and sets
`_shell_executable='/usr/bin/fish'` (`:43`) — it needs fish **and** Oh My Fish
already installed for `USER`. In the field run it worked only because `user:add`
ran first and installs both. Declaring `user:add` as a dependency would be wrong:
it would create a user that is usually meant to already exist.
**Fix.** Make the shell a parameter — `SHELL: 'fish' | 'bash'` — so the cube can
be standalone, as its manifest already claims, without taking fish away from
anyone using it today.
```js
SHELL: z
.enum(['fish', 'bash'])
.describe('Login shell to install through. fish needs Oh My Fish; bash needs nothing')
.default('fish'),
```
**Default stays `fish`, deliberately.** nvm wires itself into whichever shell
installed it, so switching the default would leave an existing user — whose login
shell `user:add` set to fish — with node installed and invisible. Additive
change; the escape hatch for a fresh host is one variable.
Three places differ, and only three:
| | `fish` | `bash` |
| --- | --- | --- |
| `_shell_executable` | `/usr/bin/fish` | `/bin/bash` |
| Loading nvm | `omf install nvm` — the plugin defines `nvm` as a fish function that every login shell loads | `export NVM_DIR="$HOME/.nvm"; . "$NVM_DIR/nvm.sh"` |
| Making `npm` reachable | the plugin activates the `default` alias on load | `nvm use <ALIAS>` first |
The bash arm has one non-obvious constraint: **every entry in `commands` is its
own shell**, so sourcing `nvm.sh` and using `nvm` have to be a single entry.
Sourcing cannot be skipped either — nvm's installer appends to `~/.bashrc`, and
Ubuntu's `~/.bashrc` returns at line 1 for a non-interactive shell, so the hook
never runs under `su -c`. fish has no equivalent problem, which is presumably why
it was chosen.
That same constraint makes `set -gx NVM_DIR $HOME/.nvm` (`:38`) **dead today** —
its own shell, exported, exits. Drop it; the fish plugin sets `NVM_DIR` itself.
**Guard.** With `SHELL: 'fish'` on a host without fish, fail early and legibly
rather than inside `omf`:
```python
if SHELL == 'fish' and not host.get_fact(Which, 'fish'):
raise DeployError(
f'runtime:nodevm: SHELL is "fish" but fish is not installed for {USER}. '
'Run user:add first, or set SHELL=bash.'
)
```
Deliberately checks the binary only. Oh My Fish is a set of fish functions with
no binary to probe and no fixed path, so a check for it would be guesswork; if
fish is present and omf is not, `omf install nvm` fails with its own clear
message. Half a guard that is certain beats a whole one that is not.
**While in the file** — `deploy.py:1-6` imports `npm` and `python` and never uses
them, and assigns `hasNode = host.get_fact(Which, 'node')`, also unused. The
`Which` import stops being dead the moment the guard lands.
**Verify.** No test harness reaches a cube deploy script, so this is acceptance,
not unit: [§7.3](#7-harness-fix-and-acceptance-run) runs `runtime:nodevm` with
`SHELL=bash` on a **fresh** VM where `user:add` has not run, and confirms
`node -v` and the `GLOBAL_PACKAGES` list for `USER`. That is the case the cube has
never survived.
### 5.4 `VERSION` accepts `null` and would install `None`
`z.nullable(z.string()).default('v22.20.0')`, and `deploy.py:36` interpolates it
straight into `nvm install {VERSION}`. Either drop `nullable`, or handle `None`
as "latest LTS" — which is what the README claims the cube does anyway.
### 5.5 `user/add/README.md` — trim, do not rewrite
Every claim it makes was verified true in the field run, and its notes on *why*
`PUBKEY` has no default and *why* the generated password was removed are the best
documentation in the repo. Its last ~50 lines are generic Fish keybinding tips
(`Ctrl+L` → clear the terminal) unrelated to the cube. Move them somewhere they
belong or delete them; leave the rest alone.
---
## 6. Documentation sweep
Small, mechanical, no code.
| File | Change |
| --- | --- |
| `docs/VAGRANT.md` | Never states the `@vagrant/<name>` host syntax — the field run inferred it from an unrelated `@docker/` example. Add `"hosts": ["@vagrant/nopytestvm"]` and one line tying the VM name to it. Add `vagrant destroy -f` for cleanup. |
| `README.md` (root) | Lists a `cubes/` directory at the repo root that no longer exists (`:10`); describes `typecheck` as `tsc --build --noEmit` (`:31`), which TS rejects outright for a project with references. |
| `packages/nopy/README.md` | Top-level `--help` lists only `-V`/`-h`, then the *Examples* block uses `-R`, `-n`, `-P`, `-l`, `-s` — all of which live on `install`. Either add a "these are `install` options" line to the help text (`nopy.cli.ts:56-76`) or promote the common ones. |
| `packages/nopy/README.md` | The host picker offers `docker`, `vagrant`, `@vagrant/…`, `custom`; the first two appear in no document. Add the two connector shortcuts and what they prompt for. |
| `docs/SESSION_FORMAT.md` | Uses `.session.json` throughout while the README uses `.nopysession.json`. Pick one — `.nopysession.json` — and align both, together with [§3.4](#34-listsessions-does-not-match-the-documented-filename-audit-25). |
Also worth stating once, somewhere prominent: the accuracy failures cluster on
one seam. Everything a human reads on screen matched the docs; everything
machine-facing had drifted — `--json`, the session format, `-s` on replay, `-l`
and history, `nodevm`'s parameters, the install command. That is not random rot,
it is the interactive surface being maintained by daily use while the scripting
surface was documented from intent. Phases 2 and 3 are the correction, in the two
ways available: `--json` was documented from intent and never built, so it goes;
the rest was built and then drifted, so it gets fixed. Keeping it corrected means
what remains of the scripting surface — `--print-only`, sessions, history —
needs tests that assert on **stdout**, not prose.
---
## 7. Harness fix and acceptance run
**7.1** Fix the PTY driver before it lies again: `drive.py` and `expect.py` must
issue `TIOCSWINSZ` after `pty.fork()`.
```python
fcntl.ioctl(fd, termios.TIOCSWINSZ, struct.pack('HHHH', rows, cols, 0, 0))
```
Worth keeping the drivers — they are the only way to test the TUI end to end —
so they belong in the repo under `scripts/`, not in a temp folder.
**7.2** Add a 0-rows regression test that exercises §4.4 directly: spawn the CLI
under a 0×0 PTY and assert the form still yields values. It has to be a real
child process for the same reason `cubes.resolve-hook.test.ts` does — inside a
vitest worker there is no TTY to misreport.
**7.3** Acceptance: re-run the field scenario from an empty directory —
`vagrant up`, install the bundle with the [§4.1](#41-the-documented-install-command-404s)
command, deploy `user:add` and `runtime:nodevm` **interactively** (not from a
hand-written session), then check:
- `npm ls -g` contains what `GLOBAL_PACKAGES` asked for (§5.1)
- on a **second, fresh** VM where `user:add` has *not* run, `runtime:nodevm` with
`SHELL=bash` installs node and the global packages; with `SHELL=fish` it fails
in one line naming the missing shell rather than inside `omf` (§5.3)
- `nopy install -P 2>/dev/null` prints the deploy commands and nothing else — no
banner, no progress lines, no update hint (§2.3)
- `nopy install --json` is rejected as an unknown option (§2.1)
- `nopy history` lists the `-l` run (§3.2)
- a dry run with a secret under `env` prints `********` on every cube (§1.1)
- `nopy` in an unconfigured directory prints one line (§4.3)
### What the run found
Every check above passed. One deviation and three findings.
**Deviation.** The bundle was installed from `pnpm pack` tarballs of the three
packages rather than from Gitea, because the cube fixes this plan makes are not
in any published snapshot and publishing one means pushing to `main`. The install
still goes through `cubePackages` → `node_modules` → `<root>/cubes`, which is the
part §4.1 is about; what it does *not* exercise is the registry and dist-tag half
of the documented command.
Two VMs, as specified: the first got `user:add` then `runtime:nodevm` with
`SHELL=fish`, the second (destroyed and recreated, no fish, no `user:add`) got
`runtime:nodevm` alone under both shells. Driven through `scripts/expect.py`, so
the interactive path is what was exercised.
**Findings**, all recorded in `DOCS-AUDIT.md`:
- §6.8 — `--print-only` was recorded in history where `--dry-run` was not, so a
`-P` pass displaced the last real deployment at the head of what `-R` repeats.
Fixed.
- §6.9 — picking `user:add` and `runtime:nodevm` together resolves nodevm first.
The first write-up blamed the ordering and was wrong: emission is already
post-order over `dependencies()`, so a declared edge wins over list order
whichever way round the two were listed, and a test now pins that. What list
order decides is where a cube with *no* edge lands — and `runtime:nodevm`
declares none, deliberately, because `user:add` creates a user. §5.3's
`DeployError` is the guard for that pair; the acceptance run used two
invocations.
- §6.10 — `runtime:nodevm` installed apt packages without refreshing the index,
which only surfaced once §5.3 let the cube run on a box where `apt:essentials`
had not. Fixed.
---
## Suggested order
1. **Phase 1** — the leak. Security, and the fix is contained.
2. **Phase 4.3–4.5** — the error boundary, the terminal proxy, and the
completeness check. Small, and they stop a silently wrong deployment.
3. **Phase 2** — remove `--json`. Mostly deletion, and it settles what the
scripting surface *is* before phase 6 documents it.
4. **Phase 6 + 4.1 + 4.2** — documentation. No code, immediate payoff for the
next new user.
5. **Phase 5** — cubes. Independent of everything above; ships with the bundle,
not the CLI.
6. **Phase 3** — session and replay. Largest surface, lowest severity.
7. **Phase 7** — harness and acceptance, last, so it exercises all of it.
+75 -17
View File
@@ -131,7 +131,7 @@ tarball.
```
checkout → resolve tag → check secrets
→ pnpm → node → cache → install → check linked deps are released
→ pnpm → node → cache → install
→ lint:ci → typecheck → test:coverage → build → verify-pack
→ publish to Gitea → publish to npmjs → delete .npmrc
→ create the Gitea release → step summary
@@ -141,13 +141,14 @@ Tag resolution and the secret check run **before** anything is installed or
built, so a malformed tag or a missing token fails in seconds instead of after
the whole gate.
*Check linked deps are released* asks npmjs whether every `workspace:` dependency
of the package being released already exists at the version pnpm is about to
bake in (`scripts/linked-deps.mjs` → `npm view`). Tagging `nopy-v1.3.0` while
`@bitsquare/nopy-cubes@1.1.0` is still unpublished would otherwise ship a tarball
nobody can install, and npmjs only lets you unpublish for 72 hours. The check is
npmjs-only: it runs before any credentials are written, and npmjs is the registry
where the mistake is permanent.
There used to be a *check linked deps are released* step between install and
lint, refusing to publish a package whose `workspace:` dependency was not yet on
npmjs. It is gone: [`scripts/release.mjs`](#cutting-a-release) is what creates
release tags now, and it pushes them dependency-first and waits for each version
to resolve on npmjs before pushing the next — so the ordering is enforced before
CI sees a tag rather than after. The trade is that a tag pushed by hand is no
longer caught; `node scripts/linked-deps.mjs <dir>` still prints what a package
would bake in if you want to check yourself.
## The verification gate
@@ -236,14 +237,60 @@ edit is discarded with the workspace and is never committed.
## Cutting a release
```sh
pnpm run release
```
`scripts/release.mjs` does the whole sequence: pick the packages, pick each
version, write the release notes, run the gate, commit, tag and push. Everything
below describes what it does and how to do it by hand.
It runs in this order, and the order is the point:
1. **Preflight.** Refuses a dirty working tree (a release commit must contain the
bump and nothing else), warns if you are not on `main`, and refuses to run
when `main` is behind the remote — a tag on a stale commit ships a tree
nobody reviewed.
2. **Pick.** A checklist of the publishable packages, each annotated with its
local version and what npmjs already has. If you select a package that others
link to, it says so and offers to add them.
3. **Version.** `patch`/`minor`/`major`/`prerelease` computed from the manifest,
or type your own. Versions already on npmjs, and versions whose tag exists,
are shown struck out and cannot be chosen. A version that will not move
`latest` gets a warning rather than a refusal.
4. **Notes.** Opens `$EDITOR` seeded with the commits since the package's last
tag, and prepends the result to `packages/<pkg>/CHANGELOG.md` in the format
the release body parser expects.
5. **Verify.** `lint:ci → typecheck → test:coverage → build → verify-pack`,
against the bumped tree and **before** the commit, so a failure leaves nothing
to unpick — it offers to restore the tree instead.
6. **Commit, tag, push.** One commit, one annotated tag per package, then the
branch, then the tags **dependency-first**. After each tag it polls npmjs
until that exact version resolves before pushing the next.
Useful flags:
```sh
pnpm run release -- --dry-run # print the plan, change nothing
pnpm run release -- -p nopy -v minor # skip the pickers
pnpm run release -- -p nopy-cubes nopy # several, ordered automatically
pnpm run release -- --no-verify # skip the gate (it still runs in CI)
pnpm run release -- --no-wait # push tags back to back
```
The push uses `SKIP_SIMPLE_GIT_HOOKS=1`, because the `pre-push` gate is the same
one step 5 just ran against the same tree.
### By hand
1. Bump `version` in `packages/<pkg>/package.json`.
2. Add a changelog entry (see below).
3. Commit, merge to `main`, and let the snapshot workflow go green.
4. Tag that commit and push the tag:
```sh
git tag nopy-v1.2.0
git push origin nopy-v1.2.0
git tag nopy-v<version>
git push origin nopy-v<version>
```
The tag name is `<directory>-v<version>` — the directory under `packages/`, not
@@ -267,10 +314,13 @@ waiting for each run to go green:
nopy-cubes → nopy, nopy-cubes-core (these two are independent of each other)
```
Release `nopy` first and the run stops at the *check linked deps* step, telling
you the `nopy-cubes` version it wanted is not on npmjs. That is the guard working;
release `nopy-cubes`, then re-tag. `node scripts/publish-order.mjs` prints the
order if you would rather not reason about it.
`pnpm run release` handles this for you — it sorts the selection over the
`workspace:` edges and will not push `nopy`'s tag until `nopy-cubes`'s new
version answers on npmjs. Releasing by hand, you own it: tag `nopy` first and its
run publishes a tarball requiring a `nopy-cubes` version that does not exist, and
npmjs only lets you unpublish for 72 hours. `node scripts/publish-order.mjs`
prints the order, and `node scripts/linked-deps.mjs <dir>` prints exactly which
versions a package would bake in.
Bumping `nopy-cubes` means bumping the packages that depend on it in the same
change — the `workspace:*` range resolves to whatever version is in the workspace
@@ -299,10 +349,12 @@ What a successful run leaves behind:
## Changelogs and release notes
Neither package has a `CHANGELOG.md` yet. Without one, the Gitea release body is
just the install snippet — nothing fails.
`pnpm run release` writes these for you — it opens `$EDITOR` seeded with the
commits since the package's last tag and prepends a `## <version> — <date>`
section, creating the file the first time. A package with no `CHANGELOG.md` is
fine: the Gitea release body degrades to the install snippet and nothing fails.
When you add one, `release.yml` extracts the section for the version being
`release.yml` extracts the section for the version being
released. The parser is deliberately dumb: it looks for the first `## ` heading
whose text contains the version string, and takes every line until the next `## `
heading. Any of these work:
@@ -644,6 +696,12 @@ node scripts/publish-order.mjs # the order to release in
node scripts/linked-deps.mjs packages/nopy # what must be on the registry first
```
Rehearse a release without touching anything:
```sh
pnpm run release -- --dry-run
```
See what is on each registry, and which versions Gitea has that npmjs does not:
```sh
+9 -4
View File
@@ -4,10 +4,11 @@ Infrastructure tooling monorepo: two published CLIs plus the pyinfra "cubes"
they deploy.
| Path | Package | Binary | What it is |
| ----------------- | ------------------ | -------- | --------------------------------------------------- |
| -------------------------- | ---------------------------- | -------- | --------------------------------------------------- |
| `packages/nopy` | `@bitsquare/nopy` | `nopy` | interactive pyinfra script management and execution |
| `packages/keyman` | `@bitsquare/keyman` | `keyman` | SSH key management with `age` encryption |
| `cubes/` | — | — | the deployment units `nopy` runs |
| `packages/nopy-cubes` | `@bitsquare/nopy-cubes` | — | the authoring surface a cube's `manifest.mjs` imports |
| `packages/nopy-cubes-core` | `@bitsquare/nopy-cubes-core` | — | the core bundle of deployment units `nopy` runs |
```sh
npm install -g @bitsquare/nopy @bitsquare/keyman
@@ -28,7 +29,7 @@ pnpm install
| Command | Does |
| --------------------------- | --------------------------------------------------- |
| `pnpm run build` | compiles both packages with `tsc` |
| `pnpm run typecheck` | `tsc --build --noEmit` across the workspace |
| `pnpm run typecheck` | `tsc --build` across the workspace (see below) |
| `pnpm run lint` | Biome check |
| `pnpm run lint:fix` | Biome check with fixes applied |
| `pnpm test` | vitest, both packages |
@@ -36,7 +37,11 @@ pnpm install
| `pnpm run coverage:summary` | renders the last coverage run as a Markdown table |
`typescript` is on the 7.x native compiler, so `tsc` *is* the fast one — there is
no separate `tsgo` binary to keep in sync. Each package also has a dev-run script
no separate `tsgo` binary to keep in sync. `typecheck` is plain `tsc --build`,
not `--noEmit`: once a project has `references`, TypeScript rejects `--noEmit`
outright (TS6310), because a composite project has to emit the declarations its
dependents read. So the typecheck writes `dist` as a side effect — gitignored,
and it means the gate also proves the build works. Each package also has a dev-run script
(`pnpm --filter @bitsquare/nopy run nopy`) that executes the TypeScript sources
directly through `tsx`.
Vendored
+88
View File
@@ -1,6 +1,27 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
require 'fileutils'
# A destroyed-and-recreated box generates fresh SSH host keys, so the entry in
# ~/.ssh/known_hosts for [127.0.0.1]:2222 goes stale and pyinfra aborts with
# "Host key ... does not match" — it reads the real known_hosts, because its
# @vagrant connector copies only HostName/Port/User/IdentityFile out of
# `vagrant ssh-config` and drops the StrictHostKeyChecking/UserKnownHostsFile
# lines vagrant emits. Nor would relaxing that help: paramiko rejects a
# *mismatched* key before any policy is consulted.
#
# So: keep one keypair on the host, install it into every incarnation of the
# VM, and pin the known_hosts entry to it after boot.
HOSTKEY_DIR = File.join(__dir__, '.vagrant-hostkeys')
HOSTKEY_PATH = File.join(HOSTKEY_DIR, 'ssh_host_ed25519_key')
unless File.exist?(HOSTKEY_PATH)
FileUtils.mkdir_p(HOSTKEY_DIR)
system('ssh-keygen', '-q', '-t', 'ed25519', '-N', '', '-C', 'ansiblingsvm', '-f', HOSTKEY_PATH) \
or raise "Vagrantfile: ssh-keygen failed to create #{HOSTKEY_PATH}"
end
Vagrant.configure("2") do |config|
config.vm.provider "vmware_desktop" do |vmware|
vmware.gui = false
@@ -19,4 +40,71 @@ Vagrant.configure("2") do |config|
# echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICpnZ6IxwQKL1rBE4dy7w5Sd3s2tLFZUDfjH87C1QIlc bdiedrichsen@Benjamins-MBP.lan" >> ~/.ssh/authorized_keys
# chmod 600 ~/.ssh/authorized_keys
#SHELL
# Land the private key as the vagrant user first; the shell provisioner
# below is what moves it into /etc/ssh with root ownership and 0600.
config.vm.provision "hostkey-upload",
type: "file",
run: "always",
source: HOSTKEY_PATH,
destination: "/tmp/ssh_host_ed25519_key"
config.vm.provision "hostkey-upload-pub",
type: "file",
run: "always",
source: "#{HOSTKEY_PATH}.pub",
destination: "/tmp/ssh_host_ed25519_key.pub"
# Idempotent: only restarts sshd when the key actually changed, so a
# `vagrant up` on an untouched VM does not bounce the connection.
config.vm.provision "hostkey-install",
type: "shell",
run: "always",
inline: <<-SHELL
set -eu
if cmp -s /tmp/ssh_host_ed25519_key /etc/ssh/ssh_host_ed25519_key \\
&& [ -f /etc/ssh/sshd_config.d/99-pinned-hostkey.conf ]; then
rm -f /tmp/ssh_host_ed25519_key /tmp/ssh_host_ed25519_key.pub
echo "host key already pinned"
exit 0
fi
install -o root -g root -m 600 /tmp/ssh_host_ed25519_key /etc/ssh/ssh_host_ed25519_key
install -o root -g root -m 644 /tmp/ssh_host_ed25519_key.pub /etc/ssh/ssh_host_ed25519_key.pub
rm -f /tmp/ssh_host_ed25519_key /tmp/ssh_host_ed25519_key.pub
# Offer *only* this key. Ubuntu's sshd_config Includes sshd_config.d/*
# before its own (commented-out) HostKey lines, and naming any HostKey
# replaces the built-in default set — so a regenerated RSA or ECDSA key
# can never become the identity a client pins.
mkdir -p /etc/ssh/sshd_config.d
echo "HostKey /etc/ssh/ssh_host_ed25519_key" > /etc/ssh/sshd_config.d/99-pinned-hostkey.conf
chmod 644 /etc/ssh/sshd_config.d/99-pinned-hostkey.conf
sshd -t
systemctl restart ssh 2>/dev/null || service ssh restart
echo "host key pinned"
SHELL
# Established sessions survive the sshd restart above, but the *next*
# connection sees the new key — so refresh known_hosts on the host from the
# public key we already hold, rather than blind-trusting a keyscan.
config.trigger.after [:up, :provision, :reload] do |trigger|
trigger.name = "pin known_hosts entry"
trigger.ruby do |_env, machine|
info = machine.ssh_info
next if info.nil?
entry = "[#{info[:host]}]:#{info[:port]} #{File.read("#{HOSTKEY_PATH}.pub").split[0, 2].join(' ')}"
known_hosts = File.expand_path('~/.ssh/known_hosts')
FileUtils.mkdir_p(File.dirname(known_hosts), mode: 0o700)
FileUtils.touch(known_hosts) unless File.exist?(known_hosts)
system('ssh-keygen', '-q', '-R', "[#{info[:host]}]:#{info[:port]}", '-f', known_hosts,
out: File::NULL, err: File::NULL)
FileUtils.rm_f("#{known_hosts}.old")
File.open(known_hosts, 'a') { |f| f.puts(entry) }
machine.ui.info("known_hosts pinned to #{entry.split[1, 2].first} for #{info[:host]}:#{info[:port]}")
end
end
end
+6 -1
View File
@@ -14,6 +14,7 @@
"test:coverage": "pnpm -r run test:coverage",
"coverage:summary": "node scripts/coverage-summary.mjs",
"registry:status": "node scripts/registry-status.mjs",
"release": "node scripts/release.mjs",
"try:snapshot": "node scripts/try-snapshot.mjs",
"typecheck": "tsc --build",
"lint": "biome check .",
@@ -31,7 +32,11 @@
"@bitsquare/nopy-cubes-core": "workspace:*",
"@logtape/logtape": "^2.2.4",
"@types/node": "^26.1.1",
"commander": "^15.0.0",
"enquirer": "^2.4.1",
"semver": "^7.8.5",
"simple-git-hooks": "^2.13.1",
"typescript": "^7.0.2"
"typescript": "^7.0.2",
"zx": "^8.8.5"
}
}
+212 -109
View File
@@ -1,104 +1,80 @@
# Keyman - SSH Key Management with Age Encryption
# keyman — SSH key management with an age-encrypted vault
Keyman is a simple command line tool built around the `age` encryption tool. It allows you to manage SSH keys in public GitHub repositories securely by encrypting the private keys.
keyman keeps SSH private keys in a vault you can commit. Each key is encrypted
with [age](https://github.com/FiloSottile/age) to a single recipient — the vault's
identity file — which is the one thing that has to stay out of the repository.
## Features
It is an interactive menu rather than a set of subcommands: point it at a vault,
pick an operation, repeat until you quit.
- 🔐 Encrypt SSH private keys with age encryption
- 📁 Organized vault structure: `vault/keys/` for encrypted keys, `vault/tmp/` for decrypted keys
- ⚙️ Configurable via `.keymanrc.json` with sensible defaults
- 🔍 Interactive CLI for encrypting, decrypting, and listing keys
- 🔄 Support for key rotation
## Requirements
## Quick Start
`age`, `age-keygen` and `ssh-keygen` on `PATH`. keyman shells out to all three and
names the missing one instead of failing obscurely.
### 1. Generate Age Encryption Key
## Installing
```bash
# Create vault structure
mkdir -p vault/keys vault/tmp
```sh
npm install -g @bitsquare/keyman@main \
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
```
# Generate age encryption key (keep this secret!)
Point the **scope** at that registry rather than setting a bare `--registry`: it
serves `@bitsquare` packages only and does not proxy npmjs, so every other
dependency has to keep resolving from npmjs. Reading needs no token while the
repository is public, and the same line works with `pnpm`.
`@main` is a snapshot of the default branch, published on every push. Name a tag —
there is no `latest` on that registry yet, so an untagged install resolves to
nothing, and keyman has not been released to npmjs. `keyman self-update` keeps you
on whichever channel you installed from.
## Quick start
```sh
# The vault identity. The only secret in the vault, and the only thing here you
# cannot regenerate — back it up somewhere that is not this repository.
mkdir -p vault
age-keygen -o vault/age.key
# Add to .gitignore
echo "vault/age.key" >> .gitignore
echo "vault/tmp/" >> .gitignore
```
### 2. Generate SSH Keys
```bash
# Generate SSH key pair
ssh-keygen -t ed25519 -f vault/tmp/id_deploy -N "" -C "deploy@myapp.dev"
```
### 3. Run Keyman
```bash
# Run keyman interactively
# Run keyman against it.
VAULT_ROOT=./vault keyman
# Or if you have .keymanrc.json configured, just run:
keyman
```
## Configuration
On startup keyman creates `keys/` and `tmp/` under the vault at `0700` and writes
a `.gitignore` beside them covering the identity and `tmp/`, so a fresh vault
cannot be committed by accident.
Keyman uses sensible defaults but can be customized via `.keymanrc.json`:
```json
{
"vaultRoot": "./vault",
"keysDir": "keys",
"tmpDir": "tmp",
"ageKeyFile": "age.key"
}
```
### Configuration Priority
1. **VAULT_ROOT** environment variable (highest priority)
2. **.keymanrc.json** file (searched from current directory upward)
3. **Default values** (lowest priority)
### Default Values
- `vaultRoot`: `"vault"`
- `keysDir`: `"keys"`
- `tmpDir`: `"tmp"`
- `ageKeyFile`: `"age.key"`
## Vault Structure
```
project/
├── vault/
│ ├── age.key # Master encryption key (NEVER commit!)
│ ├── keys/ # Encrypted keys (safe to commit)
│ │ └── deploy/ # Each key has its own folder
│ │ ├── id_deploy.pub # Public key
│ │ └── id_deploy.age # Encrypted private key
│ └── tmp/ # Decrypted keys (NEVER commit!)
│ ├── id_deploy # Decrypted private key
│ └── id_deploy.pub # Public key
└── .keymanrc.json # Configuration (optional)
```
Then pick **🆕 Generate key**: it makes the key pair and encrypts it into the vault
in one step. `ssh-keygen` collects the passphrase itself — keyman never sees it, so
it can never put it on a command line.
## Operations
Keyman provides an interactive menu-driven interface with the following operations:
Every operation returns to the menu, so a session can run several.
- **📋 List keys** - Compact view showing all keys with checkbox indicators for their locations
- **🔒 Encrypt keys** - Encrypt SSH keys from `vault/tmp/` and store in `vault/keys/`
- **🔓 Decrypt keys** - Decrypt keys from `vault/keys/` to `vault/tmp/` or `~/.ssh/`
- **❌ Quit** - Exit the program
- **📋 List keys** — every key it can see and where it is: encrypted in the vault,
decrypted in `tmp/`, live in `~/.ssh`, or some combination.
- **📝 Copy public key** — the public half of a key in `~/.ssh` or `tmp/`, to the
clipboard via whichever of `pbcopy`, `clip`, `wl-copy`, `xclip` or `xsel` exists.
With none of them, it prints the key instead.
- **🆕 Generate key** — an `ed25519` or 4096-bit `rsa` pair into `tmp/`, encrypted
into the vault straight away.
- **🔒 Encrypt keys** — pick from the private keys in `~/.ssh` *and* `tmp/`; each
goes to `<keysDir>/<name>/` with its public half beside it. A key that has no
`.pub` file gets one derived with `ssh-keygen -y`. One key failing costs only
that key.
- **🔓 Decrypt keys** — pick from the vault and decrypt to `tmp/` or `~/.ssh`.
Never overwrites a file without asking first, and the plaintext key is `0600`
from the moment it exists.
- **🔄 Rotate key** — a replacement for a vault key, encrypted *alongside* the
original. See below.
- **🗑️ Retire key** — the other half of a rotation: delete a vault key and its
plaintext copies, after listing every path that goes.
- **🧹 Clear decrypted keys** — remove the plaintext keys from `tmp/`.
- **❌ Quit**
After completing any operation, keyman automatically returns to the main menu, allowing you to perform multiple operations in a single session without restarting the tool.
### List Keys Output
The list command shows a compact, unified view of all SSH keys with their locations:
### Listing
```
🔑 SSH Keys:
@@ -117,36 +93,163 @@ The list command shows a compact, unified view of all SSH keys with their locati
⚠️ = Unmanaged (in .ssh or tmp, not encrypted in vault)
```
**Features:**
- Public keys are indicated with `(.pub)` suffix instead of separate entries
- Status emoji shows management state at a glance
- Checkboxes `[✓]` show presence in three locations:
- **[Vault]** - Encrypted in vault/keys/
- **[Tmp]** - Decrypted in vault/tmp/
- **[.ssh]** - Active in ~/.ssh/
- Alphabetically sorted for easy scanning
- New **🔓** status for keys decrypted to tmp but not yet in .ssh
`(.pub)` means a public key was found next to the private one, in either location.
The rows are sorted by name.
## Example Usage
### Rotating a key
```bash
# Using environment variable
VAULT_ROOT=../../vault keyman
Rotation is deliberately two operations, because both keys have to exist at once:
# Using default configuration
keyman
1. **🔄 Rotate key**, and pick `prod`. keyman generates `id_prod-2` in `tmp/`,
encrypts it to `keys/prod-2/`, and prints both public keys. `prod` is untouched.
2. Add the `prod-2` public key wherever `prod` is authorized.
3. Check that you can log in with `tmp/id_prod-2`.
4. Remove the `prod` public key from those hosts.
5. **🗑️ Retire key**, and pick `prod`.
# Keyman will show:
# 📁 Vault Root: /path/to/vault
# 🔑 Keys Directory: /path/to/vault/keys
# 📂 Temp Directory: /path/to/vault/tmp
# 🔐 Age Key: /path/to/vault/age.key
The name has to change: the vault directory is derived from it, so a replacement
also called `prod` *is* the `prod` entry. Rotating again continues the series
(`prod-2` → `prod-3`), and a version already taken — in the vault, in `tmp/` or in
`~/.ssh` — is skipped rather than overwritten.
Doing it in one step instead is what this shape avoids: replace the key in the
vault and you have locked yourself out of the host you were rotating for, because
the replacement is not on it yet and the only copy of the key that is has gone.
Retiring warns when nothing in the vault supersedes the key, and then asks you to
type its name.
### The `id_` prefix
keyman manages keys named `id_*`; the vault directory for `id_prod` is `prod`.
A private key named anything else is not offered by any operation — but List, Copy
and Encrypt report the ones they found, with a count and the reason, so it is
never silently invisible. Rename it to `id_<name>` to bring it in.
## Command line
```
keyman — SSH key management and an age-encrypted key vault
Usage
keyman start the interactive menu
keyman self-update update keyman itself (alias: upgrade)
Flags
-h, --help print this help and exit
-V, --version print the version and exit
--print-config print the resolved paths and the config files
they came from, as JSON, and exit
--self-update same as the self-update subcommand
Flags for self-update
--channel <latest|next|main> channel to update from
(default: derived from the running version)
--registry <url> registry to query instead of the configured one
-n, --dry-run print the install command without running it
-f, --force reinstall even when already up to date
Environment
VAULT_ROOT overrides vaultRoot from .keymanrc.json
KEYMAN_REGISTRY registry for the update check and self-update
KEYMAN_REGISTRY_TOKEN bearer token for a private registry
KEYMAN_NO_UPDATE_CHECK set to 1 to skip the once-a-day update check
(also skipped whenever CI is set)
KEYMAN_PACKAGE_MANAGER npm | pnpm | yarn | bun for the install command
Configuration is read from .keymanrc.json, merged from the current directory
upwards and then from ~/.keymanrc.json.
```
## Best Practices
The update channel is derived from the version you are running — a `-main.` build
checks `main`, any other prerelease checks `next`, a clean version checks `latest`
— so an update cannot quietly move you to a different channel. The check runs at
most once a day and prints its hint to **stderr**, which keeps `--print-config`
machine-readable.
1. **Never commit** `vault/age.key` or `vault/tmp/` to version control
2. **Always backup** your `age.key` securely (password manager, encrypted USB)
3. **Commit** `vault/keys/` - encrypted keys are safe to share
4. **Use environment variables** for CI/CD: `VAULT_ROOT=/path/to/vault keyman`
5. **Keep .keymanrc.json** in your project root for team consistency
## Configuration
`.keymanrc.json`, with every key optional:
```json
{
"vaultRoot": "vault",
"keysDir": "keys",
"tmpDir": "tmp",
"ageKeyFile": "age.key"
}
```
| key | default | meaning |
| ------------ | ---------- | ---------------------------------------------------- |
| `vaultRoot` | `vault` | the vault directory; everything else lives inside it |
| `keysDir` | `keys` | the encrypted keys — the part that is safe to commit |
| `tmpDir` | `tmp` | decrypted keys, in plaintext |
| `ageKeyFile` | `age.key` | the age identity the vault encrypts to |
The last three are resolved against `vaultRoot` unless they are absolute. A
relative `vaultRoot` **in a config file** is resolved against that file's
directory, so a repository config keeps meaning the same vault from any
subdirectory; the built-in default is resolved against the current directory.
Files are read from `~/.keymanrc.json` first, then from the filesystem root down
to the current directory, so the nearest file wins key by key. `VAULT_ROOT` in the
environment beats all of them. A file that is not valid JSON is skipped with a
warning rather than taken as fatal, and a key keyman does not know is reported
instead of silently dropped — `{"vaultroot": "…"}` used to be indistinguishable
from an empty file.
`keyman --print-config` answers what all of that resolved to, and which files it
came from:
```sh
$ keyman --print-config
{"vaultRoot":"/srv/infra/vault","keysDir":"/srv/infra/vault/keys","tmpDir":"/srv/infra/vault/tmp","keyPath":"/srv/infra/vault/age.key","configFiles":["/srv/infra/.keymanrc.json"]}
```
## Vault layout
```
project/
├── vault/
│ ├── .gitignore # written by keyman: the identity and tmp/, not keys/
│ ├── age.key # the vault identity (NEVER commit)
│ ├── keys/ # encrypted keys (safe to commit)
│ │ └── deploy/ # one directory per key, named without the id_ prefix
│ │ ├── id_deploy.age # the private key, encrypted to the vault recipient
│ │ └── id_deploy.pub # the public key
│ └── tmp/ # decrypted keys (NEVER commit)
│ ├── id_deploy
│ └── id_deploy.pub
└── .keymanrc.json # optional
```
With a custom `keysDir` or `tmpDir`, those two names change and nothing else does.
## Practices this tool assumes
1. **Back up `age.key`** somewhere outside the repository. It is the only thing
that can decrypt the vault, and nothing in the vault can reconstruct it.
2. **Commit `keys/`.** Encrypted keys are the point; a vault nobody shares is a
directory.
3. **Do not commit the identity or `tmp/`.** keyman writes a `.gitignore` for
this, and never overwrites one you wrote yourself — check it if you brought
your own.
4. **Clear `tmp/` when you are done with it** (🧹), so plaintext keys do not
outlive the reason they were decrypted.
5. **Keep `.keymanrc.json` in the project root** so everyone resolves the same
vault, and use `VAULT_ROOT` for the exceptions.
## Upgrading from a version before 0.7.0
`keysDir` and `tmpDir` used to be honoured by some operations and ignored by
others, which left anyone with custom names holding a **split vault**: `generate`
and `list` used the configured directories while `encrypt` and `decrypt` used
`<vaultRoot>/keys` and `<vaultRoot>/tmp`. All of them agree now, so anything
written by the old `encrypt` needs moving once:
```sh
mv <vaultRoot>/keys/* <vaultRoot>/<keysDir>/
```
Nobody on the default names is affected — for them the two halves were the same
directory all along.
+822
View File
@@ -0,0 +1,822 @@
# keyman audit
A review of `packages/keyman` for defects, unimplemented features, and drift
between the code and the documents that describe it.
Severity is about what it costs a user:
- **🔴 broken** — normal use produces a crash, data loss, or a silently wrong result.
- **🟠 misleading** — the code or a document states something that is not true.
- **🟡 gap** — something real that nothing mentions, or dead weight nobody uses.
Verified against `75983ab` with no uncommitted changes in the package. Line
numbers are from that state. Findings marked **verified** were reproduced by
running the code, not inferred from reading it; the reproduction is quoted.
Baseline: 162 tests pass, 98.9 % lines / 96.2 % branches. High coverage is
context for §1.2, not a defence of it.
## Status
**All 30 findings are closed except the second half of §1.8**, over the ten phases
of `PLAN.md`. Each one keeps its original text as the record, with what closed it
quoted underneath; the line numbers still point at `75983ab`, so they are history
rather than directions. The one deliberate omission is making keys not named `id_*`
*manageable* — they are now reported rather than silently skipped, and the rest is a
change to the on-disk layout that wanted sizing first.
Where the audit ends: 336 tests, 99.3 % statements / 95.7 % branches.
---
## Contents
- [1. Defects](#1-defects)
- [2. Security](#2-security)
- [3. Unimplemented and dead](#3-unimplemented-and-dead)
- [4. Public API and packaging](#4-public-api-and-packaging)
- [5. Documentation drift](#5-documentation-drift)
- [6. Checked and accurate](#6-checked-and-accurate)
- [Suggested order of attack](#suggested-order-of-attack)
---
## 1. Defects
### 1.1 ✅ `keysDir` and `tmpDir` are honoured by half the tool — **fixed**
> **Closed in Phase 5.** `main.ts` passes `paths.keysDir` and `paths.tmpDir` to
> `encrypt` and `decrypt`, neither of which joins `vaultRoot` itself any more, so all
> five operations agree on the configured directories. `tests/vault-layout.test.ts` is
> the regression test: non-default names in a real config file, the real loader, one
> encrypt, and a listing that has to show the key in the vault column.
`resolveConfigPaths()` (`keyman.config.ts:249-259`) resolves all four paths from
the config, and `keyman.main.ts:18-21` prints them. But dispatch is inconsistent
about what it hands each operation:
| Operation | receives | uses |
| --- | --- | --- |
| `listKeys` (`main.ts:67`) | `paths.keysDir` | the configured directory |
| `generateKey` (`main.ts:73`) | `paths.keysDir`, `paths.tmpDir` | the configured directories |
| `copyKey` (`main.ts:70`) | `paths.tmpDir` | the configured directory |
| `encryptKeys` (`main.ts:76`) | `paths.vaultRoot` | **hardcoded** `<vaultRoot>/keys` (`encrypt.ts:38`) |
| `decryptKeys` (`main.ts:84`) | `paths.vaultRoot` | **hardcoded** `<vaultRoot>/keys` (`decrypt.ts:7`) and `<vaultRoot>/tmp` (`decrypt.ts:39,43`) |
With the defaults the two halves agree, which is why this is invisible. Set
either sub-directory and the vault splits in two.
**Verified.** With `{"vaultRoot":"./v","keysDir":"encrypted","tmpDir":"plain"}`,
`keyman --print-config` reports:
```json
{"...":"...","keysDir":"…/v/encrypted","tmpDir":"…/v/plain","keyPath":"…/v/age.key"}
```
so `generate` writes to `v/encrypted/<name>/` and `list` scans `v/encrypted`,
while `encrypt` writes to `v/keys`, and `decrypt` reads `v/keys` and writes
`v/tmp` — never touching either configured directory. Concretely:
- encrypt a key, then list it → the list shows nothing in `[Vault]`.
- generate a key, then decrypt it → "⚠️ No encrypted keys found."
- decrypt to local, then encrypt → the key is not offered, because `encrypt`
reads the configured `tmpDir` while `decrypt` wrote to the hardcoded one.
No error at any point. The user has two vaults and one of them is invisible to
whichever operation they try next.
The current behaviour is locked in by tests: `main.test.ts:164-187` asserts
`vaultRoot` is what encrypt and decrypt receive ("encrypts keys into the vault
root"), and `encrypt.test.ts:95` / `decrypt.test.ts:53` assert the literal
`keys` segment. Fixing this means changing those assertions.
**Fix.** Pass `paths.keysDir` and `paths.tmpDir` into `encryptKeys` and
`decryptKeys` and delete the three `path.join(vaultDir, 'keys' | 'tmp')` calls.
Neither function has a use for `vaultRoot` once that is done, so the parameter
goes away rather than becoming a second source of truth.
See also §5.1 — `DOCS-AUDIT.md` currently lists this layout under *checked and
accurate*.
### 1.2 ✅ Encrypt and decrypt crash with a raw stack trace on a first run — **fixed**
> **Closed in Phases 1 and 2.** `keyman.cli.ts` is an error boundary — a
> `UsageError` prints one line, anything else prints its message and exits 1, and
> neither prints a stack. The two readdirs that threw are guarded (§1.5).
Three `readdirSync` calls have no `existsSync` guard:
- `encrypt.ts:13` — `~/.ssh`, which nothing creates.
- `encrypt.ts:16` — the tmp directory.
- `decrypt.ts:8` — `<vault>/keys`, which nothing creates either.
`keyman.main.ts:40-41` creates `vaultRoot` and `tmpDir`. It does **not** create
`keysDir`, so `decrypt` on a fresh vault throws instead of printing its
"⚠️ No encrypted keys found." message — the message is unreachable until the
directory exists for some other reason.
**Verified**, calling both functions directly against a vault laid out the way
`main.ts` lays it out:
```
--- A: decryptKeys with no vault/keys directory ---
THREW: Error ENOENT ENOENT: no such file or directory, scandir '…/vault/keys'
--- B: encryptKeys with no ~/.ssh directory ---
THREW: Error ENOENT ENOENT: no such file or directory, scandir '…/home/.ssh'
```
What the user sees is worse than the exception, because of `keyman.cli.ts:82`:
```ts
keyman();
```
Not awaited, no `.catch`. Any rejection anywhere in the menu loop becomes an
unhandled rejection: Node prints the stack and exits non-zero, and the menu loop
— whose whole point (`README.md:97`) is that you can run several operations in
one session — is gone.
`copyKey` guards (`copy.ts:8`) and `listKeys` guards all three of its
directories (`list.ts:22,50,78`). Encrypt and decrypt are the outliers, not the
rule.
Worth noting where the coverage numbers sit: `keyman.encrypt.ts` and
`keyman.decrypt.ts` are both at **100 % lines, 100 % branches**. Every test
creates the directories in `beforeEach` (`encrypt.test.ts:46-47`,
`decrypt.test.ts:54-55`), so the missing guard is not a branch that went
uncovered — it is a branch that was never written. Line coverage measures lines
executed, not inputs considered.
**Half closed (Phase 1).** `keyman()` is now awaited inside a `catch`, so a
rejection is one line rather than an unhandled-rejection stack trace. The missing
`existsSync` guards — and with them the menu loop surviving a failed operation —
are Phase 2.
### 1.3 ✅ A missing `age.key` becomes `age -r null` — **fixed**
> **Closed in Phase 3.** The recipient is resolved once per session, before any
> operation that needs one. A null aborts *that operation* with
> `age-keygen -o <path>` as the remedy and returns to the menu, and is retried on the
> next attempt, so creating the identity mid-session works. `age -r null` is now
> unreachable.
`keyman.main.ts:73` and `:80` assert away a null:
```ts
extractAgePublicKey(paths.keyPath)!
```
`extractAgePublicKey` returns `string | null` (`utils.ts:8-22`) and returns null
in three cases: the file is missing, it is unreadable, or it parses but has no
`# public key:` line. In all three it prints an error and returns — and the
non-null assertion carries that null straight into an `execa` argv.
**Verified**, both halves:
```
❌ ERROR: Age key file not found at /nope/age.key
extractAgePublicKey(missing) = null
execa with null recipient THREW: ExecaError | Command failed with exit code 1: age -r null -o /tmp/x.age /etc/hosts
```
execa stringifies the null, so the recipient becomes the literal `"null"`.
The two call sites fail differently, and the generate path fails worse:
- `generateKey` runs `ssh-keygen` **first** (`generate.ts:59`) and `age` second
(`generate.ts:68`). Its `try/catch` swallows the failure into "❌ Error
generating/encrypting key", but by then the private key is on disk in `tmpDir`
in plaintext, and the user has been told the operation failed. Nothing tells
them a key was left behind.
- `encryptKeys` has no `try/catch` at all, so it takes the §1.2 path: unhandled
rejection, stack trace, session over.
**Fix.** Resolve the recipient once, before dispatch, and treat null as a
recoverable condition: print what to run (`age-keygen -o <keyPath>`) and return
to the menu. The type already says this is possible; the `!` is the only thing
claiming otherwise.
### 1.4 ✅ Decrypting into `~/.ssh` silently overwrites an existing key — **fixed**
> **Closed in Phase 4.** Every collision is settled before anything is written: a
> confirmation per key defaulting to no, and a skip that says what it kept. The user
> is answering about files that still exist.
`decrypt.ts:47-49` writes the decrypted key and copies the public key with no
existence check, no confirmation, and no backup.
**Verified** that `age -o` does not refuse an existing file:
```
before: PRECIOUS EXISTING KEY
age -o exit=0 (overwrote)
after: secret
```
So selecting `prod` with the `SSH (~/.ssh)` destination replaces
`~/.ssh/id_prod` outright. If the vault copy is stale, or the folder name
happens to collide with an unrelated local key, the local key is gone — and this
is the one operation in the tool that writes outside the vault, into the
directory the user's actual SSH access depends on.
The `Local (vault/tmp)` destination has the same behaviour but a much lower cost,
since `vault/tmp` is scratch space by design.
**Fix.** Check both output paths before decrypting anything and prompt per
collision, or refuse and name the file. A `--force` equivalent can come later;
the current default should not be "overwrite".
### 1.5 ✅ `age` or `ssh-keygen` missing is unhandled in encrypt and decrypt — **fixed**
> **Closed in Phase 2.** `runTool` turns `ENOENT` into a `ToolNotFoundError`
> whose message is an instruction, and keeps it distinct from a tool that ran and
> refused — whose reason is on stderr and nowhere in execa's message. `encrypt`
> re-throws it instead of counting it against one key.
Same missing `try/catch` as §1.3. `generateKey` (`generate.ts:51-76`) and
`copyKey` (`copy.ts:43-57`) both wrap their `execa` calls and report a failure;
`encryptKeys` and `decryptKeys` do not. On a machine without `age` on `PATH` —
the one hard external requirement, per `CLAUDE.md` — choosing Encrypt from the
menu produces an `ENOENT` stack trace rather than "install age".
### 1.6 ✅ Encrypt copies `.pub` unconditionally and aborts the batch midway — **fixed**
> **Closed in Phase 6.** `storeInVault` reads the `.pub` *before* the vault
> directory exists and derives a missing one with `ssh-keygen -y` — stdout piped,
> stdin and stderr inherited, because the passphrase prompt goes to stderr — storing
> the private key alone if it cannot. And a failing key costs one key: `encrypt`
> collects the failures and names them at the end.
`encrypt.ts:45`:
```ts
fs.copyFileSync(`${keyPath}.pub`, path.join(vaultPath, `${key}.pub`));
```
The selection list is built from *private* keys only (`encrypt.ts:14,17` filter
out `.pub`), so a private key with no `.pub` sibling is offered — and that is a
legal state, since `ssh-keygen -y` regenerates a public key on demand and people
do delete them.
When it happens, `age` has already written the `.age` file, so the throw leaves
the vault holding an encrypted key with no public key. Worse, the throw escapes
the `for` loop: every remaining selected key is skipped, with no output saying
so, and the process dies via §1.2.
`generate.ts:71` has the same shape but is far less likely to fire, since
`ssh-keygen` just wrote the file.
**Fix.** Derive the public key with `ssh-keygen -y -f <key>` when the sibling is
absent, and wrap the loop body so one bad key costs one key rather than the
batch.
### 1.7 ✅ `/home/<user>` is hardcoded — **fixed**
> **Closed in Phase 8.** `keyman.home.ts` resolves a named user against the
> sibling of the current home first, then `/home/<user>` and `/Users/<user>`, and
> reports every path it tried. An unset `HOME` falls back to the passwd entry instead
> of resolving `.ssh` against the filesystem root.
`keyman.main.ts:33`:
```ts
const homeDir = user === '@current' ? process.env.HOME || '' : `/home/${user}`;
```
On macOS other users live under `/Users/`, and this tool is otherwise
macOS-specific (§1.9). Nothing checks the directory exists, so a wrong guess
feeds a nonexistent `sshDir` into §1.2 rather than into an error message.
`main.test.ts:198-204` locks in `/home/deploy/.ssh`.
**Fix.** `os.userInfo()` for the current user, and for a named user either look
the home directory up (`getent passwd` / `dscl`) or ask for the path outright.
Failing that, check `existsSync` and say so.
### 1.8 🟡 Keys not named `id_*` are invisible, silently — **half fixed**
> **Partly closed in Phase 8 — the rest is open.** `scanPrivateKeys` classifies a
> file by reading its first 64 bytes for a private-key header, and List, Copy and
> Encrypt report what they skipped, with the count, the directory and the reason. So
> the keys are no longer *silently* invisible.
>
> They are still not manageable. The vault layout derives `id_<dir>` from the
> directory name in four places, so accepting other names changes what is on disk;
> the plan asked for that to be sized before being committed to, and the report is
> the tenth of the work that closes most of the surprise. Left deliberately.
Every discovery filter requires the prefix: `copy.ts:9`, `encrypt.ts:14,17`,
`list.ts:23,51`, and `decrypt.ts:9` reconstructs `id_${dir}`. A key called
`deploy_ed25519` cannot be listed, copied, or encrypted, and nothing says why —
it simply is not in the menu.
`generateKey` enforces the prefix (`generate.ts:43`), so keys keyman creates are
always fine. The gap only bites pre-existing keys, which is exactly the
population a key manager is adopted to take over.
### 1.9 ✅ `pbcopy` is hardcoded — **fixed**
> **Closed in Phase 8.** `keyman.clipboard.ts` picks by platform — `pbcopy`,
> `clip`, or `wl-copy` → `xclip` → `xsel` — falls through only on `ENOENT` (a tool
> that ran and refused is a real error, not an absent tool), and prints the key when
> nothing is installed, since printing it was always the point.
`copy.ts:49`, with the comment above it admitting the shortcut:
```ts
// Since the environment is Darwin, we prioritize pbcopy, but we can add others for completeness
```
On Linux or Windows, Copy public key always fails. It fails *cleanly* — the
`try/catch` reports "❌ Failed to copy to clipboard" — but the package declares
only `"node": ">=22"` in `engines` and the README says nothing, so nothing warns
before install. `xclip`/`wl-copy`/`clip.exe` by platform is a handful of lines;
alternatively print the key to stdout as a fallback so the operation is never a
dead end.
### 1.10 ✅ Smaller things — **fixed**
> **Closed in Phases 2, 6 and 8.** All four: the `statSync` takes
> `throwIfNoEntry: false` and still follows a symlink to a real directory; both
> `replace` calls are anchored to `/^id_/`; a failed `age` now removes the file it
> named and the directory while it is empty, so nothing half-made is left claiming to
> hold a key; and both `console.log` debug lines are gone.
- **`listKeys` throws on a broken symlink.** `list.ts:80` calls `fs.statSync` on
every entry in the keys directory; a dangling symlink throws `ENOENT`, and
`listKeys` has no `try/catch`, so it exits via §1.2. `lstatSync`, or a
`withFileTypes` readdir, or a guard.
- **`key.replace('id_', '')` is unanchored** (`encrypt.ts:38`, `generate.ts:63`).
Every input is prefix-filtered today, so the first match *is* the prefix and
the behaviour is correct — it is a trap left for whoever loosens §1.8.
`replace(/^id_/, '')` costs nothing.
- **An `age` failure leaves an empty vault directory.** `generate.ts:65` creates
`<keysDir>/<name>/` before `generate.ts:68` runs `age`.
`generate.test.ts:150-163` asserts the `.pub` is absent afterwards but not the
directory, so this passes today. It makes the folder show up in
`decrypt`'s scan as a candidate that filters back out — harmless, untidy.
- **Debug output still in shipped code.** `encrypt.ts:18-19`
(`console.log(tmpKeys); console.log(sshKeys);`) is already tracked as
`DOCS-AUDIT.md` §6.4. `decrypt.ts:10` — a `console.log(keyfile)` *inside a
`filter` callback*, printing one line per vault directory — is not, and is the
more visible of the two.
---
## 2. Security
### 2.1 ✅ Decrypted private keys are world-readable before the chmod — **fixed**
> **Closed in Phase 4.** `fs.chmodSync(…, 0o600)` in-process, immediately after
> `age` returns — no `cp` or `chmod` spawn, so there is no window and no failure mode
> that leaves the mode behind. Confirmed again while probing Phase 10: age still
> writes 0644, and `ssh-keygen -y` refuses such a file outright.
`decrypt.ts:47-50` decrypts, then copies, then chmods — in three separate
processes:
```ts
await execa('age', ['-d', '-i', ageKey, '-o', privateKeyOut, encryptedKey]);
await execa('cp', [publicKey, publicKeyOut]);
await execa('chmod', ['600', privateKeyOut]);
```
**Verified** what `age` creates, and what `mkdirSync` at `main.ts:41` creates:
```
-rw-r--r-- …/out ← the decrypted private key, as age leaves it
drwxr-xr-x …/tmpdir ← vault/tmp, as keyman creates it
```
So a plaintext private key exists at `0644` for the lifetime of two process
spawns, inside a `0755` directory any local user can traverse. If the `chmod`
fails or the process is killed in between, it stays `0644` — and because
`decryptKeys` has no `try/catch` (§1.5), a failing `chmod` also kills the
session before the next key is even attempted.
**Fix.** `fs.chmodSync` immediately after `age` returns rather than a third
spawn; create `tmpDir` with `{mode: 0o700}` and `~/.ssh` likewise if it is
missing. Replacing `cp` and `chmod` with `fs.copyFileSync` / `fs.chmodSync` also
removes two shell-outs that do not work on Windows and cuts three spawns per key
to one.
### 2.2 ✅ The passphrase is passed on the `ssh-keygen` command line — **fixed**
> **Closed in Phase 6.** The prompt is gone and so is `-N`: `ssh-keygen` collects
> and confirms the passphrase itself with stdio inherited. A passphrase keyman never
> learns cannot leak from keyman — and a test asserts it never asks for one.
`generate.ts:53`:
```ts
const args = ['-t', algorithm, '-f', keyPath, '-N', password, '-C', identity];
```
argv is world-readable on both Linux (`/proc/<pid>/cmdline`) and macOS
(`ps -o command`) for the lifetime of the process. Any other user on the machine
can read the passphrase of a key being generated. `generate.test.ts:78-87`
asserts this exact argv.
**Fix.** Omit `-N` entirely and let `ssh-keygen` prompt on the tty — it already
asks twice and confirms, so keyman's own password prompt (`generate.ts:26-33`)
can go away rather than being replaced. That keeps the passphrase off argv
without keyman ever holding it.
### 2.3 ✅ The age recipient is trusted from a comment, never verified — **fixed**
> **Closed in Phase 3.** `age-keygen -y` derives the recipient from the secret
> key, so it cannot disagree with it. The comment survives only as a fallback for a
> machine with no `age-keygen`, behind a warning that it is unverified — and
> deliberately *not* as a fallback for `age-keygen` refusing the file, which means
> age cannot read the identity at all.
`extractAgePublicKey` (`utils.ts:16`) regexes the recipient out of a comment
line in the identity file:
```ts
fileContents.match(/^# public key:\s*(age1[^\s]+)/m)
```
Nothing checks it corresponds to the private key in that same file. Edit the
comment — or concatenate two key files — and every subsequent encryption goes to
a recipient the local identity cannot decrypt. The failure surfaces only later,
at decrypt time, on keys that may no longer exist in plaintext anywhere.
**Fix.** `age-keygen -y <keyPath>` derives the public key *from the private key*
and is exactly the tool for this. Note that `age-keygen` is currently not
invoked anywhere in the source, despite `CLAUDE.md` listing it among the
binaries keyman shells out to (§5.5).
### 2.4 ✅ Nothing manages the plaintext left in `vault/tmp` — **fixed**
> **Closed in Phase 8.** A **🧹 Clear decrypted keys** operation that lists what it
> will delete and asks before deleting it, plus a `.gitignore` written beside the
> vault covering the identity and the tmp directory — never overwriting one that is
> already there, and never claiming to cover a path outside the vault.
Decrypted keys accumulate in `vault/tmp` indefinitely. There is no shred
operation, no warning on exit, and keyman never writes the `.gitignore` its own
README (`README.md:25-26`, `:148`) tells the user to write by hand. The only
signal is the 🔓 marker in `listKeys`, which the user has to go looking for.
A "Clear decrypted keys" menu entry and a `.gitignore` written alongside the
vault on first run would cost little and close the most likely way a private key
reaches a public repository — which is the threat this tool exists to address.
---
## 3. Unimplemented and dead
### 3.1 ✅ There is no `--help` — **fixed**
> **Closed in Phase 1.** `helpText()` in `keyman.args.ts`, checked against the
> parser's own flag table by a test so a new flag cannot ship undocumented, and now
> quoted verbatim in the README by a second test (§5.3).
`keyman.cli.ts` handles `--print-config`, `--version`/`-V`, and
`self-update`/`upgrade`, then falls through to the interactive session. `--help`
is not among them, and neither is any unknown-flag handling.
**Verified.** `keyman --help` with no tty:
```
📁 Vault Root: …
? Specify USER (default: @current): (@current)
…/@inquirer/core/dist/lib/create-prompt.js:67
reject(new ExitPromptError(`User force closed the prompt with ${code} ${signal}`));
```
Two problems in one output. `--help` starts a session instead of describing the
tool, and because of §1.2 the resulting `ExitPromptError` is an unhandled
rejection with a stack trace. That second half is what a user gets from **Ctrl-C
at any prompt** — the normal way to leave an interactive CLI produces a crash
dump.
`keyman --vault foo` is likewise accepted and ignored.
**Fix.** `--help` listing the flags, the two subcommands, and the `KEYMAN_*`
environment variables (§5.3); an unknown-flag error; and a `catch` in
`keyman.cli.ts` that treats `ExitPromptError` as "goodbye" and anything else as
a one-line error. nopy uses Commander for this; keyman need not, but it does
need the behaviour.
**Closed (Phase 1).** `src/keyman.args.ts` owns the parse and the help text; the
`catch` around `keyman()` in `keyman.cli.ts` turns `ExitPromptError` into
"👋 Goodbye!" and exit 0, and anything else into one line and exit 1.
### 3.2 ✅ `flagValue` accepts things that are not values — **fixed**
> **Closed in Phase 1.** `parseArgs` rejects a value flag with no value, a boolean
> flag given one, an unknown flag, an unknown command, an unknown channel, and a
> self-update-only flag used without `self-update`. `--channel --force` is now a
> usage error rather than a request for a dist-tag that cannot exist.
`keyman.cli.ts:24-27` is `args.indexOf(name)` and `args[index + 1]`:
- `--channel=main` is not recognised.
- `--channel` as the last argument yields `undefined`.
- `keyman self-update --channel --force` sets the channel to `"--force"`, which
is cast to `Channel` (`cli.ts:47`) and flows into the dist-tag lookup at
`update.ts:176` as a key that cannot exist. The registry answers, the tag is
absent, and the user is told "Could not reach <registry>" — which is false.
Validating against the three legal channels would turn all three into one clear
error.
**Closed (Phase 1).** `parseArgs` accepts both `--flag value` and `--flag=value`,
rejects a flag swallowed as another flag's value, and validates `--channel`
against `CHANNELS`.
### 3.3 ✅ The `resolution` merge machinery has no effect — **fixed**
> **Closed in Phase 7 — deleted.** Every keyman property is a string, so a child
> simply wins; `mergeConfigs` is one spread with a comment recording why nopy needs
> more and keyman does not. `keyman.config.ts` lost ~45 lines.
`keyman.config.ts` carries `ResolutionStrategy`, `KeymanResolutionConfig`,
`mergeValue` and `mergeConfigs` — roughly 45 lines, imported from nopy's design.
Every property in `KeymanConfigSchema` is a `z.string()`. For two strings,
`mergeValue` returns `childValue` in the `override` branch (`:121-123`) and
returns `childValue` again from the primitive fallthrough (`:156`). The two
strategies are indistinguishable for every key the schema permits, and the
array-concat and deep-merge branches are unreachable through a valid config —
unknown keys pass through the merge but are then stripped by
`KeymanConfigSchema.parse` (§3.5).
So the documented knob does nothing. The doc comment at `:186-194` advertises it:
```json
{ "vaultRoot": "../vault", "resolution": { "vaultRoot": "override" } }
```
and `config.test.ts:212` — "honours an explicit override strategy" — passes for
a case where plain merge gives the same answer, so the test does not distinguish
them either.
This is a choice to make, not a bug to fix. Either drop the machinery and the
comment, or keep it deliberately as the shape a future object-valued or
array-valued option would need — and say so in a comment, since right now it
reads as functional.
### 3.4 ✅ `getConfigPaths()` is exported, tested, and called by nothing — **fixed**
> **Closed in Phase 7.** `describeConfig()` calls it, so `--print-config` prints
> `configFiles` — the files that were merged, in merge order. That was the one
> question the flag could not answer, and it existed only as unstructured stderr.
`keyman.config.ts:265` is used only by `config.test.ts:122,131`. It is not
re-exported from `src/index.ts` and not called by the CLI. nopy's equivalent
feeds `nopy.main.ts:64`.
The absence is felt: `--print-config` prints the *resolved paths* only, so there
is no way to ask which config files were consulted. That information exists only
as a stderr side effect of `loadConfig` ("✅ Loaded configuration from …"), which
is not machine-readable and is interleaved with warnings. Folding
`getConfigPaths()` into the `--print-config` JSON makes the function earn its
keep and makes the escape hatch answer the question it is for.
### 3.5 ✅ A typo in `.keymanrc.json` is silent — **fixed**
> **Closed in Phase 7.** `warnUnknownKeys` names the file, the keys it ignored and
> the keys it knows. Warned rather than fatal, which is this module's posture
> throughout, and warned per file because that is the only place the filename is in
> hand.
`KeymanConfigSchema` is a plain `z.object`, which strips unknown keys.
**Verified.** With `{"vaultRoot":"./v","vaultroot":"typo", …}`, the lowercase key
is dropped without a word and `--print-config` reports the vault from the
correct key. Had only the typo been present, the user would get the `vault`
default and no clue.
`.strict()` — or keeping the strip and logging the leftover keys as a warning —
turns a silently wrong vault into one line of output. Since `loadConfig` already
degrades to defaults rather than throwing, a warning fits the module's existing
posture better than a hard failure.
### 3.6 ✅ "Support for key rotation" does not exist — **fixed**
> **Closed in Phase 10 — built.** `keyman.rotate.ts`: **🔄 Rotate key** generates a
> replacement under the next name in the series and encrypts it *alongside* the
> original, and **🗑️ Retire key** deletes the superseded key after listing every path
> that goes, asking for the name to be typed out when nothing in the vault supersedes
> it. Two operations rather than one, because a rotation that replaces the key in
> place locks you out of the host you were rotating for.
`README.md:11`. `grep -rn "rotat" packages/keyman/src/` returns nothing. Already
tracked as `DOCS-AUDIT.md` §2.10, still open. Rotation is a genuinely useful
operation for this tool — generate a replacement, encrypt it, keep the old one
until the new one is deployed — so this is worth building rather than deleting.
### 3.7 ✅ "Copy public key and create README" — **fixed**
> **Closed in Phase 5.** The comment went with the rewrite of `encrypt`. No
> per-key README was ever written and nothing claims one now; the `README.md` fixture
> in `decrypt.test.ts` is a stray-file case, which `listVaultKeys` ignores.
`encrypt.ts:44` says it; no README is written. Suggestively,
`decrypt.test.ts:75` places a `README.md` inside the keys directory as a
fixture, so a per-key README appears to have been the intent once. Either build
it or drop the half of the comment that lies.
---
## 4. Public API and packaging
### 4.1 ✅ A shebang on the library entry point — **fixed**
> **Closed in Phase 9.** The shebang is gone, with a comment saying why the file
> does not want one. Verified against the built `dist/index.js`.
`src/index.ts:1` is `#!/usr/bin/env node`. The bin is `dist/keyman.cli.js`
(`package.json:28`); `index.ts` is the `exports["."]` target and is only ever
imported. nopy's `src/index.ts` has no shebang. Harmless, and a copy-paste
artefact.
### 4.2 ✅ The exported functions' types are not exported — **fixed**
> **Closed in Phase 9.** `KeymanConfig` and `KeymanConfigFile` are exported;
> `ResolutionStrategy` and `KeymanResolutionConfig` no longer exist (§3.3).
`src/index.ts:2` exports `loadConfig` and `resolveConfigPaths`. It does not
export `KeymanConfig`, `KeymanConfigFile`, `ResolutionStrategy` or
`KeymanResolutionConfig`, so a TypeScript consumer cannot name what `loadConfig`
returns or what `resolveConfigPaths` takes. This is the same one-line omission
`CLAUDE.md` already records for nopy's `CubePackageRef`.
### 4.3 ✅ `export * from './keyman.main.js'` exports only `keyman()` — **fixed**
> **Closed in Phase 9 — decided, and written down.** The surface is deliberately
> narrow: config resolution, the update machinery, and `keyman()`. The operation
> modules stay internal because every one of them prompts, prints and spawns, so
> there is nothing to do with a single one except rebuild the menu around it. The
> rule is now a comment at the top of `src/index.ts` rather than an accident.
The five operation modules and `extractAgePublicKey` are not on the public
surface, so the package is consumable as a library only as "run the entire
interactive menu". That may well be intended — but then `loadConfig` and
`resolveConfigPaths` being exported is the odd part, since a consumer can obtain
the paths and do nothing with them.
### 4.4 ✅ Update-module constants are half re-exported — **fixed**
> **Closed in Phase 9.** `export * from './keyman.update.js'`, so the rule is
> "all of it" and the list cannot drift again. Verified by importing the built
> `dist/index.js` and reading its keys.
`keyman.update.ts` exports `SCOPE`, `UPDATE_CACHE_DIR`, `UPDATE_CACHE_FILE`,
`DEFAULT_FETCH_TIMEOUT_MS` and `DEFAULT_CONFIG_TIMEOUT_MS`; `src/index.ts:12-31`
re-exports neither, while re-exporting `DEFAULT_CHECK_INTERVAL_MS` and
`NPMJS_REGISTRY`. Pick one rule.
---
## 5. Documentation drift
### 5.1 ✅ `DOCS-AUDIT.md` lists §1.1 under *checked and accurate* — **fixed**
> **Closed in Phase 5.** The claim `DOCS-AUDIT.md` makes — that the documented
> vault layout matches the code — is now *true*, which is the substance of it; §1.1 is
> what made it false. The entry has been amended to say what it actually checked.
`DOCS-AUDIT.md:826-827`:
> **keyman config** — priority (`VAULT_ROOT` > file > defaults), the four default
> values, and the vault layout match `keyman.config.ts` and `keyman.encrypt.ts`.
The first two clauses are correct. The third holds only because
`keyman.encrypt.ts` hardcodes `keys` — checking the documented layout against
the file that ignores the config is what made §1.1 invisible. The entry should
move out of section 7 and point at §1.1.
### 5.2 ✅ `README.md` operations list — **fixed**
> **Closed in Phase 9.** All nine menu entries are documented, and a test asserts
> the README contains every label `keyman.main.ts` offers, so a tenth cannot arrive
> undocumented. Encrypt is described as it behaves: the union of `~/.ssh` and the tmp
> directory.
`DOCS-AUDIT.md` §2.10, re-verified: `README.md:90-96` lists four menu entries;
`main.ts:54-61` has six. `Copy public key` and `Generate key` are undocumented —
the latter being the only in-tool way to create a key, which is why the Quick
Start at `README.md:33` tells the user to run `ssh-keygen` by hand.
`README.md:93` says encrypt takes keys "from `vault/tmp/`"; `encrypt.ts:12-20`
unions `~/.ssh` and tmp and offers both.
### 5.3 ✅ The README documents none of the CLI surface — **fixed**
> **Closed in Phase 9.** The README carries `helpText()` verbatim — every flag,
> both subcommand spellings, and all five environment variables — with a test that
> fails if the two diverge. Installation, the update channels and the once-a-day
> check are documented too.
`README.md` covers the interactive menu and the config file. It does not mention:
- `self-update` / `upgrade`, `--dry-run`, `--force`, `--channel`, `--registry`
- `--version` / `-V`, `--print-config`
- `KEYMAN_REGISTRY`, `KEYMAN_REGISTRY_TOKEN`, `KEYMAN_NO_UPDATE_CHECK`,
`KEYMAN_PACKAGE_MANAGER`
- the once-a-day update check, or that it is disabled when `CI` is set
`README.PUBLISH.md:552-578` documents all of it, but `package.json:37-41` ships
only `dist`, `README.md` and `LICENSE` — so a reader on the registry sees none of
it. This is the same shape as the nopy README problem closed as
`DOCS-AUDIT.md` §2.9, and keyman is now the worse of the two.
### 5.4 ✅ The README presents a configurable layout that is half-real — **fixed**
> **Closed in Phase 9.** The section documents what §1.1 made true: the three
> inner names resolve against `vaultRoot`, a relative `vaultRoot` in a config file
> resolves against that file's directory, and the built-in default resolves against
> the current directory. It ends with the migration note for a vault written by the
> old `encrypt`.
`README.md:46-70` documents `keysDir` and `tmpDir` as configuration, and
`:72-86` draws the default tree. Per §1.1 the first is only half true. Whichever
way §1.1 is resolved, this section needs an edit.
### 5.5 ✅ `CLAUDE.md` names a binary keyman never runs — **fixed**
> **Closed in Phases 3 and 9.** `age-keygen` became true in Phase 3 (`-y`, to
> derive the recipient), and `cp`/`chmod` stopped being spawned in Phase 4.
> `CLAUDE.md` now says all of that, records the deliberate `resolution` divergence
> from nopy, and lists the operations the menu actually has.
> Encryption shells out to `age` / `age-keygen` / `ssh-keygen`, which must be on
> `PATH`.
`age-keygen` appears nowhere in `packages/keyman/src`. It appears in
`README.md:22` as a manual setup step, which is presumably where the claim came
from. Either note it as a prerequisite the user runs rather than something
keyman invokes, or make §2.3 true and turn the claim into fact.
`CLAUDE.md` also does not mention that `decryptKeys` shells out to `cp` and
`chmod` (`decrypt.ts:49-50`) — see §2.1, where the recommendation is to stop.
### 5.6 ✅ The update module has not drifted from nopy's
`keyman.update.ts` and `nopy.update.ts` are described in `CLAUDE.md` as "two
near-identical copies of one module", the duplication deliberate. Diffed with
package names normalised: **every difference is a doc comment.** No behavioural
drift at all. The stated risk of the duplication has not materialised; nopy's
copy simply carries fuller comments, and porting the better ones over would cost
nothing.
---
## 6. Checked and accurate
- **Config precedence.** `VAULT_ROOT` > config file > defaults
(`config.ts:249-259`), matching `README.md:59-70`. Verified via
`--print-config`.
- **Upward traversal and the home-directory config.** `findConfigFiles`
(`config.ts:85-110`) collects root-first and de-duplicates the home config
when it is also an ancestor (`:105`).
- **`loadConfig` never throws.** Invalid JSON is skipped per file (`:220-226`)
and a failed final validation degrades to defaults (`:232-241`) — which is the
documented difference from nopy's behaviour, and it holds.
- **`extractAgePublicKey` is honest about failure.** It returns `null` in every
failure mode and prints why; the defect in §1.3 is entirely in the caller's
`!`.
- **The menu loop.** Returns to the menu after every operation
(`main.ts:45-91`), as `README.md:97` says.
- **The four default values** and the `id_<name>.age` / `id_<name>.pub` layout
inside a per-key folder, as drawn at `README.md:72-86`.
- **`listKeys` status logic** (`list.ts:127-128`) matches its legend and the
README's, including the 🔓 state.
- **The update module**, in full — see §5.6.
---
## Suggested order of attack
> Superseded by `PLAN.md`, which turned this into ten phases and is the record of
> what was actually done in what order. Kept because the reasoning about which
> findings share a shape is still the reason the phases group the way they do.
**1 — the crashes, together.** §1.2, §1.3, §1.5 and §1.10's `statSync` are all
the same shape: an unguarded call in a function with no error boundary, reaching
a `keyman()` that is never awaited. One `catch` in `keyman.cli.ts` that
distinguishes `ExitPromptError` from a real failure, plus `existsSync` guards and
`try/catch` in encrypt and decrypt, closes all of them and most of §3.1's second
half. This is the smallest change with the largest effect on what a first run
feels like.
**2 — §1.4 and §2.1.** Both are in `decryptKeys`, both are about writing outside
the vault, and one of them destroys data. Replacing `cp`/`chmod` with the `fs`
equivalents is part of the same edit.
**3 — §1.1.** Mechanical, but it changes four test assertions, so it wants to be
its own commit. Fix `DOCS-AUDIT.md` §5.1 in the same one.
**4 — decide on §3.3 and §3.6.** Both are features the documentation claims and
the code does not have; both are decisions rather than fixes. Rotation is worth
building. The `resolution` machinery probably is not, and deleting it would take
`keyman.config.ts` from 267 lines to around 220.
**5 — §5.2, §5.3 and §5.4** are one rewrite of `README.md`. It is the only
document that ships, and it currently describes two thirds of the menu and none
of the command line.
**6 — the rest.** §2.2 (drop the passphrase prompt, let `ssh-keygen` ask), §2.3
(`age-keygen -y`), §2.4 (a shred operation), §1.6 through §1.9, §3.2, §3.4,
§3.5, and the §4 one-liners.
+452
View File
@@ -0,0 +1,452 @@
# keyman remediation plan
Turns [`AUDIT.md`](./AUDIT.md) into sequenced work. Each phase is one commit,
independently landable, gate-green on its own. Section references (§) are to
`AUDIT.md`.
Ordering is by *blast radius per unit of risk*, not by severity: the error
boundary comes first because it makes every later phase's failure mode legible,
and the config threading comes late because it is the only phase that rewrites
existing test assertions.
## Status
**All ten phases have landed**, one commit each, on the `keyman-remediation`
branch. Three deviations worth knowing about:
- **Phase 6's literal instruction was impossible.** "Move the `mkdirSync` after
`age` succeeds" cannot be done — `age -o` will not create its output directory.
The goal (no leftover directory) is met by cleaning up on failure instead, which
also removes a truncated `.age` the plan had not accounted for.
- **§1.8 is half done, deliberately**, exactly as the plan asked: the skipped-key
report is in, the layout change that would make non-`id_*` keys manageable is
not. See `AUDIT.md` §1.8.
- **Rollout has not been done.** No version bump, no tag, nothing published — the
cut points below are still proposals, and pushing this branch to `main` would
publish a snapshot, so that is the user's call to make.
Both open decisions were resolved the way the plan recommended: the `resolution`
machinery was deleted, and rotation was built.
## Verified before planning
Four things the fixes depend on, checked by running them rather than assumed —
two of them changed the prescription:
| Check | Result | Consequence |
| --- | --- | --- |
| `ssh-keygen` with `-N` omitted | Prompts `Enter passphrase … (empty for no passphrase)` **and** confirms | §2.2 fix works: omit `-N`, inherit stdio, keyman never holds the passphrase |
| `ssh-keygen -y -f <encrypted key>` | **Prompts for the passphrase** | §1.6 fix cannot be a silent spawn — needs `stdio: 'inherit'` and a skip path |
| `@inquirer/core` from keyman | `ERR_MODULE_NOT_FOUND` — transitive via `inquirer`, not a direct dep | Detect `ExitPromptError` by `error.name`, never by import |
| `z.strictObject` in zod 4.4.3 | Available; reports `unrecognized_keys` with a `keys` array | §3.5 has a hard-failure option, though the plan prefers a warning |
## What is not a breaking change
Per §4.3, `src/index.ts` exports only `keyman`, `loadConfig`,
`resolveConfigPaths` and the update module. `encryptKeys`, `decryptKeys`,
`generateKey`, `listKeys`, `copyKey` and `extractAgePublicKey` are **not** on the
public surface, so every signature change below is internal. Phases 2–6 are not
semver-breaking.
The one user-visible behaviour change is Phase 5 — see [Migration](#migration).
## Gate discipline
`lint:ci` → `typecheck` → `test:coverage` runs on `pre-push` and in CI. Two
standing constraints:
- **Every phase lands its tests with its fix.** No phase may leave a red gate,
so there is no "write the failing tests first" commit.
- **`keyman.cli.ts` is excluded from coverage** (`vitest.config.ts:18`). Per
`CLAUDE.md`, *adding logic to those files means moving it somewhere covered* —
which is why Phase 1 extracts argument parsing into a new module rather than
growing `cli.ts`.
Per-phase verification is `pnpm --filter @bitsquare/keyman run test`; the full
gate (`pnpm run lint:ci && pnpm run typecheck && pnpm run test:coverage`) before
each push.
---
## Phase 1 — Error boundary, `--help`, argument validation
Closes §3.1, §3.2, and the second half of §1.2 (the crash dump).
First because it is pure addition, touches no operation module, and converts
every latent throw in phases 2–6 from a stack dump into a line of text. The
`ExitPromptError` half is independently worth shipping: today **Ctrl-C at any
prompt** produces a crash dump.
**New file `src/keyman.args.ts`** (covered by the gate, unlike `cli.ts`):
- `parseArgs(argv: string[]): ParsedArgs` — supports `--flag value` *and*
`--flag=value`, rejects a flag consumed as another flag's value, rejects
unknown flags, and validates `--channel` against `'latest' | 'next' | 'main'`
so §3.2's false "Could not reach <registry>" cannot happen.
- `helpText(): string` — flags, both subcommands, and the four `KEYMAN_*`
variables. This is the text Phase 9 keeps in step with the README.
**`src/keyman.cli.ts`** stays wiring: dispatch on the parse result, and
```ts
try {
await keyman();
} catch (error) {
if ((error as { name?: string }).name === 'ExitPromptError') {
console.log('\n👋 Goodbye!\n');
process.exit(0);
}
console.error(`❌ ${error instanceof Error ? error.message : error}`);
process.exit(1);
}
```
`error.name`, not `instanceof` — `@inquirer/core` is not a direct dependency and
does not resolve from this package.
**Tests** — new `tests/args.test.ts`: each rejection, both flag forms, the
channel whitelist, and that `helpText()` names every flag `parseArgs` accepts
(so the two cannot drift).
**Done when** `keyman --help` prints usage and exits 0 without loading config or
prompting; `keyman --bogus` errors; Ctrl-C prints Goodbye and exits 0.
---
## Phase 2 — Guards and error handling in encrypt/decrypt
Closes §1.2 (first half), §1.5, and §1.10's `statSync`.
- `encrypt.ts:13,16` and `decrypt.ts:8` — `existsSync` guard, falling through to
the "⚠️ No …" message each function already has but cannot currently reach.
- `main.ts:40-41` — create `keysDir` alongside `vaultRoot` and `tmpDir`. Use
`{recursive: true, mode: 0o700}` now, so Phase 4 does not have to revisit it.
- Wrap the `age` spawns in both functions. An `ENOENT` on the binary gets its own
message ("`age` was not found on PATH") — it is the one hard external
requirement and currently the least legible failure.
- `list.ts:80` — `readdirSync(dir, {withFileTypes: true})` instead of
`statSync` per entry, which also drops N stat calls and fixes the broken-symlink
throw.
- Delete the debug logging while in these files: `encrypt.ts:18-19` and
`decrypt.ts:10` (§1.10). That also closes `DOCS-AUDIT.md` §6.4's open bullet.
**Tests** — the cases the current suite structurally cannot have, because every
`beforeEach` pre-creates the directories: encrypt with no `~/.ssh`, encrypt with
no tmp, decrypt with no `<vault>/keys`, each asserting the warning and no throw.
Plus `list` with a dangling symlink in the keys directory.
**Note on coverage.** `encrypt.ts` and `decrypt.ts` are at 100 % lines and
branches *today*. The number will not move; the tests are the point.
**Done when** a first run against an empty vault can reach every menu entry and
return to the menu.
---
## Phase 3 — Resolve the age recipient once, and derive it properly
Closes §1.3 and §2.3, and makes `CLAUDE.md`'s `age-keygen` claim true (§5.5).
Two changes that belong together because both are about the recipient:
1. **`utils.ts` — derive, don't scrape.** `extractAgePublicKey` currently regexes
`# public key:` out of a comment (`utils.ts:16`) and trusts it. Replace with
`age-keygen -y <keyPath>`, which derives the public key *from the private key*
and cannot disagree with it. Keep the comment parse as a fallback for when
`age-keygen` is absent, behind a warning that the recipient is unverified.
The function becomes `async`.
2. **`main.ts:73,80` — delete both `!`.** Resolve the recipient once before the
`switch`, and treat `null` as recoverable: print the remedy
(`age-keygen -o <keyPath>`) and `break` back to the menu. This is the whole of
§1.3 — the type already said null was possible.
Sequencing matters inside the phase: fix the call site first. Without it, a
missing key file still reaches `age -r null`, and the generate path still leaves
a **plaintext private key in `tmpDir`** after telling the user the operation
failed.
**Tests** — `utils.test.ts` gains the `age-keygen -y` path with `execa` mocked,
the fallback-with-warning path, and the both-unavailable path. `main.test.ts`
gains: missing recipient → neither `generateKey` nor `encryptKeys` is called, a
remedy is printed, and the menu loop continues.
**Done when** `keyman` against a vault with no `age.key` reaches the menu,
refuses generate and encrypt with a remedy, and still offers list and decrypt.
---
## Phase 4 — Decrypt: stop overwriting, stop the 0644 window
Closes §1.4 and §2.1. The highest-value phase — §1.4 is the only finding that
destroys data the user did not ask to touch.
- **Collision check before any decryption.** Both output paths, both modes.
Prompt per collision, defaulting to skip; `~/.ssh` deserves the friction more
than `vault/tmp` does, but the check is the same code.
- **Replace the shell-outs** (`decrypt.ts:49-50`) with `fs.copyFileSync` and
`fs.chmodSync`. Three spawns per key become one, it works on Windows, and it
removes a `cp` that overwrites unconditionally.
- **Close the permission window.** Verified: `age -o` creates the file `0644`
and `mkdirSync` creates `vault/tmp` as `0755`, so a plaintext key is
world-readable for the duration of two process spawns — and stays `0644` if the
`chmod` fails. `fs.chmodSync` immediately after `age` resolves; `mode: 0o700`
on the directory (already done in Phase 2); create `~/.ssh` `0700` if absent.
**Test rework — the fiddliest in the plan.** `decrypt.test.ts` asserts on the
mocked spawns: `argsOf('cp')` (`:98,111`) and `argsOf('chmod')` (`:99,112`) both
disappear, and `execa` is mocked with a bare `mockResolvedValue` (`:57`) that
writes no output file. Once `copyFileSync` is real it needs a real file, so the
mock must write to its `-o` argument the way `encrypt.test.ts:51-54` already
does. Assert the on-disk result and mode instead of the argv — a better test
than the one it replaces, since it checks the outcome rather than the mechanism.
`execa` call counts also change (`:123`: six spawns → two).
**Done when** decrypting onto an existing key requires a confirmation, and the
decrypted key is never observable at anything but `0600`.
---
## Phase 5 — Thread `keysDir` and `tmpDir` through encrypt and decrypt
Closes §1.1 and the `DOCS-AUDIT.md` entry in §5.1.
Mechanical, but it is the one phase that rewrites assertions that pass today, so
it stays its own commit with nothing else in it.
- `encryptKeys(sshDir, keysDir, tmpDir, pubkey)` — drop `vaultDir`, delete the
hardcoded `path.join(vaultDir, 'keys')` (`encrypt.ts:38`).
- `decryptKeys(sshDir, keysDir, tmpDir, ageKey)` — drop `vaultDir`, delete the
hardcoded joins at `decrypt.ts:7,39,43`.
- `main.ts:76,84` — pass `paths.keysDir` and `paths.tmpDir`. Neither function has
any remaining use for `vaultRoot`, so the parameter goes rather than becoming a
second source of truth.
**Assertions to change** — all four, named so the diff is reviewable:
| Location | Today | After |
| --- | --- | --- |
| `main.test.ts:164-175` | "encrypts keys into the vault root", asserts `paths.vaultRoot` | asserts `paths.keysDir`, `paths.tmpDir` |
| `main.test.ts:177-187` | asserts `paths.vaultRoot` | asserts `paths.keysDir`, `paths.tmpDir` |
| `encrypt.test.ts:95,115,128-129` | `path.join(vaultDir, 'keys', …)` | `path.join(keysDir, …)` |
| `decrypt.test.ts:53,89` | `keyDir = path.join(vaultDir, 'keys')` | `keysDir` passed in directly |
**New test, the one that would have caught this:** a config with
`keysDir: 'encrypted'` and `tmpDir: 'plain'`, encrypt a key, then list it, and
assert the listing shows it in `[Vault]`. That round trip fails today and is the
regression worth owning.
**Also in this commit:** move the `DOCS-AUDIT.md:826-827` bullet out of *checked
and accurate* and point it at this finding. It was verified against
`keyman.encrypt.ts` — the file that ignores the config — which is precisely how
§1.1 stayed invisible.
---
## Phase 6 — Generate: passphrase off argv, and `.pub` recovery
Closes §2.2, §1.6, and §1.10's leftover-directory bullet.
**Passphrase (§2.2).** Verified: omitting `-N` makes `ssh-keygen` prompt *and*
confirm. So delete keyman's own password prompt (`generate.ts:26-33`), omit `-N`,
and spawn with `stdio: 'inherit'`. The passphrase never enters keyman's memory
and never reaches argv — strictly better than routing it more carefully, and it
deletes code. `generate.test.ts:78-87` loses `-N`/`'pw'` from the expected argv
and the password-prompt case goes away.
**Missing `.pub` (§1.6).** The selection list is built from private keys only, so
an orphan private key is offered and `copyFileSync` throws *after* `age` has
written the `.age` file — leaving a vault entry with no public key and killing
the rest of the batch. Fix in two parts:
- Derive it with `ssh-keygen -y -f <key>` when the sibling is absent. **Verified
that this prompts for a passphrase on an encrypted key**, so it needs
`stdio: 'inherit'` and a clean skip when the user cannot or will not supply it
— not a silent spawn whose stdout is captured.
- Wrap the loop body in `encrypt.ts:36-48` per key, so one bad key costs one key.
Report the failures at the end rather than dying at the first.
**Leftover directory.** `generate.ts:65` creates `<keysDir>/<name>/` before
`age` runs at `:68`. Move the `mkdirSync` after `age` succeeds.
---
## Phase 7 — Config: warn on typos, decide on the dead machinery
Closes §3.5, §3.4, and asks for a decision on §3.3.
**Typos (§3.5).** Verified: `{"vaultroot": "…"}` is silently stripped by
`z.object`. Warn per file rather than failing — diff `Object.keys(rawConfig)`
against the schema keys plus `resolution` inside the existing per-file loop
(`config.ts:210-227`), where the filename is in hand. That names the offending
file, which `z.strictObject` cannot do from the merged result, and it preserves
the module's documented posture of degrading to defaults rather than throwing.
(`z.strictObject` is available in zod 4.4.3 and reports `unrecognized_keys` with
a `keys` array, if a hard failure is preferred later.)
**`getConfigPaths` (§3.4).** Add it to the `--print-config` JSON as
`configFiles`. The function is currently exercised only by its own test, and
`--print-config` currently cannot answer *which files were read* — that exists
only as unstructured stderr from `loadConfig`. One change fixes both.
**Decision needed — the `resolution` machinery (§3.3).** Roughly 45 lines
(`config.ts:23-30,115-157`) that cannot affect a valid config, because every
schema property is a `string` and both strategies return `childValue` for
primitives. `config.test.ts:212` "honours an explicit override strategy" passes
either way.
- **Recommended: delete it**, along with the doc comment at `:186-194` that
advertises it. `keyman.config.ts` goes from 267 lines to roughly 220, and the
config file stops documenting a knob that does nothing.
- **Alternative: keep it** as the shape a future array- or object-valued option
would need — but then say so in a comment, because today it reads as
functional, and make `config.test.ts:212` assert something that distinguishes
the two strategies (which requires a non-string property to exist first).
Deleting is the smaller lie. It also diverges from nopy, where the machinery
*is* load-bearing — worth a line in `CLAUDE.md` so the divergence reads as
deliberate.
---
## Phase 8 — Portability and the gaps that make keys invisible
Closes §1.7, §1.8, §1.9, §2.4. Independent of each other; split if any grows.
- **Clipboard (§1.9).** `pbcopy` / `wl-copy` / `xclip` / `clip.exe` by platform,
falling back to printing the key to stdout so the operation is never a dead
end. Delete the comment at `copy.ts:46-48` that admits the shortcut.
- **Home directory (§1.7).** `os.userInfo()` for the current user; for a named
user, look the home directory up rather than assuming `/home/<user>` — wrong on
the one platform the tool currently supports. Check `existsSync` and say so,
instead of feeding a nonexistent path into a `readdir`.
`main.test.ts:198-204` changes.
- **Non-`id_*` keys (§1.8).** Relax the filters (`copy.ts:9`, `encrypt.ts:14,17`,
`list.ts:23,51`) to *any* private key with a recognisable header, or at minimum
print a count of the keys that were skipped and why. Today a key named
`deploy_ed25519` is simply absent from the menu — and pre-existing keys are the
population a key manager is adopted to take over. `decrypt.ts:9` reconstructs
`id_${dir}` from the folder name, so the vault layout has the assumption baked
in; relaxing discovery means storing the real filename per key, which is the
largest single item in this plan. **Size it before committing to it** — a
skipped-key count is a tenth of the work and closes most of the surprise.
- **Plaintext hygiene (§2.4).** A "🧹 Clear decrypted keys" menu entry, and write
a `.gitignore` next to the vault on first run covering `age.key` and `tmp/` —
which `README.md:25-26` currently tells the user to do by hand. This is the
cheapest guard against the exact failure the tool exists to prevent.
---
## Phase 9 — Documentation
Closes §5.2, §5.3, §5.4, §5.5. Last, so it documents what the code now does.
- **`README.md` — the only shipped document** (`package.json:37-41` ships `dist`,
`README.md`, `LICENSE`). Currently describes four of six menu entries, invents
key rotation, and mentions none of `self-update`, `--print-config`,
`--version`, `--help`, or the four `KEYMAN_*` variables. `README.PUBLISH.md`
has all of it and never reaches a reader on the registry. Reuse Phase 1's
`helpText()` as the source for the CLI section so the two cannot drift.
- **`README.md:46-86`** — the configuration and vault-layout sections, now that
Phase 5 makes `keysDir`/`tmpDir` real, plus the migration note below.
- **`README.md:11`** — drop "Support for key rotation" unless Phase 10 lands
first.
- **`README.md:33`** — stop telling the user to shell out to `ssh-keygen`; the
Generate operation exists.
- **`CLAUDE.md`** — `age-keygen` becomes true in Phase 3; note that `cp`/`chmod`
are gone (Phase 4) and record the `resolution` divergence from nopy (Phase 7).
- **`AUDIT.md`** — mark findings closed, keeping their text as the record, the way
`DOCS-AUDIT.md` does.
---
## Phase 10 — Key rotation (decision required)
§3.6. `README.md:11` has advertised it since before this audit;
`grep -rn "rotat" packages/keyman/src/` returns nothing.
Unlike the rest of this plan it is a feature, not a repair, and it is the one
item that could reasonably be dropped instead. **Recommendation: build it** —
rotation is the operation that makes a key vault worth having, and the pieces all
exist by Phase 6 (generate under a new name, encrypt, keep the old key until the
replacement is deployed, then shred). Sketch:
1. Pick an existing vault key.
2. Generate a replacement into `tmpDir` under a versioned name.
3. Encrypt it alongside the current one — never replacing it.
4. Report both public keys, so the new one can be deployed before the old one
goes.
5. A separate "retire" step that removes the superseded key once the user
confirms.
Steps 3 and 4 are the whole value: a rotation that atomically replaces the key is
a rotation that locks you out of the host you were rotating for. If this is
deferred, delete the README claim in Phase 9 instead.
---
## Migration
Phase 5 is the only user-visible change. Anyone with a custom `keysDir` or
`tmpDir` currently has a **split vault** — `generate` and `list` on the
configured directory, `encrypt` and `decrypt` on `<vaultRoot>/keys` and
`<vaultRoot>/tmp`. After Phase 5 all five agree on the configured directory, so
anything written by `encrypt` before the upgrade needs moving:
```sh
mv <vaultRoot>/keys/* <vaultRoot>/<keysDir>/
```
Nobody on the defaults is affected, since the two halves coincide there. The
README gets this as a note, and it is worth a line in the release notes for
whichever version carries Phase 5.
## Rollout
Per `CLAUDE.md`: bump `packages/keyman/package.json`, land on `main`, then tag
`keyman-v<version>`.
- **Snapshots come free.** Every push to `main` publishes
`<version>-main.<run>.g<sha>` to Gitea under the `main` dist-tag, so each phase
is installable for testing without a release. `pnpm run try:snapshot` installs
one into a throwaway project.
- **Suggested cut points.** After Phase 4 as `0.6.0` — error boundary, guards,
recipient handling and the data-loss fix, which is the set worth getting to
users first. After Phase 9 as `0.7.0`, carrying the Phase 5 migration note.
- **keyman can reach npmjs.** It has no `workspace:*` dependencies (`execa`,
`inquirer`, `semver`, `zod` only), so `scripts/linked-deps.mjs` has nothing to
block on — unlike `nopy`, which `CLAUDE.md` records as gated behind
`nopy-cubes` shipping. keyman has never been published to npmjs; `0.6.0` could
be the first, and versions being `0.x.y` rather than `1.0.0-alphaN` means the
`latest` dist-tag will now actually move.
- **`pnpm publish`, never `npm publish`** — no `workspace:` ranges here, but the
rule is repo-wide and `scripts/verify-pack.mjs` enforces it in both workflows.
## Sequencing at a glance
```
1 cli boundary + --help + args §3.1 §3.2 §1.2(half) isolated, pure addition
2 guards in encrypt/decrypt/list §1.2 §1.5 §1.10 new tests only
3 age recipient, once and derived §1.3 §2.3 §5.5 signature → async
4 decrypt: no clobber, no 0644 §1.4 §2.1 reworks decrypt.test.ts
── cut 0.6.0 ──
5 thread keysDir/tmpDir §1.1 §5.1 rewrites 4 assertions
6 generate: -N gone, .pub recovery §2.2 §1.6 §1.10 reworks generate.test.ts
7 config: warn, prune, print §3.5 §3.4 §3.3* *decision
8 portability + hygiene §1.7 §1.8 §1.9 §2.4 §1.8 needs sizing
9 documentation §5.2 §5.3 §5.4 §5.5 README is the shipped one
── cut 0.7.0 ──
10 rotation §3.6* *decision: build or delete
```
Phases 1–6 are repairs and want to land in order. 7 and 8 are independent of each
other and of 5–6. 9 depends on everything before it. 10 is optional and gates
one line of Phase 9.
## Open decisions
Neither blocks Phase 1. Both change scope where they land:
1. **§3.3, at Phase 7** — delete the inert `resolution` machinery (recommended,
−45 lines) or keep it as future shape with a comment saying so.
2. **§3.6, at Phase 10** — build rotation (recommended) or delete the README
claim in Phase 9.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@bitsquare/keyman",
"version": "0.5.0",
"version": "0.7.3",
"description": "A system to simplify ssh key management",
"keywords": [
"ssh",
+14 -30
View File
@@ -1,31 +1,15 @@
#!/usr/bin/env node
export { loadConfig, resolveConfigPaths } from './keyman.config.js';
/**
* The library surface — `exports["."]`, imported and never executed, which is why
* it no longer carries the bin's shebang (AUDIT §4.1). The bin is
* `dist/keyman.cli.js`.
*
* Deliberately narrow: config resolution, the update machinery, and `keyman()` to
* run the menu. The operation modules stay internal — every one of them prompts,
* prints and spawns, so there is nothing to do with a single one except reproduce
* the menu around it (§4.3). The update module is re-exported wholesale rather
* than by name list, because the list had drifted to half of it (§4.4).
*/
export type { KeymanConfig, KeymanConfigFile } from './keyman.config.js';
export { describeConfig, loadConfig, resolveConfigPaths } from './keyman.config.js';
export * from './keyman.main.js';
export type {
Channel,
CommandRunner,
PackageManager,
SelfUpdateResult,
UpdateCache,
UpdateStatus,
} from './keyman.update.js';
export {
buildSelfUpdateCommand,
channelForVersion,
checkForUpdate,
DEFAULT_CHECK_INTERVAL_MS,
detectPackageManager,
fetchChannelVersion,
formatCommand,
formatUpdateNotice,
getUpdateCachePath,
isUpdateCheckDisabled,
NPMJS_REGISTRY,
normalizeRegistry,
PACKAGE_NAME,
readUpdateCache,
resolveRegistry,
selfUpdate,
updateNotice,
writeUpdateCache,
} from './keyman.update.js';
export * from './keyman.update.js';
+204
View File
@@ -0,0 +1,204 @@
/**
* Argv parsing for the keyman CLI.
*
* Separate from `keyman.cli.ts` because that file is excluded from coverage: it
* is meant to be wiring, and *which flag takes a value* and *which channel names
* are legal* are behaviour. The old inline `indexOf` reader accepted
* `--channel --force`, which reached the registry as a dist-tag that cannot
* exist and reported an unreachable registry instead of a bad flag.
*/
import type { Channel } from './keyman.update.js';
/** The channels `--channel` accepts, in the order the error message lists them */
export const CHANNELS: readonly Channel[] = ['latest', 'next', 'main'];
/** Flags that consume the next token, or the suffix of a `--flag=value` */
const VALUE_FLAGS: readonly string[] = ['--channel', '--registry'];
/** Flags that stand alone, short aliases included */
const BOOLEAN_FLAGS: readonly string[] = [
'--help',
'-h',
'--version',
'-V',
'--print-config',
'--self-update',
'--dry-run',
'-n',
'--force',
'-f',
];
/**
* Flags that only mean anything to `self-update`. Named so that using one on its
* own is an error rather than a silent no-op.
*/
const SELF_UPDATE_ONLY: readonly string[] = [
'--channel',
'--registry',
'--dry-run',
'-n',
'--force',
'-f',
];
/** Every flag the parser accepts — the list `helpText()` is checked against */
export const KNOWN_FLAGS: readonly string[] = [...BOOLEAN_FLAGS, ...VALUE_FLAGS];
const SUBCOMMANDS: readonly string[] = ['self-update', 'upgrade'];
export type ParsedArgs =
| { command: 'help' }
| { command: 'version' }
| { command: 'print-config' }
| { command: 'interactive' }
| {
command: 'self-update';
dryRun: boolean;
force: boolean;
channel?: Channel;
registry?: string;
};
/**
* A mistake in the invocation. Carries a message meant for the user, so the CLI
* can print one line instead of a stack trace.
*/
export class UsageError extends Error {
constructor(message: string) {
super(message);
this.name = 'UsageError';
}
}
/**
* Turns argv (already sliced past `node` and the script) into one command.
*
* @throws {UsageError} on an unknown flag or command, a value flag with no
* value, a boolean flag given one, or a channel that is not a real channel
*/
export function parseArgs(argv: string[]): ParsedArgs {
// Before tokenising, so that help answers a line it could not otherwise parse.
// Exact tokens only: `--registry=--help` is a (bad) registry, not a request.
if (argv.some((token) => token === '--help' || token === '-h')) {
return { command: 'help' };
}
const flags = new Set<string>();
const values = new Map<string, string>();
let subcommand: string | undefined;
for (let index = 0; index < argv.length; index++) {
const token = argv[index];
if (!token.startsWith('-')) {
if (!SUBCOMMANDS.includes(token)) {
throw new UsageError(`Unknown command: ${token}`);
}
if (subcommand) {
throw new UsageError(`Unexpected argument: ${token}`);
}
subcommand = token;
continue;
}
const equals = token.indexOf('=');
const name = equals === -1 ? token : token.slice(0, equals);
if (VALUE_FLAGS.includes(name)) {
// A value that looks like a flag is a forgotten value, not a value —
// unless it was written as --flag=-value and therefore meant.
const inline = equals === -1 ? undefined : token.slice(equals + 1);
const value = inline ?? argv[++index];
if (!value || (inline === undefined && value.startsWith('-'))) {
throw new UsageError(`${name} expects a value`);
}
values.set(name, value);
continue;
}
if (!BOOLEAN_FLAGS.includes(name)) {
throw new UsageError(`Unknown flag: ${name}`);
}
if (equals !== -1) {
throw new UsageError(`${name} does not take a value`);
}
flags.add(name);
}
const given = (...names: string[]) => names.some((name) => flags.has(name));
const isSelfUpdate = subcommand !== undefined || flags.has('--self-update');
if (!isSelfUpdate) {
const stray = [...values.keys(), ...flags].find((name) => SELF_UPDATE_ONLY.includes(name));
if (stray) {
throw new UsageError(`${stray} is only valid with \`keyman self-update\``);
}
}
if (flags.has('--print-config')) {
return { command: 'print-config' };
}
if (given('--version', '-V')) {
return { command: 'version' };
}
if (isSelfUpdate) {
const channel = values.get('--channel');
if (channel !== undefined && !CHANNELS.includes(channel as Channel)) {
throw new UsageError(`Unknown channel: ${channel} (expected ${CHANNELS.join(', ')})`);
}
return {
command: 'self-update',
dryRun: given('--dry-run', '-n'),
force: given('--force', '-f'),
channel: channel as Channel | undefined,
registry: values.get('--registry'),
};
}
return { command: 'interactive' };
}
/**
* What `--help` prints.
*
* Hand-written rather than generated from the flag tables, so that adding a flag
* to the parser without documenting it fails a test instead of shipping.
*/
export function helpText(): string {
return `keyman — SSH key management and an age-encrypted key vault
Usage
keyman start the interactive menu
keyman self-update update keyman itself (alias: upgrade)
Flags
-h, --help print this help and exit
-V, --version print the version and exit
--print-config print the resolved paths and the config files
they came from, as JSON, and exit
--self-update same as the self-update subcommand
Flags for self-update
--channel <${CHANNELS.join('|')}> channel to update from
(default: derived from the running version)
--registry <url> registry to query instead of the configured one
-n, --dry-run print the install command without running it
-f, --force reinstall even when already up to date
Environment
VAULT_ROOT overrides vaultRoot from .keymanrc.json
KEYMAN_REGISTRY registry for the update check and self-update
KEYMAN_REGISTRY_TOKEN bearer token for a private registry
KEYMAN_NO_UPDATE_CHECK set to 1 to skip the once-a-day update check
(also skipped whenever CI is set)
KEYMAN_PACKAGE_MANAGER npm | pnpm | yarn | bun for the install command
Configuration is read from .keymanrc.json, merged from the current directory
upwards and then from ~/.keymanrc.json.
`;
}
+86
View File
@@ -0,0 +1,86 @@
import fs from 'node:fs';
import path from 'node:path';
import inquirer from 'inquirer';
import { scanPrivateKeys } from './keyman.keys.js';
/**
* Writes a `.gitignore` beside the vault, once.
*
* The README told the user to do this by hand. A vault holds the age identity and,
* whenever anything has been decrypted, plaintext private keys — committing it is
* the exact failure the tool exists to prevent, and it is one file to prevent it.
*
* Never overwritten: an existing file may say more than this one does.
*/
export function writeVaultGitignore(vaultRoot: string, tmpDir: string, keyPath: string) {
const gitignore = path.join(vaultRoot, '.gitignore');
if (fs.existsSync(gitignore)) {
return;
}
// Both are configurable and may be absolute, so either can sit outside the vault.
// A .gitignore cannot speak about a path above itself, and claiming to would be
// worse than saying nothing.
const inside = (target: string) => {
const relative = path.relative(vaultRoot, target);
return relative && !relative.startsWith('..') && !path.isAbsolute(relative) ? relative : null;
};
const tmp = inside(tmpDir);
const key = inside(keyPath);
const lines = [
'# Written by keyman. The encrypted keys under the keys directory are safe to',
'# commit; nothing else here is.',
...(key ? [key, `${key}.pub`] : []),
...(tmp ? [`${tmp}/`] : []),
'',
];
fs.writeFileSync(gitignore, lines.join('\n'), { mode: 0o600 });
}
/**
* Deletes the decrypted keys in the vault's tmp directory.
*
* The counterpart to `decrypt`, which had none: a plaintext private key stayed
* there until someone remembered it, and "someone remembered" is not a security
* control. Only the key pairs are removed — anything else in the directory is not
* keyman's to delete.
*/
export async function clearDecryptedKeys(tmpDir: string) {
const { keys } = scanPrivateKeys(tmpDir);
if (keys.length === 0) {
console.log(`✅ Nothing decrypted in ${tmpDir}.`);
return;
}
console.log(`\n🔓 Decrypted keys in ${tmpDir}:`);
for (const key of keys) {
console.log(` ${key}`);
}
const { confirmed } = await inquirer.prompt<{ confirmed: boolean }>([
{
type: 'confirm',
name: 'confirmed',
message: `Delete ${keys.length === 1 ? 'this key' : `these ${keys.length} keys`}?`,
// A key that exists only here — generated and not yet deployed — is gone for
// good, so this is not a question to answer by pressing return.
default: false,
},
]);
if (!confirmed) {
console.log('⏭️ Nothing was deleted.');
return;
}
for (const key of keys) {
for (const file of [key, `${key}.pub`]) {
fs.rmSync(path.join(tmpDir, file), { force: true });
}
console.log(`🧹 Removed ${key}`);
}
}
+35 -19
View File
@@ -1,9 +1,9 @@
#!/usr/bin/env node
import { createRequire } from 'node:module';
import { loadConfig, resolveConfigPaths } from './keyman.config.js';
import { helpText, type ParsedArgs, parseArgs, UsageError } from './keyman.args.js';
import { describeConfig } from './keyman.config.js';
import { keyman } from './keyman.main.js';
import type { Channel } from './keyman.update.js';
import { formatCommand, selfUpdate, updateNotice } from './keyman.update.js';
const { version, buildInfo } = createRequire(import.meta.url)('../package.json') as {
@@ -18,35 +18,40 @@ const { version, buildInfo } = createRequire(import.meta.url)('../package.json')
*/
const versionLabel = buildInfo?.commit ? `${version} (${buildInfo.commit})` : version;
const args = process.argv.slice(2);
/** Reads `--flag value` out of argv, or undefined when the flag is absent */
function flagValue(name: string): string | undefined {
const index = args.indexOf(name);
return index === -1 ? undefined : args[index + 1];
let parsed: ParsedArgs;
try {
parsed = parseArgs(process.argv.slice(2));
} catch (error) {
if (!(error instanceof UsageError)) throw error;
console.error(`❌ ${error.message}`);
console.error('Run `keyman --help` for usage.');
process.exit(2);
}
if (args.includes('--print-config')) {
const config = loadConfig();
const paths = resolveConfigPaths(config);
console.log(JSON.stringify(paths));
if (parsed.command === 'help') {
console.log(helpText());
process.exit(0);
}
if (args.includes('--version') || args.includes('-V')) {
if (parsed.command === 'print-config') {
console.log(JSON.stringify(describeConfig()));
process.exit(0);
}
if (parsed.command === 'version') {
console.log(versionLabel);
process.exit(0);
}
if (args[0] === 'self-update' || args[0] === 'upgrade' || args.includes('--self-update')) {
const dryRun = args.includes('--dry-run') || args.includes('-n');
if (parsed.command === 'self-update') {
const { dryRun } = parsed;
try {
const result = await selfUpdate({
currentVersion: version,
channel: flagValue('--channel') as Channel | undefined,
registry: flagValue('--registry'),
channel: parsed.channel,
registry: parsed.registry,
dryRun,
force: args.includes('--force') || args.includes('-f'),
force: parsed.force,
});
const { status } = result;
@@ -79,4 +84,15 @@ if (notice) {
console.error(`\n${notice}\n`);
}
keyman();
try {
await keyman();
} catch (error) {
// Ctrl-C at any inquirer prompt lands here. `name`, not `instanceof`:
// @inquirer/core is transitive and does not resolve from this package.
if ((error as { name?: string }).name === 'ExitPromptError') {
console.log('\n👋 Goodbye!\n');
process.exit(0);
}
console.error(`❌ ${error instanceof Error ? error.message : error}`);
process.exit(1);
}
+56
View File
@@ -0,0 +1,56 @@
import { runTool, ToolNotFoundError } from './keyman.utils.js';
/** A clipboard command and the argv it wants, in the order they are tried. */
interface ClipboardTool {
binary: string;
args: string[];
}
/**
* The clipboard commands worth trying on a platform, best first.
*
* Linux is a list rather than a choice because there is no single answer:
* `wl-copy` under Wayland, `xclip`/`xsel` under X11, and a user may have any
* subset installed. Trying them in order and moving on from an absent one costs a
* failed spawn and removes the need to detect the session type.
*/
export function clipboardTools(platform: string = process.platform): ClipboardTool[] {
switch (platform) {
case 'darwin':
return [{ binary: 'pbcopy', args: [] }];
case 'win32':
return [{ binary: 'clip', args: [] }];
default:
return [
{ binary: 'wl-copy', args: [] },
{ binary: 'xclip', args: ['-selection', 'clipboard'] },
{ binary: 'xsel', args: ['--clipboard', '--input'] },
];
}
}
/**
* Puts `text` on the system clipboard.
*
* keyman used to spawn `pbcopy` unconditionally, with a comment saying so — which
* made "copy public key" a dead end on every platform but macOS, and reported it
* as a clipboard failure rather than as a missing tool.
*
* @returns the command that took it, or null if none was available
*/
export async function copyToClipboard(text: string, platform?: string): Promise<string | null> {
for (const { binary, args } of clipboardTools(platform)) {
try {
await runTool(binary, args, { input: text });
return binary;
} catch (error) {
// Only an absent tool is worth trying the next candidate for. One that ran
// and refused has an opinion, and repeating the paste elsewhere is not it.
if (!(error instanceof ToolNotFoundError)) {
throw error;
}
}
}
return null;
}
+41 -87
View File
@@ -15,27 +15,11 @@ const KeymanConfigSchema = z.object({
export type KeymanConfig = z.infer<typeof KeymanConfigSchema>;
/**
* Resolution strategy for merging config properties
* - 'merge': Arrays are concatenated, objects are deep merged (default)
* - 'override': Child value completely replaces parent value
*/
export type ResolutionStrategy = 'merge' | 'override';
/** Raw config file structure */
export type KeymanConfigFile = Partial<KeymanConfig>;
/**
* Resolution configuration for customizing merge behavior
*/
export type KeymanResolutionConfig = {
[K in keyof KeymanConfig]?: ResolutionStrategy;
};
/**
* Raw config file structure (includes resolution)
*/
export interface KeymanConfigFile extends Partial<KeymanConfig> {
/** Customize merge behavior for specific properties */
resolution?: KeymanResolutionConfig;
}
/** Every key a config file may set. */
const KNOWN_KEYS = Object.keys(KeymanConfigSchema.shape) as (keyof KeymanConfig)[];
/**
* Default configuration values
@@ -110,71 +94,36 @@ function findConfigFiles(startDir: string): string[] {
}
/**
* Deep merges two values based on resolution strategy
* Reports keys a config file sets that keyman does not read.
*
* `z.object` strips them silently, so `{"vaultroot": "…"}` used to be
* indistinguishable from an empty file — the vault quietly stayed at the default
* and nothing said why. Warned rather than fatal, which is this module's posture
* throughout, and warned *here* because this is the only place the filename is in
* hand: `z.strictObject` on the merged result cannot name the file that said it.
*/
function mergeValue(
parentValue: unknown,
childValue: unknown,
strategy: ResolutionStrategy
): unknown {
// Override strategy: child replaces parent completely
if (strategy === 'override') {
return childValue;
}
function warnUnknownKeys(configFile: KeymanConfigFile, configPath: string): void {
const unknown = Object.keys(configFile).filter(
(key) => !KNOWN_KEYS.includes(key as keyof KeymanConfig)
);
// Merge strategy (default)
if (Array.isArray(parentValue) && Array.isArray(childValue)) {
// Concatenate arrays, remove duplicates for primitives
const combined = [...parentValue, ...childValue];
if (combined.every((v) => typeof v !== 'object')) {
return [...new Set(combined)];
if (unknown.length > 0) {
console.warn(
`⚠️ ${configPath}: ignoring unknown ${unknown.length === 1 ? 'key' : 'keys'} ${unknown.join(', ')}. Known keys: ${KNOWN_KEYS.join(', ')}.`
);
}
return combined;
}
if (
typeof parentValue === 'object' &&
parentValue !== null &&
typeof childValue === 'object' &&
childValue !== null &&
!Array.isArray(parentValue) &&
!Array.isArray(childValue)
) {
// Deep merge objects
const result: Record<string, unknown> = { ...parentValue };
for (const [key, value] of Object.entries(childValue)) {
if (key in result) {
result[key] = mergeValue(result[key], value, 'merge');
} else {
result[key] = value;
}
}
return result;
}
// Primitives: child overrides parent
return childValue;
}
/**
* Merges a child config into a parent config
* Merges a child config into a parent config.
*
* Every property is a string, so a child simply wins. keyman deliberately has
* none of nopy's `resolution` machinery: deep-merge and array-concatenation
* strategies are meaningful there because its config holds arrays and objects,
* and here they would be 45 lines that cannot change an outcome.
*/
function mergeConfigs(parent: KeymanConfig, childFile: KeymanConfigFile): KeymanConfig {
const resolution = childFile.resolution || {};
const result: Record<string, unknown> = { ...parent };
for (const [key, value] of Object.entries(childFile)) {
if (key === 'resolution') continue; // Skip resolution property itself
const strategy = resolution[key as keyof KeymanConfig] || 'merge';
if (key in result) {
result[key] = mergeValue(result[key], value, strategy);
} else {
result[key] = value;
}
}
return result as unknown as KeymanConfig;
return { ...parent, ...childFile };
}
/**
@@ -183,16 +132,6 @@ function mergeConfigs(parent: KeymanConfig, childFile: KeymanConfigFile): Keyman
* Searches for `.keymanrc.json` by traversing upwards from cwd to root.
* Multiple config files are merged, with child configs overriding parent configs.
*
* Use the `resolution` property to customize merge behavior:
* ```json
* {
* "vaultRoot": "../vault",
* "resolution": {
* "vaultRoot": "override"
* }
* }
* ```
*
* @returns Validated keyman configuration
*/
export function loadConfig(): KeymanConfig {
@@ -211,6 +150,7 @@ export function loadConfig(): KeymanConfig {
try {
const content = fs.readFileSync(configPath, 'utf-8');
const rawConfig = JSON.parse(content) as KeymanConfigFile;
warnUnknownKeys(rawConfig, configPath);
// Resolve path properties relative to the config file's directory
const configDir = path.dirname(configPath);
const resolvedConfig = resolvePathsRelativeToConfig(rawConfig, configDir);
@@ -265,3 +205,17 @@ export function resolveConfigPaths(config: KeymanConfig) {
export function getConfigPaths(): string[] {
return findConfigFiles(process.cwd());
}
/**
* What `--print-config` prints.
*
* `configFiles` is the question the flag could not answer before: which files
* were read, in the order they were merged. It existed only as unstructured
* stderr from `loadConfig`, which is exactly the wrong place for it — the JSON is
* the machine-readable half.
*/
export function describeConfig(): ReturnType<typeof resolveConfigPaths> & {
configFiles: string[];
} {
return { ...resolveConfigPaths(loadConfig()), configFiles: getConfigPaths() };
}
+24 -19
View File
@@ -1,18 +1,19 @@
import fs from 'node:fs';
import path from 'node:path';
import { execa } from 'execa';
import inquirer from 'inquirer';
import { copyToClipboard } from './keyman.clipboard.js';
import { reportSkippedKeys, scanPrivateKeys } from './keyman.keys.js';
export async function copyKey(sshDir: string, tmpDir: string) {
const getKeys = (dir: string) => {
if (!fs.existsSync(dir)) return [];
return fs.readdirSync(dir).filter((key) => key.startsWith('id_') && !key.endsWith('.pub'));
};
const ssh = scanPrivateKeys(sshDir);
const tmp = scanPrivateKeys(tmpDir);
const sshKeys = getKeys(sshDir);
const tmpKeys = getKeys(tmpDir);
const keys = [...new Set([...ssh.keys, ...tmp.keys])];
const keys = [...new Set([...sshKeys, ...tmpKeys])];
// Before the empty check: "no SSH keys found" next to four unmanageable ones is
// the case the report exists for.
reportSkippedKeys(ssh.skipped, sshDir);
reportSkippedKeys(tmp.skipped, tmpDir);
if (keys.length === 0) {
console.log('⚠️ No SSH keys found.');
@@ -21,7 +22,7 @@ export async function copyKey(sshDir: string, tmpDir: string) {
const { selectedKey } = await inquirer.prompt<{ selectedKey: string }>([
{
type: 'list',
type: 'select',
name: 'selectedKey',
message: 'Select key to copy public key from:',
choices: keys,
@@ -40,19 +41,23 @@ export async function copyKey(sshDir: string, tmpDir: string) {
return;
}
try {
const pubKeyContent = fs.readFileSync(pubKeyPath, 'utf-8').trim();
// Detect OS and use appropriate clipboard command
// Since the environment is Darwin, we prioritize pbcopy, but we can add others for completeness or use a simple check.
// For this specific request on Darwin:
const proc = execa('pbcopy');
proc.stdin?.write(pubKeyContent);
proc.stdin?.end();
await proc;
try {
const tool = await copyToClipboard(pubKeyContent);
console.log(`✅ Public key for ${selectedKey} copied to clipboard!`);
if (tool) {
console.log(`✅ Public key for ${selectedKey} copied to clipboard via ${tool}!`);
return;
}
console.warn('⚠️ No clipboard command found.');
} catch (error) {
console.error(`❌ Failed to copy to clipboard: ${error}`);
console.error(
`❌ Failed to copy to clipboard: ${error instanceof Error ? error.message : error}`
);
}
// Printing it is the point of the operation; the clipboard was only the
// convenient way to deliver it. A public key is not a secret.
console.log(`\n${pubKeyContent}\n`);
}
+81 -26
View File
@@ -1,15 +1,22 @@
import fs from 'node:fs';
import path from 'node:path';
import { execa } from 'execa';
import inquirer from 'inquirer';
import { runTool } from './keyman.utils.js';
import { listVaultKeys } from './keyman.vault.js';
export async function decryptKeys(sshDir: string, vaultDir: string, ageKey: string) {
const keyDir = path.join(vaultDir, 'keys');
const vaultKeys = fs.readdirSync(keyDir).filter((key) => {
const keyfile = path.join(keyDir, key, `id_${key}.age`);
console.log(keyfile);
return fs.existsSync(keyfile);
});
/** The two decryption targets. Values, so the label can name the real directory. */
const LOCAL_MODE = 'local';
interface DecryptPlan {
key: string;
encryptedKey: string;
publicKey: string;
privateKeyOut: string;
publicKeyOut: string;
}
export async function decryptKeys(sshDir: string, keysDir: string, tmpDir: string, ageKey: string) {
const vaultKeys = listVaultKeys(keysDir);
if (vaultKeys.length === 0) {
console.log('⚠️ No encrypted keys found.');
@@ -24,30 +31,78 @@ export async function decryptKeys(sshDir: string, vaultDir: string, ageKey: stri
choices: vaultKeys,
},
{
type: 'list',
type: 'select',
name: 'decryptMode',
message: 'Choose decryption location:',
choices: ['Local (vault/tmp)', 'SSH (~/.ssh)'],
// Named after the directories actually in use, which are configurable.
choices: [
{ name: `Local (${tmpDir})`, value: LOCAL_MODE },
{ name: `SSH (${sshDir})`, value: 'ssh' },
],
},
]);
for (const key of selectedKeys) {
const encryptedKey = path.join(keyDir, key, `id_${key}.age`);
const publicKey = path.join(keyDir, key, `id_${key}.pub`);
const privateKeyOut =
decryptMode === 'Local (vault/tmp)'
? path.join(vaultDir, 'tmp', `id_${key}`)
: path.join(sshDir, `id_${key}`);
const publicKeyOut =
decryptMode === 'Local (vault/tmp)'
? path.join(vaultDir, 'tmp', `id_${key}.pub`)
: path.join(sshDir, `id_${key}.pub`);
const outDir = decryptMode === LOCAL_MODE ? tmpDir : sshDir;
// Decrypt key
await execa('age', ['-d', '-i', ageKey, '-o', privateKeyOut, encryptedKey]);
const plans: DecryptPlan[] = selectedKeys.map((key: string) => ({
key,
encryptedKey: path.join(keysDir, key, `id_${key}.age`),
publicKey: path.join(keysDir, key, `id_${key}.pub`),
privateKeyOut: path.join(outDir, `id_${key}`),
publicKeyOut: path.join(outDir, `id_${key}.pub`),
}));
await execa('cp', [publicKey, publicKeyOut]);
await execa('chmod', ['600', privateKeyOut]);
console.log(`✅ Decrypted: ${privateKeyOut}`);
// Every collision is settled before anything is written. `age -d -o` and the
// old `cp` both overwrote silently, so decrypting a vault key on top of a
// newer working key destroyed it with no prompt and no copy — and the user is
// answering these questions about files that still exist.
const approved: DecryptPlan[] = [];
for (const plan of plans) {
const existing = [plan.privateKeyOut, plan.publicKeyOut].filter((file) => fs.existsSync(file));
if (existing.length === 0) {
approved.push(plan);
continue;
}
const { overwrite } = await inquirer.prompt<{ overwrite: boolean }>([
{
type: 'confirm',
name: 'overwrite',
message: `${existing.join(', ')} already present. Overwrite?`,
default: false,
},
]);
if (overwrite) {
approved.push(plan);
} else {
console.log(`⏭️ Skipped ${plan.key} — kept what was already there.`);
}
}
if (approved.length === 0) {
return;
}
// 0700: ~/.ssh may not exist yet, and it is about to hold a private key.
fs.mkdirSync(outDir, { recursive: true, mode: 0o700 });
for (const plan of approved) {
await runTool('age', ['-d', '-i', ageKey, '-o', plan.privateKeyOut, plan.encryptedKey]);
// Immediately, and in-process: age creates its output 0644 regardless of
// umask, so this used to be a world-readable private key for the length of
// two process spawns — and stayed 0644 whenever the chmod itself failed.
fs.chmodSync(plan.privateKeyOut, 0o600);
if (fs.existsSync(plan.publicKey)) {
fs.copyFileSync(plan.publicKey, plan.publicKeyOut);
} else {
console.log(
`⚠️ ${plan.key} has no public key in the vault; only the private key was written.`
);
}
console.log(`✅ Decrypted: ${plan.privateKeyOut}`);
}
}
+32 -24
View File
@@ -1,24 +1,19 @@
import fs from 'node:fs';
import path from 'node:path';
import { execa } from 'execa';
import inquirer from 'inquirer';
import { reportSkippedKeys, scanPrivateKeys } from './keyman.keys.js';
import { ToolNotFoundError } from './keyman.utils.js';
import { storeInVault } from './keyman.vault.js';
export async function encryptKeys(
sshDir: string,
vaultDir: string,
tmpDir: string,
pubkey: string
) {
const sshKeys = fs
.readdirSync(sshDir)
.filter((key) => key.startsWith('id_') && !key.endsWith('.pub'));
const tmpKeys = fs
.readdirSync(tmpDir)
.filter((key) => key.startsWith('id_') && !key.endsWith('.pub'));
console.log(tmpKeys);
console.log(sshKeys);
export async function encryptKeys(sshDir: string, keysDir: string, tmpDir: string, pubkey: string) {
const ssh = scanPrivateKeys(sshDir);
const tmp = scanPrivateKeys(tmpDir);
const sshKeys = ssh.keys;
const tmpKeys = tmp.keys;
const keys = [...new Set([...sshKeys, ...tmpKeys])];
reportSkippedKeys(ssh.skipped, sshDir);
reportSkippedKeys(tmp.skipped, tmpDir);
if (keys.length === 0) {
console.log('⚠️ No private SSH keys found to encrypt.');
return;
@@ -33,17 +28,30 @@ export async function encryptKeys(
},
]);
const failed: string[] = [];
for (const key of selectedKeys) {
const keyPath = path.join(tmpKeys.includes(key) ? tmpDir : sshDir, key);
const vaultPath = path.join(vaultDir, 'keys', key.replace('id_', ''));
fs.mkdirSync(vaultPath, { recursive: true });
// Encrypt key using `age`
await execa('age', ['-r', pubkey, '-o', path.join(vaultPath, `${key}.age`), keyPath]);
try {
await storeInVault(keyPath, keysDir, pubkey);
} catch (error) {
// One bad key costs one key. Selecting ten and losing the last nine to an
// unreadable first one was the old behaviour, and nothing afterwards said
// which of the ten had made it into the vault.
if (error instanceof ToolNotFoundError) {
// Not a per-key problem: age is missing for all of them, so nine more
// identical failures would tell the user nothing new.
throw error;
}
failed.push(key);
console.error(`❌ ${key}: ${error instanceof Error ? error.message : error}`);
}
}
// Copy public key and create README
fs.copyFileSync(`${keyPath}.pub`, path.join(vaultPath, `${key}.pub`));
console.log(`🔒 Encrypted and stored: ${vaultPath}/${key}`);
if (failed.length > 0) {
console.log(
`\n⚠️ ${failed.length} of ${selectedKeys.length} selected keys were not stored: ${failed.join(', ')}`
);
}
}
+76 -44
View File
@@ -1,12 +1,27 @@
import fs from 'node:fs';
import path from 'node:path';
import { execa } from 'execa';
import inquirer from 'inquirer';
import { runTool } from './keyman.utils.js';
import { storeInVault } from './keyman.vault.js';
export async function generateKey(tmpDir: string, keysDir: string, pubkey: string) {
export interface KeyOptions {
algorithm: string;
identity: string;
}
/**
* How a new key pair should be made: the algorithm and the comment.
*
* Shared with rotation, which asks the same two questions about a key whose name
* it works out for itself.
*
* @param defaultIdentity offered as the answer — the comment of the key being
* replaced, when there is one
*/
export async function promptKeyOptions(defaultIdentity?: string): Promise<KeyOptions> {
const { algorithm } = await inquirer.prompt<{ algorithm: string }>([
{
type: 'list',
type: 'select',
name: 'algorithm',
message: 'Select algorithm:',
choices: ['ed25519', 'rsa'],
@@ -14,6 +29,57 @@ export async function generateKey(tmpDir: string, keysDir: string, pubkey: strin
},
]);
const { identity } = await inquirer.prompt<{ identity: string }>([
{
type: 'input',
name: 'identity',
message: 'Enter key identity (comment):',
default: defaultIdentity,
},
]);
return { algorithm, identity };
}
/**
* Generates one key pair at `keyPath`, reporting a failure rather than throwing.
*
* @returns whether the key pair was written
*/
export async function createKeyPair(
keyPath: string,
algorithm: string,
identity: string
): Promise<boolean> {
const fileName = path.basename(keyPath);
if (fs.existsSync(keyPath)) {
console.error(`❌ Error: Key file ${fileName} already exists in ${path.dirname(keyPath)}`);
return false;
}
const args = ['-t', algorithm, '-f', keyPath, '-C', identity];
if (algorithm === 'rsa') {
args.push('-b', '4096');
}
try {
console.log(`Generating ${algorithm} key pair...`);
// No `-N`, and stdio inherited: ssh-keygen asks for the passphrase itself and
// confirms it. keyman used to prompt for it and pass it as `-N <value>`,
// which put the passphrase in this process's argv — readable by any user on
// the box via `ps` for as long as the spawn lived, and in keyman's memory
// before that. A passphrase keyman never learns cannot be leaked by keyman.
await runTool('ssh-keygen', args, { stdio: 'inherit' });
console.log(`✅ Key generated: ${keyPath}`);
return true;
} catch (error) {
console.error(`❌ Error generating key: ${error instanceof Error ? error.message : error}`);
return false;
}
}
export async function generateKey(tmpDir: string, keysDir: string, pubkey: string) {
const { keyName } = await inquirer.prompt<{ keyName: string }>([
{
type: 'input',
@@ -23,55 +89,21 @@ export async function generateKey(tmpDir: string, keysDir: string, pubkey: strin
},
]);
const { password } = await inquirer.prompt<{ password: string }>([
{
type: 'password',
name: 'password',
message: 'Enter passphrase (leave empty for no passphrase):',
mask: '*',
},
]);
const { identity } = await inquirer.prompt<{ identity: string }>([
{
type: 'input',
name: 'identity',
message: 'Enter key identity (comment):',
},
]);
const { algorithm, identity } = await promptKeyOptions();
const fileName = keyName.startsWith('id_') ? keyName : `id_${keyName}`;
const keyPath = path.join(tmpDir, fileName);
if (fs.existsSync(keyPath)) {
console.error(`❌ Error: Key file ${fileName} already exists in ${tmpDir}`);
if (!(await createKeyPair(keyPath, algorithm, identity))) {
return;
}
try {
console.log(`Generating ${algorithm} key pair...`);
const args = ['-t', algorithm, '-f', keyPath, '-N', password, '-C', identity];
if (algorithm === 'rsa') {
args.push('-b', '4096');
}
await execa('ssh-keygen', args);
console.log(`✅ Key generated: ${keyPath}`);
// Encrypt the key
const folderName = fileName.replace('id_', '');
const vaultPath = path.join(keysDir, folderName);
fs.mkdirSync(vaultPath, { recursive: true });
// Encrypt key using `age`
await execa('age', ['-r', pubkey, '-o', path.join(vaultPath, `${fileName}.age`), keyPath]);
// Copy public key
fs.copyFileSync(`${keyPath}.pub`, path.join(vaultPath, `${fileName}.pub`));
console.log(`🔒 Encrypted and stored: ${vaultPath}`);
await storeInVault(keyPath, keysDir, pubkey);
} catch (error) {
console.error(`❌ Error generating/encrypting key: ${error}`);
// The private key is still in tmpDir, so this is recoverable by encrypting it
// — which is why it does not read as having lost the key.
console.error(`❌ Error encrypting key: ${error instanceof Error ? error.message : error}`);
console.error(` ${keyPath} was generated; encrypt it once the problem is fixed.`);
}
}
+70
View File
@@ -0,0 +1,70 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
/** The answer the USER prompt defaults to: whoever is running keyman. */
export const CURRENT_USER = '@current';
/**
* The home directory of the current user.
*
* `HOME` first, because a user who set it meant it, and `os.userInfo()` after,
* which reads the passwd database and so still answers when `HOME` is unset — a
* cron job, a `su` without `-l`, a container entrypoint. `process.env.HOME || ''`
* treated all of those as a fatal error.
*/
function currentHome(): string | null {
if (process.env.HOME) {
return process.env.HOME;
}
try {
return os.userInfo().homedir || null;
} catch {
// uv_os_get_passwd can fail outright when there is no passwd entry for the uid.
return null;
}
}
/**
* Where another user's home directory is, without asking the system.
*
* The sibling of the current user's home comes first because it is right wherever
* homes live together, whatever that directory is called — `/Users` on macOS,
* `/home` on Linux, `/export/home` on the odd installation. keyman previously
* hardcoded `/home/<user>`, which is wrong on the one platform it was written on.
*
* The candidates are checked for existence rather than guessed at, so a wrong one
* produces an error naming what was tried instead of an empty `readdir`.
*/
function candidateHomes(user: string): string[] {
const home = currentHome();
const siblings = home ? [path.join(path.dirname(home), user)] : [];
return [...new Set([...siblings, path.join('/home', user), path.join('/Users', user)])];
}
/**
* Resolves the home directory for an answer to the USER prompt.
*
* @returns the directory, or null with the reason already reported
*/
export function resolveHomeDir(user: string): string | null {
if (user === CURRENT_USER) {
const home = currentHome();
if (!home) {
console.error('❌ Unable to determine HOME directory for the current user.');
return null;
}
return home;
}
const candidates = candidateHomes(user);
const found = candidates.find((candidate) => fs.existsSync(candidate));
if (!found) {
console.error(`❌ No home directory found for ${user}. Tried: ${candidates.join(', ')}`);
return null;
}
return found;
}
+93
View File
@@ -0,0 +1,93 @@
import fs from 'node:fs';
import path from 'node:path';
/** Present in the first line of every private key format ssh-keygen writes. */
const PRIVATE_KEY_MARKER = 'PRIVATE KEY-----';
/** Enough for `-----BEGIN OPENSSH PRIVATE KEY-----`, and no more of a key than needed. */
const HEADER_BYTES = 64;
/**
* Whether a file opens with a private key header.
*
* A bounded read of the first line, not the file: classifying a key is no reason
* to pull one into memory.
*/
function looksLikePrivateKey(file: string): boolean {
let handle: number | undefined;
try {
handle = fs.openSync(file, 'r');
const buffer = Buffer.alloc(HEADER_BYTES);
const read = fs.readSync(handle, buffer, 0, HEADER_BYTES, 0);
return buffer.subarray(0, read).toString('latin1').includes(PRIVATE_KEY_MARKER);
} catch {
// A directory, a socket, a file with no read permission — none of them a key.
return false;
} finally {
if (handle !== undefined) {
fs.closeSync(handle);
}
}
}
export interface PrivateKeyScan {
/** Keys keyman can manage: named `id_*`, which is what the vault layout assumes. */
keys: string[];
/** Private keys it found and cannot manage, because they are named otherwise. */
skipped: string[];
}
/**
* The private keys in a directory that may not exist.
*
* A first run has neither `~/.ssh` nor the tmp directory, and an unguarded readdir
* there threw before the "nothing to encrypt" message could be reached.
*
* `skipped` exists because the `id_*` filter is silent: a key named
* `deploy_ed25519` was simply absent from every menu, and pre-existing keys are
* the population a key manager gets adopted to take over. Reporting them is not
* managing them — see `reportSkippedKeys`.
*/
export function scanPrivateKeys(dir: string): PrivateKeyScan {
if (!fs.existsSync(dir)) {
return { keys: [], skipped: [] };
}
const keys: string[] = [];
const skipped: string[] = [];
// Sorted, because readdir order is the filesystem's business and a menu's order
// should not depend on it.
for (const file of fs.readdirSync(dir).sort()) {
if (file.endsWith('.pub')) {
continue;
}
if (file.startsWith('id_')) {
// Not content-checked: what the menus offered has not changed.
keys.push(file);
} else if (looksLikePrivateKey(path.join(dir, file))) {
skipped.push(file);
}
}
return { keys, skipped };
}
/**
* Says which private keys were found and left alone, and why.
*
* The vault stores a key as `<name minus id_>/id_<name>.age` and `decrypt`
* reconstructs the filename from the directory, so the prefix is baked into the
* on-disk layout — which is why this is a report and not a fix.
*/
export function reportSkippedKeys(skipped: string[], dir: string): void {
if (skipped.length === 0) {
return;
}
const plural = skipped.length === 1 ? 'key' : 'keys';
console.log(
`ℹ️ Skipped ${skipped.length} private ${plural} in ${dir} not named id_*: ${skipped.join(', ')}`
);
console.log(' The vault layout requires the id_ prefix; rename to manage them here.');
}
+12 -2
View File
@@ -1,5 +1,6 @@
import fs from 'node:fs';
import path from 'node:path';
import { reportSkippedKeys, scanPrivateKeys } from './keyman.keys.js';
interface KeyInfo {
name: string;
@@ -76,9 +77,12 @@ export async function listKeys(sshDir: string, vaultDir: string, tmpDir: string)
// Scan vault directory
if (fs.existsSync(vaultDir)) {
// throwIfNoEntry keeps a dangling symlink from aborting the whole listing;
// the stat still follows a symlink to a real directory, which withFileTypes
// would have reported as a link and skipped.
const vaultDirs = fs.readdirSync(vaultDir).filter((dir) => {
const stat = fs.statSync(path.join(vaultDir, dir));
return stat.isDirectory();
const stat = fs.statSync(path.join(vaultDir, dir), { throwIfNoEntry: false });
return stat?.isDirectory() ?? false;
});
for (const dir of vaultDirs) {
@@ -102,6 +106,12 @@ export async function listKeys(sshDir: string, vaultDir: string, tmpDir: string)
}
}
// A listing that omits keys without saying so is the worst place for the id_
// assumption to be invisible: this is the screen a user checks it against.
for (const dir of [sshDir, tmpDir]) {
reportSkippedKeys(scanPrivateKeys(dir).skipped, dir);
}
// Display results
if (keyMap.size === 0) {
console.log('⚠️ No SSH keys found.\n');
+55 -17
View File
@@ -1,12 +1,15 @@
import fs from 'node:fs';
import path from 'node:path';
import inquirer from 'inquirer';
import { clearDecryptedKeys, writeVaultGitignore } from './keyman.clear.js';
import { loadConfig, resolveConfigPaths } from './keyman.config.js';
import { copyKey } from './keyman.copy.js';
import { decryptKeys } from './keyman.decrypt.js';
import { encryptKeys } from './keyman.encrypt.js';
import { generateKey } from './keyman.generate.js';
import { CURRENT_USER, resolveHomeDir } from './keyman.home.js';
import { listKeys } from './keyman.list.js';
import { retireKey, rotateKey } from './keyman.rotate.js';
import { extractAgePublicKey } from './keyman.utils.js';
// 🔹 Main function to resolve paths and manage flow
@@ -25,20 +28,36 @@ export async function keyman() {
{
type: 'input',
name: 'user',
message: 'Specify USER (default: @current):',
default: '@current',
message: `Specify USER (default: ${CURRENT_USER}):`,
default: CURRENT_USER,
},
]);
const homeDir = user === '@current' ? process.env.HOME || '' : `/home/${user}`;
const homeDir = resolveHomeDir(user);
if (!homeDir) {
console.error('Error: Unable to determine HOME directory.');
process.exit(1);
}
const sshDir = path.join(homeDir, '.ssh');
fs.mkdirSync(paths.vaultRoot, { recursive: true });
fs.mkdirSync(paths.tmpDir, { recursive: true });
// 0700 because the vault holds the age identity and, in tmp, plaintext private
// keys. keysDir is created here too: decrypt used to read it before anything
// created it.
for (const dir of [paths.vaultRoot, paths.keysDir, paths.tmpDir]) {
fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
}
writeVaultGitignore(paths.vaultRoot, paths.tmpDir, paths.keyPath);
// Resolved on demand, because only generate and encrypt need a recipient, and
// remembered once it succeeds. Retried while it has not: creating the identity
// mid-session should not mean restarting.
let recipient: string | null = null;
const ageRecipient = async () => {
recipient ??= await extractAgePublicKey(paths.keyPath);
if (!recipient) {
console.error(` Create one with: age-keygen -o ${paths.keyPath}`);
}
return recipient;
};
// Main loop - keep showing menu until user quits
let running = true;
@@ -48,7 +67,7 @@ export async function keyman() {
// 🔹 Show category selection
const { category } = await inquirer.prompt<{ category: string }>([
{
type: 'list',
type: 'select',
name: 'category',
message: 'Select operation:',
choices: [
@@ -57,6 +76,9 @@ export async function keyman() {
{ name: '🆕 Generate key', value: 'generate' },
{ name: '🔒 Encrypt keys', value: 'encrypt' },
{ name: '🔓 Decrypt keys', value: 'decrypt' },
{ name: '🔄 Rotate key', value: 'rotate' },
{ name: '🗑️ Retire key', value: 'retire' },
{ name: '🧹 Clear decrypted keys', value: 'clear' },
{ name: '❌ Quit', value: 'quit' },
],
},
@@ -69,19 +91,35 @@ export async function keyman() {
case 'copy':
await copyKey(sshDir, paths.tmpDir);
break;
case 'generate':
await generateKey(paths.tmpDir, paths.keysDir, extractAgePublicKey(paths.keyPath)!);
case 'generate': {
const pubkey = await ageRecipient();
if (pubkey) {
await generateKey(paths.tmpDir, paths.keysDir, pubkey);
}
break;
case 'encrypt':
await encryptKeys(
sshDir,
paths.vaultRoot,
paths.tmpDir,
extractAgePublicKey(paths.keyPath)!
);
}
case 'encrypt': {
const pubkey = await ageRecipient();
if (pubkey) {
await encryptKeys(sshDir, paths.keysDir, paths.tmpDir, pubkey);
}
break;
}
case 'decrypt':
await decryptKeys(sshDir, paths.vaultRoot, paths.keyPath);
await decryptKeys(sshDir, paths.keysDir, paths.tmpDir, paths.keyPath);
break;
case 'rotate': {
const pubkey = await ageRecipient();
if (pubkey) {
await rotateKey(sshDir, paths.keysDir, paths.tmpDir, pubkey);
}
break;
}
case 'retire':
await retireKey(sshDir, paths.keysDir, paths.tmpDir);
break;
case 'clear':
await clearDecryptedKeys(paths.tmpDir);
break;
case 'quit':
console.log('\n👋 Goodbye!\n');
+256
View File
@@ -0,0 +1,256 @@
/**
* Key rotation, in two halves that are deliberately not one operation.
*
* `rotateKey` only ever *adds*: a replacement key generated under the next name in
* the series and encrypted alongside the key it replaces. `retireKey` is what
* finally deletes the old one, once the user says the replacement is deployed.
*
* Rotating in place — overwriting the key, or deleting it in the same breath —
* locks you out of the host you were rotating for: the replacement is not on it
* yet, and the only copy of the key that is has gone. The gap between the two
* operations is where you add the new public key and check that it works.
*/
import fs from 'node:fs';
import path from 'node:path';
import inquirer from 'inquirer';
import { createKeyPair, promptKeyOptions } from './keyman.generate.js';
import { scanPrivateKeys } from './keyman.keys.js';
import { listVaultKeys, storeInVault } from './keyman.vault.js';
interface Series {
base: string;
version: number;
}
/** `prod-2` → base `prod`, version 2. An unsuffixed name is version 1. */
function series(key: string): Series {
const match = /^(.+)-(\d+)$/.exec(key);
return match ? { base: match[1], version: Number(match[2]) } : { base: key, version: 1 };
}
/**
* The name for the replacement of `key`: same series, next version up.
*
* The name has to change. The vault layout derives the directory from it, so a
* replacement also called `prod` *is* the `prod` entry — and holding both at once
* is the whole point of rotating this way.
*
* @param taken every name already in use, in the vault or as a plaintext key, so
* the suffix skips a version that was made by hand
*/
export function nextRotationName(key: string, taken: string[]): string {
const { base, version } = series(key);
let next = version + 1;
for (const name of taken) {
const other = series(name);
if (other.base === base && other.version >= next) {
next = other.version + 1;
}
}
return `${base}-${next}`;
}
/** The latest key in the vault that comes after `key` in its series, if any. */
export function supersededBy(key: string, vaultKeys: string[]): string | null {
const { base, version } = series(key);
let successor: string | null = null;
let highest = version;
for (const name of vaultKeys) {
const other = series(name);
if (other.base === base && other.version > highest) {
successor = name;
highest = other.version;
}
}
return successor;
}
/** The bare names of the plaintext keys in `dir`, matching the vault's naming. */
function plaintextNames(dir: string): string[] {
return scanPrivateKeys(dir).keys.map((file) => file.replace(/^id_/, ''));
}
/** The comment on a stored public key, so a rotation can carry it over. */
function storedComment(publicKeyFile: string): string | undefined {
if (!fs.existsSync(publicKeyFile)) {
return undefined;
}
// `<type> <base64> <comment...>`: the comment is optional and may hold spaces.
const comment = fs.readFileSync(publicKeyFile, 'utf-8').trim().split(/\s+/).slice(2).join(' ');
return comment || undefined;
}
/** Prints a public key for copying, or says why it cannot. */
function showPublicKey(label: string, file: string): void {
console.log(`\n ${label}`);
if (fs.existsSync(file)) {
console.log(` ${fs.readFileSync(file, 'utf-8').trim()}`);
} else {
console.log(` (none stored at ${file})`);
}
}
function isFile(file: string): boolean {
return fs.statSync(file, { throwIfNoEntry: false })?.isFile() ?? false;
}
/**
* Generates a replacement for a vault key and stores it beside the original.
*
* Nothing is deleted or overwritten; `retireKey` is the other half.
*/
export async function rotateKey(
sshDir: string,
keysDir: string,
tmpDir: string,
pubkey: string
): Promise<void> {
const vaultKeys = listVaultKeys(keysDir);
if (vaultKeys.length === 0) {
console.log('⚠️ No encrypted keys to rotate — generate or encrypt one first.');
return;
}
const { key } = await inquirer.prompt<{ key: string }>([
{
type: 'select',
name: 'key',
message: 'Select the key to rotate:',
choices: vaultKeys,
},
]);
const currentPublicKey = path.join(keysDir, key, `id_${key}.pub`);
const { algorithm, identity } = await promptKeyOptions(storedComment(currentPublicKey));
const replacement = nextRotationName(key, [
...vaultKeys,
...plaintextNames(tmpDir),
...plaintextNames(sshDir),
]);
const keyPath = path.join(tmpDir, `id_${replacement}`);
console.log(`\n🔄 Rotating ${key} → ${replacement}`);
console.log(` ${key} is left exactly as it is, in the vault and on its hosts.\n`);
fs.mkdirSync(tmpDir, { recursive: true, mode: 0o700 });
if (!(await createKeyPair(keyPath, algorithm, identity))) {
return;
}
try {
await storeInVault(keyPath, keysDir, pubkey);
} catch (error) {
console.error(
`❌ Error encrypting the replacement: ${error instanceof Error ? error.message : error}`
);
console.error(` ${keyPath} was generated; encrypt it once the problem is fixed.`);
return;
}
showPublicKey(`Current — still valid (${key}):`, currentPublicKey);
showPublicKey(`Replacement — deploy this (${replacement}):`, `${keyPath}.pub`);
console.log('\n Next:');
console.log(` 1. Add the replacement public key wherever ${key} is authorized.`);
console.log(` 2. Check that you can log in with ${keyPath}.`);
console.log(` 3. Remove ${key} from those hosts, then retire it here.\n`);
}
/**
* Deletes a vault key and its plaintext copies, after saying exactly what goes.
*
* The second half of a rotation, and the only operation in keyman that destroys an
* encrypted key.
*/
export async function retireKey(sshDir: string, keysDir: string, tmpDir: string): Promise<void> {
const vaultKeys = listVaultKeys(keysDir);
if (vaultKeys.length === 0) {
console.log('⚠️ No encrypted keys in the vault.');
return;
}
const { key } = await inquirer.prompt<{ key: string }>([
{
type: 'select',
name: 'key',
message: 'Select the key to retire:',
choices: vaultKeys,
},
]);
const vaultPath = path.join(keysDir, key);
const files = [
...fs.readdirSync(vaultPath).map((file) => path.join(vaultPath, file)),
path.join(tmpDir, `id_${key}`),
path.join(tmpDir, `id_${key}.pub`),
path.join(sshDir, `id_${key}`),
path.join(sshDir, `id_${key}.pub`),
].filter(isFile);
const successor = supersededBy(key, vaultKeys);
console.log(`\n🗑️ Retiring ${key} deletes:`);
for (const file of files) {
console.log(` ${file}`);
}
if (successor) {
console.log(`\n ${successor} is in the vault and supersedes ${key}.`);
} else {
console.log(`\n⚠️ Nothing in the vault supersedes ${key}: this deletes the only copy.`);
}
const { confirmed } = await inquirer.prompt<{ confirmed: boolean }>([
{
type: 'confirm',
name: 'confirmed',
message: `Delete ${files.length} ${files.length === 1 ? 'file' : 'files'}?`,
default: false,
},
]);
if (!confirmed) {
console.log(' Nothing was deleted.');
return;
}
// Typed out when there is no successor, because that is the deletion this tool
// exists to prevent: an encrypted key nothing replaces is the only copy there is,
// and a y/n is one keystroke away from an irreversible one.
if (!successor) {
const { typed } = await inquirer.prompt<{ typed: string }>([
{
type: 'input',
name: 'typed',
message: `Type ${key} to confirm:`,
},
]);
if (typed.trim() !== key) {
console.log(' Name did not match — nothing was deleted.');
return;
}
}
for (const file of files) {
fs.rmSync(file, { force: true });
console.log(` Removed ${file}`);
}
try {
// Only while empty: anything left in there was not ours to delete.
fs.rmdirSync(vaultPath);
} catch {
console.log(` Kept ${vaultPath} — it still holds other files.`);
}
console.log(`✅ Retired ${key}.`);
}
+79 -4
View File
@@ -1,16 +1,91 @@
import fs from 'node:fs';
import { execa, type Options } from 'execa';
/** A binary keyman needs is not installed — recoverable, unlike a tool refusing */
export class ToolNotFoundError extends Error {
constructor(readonly binary: string) {
super(`\`${binary}\` was not found on PATH. Install it and try again.`);
this.name = 'ToolNotFoundError';
}
}
/**
* Extracts the public key from an age key file.
* @param keyFilePath Path to the age key file.
* @returns The public key as a string, or null if not found.
* Runs one of the external binaries keyman depends on.
*
* Two failures are worth telling apart, and an execa error tells a reader
* neither: the binary not being installed (`ENOENT`, whose message is
* `spawn <name> ENOENT`) and the binary refusing (whose reason is on stderr and
* nowhere in the thrown message). `age` is a hard requirement, so its absence
* has to read as an instruction.
*
* Returns only `stdout` — annotated rather than inferred because execa's result
* type cannot be named from here (TS2883), and it is all any caller wants. Empty
* when the output went somewhere else, as with `stdio: 'inherit'`.
*/
export function extractAgePublicKey(keyFilePath: string): string | null {
export async function runTool(
binary: string,
args: string[],
options?: Options
): Promise<{ stdout: string }> {
try {
// Called without the third argument when there are no options, so a test
// asserting on the spawn sees the call it wrote.
const result = options ? await execa(binary, args, options) : await execa(binary, args);
return { stdout: typeof result.stdout === 'string' ? result.stdout : '' };
} catch (error) {
const failure = error as { code?: string; stderr?: string; shortMessage?: string };
if (failure.code === 'ENOENT') {
throw new ToolNotFoundError(binary);
}
throw new Error(`\`${binary}\` failed: ${failure.stderr?.trim() || failure.shortMessage}`);
}
}
/**
* The age recipient a vault encrypts to, derived from its identity file.
*
* `age-keygen -y` derives the public key from the secret key, so it cannot
* disagree with it. The `# public key:` comment can: it is ordinary text that
* nothing re-checks, and a wrong one encrypts the vault to a recipient nobody
* holds the private half of. Verified — rewriting the comment does not change
* what `-y` reports.
*
* The comment stays as a fallback for a machine with no `age-keygen`, behind a
* warning that it is unverified. It is *not* a fallback for `age-keygen`
* refusing the file: that means age cannot read the identity, and trusting the
* comment then would encrypt to a recipient the vault could never decrypt with.
*
* @returns the recipient, or null with the reason already reported
*/
export async function extractAgePublicKey(keyFilePath: string): Promise<string | null> {
if (!fs.existsSync(keyFilePath)) {
console.error(`❌ ERROR: Age key file not found at ${keyFilePath}`);
return null;
}
try {
const { stdout } = await runTool('age-keygen', ['-y', keyFilePath]);
const derived = stdout.trim();
if (derived.startsWith('age1')) {
return derived;
}
console.error(`❌ ERROR: age-keygen derived no public key from ${keyFilePath}`);
return null;
} catch (error) {
if (!(error instanceof ToolNotFoundError)) {
console.error(`❌ ERROR: ${error instanceof Error ? error.message : error}`);
return null;
}
console.warn(
`⚠️ age-keygen is not installed — reading the public key from the comment in ${keyFilePath}, unverified against the secret key.`
);
}
return publicKeyFromComment(keyFilePath);
}
/** The `# public key:` line: a claim about the key rather than a derivation from it */
function publicKeyFromComment(keyFilePath: string): string | null {
try {
const fileContents = fs.readFileSync(keyFilePath, 'utf-8');
const publicKeyMatch = fileContents.match(/^# public key:\s*(age1[^\s]+)/m);
+109
View File
@@ -0,0 +1,109 @@
import fs from 'node:fs';
import path from 'node:path';
import { runTool } from './keyman.utils.js';
/**
* The vault entries that hold an encrypted key, sorted.
*
* A directory counts as an entry when it holds `id_<dir>.age` — the layout
* `storeInVault` writes and `decrypt` reads back — which is what keeps a stray
* file, or a directory whose encryption failed, out of every menu built from this.
* Sorted because the order otherwise comes from the filesystem.
*/
export function listVaultKeys(keysDir: string): string[] {
// Nothing creates the keys directory until the first encrypt, so on a fresh
// vault this readdir threw instead of reporting an empty one.
if (!fs.existsSync(keysDir)) {
return [];
}
return fs
.readdirSync(keysDir)
.filter((key) => fs.existsSync(path.join(keysDir, key, `id_${key}.age`)))
.sort();
}
/**
* The public half of a private key, derived if the sibling file is missing.
*
* `encrypt` builds its selection list from private keys only, so a key whose
* `.pub` was deleted is offered like any other. Reading the sibling blindly meant
* finding out it was absent *after* `age` had written the encrypted key — a vault
* entry with no public key, and an exception that killed the rest of the batch.
*
* @returns the public key text, or null with the reason already reported
*/
async function publicKeyFor(keyPath: string): Promise<string | null> {
const sibling = `${keyPath}.pub`;
if (fs.existsSync(sibling)) {
return fs.readFileSync(sibling, 'utf-8');
}
const fileName = path.basename(keyPath);
console.log(`ℹ️ ${fileName} has no .pub file — deriving it with ssh-keygen.`);
try {
// stdin and stderr inherited, stdout piped: verified that `ssh-keygen -y`
// prompts for the passphrase of an encrypted key, and that it prompts on
// *stderr*. Capturing everything would hide the prompt and then fail on the
// passphrase nobody was asked for; inheriting everything would lose the key.
const { stdout } = await runTool('ssh-keygen', ['-y', '-f', keyPath], {
stdio: ['inherit', 'pipe', 'inherit'],
});
const derived = stdout.trim();
if (derived) {
return `${derived}\n`;
}
} catch (error) {
console.warn(`⚠️ ${fileName}: ${error instanceof Error ? error.message : error}`);
}
console.warn(`⚠️ ${fileName}: no public key could be derived; storing the private key alone.`);
return null;
}
/**
* Encrypts one private key into `<keysDir>/<name>/`, alongside its public half.
*
* Shared by `encrypt` and `generate`, which were two copies of it.
*
* @returns the vault directory the key was stored in
*/
export async function storeInVault(
keyPath: string,
keysDir: string,
pubkey: string
): Promise<string> {
const fileName = path.basename(keyPath);
const vaultPath = path.join(keysDir, fileName.replace(/^id_/, ''));
// Before the directory exists, so a key that cannot be read does not leave one.
const publicKey = await publicKeyFor(keyPath);
fs.mkdirSync(vaultPath, { recursive: true, mode: 0o700 });
const encryptedKey = path.join(vaultPath, `${fileName}.age`);
try {
await runTool('age', ['-r', pubkey, '-o', encryptedKey, keyPath]);
} catch (error) {
// age writes into a directory that has to exist already, so a failure here
// leaves one behind — and possibly a truncated .age file, which `list` would
// count as a vault entry and `decrypt` would offer. Both are ours: the file
// because we named it, the directory only while it is empty, since one
// holding an earlier key is not.
fs.rmSync(encryptedKey, { force: true });
try {
fs.rmdirSync(vaultPath);
} catch {
// ENOTEMPTY — something else was already stored here.
}
throw error;
}
if (publicKey !== null) {
fs.writeFileSync(path.join(vaultPath, `${fileName}.pub`), publicKey);
}
console.log(`🔒 Encrypted and stored: ${path.join(vaultPath, `${fileName}.age`)}`);
return vaultPath;
}
+155
View File
@@ -0,0 +1,155 @@
/**
* Tests for keyman's argv parsing.
*
* The old inline reader in keyman.cli.ts turned three different mistakes into
* silence or into a wrong diagnosis, so the interesting cases here are the
* rejections rather than the happy paths.
*/
import { describe, expect, it } from 'vitest';
import { CHANNELS, helpText, KNOWN_FLAGS, parseArgs, UsageError } from '../src/keyman.args.js';
describe('parseArgs', () => {
it('defaults to the interactive session', () => {
expect(parseArgs([])).toEqual({ command: 'interactive' });
});
it.each([
[['--help'], 'help'],
[['-h'], 'help'],
[['--version'], 'version'],
[['-V'], 'version'],
[['--print-config'], 'print-config'],
] as const)('%s selects %s', (argv, command) => {
expect(parseArgs([...argv])).toEqual({ command });
});
it('answers --help even when the rest of the line is wrong', () => {
expect(parseArgs(['--bogus', '--help'])).toEqual({ command: 'help' });
expect(parseArgs(['--help', '--channel'])).toEqual({ command: 'help' });
});
describe('self-update', () => {
it.each(['self-update', 'upgrade'])('is selected by the %s subcommand', (subcommand) => {
expect(parseArgs([subcommand])).toEqual({
command: 'self-update',
dryRun: false,
force: false,
channel: undefined,
registry: undefined,
});
});
it('is selected by --self-update', () => {
expect(parseArgs(['--self-update'])).toMatchObject({ command: 'self-update' });
});
it('collects its flags, long and short', () => {
expect(parseArgs(['self-update', '--dry-run', '--force'])).toMatchObject({
dryRun: true,
force: true,
});
expect(parseArgs(['self-update', '-n', '-f'])).toMatchObject({
dryRun: true,
force: true,
});
});
it.each(['--channel main', '--channel=main'])('accepts %s', (form) => {
expect(parseArgs(['self-update', ...form.split(' ')])).toMatchObject({ channel: 'main' });
});
it('accepts every real channel', () => {
for (const channel of CHANNELS) {
expect(parseArgs(['self-update', '--channel', channel])).toMatchObject({ channel });
}
});
it('reads a registry in either form', () => {
expect(parseArgs(['self-update', '--registry', 'https://r.example'])).toMatchObject({
registry: 'https://r.example',
});
expect(parseArgs(['self-update', '--registry=https://r.example'])).toMatchObject({
registry: 'https://r.example',
});
});
it('keeps a value that starts with a dash when it was written inline', () => {
expect(parseArgs(['self-update', '--registry=-weird'])).toMatchObject({
registry: '-weird',
});
});
});
describe('rejections', () => {
const reject = (argv: string[]) => () => parseArgs(argv);
it('rejects a channel that is not a channel', () => {
expect(reject(['self-update', '--channel', 'stable'])).toThrow(UsageError);
expect(reject(['self-update', '--channel', 'stable'])).toThrow(
'Unknown channel: stable (expected latest, next, main)'
);
});
it('rejects the next flag being eaten as a value', () => {
// The bug this whole module exists for: --channel --force used to set the
// channel to "--force" and report an unreachable registry.
expect(reject(['self-update', '--channel', '--force'])).toThrow('--channel expects a value');
});
it('rejects a value flag with nothing after it', () => {
expect(reject(['self-update', '--channel'])).toThrow('--channel expects a value');
expect(reject(['self-update', '--registry='])).toThrow('--registry expects a value');
});
it('rejects a boolean flag given a value', () => {
expect(reject(['--dry-run=yes'])).toThrow('--dry-run does not take a value');
});
it('rejects unknown flags and commands', () => {
expect(reject(['--vault', 'foo'])).toThrow('Unknown flag: --vault');
expect(reject(['-x'])).toThrow('Unknown flag: -x');
expect(reject(['encrypt'])).toThrow('Unknown command: encrypt');
expect(reject(['self-update', 'upgrade'])).toThrow('Unexpected argument: upgrade');
});
it.each(['--channel', '--registry', '--dry-run', '-n', '--force', '-f'])(
'rejects %s without self-update rather than ignoring it',
(flag) => {
const argv = flag === '--channel' || flag === '--registry' ? [flag, 'main'] : [flag];
expect(reject(argv)).toThrow('is only valid with `keyman self-update`');
}
);
it('rejects a self-update flag alongside another command', () => {
expect(reject(['--print-config', '--force'])).toThrow('--force is only valid');
});
});
});
describe('helpText', () => {
it('documents every flag the parser accepts', () => {
const text = helpText();
for (const flag of KNOWN_FLAGS) {
expect(text, `${flag} is missing from --help`).toContain(flag);
}
});
it('names both subcommands, every channel, and the environment variables', () => {
const text = helpText();
expect(text).toContain('self-update');
expect(text).toContain('upgrade');
for (const channel of CHANNELS) {
expect(text).toContain(channel);
}
for (const variable of [
'VAULT_ROOT',
'KEYMAN_REGISTRY',
'KEYMAN_REGISTRY_TOKEN',
'KEYMAN_NO_UPDATE_CHECK',
'KEYMAN_PACKAGE_MANAGER',
]) {
expect(text).toContain(variable);
}
});
});
+158
View File
@@ -0,0 +1,158 @@
/**
* Tests for the plaintext hygiene helpers: the vault .gitignore and the
* clear-decrypted-keys operation.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const { prompt } = vi.hoisted(() => ({ prompt: vi.fn() }));
vi.mock('inquirer', () => ({ default: { prompt } }));
import { clearDecryptedKeys, writeVaultGitignore } from '../src/keyman.clear.js';
describe('writeVaultGitignore', () => {
let vaultRoot: string;
const read = () => fs.readFileSync(path.join(vaultRoot, '.gitignore'), 'utf-8');
beforeEach(() => {
vaultRoot = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-ignore-')));
});
afterEach(() => {
fs.rmSync(vaultRoot, { recursive: true, force: true });
});
it('ignores the identity and the decrypted keys, not the encrypted ones', () => {
writeVaultGitignore(vaultRoot, path.join(vaultRoot, 'tmp'), path.join(vaultRoot, 'age.key'));
const contents = read();
expect(contents).toContain('age.key\n');
expect(contents).toContain('age.key.pub');
expect(contents).toContain('tmp/');
// The encrypted keys are the thing worth committing.
expect(contents).not.toContain('keys/');
});
it('uses the configured names', () => {
writeVaultGitignore(
vaultRoot,
path.join(vaultRoot, 'plain'),
path.join(vaultRoot, 'identity.age')
);
expect(read()).toContain('plain/');
expect(read()).toContain('identity.age');
});
it('says nothing about a directory outside the vault', () => {
writeVaultGitignore(vaultRoot, '/elsewhere/tmp', path.join(vaultRoot, 'age.key'));
// A .gitignore cannot speak for a path above itself, and pretending otherwise
// would read as protection that is not there.
expect(read()).not.toContain('elsewhere');
expect(read()).toContain('age.key');
});
it('never overwrites an existing file', () => {
fs.writeFileSync(path.join(vaultRoot, '.gitignore'), 'mine\n');
writeVaultGitignore(vaultRoot, path.join(vaultRoot, 'tmp'), path.join(vaultRoot, 'age.key'));
expect(read()).toBe('mine\n');
});
it('creates it private to the owner', () => {
writeVaultGitignore(vaultRoot, path.join(vaultRoot, 'tmp'), path.join(vaultRoot, 'age.key'));
expect(fs.statSync(path.join(vaultRoot, '.gitignore')).mode & 0o777).toBe(0o600);
});
});
describe('clearDecryptedKeys', () => {
let tmpDir: string;
let logSpy: ReturnType<typeof vi.spyOn>;
const messages = () => logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
const decrypted = (name: string) => {
fs.writeFileSync(path.join(tmpDir, name), 'PRIVATE');
fs.writeFileSync(path.join(tmpDir, `${name}.pub`), 'PUBLIC');
};
beforeEach(() => {
vi.clearAllMocks();
tmpDir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-clear-')));
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
prompt.mockResolvedValue({ confirmed: true });
});
afterEach(() => {
vi.restoreAllMocks();
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('says so when there is nothing to clear', async () => {
await clearDecryptedKeys(tmpDir);
expect(messages()).toContain('Nothing decrypted');
expect(prompt).not.toHaveBeenCalled();
});
it('does not mind a tmp directory that was never created', async () => {
fs.rmSync(tmpDir, { recursive: true });
await expect(clearDecryptedKeys(tmpDir)).resolves.toBeUndefined();
});
it('removes each key and its public half', async () => {
decrypted('id_prod');
decrypted('id_stage');
await clearDecryptedKeys(tmpDir);
expect(fs.readdirSync(tmpDir)).toEqual([]);
expect(messages()).toContain('Removed id_prod');
});
it('lists what it is about to delete before asking', async () => {
decrypted('id_prod');
await clearDecryptedKeys(tmpDir);
const askedAt = messages().indexOf('id_prod');
expect(askedAt).toBeGreaterThanOrEqual(0);
expect(prompt.mock.calls[0][0][0]).toMatchObject({ type: 'confirm', default: false });
});
it('keeps everything when the confirmation is declined', async () => {
decrypted('id_prod');
prompt.mockResolvedValue({ confirmed: false });
await clearDecryptedKeys(tmpDir);
expect(fs.existsSync(path.join(tmpDir, 'id_prod'))).toBe(true);
expect(messages()).toContain('Nothing was deleted');
});
it('leaves files that are not keys alone', async () => {
decrypted('id_prod');
fs.writeFileSync(path.join(tmpDir, 'notes.md'), 'mine');
await clearDecryptedKeys(tmpDir);
expect(fs.readdirSync(tmpDir)).toEqual(['notes.md']);
});
it('does not fail on a key whose public half is missing', async () => {
fs.writeFileSync(path.join(tmpDir, 'id_prod'), 'PRIVATE');
await clearDecryptedKeys(tmpDir);
expect(fs.readdirSync(tmpDir)).toEqual([]);
});
});
+85
View File
@@ -0,0 +1,85 @@
/**
* Tests for the clipboard layer.
*
* The platform is passed in rather than stubbed, so every branch is reachable from
* the one machine the suite runs on.
*/
import { beforeEach, describe, expect, it, vi } from 'vitest';
const { execa } = vi.hoisted(() => ({ execa: vi.fn() }));
vi.mock('execa', () => ({ execa }));
import { clipboardTools, copyToClipboard } from '../src/keyman.clipboard.js';
describe('clipboardTools', () => {
it.each([
['darwin', ['pbcopy']],
['win32', ['clip']],
['linux', ['wl-copy', 'xclip', 'xsel']],
// Anything unrecognised gets the X11/Wayland list rather than nothing: a BSD
// running the same desktop stack is closer to linux than to no answer.
['freebsd', ['wl-copy', 'xclip', 'xsel']],
])('offers the right commands on %s', (platform, expected) => {
expect(clipboardTools(platform).map((t) => t.binary)).toEqual(expected);
});
it('passes the clipboard selection to the X11 tools', () => {
const byBinary = new Map(clipboardTools('linux').map((t) => [t.binary, t.args]));
// Without these, xclip and xsel write to the primary selection, which is not
// the clipboard a paste reads from.
expect(byBinary.get('xclip')).toEqual(['-selection', 'clipboard']);
expect(byBinary.get('xsel')).toEqual(['--clipboard', '--input']);
});
});
describe('copyToClipboard', () => {
const notFound = () =>
execa.mockImplementation(async () => {
throw Object.assign(new Error('spawn ENOENT'), { code: 'ENOENT' });
});
beforeEach(() => {
vi.clearAllMocks();
execa.mockResolvedValue({ stdout: '' });
});
it('pipes the text to the first available command', async () => {
const tool = await copyToClipboard('ssh-ed25519 AAAA', 'darwin');
expect(tool).toBe('pbcopy');
expect(execa).toHaveBeenCalledWith('pbcopy', [], { input: 'ssh-ed25519 AAAA' });
});
it('moves on from a command that is not installed', async () => {
execa.mockImplementation(async (binary: string) => {
if (binary !== 'xclip') {
throw Object.assign(new Error('spawn ENOENT'), { code: 'ENOENT' });
}
return { stdout: '' };
});
expect(await copyToClipboard('key', 'linux')).toBe('xclip');
expect(execa.mock.calls.map((c) => c[0])).toEqual(['wl-copy', 'xclip']);
});
it('reports that nothing was available rather than throwing', async () => {
notFound();
expect(await copyToClipboard('key', 'linux')).toBeNull();
expect(execa).toHaveBeenCalledTimes(3);
});
it('surfaces a command that ran and refused', async () => {
execa.mockImplementation(async () => {
throw Object.assign(new Error('failed'), { stderr: 'Error: No protocol specified' });
});
// A tool with an opinion is not an absent tool: trying the next one would
// hide a real problem behind a second failure.
await expect(copyToClipboard('key', 'linux')).rejects.toThrow('No protocol specified');
expect(execa).toHaveBeenCalledTimes(1);
});
});
+84 -27
View File
@@ -11,6 +11,7 @@ import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import {
describeConfig,
getConfigPaths,
type KeymanConfigFile,
loadConfig,
@@ -208,49 +209,82 @@ describe('keyman config', () => {
});
});
describe('merge strategy', () => {
it('honours an explicit override strategy', () => {
write(rootDir, { vaultRoot: '/parent-vault' });
const child = path.join(rootDir, 'nested');
write(child, { vaultRoot: '/child-vault', resolution: { vaultRoot: 'override' } });
process.chdir(child);
describe('unknown keys', () => {
/** What a config file is likely to get wrong: the casing of a real key. */
const TYPO = { vaultroot: '/somewhere-else' } as unknown as KeymanConfigFile;
expect(loadConfig().vaultRoot).toBe('/child-vault');
it('names the file, the key and what it could have been', () => {
write(rootDir, TYPO);
loadConfig();
const warned = messages(warnSpy);
expect(warned).toContain(path.join(rootDir, '.keymanrc.json'));
expect(warned).toContain('vaultroot');
// Without the list of known keys the warning says a key is wrong without
// saying what right looks like, which for a casing slip is most of the work.
expect(warned).toContain('vaultRoot');
});
it('never surfaces the resolution key in the loaded config', () => {
write(rootDir, { keysDir: 'my-keys', resolution: { keysDir: 'override' } });
expect(loadConfig()).not.toHaveProperty('resolution');
});
it('tolerates and drops array-valued keys the schema does not define', () => {
write(rootDir, { extra: ['a', 'b'] } as unknown as KeymanConfigFile);
const child = path.join(rootDir, 'nested');
write(child, { extra: ['b', 'c'], keysDir: 'my-keys' } as unknown as KeymanConfigFile);
process.chdir(child);
it('still applies the keys it does understand', () => {
write(rootDir, { ...TYPO, keysDir: 'my-keys' });
const config = loadConfig();
expect(config).toEqual({ ...DEFAULTS, keysDir: 'my-keys' });
expect(config.keysDir).toBe('my-keys');
expect(config.vaultRoot).toBe(DEFAULTS.vaultRoot);
});
it('tolerates and drops object-valued keys the schema does not define', () => {
write(rootDir, { extra: { a: 1 } } as unknown as KeymanConfigFile);
it('blames the file that said it, not the merged result', () => {
write(rootDir, {});
const child = path.join(rootDir, 'nested');
write(child, { extra: { a: 2, b: 3 } } as unknown as KeymanConfigFile);
write(child, TYPO);
process.chdir(child);
expect(loadConfig()).toEqual(DEFAULTS);
loadConfig();
expect(messages(warnSpy)).toContain(path.join(child, '.keymanrc.json'));
expect(messages(warnSpy)).not.toContain(path.join(rootDir, '.keymanrc.json'));
});
it('tolerates arrays of objects, which cannot be de-duplicated', () => {
write(rootDir, { extra: [{ a: 1 }] } as unknown as KeymanConfigFile);
it('lists every unknown key in one warning per file', () => {
write(rootDir, { nope: 1, alsoNope: 2 } as unknown as KeymanConfigFile);
loadConfig();
expect(warnSpy).toHaveBeenCalledTimes(1);
expect(messages(warnSpy)).toContain('nope, alsoNope');
expect(messages(warnSpy)).toContain('unknown keys');
});
it('says key, singular, for one of them', () => {
write(rootDir, TYPO);
loadConfig();
expect(messages(warnSpy)).toContain('unknown key ');
});
it('says nothing about a file that sets only known keys', () => {
write(rootDir, { keysDir: 'my-keys', tmpDir: 'my-tmp' });
loadConfig();
expect(warnSpy).not.toHaveBeenCalled();
});
it.each([
['array-valued', { extra: ['a', 'b'] }],
['object-valued', { extra: { a: 1 } }],
['an array of objects', { extra: [{ a: 1 }] }],
])('drops a %s unknown key rather than merging it in', (_label, extra) => {
write(rootDir, extra as unknown as KeymanConfigFile);
const child = path.join(rootDir, 'nested');
write(child, { extra: [{ a: 2 }] } as unknown as KeymanConfigFile);
write(child, { ...extra, keysDir: 'my-keys' } as unknown as KeymanConfigFile);
process.chdir(child);
expect(loadConfig()).toEqual(DEFAULTS);
// The schema strips them; nothing in keyman merges an array or an object.
expect(loadConfig()).toEqual({ ...DEFAULTS, keysDir: 'my-keys' });
});
});
@@ -291,4 +325,27 @@ describe('keyman config', () => {
expect(paths.keysDir).toBe('/elsewhere/keys');
});
});
describe('describeConfig', () => {
it('reports the resolved paths and the files they came from', () => {
write(rootDir, { keysDir: 'my-keys', vaultRoot: 'vault' });
const child = path.join(rootDir, 'nested');
write(child, { tmpDir: 'my-tmp' });
process.chdir(child);
expect(describeConfig()).toEqual({
vaultRoot: path.join(rootDir, 'vault'),
keysDir: path.join(rootDir, 'vault', 'my-keys'),
tmpDir: path.join(rootDir, 'vault', 'my-tmp'),
keyPath: path.join(rootDir, 'vault', 'age.key'),
// Parent first, the order they were merged in — which is the only way to
// read a surprising value back to the file responsible for it.
configFiles: [path.join(rootDir, '.keymanrc.json'), path.join(child, '.keymanrc.json')],
});
});
it('reports an empty list when nothing was found', () => {
expect(describeConfig().configFiles).toEqual([]);
});
});
});
+39 -14
View File
@@ -10,11 +10,7 @@ import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const { execa, prompt, stdin } = vi.hoisted(() => ({
execa: vi.fn(),
prompt: vi.fn(),
stdin: { write: vi.fn(), end: vi.fn() },
}));
const { execa, prompt } = vi.hoisted(() => ({ execa: vi.fn(), prompt: vi.fn() }));
vi.mock('execa', () => ({ execa }));
vi.mock('inquirer', () => ({ default: { prompt } }));
@@ -27,6 +23,7 @@ describe('copyKey', () => {
let tmpDir: string;
let logSpy: ReturnType<typeof vi.spyOn>;
let errorSpy: ReturnType<typeof vi.spyOn>;
let warnSpy: ReturnType<typeof vi.spyOn>;
const touch = (dir: string, file: string, contents = '') => {
fs.mkdirSync(dir, { recursive: true });
@@ -39,6 +36,9 @@ describe('copyKey', () => {
/** The choices offered by the last inquirer.prompt call. */
const choices = () => prompt.mock.calls.at(-1)?.[0][0].choices as string[];
/** What was piped into the clipboard command. */
const piped = () => (execa.mock.calls.at(-1)?.[2] as { input?: string } | undefined)?.input;
beforeEach(() => {
vi.clearAllMocks();
root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-copy-')));
@@ -46,9 +46,9 @@ describe('copyKey', () => {
tmpDir = path.join(root, 'tmp');
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
const proc = Object.assign(Promise.resolve({ exitCode: 0 }), { stdin });
execa.mockReturnValue(proc);
execa.mockResolvedValue({ stdout: '' });
});
afterEach(() => {
@@ -93,9 +93,7 @@ describe('copyKey', () => {
await copyKey(sshDir, tmpDir);
expect(execa).toHaveBeenCalledWith('pbcopy');
expect(stdin.write).toHaveBeenCalledWith('ssh-ed25519 AAAA tmp');
expect(stdin.end).toHaveBeenCalled();
expect(piped()).toBe('ssh-ed25519 AAAA tmp');
expect(messages(logSpy)).toContain('copied to clipboard');
});
@@ -106,7 +104,7 @@ describe('copyKey', () => {
await copyKey(sshDir, tmpDir);
expect(stdin.write).toHaveBeenCalledWith('ssh-ed25519 AAAA ssh');
expect(piped()).toBe('ssh-ed25519 AAAA ssh');
});
it('reports a missing public key without invoking the clipboard', async () => {
@@ -123,11 +121,38 @@ describe('copyKey', () => {
touch(sshDir, 'id_prod');
touch(sshDir, 'id_prod.pub', 'ssh-ed25519 AAAA ssh');
prompt.mockResolvedValue({ selectedKey: 'id_prod' });
execa.mockImplementation(() => {
throw new Error('pbcopy missing');
});
execa.mockRejectedValue(Object.assign(new Error('refused'), { stderr: 'no display' }));
await expect(copyKey(sshDir, tmpDir)).resolves.toBeUndefined();
expect(messages(errorSpy)).toContain('Failed to copy to clipboard');
});
it('prints the key when no clipboard command exists at all', async () => {
touch(sshDir, 'id_prod');
touch(sshDir, 'id_prod.pub', 'ssh-ed25519 AAAA ssh');
prompt.mockResolvedValue({ selectedKey: 'id_prod' });
execa.mockImplementation(async () => {
throw Object.assign(new Error('spawn ENOENT'), { code: 'ENOENT' });
});
await copyKey(sshDir, tmpDir);
// The operation is "give me this public key". Without a clipboard it is still
// answerable, and it used to be a dead end on every platform but macOS.
expect(messages(logSpy)).toContain('ssh-ed25519 AAAA ssh');
expect(messages(warnSpy)).toContain('No clipboard command found');
});
it('names the private keys it cannot manage', async () => {
touch(sshDir, 'id_prod');
touch(sshDir, 'id_prod.pub', 'PUBLIC');
touch(sshDir, 'deploy_ed25519', '-----BEGIN OPENSSH PRIVATE KEY-----\nAAAA\n');
prompt.mockResolvedValue({ selectedKey: 'id_prod' });
await copyKey(sshDir, tmpDir);
expect(choices()).toEqual(['id_prod']);
expect(messages(logSpy)).toContain('deploy_ed25519');
expect(messages(logSpy)).toContain('not named id_*');
});
});
+182 -34
View File
@@ -1,8 +1,10 @@
/**
* Tests for decryptKeys.
*
* age, cp and chmod are all mocked; the assertions cover which keys are
* offered and exactly where each decrypted key is written.
* Only `age` is mocked, and its stand-in writes the output file the way age
* would: the copy and the chmod are now real fs calls, so the assertions are on
* what ends up on disk and at what mode rather than on which binaries were
* spawned.
*/
import fs from 'node:fs';
@@ -17,27 +19,35 @@ vi.mock('inquirer', () => ({ default: { prompt } }));
import { decryptKeys } from '../src/keyman.decrypt.js';
const LOCAL = 'Local (vault/tmp)';
const SSH = 'SSH (~/.ssh)';
const LOCAL = 'local';
const SSH = 'ssh';
describe('decryptKeys', () => {
let root: string;
let sshDir: string;
let vaultDir: string;
let keyDir: string;
let keysDir: string;
let tmpDir: string;
let logSpy: ReturnType<typeof vi.spyOn>;
const AGE_KEY = '/vault/age.key';
/** Creates <vault>/keys/<name>/id_<name>.{age,pub}. */
const vaultKey = (name: string) => {
const dir = path.join(keyDir, name);
const dir = path.join(keysDir, name);
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, `id_${name}.age`), 'ENCRYPTED');
fs.writeFileSync(path.join(dir, `id_${name}.pub`), 'PUBLIC');
fs.writeFileSync(path.join(dir, `id_${name}.age`), `ENCRYPTED ${name}`);
fs.writeFileSync(path.join(dir, `id_${name}.pub`), `PUBLIC ${name}`);
};
const choices = () => prompt.mock.calls.at(-1)?.[0][0].choices as string[];
/** Answers the selection prompt, then every overwrite confirmation. */
const answers = (selectedKeys: string[], decryptMode = LOCAL, overwrite = false) => {
prompt.mockImplementation(async (questions: { name: string }[]) =>
questions[0].name === 'selectedKeys' ? { selectedKeys, decryptMode } : { overwrite }
);
};
const choices = () => prompt.mock.calls[0]?.[0][0].choices as string[];
const argsOf = (binary: string) =>
execa.mock.calls.find((c) => c[0] === binary)?.[1] as string[] | undefined;
@@ -45,16 +55,26 @@ describe('decryptKeys', () => {
const messages = (spy: ReturnType<typeof vi.spyOn>) =>
spy.mock.calls.map((c) => c.join(' ')).join('\n');
const modeOf = (file: string) => fs.statSync(file).mode & 0o777;
beforeEach(() => {
vi.clearAllMocks();
root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-decrypt-')));
sshDir = path.join(root, '.ssh');
vaultDir = path.join(root, 'vault');
keyDir = path.join(vaultDir, 'keys');
fs.mkdirSync(keyDir, { recursive: true });
keysDir = path.join(vaultDir, 'keys');
tmpDir = path.join(vaultDir, 'tmp');
fs.mkdirSync(keysDir, { recursive: true });
fs.mkdirSync(sshDir, { recursive: true });
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
execa.mockResolvedValue({ exitCode: 0 });
// Stand in for `age -d`: write the plaintext to -o, 0644 as age does.
execa.mockImplementation(async (_binary: string, args: string[]) => {
const out = args[args.indexOf('-o') + 1];
fs.mkdirSync(path.dirname(out), { recursive: true });
fs.writeFileSync(out, 'PLAINTEXT', { mode: 0o644 });
return { exitCode: 0 };
});
});
afterEach(() => {
@@ -63,72 +83,200 @@ describe('decryptKeys', () => {
});
it('warns when the vault holds no encrypted keys', async () => {
await decryptKeys(sshDir, vaultDir, AGE_KEY);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(messages(logSpy)).toContain('No encrypted keys found.');
expect(prompt).not.toHaveBeenCalled();
});
it('warns instead of throwing when the vault has no keys directory', async () => {
fs.rmSync(keysDir, { recursive: true });
await expect(decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY)).resolves.toBeUndefined();
expect(messages(logSpy)).toContain('No encrypted keys found.');
});
it('reports a missing age binary rather than an ENOENT', async () => {
vaultKey('prod');
answers(['prod']);
execa.mockImplementation(async () => {
throw Object.assign(new Error('spawn age ENOENT'), { code: 'ENOENT' });
});
await expect(decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY)).rejects.toThrow(
'`age` was not found on PATH'
);
});
it('offers only directories that actually contain an encrypted key', async () => {
vaultKey('prod');
fs.mkdirSync(path.join(keyDir, 'empty'), { recursive: true });
fs.writeFileSync(path.join(keyDir, 'README.md'), '');
prompt.mockResolvedValue({ selectedKeys: [], decryptMode: LOCAL });
fs.mkdirSync(path.join(keysDir, 'empty'), { recursive: true });
fs.writeFileSync(path.join(keysDir, 'README.md'), '');
answers([]);
await decryptKeys(sshDir, vaultDir, AGE_KEY);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(choices()).toEqual(['prod']);
});
it('decrypts into the vault tmp directory', async () => {
vaultKey('prod');
prompt.mockResolvedValue({ selectedKeys: ['prod'], decryptMode: LOCAL });
answers(['prod']);
await decryptKeys(sshDir, vaultDir, AGE_KEY);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
const out = path.join(vaultDir, 'tmp', 'id_prod');
const out = path.join(tmpDir, 'id_prod');
expect(argsOf('age')).toEqual([
'-d',
'-i',
AGE_KEY,
'-o',
out,
path.join(keyDir, 'prod', 'id_prod.age'),
path.join(keysDir, 'prod', 'id_prod.age'),
]);
expect(argsOf('cp')).toEqual([path.join(keyDir, 'prod', 'id_prod.pub'), `${out}.pub`]);
expect(argsOf('chmod')).toEqual(['600', out]);
expect(fs.readFileSync(out, 'utf-8')).toBe('PLAINTEXT');
expect(fs.readFileSync(`${out}.pub`, 'utf-8')).toBe('PUBLIC prod');
expect(messages(logSpy)).toContain(`Decrypted: ${out}`);
});
it('decrypts into the .ssh directory when asked', async () => {
vaultKey('prod');
prompt.mockResolvedValue({ selectedKeys: ['prod'], decryptMode: SSH });
answers(['prod'], SSH);
await decryptKeys(sshDir, vaultDir, AGE_KEY);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
const out = path.join(sshDir, 'id_prod');
expect(argsOf('age')?.[4]).toBe(out);
expect(argsOf('cp')?.[1]).toBe(`${out}.pub`);
expect(argsOf('chmod')).toEqual(['600', out]);
expect(fs.readFileSync(`${out}.pub`, 'utf-8')).toBe('PUBLIC prod');
});
it('decrypts every selected key', async () => {
it('creates the .ssh directory when it does not exist, private to the owner', async () => {
fs.rmSync(sshDir, { recursive: true });
vaultKey('prod');
answers(['prod'], SSH);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(modeOf(sshDir)).toBe(0o700);
expect(fs.existsSync(path.join(sshDir, 'id_prod'))).toBe(true);
});
it('leaves the private key at 0600, never observable at what age wrote', async () => {
vaultKey('prod');
answers(['prod']);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(modeOf(path.join(tmpDir, 'id_prod'))).toBe(0o600);
});
it('decrypts every selected key with one spawn each', async () => {
vaultKey('prod');
vaultKey('stage');
prompt.mockResolvedValue({ selectedKeys: ['prod', 'stage'], decryptMode: LOCAL });
answers(['prod', 'stage']);
await decryptKeys(sshDir, vaultDir, AGE_KEY);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
// age, cp and chmod for each of the two keys.
expect(execa).toHaveBeenCalledTimes(6);
// One age per key: the cp and chmod spawns are gone.
expect(execa).toHaveBeenCalledTimes(2);
expect(execa.mock.calls.every((c) => c[0] === 'age')).toBe(true);
});
it('does nothing when the selection is empty', async () => {
vaultKey('prod');
prompt.mockResolvedValue({ selectedKeys: [], decryptMode: LOCAL });
answers([]);
await decryptKeys(sshDir, vaultDir, AGE_KEY);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(execa).not.toHaveBeenCalled();
});
it('writes the private key even when the vault entry has no public key', async () => {
vaultKey('prod');
fs.rmSync(path.join(keysDir, 'prod', 'id_prod.pub'));
answers(['prod']);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(fs.existsSync(path.join(tmpDir, 'id_prod'))).toBe(true);
expect(messages(logSpy)).toContain('has no public key in the vault');
});
describe('when the target already exists', () => {
const existing = (dir: string, name = 'id_prod') => {
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, name), 'PRECIOUS EXISTING KEY');
return path.join(dir, name);
};
it('keeps the existing key by default', async () => {
vaultKey('prod');
const target = existing(tmpDir);
answers(['prod']);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(fs.readFileSync(target, 'utf-8')).toBe('PRECIOUS EXISTING KEY');
expect(execa).not.toHaveBeenCalled();
expect(messages(logSpy)).toContain('Skipped prod');
});
it('asks before overwriting, defaulting to no', async () => {
vaultKey('prod');
existing(tmpDir);
answers(['prod']);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
const confirm = prompt.mock.calls.at(-1)?.[0][0];
expect(confirm).toMatchObject({ type: 'confirm', default: false });
expect(confirm.message).toContain(path.join(tmpDir, 'id_prod'));
});
it('overwrites once confirmed', async () => {
vaultKey('prod');
const target = existing(tmpDir);
answers(['prod'], LOCAL, true);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(fs.readFileSync(target, 'utf-8')).toBe('PLAINTEXT');
});
it('asks about an existing public key too', async () => {
vaultKey('prod');
existing(tmpDir, 'id_prod.pub');
answers(['prod']);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(execa).not.toHaveBeenCalled();
expect(prompt.mock.calls.at(-1)?.[0][0].message).toContain('id_prod.pub');
});
it('protects a key in .ssh the same way', async () => {
vaultKey('prod');
const target = existing(sshDir);
answers(['prod'], SSH);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
expect(fs.readFileSync(target, 'utf-8')).toBe('PRECIOUS EXISTING KEY');
});
it('settles every collision before decrypting anything', async () => {
vaultKey('prod');
vaultKey('stage');
existing(tmpDir);
answers(['prod', 'stage']);
await decryptKeys(sshDir, keysDir, tmpDir, AGE_KEY);
// stage is written, prod is kept — and the question about prod was asked
// before either was touched.
expect(fs.existsSync(path.join(tmpDir, 'id_stage'))).toBe(true);
expect(fs.readFileSync(path.join(tmpDir, 'id_prod'), 'utf-8')).toBe('PRECIOUS EXISTING KEY');
expect(execa).toHaveBeenCalledTimes(1);
});
});
});
+146 -16
View File
@@ -20,7 +20,7 @@ import { encryptKeys } from '../src/keyman.encrypt.js';
describe('encryptKeys', () => {
let root: string;
let sshDir: string;
let vaultDir: string;
let keysDir: string;
let tmpDir: string;
let logSpy: ReturnType<typeof vi.spyOn>;
@@ -41,7 +41,7 @@ describe('encryptKeys', () => {
vi.clearAllMocks();
root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-encrypt-')));
sshDir = path.join(root, '.ssh');
vaultDir = path.join(root, 'vault');
keysDir = path.join(root, 'vault', 'keys');
tmpDir = path.join(root, 'vault', 'tmp');
fs.mkdirSync(sshDir, { recursive: true });
fs.mkdirSync(tmpDir, { recursive: true });
@@ -60,17 +60,44 @@ describe('encryptKeys', () => {
});
it('warns when there is nothing to encrypt', async () => {
await encryptKeys(sshDir, vaultDir, tmpDir, PUBKEY);
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(messages(logSpy)).toContain('No private SSH keys found to encrypt.');
expect(prompt).not.toHaveBeenCalled();
});
it('warns instead of throwing when the .ssh directory does not exist', async () => {
fs.rmSync(sshDir, { recursive: true });
await expect(encryptKeys(sshDir, keysDir, tmpDir, PUBKEY)).resolves.toBeUndefined();
expect(messages(logSpy)).toContain('No private SSH keys found to encrypt.');
});
it('still offers the .ssh keys when the tmp directory does not exist', async () => {
fs.rmSync(tmpDir, { recursive: true });
key(sshDir, 'id_prod', 'ssh');
prompt.mockResolvedValue({ selectedKeys: [] });
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(choices()).toEqual(['id_prod']);
});
it('reports a missing age binary rather than an ENOENT', async () => {
key(sshDir, 'id_prod', 'ssh');
prompt.mockResolvedValue({ selectedKeys: ['id_prod'] });
execa.mockRejectedValue(Object.assign(new Error('spawn age ENOENT'), { code: 'ENOENT' }));
await expect(encryptKeys(sshDir, keysDir, tmpDir, PUBKEY)).rejects.toThrow(
'`age` was not found on PATH'
);
});
it('ignores public keys and unrelated files when building the list', async () => {
fs.writeFileSync(path.join(sshDir, 'known_hosts'), '');
fs.writeFileSync(path.join(sshDir, 'id_orphan.pub'), 'PUBLIC');
await encryptKeys(sshDir, vaultDir, tmpDir, PUBKEY);
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(messages(logSpy)).toContain('No private SSH keys found to encrypt.');
});
@@ -81,7 +108,7 @@ describe('encryptKeys', () => {
key(tmpDir, 'id_stage', 'tmp');
prompt.mockResolvedValue({ selectedKeys: [] });
await encryptKeys(sshDir, vaultDir, tmpDir, PUBKEY);
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(choices()).toEqual(['id_prod', 'id_stage']);
});
@@ -90,9 +117,9 @@ describe('encryptKeys', () => {
key(sshDir, 'id_prod', 'ssh');
prompt.mockResolvedValue({ selectedKeys: ['id_prod'] });
await encryptKeys(sshDir, vaultDir, tmpDir, PUBKEY);
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
const vaultPath = path.join(vaultDir, 'keys', 'prod');
const vaultPath = path.join(keysDir, 'prod');
expect(execa).toHaveBeenCalledWith('age', [
'-r',
PUBKEY,
@@ -109,12 +136,10 @@ describe('encryptKeys', () => {
key(tmpDir, 'id_prod', 'tmp');
prompt.mockResolvedValue({ selectedKeys: ['id_prod'] });
await encryptKeys(sshDir, vaultDir, tmpDir, PUBKEY);
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(execa.mock.calls[0][1]).toContain(path.join(tmpDir, 'id_prod'));
expect(fs.readFileSync(path.join(vaultDir, 'keys', 'prod', 'id_prod.pub'), 'utf-8')).toBe(
'PUBLIC tmp'
);
expect(fs.readFileSync(path.join(keysDir, 'prod', 'id_prod.pub'), 'utf-8')).toBe('PUBLIC tmp');
});
it('encrypts every selected key', async () => {
@@ -122,20 +147,125 @@ describe('encryptKeys', () => {
key(sshDir, 'id_stage', 'ssh');
prompt.mockResolvedValue({ selectedKeys: ['id_prod', 'id_stage'] });
await encryptKeys(sshDir, vaultDir, tmpDir, PUBKEY);
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(execa).toHaveBeenCalledTimes(2);
expect(fs.existsSync(path.join(vaultDir, 'keys', 'prod', 'id_prod.age'))).toBe(true);
expect(fs.existsSync(path.join(vaultDir, 'keys', 'stage', 'id_stage.age'))).toBe(true);
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.age'))).toBe(true);
expect(fs.existsSync(path.join(keysDir, 'stage', 'id_stage.age'))).toBe(true);
});
it('does nothing when the selection is empty', async () => {
key(sshDir, 'id_prod', 'ssh');
prompt.mockResolvedValue({ selectedKeys: [] });
await encryptKeys(sshDir, vaultDir, tmpDir, PUBKEY);
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(execa).not.toHaveBeenCalled();
expect(fs.existsSync(path.join(vaultDir, 'keys'))).toBe(false);
expect(fs.existsSync(keysDir)).toBe(false);
});
describe('a key with no .pub file', () => {
/** A private key without its sibling — what the selection list offers anyway. */
const orphan = (dir: string, name: string) => {
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, name), `PRIVATE ${name}`);
};
it('derives the public key with ssh-keygen', async () => {
orphan(sshDir, 'id_prod');
prompt.mockResolvedValue({ selectedKeys: ['id_prod'] });
execa.mockImplementation(async (binary: string, args: string[]) => {
if (binary === 'ssh-keygen') return { stdout: 'ssh-ed25519 AAAA derived' };
fs.writeFileSync(args[args.indexOf('-o') + 1], 'ENCRYPTED');
return { stdout: '' };
});
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(execa).toHaveBeenCalledWith(
'ssh-keygen',
['-y', '-f', path.join(sshDir, 'id_prod')],
// stderr inherited so the passphrase prompt is visible, stdout piped so
// the derived key can be captured.
{ stdio: ['inherit', 'pipe', 'inherit'] }
);
expect(fs.readFileSync(path.join(keysDir, 'prod', 'id_prod.pub'), 'utf-8')).toBe(
'ssh-ed25519 AAAA derived\n'
);
});
it('stores the private key alone when the derivation fails', async () => {
orphan(sshDir, 'id_prod');
prompt.mockResolvedValue({ selectedKeys: ['id_prod'] });
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
execa.mockImplementation(async (binary: string, args: string[]) => {
if (binary === 'ssh-keygen') {
throw Object.assign(new Error('bad passphrase'), { stderr: 'incorrect passphrase' });
}
fs.writeFileSync(args[args.indexOf('-o') + 1], 'ENCRYPTED');
return { stdout: '' };
});
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
// The encrypted key is what matters; the .pub is recoverable from it later.
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.age'))).toBe(true);
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.pub'))).toBe(false);
expect(messages(warnSpy)).toContain('no public key could be derived');
});
});
describe('when one key of several fails', () => {
beforeEach(() => {
key(sshDir, 'id_prod', 'ssh');
key(sshDir, 'id_stage', 'ssh');
prompt.mockResolvedValue({ selectedKeys: ['id_prod', 'id_stage'] });
// age refuses the first key only.
execa.mockImplementation(async (_binary: string, args: string[]) => {
if (args.some((arg) => arg.endsWith('id_prod'))) {
throw Object.assign(new Error('age refused'), { stderr: 'no identity' });
}
fs.writeFileSync(args[args.indexOf('-o') + 1], 'ENCRYPTED');
return { stdout: '' };
});
});
it('still encrypts the rest', async () => {
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(fs.existsSync(path.join(keysDir, 'stage', 'id_stage.age'))).toBe(true);
});
it('reports which keys were not stored', async () => {
const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
expect(messages(errorSpy)).toContain('id_prod');
expect(messages(logSpy)).toContain('1 of 2 selected keys were not stored: id_prod');
});
it('leaves no vault entry for the key that failed', async () => {
await encryptKeys(sshDir, keysDir, tmpDir, PUBKEY);
// Not even an empty directory: list counts a directory with an .age in it,
// and a truncated .age would be offered for decryption.
expect(fs.existsSync(path.join(keysDir, 'prod'))).toBe(false);
});
});
it('gives up immediately when age is not installed', async () => {
key(sshDir, 'id_prod', 'ssh');
key(sshDir, 'id_stage', 'ssh');
prompt.mockResolvedValue({ selectedKeys: ['id_prod', 'id_stage'] });
execa.mockImplementation(async () => {
throw Object.assign(new Error('spawn age ENOENT'), { code: 'ENOENT' });
});
// Not a per-key failure: nine more identical errors help nobody.
await expect(encryptKeys(sshDir, keysDir, tmpDir, PUBKEY)).rejects.toThrow(
'`age` was not found on PATH'
);
expect(execa).toHaveBeenCalledTimes(1);
});
});
+33 -12
View File
@@ -42,6 +42,10 @@ describe('generateKey', () => {
const argsOf = (binary: string) =>
execa.mock.calls.find((c) => c[0] === binary)?.[1] as string[] | undefined;
/** The options of the mocked call to `binary`. */
const optionsOf = (binary: string) =>
execa.mock.calls.find((c) => c[0] === binary)?.[2] as { stdio?: unknown } | undefined;
const messages = (spy: ReturnType<typeof vi.spyOn>) =>
spy.mock.calls.map((c) => c.join(' ')).join('\n');
@@ -64,7 +68,7 @@ describe('generateKey', () => {
return { exitCode: 0 };
});
answer({ algorithm: 'ed25519', keyName: 'prod', password: 'pw', identity: 'me@host' });
answer({ algorithm: 'ed25519', keyName: 'prod', identity: 'me@host' });
});
afterEach(() => {
@@ -80,16 +84,24 @@ describe('generateKey', () => {
'ed25519',
'-f',
path.join(tmpDir, 'id_prod'),
'-N',
'pw',
'-C',
'me@host',
]);
expect(messages(logSpy)).toContain('Key generated');
});
it('never handles the passphrase itself', async () => {
await generateKey(tmpDir, keysDir, PUBKEY);
// No -N, so ssh-keygen prompts and confirms; inherited stdio is what makes
// that prompt reach the terminal. The passphrase never touches argv.
expect(argsOf('ssh-keygen')).not.toContain('-N');
expect(optionsOf('ssh-keygen')).toEqual({ stdio: 'inherit' });
expect(prompt.mock.calls.map((c) => c[0][0].name)).not.toContain('password');
});
it('does not prefix a key name that already starts with id_', async () => {
answer({ algorithm: 'ed25519', keyName: 'id_prod', password: '', identity: '' });
answer({ algorithm: 'ed25519', keyName: 'id_prod', identity: '' });
await generateKey(tmpDir, keysDir, PUBKEY);
@@ -97,7 +109,7 @@ describe('generateKey', () => {
});
it('requests a 4096 bit key for rsa', async () => {
answer({ algorithm: 'rsa', keyName: 'prod', password: '', identity: '' });
answer({ algorithm: 'rsa', keyName: 'prod', identity: '' });
await generateKey(tmpDir, keysDir, PUBKEY);
@@ -139,17 +151,22 @@ describe('generateKey', () => {
expect(fs.readFileSync(path.join(tmpDir, 'id_prod'), 'utf-8')).toBe('EXISTING');
});
it('reports a failure from ssh-keygen without leaving a vault entry', async () => {
execa.mockRejectedValue(new Error('ssh-keygen exploded'));
it('reports a failure from ssh-keygen without reaching age', async () => {
execa.mockImplementation(async () => {
throw Object.assign(new Error('ssh-keygen exploded'), { stderr: 'ssh-keygen exploded' });
});
await expect(generateKey(tmpDir, keysDir, PUBKEY)).resolves.toBeUndefined();
expect(messages(errorSpy)).toContain('Error generating/encrypting key');
expect(messages(errorSpy)).toContain('Error generating key');
expect(argsOf('age')).toBeUndefined();
expect(fs.existsSync(path.join(keysDir, 'prod'))).toBe(false);
});
it('reports a failure from age', async () => {
it('reports a failure from age and says the key is still there to encrypt', async () => {
execa.mockImplementation(async (binary: string, args: string[]) => {
if (binary === 'age') throw new Error('age exploded');
if (binary === 'age') {
throw Object.assign(new Error('age exploded'), { stderr: 'age exploded' });
}
const keyPath = args[args.indexOf('-f') + 1];
fs.writeFileSync(keyPath, 'PRIVATE');
fs.writeFileSync(`${keyPath}.pub`, 'ssh-ed25519 AAAA generated');
@@ -158,7 +175,11 @@ describe('generateKey', () => {
await generateKey(tmpDir, keysDir, PUBKEY);
expect(messages(errorSpy)).toContain('Error generating/encrypting key');
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.pub'))).toBe(false);
expect(messages(errorSpy)).toContain('Error encrypting key');
// The generated key is the thing of value, and it survived.
expect(fs.existsSync(path.join(tmpDir, 'id_prod'))).toBe(true);
expect(messages(errorSpy)).toContain(path.join(tmpDir, 'id_prod'));
// And no half-made vault entry was left claiming to hold it.
expect(fs.existsSync(path.join(keysDir, 'prod'))).toBe(false);
});
});
+120
View File
@@ -0,0 +1,120 @@
/**
* Tests for home directory resolution.
*
* Real directories under os.tmpdir() stand in for home directories, since the
* whole point of the module is that it checks whether a path exists rather than
* assuming a layout.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { CURRENT_USER, resolveHomeDir } from '../src/keyman.home.js';
describe('resolveHomeDir', () => {
let homes: string;
let originalHome: string | undefined;
let errorSpy: ReturnType<typeof vi.spyOn>;
const messages = () => errorSpy.mock.calls.map((c) => c.join(' ')).join('\n');
beforeEach(() => {
originalHome = process.env.HOME;
homes = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-homes-')));
process.env.HOME = path.join(homes, 'alice');
fs.mkdirSync(process.env.HOME, { recursive: true });
errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
});
afterEach(() => {
vi.restoreAllMocks();
if (originalHome === undefined) {
delete process.env.HOME;
} else {
process.env.HOME = originalHome;
}
fs.rmSync(homes, { recursive: true, force: true });
});
describe('the current user', () => {
it('uses HOME when it is set', () => {
expect(resolveHomeDir(CURRENT_USER)).toBe(path.join(homes, 'alice'));
});
it('falls back to the passwd entry when HOME is unset', () => {
delete process.env.HOME;
// Not asserted as a literal: what matters is that an unset HOME is no longer
// a fatal error, which is what `process.env.HOME || ''` made it.
expect(resolveHomeDir(CURRENT_USER)).toBe(os.userInfo().homedir);
});
it('reports the failure when neither is available', () => {
delete process.env.HOME;
vi.spyOn(os, 'userInfo').mockImplementation(() => {
throw new Error('no passwd entry for uid');
});
expect(resolveHomeDir(CURRENT_USER)).toBeNull();
expect(messages()).toContain('Unable to determine HOME directory');
});
it('treats an empty passwd home as no answer', () => {
delete process.env.HOME;
vi.spyOn(os, 'userInfo').mockReturnValue({
...os.userInfo(),
homedir: '',
});
expect(resolveHomeDir(CURRENT_USER)).toBeNull();
});
});
describe('another user', () => {
it('looks beside the current home, whatever that directory is called', () => {
const bob = path.join(homes, 'bob');
fs.mkdirSync(bob);
// The old code hardcoded /home/<user>, which is wrong on macOS — where homes
// live in /Users — and on any host that puts them anywhere else.
expect(resolveHomeDir('bob')).toBe(bob);
});
it('still tries the conventional locations with no current home to go by', () => {
delete process.env.HOME;
vi.spyOn(os, 'userInfo').mockImplementation(() => {
throw new Error('no passwd entry for uid');
});
// Not knowing where *this* user lives is no reason to give up on another.
expect(resolveHomeDir('nobody')).toBeNull();
expect(messages()).toContain('/home/nobody, /Users/nobody');
expect(messages()).not.toContain('Unable to determine HOME');
});
it('reports every path it tried when there is no such home', () => {
expect(resolveHomeDir('nobody')).toBeNull();
expect(messages()).toContain('No home directory found for nobody');
expect(messages()).toContain(path.join(homes, 'nobody'));
expect(messages()).toContain('/home/nobody');
expect(messages()).toContain('/Users/nobody');
});
it('still checks the conventional locations when HOME is somewhere odd', () => {
process.env.HOME = path.join(homes, 'alice');
const conventional = process.platform === 'darwin' ? '/Users' : '/home';
const existing = fs
.readdirSync(conventional)
.find((entry) =>
fs.statSync(path.join(conventional, entry), { throwIfNoEntry: false })?.isDirectory()
);
// Skipped rather than asserted blind if the machine has no such user.
if (existing) {
expect(resolveHomeDir(existing)).toBe(path.join(conventional, existing));
}
});
});
});
+128
View File
@@ -0,0 +1,128 @@
/**
* Tests for private key discovery.
*
* Real files, because the classification is a bounded read of a real header.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { reportSkippedKeys, scanPrivateKeys } from '../src/keyman.keys.js';
const OPENSSH = '-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXk\n';
const RSA_PEM = '-----BEGIN RSA PRIVATE KEY-----\nProc-Type: 4,ENCRYPTED\n';
const PKCS8 = '-----BEGIN PRIVATE KEY-----\nMIIB\n';
describe('scanPrivateKeys', () => {
let dir: string;
const write = (name: string, contents: string) =>
fs.writeFileSync(path.join(dir, name), contents);
beforeEach(() => {
dir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-keys-')));
});
afterEach(() => {
fs.rmSync(dir, { recursive: true, force: true });
});
it('returns nothing for a directory that does not exist', () => {
expect(scanPrivateKeys(path.join(dir, 'nope'))).toEqual({ keys: [], skipped: [] });
});
it('offers the id_ keys and not their public halves', () => {
write('id_prod', OPENSSH);
write('id_prod.pub', 'ssh-ed25519 AAAA');
expect(scanPrivateKeys(dir)).toEqual({ keys: ['id_prod'], skipped: [] });
});
it('sorts the keys, so the menu order does not come from the filesystem', () => {
for (const name of ['id_stage', 'id_alpha', 'id_prod']) {
write(name, OPENSSH);
}
expect(scanPrivateKeys(dir).keys).toEqual(['id_alpha', 'id_prod', 'id_stage']);
});
it('offers an id_ file without checking what is in it', () => {
// Unchanged from before the scan existed: whatever was offered still is.
write('id_prod', 'not a key at all');
expect(scanPrivateKeys(dir).keys).toEqual(['id_prod']);
});
it.each([
['an OpenSSH key', OPENSSH],
['an encrypted PEM key', RSA_PEM],
['a PKCS#8 key', PKCS8],
])('reports %s that is not named id_*', (_label, contents) => {
write('deploy_ed25519', contents);
expect(scanPrivateKeys(dir)).toEqual({ keys: [], skipped: ['deploy_ed25519'] });
});
it('ignores the other files a .ssh directory is full of', () => {
write('known_hosts', 'github.com ssh-ed25519 AAAA');
write('config', 'Host *\n AddKeysToAgent yes\n');
write('authorized_keys', 'ssh-ed25519 AAAA');
fs.mkdirSync(path.join(dir, 'sockets'));
expect(scanPrivateKeys(dir)).toEqual({ keys: [], skipped: [] });
});
it('ignores a path it cannot read', () => {
// A dangling symlink, not a 0o000 file: root reads a 0o000 file happily, so
// the mode-based version of this passed here and failed on the CI runner,
// which is a container running as root. ENOENT nobody can override.
fs.symlinkSync(path.join(dir, 'gone'), path.join(dir, 'secret'));
// Reported as not-a-key rather than crashing the menu it was building.
expect(scanPrivateKeys(dir).skipped).toEqual([]);
});
it('does not read past the header', () => {
// The marker is in the first line; a mention further down is not a key.
write('decoy', `${'x'.repeat(200)}\nPRIVATE KEY-----\n`);
expect(scanPrivateKeys(dir).skipped).toEqual([]);
});
});
describe('reportSkippedKeys', () => {
let logSpy: ReturnType<typeof vi.spyOn>;
const messages = () => logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
beforeEach(() => {
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
});
afterEach(() => {
vi.restoreAllMocks();
});
it('says nothing when nothing was skipped', () => {
reportSkippedKeys([], '/home/alice/.ssh');
expect(logSpy).not.toHaveBeenCalled();
});
it('names the keys, the directory and the reason', () => {
reportSkippedKeys(['deploy_ed25519', 'backup_rsa'], '/home/alice/.ssh');
expect(messages()).toContain('deploy_ed25519, backup_rsa');
expect(messages()).toContain('/home/alice/.ssh');
expect(messages()).toContain('2 private keys');
// Without the reason the message is a complaint rather than an instruction.
expect(messages()).toContain('rename');
});
it('says key, singular, for one of them', () => {
reportSkippedKeys(['deploy_ed25519'], '/home/alice/.ssh');
expect(messages()).toContain('1 private key ');
});
});
+19
View File
@@ -182,6 +182,25 @@ describe('listKeys', () => {
expect(row('id_real')).toBeDefined();
});
it('keeps listing when the vault holds a dangling symlink', async () => {
vaultKey('real');
fs.symlinkSync(path.join(root, 'gone'), path.join(vaultDir, 'broken'));
await expect(listKeys(sshDir, vaultDir, tmpDir)).resolves.toBeUndefined();
expect(row('id_real')).toBeDefined();
});
it('follows a symlink pointing at a real vault directory', async () => {
const elsewhere = path.join(root, 'elsewhere', 'prod');
touch(elsewhere, 'id_prod.age');
fs.mkdirSync(vaultDir, { recursive: true });
fs.symlinkSync(elsewhere, path.join(vaultDir, 'prod'));
await listKeys(sshDir, vaultDir, tmpDir);
expect(row('id_prod')).toBeDefined();
});
it('ignores loose files sitting next to the vault directories', async () => {
vaultKey('real');
fs.writeFileSync(path.join(vaultDir, 'README.md'), '');
+141 -9
View File
@@ -19,6 +19,8 @@ const {
generateKey,
encryptKeys,
decryptKeys,
rotateKey,
retireKey,
extractAgePublicKey,
} = vi.hoisted(() => ({
prompt: vi.fn(),
@@ -29,6 +31,8 @@ const {
generateKey: vi.fn(),
encryptKeys: vi.fn(),
decryptKeys: vi.fn(),
rotateKey: vi.fn(),
retireKey: vi.fn(),
extractAgePublicKey: vi.fn(),
}));
@@ -39,6 +43,7 @@ vi.mock('../src/keyman.copy.js', () => ({ copyKey }));
vi.mock('../src/keyman.generate.js', () => ({ generateKey }));
vi.mock('../src/keyman.encrypt.js', () => ({ encryptKeys }));
vi.mock('../src/keyman.decrypt.js', () => ({ decryptKeys }));
vi.mock('../src/keyman.rotate.js', () => ({ rotateKey, retireKey }));
vi.mock('../src/keyman.utils.js', () => ({ extractAgePublicKey }));
import { keyman } from '../src/keyman.main.js';
@@ -81,7 +86,7 @@ describe('keyman', () => {
ageKeyFile: 'age.key',
});
resolveConfigPaths.mockReturnValue(paths);
extractAgePublicKey.mockReturnValue('age1recipient');
extractAgePublicKey.mockResolvedValue('age1recipient');
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
@@ -106,6 +111,17 @@ describe('keyman', () => {
expect(output()).toContain(paths.keyPath);
expect(fs.existsSync(paths.vaultRoot)).toBe(true);
expect(fs.existsSync(paths.tmpDir)).toBe(true);
// keysDir too: decrypt reads it, and nothing created it before the first
// encrypt, so a fresh vault could not be decrypted from.
expect(fs.existsSync(paths.keysDir)).toBe(true);
});
it('creates the vault directories private to the owner', async () => {
await keyman();
for (const dir of [paths.vaultRoot, paths.keysDir, paths.tmpDir]) {
expect(fs.statSync(dir).mode & 0o777, dir).toBe(0o700);
}
});
it('quits without running any operation', async () => {
@@ -125,6 +141,9 @@ describe('keyman', () => {
'generate',
'encrypt',
'decrypt',
'rotate',
'retire',
'clear',
'quit',
]);
});
@@ -161,31 +180,110 @@ describe('keyman', () => {
expect(generateKey).toHaveBeenCalledWith(paths.tmpDir, paths.keysDir, 'age1recipient');
});
it('encrypts keys into the vault root', async () => {
it('encrypts keys into the configured keys directory', async () => {
menu(['encrypt']);
await keyman();
expect(encryptKeys).toHaveBeenCalledWith(
path.join(process.env.HOME as string, '.ssh'),
paths.vaultRoot,
paths.keysDir,
paths.tmpDir,
'age1recipient'
);
});
it('decrypts keys using the age identity file', async () => {
it('decrypts from the configured keys directory using the age identity file', async () => {
menu(['decrypt']);
await keyman();
expect(decryptKeys).toHaveBeenCalledWith(
path.join(process.env.HOME as string, '.ssh'),
paths.vaultRoot,
paths.keysDir,
paths.tmpDir,
paths.keyPath
);
});
it('rotates a key with the age recipient, against the same directories', async () => {
menu(['rotate']);
await keyman();
expect(rotateKey).toHaveBeenCalledWith(
path.join(process.env.HOME as string, '.ssh'),
paths.keysDir,
paths.tmpDir,
'age1recipient'
);
});
it('retires a key without needing a recipient', async () => {
menu(['retire']);
await keyman();
expect(retireKey).toHaveBeenCalledWith(
path.join(process.env.HOME as string, '.ssh'),
paths.keysDir,
paths.tmpDir
);
// Retiring only deletes, so it works with no age identity at all.
expect(extractAgePublicKey).not.toHaveBeenCalled();
});
describe('without an age recipient', () => {
beforeEach(() => {
extractAgePublicKey.mockResolvedValue(null);
});
it.each([
['generate', generateKey],
['encrypt', encryptKeys],
['rotate', rotateKey],
])('refuses %s with a remedy instead of passing null to age', async (choice, operation) => {
menu([choice]);
await keyman();
expect(operation).not.toHaveBeenCalled();
const reported = errorSpy.mock.calls.map((c) => c.join(' ')).join('\n');
expect(reported).toContain(`age-keygen -o ${paths.keyPath}`);
// The whole point: the loop survives and quit is still reached.
expect(output()).toContain('Goodbye!');
});
it('still allows the operations that need no recipient', async () => {
menu(['list', 'decrypt', 'retire']);
await keyman();
expect(listKeys).toHaveBeenCalled();
expect(decryptKeys).toHaveBeenCalled();
expect(retireKey).toHaveBeenCalled();
});
it('retries the lookup, so creating the identity mid-session works', async () => {
extractAgePublicKey.mockResolvedValueOnce(null).mockResolvedValueOnce('age1later');
menu(['generate', 'generate']);
await keyman();
expect(extractAgePublicKey).toHaveBeenCalledTimes(2);
expect(generateKey).toHaveBeenCalledTimes(1);
expect(generateKey).toHaveBeenCalledWith(paths.tmpDir, paths.keysDir, 'age1later');
});
});
it('resolves the recipient once for repeated operations', async () => {
menu(['generate', 'encrypt']);
await keyman();
expect(extractAgePublicKey).toHaveBeenCalledTimes(1);
});
it('keeps showing the menu until the user quits', async () => {
menu(['list', 'copy', 'list']);
@@ -196,21 +294,55 @@ describe('keyman', () => {
});
it('targets another user home directory when a user is named', async () => {
// A real sibling of the current HOME, because resolveHomeDir checks that the
// directory exists rather than assuming a layout.
const deployHome = path.join(root, 'deploy');
fs.mkdirSync(deployHome, { recursive: true });
menu(['list'], 'deploy');
await keyman();
expect(listKeys).toHaveBeenCalledWith('/home/deploy/.ssh', paths.keysDir, paths.tmpDir);
expect(listKeys).toHaveBeenCalledWith(
path.join(deployHome, '.ssh'),
paths.keysDir,
paths.tmpDir
);
});
it('aborts when the home directory cannot be determined', async () => {
delete process.env.HOME;
it('aborts when the named user has no home directory', async () => {
menu(['list'], 'nobody-at-all');
const exit = vi.spyOn(process, 'exit').mockImplementation(() => {
throw new Error('process.exit');
});
await expect(keyman()).rejects.toThrow('process.exit');
expect(exit).toHaveBeenCalledWith(1);
expect(errorSpy.mock.calls[0][0]).toContain('Unable to determine HOME directory');
expect(errorSpy.mock.calls[0][0]).toContain('No home directory found');
});
it('writes a .gitignore next to the vault so it cannot be committed', async () => {
await keyman();
const contents = fs.readFileSync(path.join(paths.vaultRoot, '.gitignore'), 'utf-8');
// The README used to ask the user to do this by hand.
expect(contents).toContain('age.key');
expect(contents).toContain('tmp/');
});
it('clears the decrypted keys on request', async () => {
fs.mkdirSync(paths.tmpDir, { recursive: true });
fs.writeFileSync(path.join(paths.tmpDir, 'id_prod'), 'PRIVATE');
// Not the `menu` helper: this one has to answer the confirmation too.
const queue = ['clear', 'quit'];
prompt.mockImplementation(async (questions: { name: string }[]) => {
const { name } = questions[0];
if (name === 'user') return { user: '@current' };
if (name === 'confirmed') return { confirmed: true };
return { category: queue.shift() };
});
await keyman();
expect(fs.existsSync(path.join(paths.tmpDir, 'id_prod'))).toBe(false);
});
});
+47
View File
@@ -0,0 +1,47 @@
/**
* The README is the only document that ships (`package.json` files: dist,
* README.md, LICENSE), so a reader on the registry sees it and nothing else. It
* had drifted to describing four of the menu's entries and none of the command
* line; these assertions are the parts that can drift again silently.
*/
import fs from 'node:fs';
import path from 'node:path';
import { describe, expect, it } from 'vitest';
import { helpText } from '../src/keyman.args.js';
const README = fs.readFileSync(path.join(import.meta.dirname, '..', 'README.md'), 'utf-8');
describe('README', () => {
it('quotes --help verbatim', () => {
// Copied rather than described, and checked rather than trusted: a flag added
// to helpText() now fails here instead of shipping undocumented.
expect(README).toContain(helpText().trim());
});
it('documents every menu operation', () => {
const main = fs.readFileSync(
path.join(import.meta.dirname, '..', 'src', 'keyman.main.ts'),
'utf-8'
);
const labels = [...main.matchAll(/\{ name: '([^']+)', value: '[a-z]+' \}/g)].map((m) => m[1]);
// The labels themselves, emoji included, so a renamed entry is caught too —
// but with runs of whitespace collapsed, because some of them carry a second
// space to align a variation-selector emoji in a terminal, and prose should
// not have to reproduce that.
const collapse = (text: string) => text.replace(/\s+/g, ' ');
const readme = collapse(README);
expect(labels.length).toBe(9);
for (const label of labels) {
expect(readme, label).toContain(collapse(label));
}
});
it('documents every configuration key', () => {
for (const key of ['vaultRoot', 'keysDir', 'tmpDir', 'ageKeyFile']) {
expect(README, key).toContain(key);
}
});
});
+471
View File
@@ -0,0 +1,471 @@
/**
* Tests for rotation and retirement.
*
* ssh-keygen and age are mocked; the ssh-keygen stand-in writes the pair the real
* binary would, so the vault write is a real one. Everything the operations claim
* about the filesystem is asserted against the filesystem.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const { execa, prompt } = vi.hoisted(() => ({ execa: vi.fn(), prompt: vi.fn() }));
vi.mock('execa', () => ({ execa }));
vi.mock('inquirer', () => ({ default: { prompt } }));
import { nextRotationName, retireKey, rotateKey, supersededBy } from '../src/keyman.rotate.js';
describe('nextRotationName', () => {
it('starts a series at 2, so the first key keeps its plain name', () => {
expect(nextRotationName('prod', ['prod'])).toBe('prod-2');
});
it('continues an existing series', () => {
expect(nextRotationName('prod-2', ['prod', 'prod-2'])).toBe('prod-3');
});
it('skips past a version that already exists', () => {
// Rotating the original again after prod-2 and prod-3 exist: -2 is taken.
expect(nextRotationName('prod', ['prod', 'prod-2', 'prod-3'])).toBe('prod-4');
});
it('ignores other series', () => {
expect(nextRotationName('prod', ['prod', 'stage-7', 'prod-backup'])).toBe('prod-2');
});
it('treats a name that ends in a number as its own series', () => {
// `web2` is a host name, not a version — the separator is what makes a series.
expect(nextRotationName('web2', ['web2'])).toBe('web2-2');
});
it('keeps a hyphenated base intact', () => {
expect(nextRotationName('build-agent', ['build-agent'])).toBe('build-agent-2');
expect(nextRotationName('build-agent-2', ['build-agent-2'])).toBe('build-agent-3');
});
});
describe('supersededBy', () => {
it('finds the replacement of a key', () => {
expect(supersededBy('prod', ['prod', 'prod-2'])).toBe('prod-2');
});
it('answers with the latest one', () => {
expect(supersededBy('prod', ['prod', 'prod-2', 'prod-3'])).toBe('prod-3');
});
it('says nothing supersedes the newest key in a series', () => {
expect(supersededBy('prod-3', ['prod', 'prod-2', 'prod-3'])).toBeNull();
});
it('does not count an unrelated key', () => {
expect(supersededBy('prod', ['prod', 'stage-9'])).toBeNull();
});
});
describe('rotateKey', () => {
let root: string;
let sshDir: string;
let keysDir: string;
let tmpDir: string;
let logSpy: ReturnType<typeof vi.spyOn>;
let errorSpy: ReturnType<typeof vi.spyOn>;
const PUBKEY = 'age1recipient';
/** Creates <keysDir>/<name>/id_<name>.{age,pub}. */
const vaultKey = (name: string, comment = 'me@host') => {
const dir = path.join(keysDir, name);
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, `id_${name}.age`), `ENCRYPTED ${name}`);
fs.writeFileSync(path.join(dir, `id_${name}.pub`), `ssh-ed25519 AAAA${name} ${comment}\n`);
};
/** Answers each prompt by the name of the question it asks. */
const answer = (answers: Record<string, unknown>) => {
prompt.mockImplementation(async (questions: { name: string }[]) => {
const { name } = questions[0];
return { [name]: answers[name] };
});
};
const question = (name: string) =>
prompt.mock.calls.map((c) => c[0][0]).find((q) => q.name === name);
const argsOf = (binary: string) =>
execa.mock.calls.find((c) => c[0] === binary)?.[1] as string[] | undefined;
const messages = (spy: ReturnType<typeof vi.spyOn>) =>
spy.mock.calls.map((c) => c.join(' ')).join('\n');
beforeEach(() => {
vi.clearAllMocks();
root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-rotate-')));
sshDir = path.join(root, '.ssh');
keysDir = path.join(root, 'vault', 'keys');
tmpDir = path.join(root, 'vault', 'tmp');
fs.mkdirSync(keysDir, { recursive: true });
fs.mkdirSync(sshDir, { recursive: true });
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
execa.mockImplementation(async (binary: string, args: string[]) => {
if (binary === 'ssh-keygen') {
const keyPath = args[args.indexOf('-f') + 1];
fs.writeFileSync(keyPath, 'PRIVATE');
fs.writeFileSync(`${keyPath}.pub`, `ssh-ed25519 NEWKEY ${args[args.indexOf('-C') + 1]}\n`);
}
if (binary === 'age') {
// Written, not just recorded: what the vault ends up holding is the thing
// under test, and a later listing has to see the new entry.
fs.writeFileSync(args[args.indexOf('-o') + 1], 'ENCRYPTED');
}
return { exitCode: 0 };
});
answer({ key: 'prod', algorithm: 'ed25519', identity: 'me@host' });
});
afterEach(() => {
vi.restoreAllMocks();
fs.rmSync(root, { recursive: true, force: true });
});
it('says there is nothing to rotate on an empty vault', async () => {
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(messages(logSpy)).toContain('No encrypted keys to rotate');
expect(prompt).not.toHaveBeenCalled();
});
it('offers the vault keys', async () => {
vaultKey('stage');
vaultKey('prod');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(question('key').choices).toEqual(['prod', 'stage']);
});
it('generates the replacement into the tmp directory under the next name', async () => {
vaultKey('prod');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(argsOf('ssh-keygen')).toEqual([
'-t',
'ed25519',
'-f',
path.join(tmpDir, 'id_prod-2'),
'-C',
'me@host',
]);
});
it('leaves the rotated key untouched in the vault', async () => {
vaultKey('prod');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
// The whole point of rotating this way: both keys are in the vault, and the
// one that is deployed is byte for byte what it was.
expect(fs.readFileSync(path.join(keysDir, 'prod', 'id_prod.age'), 'utf-8')).toBe(
'ENCRYPTED prod'
);
expect(fs.existsSync(path.join(keysDir, 'prod-2', 'id_prod-2.age'))).toBe(true);
});
it('encrypts the replacement to the vault recipient', async () => {
vaultKey('prod');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(argsOf('age')).toEqual([
'-r',
PUBKEY,
'-o',
path.join(keysDir, 'prod-2', 'id_prod-2.age'),
path.join(tmpDir, 'id_prod-2'),
]);
});
it('offers the comment of the key being replaced', async () => {
vaultKey('prod', 'deploy@prod');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(question('identity').default).toBe('deploy@prod');
});
it('offers no comment when the stored public key has none', async () => {
vaultKey('prod');
fs.writeFileSync(path.join(keysDir, 'prod', 'id_prod.pub'), 'ssh-ed25519 AAAAprod\n');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(question('identity').default).toBeUndefined();
});
it('rotates a key whose public half was never stored', async () => {
vaultKey('prod');
fs.rmSync(path.join(keysDir, 'prod', 'id_prod.pub'));
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(question('identity').default).toBeUndefined();
// Reported rather than printed as a blank line, since the user needs it to
// know what to remove from the host afterwards.
expect(messages(logSpy)).toContain('none stored at');
expect(fs.existsSync(path.join(keysDir, 'prod-2', 'id_prod-2.age'))).toBe(true);
});
it('prints both public keys and what to do with them', async () => {
vaultKey('prod', 'deploy@prod');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
const output = messages(logSpy);
expect(output).toContain('ssh-ed25519 AAAAprod deploy@prod');
expect(output).toContain('ssh-ed25519 NEWKEY me@host');
// Deploy-then-retire, in that order: the reverse locks you out.
expect(output).toContain('Add the replacement public key');
expect(output).toContain('retire');
});
it('skips a name taken by a plaintext key outside the vault', async () => {
vaultKey('prod');
fs.writeFileSync(path.join(sshDir, 'id_prod-2'), 'PRIVATE');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
// Generating id_prod-2 would have refused, or worse asked ssh-keygen to
// overwrite a private key that is in use.
expect(argsOf('ssh-keygen')).toContain(path.join(tmpDir, 'id_prod-3'));
expect(fs.readFileSync(path.join(sshDir, 'id_prod-2'), 'utf-8')).toBe('PRIVATE');
});
it('skips a name taken by an earlier rotation still in tmp', async () => {
vaultKey('prod');
fs.mkdirSync(tmpDir, { recursive: true });
fs.writeFileSync(path.join(tmpDir, 'id_prod-2'), 'PRIVATE');
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(argsOf('ssh-keygen')).toContain(path.join(tmpDir, 'id_prod-3'));
});
it('requests a 4096 bit key for rsa', async () => {
vaultKey('prod');
answer({ key: 'prod', algorithm: 'rsa', identity: '' });
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(argsOf('ssh-keygen')?.slice(-2)).toEqual(['-b', '4096']);
});
it('stops at a failure from ssh-keygen without touching the vault', async () => {
vaultKey('prod');
execa.mockImplementation(async () => {
throw Object.assign(new Error('ssh-keygen exploded'), { stderr: 'ssh-keygen exploded' });
});
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(messages(errorSpy)).toContain('Error generating key');
expect(fs.existsSync(path.join(keysDir, 'prod-2'))).toBe(false);
});
it('reports a failure from age and says where the replacement is', async () => {
vaultKey('prod');
execa.mockImplementation(async (binary: string, args: string[]) => {
if (binary === 'age') {
throw Object.assign(new Error('age exploded'), { stderr: 'age exploded' });
}
const keyPath = args[args.indexOf('-f') + 1];
fs.writeFileSync(keyPath, 'PRIVATE');
fs.writeFileSync(`${keyPath}.pub`, 'ssh-ed25519 NEWKEY me@host\n');
return { exitCode: 0 };
});
await rotateKey(sshDir, keysDir, tmpDir, PUBKEY);
expect(messages(errorSpy)).toContain('Error encrypting the replacement');
expect(messages(errorSpy)).toContain(path.join(tmpDir, 'id_prod-2'));
// No summary: nothing was stored, so there is nothing to deploy yet.
expect(messages(logSpy)).not.toContain('Add the replacement public key');
});
});
describe('retireKey', () => {
let root: string;
let sshDir: string;
let keysDir: string;
let tmpDir: string;
let logSpy: ReturnType<typeof vi.spyOn>;
const vaultKey = (name: string) => {
const dir = path.join(keysDir, name);
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, `id_${name}.age`), `ENCRYPTED ${name}`);
fs.writeFileSync(path.join(dir, `id_${name}.pub`), `PUBLIC ${name}`);
};
const answer = (answers: Record<string, unknown>) => {
prompt.mockImplementation(async (questions: { name: string }[]) => {
const { name } = questions[0];
return { [name]: answers[name] };
});
};
const question = (name: string) =>
prompt.mock.calls.map((c) => c[0][0]).find((q) => q.name === name);
const messages = () => logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
beforeEach(() => {
vi.clearAllMocks();
root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-retire-')));
sshDir = path.join(root, '.ssh');
keysDir = path.join(root, 'vault', 'keys');
tmpDir = path.join(root, 'vault', 'tmp');
fs.mkdirSync(keysDir, { recursive: true });
fs.mkdirSync(sshDir, { recursive: true });
fs.mkdirSync(tmpDir, { recursive: true });
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
answer({ key: 'prod', confirmed: true, typed: 'prod' });
});
afterEach(() => {
vi.restoreAllMocks();
fs.rmSync(root, { recursive: true, force: true });
});
it('says there is nothing to retire on an empty vault', async () => {
await retireKey(sshDir, keysDir, tmpDir);
expect(messages()).toContain('No encrypted keys in the vault');
expect(prompt).not.toHaveBeenCalled();
});
it('removes the vault entry and its directory', async () => {
vaultKey('prod');
vaultKey('prod-2');
await retireKey(sshDir, keysDir, tmpDir);
expect(fs.existsSync(path.join(keysDir, 'prod'))).toBe(false);
// Only the one that was named.
expect(fs.existsSync(path.join(keysDir, 'prod-2', 'id_prod-2.age'))).toBe(true);
});
it('removes the plaintext copies as well', async () => {
vaultKey('prod');
vaultKey('prod-2');
for (const dir of [sshDir, tmpDir]) {
fs.writeFileSync(path.join(dir, 'id_prod'), 'PRIVATE');
fs.writeFileSync(path.join(dir, 'id_prod.pub'), 'PUBLIC');
}
await retireKey(sshDir, keysDir, tmpDir);
expect(fs.readdirSync(sshDir)).toEqual([]);
expect(fs.readdirSync(tmpDir)).toEqual([]);
});
it('lists every path before asking, and asks with a no default', async () => {
vaultKey('prod');
vaultKey('prod-2');
fs.writeFileSync(path.join(sshDir, 'id_prod'), 'PRIVATE');
await retireKey(sshDir, keysDir, tmpDir);
expect(messages()).toContain(path.join(keysDir, 'prod', 'id_prod.age'));
expect(messages()).toContain(path.join(sshDir, 'id_prod'));
expect(question('confirmed')).toMatchObject({ type: 'confirm', default: false });
expect(question('confirmed').message).toContain('3 files');
});
it('counts one file as one file', async () => {
vaultKey('prod-2');
fs.mkdirSync(path.join(keysDir, 'prod'));
fs.writeFileSync(path.join(keysDir, 'prod', 'id_prod.age'), 'ENCRYPTED');
await retireKey(sshDir, keysDir, tmpDir);
expect(question('confirmed').message).toContain('1 file?');
expect(fs.existsSync(path.join(keysDir, 'prod'))).toBe(false);
});
it('says what supersedes the key it is about to delete', async () => {
vaultKey('prod');
vaultKey('prod-2');
await retireKey(sshDir, keysDir, tmpDir);
expect(messages()).toContain('prod-2 is in the vault and supersedes prod');
expect(question('typed')).toBeUndefined();
});
it('keeps everything when the confirmation is declined', async () => {
vaultKey('prod');
vaultKey('prod-2');
answer({ key: 'prod', confirmed: false });
await retireKey(sshDir, keysDir, tmpDir);
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.age'))).toBe(true);
expect(messages()).toContain('Nothing was deleted');
});
describe('a key nothing replaces', () => {
beforeEach(() => {
vaultKey('prod');
});
it('warns that this is the only copy', async () => {
await retireKey(sshDir, keysDir, tmpDir);
expect(messages()).toContain('Nothing in the vault supersedes prod');
});
it('asks for the name to be typed out, and deletes when it matches', async () => {
await retireKey(sshDir, keysDir, tmpDir);
// A y/n is one keystroke from an irreversible deletion; this is not.
expect(question('typed')).toBeDefined();
expect(fs.existsSync(path.join(keysDir, 'prod'))).toBe(false);
});
it('deletes nothing when the typed name does not match', async () => {
answer({ key: 'prod', confirmed: true, typed: 'prodd' });
await retireKey(sshDir, keysDir, tmpDir);
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.age'))).toBe(true);
expect(messages()).toContain('Name did not match');
});
it('accepts the name with stray whitespace', async () => {
answer({ key: 'prod', confirmed: true, typed: ' prod ' });
await retireKey(sshDir, keysDir, tmpDir);
expect(fs.existsSync(path.join(keysDir, 'prod'))).toBe(false);
});
});
it('keeps a vault directory that holds something else', async () => {
vaultKey('prod');
vaultKey('prod-2');
fs.mkdirSync(path.join(keysDir, 'prod', 'notes'));
await retireKey(sshDir, keysDir, tmpDir);
// The .age and .pub are gone; the directory stays, and says why.
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.age'))).toBe(false);
expect(fs.existsSync(path.join(keysDir, 'prod', 'notes'))).toBe(true);
expect(messages()).toContain('it still holds other files');
});
});
+56
View File
@@ -0,0 +1,56 @@
/**
* Tests for runTool.
*
* These spawn real processes rather than mocking execa. What runTool exists for
* is the shape of an execa failure — a mock would assert only what this test
* already assumes. It lives apart from utils.test.ts, which mocks execa to test
* the callers.
*/
import { describe, expect, it } from 'vitest';
import { runTool, ToolNotFoundError } from '../src/keyman.utils.js';
describe('runTool', () => {
it('returns stdout on success', async () => {
const result = await runTool('node', ['-e', 'process.stdout.write("hi")']);
expect(result.stdout).toBe('hi');
});
it('passes options through', async () => {
const result = await runTool('node', ['-e', 'process.stdout.write(process.env.PROBE ?? "")'], {
env: { PROBE: 'from-options' },
});
expect(result.stdout).toBe('from-options');
});
it('reports empty stdout when the output went elsewhere', async () => {
const result = await runTool('node', ['-e', 'process.stdout.write("hi")'], {
stdout: 'ignore',
});
expect(result.stdout).toBe('');
});
it('turns a missing binary into an instruction rather than an ENOENT', async () => {
const failure = runTool('keyman-no-such-binary', []);
await expect(failure).rejects.toThrow(ToolNotFoundError);
await expect(failure).rejects.toThrow(
'`keyman-no-such-binary` was not found on PATH. Install it and try again.'
);
});
it('surfaces what the binary wrote to stderr', async () => {
await expect(
runTool('node', ['-e', 'process.stderr.write("no recipient\\n"); process.exit(1)'])
).rejects.toThrow('`node` failed: no recipient');
});
it('falls back to the command summary when stderr is empty', async () => {
await expect(runTool('node', ['-e', 'process.exit(3)'])).rejects.toThrow(
/`node` failed: .*exit code 3/
);
});
});
+90 -22
View File
@@ -1,19 +1,30 @@
/**
* Tests for extractAgePublicKey.
*
* Runs against real files in a temp directory: the function is a thin wrapper
* around fs plus a regex, and faking fs would only test the fake.
* Real files in a temp directory, but a mocked execa: the recipient is now
* derived by spawning `age-keygen -y`, and the gate cannot depend on age being
* installed on the machine running it. runTool itself is tested against real
* processes in tool.test.ts.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const { execa } = vi.hoisted(() => ({ execa: vi.fn() }));
vi.mock('execa', () => ({ execa }));
import { extractAgePublicKey } from '../src/keyman.utils.js';
const DERIVED = 'age1derivedfromthesecretkey';
const IN_COMMENT = 'age1fromthecomment';
describe('extractAgePublicKey', () => {
let tmpDir: string;
let errorSpy: ReturnType<typeof vi.spyOn>;
let warnSpy: ReturnType<typeof vi.spyOn>;
const keyFile = (contents: string) => {
const file = path.join(tmpDir, 'age.key');
@@ -21,9 +32,33 @@ describe('extractAgePublicKey', () => {
return file;
};
/** A well-formed identity file, whose comment can be made to disagree */
const identity = (comment = DERIVED) =>
keyFile(
['# created: 2026-01-01T00:00:00Z', `# public key: ${comment}`, 'AGE-SECRET-KEY-1QQQ'].join(
'\n'
)
);
/**
* Makes age-keygen unavailable, the one case that falls back to the comment.
*
* Throws from an implementation rather than using mockRejectedValue: that
* builds its rejected promise when the mock is configured, and configuring it
* in a beforeEach leaves the rejection unhandled for a tick.
*/
const noAgeKeygen = () => {
execa.mockImplementation(async () => {
throw Object.assign(new Error('spawn age-keygen ENOENT'), { code: 'ENOENT' });
});
};
beforeEach(() => {
vi.clearAllMocks();
tmpDir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-utils-')));
errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
execa.mockResolvedValue({ stdout: `${DERIVED}\n` });
});
afterEach(() => {
@@ -31,49 +66,82 @@ describe('extractAgePublicKey', () => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('returns the public key from a standard age key file', () => {
const file = keyFile(
[
'# created: 2026-01-01T00:00:00Z',
'# public key: age1abc123xyz',
'AGE-SECRET-KEY-1QQQ',
].join('\n')
);
it('derives the recipient from the secret key with age-keygen', async () => {
const file = identity();
expect(extractAgePublicKey(file)).toBe('age1abc123xyz');
await expect(extractAgePublicKey(file)).resolves.toBe(DERIVED);
expect(execa).toHaveBeenCalledWith('age-keygen', ['-y', file]);
expect(warnSpy).not.toHaveBeenCalled();
});
it('tolerates extra whitespace after the label', () => {
const file = keyFile('# public key: age1spaced\n');
it('prefers the derived key over a comment that disagrees', async () => {
// §2.3: the comment is editable text, and this is what makes it not matter.
const file = identity('age1staleorforged');
expect(extractAgePublicKey(file)).toBe('age1spaced');
await expect(extractAgePublicKey(file)).resolves.toBe(DERIVED);
});
it('returns null and reports when the file does not exist', () => {
it('returns null and reports when the file does not exist', async () => {
const missing = path.join(tmpDir, 'nope.key');
expect(extractAgePublicKey(missing)).toBeNull();
await expect(extractAgePublicKey(missing)).resolves.toBeNull();
expect(errorSpy.mock.calls[0][0]).toContain('Age key file not found');
expect(execa).not.toHaveBeenCalled();
});
it('returns null when the file has no public key line', () => {
it('returns null when age-keygen refuses the file, without trusting the comment', async () => {
const file = identity(IN_COMMENT);
execa.mockRejectedValue(
Object.assign(new Error('failed'), { exitCode: 1, stderr: 'age-keygen: error: malformed' })
);
await expect(extractAgePublicKey(file)).resolves.toBeNull();
expect(errorSpy.mock.calls[0][0]).toContain('malformed');
});
it('returns null when age-keygen prints something that is not a recipient', async () => {
const file = identity();
execa.mockResolvedValue({ stdout: 'Public key: (none)\n' });
await expect(extractAgePublicKey(file)).resolves.toBeNull();
expect(errorSpy.mock.calls[0][0]).toContain('derived no public key');
});
describe('without age-keygen installed', () => {
beforeEach(noAgeKeygen);
it('falls back to the comment, warning that it is unverified', async () => {
const file = identity(IN_COMMENT);
await expect(extractAgePublicKey(file)).resolves.toBe(IN_COMMENT);
expect(warnSpy.mock.calls[0][0]).toContain('unverified');
});
it('tolerates extra whitespace after the label', async () => {
const file = keyFile('# public key: age1spaced\n');
await expect(extractAgePublicKey(file)).resolves.toBe('age1spaced');
});
it('returns null when the file has no public key line', async () => {
const file = keyFile('AGE-SECRET-KEY-1QQQ\n');
expect(extractAgePublicKey(file)).toBeNull();
await expect(extractAgePublicKey(file)).resolves.toBeNull();
expect(errorSpy).not.toHaveBeenCalled();
});
it('ignores a key that is not on its own line', () => {
it('ignores a key that is not on its own line', async () => {
const file = keyFile('prefix # public key: age1inline\n');
expect(extractAgePublicKey(file)).toBeNull();
await expect(extractAgePublicKey(file)).resolves.toBeNull();
});
it('returns null and reports when the file cannot be read', () => {
it('returns null and reports when the file cannot be read', async () => {
const asDirectory = path.join(tmpDir, 'age.key');
fs.mkdirSync(asDirectory);
expect(extractAgePublicKey(asDirectory)).toBeNull();
await expect(extractAgePublicKey(asDirectory)).resolves.toBeNull();
expect(errorSpy.mock.calls[0][0]).toContain('Failed to read key file');
});
});
});
+127
View File
@@ -0,0 +1,127 @@
/**
* End-to-end over the configured vault layout.
*
* Everything except `age` and the prompts is real here — the config loader, the
* path resolution, encrypt and list all run — because the bug this covers lived
* in the seam between them: encrypt wrote to `vaultRoot` while list read from
* `keysDir`, so with the defaults (`keysDir: 'keys'`) an encrypted key was
* invisible to the very next listing. Every unit suite passed throughout, since
* each was told which directory to use.
*
* Non-default names on purpose: `keys`/`tmp` would also pass against a function
* that ignored the config entirely.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const { execa, prompt } = vi.hoisted(() => ({ execa: vi.fn(), prompt: vi.fn() }));
vi.mock('execa', () => ({ execa }));
vi.mock('inquirer', () => ({ default: { prompt } }));
import { keyman } from '../src/keyman.main.js';
describe('the configured vault layout', () => {
let root: string;
let project: string;
let home: string;
let sshDir: string;
let vaultRoot: string;
let cwd: string;
let env: NodeJS.ProcessEnv;
let logSpy: ReturnType<typeof vi.spyOn>;
const output = () => logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
/** The one line of the listing table describing `name`. */
const listingRow = (name: string) =>
output()
.split('\n')
.find((line) => line.includes(name) && line.includes('['));
beforeEach(() => {
vi.clearAllMocks();
cwd = process.cwd();
env = { ...process.env };
root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-layout-')));
project = path.join(root, 'project');
home = path.join(root, 'home');
sshDir = path.join(home, '.ssh');
vaultRoot = path.join(project, 'vault');
fs.mkdirSync(project, { recursive: true });
fs.mkdirSync(sshDir, { recursive: true });
fs.writeFileSync(path.join(sshDir, 'id_prod'), 'PRIVATE');
fs.writeFileSync(path.join(sshDir, 'id_prod.pub'), 'PUBLIC');
fs.writeFileSync(
path.join(project, '.keymanrc.json'),
JSON.stringify({ vaultRoot: 'vault', keysDir: 'encrypted', tmpDir: 'plain' })
);
// HOME also redirects os.homedir(), so the real ~/.keymanrc.json cannot
// reach the loader and make this test depend on the machine it runs on.
process.env.HOME = home;
delete process.env.VAULT_ROOT;
process.chdir(project);
// The age identity has to exist before extractAgePublicKey will shell out.
fs.mkdirSync(vaultRoot, { recursive: true });
fs.writeFileSync(path.join(vaultRoot, 'age.key'), 'AGE-SECRET-KEY-1');
execa.mockImplementation(async (binary: string, args: string[]) => {
if (binary === 'age-keygen') {
return { stdout: 'age1recipient' };
}
fs.writeFileSync(args[args.indexOf('-o') + 1], 'ENCRYPTED');
return { stdout: '' };
});
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
vi.spyOn(console, 'error').mockImplementation(() => {});
});
afterEach(() => {
vi.restoreAllMocks();
process.chdir(cwd);
process.env = env;
fs.rmSync(root, { recursive: true, force: true });
});
/** Walks the menu, answering encrypt's key selection along the way. */
const run = (categories: string[]) => {
const queue = [...categories, 'quit'];
prompt.mockImplementation(async (questions: { name: string }[]) => {
switch (questions[0].name) {
case 'user':
return { user: '@current' };
case 'selectedKeys':
return { selectedKeys: ['id_prod'] };
default:
return { category: queue.shift() };
}
});
return keyman();
};
it('encrypts into the configured keys directory, where the listing looks', async () => {
await run(['encrypt', 'list']);
expect(fs.existsSync(path.join(vaultRoot, 'encrypted', 'prod', 'id_prod.age'))).toBe(true);
// ✅ is reachable only via inVault && inSsh, and the columns are
// [vault] [tmp] [.ssh] — either alone would pass on a blank vault column.
expect(listingRow('id_prod')).toContain('✅');
expect(listingRow('id_prod')).toMatch(/\[✓]\s+\[ ]\s+\[✓]/);
});
it('honours the configured directory names for every path it prints', async () => {
await run([]);
expect(output()).toContain(path.join(vaultRoot, 'encrypted'));
expect(output()).toContain(path.join(vaultRoot, 'plain'));
});
});
+167
View File
@@ -0,0 +1,167 @@
/**
* Tests for storeInVault, the write path encrypt and generate share.
*
* Its ordinary use is covered through those two callers; what is here is the
* behaviour that is awkward to reach from either — an ssh-keygen that succeeds
* without printing anything, and a failure over an entry that already exists.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const { execa } = vi.hoisted(() => ({ execa: vi.fn() }));
vi.mock('execa', () => ({ execa }));
import { listVaultKeys, storeInVault } from '../src/keyman.vault.js';
describe('listVaultKeys', () => {
let keysDir: string;
const entry = (name: string, file = `id_${name}.age`) => {
fs.mkdirSync(path.join(keysDir, name), { recursive: true });
fs.writeFileSync(path.join(keysDir, name, file), 'ENCRYPTED');
};
beforeEach(() => {
keysDir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-vaultlist-')));
});
afterEach(() => {
fs.rmSync(keysDir, { recursive: true, force: true });
});
it('is empty for a keys directory that was never created', () => {
expect(listVaultKeys(path.join(keysDir, 'nope'))).toEqual([]);
});
it('sorts the entries rather than taking the filesystem order', () => {
for (const name of ['stage', 'alpha', 'prod']) {
entry(name);
}
expect(listVaultKeys(keysDir)).toEqual(['alpha', 'prod', 'stage']);
});
it('ignores a directory with no encrypted key in it', () => {
entry('prod');
// The shape a failed encryption used to leave behind, and a plain mistake.
fs.mkdirSync(path.join(keysDir, 'empty'));
entry('notes', 'README.md');
expect(listVaultKeys(keysDir)).toEqual(['prod']);
});
it('ignores a loose file', () => {
entry('prod');
fs.writeFileSync(path.join(keysDir, 'id_stage.age'), 'ENCRYPTED');
expect(listVaultKeys(keysDir)).toEqual(['prod']);
});
});
describe('storeInVault', () => {
let root: string;
let keysDir: string;
let keyPath: string;
let logSpy: ReturnType<typeof vi.spyOn>;
let warnSpy: ReturnType<typeof vi.spyOn>;
const PUBKEY = 'age1recipient';
const messages = (spy: ReturnType<typeof vi.spyOn>) =>
spy.mock.calls.map((c) => c.join(' ')).join('\n');
beforeEach(() => {
vi.clearAllMocks();
root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'keyman-vault-')));
keysDir = path.join(root, 'keys');
keyPath = path.join(root, 'id_prod');
fs.writeFileSync(keyPath, 'PRIVATE');
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
});
afterEach(() => {
vi.restoreAllMocks();
fs.rmSync(root, { recursive: true, force: true });
});
it('warns when ssh-keygen succeeds but prints no key', async () => {
execa.mockImplementation(async (binary: string, args: string[]) => {
if (binary === 'ssh-keygen') return { stdout: ' \n' };
fs.writeFileSync(args[args.indexOf('-o') + 1], 'ENCRYPTED');
return { stdout: '' };
});
await storeInVault(keyPath, keysDir, PUBKEY);
// An exit code of 0 is not a public key: writing a .pub holding whitespace
// would put a file in the vault that no host would ever accept.
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.pub'))).toBe(false);
expect(messages(warnSpy)).toContain('no public key could be derived');
expect(fs.existsSync(path.join(keysDir, 'prod', 'id_prod.age'))).toBe(true);
});
it('writes the public half at the same time as the encrypted key', async () => {
fs.writeFileSync(`${keyPath}.pub`, 'ssh-ed25519 AAAA sibling');
execa.mockImplementation(async (_binary: string, args: string[]) => {
fs.writeFileSync(args[args.indexOf('-o') + 1], 'ENCRYPTED');
return { stdout: '' };
});
const vaultPath = await storeInVault(keyPath, keysDir, PUBKEY);
expect(vaultPath).toBe(path.join(keysDir, 'prod'));
expect(fs.readFileSync(path.join(vaultPath, 'id_prod.pub'), 'utf-8')).toBe(
'ssh-ed25519 AAAA sibling'
);
// No ssh-keygen: the sibling was there, so nothing needed deriving.
expect(execa.mock.calls.every((c) => c[0] === 'age')).toBe(true);
});
it('creates the vault entry private to the owner', async () => {
fs.writeFileSync(`${keyPath}.pub`, 'PUBLIC');
execa.mockImplementation(async (_binary: string, args: string[]) => {
fs.writeFileSync(args[args.indexOf('-o') + 1], 'ENCRYPTED');
return { stdout: '' };
});
await storeInVault(keyPath, keysDir, PUBKEY);
expect(fs.statSync(path.join(keysDir, 'prod')).mode & 0o777).toBe(0o700);
});
describe('when age fails', () => {
beforeEach(() => {
fs.writeFileSync(`${keyPath}.pub`, 'PUBLIC');
execa.mockImplementation(async (_binary: string, args: string[]) => {
// Half-written output, the way a failing age can leave it.
fs.writeFileSync(args[args.indexOf('-o') + 1], 'TRUNC');
throw Object.assign(new Error('age refused'), { stderr: 'no recipient' });
});
});
it('leaves no truncated key behind for decrypt to offer', async () => {
await expect(storeInVault(keyPath, keysDir, PUBKEY)).rejects.toThrow('`age` failed');
expect(fs.existsSync(path.join(keysDir, 'prod'))).toBe(false);
});
it('keeps an entry that was already there', async () => {
const vaultPath = path.join(keysDir, 'prod');
fs.mkdirSync(vaultPath, { recursive: true });
fs.writeFileSync(path.join(vaultPath, 'id_prod.pub'), 'THE OLD PUBLIC KEY');
await expect(storeInVault(keyPath, keysDir, PUBKEY)).rejects.toThrow('`age` failed');
// Cleaning up after a failure must not take the previous key with it.
expect(fs.readFileSync(path.join(vaultPath, 'id_prod.pub'), 'utf-8')).toBe(
'THE OLD PUBLIC KEY'
);
expect(messages(logSpy)).not.toContain('Encrypted and stored');
});
});
});
+10 -1
View File
@@ -6,9 +6,18 @@ base packages, users, SSH, firewalling, networking, web serving and runtimes.
## Install
```sh
pnpm add -D @bitsquare/nopy-cubes-core
pnpm add -D @bitsquare/nopy-cubes-core@main \
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
```
**Both halves are required today.** This bundle has not been published to npmjs
yet, so it comes from the Gitea registry — and that registry publishes no
`latest` tag, so an *untagged* install resolves to nothing at all. Name `@main`
(a snapshot of every commit) or `@next` (a prerelease) explicitly. Point the
**scope** at Gitea rather than setting a bare `registry=`: Gitea serves
`@bitsquare` only and does not proxy npmjs, so everything else must keep
resolving from there. Reading needs no token while the repository is public.
Then name it in `.nopyrc.json`:
```json
@@ -1,76 +0,0 @@
# cockpit
**Install Cockpit web-based server management interface**
## Purpose
This cube installs Cockpit, a powerful web-based interface for managing Linux servers, making system administration accessible through your browser.
## What is Cockpit?
Cockpit is a modern, interactive server admin interface that runs in your web browser. It provides:
- **Real-time monitoring**: CPU, memory, disk, and network usage graphs
- **Container management**: View and manage Docker containers
- **Service management**: Start, stop, and manage systemd services
- **Storage administration**: Manage disks, RAID, and filesystems
- **Network configuration**: Configure network interfaces and firewall
- **Terminal access**: Built-in terminal for command-line access
- **User management**: Create and manage user accounts
- **Software updates**: View and apply system updates
Think of it as a control panel for your Linux server - all accessible from any web browser.
## What This Cube Does
1. Installs the `cockpit` package
2. Installs `sscg` (Simple Signed Certificate Generator) for HTTPS support
3. Starts the Cockpit service
4. Makes Cockpit accessible on port 9090
## Configuration
This cube currently has no configurable parameters.
## Dependencies
None - this cube can run standalone.
## Post-Installation
Access Cockpit by navigating to:
```
https://your-server-ip:9090
```
Login with any valid system user account (e.g., root or a user created with the `user-add` cube).
### Security Notes
- Cockpit uses HTTPS by default (self-signed certificate)
- Your browser will show a security warning on first access (expected with self-signed certs)
- Consider using UFW to restrict access: `sudo ufw allow from YOUR_IP to any port 9090`
- Disable Cockpit when not in use: `sudo systemctl stop cockpit.socket`
## Common Use Cases
- Monitor server performance in real-time
- Manage Docker containers without command-line
- View system logs and troubleshoot issues
- Configure network settings
- Apply system updates
- Manage storage and filesystems
## Managing Cockpit
Start/stop Cockpit:
```bash
sudo systemctl start cockpit.socket
sudo systemctl stop cockpit.socket
sudo systemctl status cockpit.socket
```
Disable Cockpit from starting on boot:
```bash
sudo systemctl disable cockpit.socket
```
@@ -1,13 +0,0 @@
from pyinfra import host
from pyinfra.operations import server, apt
apt.packages(
packages=[ "sscg cockpit"],
present=True,
_sudo=True
)
server.service(
'cockpit',
running=True,
)
@@ -1,7 +0,0 @@
import { Manifest } from '@bitsquare/nopy-cubes';
export default Manifest({
id: 'admin:cockpit',
name: 'Install cockpit and utils',
dependencies: () => [],
});
@@ -35,7 +35,9 @@ Key benefits:
## Configuration
This cube currently has no configurable parameters.
| Variable | Default | What it does |
| --- | --- | --- |
| `DISTRO` | `ubuntu` | which of Docker's package repositories to add — `ubuntu` or `debian`. It selects the download path and nothing else; the release codename comes from the host's own `/etc/os-release`. |
## Dependencies
@@ -1,69 +1,79 @@
# nodevm
**Install Node.js with essential global packages**
**Install Node.js through nvm, for one user, with global packages**
## Purpose
This cube installs the latest LTS (Long Term Support) version of Node.js along with essential global npm packages commonly needed for development and deployment.
Installs [nvm](https://github.com/nvm-sh/nvm) into a single user's home
directory, uses it to install one pinned Node.js version under an alias, and
installs a list of global npm packages for that user.
## What is Node.js?
Node.js is a JavaScript runtime built on Chrome's V8 engine that allows you to run JavaScript on the server. It's widely used for:
- Building web servers and APIs
- Command-line tools
- Build tools and task runners
- Real-time applications (chat, notifications)
- Microservices
Per-user, not system-wide. Nothing is placed on the system `PATH`, and another
user on the same host is unaffected — which is the point: version pinning belongs
to whoever runs the app.
## What This Cube Does
1. **Installs Node.js LTS**
- Downloads and runs the official NodeSource setup script
- Installs the latest LTS version of Node.js
- Includes npm (Node Package Manager)
2. **Installs build dependencies**
- `libssl-dev` - SSL/TLS libraries
- `libtool` - Library building tools
- `cmake` - Cross-platform build system
- `libpng-dev`, `libjpeg-dev`, `libvips-dev` - Image processing libraries
3. **Installs global npm packages**
- **npm@11.1.0** - Latest npm version
- **pm2** - Production process manager for Node.js apps
- **yarn** - Alternative package manager
- **local-web-server** - Local development web server
- **node-gyp** - Node.js native addon build tool
- **inquirer** - Interactive command-line prompts
- **execa** - Better child process execution
- **@dotenvx/dotenvx** - Environment variable management
1. **Installs build dependencies** with apt, as root — `build-essential`,
`libssl-dev`, `libtool`, `cmake`, and the cairo/pango/png/jpeg/vips/rsvg/pixman
headers that native addons need. The package index is refreshed first: a box
nobody has updated lists .deb versions the mirror has already dropped.
2. **Installs nvm** for `USER` via the official install script, then
`nvm install <VERSION>` and `nvm alias <ALIAS> <VERSION>`.
3. **Installs `GLOBAL_PACKAGES`** with `npm install -g`, as `USER`.
## Configuration
This cube currently has no configurable parameters.
| Variable | Default | What it does |
| --- | --- | --- |
| `VERSION` | `v22.20.0` | the Node.js version nvm installs. A pin, not "latest LTS" — nvm's own version strings work, so `--lts` or `22` are accepted too. |
| `USER` | `vagrant` | the user nvm is installed **for**. Everything lands in that user's `~/.nvm`. |
| `ALIAS` | `nodelts` | the nvm alias pointing at `VERSION`, so later cubes and scripts can say `nvm use nodelts` without knowing the number. |
| `GLOBAL_PACKAGES` | `pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx` | space-separated, passed to one `npm install -g`. Setting it **replaces** the list rather than adding to it. |
| `SHELL` | `fish` | the login shell to install through — `fish` or `bash`. See below. |
### `SHELL`
nvm wires itself into whichever shell installed it, so this is not cosmetic.
- **`fish`** (default) additionally requires **Oh My Fish**, because loading nvm
goes through the `omf install nvm` plugin. `user:add` installs both, which is
the usual way a host arrives here. The cube fails with one line, before
changing anything, if `SHELL=fish` on a host with no fish.
- **`bash`** needs nothing beyond bash. Use it on a host where `user:add` has not
run.
The default stays `fish` so that an existing user — whose login shell `user:add`
set to fish — keeps getting a Node that their shell can actually see. Switching
would install it invisibly.
## Dependencies
None - this cube can run standalone.
None declared: the cube runs standalone. With `SHELL=fish` it does have a real
prerequisite (fish + Oh My Fish, which `user:add` provides), but `user:add` is
deliberately not a declared dependency — it would *create* a user who is normally
meant to already exist. `SHELL=bash` is the standalone path.
## Post-Installation
Verify installation:
`node` is on `USER`'s `PATH` in a login shell, not in root's and not in a
non-interactive one. To check:
```bash
node --version
npm --version
su - <USER> -c 'node --version && npm --version'
```
Common commands:
- Run a Node.js app: `node app.js`
- Start with PM2: `pm2 start app.js`
- Install packages: `npm install <package>`
- Use yarn: `yarn add <package>`
From a bash script that is not a login shell, load nvm first:
## PM2 - Process Manager
```bash
export NVM_DIR="$HOME/.nvm"; . "$NVM_DIR/nvm.sh"
nvm use nodelts
```
PM2 is included for production deployments. Common PM2 commands:
## PM2 — process manager
`pm2` is in the default `GLOBAL_PACKAGES`, so it is installed unless you replaced
the list.
```bash
pm2 start app.js # Start application
@@ -75,9 +85,11 @@ pm2 startup # Enable PM2 on boot
pm2 save # Save current process list
```
`pm2 startup` prints a `sudo` command to run; it does not enable itself.
## Notes
- Node.js is installed system-wide
- Global packages are accessible to all users
- npm cache is stored in `~/.npm`
- Use `nvm` if you need multiple Node.js versions
- Node.js is installed **per user**, under `~/.nvm` for `USER`.
- Global packages belong to that user too, not to everyone on the host.
- Run the cube again with a different `VERSION` and `ALIAS` to have several
versions side by side; nvm is built for exactly that.
@@ -1,13 +1,51 @@
from pyinfra.operations import server, apt, npm, python
from pyinfra.operations import server, apt
from pyinfra import host
from pyinfra.facts.files import Directory
from pyinfra.facts.server import Which
from pyinfra.api.exceptions import DeployError
hasNode = host.get_fact(Which, 'node')
VERSION = host.data.VERSION
ALIAS = host.data.ALIAS
GLOBAL_PACKAGES = host.data.GLOBAL_PACKAGES
USER = host.data.USER
SHELL = host.data.SHELL
INSTALL_NVM = "curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash"
# Under bash, every entry in `commands` is its own shell, so loading nvm and
# using it have to be one entry. Loading cannot be skipped either: nvm's
# installer appends its hook to ~/.bashrc, and Ubuntu's ~/.bashrc returns at
# line 1 for a non-interactive shell, so under `su -c` the hook never runs.
LOAD_NVM = 'export NVM_DIR="$HOME/.nvm"; . "$NVM_DIR/nvm.sh"'
# The binary only. Oh My Fish is a set of fish functions with no binary and no
# fixed path, so probing for it would be guesswork — and if fish is there while
# omf is not, `omf install nvm` says so itself. Half a guard that is certain
# beats a whole one that is not.
if SHELL == 'fish' and not host.get_fact(Which, 'fish'):
raise DeployError(
f'runtime:nodevm: SHELL is "fish" but fish is not installed for {USER}. '
'Run user:add first, or set SHELL=bash.'
)
if SHELL == 'fish':
shell_executable = '/usr/bin/fish'
# The omf plugin defines `nvm` as a fish function that every login shell
# loads, and activates the `default` alias on load — so `nvm` and `npm` are
# both reachable in any later shell without setup, and NVM_DIR is set for us.
nvm_commands = [
INSTALL_NVM,
'omf install nvm',
f'nvm install {VERSION}',
f'nvm alias {ALIAS} {VERSION}',
]
npm_commands = [f'npm install -g {GLOBAL_PACKAGES}']
else:
shell_executable = '/bin/bash'
nvm_commands = [
INSTALL_NVM,
f'{LOAD_NVM}; nvm install {VERSION}; nvm alias {ALIAS} {VERSION}',
]
npm_commands = [f'{LOAD_NVM}; nvm use {ALIAS}; npm install -g {GLOBAL_PACKAGES}']
apt.packages(
name=f'Install nodejs tools',
@@ -26,33 +64,30 @@ apt.packages(
'librsvg2-dev',
'libpixman-1-dev',
],
# Every other cube that installs packages refreshes the index first, and
# this one only got away without it while `user:add` ran ahead of it and
# dragged in `apt:essentials`. On a box nobody has updated, the index
# names .deb versions the mirror has already superseded and the fetch
# 404s — the same "assumes a predecessor cube ran" defect as the shell.
update=True,
_sudo = True,
)
server.shell(
commands=[
"curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash",
"omf install nvm",
f"nvm install {VERSION}",
f"nvm alias {ALIAS} {VERSION}",
"set -gx NVM_DIR $HOME/.nvm",
],
name=f'Install nvm and node {VERSION} for {USER}',
commands=nvm_commands,
_sudo=True,
_su_user=USER,
_use_su_login=True,
_shell_executable='/usr/bin/fish'
_shell_executable=shell_executable,
)
server.shell(
commands=[
"npm install -g pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx"
],
name=f'Install global packages for {USER}',
commands=npm_commands,
_sudo=True,
_su_user=USER,
_use_su_login=True,
_shell_executable='/usr/bin/fish'
_shell_executable=shell_executable,
)
@@ -7,14 +7,24 @@ export default Manifest({
dependencies: () => [],
schema: z.object({
VERSION: z
.nullable(z.string())
.string()
.describe('Node.js version to install. It is recommended to use semver notation')
.default('v22.20.0'),
USER: z.string().describe('Username for which to install nodejs').default('vagrant'),
ALIAS: z.string().describe('The alias for this node version').default('nodelts'),
// The list the deploy script used to hardcode. It is the default rather than
// a constant so that setting the variable adds to nothing and replaces
// everything — which is what "space-separated list" reads as.
GLOBAL_PACKAGES: z
.string()
.describe('Space-separated list of global npm packages to install')
.default('npm-check-updates'),
.default('pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx'),
// fish is the default because nvm wires itself into whichever shell installed
// it: switching would leave an existing user — whose login shell `user:add`
// set to fish — with node installed and invisible.
SHELL: z
.enum(['fish', 'bash'])
.describe('Login shell to install through. fish needs Oh My Fish; bash needs nothing')
.default('fish'),
}),
});
@@ -1,176 +1,61 @@
# TypeStack Install Cube
# autostart
Deploys a Node.js/TypeScript application from a Git repository as a systemd service with Docker Compose and PM2 support.
**Enable and start an existing systemd service**
## Features
## Purpose
- Clones Git repository
- Installs dependencies with Yarn
- Builds the application
- Starts Docker Compose services
- Creates a systemd service for automatic startup
- Configures PM2 for process management
- Automatic restart on failure
Takes a systemd unit that is already installed on the host and decides whether
it runs: `systemctl enable` plus `systemctl start`, or neither.
## Requirements
- Git (for cloning repository)
- Yarn (for dependency management)
- Docker and Docker Compose
- PM2 (for process management)
- Node.js/NVM installed
- SSH key access to the repository (if using private repos)
## Configuration Parameters
### Required
- **USER**: System user to run the application (default: `teclabmin`)
- **REPO**: Git repository URL (default: `git@github.com:bennidi/teclab-flintstone.git`)
- **APP**: Application name/directory name (default: `flintstone`)
### Optional
- **ENV**: Application environment (default: `production`)
- **AUTOSTART**: Enable and start service immediately (default: `True`)
- **NODE_PATH**: Path to Node.js binaries (default: `/home/teclabmin/.nvm/versions/node/v21.7.3/bin`)
## Example Usage
### Basic Configuration
```json
{
"USER": "myuser",
"REPO": "git@github.com:myorg/myapp.git",
"APP": "myapp"
}
```
### Advanced Configuration
```json
{
"USER": "appuser",
"REPO": "git@github.com:myorg/myapp.git",
"APP": "myapp",
"ENV": "staging",
"AUTOSTART": false,
"NODE_PATH": "/home/appuser/.nvm/versions/node/v20.0.0/bin"
}
```
It does **not** create the unit. Something else — a package, another cube, a
`files.template` — has to have put `<APP>.service` on the host first. This cube
is the switch, not the wiring.
## What This Cube Does
1. **Clone Repository**: Clones the specified Git repository to `/home/<USER>/<APP>`
2. **Install Dependencies**: Runs `yarn install` to install all dependencies
3. **Build Application**: Runs `yarn build` to compile the application
4. **Start Docker Services**: Runs `docker compose up -d` to start containerized services
5. **Create Startup Script**: Creates `/home/<USER>/<APP>.service.sh` that:
- Starts Docker Compose services
- Starts PM2 with ecosystem.config.js
6. **Create Systemd Service**: Creates `/etc/systemd/system/<APP>.service` that:
- Runs after Docker service
- Uses the specified user
- Configures proper environment (HOME, PATH)
- Auto-restarts on failure
7. **Enable & Start Service**: Enables and starts the service (if AUTOSTART=True)
With `AUTOSTART=True` (the default), two `systemd.service` operations against
`<APP>`: one setting `enabled=True` so the unit comes up on boot, one setting
`running=True` so it comes up now. Both are idempotent — a unit already enabled
and running is left alone.
## Service Management
With `AUTOSTART=False`, nothing is changed. The cube prints the two commands you
would run by hand and exits, which is the point of the flag: install now, decide
later.
### Check service status
## Configuration
| Variable | Default | What it does |
| --- | --- | --- |
| `APP` | *(required)* | the systemd unit name, without the `.service` suffix — `flintstone` for `/etc/systemd/system/flintstone.service`. This is what `systemctl` is actually pointed at. |
| `SERVICE_NAME` | `Application` | a display name, used only in the operation labels pyinfra prints and in the `AUTOSTART=False` message. Changing it changes what you read, not what happens. |
| `AUTOSTART` | `true` | whether to enable and start the unit at all. |
`APP` has no default, so `nopy -D` (`--use-defaults`) fails by name rather than
guessing. Supply it under `env` in `.nopyrc.json`, from a dependency, or at the
prompt.
## Dependencies
None declared, and none implied beyond the unit file itself. `systemd.service`
is a pyinfra built-in; there is nothing to install.
## Post-Installation
```bash
sudo systemctl status <APP>
systemctl status <APP> # is it running?
systemctl is-enabled <APP> # will it come back after a reboot?
journalctl -u <APP> -f # follow its log
```
### Start the service
```bash
sudo systemctl start <APP>
```
### Stop the service
```bash
sudo systemctl stop <APP>
```
### Restart the service
```bash
sudo systemctl restart <APP>
```
### View service logs
```bash
sudo journalctl -u <APP> -f
```
### Disable autostart
```bash
sudo systemctl disable <APP>
```
## File Structure
After deployment:
```
/home/<USER>/
├── <APP>/ # Application directory
│ ├── ecosystem.config.js # PM2 configuration
│ ├── docker-compose.yml # Docker services
│ └── ... # Application files
├── <APP>.service.sh # Startup script
/etc/systemd/system/
└── <APP>.service # Systemd service file
```
## Troubleshooting
### Service fails to start
1. Check service logs:
```bash
sudo journalctl -u <APP> -n 50
```
2. Verify Docker is running:
```bash
sudo systemctl status docker
```
3. Check if Node.js path is correct:
```bash
which node
which pm2
```
### Repository clone fails
- Ensure SSH keys are properly configured for the user
- Test SSH access: `ssh -T git@github.com`
- Check repository URL is correct
### Docker Compose fails
- Verify Docker is installed and running
- Check docker-compose.yml exists in the application directory
- Ensure user has Docker permissions: `sudo usermod -aG docker <USER>`
### PM2 not starting
- Verify PM2 is installed: `pm2 --version`
- Check ecosystem.config.js exists
- Verify NODE_PATH includes PM2 binary location
If the run fails with *Unit `<APP>.service` could not be found*, the unit was
never installed — see Purpose. `systemctl daemon-reload` is worth trying if the
file was written after systemd last read the directory.
## Notes
- The service type is set to `forking` to support PM2's daemon mode
- Service will auto-restart on failure with a 5-second delay
- Maximum 5 restart attempts in the burst period
- The service waits for Docker to be ready before starting
- Environment variables can be configured in the ecosystem.config.js file
- Enabling and starting are separate systemd concepts and this cube always does
both or neither. If you need one without the other, call `systemd.service`
from your own deploy script.
- `SERVICE_NAME` is deliberately not passed to systemd. The unit is identified
by `APP` alone, so a wrong `SERVICE_NAME` is a cosmetic mistake rather than a
cube that manages the wrong service.
@@ -1,8 +1,10 @@
from pyinfra.operations import systemd
from pyinfra.operations import server, systemd
from pyinfra import host
APP = host.data.APP
SERVICE_NAME = host.data.SERVICE_NAME
AUTOSTART = host.data.AUTOSTART
# Enable and start the service based on AUTOSTART flag
if AUTOSTART:
@@ -47,22 +47,19 @@ This cube creates a new user account with a modern shell environment (Fish), SSH
credential nobody had seen, and replaying that run produced a different one.
- **GROUPS** (string, default: `''`)
- Comma-separated list of additional groups (e.g., `"docker,sudo"`)
- Space-separated list of additional groups (e.g., `"docker sudo"`)
- Common groups:
- `docker` - Run Docker without sudo
- `sudo` - Administrative privileges
- `www-data` - Web server file access
- **PUBKEY** (string, **required** — no default)
- **PUBKEY** (string, default: `''`)
- SSH public key to authorize for the user
- Should be your public key for passwordless SSH access
- There is deliberately no default. It used to be a specific personal key, so
accepting the default authorized *someone else's* key on the new account.
No key would be a sensible guess, so the cube asks instead.
- Because it is required, `--use-defaults` refuses to run this cube unless
`PUBKEY` comes from `env` in `.nopyrc.json`, a dependency, or a hook.
- Submitting an empty value at the prompt authorizes no key at all (the account
is still created, with password login only).
- Empty (the default) authorizes no key at all — the account is created with
password login only. Some users simply do not need one.
- The default is deliberately empty, never a specific key. It used to be a
personal key, so accepting the default authorized *someone else's* key on
the new account.
## Dependencies
@@ -87,57 +84,15 @@ After deployment:
## Notes
- If the user already exists, the cube does nothing at all — rerunning it would
reset the password and overwrite `~/.config/fish`, so an existing account is
left untouched.
- The user's home directory is created at `/home/{USER}`
- Fish configuration is stored in `/home/{USER}/.config/fish/`
- Oh My Fish provides package management: `omf install <package>`
- To switch shells: `chsh -s /bin/bash` (or back to fish: `chsh -s /usr/bin/fish`)
---
# 📌 Most Useful Fish Key Bindings (with Fisher Extensions)
## 🐟 Default Fish Key Bindings
- `Ctrl + C` → Cancel the current command
- `Ctrl + D` → Exit the shell (or logout if in SSH)
- `Ctrl + L` → Clear the terminal
- `Ctrl + R` → Search command history (enhanced by `fzf.fish`)
- `Ctrl + U` → Delete the entire command line
- `Ctrl + W` → Delete the last word
- `Alt + ← / →` → Move backward/forward by a word
## 🔍 Enhanced with `fzf.fish`
- `Ctrl + R` → **Fuzzy search command history**
- `Ctrl + T` → **Fuzzy search and insert file path**
- `Alt + C` → **Fuzzy search directories (`cd` with `z`)**
## 📂 Directory Navigation (with `z`)
- `z <dir>` → Jump to a frequently used directory
- `z -l` → List most-used directories
- `z -c` → Remove a directory from `z`'s database
## 🔄 Process & Job Management
- `Ctrl + Z` → Suspend the current process
- `fg` → Bring a suspended process back to foreground
- `jobs` → List background jobs
## 🎨 Other Handy Shortcuts
- `fish_vi_key_bindings` → Enable Vi mode (press `Esc` for normal mode)
- `Ctrl + G` → Show Git status (if using `fzf.fish`)
- `Ctrl + E` → Edit command line in `$EDITOR`
## ⚙️ Useful Commands for Key Binding
```fish
# Set Fish default key bindings
fish_default_key_bindings
# Enable Vi mode
fish_vi_key_bindings
# Rebind a custom key (Example: Ctrl + G for git status)
bind \cg 'git status'
Fish's own key bindings and the plugins this cube installs are documented
upstream — `fish_key_reader` lists what is bound, and `omf help` what is
installed. They used to be reproduced here at length, which is not something
this cube knows anything about.
@@ -1,6 +1,6 @@
from pyinfra import host
from pyinfra.operations import server, files, apt
from io import StringIO
from pyinfra.facts.server import Users
# Define the username, password, and public key for the new admin user
USER = host.data.USER
@@ -11,13 +11,19 @@ PASSWORD = host.data.PASSWORD
# line, so an absent key means no key rather than a blank one.
PUBKEY = host.data.PUBKEY
PUBKEYS = [PUBKEY] if PUBKEY and str(PUBKEY).strip() else []
GROUPS = list(filter(None, map(str.strip, str(host.data.GROUPS).split())))
GROUPS = str(host.data.GROUPS).split()
FISH_PATH = "/usr/bin/fish"
FISH_CONFIG_DIR = f"{HOME_DIR}/.config/fish"
FISH_CONFIG_FILE = f"{FISH_CONFIG_DIR}/config.fish"
FISH_RC_DIR = f"{FISH_CONFIG_DIR}/rc"
SSH_AGENT_SCRIPT = f"{FISH_RC_DIR}/ssh-agent.fish"
# An existing user is left entirely alone — everything below would reset the
# password and overwrite ~/.config/fish, clobbering whatever the user has
# changed since their account was created.
if host.get_fact(Users).get(USER):
host.noop(f"User {USER} already exists")
else:
apt.packages(
name='Ensure fish shell is installed',
packages=[ 'fish'],
@@ -17,12 +17,14 @@ export default Manifest({
PASSWORD: z.string().describe('Password for the new user account').default('changeme'),
GROUPS: z
.string()
.describe('Comma-separated list of additional groups (e.g., "docker,sudo")')
.describe('Space-separated list of additional groups (e.g., "docker sudo")')
.default(''),
// Empty by default, never a specific key: this used to carry a personal
// key, which meant an unattended run authorised someone else's key on the
// new account. Empty means no key is authorised — some users need none.
PUBKEY: z
.string()
.describe('SSH public key to authorize for the user (empty for none)')
.default(''),
// No default on purpose. This used to carry a specific personal key, which
// meant an unattended run authorised someone else's key on the new account.
// Leaving it required makes `--use-defaults` refuse by name instead of
// guessing, and there is no key that would be a sensible guess.
PUBKEY: z.string().describe('SSH public key to authorize for the user'),
}),
});
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@bitsquare/nopy-cubes-core",
"version": "0.5.0",
"version": "1.0.2",
"description": "The core nopy cube bundle: apt, users, ssh, networking, services and runtimes.",
"keywords": [
"nopy",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@bitsquare/nopy-cubes",
"version": "0.5.0",
"version": "1.0.2",
"description": "Authoring types for nopy cubes: the Manifest factory and the Cube contract.",
"keywords": [
"nopy",
+12
View File
@@ -192,6 +192,18 @@ export class Cube<Schema extends AnyObjectSchema = AnyObjectSchema> {
return defaults as z.infer<Schema>;
}
/**
* Every key the schema declares, required or not.
*
* The question this answers is "does this cube claim to know about KEY", which
* is not the same as "does it have a value for it" — a cube can read a key off
* `host.data` that only the config `env` supplies. That distinction is what
* decides whether a secret is allowed to travel to it.
*/
schemaKeys(): string[] {
return Object.keys(this.manifest.schema.shape);
}
/**
* Schema keys that have to be supplied from somewhere: no `.default()`, and
* not optional. Nothing else can fill them in, so a run that cannot prompt
+132 -30
View File
@@ -1,28 +1,30 @@
# Nopy
A CLI tool that simplifies **pyinfra** script management and execution, providing an interactive workflow for deploying infrastructure configurations ("cubes") to remote hosts.
A CLI tool that simplifies **pyinfra** script management and execution, providing an interactive workflow for deploying infrastructure configurations `cubes` to remote hosts.
## Overview
Nopy wraps pyinfra with structure, validation, and an interactive experience for managing complex infrastructure deployments. It organizes deployments into self-contained "cubes" with dependency management, schema validation, and lifecycle hooks.
Nopy wraps [pyinfra](https://pyinfra.com/) in the javascript ecosystem to provide an interactive experience for managing repeatable infrastructure deployments. It organizes deployments into self-contained units - called `cubes` - adding support for transitive dependency management, user input validation, and different lifecycle hooks.
## Features
## Features in a Nutshell
- **Dependency resolution** with topological sorting
- **Before/after hooks** for multi-cube orchestration
- **Manifest files** to support declarative description of user inputs and orchestration semantics per cube
- **Dependency resolution** in dependency order, with cycle detection
- **Before/after hooks** for programmable, multi-cube orchestration
- **SSH key or password authentication**
- **Default values** with optional customization via manifest `env`
- **Schema validation** using Zod
- **Schema validation** and **type coercion** using Zod
- **Recursive cube directory discovery**
- **Dry-run mode** for previewing deployments
- **JSON output** for CI/CD integration
- **Session history** with replay capability
- **Dry-run mode** for previewing deployment scenarios
- **Pipeable output** for CI/CD integration — the plan on stdout, everything else on stderr
- **Session history** for fast replay during development
- **Multi-layered** config files with natural discovery and deterministic parameter resolution
## Workflow
1. **Load cubes** - Discovers and validates cubes from configured directories
2. **Interactive prompts** - Select cubes, target host, and authentication method
3. **Dependency resolution** - Topologically sorts cubes based on dependencies
3. **Dependency resolution** - Resolves each cube's dependencies before the cube itself, so the deploy order is a topological order of the graph; a cycle is reported by name rather than recursed into
4. **Variable assignment** - Validates and collects configuration with schema validation
5. **Execute hooks** - Runs before/after hooks for orchestration
6. **Deploy** - Sequentially executes pyinfra commands
@@ -33,23 +35,20 @@ Nopy wraps pyinfra with structure, validation, and an interactive experience for
A cube is a **directory** containing two files:
- **JavaScript manifest**: `manifest.mjs` defining schema, dependencies, defaults, secrets, and hooks
- **JavaScript manifest**: `manifest.mjs` defining schema, dependencies, defaults, secrets (encrypted only), and hooks
- **Python deployment script**: `deploy.py`, a plain pyinfra script
Configuration variables are declared in the manifest and validated with Zod schemas before the deployment script runs.
```
cubes/
├── .npcubes
└── apt/
└── install/
├── manifest.mjs
└── deploy.py
```
Any directory holding both files is treated as a cube, so cubes can be nested as deeply as you like to group them by topic. Discovery is recursive; directories starting with `.` and `node_modules` are skipped. Additional files in the cube directory (a `README.md`, config templates, and so on) are ignored by the loader and can be referenced from the deploy script — the script runs with its cube directory as the working directory.
The prefixed forms `<cube-name>.manifest.mjs` and `<cube-name>.deploy.py` are also still recognized, but plain `manifest.mjs` / `deploy.py` is the current convention.
Any directory holding both files is treated as a cube, so cubes can be nested and grouped by topic. Discovery is recursive; hidden directories starting with `.` and `node_modules` are skipped. Additional files in the cube directory (a `README.md`, config templates, and so on) are ignored by the loader but can be referenced from the deploy script — ** the pyinfra script runs with its cube directory as the working directory**.
A cube's identity comes from the manifest's `id` field (see below). If `id` is omitted, nopy falls back to an `[id]` prefix in the manifest `name`, and finally to the directory's own name. Note that the id does not have to mirror the folder path — `cubes/network/tailscale` declares `id: 'net:tailscale'`.
@@ -65,7 +64,7 @@ export default cubes.Manifest({
dependencies: () => [],
schema: z.object({
UPDATE: z.boolean().describe('Update package cache').default(false),
PACKAGES: z.string().describe('Space-separated list of packages').default('vim htop'),
PACKAGES: z.string().describe('Space-separated list of packages').default('curl htop'),
})
})
```
@@ -118,7 +117,7 @@ A variable can be set from several places in one run. Every assignment is kept,
This allows cubes to ship with reasonable defaults while still allowing users to override them globally via `.nopyrc.json` or interactively during deployment. Because `env` outranks the schema, `.nopyrc.json` is also what steers a run started with `--use-defaults`, which never prompts.
`prompt` and `param` rarely compete: a key a dependency supplies is left out of the prompt entirely, so the user is only ever asked about the keys nothing else has set.
`prompt` and `param` rarely compete: a key supplied by a dependency is left out of the user input prompt entirely.
Ranking by origin rather than by arrival order is what makes replay work: a recorded value is applied *before* the cube would be prompted for, and prompting can still override it, but a `--data` value pushed in by a dependency is never clobbered by a stale recording.
@@ -142,13 +141,25 @@ export default cubes.Manifest({
Every entry must be a key of `schema`; naming anything else is a manifest error and aborts the run, so a typo fails loudly instead of silently leaving a value unprotected.
Declaring a key a secret changes three things:
Declaring a key a secret changes four things:
- **It is never written to a session file or to the history.** Everything else the run settled on is recorded — including values that came from a `.default()` — but declared secrets are left out.
- **It is masked wherever a command or a plan is printed** — `--dry-run`, `--print-only`, and the debug log all show `********` in place of the value, in the variable list *and* in the `pyinfra` command line above it. The SSH password passed via `--password` is masked the same way, whether or not any cube declares secrets.
- **It is re-prompted on replay**, since there is nothing recorded to replay from (see [Session Recording and Replay](#session-recording-and-replay)).
- **It stops travelling.** Ordinary `env` values are seeded onto every cube in the run, because a cube may read a key off `host.data` that its own schema never declared. A secret is the exception: it reaches only the cubes whose `schema` names it. Otherwise putting a password under `env` — which is what unattended replay asks you to do — would put it on the command line of every unrelated cube, where nothing masks it because that cube never called it a secret.
Nopy does not guess. A key called `PASSWORD` in a manifest that declares no `secrets` is treated as an ordinary variable — recorded, and printed in the clear.
Declaring is global, masking is global. A key any manifest calls a secret is masked and kept out of sessions on every cube it lands on, even one whose own manifest forgot to list it. What is *not* global is the guess: a key called `PASSWORD` that no manifest declares anywhere is an ordinary variable — broadcast, recorded, and printed in the clear.
For a sensitive `env` value that no cube declares at all — a token only a hook reads, say — name it in the config instead:
```json
{
"secrets": ["DEPLOY_TOKEN"],
"env": { "DEPLOY_TOKEN": "..." }
}
```
Entries here behave exactly like a manifest's: masked, never recorded, and delivered only to cubes that declare them.
Three limits are worth knowing, because `secrets` keeps a value out of the files nopy writes and nothing more:
@@ -168,6 +179,7 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
"env": {
"SHARED_VAR": "value"
},
"secrets": ["DEPLOY_TOKEN"],
"log": {
"verbosity": "info",
"debug": false
@@ -184,8 +196,36 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
`history` controls automatic session recording (see [Deployment History](#deployment-history)), and `execution.continueOnError` sets the default for `--continue-on-error`.
`secrets` names `env` keys to treat as sensitive that no manifest declares — it is the config-side half of a manifest's `secrets`, and behaves identically. See [Secrets](#secrets).
`hosts` seeds the target picker; see [Target hosts](#target-hosts) for what else that picker offers.
`cubeDirs` holds paths, `cubePackages` holds installed npm packages that ship cubes — see [Cube Discovery](#cube-discovery) below and [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md) for publishing your own. Both are additive, and both resolve relative to the config file that named them, not to the working directory: a `.nopyrc.json` two levels up may name a package that only exists in *its* `node_modules`.
#### Target hosts
The host prompt offers more than the `hosts` array. Two entries at the top are
shortcuts for pyinfra's local connectors, each asking one follow-up question and
assembling the host string from the answer:
| Picked | Asks for | Becomes |
| --- | --- | --- |
| `docker` | a container name/id, **or** an image reference | `@docker/<answer>` |
| `vagrant` | the machine name (default `default`) | `@vagrant/<answer>` |
| *(a configured host)* | — | itself |
| `custom` | any address | itself |
The two connector forms can equally be written into `hosts` directly — a session
records whatever string the run used, so `"hosts": ["@vagrant/nopytestvm"]` and
picking `vagrant` are the same thing to everything downstream.
The docker answer is deliberately not validated as one kind or the other, because
the two mean very different things and only the connector can tell them apart (it
looks for a matching container first). A **container** is mutated in place and
left running; an **image** makes pyinfra start a throwaway container, apply the
deploy, commit the result as a new image and print its id. See
[DOCKER.md](docs/DOCKER.md) and [VAGRANT.md](docs/VAGRANT.md).
#### Logging Configuration
Control pyinfra output verbosity and debug information using the `log` configuration object:
@@ -259,12 +299,17 @@ Sessions are stored in `.nopysession.json` files with the following structure:
- **`env`**: The `env` block of `.nopyrc.json` as it stood at record time, kept for reference
- **`hosts`**: Array of target hosts
- **`auth`**: Authentication configuration (passwords are never stored)
- **`version`**, **`timestamp`**, **`name`**: stamped on every session nopy writes — the format version, the ISO 8601 record time, and a one-line description in the same `date - cubes → hosts` form the history list uses
Only `cubes` and `auth` are required. A hand-written session may omit the rest, and one that predates the stamp still loads; a `version` this build does not recognise is a warning on stderr, never a refusal.
`auth.method` has a third value the picker never offers: **`ssh`**, meaning the connector owns authentication and nopy supplies none. It is what an `@vagrant/` or `@docker/` host gets, which is why replaying one asks for nothing.
**What is recorded:** every value each cube settled on, regardless of where it came from — a value the user typed, one inherited from `.nopyrc.json` `env`, one a dependency supplied, and one that fell through to the schema's `.default()` are all written out the same way. A session is therefore a full snapshot rather than a diff, and a `--use-defaults` run produces a session with real values in it instead of an empty one.
The consequence is that replay is faithful rather than re-derived: the recorded value outranks the current `.nopyrc.json` `env` and the current schema default, so editing either one does not silently change what a replay does. To pick up a new default, record a fresh session.
**Security Note**: Passwords are never stored in session files. This covers both the SSH password — a session records the auth *method* and username, never the credential — and any schema key a cube's manifest lists under [`secrets`](#secrets). Both are re-prompted on replay.
**Security Note**: Passwords are never stored in session files. This covers both the SSH password — a session records the auth *method* and username, never the credential — and any schema key a cube's manifest lists under [`secrets`](#secrets). Both are re-prompted on replay. The rule applies to the session's `env` block as well as to each cube's `variables`, so a declared secret set in `.nopyrc.json` is left out of the recorded copy rather than written back out in plaintext.
#### Recording a Session
@@ -274,6 +319,9 @@ nopy install --save-session my-deployment.nopysession.json
# With defaults (no prompts for variables)
nopy install -D --save-session automated-deployment.nopysession.json
# Also works on a replay — the resolved cube set is what you asked to capture
nopy install -R --save-session repeat-of-the-last-run.nopysession.json
```
#### Replaying a Session
@@ -288,7 +336,9 @@ nopy install --load-session my-deployment.nopysession.json
A replay runs straight through without asking anything, with three exceptions. Password authentication always re-prompts. A session with no recorded host falls back to the host picker. And a cube is re-prompted for its declared secrets, plus for any required variable the session has no value for — which happens when the cube's schema has gained a field since the session was written.
Those re-prompts are what a session cannot supply, so `--use-defaults` cannot paper over them: combining `-D` with a replay that needs either fails with a message naming the keys rather than deploying with a placeholder. Put the values under `env` in `.nopyrc.json` to make such a replay unattended.
Those re-prompts are what a session cannot supply, so `--use-defaults` cannot paper over them: combining `-D` with a replay that needs either fails with a message naming the keys rather than deploying with a placeholder. Put the values under `env` in `.nopyrc.json` — or pass them from a dependency — to make such a replay unattended. A secret supplied that way reaches only the cubes that declare it, so this does not broadcast it across the run; see [Secrets](#secrets).
A schema `.default()` is deliberately *not* accepted in its place. The recorded answer is gone on purpose, so falling back to the manifest would deploy a different credential than the run being replayed, and say nothing about it.
### Cube Discovery
@@ -314,13 +364,17 @@ A cube package is an ordinary npm package that ships its cubes in a `cubes/` dir
Install it and name it — nothing needs linking or copying:
```sh
pnpm add -D @bitsquare/nopy-cubes-core
pnpm add -D @bitsquare/nopy-cubes-core@main \
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
```
```json
{ "cubePackages": ["@bitsquare/nopy-cubes-core"] }
```
The tag and the registry flag are both required for this bundle today — see
[Installation](#installation).
Naming a package is a statement that cubes are expected from it, so anything wrong is an error that aborts the run rather than a silent skip: the package is not installed, it has neither a `cubes/` directory nor a `nopy.cubes` override, its `nopy.cubes` is malformed, or an entry points at a directory that does not exist or lies outside the package.
#### Ids are claimed globally
@@ -331,6 +385,18 @@ Writing cubes to publish is covered in [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md).
## Command Line Usage
### Requirements
| | |
| --- | --- |
| **Node** | ≥ 22 |
| **pyinfra** | on `PATH` — `pipx install pyinfra` |
| **the connector** | `vagrant` or `docker` on `PATH`, if you deploy to one |
nopy builds pyinfra command lines and spawns them; it does not vendor pyinfra and
will not install it for you. A missing `pyinfra` surfaces as a spawn failure on
the first deploy, after every prompt has been answered.
### Installation
```bash
@@ -341,13 +407,21 @@ The cubes live in a separate bundle, installed into whichever project describes
your infrastructure and named in its `.nopyrc.json`:
```bash
pnpm add -D @bitsquare/nopy-cubes-core
pnpm add -D @bitsquare/nopy-cubes-core@main \
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
```
```json
{ "hosts": ["your-host"], "cubePackages": ["@bitsquare/nopy-cubes-core"] }
```
**The tag and the registry flag are both required for the bundle today.** It has
not been published to npmjs yet, and the Gitea registry publishes no `latest`
tag, so a plain `pnpm add -D @bitsquare/nopy-cubes-core` fails with a 404 against
npmjs and an untagged Gitea install resolves to nothing. Name `@main` or `@next`
explicitly. See [Channels](#channels) for what the tags mean and how to set the
scope persistently. The CLI itself is on npmjs and installs without either.
#### Channels
Three dist-tags are published, and the one you install from is the one you stay
@@ -406,8 +480,8 @@ npm install -g @bitsquare/nopy@latest
```
Once a day, `nopy` checks its channel in the background and prints a one-line
hint to **stderr** when a newer version exists — never to stdout, so `--json`
and `--print-only` output stay clean. The answer is cached in
hint to **stderr** when a newer version exists — never to stdout, so a piped
`--print-only` stays clean. The answer is cached in
`~/.nopy/update-check.json`; a registry that is slow or unreachable is given
1.5 seconds and then ignored.
@@ -427,6 +501,19 @@ pnpm --filter @bitsquare/nopy run nopy # runs the CLI from source via tsx
### Basic Commands
**Start a new project**:
```bash
nopy init
```
Writes two files into the current directory and touches nothing that already
exists (`--force` overwrites): a starter `.nopyrc.json` — the file without
which `nopy install` refuses to run — and `NOPY.LLM.md`, a bundled usage guide
written for AI assistants. Point your coding agent at it (or let it discover
the file) and it can answer nopy questions, write cubes, and plan deployments
from project-local context instead of guessing.
**Install cubes (default command)**:
```bash
@@ -493,11 +580,14 @@ Every deployment is automatically recorded to a `.nopy.history.json` file in the
The recording happens before the deploy commands run, so a **failed** deployment is recorded too — `-R` is the quick way to retry one after fixing the cause. Replaying a session with `-R` or `-H` does not itself create a new entry, so repeating never pushes the original run out of the list.
A `--load-session` run *is* recorded, and the distinction is the point: a session file has never been in history, so without the entry `nopy history` would report nothing afterwards and `-R` would have nothing to repeat.
A run is *not* recorded when:
- `--dry-run` or `--no-history` is passed
- `--dry-run`, `--print-only` or `--no-save-history` is passed — the first two deploy nothing, and history is what `-R` repeats
- No cubes were selected, so there was nothing to deploy
- `history.autoSave` is set to `false` in `.nopyrc.json`
- it is a `-R` or `-H` replay, as above
Because the history file is resolved against the current working directory, each project keeps its own history — running nopy from a different directory will not find the previous run. As with session files, passwords are never stored and are re-prompted on replay.
@@ -534,14 +624,26 @@ nopy install --dry-run
Shows the execution plan including commands, environment variables, and targets without running anything. Sensitive data is masked in output.
**JSON output (for CI/CD)**:
**CI/CD**:
```bash
nopy install --json
nopy history --json
nopy install --print-only > plan.txt # the commands, and nothing else
nopy install -D # run it; exit code 1 if any cube failed
```
Machine-readable JSON output for scripting and CI/CD integration.
There is no `--json` on `install`, deliberately. A deploy runs pyinfra with
inherited stdio, so during a run nopy does not own its own stdout — pyinfra does,
and writes an unbounded amount to it. Anything nopy appended afterwards would not
be parseable by any definition a caller could rely on. Two things are guaranteed
instead:
- **stdout carries the deploy commands and pyinfra's own output. Everything nopy
says about itself — the config banner, progress lines, warnings, the update
hint, errors — goes to stderr.** So `--print-only` redirects cleanly.
- **The exit code is the verdict**: `1` if any cube failed, `0` otherwise.
`nopy history --json` is unaffected and is how a script finds the id to pass to
`-H`.
**Continue on error**:
+146 -62
View File
@@ -25,6 +25,7 @@ If you are writing cubes rather than calling nopy from code, you want
- [Workflow Module](#workflow-module)
- [Session Module](#session-module)
- [History Module](#history-module)
- [Init Module](#init-module)
- [Config Module](#config-module)
- [Prompts Module](#prompts-module)
- [Update Module](#update-module)
@@ -137,6 +138,7 @@ class Cube<Schema extends AnyObjectSchema = AnyObjectSchema> {
get secrets(): string[]; // manifest.secrets ?? []
getDefaults(): z.infer<Schema>;
schemaKeys(): string[];
requiredKeys(): string[];
isSecret(key: string): boolean;
}
@@ -151,6 +153,11 @@ single required field used to leave the cube with no variables at all.
and not optional. A `--use-defaults` run that cannot supply one aborts by name
rather than deploying the cube with the value missing.
`schemaKeys()` returns every declared key, required or not. It answers a
different question — whether the cube *claims to know about* a key, rather than
whether it has a value for one — and that is what decides whether a secret in the
config `env` is allowed to reach it.
### `CubeSource`
Where a cube came from. Carried because a cube's directory does not say how it
@@ -264,13 +271,12 @@ const result = await nopy({ useDefaults: true, dryRun: true });
|------|------|---------|-------------|
| `useDefaults` | `boolean` | `false` | Skip the variable prompts. A cube with a required key nothing supplied aborts the run by name. |
| `useAuthKey` | `boolean` | `false` | Force SSH key auth, skipping the auth prompt. |
| `saveSession` | `string` | – | Path to write the session to. **Ignored during a replay.** |
| `saveSession` | `string` | – | Path to write the session to. Honoured on a replay too. |
| `loadSession` | `string` | – | Path to a session file to replay. |
| `replaySession` | `NopySession` | – | A session object to replay, used by `-R` / `-H` from history. Takes precedence over `loadSession`. |
| `dryRun` | `boolean` | `false` | Print the execution plan instead of running it. |
| `printOnly` | `boolean` | `false` | Print the built pyinfra commands and return; the executor is never reached. |
| `continueOnError` | `boolean` | `false` | Keep going after a cube fails. |
| `jsonOutput` | `boolean` | `false` | Suppress the config banner and progress lines. See [Known gaps](#known-gaps). |
| `saveToHistory` | `boolean` | `true` | Record the session in `.nopy.history.json`. |
**Returns:** `Promise<NopyResult | undefined>` — `undefined` when cube loading
@@ -434,21 +440,30 @@ Recursive, per (cube, host):
5. emit the deploy call;
6. run `after` hooks.
There is no separate topological sort — the ordering falls out of the recursion,
and a `${cubeId}:${host}` set makes emission idempotent. Consequently there is no
cycle detection either: two mutually dependent cubes recurse until the stack
overflows.
There is no separate topological sort — emission is post-order, so a dependency
is always emitted ahead of its dependent and the ordering *is* topological
without an algorithm computing it. A `${cubeId}:${host}` set makes emission
idempotent.
**Throws** when the cube id is unknown, when `useDefaults` cannot fill a required
key, when a replay would need a value only the user has (secrets are never
recorded), and when a cancelled prompt leaves a required key empty.
Cycles are detected by the resolution stack rather than by the sort that does not
exist: a (cube, host) pair re-entered while it is still resolving raises with the
whole path named — `Circular dependency on host1: a → b → c → a`. The stack is
separate from the idempotence set on purpose, since re-entering a *finished* cube
with different `param` overrides is legitimate and a dependency or hook may do it.
The command it builds:
**Throws** when the cube id is unknown, when the dependency graph contains a
cycle, when `useDefaults` cannot fill a required key, when a replay would need a
value only the user has (secrets are never recorded), and when a cancelled prompt
leaves a required key empty.
The command it builds — an argv array, one element per argument, nothing quoted:
```
pyinfra <host> -y [--user U --password P] --data "K=V" … --chdir <cubeDir> <cubeDir>/<deployScript>
pyinfra <host> -y [-v|-vv|-vvv] [--debug] [--user U --password P] --data K=V … --chdir <cubeDir> <cubeDir>/<deployScript>
```
The verbosity and debug flags come from `config.log` via `logConfigToFlags()`.
---
## Variables Module
@@ -507,9 +522,10 @@ displaced stays visible underneath. The trace is never persisted.
```typescript
class Variables {
constructor(env?: TVariables);
constructor(env?: TVariables, globalSecrets?: Iterable<string>);
declareSecrets(cube: string, keys: readonly string[]): void;
declareSchema(cube: string, keys: readonly string[]): void;
isSecret(cube: string, name: string): boolean;
assign(cube: string, origin: Origin, values?: TVariables): void;
@@ -527,6 +543,25 @@ const MASK = '********';
`declareSecrets()` is retroactive as well as prospective, so it does not matter
whether the caller declares before or after the values arrive.
`globalSecrets` is every key *any* manifest declares secret, plus the config's
own `secrets` list. `nopy()` computes it once after `loadCubes()`, before the
first cube resolves, so resolution order cannot change whether a value is treated
as a credential. It does two things:
- `isSecret()` is true for such a key on **every** cube, so a manifest that lists
`PASSWORD` in `schema` and forgets it in `secrets` still gets masking and still
keeps the value out of the session.
- The config `env` stops being broadcast for it. Ordinary `env` keys are seeded
onto every cube — deliberately, since a cube may read a key off `host.data`
that it never declared — but a secret reaches only the cubes whose
`schemaKeys()` include it.
`declareSchema()` is what supplies those keys, and it has an ordering
requirement: call it before anything assigns to the cube, because the first
assignment is what seeds `env`. `BuildContext.resolveCube` calls it immediately
after `declareSecrets()`. It deliberately does not create the cube's bucket
itself.
`persistable()` leaves a secret out entirely rather than masking it, so a replay
sees it as absent and asks for it again. That is why replaying a session whose
cubes declare secrets is interactive even under `-D` — a `-D` replay that would
@@ -574,9 +609,14 @@ interface ExecutionOptions {
### `executeDeployCalls(calls, options?)`
Runs the calls **sequentially**, in the order they were built, through
`execa({ shell: true })` with `stdio: 'inherit'` so pyinfra's output reaches the
terminal live. Stops at the first failure unless `continueOnError`. With
`dryRun`, prints the plan and returns `[]` without executing.
`execa(command[0], command.slice(1))` with `stdio: 'inherit'` so pyinfra's output
reaches the terminal live. Stops at the first failure unless `continueOnError`.
With `dryRun`, prints the plan and returns `[]` without executing.
**No shell.** It used to join `command` into one string and run it through
`execa({ shell: true })`, which made every `--data` value shell syntax: a
password or a variable containing `;`, a backtick or `$(…)` was executed rather
than passed along. Spawning the argv directly removes the parse step entirely.
```typescript
const results = await executeDeployCalls(calls, {
@@ -585,15 +625,16 @@ const results = await executeDeployCalls(calls, {
});
```
### `outputExecutionPlan(calls, asJson?)`
### `outputExecutionPlan(calls)`
```typescript
outputExecutionPlan(deployCalls); // text
outputExecutionPlan(deployCalls, true); // JSON
outputExecutionPlan(deployCalls);
```
Both forms mask secrets. Note that `executeDeployCalls` calls this without the
second argument, so `--dry-run --json` prints the text plan.
Prints the plan a `--dry-run` shows, with secrets masked. Went from
`(calls, asJson?)` to `(calls)` when `--json` was removed; the JSON branch was
unreachable from the CLI, since `executeDeployCalls` never passed the second
argument.
### `maskCommand(call)` / `maskVariables(call)`
@@ -604,9 +645,11 @@ maskVariables(call); // Record<string, string>
pyinfra takes its data on the command line, so the real values have to be in
`call.command`; these are the last point before they would reach a log, a
`--print-only` dump or a dry-run plan. `maskCommand` replaces the SSH
`--password` argument and every `--data "KEY=…"` whose key the manifest declared
a secret.
`--print-only` dump or a dry-run plan. `maskCommand` walks the argv, replaces the
element after `--password` and the value of every `--data KEY=…` whose key the
manifest declared a secret, and shell-quotes the rest so `--print-only` output
stays pasteable. It is the only thing that joins `command` into a string —
nothing executes it that way.
This covers nopy's own output only. The value still reaches pyinfra on its
command line, so it is visible in `ps` — inherent to pyinfra's `--data`
@@ -641,7 +684,7 @@ interface WorkflowResult {
authMethod: string;
username?: string;
password?: string;
isReplay: boolean;
replaySource?: 'file' | 'history'; // undefined on a fresh interactive run
}
interface WorkflowOptions {
@@ -673,10 +716,12 @@ The same, from a session object rather than a path — the `-R` / `-H` path.
```typescript
interface NopySession {
cubes: CubeSession[]; // required
auth: AuthSession; // required
version?: string;
timestamp?: string; // ISO 8601
name?: string;
cubes: CubeSession[];
hosts?: string[];
auth: AuthSession;
env?: TVariables;
}
@@ -692,7 +737,10 @@ interface AuthSession {
}
```
There is no `version` or `timestamp` field, and nothing validates compatibility.
`version` and `timestamp` are stamped on every session nopy writes and demanded
of none it reads — an older file, or a hand-written one, simply lacks them.
Nothing validates compatibility beyond a warning on an unrecognised `version`;
the constant is exported as `SESSION_VERSION`.
A `CubeSession` records every value the cube settled on, whatever its origin —
not just the prompted ones — minus anything the manifest declared a secret. So a
@@ -702,8 +750,7 @@ and `env` happen to say later.
### `saveSession(session, filePath)`
Writes JSON, creating the directory if needed. Note that `nopy()` skips this
during a replay.
Writes JSON, creating the directory if needed.
### `loadSession(filePath)`
@@ -713,7 +760,8 @@ const session = await loadSession('./deployment.session.mjs'); // default expor
```
Dispatches on the extension; `.json` and `.mjs` only. Validates that `cubes` is
an array, that `hosts` (if present) is an array, and that `auth` exists.
an array, that `hosts` (if present) is an array, and that `auth` exists. A
`version` other than `SESSION_VERSION` warns on stderr and loads anyway.
### `createSession(params)`
@@ -725,18 +773,27 @@ const session = createSession({
});
```
Stamps `version` and `timestamp`; pass `timestamp` to override the latter. It
does not derive a `name` — that needs the resolved cube list, which does not
exist yet at the point the session is created, so `nopy()` fills it in at save
time.
### `describeSession(session, timestamp)`
The one-line `date - cubes → hosts` description, shared with the history list so
that the two cannot drift.
### `listSessions(dirPath?)`
Non-recursive; matches **`*.session.json`** and **`*.session.mjs`** only.
A file named `deploy.nopysession.json` will not be listed, though `loadSession`
reads it fine.
Non-recursive; matches `*.nopysession.json`, `*.nopysession.mjs`,
`*.session.json` and `*.session.mjs`.
---
## History Module
Sessions are recorded automatically after a successful non-replay run, into
`.nopy.history.json` in the working directory.
Sessions are recorded automatically, into `.nopy.history.json` in the working
directory, before the deploy commands run — so a failed run is recorded too.
```typescript
const HISTORY_FILE = '.nopy.history.json';
@@ -767,8 +824,37 @@ interface SessionHistory {
| `removeFromHistory(id)` | `boolean` | `false` if the id was not found |
| `formatHistoryList(entries)` | `string` | what `nopy history` prints |
Recording is suppressed for a dry run, a replay, a run that built no deploy
calls, `--no-history`, and `history.autoSave: false` in the config.
Recording is suppressed for a dry run, a print-only run, a `-R`/`-H` replay out
of history, a run that built no deploy calls, `--no-save-history`, and
`history.autoSave: false` in the config. A `--load-session` run **is** recorded: it is not in history already, and
without the entry `-R` would have nothing to repeat.
---
## Init Module
Backs `nopy init`.
### `initProject(options?)`
```typescript
function initProject(options?: { force?: boolean; dir?: string }): InitFileResult[];
// InitFileResult: { file: string; path: string; status: 'created' | 'overwritten' | 'skipped' }
```
Writes `STARTER_CONFIG` as `.nopyrc.json` and the bundled `NOPY.LLM.md` usage
guide (`GUIDE_FILENAME`) into `dir` (default: the working directory). Existing
files are skipped unless `force` is set; the result names what happened to each
file. The guide template ships in `dist/templates/` and is resolved relative to
the module, so it works from source and from an installed package alike.
`STARTER_CONFIG` deliberately leaves `cubePackages` empty: naming a bundle
that is not installed is a hard error, and `init` must leave a config that
loads.
### `formatInitResults(results)`
Renders the per-file report plus the next-steps hint that `nopy init` prints.
---
@@ -785,6 +871,7 @@ interface NopyConfig {
cubeDirs: string[];
cubePackages: CubePackageRef[];
env: TVariables;
secrets?: string[]; // env keys to treat as sensitive that no manifest declares
log?: LogConfig;
history?: HistoryConfig;
execution?: ExecutionConfig;
@@ -827,9 +914,8 @@ config's `node_modules` rather than the working directory's. It is the same
problem `PATH_PROPERTIES` solves for relative `cubeDirs`, with a different answer:
a reference to resolve later instead of a rewritten path.
> The `CubePackageRef` name is currently not re-exported from the package root,
> though `NopyConfig` refers to it. Import it from `@bitsquare/nopy` and you get
> `NopyConfig` but not this type by name.
Re-exported from the package root alongside `NopyConfig`, which refers to it —
it was not, until the regeneration of this document noticed.
### `loadConfig()`
@@ -1031,7 +1117,7 @@ on `PATH`.
**never throws** — it sits in front of every command the user actually asked
for. Returns `null` immediately when `isUpdateCheckDisabled(env)`:
`NOPY_NO_UPDATE_CHECK` set to anything but `0`/`false`, or `CI` set at all. The
CLI prints it to **stderr**, so `--json` and piped stdout stay clean.
CLI prints it to **stderr**, so a piped `--print-only` stays clean.
### `selfUpdate(options)` → `SelfUpdateResult`
@@ -1052,6 +1138,7 @@ do. Returns `{status, command, ran}`; `ran` is `false` for `dryRun`, and for
## CLI Usage
```bash
nopy init # write a starter .nopyrc.json + NOPY.LLM.md here (-f overwrites)
nopy install # interactive (the default command; `nopy` alone works, as does `nopy i`)
nopy install -D # use defaults, no variable prompts
nopy install -K # force SSH key auth
@@ -1062,8 +1149,7 @@ nopy install -l ./sess.json # replay a session file
nopy install -n # dry run — print the plan, execute nothing
nopy install -P # print the built pyinfra commands and exit
nopy install -c # continue after a failure
nopy install -j # JSON output
nopy install --no-history # do not record this run
nopy install --no-save-history # do not record this run
nopy history # list recorded sessions (alias: h; -j for JSON)
nopy clear-history # drop them all
@@ -1083,8 +1169,10 @@ Exit code is 1 when any cube failed.
"up to date", since an unanswerable check is not a negative answer. See
[Known gaps](#known-gaps) for what that message conflates.
> `-H <id>` and `--no-history` share one Commander destination, so passing both
> discards the id and falls through to an interactive run.
> The suppression flag is `--no-save-history`, not `--no-history`. Commander
> derives an option's destination from its long flag with `no-` stripped, so
> `--no-history` wrote to the same `options.history` that `-H <id>` does and
> `nopy install -H abc --no-history` silently discarded the id.
---
@@ -1127,12 +1215,15 @@ export default Manifest({
});
```
> **Call `.default()` before `.describe()`.** In zod 4, `.default()` returns a
> `ZodDefault` wrapper that does not inherit `.description` from the type it
> wraps, and the prompt reads the description off the outer node. So
> `z.boolean().describe('Update cache').default(false)` prompts with the bare key
> `UPDATE`, while `z.boolean().default(false).describe('Update cache')` prompts
> with the sentence. Verified against zod 4.4.3.
> **The order of `.default()` and `.describe()` does not matter.** It used to.
> In zod 4, `.default()` returns a `ZodDefault` wrapper that does not inherit
> `.description` from the type it wraps, so
> `z.boolean().describe('Update cache').default(false)` prompted with the bare
> key `UPDATE` while the other order prompted with the sentence — a difference
> nothing announced, and one that 15 of the 22 core cubes were on the wrong side
> of. The prompt now unwraps `default`/`optional`/`nullable` looking for a
> description, so either chaining order gives the label. Verified against
> zod 4.4.3.
Every schema key reaches pyinfra as `--data KEY=value`, so `host.data.KEY` is
always defined. pyinfra parses the values itself: `"true"` arrives as a bool and
@@ -1164,20 +1255,13 @@ For packaging cubes as an installable npm bundle, see
Real behaviour that a reader would otherwise take on trust. Tracked in
`DOCS-AUDIT.md` and summarised in `CLAUDE.md`.
- **`logConfigToFlags()` is never consumed.** It is exported and unit-tested, but
nothing feeds its output into the built pyinfra command, so `log.verbosity` and
`log.debug` in `.nopyrc.json` have no effect today.
- **`--json` emits nothing on success.** `jsonOutput` suppresses the banner and
the progress lines, and prints `{success: false, errors}` when cube *loading*
fails. The success path returns `NopyResult` to the caller without printing it,
so a CI job gets pyinfra's inherited stdio and an exit code. `--dry-run --json`
prints the *text* plan.
- **No cycle detection.** Ordering is a side effect of recursion, not a
topological sort. Two mutually dependent cubes overflow the stack.
- **`DeployCall.dependencies` is always `[]`.** The field is populated nowhere;
dependency information lives in the emission order.
- **`ExecutionResult.stdout` / `.stderr` are always `undefined`,** because the
executor inherits stdio rather than capturing it.
executor inherits stdio rather than capturing it. This is also why `install`
has no `--json`: during a run nopy does not own its own stdout, so there is no
stream to put a machine-readable answer on. Use `--print-only` for the plan and
the exit code for the verdict.
- **Hook variables are not schema-validated.** The second argument to a hook is
the effective values as collected. `schema.parse()` runs in exactly one place —
`Cube.getDefaults()`, against `{}` — and prompt input is type-coerced, which is
+2 -2
View File
@@ -314,8 +314,8 @@ Rename one of them, or remove a source from .nopyrc.json.
There is deliberately no override, alias or precedence rule. If two bundles ever
claim the same id they are mutually exclusive, and the fix is upstream.
Surface the source in the interactive picker and in `--json` output so a user can
see where a cube came from before running it.
Surface the source in the interactive picker so a user can see where a cube came
from before running it.
## Phase 4 — `@bitsquare/nopy-cubes`, the authoring package — **done**
+46 -1
View File
@@ -5,7 +5,7 @@ This guide explains how to set up a local Docker container to test `nopy` deploy
## Prerequisites
- Docker installed and running on your machine.
- `nopy` installed and linked (see [README.md](./README.md)).
- `nopy` installed and linked (see [README.md](../README.md)).
## 1. Setup SSH Key (Important)
@@ -80,3 +80,48 @@ To stop and remove the container:
```bash
docker rm -f nopy-test-container
```
## Building an image instead of targeting a container
The `@docker` connector reads its identifier two ways, and the difference is
the whole feature:
| Host | What pyinfra does |
| --------------------- | ------------------------------------------------------------------------------------- |
| `@docker/<container>` | runs against that container and leaves it running — the flow above |
| `@docker/<image>` | starts a throwaway container, deploys into it, `docker commit`s it, prints the new image ID, removes the container |
It looks for a matching container first, so nothing distinguishes the two at the
prompt: pick `docker` at host selection and enter either an existing container
or an image reference such as `ubuntu:24.04`.
```
$ nopy install
? Select host from inventory docker
? Specify docker container name/id, or an image to build from: ubuntu:24.04
...
--> docker build complete, image ID: 39b782da6859
$ docker tag 39b782da6859 myapp:1.0
```
Two things the connector cannot do, both worth knowing before treating this as a
Dockerfile replacement. The commit is untagged, so the image exists only as an
ID until you tag it; and it carries the base image's metadata unchanged —
`CMD`, `ENTRYPOINT`, `ENV`, `EXPOSE` have no equivalent in a cube. When either
matters, own the container yourself and commit deliberately:
```bash
cid=$(docker run -d ubuntu:24.04 sleep infinity)
nopy install # host: docker → paste $cid at the prompt
docker commit --change 'CMD ["/usr/sbin/sshd","-D"]' "$cid" myapp:1.0
docker rm -f "$cid"
```
There is no `--host` flag; for an unattended build put the identifier in a
session file's `hosts` array (as `example.nopysession.json` does) and replay it
with `nopy install -l <file>`.
Note also that an image target starts from a fresh container every run, so every
cube reports changes every time — idempotence only shows up when you re-run
against a container id. And there is no init system in a plain container, so
service-level cubes still need the `--privileged` systemd setup above.
+348
View File
@@ -0,0 +1,348 @@
# Requirements and facts
A proposal, not a record. Nothing here is built.
The question it answers: a cube should be able to say *"in order to run, a user
named xyz must exist, with fish as its shell, in these groups"* — and the engine
should check that against the real host rather than blindly running whatever
cube happens to create such a user.
Everything under [What pyinfra actually gives us](#what-pyinfra-actually-gives-us)
was measured against pyinfra **3.5.1**. Everything under
[The proposal](#the-proposal) is design.
## Contents
- [The problem with `dependencies`](#the-problem-with-dependencies)
- [What pyinfra actually gives us](#what-pyinfra-actually-gives-us)
- [The proposal](#the-proposal)
- [Why `facts.py` exposes a function, not a script](#why-factspy-exposes-a-function-not-a-script)
- [Execution model](#execution-model)
- [Sharp edges](#sharp-edges)
- [Measured vs. assumed](#measured-vs-assumed)
---
## The problem with `dependencies`
`manifest.dependencies` conflates two things that are not the same:
- **a requirement** — "this cube needs a user xyz to exist"
- **a remedy** — "therefore run `user:add`"
Today only the remedy is expressible. `user:add` declares
`dependencies: () => ['apt:essentials']`, which means *always run
`apt:essentials`*, on every host, forever, regardless of whether anything it
installs is missing.
### What that actually costs
Be honest about this, because it is smaller than it first looks and it changes
what the feature is for.
pyinfra operations are **already fact-diffed**. `server.user(present=True,
shell=…)` gathers `server.Users` itself and no-ops when the state matches. The
current design is therefore not *incorrect* — it is idempotent. What it costs is:
1. **Prompts.** Resolving `user:add` drags `apt:essentials` into the plan *and
its variables into the prompt sequence*. The user is asked questions about a
cube they never chose. This is the real, visible tax.
2. **Time.** A no-op pyinfra run is still a connection, a fact gather, and an
operation build.
3. **Legibility.** The plan cannot say *"user xyz already exists, skipping"*.
This matters because the obvious objection to the whole feature is *"just write
`if not host.get_fact(…)` inside `deploy.py` — that is idiomatic pyinfra"*. The
answer is that nopy's layer is **planning and prompting**: deciding which cubes
enter the plan and which variables to ask for, before anything runs. A condition
inside `deploy.py` cannot help with either. That is the justification for lifting
facts into the Node runtime at all — and it means the feature buys **legibility
and prompt-avoidance, not correctness**.
## What pyinfra actually gives us
### The `fact` subcommand is not a machine interface
`pyinfra @local fact server.Groups` prints JSON — and prints it to **stderr**,
via `click.echo(jsonify(…), err=True)` in `pyinfra_cli/prints.py:print_fact`,
interleaved with `--> Loading config...` progress lines. Worse,
`_run_fact_operations` wraps each fact in `except PyinfraError: pass`, so the
command **exits 0 whether or not the fact resolved**.
Parsing that stream means fishing a JSON blob out of styled log output and
having no exit code to check. Rejected.
### A deploy file that prints to stdout is clean
pyinfra deploy files execute on the control machine at *build* time, and
`host.get_fact()` is available there — it is exactly how operations do their own
diffing. A file that gathers facts, prints JSON, and declares no operations
works:
```python
import json, sys
from pyinfra import host
from pyinfra.facts.server import Users, Which
users = host.get_fact(Users)
u = users.get(host.data.USER)
print("###NOPY-FACTS###" + json.dumps({
"host": host.name,
"exists": u is not None,
"shell": (u or {}).get("shell"),
"groups": (u or {}).get("groups", []),
}), file=sys.stdout)
```
```
$ pyinfra @local -y --data USER=someone facts.py
EXIT=0
--- STDOUT ---
###NOPY-FACTS###{"host": "@local", "exists": false, "shell": null, "groups": []}
--- STDERR ---
--> Loading config...
…
--> Results:
Operation Hosts Success Error No Change
Grand total - - - -
```
Confirmed properties:
- **stdout is exclusively ours.** Every byte pyinfra emits goes to stderr, which
is the same reason nopy's own logging goes there (`configureLogtape`).
- **`--data` flows in identically** to a deploy script. Parameterising a probe
is free.
- **Zero operations is legal.** `Grand total -`, exit 0.
- **`host.name` is available**, so multi-host output is self-tagging.
- **A custom `FactBase` subclass in a sibling module imports fine** under
`--chdir`, so a cube can ship fact classes pyinfra does not have.
- **An exception exits 1** with a traceback on stderr.
### The trap
**An unreachable host exits 0 and prints nothing.**
```
$ pyinfra nonexistent.invalid.example -y probe.py
nonexistent.invalid.example is neither an inventory file, a (list of) hosts…
EXIT=0
--- STDOUT --- (empty)
```
Absence of a probe result must be a hard failure. It must never be read as
"this host has no requirements to check", which is the shape the bug would take.
The `###NOPY-FACTS###` sentinel exists for this: one line per host is *required*,
and a missing line is an error.
### `server.Users` already covers the motivating case
It returns `shell`, `groups`, `home`, `uid`, `gid`, `comment` and `password` per
user, keyed by name. `user:add` needs almost no custom fact code — see
[Sharp edges](#sharp-edges) for why `password` is a problem.
## The proposal
Split the one idea into two manifest fields, because there are two different
objects: what a cube can **report** about a host (owned by the cube responsible
for that state) and what a cube **demands** (owned by the consumer).
A single `facts:` field cannot be both.
### `provides` — on the cube that owns the state
```js
// cubes/user/add/manifest.mjs
export default Manifest({
id: 'user:add',
provides: {
/** What the probe returns. Validated on the way back in. */
schema: z.object({
exists: z.boolean(),
shell: z.string().nullable(),
groups: z.array(z.string()),
}),
/** Schema keys the probe needs in order to look anything up. */
params: ['USER'],
},
schema: z.object({ /* … unchanged … */ }),
});
```
Plus a `facts.py` in the cube directory, discovered by convention exactly as
`deploy.py` is.
### `requires` — on the consumer
```js
requires: (vars) => [{
cube: 'user:add',
with: { USER: vars.DEPLOY_USER },
expect: z.object({
exists: z.literal(true),
shell: z.literal('/usr/bin/fish'),
groups: z.array(z.string()).refine((g) => g.includes('docker')),
}),
}],
```
Three deliberate choices:
**A requirement names its provider.** If `requires` stated only a predicate, the
engine would have to search the cube space for something that satisfies an
arbitrary Zod schema. That is a planner, and a planner is a research project.
Naming the cube keeps resolution linear and keeps the failure message legible.
**`requires` is `(vars) => …`, like `dependencies` already is.** A requirement
almost always depends on the consumer's own variables — you cannot know *which*
user to check for until the consumer has been asked.
**Zod is the predicate language.** It is already this repo's schema vocabulary,
`z.literal` and `.refine()` cover the cases, and `z.treeifyError()` produces the
failure report for free. The cost is that `.refine()` closures are not
serialisable, so a requirement can never be written into a session — only its
*result* can. See [Sharp edges](#sharp-edges).
### The engine's rule
This is the half of the design that "only run if requirements are fulfilled"
leaves out. When a requirement is **not** met, there are three possible answers:
| | |
| --- | --- |
| **Abort** | Honest, and useless. On a bare host nothing is fulfilled, so every fresh deploy fails. |
| **Skip the cube** | A silent no-op. Dangerous. |
| **Remedy** | Run the named cube. |
It has to be *remedy* — and remedy means "run the cube that provides it", which
is `dependencies` again. That is the actual insight here, and it is a much
smaller change than a parallel subsystem:
> **You do not need a new mechanism. You need dependencies to become conditional.**
So:
- requirement **met** → the provider is *not* scheduled and its variables are
*not* prompted for. This is where the prompt tax disappears.
- requirement **unmet** → the provider is scheduled with `with` applied as
overrides. `with` assigns at origin `param`, which already outranks every
other origin (`default < env < session < prompt < param`), so no new
precedence rule is needed.
## Why `facts.py` exposes a function, not a script
The obvious design is symmetry: `deploy.py` is a script, so `facts.py` is a
script. Reject it.
A recursive resolution over a dependency tree issues one probe per (cube, host,
params). At one pyinfra invocation each, that is one SSH connection each —
seconds apiece, multiplied by the tree. Unaffordable.
But probes are **pure reads with no ordering constraints between them**, which
means every probe for a given host can be gathered in a *single* pyinfra
invocation: one generated driver script that imports each cube's `facts.py`,
calls it with its params, and emits one JSON object per host. One connection per
host per resolution round, instead of one per cube.
A standalone script can only be run alone. A function can be batched:
```python
# cubes/user/add/facts.py
from pyinfra.facts.server import Users
def gather(host, params):
u = host.get_fact(Users).get(params["USER"])
return {
"exists": u is not None,
"shell": (u or {}).get("shell"),
"groups": (u or {}).get("groups", []),
}
```
Consequence: params arrive as one `--data NOPY_PROBES=<json>` blob rather than
per-cube `--data KEY=…`, since a batched run carries several cubes' params at
once. Probe results are memoised per (cube, host, params) in the `BuildContext`,
the same shape as the existing `resolvedCubes` set.
## Execution model
Probes run during **resolution**, before anything has deployed. That has a
consequence worth stating plainly rather than discovering later:
When the plan reads *"user absent → schedule `user:add` → then B"*, B's
requirement was never verified. It was **assumed** met because a remedy was
scheduled. Every tool in this space stops there.
Since the probe already exists, the loop can be closed for nearly nothing:
> **Re-run the provider's probe after it deploys, and fail loudly if the
> requirement still is not met.**
`facts.py` then serves as a post-condition test as well as a precondition check.
This is the single most valuable property of the design — it is strictly more
than Ansible's `when:` offers — and it should be built in from the start rather
than added as a later refinement.
Sketch of the resolution change in `BuildContext.resolveCube`, between steps 3
(`before` hooks) and 4 (dependencies):
1. Collect `manifest.requires?.(currentVars)`.
2. Batch every unmet-so-far probe for this host into one pyinfra run.
3. Parse each result against the provider's `provides.schema`, then against the
consumer's `expect`.
4. For each failure, `resolveCube(req.cube, host, req.with)`.
5. After that provider's deploy call executes, re-probe and assert.
Step 5 does not fit the current shape: `deployCalls` are all built first and
executed later, so a post-condition needs the executor to call back into
probing. That is the one structurally invasive part of this proposal.
## Sharp edges
**`--print-only` purity.** A dry run touches no host today. Probing breaks that
outright. Needs an explicit answer — either a probe-free plan that renders
requirements as unevaluated, or an opt-in flag. Silently connecting during what
the user believes is a dry run is not acceptable.
**Secret leakage.** `server.Users` returns the **encrypted password hash** in
its `password` field. A probe result that reaches history, a plan dump, or a log
line is a credential leak. The existing `secrets` machinery is per-*variable*
and does not apply — facts need their own redaction rule. Easy to miss, hard to
take back.
**Sessions must never persist facts.** Facts are host state at a moment in time;
a replay must re-probe, never restore. Recording them in *history* for
diagnostics is fine and useful. This is also forced by `.refine()` being
unserialisable.
**Sudo.** Many useful facts require `_sudo`, and a probe runs before the deploy
has made any of its own auth decisions. Unresolved — worth a spike.
**Cycles.** `requires` introduces a second edge type into a graph that still has
no cycle detection (see *Known drift* in `CLAUDE.md`). Conditional edges make
"A requires B, B requires A under different params" considerably more reachable
than the current unconditional ones do.
**Naming.** `provides` / `requires` over the original `facts:`, because the
field names should say which side of the relationship they belong to.
## Measured vs. assumed
Measured against pyinfra 3.5.1, on this machine:
- `fact` subcommand writes JSON to stderr and exits 0 on fact failure
- a deploy file's stdout is uncontaminated by pyinfra output
- `--data` reaches a probe as `host.data.KEY`
- a deploy file with zero operations succeeds
- a custom `FactBase` in a sibling module resolves under `--chdir`
- an exception in a deploy file exits 1
- **an unreachable host exits 0 with empty stdout**
Assumed, not yet tested:
- batching several cubes' probes into one driver script works and is meaningfully
cheaper than N invocations — this is the load-bearing cost assumption and
should be the first thing spiked
- `host.get_fact()` accepts `_sudo` from within a probe function
- the post-condition re-probe can be threaded through `executeDeployCalls`
without unpicking the build-then-execute split
+29 -12
View File
@@ -2,9 +2,14 @@
Nopy supports two session file formats: **JSON** and **MJS** (ES Module JavaScript).
The extension is what picks the loader, so a session file has to end in `.json`
or `.mjs`; anything else is refused by name. The `.nopysession.*` names used
throughout are the convention `listSessions()` looks for — `-s` and `-l` accept
any path you give them.
## Supported Formats
### JSON Format (`.session.json`)
### JSON Format (`.nopysession.json`)
Traditional JSON format for session files:
@@ -35,7 +40,7 @@ Traditional JSON format for session files:
- Cannot use dynamic values or computation
- No code reuse or imports
### MJS Format (`.session.mjs`) - **Recommended**
### MJS Format (`.nopysession.mjs`) - **Recommended**
JavaScript module format with full ES Module support:
@@ -172,7 +177,7 @@ export const commonCubes = [
];
```
**my-session.session.mjs:**
**my-session.nopysession.mjs:**
```javascript
import { productionHosts, commonCubes } from './common-config.mjs';
@@ -232,7 +237,7 @@ function generateSession(config) {
};
const content = `export default ${JSON.stringify(session, null, 2)};`;
fs.writeFileSync('generated.session.mjs', content);
fs.writeFileSync('generated.nopysession.mjs', content);
}
// Generate from external configuration
@@ -255,10 +260,10 @@ Both formats are loaded the same way:
import { loadSession } from '@bitsquare/nopy';
// Load JSON
const jsonSession = await loadSession('./my-session.session.json');
const jsonSession = await loadSession('./my-session.nopysession.json');
// Load MJS
const mjsSession = await loadSession('./my-session.session.mjs');
const mjsSession = await loadSession('./my-session.nopysession.mjs');
```
The file extension determines which loader to use.
@@ -267,7 +272,7 @@ The file extension determines which loader to use.
To convert an existing JSON session to MJS:
1. Rename the file from `.session.json` to `.session.mjs`
1. Rename the file from `.nopysession.json` to `.nopysession.mjs`
2. Add `export default` before the configuration object
3. Remove quotes from property keys (optional)
4. Add comments and dynamic values as needed
@@ -304,11 +309,12 @@ Both formats must export/contain an object with this structure:
```typescript
interface NopySession {
version: string; // Session format version
timestamp: string; // ISO timestamp
cubes: CubeSession[]; // Array of cube configurations
hosts: string[]; // Target hosts
auth: AuthSession; // Authentication configuration
cubes: CubeSession[]; // Array of cube configurations — required
auth: AuthSession; // Authentication configuration — required
version?: string; // Session format version, currently "1.0.0"
timestamp?: string; // ISO timestamp
name?: string; // One-line description
hosts?: string[]; // Target hosts
env?: Record<string, any>; // Global environment variables
}
@@ -323,6 +329,17 @@ interface AuthSession {
}
```
Only `cubes` and `auth` are demanded of a session being *read* — the loader
requires what it cannot work without and nothing else, so the sessions in these
examples are all valid, and one written before `version` existed still loads. A
session nopy *writes* always carries `version`, `timestamp` and `name`; a
`version` this build does not recognise produces a warning on stderr and loads
anyway.
`method: 'ssh'` is the third value and the one no prompt produces: it means the
connector handles authentication and nopy supplies no credential. Every
`@vagrant/` and `@docker/` host gets it.
A session nopy *writes* holds, per cube, every value that cube ran with — what
was typed, what came from `.nopyrc.json`, what a dependency supplied, and what
fell through to the schema's `.default()`. Two things are deliberately absent and
+35 -2
View File
@@ -1,7 +1,9 @@
# Vagrant
`vagrant ssh-config` to find the SSH port of the machine
`vagrant status --machine-readable` will be executed by pyinfra to get information about available VMs
A Vagrant box is the cheapest way to run a cube against a real machine you can
throw away afterwards.
## The Vagrantfile
```ruby
@@ -19,3 +21,34 @@ Vagrant.configure("2") do |config|
end
```
## Naming the machine to nopy
The host string is `@vagrant/<name>`, where `<name>` is what `config.vm.define`
declared — `nopytestvm` above. It is pyinfra's connector syntax, not nopy's, and
it is the same shape as `@docker/<container-or-image>`.
Two ways to get there. Either pick `vagrant` in the host prompt and answer the
follow-up with the machine name, which is what builds the string for you, or put
it in `.nopyrc.json` so it appears in the list directly:
```json
{
"hosts": ["@vagrant/nopytestvm"],
"cubePackages": ["@bitsquare/nopy-cubes-core"]
}
```
pyinfra runs `vagrant status --machine-readable` to find the available machines
and `vagrant ssh-config` for the SSH port, so `vagrant` has to be on `PATH` and
the box has to be `up` before a deploy.
## Cleaning up
```sh
vagrant halt # stop it, keep the disk
vagrant destroy -f # delete it — the next `vagrant up` is a fresh box
```
`destroy` is the one to use between test runs of a cube that is not idempotent:
re-running against a half-configured box tests something other than the cube.
+3 -1
View File
@@ -1,5 +1,7 @@
{
"version": "1.0.0",
"name": "Example Deployment Session",
"timestamp": "2026-07-30T09:15:00.000Z",
"cubes": [
{
"key": "apt:essentials",
@@ -8,7 +10,7 @@
}
}
],
"hosts": ["@docker/nopy-test-ubuntu"],
"hosts": ["@docker/nopy-test-container"],
"env": {
"KEY_DIR": "../../vault/tmp"
},
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@bitsquare/nopy",
"version": "0.5.0",
"version": "1.0.2",
"description": "A system to simplify pyinfra script management and execution.",
"keywords": [
"pyinfra",
@@ -43,7 +43,7 @@
},
"scripts": {
"clean": "rm -rf dist .tsbuildinfo",
"build": "tsc && cp src/cubes/*.mjs dist/cubes/",
"build": "tsc && cp src/cubes/*.mjs dist/cubes/ && mkdir -p dist/templates && cp -R src/templates/. dist/templates/",
"prepack": "pnpm run build",
"link:local": "pnpm run build && npm link",
"nopy": "tsx src/nopy.cli.ts",
+114 -24
View File
@@ -7,6 +7,8 @@ import type { Cube, CubeVariables, HookContext } from '@bitsquare/nopy-cubes';
import { getLogger } from '@logtape/logtape';
import type { Variables } from '../nopy.common.js';
import type { NopyConfig } from '../nopy.config.js';
import { logConfigToFlags } from '../nopy.config.js';
import { NopyUsageError } from '../nopy.errors.js';
import type { DeployCall } from '../nopy.executor.js';
import { VariableAssignment } from '../nopy.prompts.js';
import type { CubeSession, NopySession } from '../nopy.session.js';
@@ -21,6 +23,18 @@ export class BuildContext {
public readonly cubeSessions: CubeSession[] = [];
private readonly resolvedCubes = new Set<string>();
/**
* The (cube, host) pairs currently being resolved, innermost last.
*
* `resolvedCubes` cannot serve here: it is written by `buildDeployCall`, which
* runs *after* the recursive descent, so a cycle never reaches it — the two
* cubes just recurse until the stack overflows. It cannot be widened into a
* "seen" set either, because re-entering a cube with different `param`
* overrides is a legitimate thing for a dependency or a hook to do. What is
* never legitimate is re-entering one that has not finished.
*/
private readonly resolving: { cubeId: string; host: string }[] = [];
constructor(
public readonly allCubes: Record<string, Cube>,
public readonly variables: Variables,
@@ -45,21 +59,33 @@ export class BuildContext {
}
/**
* Fails a non-interactive run that cannot fill a required variable.
* Fails a run that cannot fill a required variable.
*
* Without this the cube would be deployed with the key simply absent from
* `--data`, and the deploy script would read `None` off `host.data`.
* `--data`, and the deploy script would read `None` off `host.data` — against
* the documented guarantee that every schema key reaches it.
*
* Runs on the interactive path too, not only under `--use-defaults`. A prompt
* is not proof of an answer: a terminal that misreports its size renders an
* empty form and submits `{}` without the user seeing a field, which is
* exactly how this was found.
*/
private assertVariablesComplete(cube: Cube): void {
const missing = this.missingRequired(cube);
if (missing.length === 0) return;
const [one, them] =
missing.length === 1 ? ['has no default value', 'it'] : ['have no default values', 'them'];
throw new Error(
`Cube "${cube.id}" cannot run with --use-defaults: ${missing.join(', ')} ${one}. ` +
const list = missing.join(', ');
const them = missing.length === 1 ? 'it' : 'them';
const have = missing.length === 1 ? 'has no default value' : 'have no default values';
throw new NopyUsageError(
this.options.useDefaults
? `Cube "${cube.id}" cannot run with --use-defaults: ${list} ${have}. ` +
`Set ${them} under "env" in .nopyrc.json, pass ${them} from a dependency, ` +
'or drop --use-defaults to be prompted.'
: `Cube "${cube.id}" is missing ${list}. Nothing supplied ${them} — the form may have ` +
`been submitted empty. Re-run and fill ${them} in, or set ${them} under "env" ` +
'in .nopyrc.json.'
);
}
@@ -81,23 +107,58 @@ export class BuildContext {
if (gaps.length === 0) return;
if (this.options.useDefaults) {
throw new Error(
`Cube "${cube.id}" cannot be replayed with --use-defaults: ${gaps.join(', ')} ` +
'would have to be entered. Secrets are never recorded in a session. ' +
'Replay without --use-defaults, or set the values under "env" in .nopyrc.json.'
// A gap is only a gap if nothing outside the session filled it. `env` and
// `param` both say deliberately what the value is, which is exactly what
// the old message told the user to do — and then failed anyway.
//
// `default` is not accepted here. The session dropped the secret on
// purpose, so falling through to a manifest default would deploy a
// different credential than the run being replayed, without saying so.
const unsatisfied = gaps.filter((key) => {
const origin = this.variables.of(cube.id, key)?.origin;
return origin !== 'env' && origin !== 'param';
});
if (unsatisfied.length > 0) {
const them = unsatisfied.length === 1 ? 'it' : 'them';
const secret = unsatisfied.some((key) => cube.secrets.includes(key));
throw new NopyUsageError(
`Cube "${cube.id}" cannot be replayed with --use-defaults: ` +
`${unsatisfied.join(', ')} would have to be entered. ` +
(secret ? 'Secrets are never recorded in a session. ' : '') +
`Set ${them} under "env" in .nopyrc.json` +
(secret ? ' (a schema default is not accepted for a secret)' : '') +
`, pass ${them} from a dependency, or replay without --use-defaults.`
);
}
return;
}
log.debug('Filling session gaps', { cubeId: cube.id, gaps });
await VariableAssignment(cube, this.variables, { keys: gaps });
// A cancelled form leaves the run short of a value it cannot invent.
const stillMissing = this.missingRequired(cube);
if (stillMissing.length > 0) {
throw new Error(
`Cube "${cube.id}" is missing ${stillMissing.join(', ')} and cannot be deployed.`
);
// A form that resolved is not a form that was answered — same check, and
// the same reason for it, as the interactive path.
this.assertVariablesComplete(cube);
}
/**
* Fails a run whose dependency graph loops back on itself.
*
* Names the whole path rather than just the repeated cube: with dependencies
* declared dynamically — and hooks free to `exec` anything at all — the edge
* that closed the loop is rarely the one you would guess from the two ends.
*/
private assertNoCycle(cubeId: string, host: string): void {
const at = this.resolving.findIndex((f) => f.cubeId === cubeId && f.host === host);
if (at === -1) return;
const path = [...this.resolving.slice(at).map((f) => f.cubeId), cubeId].join(' → ');
throw new NopyUsageError(
`Circular dependency on ${host}: ${path}. ` +
'A cube cannot depend on itself, directly or through a chain — check the ' +
'`dependencies()` of each cube named, and any `before`/`after` hook that calls `exec`.'
);
}
/**
@@ -110,15 +171,35 @@ export class BuildContext {
): Promise<void> {
const cube = this.allCubes[cubeId];
if (!cube) {
throw new Error(`Cube not found: ${cubeId}`);
throw new NopyUsageError(`Cube not found: ${cubeId}`);
}
log.debug('Resolving cube', { cubeId, host });
// 1. Declare secrets, then assign overrides and defaults. Declaring first
// means even the config `env` seeded on the cube's first assignment is
// already marked, so nothing reaches a session or a log unredacted.
this.assertNoCycle(cubeId, host);
this.resolving.push({ cubeId, host });
try {
await this.visitCube(cube, host, overrides);
} finally {
this.resolving.pop();
}
}
/**
* The body of {@link resolveCube}, once the cube is known and the cycle guard
* has admitted it.
*/
private async visitCube(cube: Cube, host: string, overrides: CubeVariables): Promise<void> {
const cubeId = cube.id;
// 1. Declare secrets and schema, then assign overrides and defaults. Both
// declarations have to come first: the cube's first assignment is what
// seeds the config `env` onto it, and by then it must already be known
// which of those keys are secret (so nothing reaches a session or a log
// unredacted) and which the cube actually declares (so a secret it does
// not declare is never seeded at all).
this.variables.declareSecrets(cubeId, cube.secrets);
this.variables.declareSchema(cubeId, cube.schemaKeys());
if (Object.keys(overrides).length > 0) {
this.variables.assign(cubeId, 'param', overrides);
}
@@ -136,6 +217,7 @@ export class BuildContext {
this.assertVariablesComplete(cube);
} else {
await VariableAssignment(cube, this.variables);
this.assertVariablesComplete(cube);
}
const currentVars = this.variables.get(cubeId);
@@ -171,6 +253,14 @@ export class BuildContext {
/**
* Builds and stores a deployment call for a resolved cube
*
* `command` is a true argv — one array element per argument, nothing
* pre-quoted. The executor spawns it without a shell, so a value containing a
* space, a quote or a `$(…)` is passed through verbatim instead of being
* re-parsed. It used to be a list of fragments joined into one shell string,
* which meant any variable value was shell syntax: an SSH password with a `;`
* in it ran whatever followed. {@link maskCommand} is the only thing that
* turns this back into a string, for display, and quotes as it goes.
*/
private buildDeployCall(cube: Cube, host: string): void {
const cubeId = cube.id;
@@ -178,17 +268,17 @@ export class BuildContext {
if (this.resolvedCubes.has(callKey)) return;
const parts: string[] = [];
const parts: string[] = [...logConfigToFlags(this.config.log)];
if (this.auth.method === 'password' && this.auth.username && this.auth.password) {
parts.push(`--user ${this.auth.username} --password ${this.auth.password}`);
parts.push('--user', this.auth.username, '--password', this.auth.password);
}
const cubeVars = this.variables.get(cubeId);
Object.entries(cubeVars).forEach(([key, value]) => {
parts.push(`--data "${key}=${value}"`);
parts.push('--data', `${key}=${value}`);
});
parts.push(`--chdir ${cube.dir}`);
parts.push('--chdir', cube.dir);
parts.push(`${cube.dir}/${cube.deployScript}`);
const command = ['pyinfra', host, '-y', ...parts];
+16
View File
@@ -10,6 +10,7 @@ export type { Assignment, Origin, TVariables, Value } from './nopy.common.js';
// Variables
export { MASK, Variable, Variables } from './nopy.common.js';
export type {
CubePackageRef,
ExecutionConfig,
HistoryConfig,
LogConfig,
@@ -21,6 +22,18 @@ export type {
} from './nopy.config.js';
// Configuration
export { getConfigPaths, loadConfig, logConfigToFlags, saveConfig } from './nopy.config.js';
export type { CreateCubeOptions } from './nopy.create-cube.js';
// Cube scaffolding
export {
assertCubeIdAvailable,
createCube,
cubeDirWarning,
DEPLOY_FILENAME,
formatCreateCubeResults,
MANIFEST_FILENAME,
suggestCubeDir,
validateCubeId,
} from './nopy.create-cube.js';
// Backwards compatibility - cubes namespace
export { cubes } from './nopy.cubes.js';
export type {
@@ -61,6 +74,9 @@ export {
removeFromHistory,
saveHistory,
} from './nopy.history.js';
export type { InitFileResult, InitFileStatus, InitOptions } from './nopy.init.js';
// Project initialization
export { formatInitResults, GUIDE_FILENAME, initProject, STARTER_CONFIG } from './nopy.init.js';
export type { NopyOptions, NopyResult } from './nopy.main.js';
// Main entry point
export { nopy } from './nopy.main.js';
+87 -23
View File
@@ -7,7 +7,16 @@
import { createRequire } from 'node:module';
import { Command } from 'commander';
import type { NopyConfig } from './nopy.config.js';
import { loadConfig } from './nopy.config.js';
import {
assertCubeIdAvailable,
createCube,
cubeDirWarning,
formatCreateCubeResults,
suggestCubeDir,
} from './nopy.create-cube.js';
import { reportError } from './nopy.errors.js';
import { exitWithFarewell, installGracefulExit, isCancellation } from './nopy.exit.js';
import {
clearHistory,
@@ -16,7 +25,9 @@ import {
getSessionById,
listHistory,
} from './nopy.history.js';
import { formatInitResults, initProject } from './nopy.init.js';
import { nopy } from './nopy.main.js';
import { CubeScaffoldPrompts } from './nopy.prompts.js';
import type { Channel } from './nopy.update.js';
import { formatCommand, selfUpdate, updateNotice } from './nopy.update.js';
@@ -33,8 +44,8 @@ const { version, buildInfo } = createRequire(import.meta.url)('../package.json')
const versionLabel = buildInfo?.commit ? `${version} (${buildInfo.commit})` : version;
/**
* Prints the update hint to stderr, so it never lands in `--json` output or in
* a `--print-only` command list being piped somewhere.
* Prints the update hint to stderr, so it never lands in a `--print-only`
* command list being piped somewhere.
*/
async function printUpdateNotice(): Promise<void> {
const notice = await updateNotice({ currentVersion: version });
@@ -57,6 +68,8 @@ program
'after',
`
Examples:
$ nopy init Set up this directory (.nopyrc.json + NOPY.LLM.md)
$ nopy create-cube Scaffold a new cube (manifest.mjs + deploy.py)
$ nopy Interactive cube selection and deployment
$ nopy -R Repeat the last deployment session
$ nopy -H <id> Run a specific session from history
@@ -67,6 +80,9 @@ Examples:
$ nopy history List all saved sessions
$ nopy clear-history Clear session history
Every flag above belongs to 'install', the default command — 'nopy -R' is
'nopy install -R'. Run 'nopy install --help' for the full list.
Session Replay:
Sessions are automatically saved to history after each deployment.
Use 'nopy history' to see available sessions and their IDs.
@@ -88,16 +104,24 @@ program
.option('-n, --dry-run', 'Show execution plan without running')
.option('-P, --print-only', 'Print deploy commands and exit (no execution)')
.option('-c, --continue-on-error', 'Continue executing after failures')
.option('-j, --json', 'Output results as JSON')
.option('--no-history', 'Do not save this session to history')
// `--no-save-history`, not `--no-history`: Commander derives the destination
// from the long flag with the `no-` stripped, so `--no-history` wrote to the
// same `options.history` that `-H <id>` does. `-H abc --no-history` set it to
// `false`, the id was silently discarded, and the run fell through to a full
// interactive session instead of replaying anything.
.option('--no-save-history', 'Do not save this session to history')
.action(async (options) => {
await printUpdateNotice();
// Loaded lazily so that --help/--version work outside a configured project.
try {
// Loaded lazily so that --help/--version work outside a configured
// project — and inside the try, so that "no .nopyrc.json here" is
// reported by `reportError` rather than escaping as an unhandled
// rejection and printing node's own stack. It is the likeliest first-run
// mistake there is.
const execConfig = loadConfig().execution ?? {};
const continueOnError = options.continueOnError ?? execConfig.continueOnError ?? false;
try {
// Handle session replay
const loadSessionPath = options.loadSession;
let sessionToReplay: { session: import('./nopy.session.js').NopySession } | undefined;
@@ -109,7 +133,9 @@ program
process.exit(1);
}
sessionToReplay = lastEntry;
console.log(`Repeating: ${lastEntry.name}\n`);
// stderr, like everything nopy says about itself — `-R --print-only` has
// to leave stdout to the commands.
console.error(`Repeating: ${lastEntry.name}\n`);
} else if (options.history) {
const entry = getSessionById(options.history);
if (!entry) {
@@ -118,7 +144,7 @@ program
process.exit(1);
}
sessionToReplay = entry;
console.log(`Running: ${entry.name}\n`);
console.error(`Running: ${entry.name}\n`);
}
const result = await nopy({
@@ -130,8 +156,7 @@ program
dryRun: options.dryRun,
printOnly: options.printOnly,
continueOnError,
jsonOutput: options.json,
saveToHistory: options.history !== false && !options.dryRun,
saveToHistory: options.saveHistory !== false && !options.dryRun,
});
// Exit with error code if deployment failed
@@ -144,20 +169,59 @@ program
// the process-level handler.
if (isCancellation(error)) exitWithFarewell();
if (options.json) {
console.log(
JSON.stringify(
{
success: false,
error: error instanceof Error ? error.message : String(error),
},
null,
2
)
);
} else {
console.error('Error:', error instanceof Error ? error.message : error, error);
reportError(error);
process.exit(1);
}
});
program
.command('init')
.description('Write a starter .nopyrc.json and the NOPY.LLM.md usage guide here')
.option('-f, --force', 'Overwrite files that already exist')
.action((options) => {
try {
const results = initProject({ force: options.force });
console.log(formatInitResults(results));
} catch (error) {
reportError(error);
process.exit(1);
}
});
program
.command('create-cube [dir]')
.description('Scaffold a new cube (manifest.mjs + deploy.py) from the bundled templates')
.option('--id <id>', 'Cube id, e.g. net:tailscale')
.option('--name <name>', 'Human-readable cube name')
.option('-f, --force', 'Overwrite existing cube files')
.action(async (dirArg: string | undefined, options) => {
try {
// Config is optional here, unlike `install`: create-cube works in a bare
// directory too; the config only improves the suggested location.
let config: NopyConfig | undefined;
try {
config = loadConfig();
} catch {
config = undefined;
}
const answers = await CubeScaffoldPrompts(
{ id: options.id, name: options.name, dir: dirArg },
(id) => suggestCubeDir(id, config)
);
await assertCubeIdAvailable(answers.id, answers.dir);
const results = createCube({ ...answers, force: options.force });
console.log(
formatCreateCubeResults(results, {
id: answers.id,
warning: cubeDirWarning(answers.dir),
})
);
} catch (error) {
if (isCancellation(error)) exitWithFarewell();
reportError(error);
process.exit(1);
}
});
+62 -2
View File
@@ -114,8 +114,21 @@ export class Variable {
export class Variables {
private readonly store: Record<string, Record<string, Variable>> = {};
private readonly secrets: Record<string, Set<string>> = {};
private readonly schemas: Record<string, Set<string>> = {};
private readonly globalSecrets: Set<string>;
constructor(readonly env: TVariables = {}) {}
/**
* @param env - the `env` block of the merged config, seeded onto every cube
* @param globalSecrets - every key *any* manifest declares secret, plus the
* config's own `secrets` list. Known up front, before the first cube
* resolves, so it does not depend on resolution order.
*/
constructor(
readonly env: TVariables = {},
globalSecrets: Iterable<string> = []
) {
this.globalSecrets = new Set(globalSecrets);
}
/**
* Marks keys of one cube as holding secrets.
@@ -132,8 +145,30 @@ export class Variables {
}
}
/**
* Records which keys a cube's schema declares.
*
* Only {@link bucket} reads this, and only to decide whether a globally
* declared secret may be seeded from `env`. Call it before anything assigns to
* the cube — it deliberately does not create the bucket itself, because
* creating it is what seeds `env`.
*/
declareSchema(cube: string, keys: readonly string[]): void {
this.schemas[cube] ??= new Set<string>();
const declared = this.schemas[cube];
for (const key of keys) declared.add(key);
}
/**
* Whether a key is sensitive for a cube.
*
* True for a key the cube's own manifest declared, and also for one *another*
* manifest declared: a value that is a secret anywhere is a secret everywhere
* it lands. That covers the manifest that lists `PASSWORD` in `schema` and
* forgets it in `secrets`.
*/
isSecret(cube: string, name: string): boolean {
return this.secrets[cube]?.has(name) ?? false;
return (this.secrets[cube]?.has(name) ?? false) || this.globalSecrets.has(name);
}
/** Records values for one cube, all at the same origin. */
@@ -176,6 +211,24 @@ export class Variables {
return values;
}
/**
* The config's `env` block minus anything declared secret — what a session's
* own `env` records.
*
* A session copies `env` verbatim for reference, which quietly undid
* {@link persistable}: a credential declared in `.nopyrc.json` was kept out of
* every cube's `variables` and then written to the same file one key higher up,
* in plaintext, along with a copy in `.nopy.history.json`. Same rule as
* `persistable`, applied to the same file.
*/
persistableEnv(): TVariables {
const values: TVariables = {};
for (const [name, value] of Object.entries(this.env)) {
if (!this.globalSecrets.has(name)) values[name] = value;
}
return values;
}
private create(cube: string, name: string, first: Assignment): Variable {
const variable = new Variable(cube, name, first);
variable.redacted = this.isSecret(cube, name);
@@ -189,6 +242,12 @@ export class Variables {
* rather than a parallel bag merged in at read time. That is what lets it
* carry an origin, show up in the trace, and lose to a prompt by the same rule
* as everything else.
*
* One key is held back: a **secret**, on a cube whose schema does not mention
* it. Broadcasting is otherwise load-bearing — a cube may legitimately read a
* key off `host.data` that it never declared — but a credential does not
* belong on the command line of every unrelated cube in the run, where nothing
* masks it because that cube never declared it sensitive.
*/
private bucket(cube: string): Record<string, Variable> {
const existing = this.store[cube];
@@ -197,6 +256,7 @@ export class Variables {
const bucket: Record<string, Variable> = {};
this.store[cube] = bucket;
for (const [name, value] of Object.entries(this.env)) {
if (this.globalSecrets.has(name) && !this.schemas[cube]?.has(name)) continue;
bucket[name] = this.create(cube, name, { value, origin: 'env' });
}
return bucket;
+12 -3
View File
@@ -6,6 +6,7 @@
import fs from 'node:fs';
import path from 'node:path';
import type { TVariables } from './nopy.common.js';
import { NopyUsageError } from './nopy.errors.js';
/**
* Log verbosity levels for pyinfra output
@@ -102,6 +103,14 @@ export interface NopyConfig {
cubePackages: CubePackageRef[];
/** Global environment variables */
env: TVariables;
/**
* `env` keys to treat as sensitive even though no manifest says so.
*
* A manifest's own `secrets` list already covers the cubes that declare the
* key. This is for the value no cube declares at all — a token a hook reads,
* say — which would otherwise be broadcast and printed in the clear.
*/
secrets?: string[];
/** Logging configuration */
log?: LogConfig;
/** Session history configuration */
@@ -120,7 +129,7 @@ const DEFAULT_CONFIG: NopyConfig = {
env: {},
};
const CONFIG_FILENAME = '.nopyrc.json';
export const CONFIG_FILENAME = '.nopyrc.json';
/**
* Finds all config files by traversing upwards from cwd to root
@@ -317,7 +326,7 @@ export function loadConfig(): NopyConfig {
const configPaths = findConfigFiles();
if (configPaths.length === 0) {
throw new Error(
throw new NopyUsageError(
`No ${CONFIG_FILENAME} found. Create one in your project directory or any parent directory.`
);
}
@@ -334,7 +343,7 @@ export function loadConfig(): NopyConfig {
config = mergeConfigs(config, resolvedConfig);
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
throw new Error(`Failed to load config ${configPath}: ${message}`);
throw new NopyUsageError(`Failed to load config ${configPath}: ${message}`);
}
}
+210
View File
@@ -0,0 +1,210 @@
/**
* Cube scaffolding — `nopy create-cube`
* @module nopy.create-cube
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { findCubeDirectories, loadCubes } from './cubes/index.js';
import type { NopyConfig } from './nopy.config.js';
import { NopyUsageError } from './nopy.errors.js';
import { type InitFileResult, writeGuarded } from './nopy.init.js';
/** What the scaffold writes — the loader's two exact-name candidates. */
export const MANIFEST_FILENAME = 'manifest.mjs';
export const DEPLOY_FILENAME = 'deploy.py';
/**
* Templates resolved relative to this module, so the same path works from
* `src/` (tsx, vitest) and from `dist/` (the build copies `src/templates`
* alongside). Named `*.example.*` because the loader declares any directory
* holding a manifest **and** a deploy script a cube — under their real names
* the template directory itself would be one, and a `cubeDirs` entry sweeping
* this package would deploy the template.
*/
const TEMPLATES: Record<string, URL> = {
[MANIFEST_FILENAME]: new URL('./templates/cube/manifest.example.mjs', import.meta.url),
[DEPLOY_FILENAME]: new URL('./templates/cube/deploy.example.py', import.meta.url),
};
const CUBE_ID_PATTERN = /^[a-z0-9][a-z0-9_.:-]*$/i;
/**
* Why `id` cannot name a cube, or `undefined` when it can. Returns the message
* rather than throwing so a prompt can use it as an inline `validate` while
* {@link createCube} turns it into the error it is.
*/
export function validateCubeId(id: string): string | undefined {
if (!id.trim()) return 'Cube id is required';
if (!CUBE_ID_PATTERN.test(id)) {
return `Cube id may hold letters, digits and ":-_." — like "net:tailscale" or "apt"`;
}
return undefined;
}
/**
* Where the prompt suggests putting a cube: the first configured cube
* directory (falling back to `./cubes`) plus the id with each `:` segment as a
* subdirectory, so `net:tailscale` lands in `cubes/net/tailscale`. Ids are
* flat and need not mirror the path — this is a suggestion, not a rule.
* Returned relative to the working directory when it is under it, because
* that is the form a prompt default should show.
*/
export function suggestCubeDir(id: string, config?: Pick<NopyConfig, 'cubeDirs'>): string {
const base = config?.cubeDirs?.[0] ?? path.resolve(process.cwd(), 'cubes');
const target = path.join(base, ...id.split(':').filter(Boolean));
const relative = path.relative(process.cwd(), target);
return relative.startsWith('..') ? target : relative;
}
/**
* Files that already make `dir` a cube, by the loader's own patterns — not
* just the two exact names the scaffold writes. A `foo.manifest.mjs` already
* present would leave the directory with two manifests and the loader picking
* whichever `readdir` returns first, so it has to block the scaffold too.
*/
function existingCubeFiles(dir: string): string[] {
if (!fs.existsSync(dir)) return [];
return fs
.readdirSync(dir)
.filter(
(name) =>
name === MANIFEST_FILENAME ||
name.endsWith('.manifest.mjs') ||
name === DEPLOY_FILENAME ||
name.endsWith('.deploy.py')
)
.sort();
}
/**
* Escapes a value for splicing into a single-quoted string literal in the
* manifest template — the cube name is free text, and an apostrophe in it
* must not produce a manifest that does not parse.
*/
function jsEscape(value: string): string {
return value.replace(/\\/g, '\\\\').replace(/'/g, "\\'");
}
export interface CreateCubeOptions {
/** Cube id, e.g. `net:tailscale`. */
id: string;
/** Human-readable name, shown in the cube list. */
name: string;
/** Target directory; created if missing. Relative paths resolve against cwd. */
dir: string;
/** Overwrite existing cube files. */
force?: boolean;
}
/**
* Refuses an id another cube already claims — at creation time, rather than
* as the loader's hard duplicate error on the next run. Best-effort: no
* config, or a loader that cannot run, skips the check (the loader still
* catches the collision later). A claim by the target directory itself is the
* `--force` re-scaffold case, not a collision.
*/
export async function assertCubeIdAvailable(id: string, dir: string): Promise<void> {
let cubes: Awaited<ReturnType<typeof loadCubes>>['cubes'];
try {
({ cubes } = await loadCubes());
} catch {
return;
}
const claimant = cubes[id];
if (!claimant || path.resolve(claimant.dir) === path.resolve(dir)) return;
const from =
claimant.source.type === 'package' ? `package ${claimant.source.packageName}` : claimant.dir;
throw new NopyUsageError(`Cube id "${id}" is already claimed by ${from}.`);
}
/**
* The hint when a cube lands where the loader will never look, or `undefined`
* when it is discoverable (or there is no config to consult — a bare
* directory gets the next-steps line about `.nopyrc.json` instead of a
* warning about one that does not exist).
*/
export function cubeDirWarning(dir: string): string | undefined {
let roots: string[];
try {
roots = findCubeDirectories();
} catch {
return undefined;
}
const target = path.resolve(dir);
const inside = roots.some((root) => {
const relative = path.relative(path.resolve(root), target);
return !relative.startsWith('..') && !path.isAbsolute(relative);
});
if (inside) return undefined;
return (
`Note: ${dir} is outside every configured cube directory — ` +
'add it to "cubeDirs" in .nopyrc.json or nopy will not find it.'
);
}
/**
* Scaffolds a cube directory from the bundled templates: `manifest.mjs` with
* the id and name spliced in, plus a minimal `deploy.py`. The result is
* loadable as-is; the schema is an example to replace.
*/
export function createCube(options: CreateCubeOptions): InitFileResult[] {
const idError = validateCubeId(options.id);
if (idError) throw new NopyUsageError(idError);
if (!options.name.trim()) throw new NopyUsageError('Cube name is required');
const dir = path.resolve(options.dir);
const force = options.force ?? false;
const existing = existingCubeFiles(dir);
if (existing.length > 0 && !force) {
throw new NopyUsageError(
`${dir} already holds cube files (${existing.join(', ')}). ` +
`Use --force to overwrite ${MANIFEST_FILENAME} and ${DEPLOY_FILENAME}.`
);
}
fs.mkdirSync(dir, { recursive: true });
return Object.entries(TEMPLATES).map(([filename, url]) => {
// Function replacements, so a `$` in a cube name is never expanded as a
// replacement pattern.
const content = fs
.readFileSync(fileURLToPath(url), 'utf-8')
.replace(/__CUBE_ID__/g, () => jsEscape(options.id))
.replace(/__CUBE_NAME__/g, () => jsEscape(options.name));
return writeGuarded(path.join(dir, filename), content, force);
});
}
/**
* The report `create-cube` prints. Lives here rather than in the CLI because
* the CLI is excluded from coverage.
*/
export function formatCreateCubeResults(
results: InitFileResult[],
options: { id: string; warning?: string }
): string {
const lines = results.map((result) =>
result.status === 'skipped'
? ` exists, skipped ${result.file} (use --force to overwrite)`
: ` ${result.status.padEnd(15)} ${result.file}`
);
lines.push(
'',
'Next steps:',
` 1. Declare the cube's variables in ${MANIFEST_FILENAME} — the schema is an example`,
` 2. Write the deployment in ${DEPLOY_FILENAME}; every schema key arrives on host.data`,
` 3. Run \`nopy\` and select ${options.id}`
);
if (options.warning) lines.push('', options.warning);
return lines.join('\n');
}
+50
View File
@@ -0,0 +1,50 @@
/**
* The errors that are the user's to fix.
* @module nopy.errors
*/
/**
* A run that failed for a reason the user can act on: no config file, a cube
* that does not exist, a required variable nothing supplied, a session file
* that will not load.
*
* The point is the *presentation*, not the control flow — nothing catches this
* to recover. A stack trace through `dist/` says nothing useful about a missing
* `.nopyrc.json`, and printing one invites the reader to look for a bug in nopy
* instead of a typo in their project. The CLI prints the message alone and keeps
* the stack behind `NOPY_DEBUG`.
*
* Mirrors keyman's `UsageError` deliberately: the two CLIs are kept in step on
* how they fail for the same reason their update modules are duplicated rather
* than shared.
*/
export class NopyUsageError extends Error {
constructor(message: string) {
super(message);
this.name = 'NopyUsageError';
}
}
/**
* Reports a failed run in as many lines as it deserves.
*
* A {@link NopyUsageError} prints as one line: it is something the reader can
* fix, and three frames into `dist/` say nothing about a missing `.nopyrc.json`
* except that it looks like a crash in nopy rather than a typo in the project.
* Everything else keeps its stack, because an unexpected failure is exactly when
* one is worth having. `NOPY_DEBUG` forces it for both.
*
* Lives here rather than in `nopy.cli.ts` because the CLI is excluded from
* coverage — it is argv wiring, and this is a decision.
*/
export function reportError(error: unknown): void {
const message = error instanceof Error ? error.message : String(error);
console.error(`Error: ${message}`);
const stack = error instanceof Error ? error.stack : undefined;
const wanted = process.env.NOPY_DEBUG || !(error instanceof NopyUsageError);
if (wanted && stack) console.error(stack);
else if (!process.env.NOPY_DEBUG) console.error('Set NOPY_DEBUG=1 for the full stack trace.');
}
+52 -25
View File
@@ -20,7 +20,11 @@ export interface DeployCall {
host: string;
/** Working directory for execution */
cwd: string;
/** Full command array */
/**
* The command as argv — `command[0]` is the executable, the rest are its
* arguments, one element each and none of them quoted. Nothing joins this to
* run it; {@link maskCommand} joins it to *show* it.
*/
command: string[];
/** Environment variables for the cube */
env: Record<string, unknown>;
@@ -30,6 +34,18 @@ export interface DeployCall {
dependencies: DependencySpec[];
}
/**
* One argv element, quoted for a POSIX shell.
*
* Display only — nothing is executed through a shell any more. The point is
* that what `--print-only` writes can be pasted into a terminal and mean the
* same thing it meant here.
*/
function shellQuote(arg: string): string {
if (arg.length > 0 && /^[\w@%+=:,./-]+$/.test(arg)) return arg;
return `'${arg.replace(/'/g, `'\\''`)}'`;
}
/**
* The command as it is safe to show: the SSH password, and every `--data KEY=…`
* whose key the manifest declared a secret, have their values replaced.
@@ -37,18 +53,37 @@ export interface DeployCall {
* pyinfra takes its data on the command line, so the real values have to be in
* `call.command` — this is the last point before they would reach a log, a
* `--print-only` dump or a dry-run plan.
*
* Walks the argv rather than pattern-matching a joined string. The old version
* bounded a `--data` value on the closing quote the builder had written, which
* tied masking to a quoting convention two modules apart; a value containing a
* `"` broke it, and it was the same brittleness that made the command a shell
* injection in the first place. Position is not guessable.
*/
export function maskCommand(call: DeployCall): string {
const command = call.command.join(' ');
const quoteMeta = (key: string) => key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const secrets = new Set(call.secrets ?? []);
const argv = call.command;
const out: string[] = [];
// The builder always quotes a `--data` value, so the closing quote bounds it.
const masked = (call.secrets ?? []).reduce(
(acc, key) => acc.replace(new RegExp(`(--data "${quoteMeta(key)}=)[^"]*"`, 'g'), `$1${MASK}"`),
command
);
for (let i = 0; i < argv.length; i++) {
const arg = argv[i];
const next = argv[i + 1];
return masked.replace(/(--password )\S+/g, `$1${MASK}`);
if (arg === '--password' && next !== undefined) {
out.push(arg, MASK);
i++;
} else if (arg === '--data' && next !== undefined) {
// Split on the first `=` only: the key cannot contain one, the value can.
const eq = next.indexOf('=');
const key = eq === -1 ? next : next.slice(0, eq);
out.push(arg, secrets.has(key) ? `${key}=${MASK}` : shellQuote(next));
i++;
} else {
out.push(shellQuote(arg));
}
}
return out.join(' ');
}
/**
@@ -107,14 +142,18 @@ export interface ExecutionOptions {
*/
async function executeCall(call: DeployCall): Promise<ExecutionResult> {
const startTime = Date.now();
const commandStr = call.command.join(' ');
const [file, ...args] = call.command;
try {
log.info(`Executing: ${call.cube} -> ${call.host}`);
log.debug(`Command: ${maskCommand(call)}`);
// Inherit stdio for live output
await execa({ shell: true })(commandStr, {
// No shell. `execa({shell: true})` used to run the whole command as one
// string, which made every variable value shell syntax — a password or a
// `--data` value containing `;`, a backtick or `$(…)` was executed rather
// than passed along. Spawning the argv directly removes the parse step;
// pyinfra is still found on PATH, and stdio stays inherited for live output.
await execa(file, args, {
cwd: call.cwd,
stdio: 'inherit',
});
@@ -143,20 +182,8 @@ async function executeCall(call: DeployCall): Promise<ExecutionResult> {
* Outputs the execution plan without running (dry run)
*
* @param calls - Array of deployment calls
* @param asJson - Output as JSON instead of text
*/
export function outputExecutionPlan(calls: DeployCall[], asJson?: boolean): void {
if (asJson) {
const plan = calls.map((call) => ({
cube: call.cube,
host: call.host,
command: maskCommand(call),
variables: maskVariables(call),
}));
console.log(JSON.stringify({ plan }, null, 2));
return;
}
export function outputExecutionPlan(calls: DeployCall[]): void {
console.log('\n=== Execution Plan (Dry Run) ===\n');
for (let i = 0; i < calls.length; i++) {
+2 -2
View File
@@ -4,7 +4,7 @@
*/
/** Parting words. Printed whenever a run ends because the user asked it to. */
export const FAREWELL = 'Bye Bye Honeypie';
export const FAREWELL = 'Bye Bye HoneyPy';
/** Conventional exit code for "terminated by SIGINT" — 128 + 2. */
export const CANCELLED_EXIT_CODE = 130;
@@ -74,7 +74,7 @@ export function restoreTerminal(): void {
* Says goodbye and leaves.
*
* The farewell goes to **stderr**, for the same reason the update hint does:
* `--json` and `--print-only` stay machine-readable no matter how the run ends.
* `--print-only` stays machine-readable no matter how the run ends.
*
* `process.exit` rather than letting the loop drain, because the prompt that
* was cancelled is still holding stdin — after the teardown above threw, its
+2 -31
View File
@@ -5,7 +5,7 @@
import fs from 'node:fs';
import path from 'node:path';
import type { NopySession } from './nopy.session.js';
import { describeSession, type NopySession } from './nopy.session.js';
/** Default number of sessions to keep in history */
export const DEFAULT_HISTORY_SIZE = 10;
@@ -72,35 +72,6 @@ export function saveHistory(history: SessionHistory): void {
fs.writeFileSync(historyPath, JSON.stringify(history, null, 2), 'utf-8');
}
/**
* Generates a history entry name from session data
*
* Format: "YYYY-MM-DD HH:mm - cube1, cube2, ..."
*
* @param session - The session to name
* @param timestamp - ISO timestamp
* @returns Human-readable name
*/
function generateEntryName(session: NopySession, timestamp: string): string {
const date = new Date(timestamp);
const dateStr = date.toLocaleString('en-US', {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
hour12: false,
});
const cubeNames = session.cubes.map((c) => c.key).join(', ');
const truncatedCubes = cubeNames.length > 40 ? `${cubeNames.substring(0, 37)}...` : cubeNames;
const hosts = session.hosts?.join(', ') || 'no host';
const truncatedHosts = hosts.length > 20 ? `${hosts.substring(0, 17)}...` : hosts;
return `${dateStr} - ${truncatedCubes} → ${truncatedHosts}`;
}
/**
* Generates a unique ID for a history entry
*/
@@ -124,7 +95,7 @@ export function addToHistory(
const entry: HistoryEntry = {
id: generateEntryId(),
name: generateEntryName(session, timestamp),
name: describeSession(session, timestamp),
timestamp,
session,
};
+111
View File
@@ -0,0 +1,111 @@
/**
* Project initialization — `nopy init`
* @module nopy.init
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { CONFIG_FILENAME, type NopyConfigFile } from './nopy.config.js';
/** The LLM-facing usage guide `init` drops next to the config. */
export const GUIDE_FILENAME = 'NOPY.LLM.md';
/**
* What a fresh project starts from. `cubePackages` stays empty on purpose:
* naming a bundle is a hard error until it is installed, and `init` must leave
* behind a config that loads.
*/
export const STARTER_CONFIG: NopyConfigFile = {
hosts: [],
cubeDirs: ['./cubes'],
cubePackages: [],
env: {},
log: {
verbosity: 'info',
debug: false,
},
};
/**
* The bundled guide, resolved relative to this module so the same path works
* from `src/` (tsx, vitest) and from `dist/` (the build copies `src/templates`
* alongside the compiled module).
*/
const TEMPLATE_URL = new URL('./templates/NOPY.LLM.md', import.meta.url);
export type InitFileStatus = 'created' | 'overwritten' | 'skipped';
/** One file `init` considered, and what happened to it. */
export interface InitFileResult {
/** Basename, for reporting. */
file: string;
/** Absolute path that was written or left alone. */
path: string;
status: InitFileStatus;
}
export interface InitOptions {
/** Overwrite files that already exist. */
force?: boolean;
/** Target directory (defaults to the working directory). */
dir?: string;
}
/**
* Writes `filePath` unless it already exists and `force` is unset, and says
* which of the three it was. Shared with `create-cube`, which scaffolds under
* the same skip/overwrite rules.
*/
export function writeGuarded(filePath: string, content: string, force: boolean): InitFileResult {
const existed = fs.existsSync(filePath);
if (existed && !force) {
return { file: path.basename(filePath), path: filePath, status: 'skipped' };
}
fs.writeFileSync(filePath, content);
return {
file: path.basename(filePath),
path: filePath,
status: existed ? 'overwritten' : 'created',
};
}
/**
* Writes a starter `.nopyrc.json` and the bundled `NOPY.LLM.md` guide into
* `dir`. Existing files are left alone unless `force` is set; either way the
* result names what happened to each file.
*/
export function initProject(options: InitOptions = {}): InitFileResult[] {
const dir = options.dir ?? process.cwd();
const force = options.force ?? false;
const config = `${JSON.stringify(STARTER_CONFIG, null, 2)}\n`;
const guide = fs.readFileSync(fileURLToPath(TEMPLATE_URL), 'utf-8');
return [
writeGuarded(path.join(dir, CONFIG_FILENAME), config, force),
writeGuarded(path.join(dir, GUIDE_FILENAME), guide, force),
];
}
/**
* The report `nopy init` prints, one line per file plus a next-steps hint.
* Lives here rather than in the CLI because the CLI is excluded from coverage.
*/
export function formatInitResults(results: InitFileResult[]): string {
const lines = results.map((result) =>
result.status === 'skipped'
? ` exists, skipped ${result.file} (use --force to overwrite)`
: ` ${result.status.padEnd(15)} ${result.file}`
);
lines.push(
'',
'Next steps:',
` 1. Add target hosts to "hosts" in ${CONFIG_FILENAME}`,
' 2. Put cubes in ./cubes, or install a bundle and list it under "cubePackages"',
' 3. Run `nopy` to deploy — NOPY.LLM.md explains the rest'
);
return lines.join('\n');
}
+50 -16
View File
@@ -15,11 +15,16 @@ import {
summarizeResults,
} from './nopy.executor.js';
import { addToHistory, DEFAULT_HISTORY_SIZE } from './nopy.history.js';
import { type NopySession, saveSession } from './nopy.session.js';
import { describeSession, type NopySession, SESSION_VERSION, saveSession } from './nopy.session.js';
import { runWorkflow } from './nopy.workflow.js';
/**
* Configures the logtape logger for console output
* Configures the logtape logger for console output.
*
* **stderr**, deliberately. stdout carries the deploy commands and pyinfra's own
* output; everything nopy says about itself goes to stderr, so `--print-only`
* can be piped somewhere. The sink used to write to stdout and was held back
* only by `--json`, which never worked and is gone.
*/
function configureLogtape(): void {
configure({
@@ -31,7 +36,7 @@ function configureLogtape(): void {
if (typeof formatted === 'string') {
const msg = formatted.replace(/\r?\n$/, '');
const props = record.properties as Record<string, unknown>;
console.log(msg, ...Object.values(props));
console.error(msg, ...Object.values(props));
}
};
})(),
@@ -55,7 +60,7 @@ function configureLogtape(): void {
configureLogtape();
/**
* Prints the active configuration summary
* Prints the active configuration summary — to stderr, see {@link configureLogtape}.
*/
function printActiveConfig(
config: import('./nopy.config.js').NopyConfig,
@@ -92,7 +97,7 @@ function printActiveConfig(
}
lines.push('');
console.log(lines.join('\n'));
console.error(lines.join('\n'));
}
/**
@@ -107,7 +112,6 @@ export interface NopyOptions {
dryRun?: boolean;
printOnly?: boolean;
continueOnError?: boolean;
jsonOutput?: boolean;
saveToHistory?: boolean;
}
@@ -138,24 +142,30 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
dryRun = false,
printOnly = false,
continueOnError = false,
jsonOutput = false,
saveToHistory = true,
} = opts;
const log = getLogger(['nopy']);
const config = loadConfig();
if (!jsonOutput && !replaySession && !loadSessionPath) {
if (!replaySession && !loadSessionPath) {
printActiveConfig(config, { continueOnError });
}
const { cubes, errors } = await loadCubes();
const variables = new Variables(config.env);
// Every key any manifest calls a secret, plus the config's own list. Computed
// before the first cube resolves, so which cube happens to run first cannot
// change whether a credential is treated as one.
const declaredSecrets = new Set([
...Object.values(cubes).flatMap((cube) => cube.secrets),
...(config.secrets ?? []),
]);
const variables = new Variables(config.env, declaredSecrets);
if (errors.length > 0) {
log.error('Errors found during cube loading:');
for (const error of errors) log.error(error);
if (jsonOutput) console.log(JSON.stringify({ success: false, errors }, null, 2));
return undefined;
}
@@ -180,7 +190,7 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
},
{
useDefaults,
isSessionReplay: workflow.isReplay,
isSessionReplay: workflow.replaySource !== undefined,
}
);
@@ -190,17 +200,43 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
}
}
// The default name needs the resolved cube list, which does not exist until
// the build has run — so it is filled in here rather than in `createSession`,
// and only when nothing supplied one. `version` sits before the spread so that
// a replayed session keeps whatever its file declared; a hand-written session
// that declared none of the three gets all three.
const timestamp = workflow.session.timestamp ?? new Date().toISOString();
const sessionForSaving: NopySession = {
version: SESSION_VERSION,
...workflow.session,
timestamp,
cubes: context.cubeSessions,
env: config.env,
// Not `config.env` — a declared secret in there would be written to the
// session file in plaintext, one key above the `variables` it was carefully
// kept out of.
env: variables.persistableEnv(),
};
sessionForSaving.name ??= describeSession(sessionForSaving, timestamp);
if (saveSessionPath && !workflow.isReplay) {
// Saved on a replay too: the resolved cube set is exactly what was asked for,
// and a session written from a replay is no less valid than one written from a
// fresh run. The old `!isReplay` guard made `nopy install -R -s out.json` exit
// 0 having written nothing.
if (saveSessionPath) {
saveSession(sessionForSaving, saveSessionPath);
}
if (saveToHistory && !dryRun && !workflow.isReplay && context.deployCalls.length > 0) {
// A `-R`/`-H` replay is already in history and re-recording it would push the
// original out of the list. A `--load-session` run is not in history at all,
// so unless it is recorded here, `nopy history` reports nothing afterwards and
// `-R` has nothing to repeat.
const recordable = workflow.replaySource !== 'history';
// `--print-only` is excluded for the same reason `--dry-run` is: neither
// deployed anything, and history is what `-R` repeats. Recording a run that
// never happened made `nopy install -P` — the safe look-before-you-leap flag —
// silently displace the last real deployment at the head of the list.
if (saveToHistory && !dryRun && !printOnly && recordable && context.deployCalls.length > 0) {
const historySize = config.history?.maxSessions ?? DEFAULT_HISTORY_SIZE;
if (config.history?.autoSave !== false) {
addToHistory(sessionForSaving, historySize);
@@ -224,10 +260,8 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
dryRun,
continueOnError,
onProgress: (result, completed, total) => {
if (!jsonOutput) {
const status = result.success ? '✓' : '✗';
log.info(`[${completed}/${total}] ${status} ${result.cube} -> ${result.host}`);
}
},
});
+159 -20
View File
@@ -9,6 +9,7 @@ import inquirer from 'inquirer';
import type { z } from 'zod';
import { type AnyObjectSchema, type Cube, zodInner, zodKind } from './cubes/index.js';
import type { Variables } from './nopy.common.js';
import { validateCubeId } from './nopy.create-cube.js';
interface CubeChoice {
/** Submitted value — enquirer returns the `name` of each selected choice. */
@@ -17,6 +18,45 @@ interface CubeChoice {
message: string;
}
/** Floor for a terminal that reports a size no prompt could render into. */
const MIN_ROWS = 24;
const MIN_COLS = 80;
/**
* The window size to hand an enquirer prompt, never smaller than {@link MIN_ROWS}.
*
* Load-bearing, not cosmetic. enquirer derives how many choices are visible from
* its height, and `utils.height` (`lib/utils.js:80`) computes a sane fallback and
* then throws it away:
*
* ```js
* let rows = (stream && stream.rows) ? stream.rows : fallback;
* if (stream && typeof stream.getWindowSize === 'function') {
* rows = stream.getWindowSize()[1]; // unconditional
* }
* ```
*
* A TTY always has `getWindowSize`, so a terminal reporting 0 rows — some CI
* pseudo-terminals, `script -q`, an editor terminal mid-startup — yields
* `height: 0`, `Math.min(limit, 0)` choices, and a form that renders nothing and
* submits `{}`. Passing `rows` bypasses that: `prompt.js:396` reads
* `this.options.rows || utils.height(...)`, so the broken function never runs.
*
* Measured on a 0×0 pty: without this the four-field form returns `{}`; with it,
* every field. No effect on a terminal that reports its size honestly.
* enquirer 2.4.1 is its final release, so the bug is not going to be fixed
* upstream.
*/
function terminalSize(out: NodeJS.WriteStream = process.stdout): {
rows: number;
columns: number;
} {
return {
rows: Math.max(out.rows || 0, MIN_ROWS),
columns: Math.max(out.columns || 0, MIN_COLS),
};
}
/**
* Fuzzy-filters the cube list against what the user has typed so far.
*
@@ -52,8 +92,8 @@ export async function CubeSelection(
// Clear terminal and move cursor to top
process.stdout.write('\x1B[2J\x1B[0f');
const terminalHeight = process.stdout.rows || 24;
const pageSize = Math.max(10, terminalHeight - 5);
const size = terminalSize();
const pageSize = Math.max(10, size.rows - 5);
console.log('\n Cube Selection\n');
console.log(' Type to filter • Space to select • Enter to confirm\n');
@@ -65,14 +105,15 @@ export async function CubeSelection(
multiple: true,
choices: cubeChoices,
suggest: suggestCubes,
...size,
});
try {
// Deliberately no catch. Swallowing a cancellation here used to return an
// empty selection, which is indistinguishable from "the user picked nothing"
// and let the run carry on to deploy zero cubes. Both ways out now travel:
// a cancellation to `isCancellation` at the CLI boundary, anything else as
// the failure it is.
return { selectedCubes: await prompt.run() };
} catch {
// User cancelled
return { selectedCubes: [] };
}
}
export async function AuthSelection(useAuthKey?: boolean): Promise<{
@@ -117,6 +158,17 @@ export async function PasswordSelection(username: string): Promise<string> {
return password;
}
/**
* Prompts for the deployment target, normalising the built-ins into the host
* strings pyinfra's connectors expect.
*
* The docker branch takes either identifier the connector accepts, and they
* mean very different things: a **container** name or id is mutated in place
* and left running, while an **image** reference makes pyinfra start a
* throwaway container, apply the deploy, commit the result as a new image and
* print its id. Only the connector can tell the two apart — it looks for a
* matching container first — so the prompt does not try to.
*/
export async function HostSelection(hosts: string[]): Promise<string> {
const selectedHost = await inquirer.prompt([
{
@@ -140,16 +192,66 @@ export async function HostSelection(hosts: string[]): Promise<string> {
},
{
type: 'input',
name: 'dockerContainer',
message: 'Specify docker container name:',
when: (answers) => answers.host === 'runtime:docker',
name: 'dockerTarget',
message: 'Specify docker container name/id, or an image to build from:',
when: (answers) => answers.host === 'docker',
validate: (value: string) => value.trim().length > 0 || 'Required',
},
]);
if (selectedHost.host === 'vagrant') return `@vagrant/${selectedHost.vagrantVM}`;
if (selectedHost.host === 'runtime:docker') return `@docker/${selectedHost.dockerContainer}`;
if (selectedHost.host === 'docker') return `@docker/${selectedHost.dockerTarget.trim()}`;
return selectedHost.customHost ?? selectedHost.host;
}
/** What `create-cube` needs to know before it can scaffold. */
export interface CubeScaffoldAnswers {
id: string;
name: string;
dir: string;
}
/**
* Asks for whatever `create-cube` was not already told on the command line —
* a flag that was given is never re-asked. The directory default is derived
* from the id, which may itself have just been typed, hence the function
* rather than a precomputed value.
*/
export async function CubeScaffoldPrompts(
given: Partial<CubeScaffoldAnswers>,
suggestDir: (id: string) => string
): Promise<CubeScaffoldAnswers> {
const answers = await inquirer.prompt([
{
type: 'input',
name: 'id',
message: 'Cube id (flat, e.g. net:tailscale):',
when: () => !given.id,
validate: (value: string) => validateCubeId(value) ?? true,
},
{
type: 'input',
name: 'name',
message: 'Cube name (the label shown in the cube list):',
when: () => !given.name,
validate: (value: string) => value.trim().length > 0 || 'Required',
},
{
type: 'input',
name: 'dir',
message: 'Directory to scaffold:',
when: () => !given.dir,
default: (soFar: { id?: string }) => suggestDir(given.id ?? soFar.id ?? ''),
validate: (value: string) => value.trim().length > 0 || 'Required',
},
]);
return {
id: given.id ?? answers.id,
name: given.name ?? answers.name,
dir: given.dir ?? answers.dir,
};
}
/**
* Turns a form answer — always a string — back into what the schema declares.
*
@@ -184,6 +286,43 @@ interface FormChoice {
initial: string;
}
/**
* The label to prompt a schema field with: its `.describe()`, read through the
* wrappers that hide it, falling back to the bare key.
*
* In zod 4 a description lives in `z.globalRegistry` keyed by the schema
* *instance*, and `.default()` returns a new `ZodDefault` around the described
* type rather than mutating it. So the wrapper carries no description of its
* own, and the chaining order used to decide whether the label survived:
*
* ```
* z.boolean().describe('Update package cache').default(false) → 'UPDATE'
* z.boolean().default(false).describe('Update package cache') → the sentence
* ```
*
* 15 of the 22 core cubes were written the first way, so most prompts showed a
* bare key. Unwrapping makes the two orders equivalent, which is the answer that
* cannot regress — the alternative was to re-order every manifest and hope the
* next one written gets it right.
*
* Discriminates on {@link zodKind}, not `instanceof`, for the reason given
* there. Falling open here only costs an ugly label, but there is no reason to.
*/
function promptLabel(zodType: unknown, key: string): string {
let current = zodType;
while (current) {
const description = (current as { description?: string }).description;
if (description) return description;
const kind = zodKind(current);
if (kind !== 'default' && kind !== 'optional' && kind !== 'nullable') break;
current = zodInner(current);
}
return key;
}
/**
* Asks the user for a cube's variables and records the answers.
*
@@ -217,19 +356,22 @@ export async function VariableAssignment<S extends AnyObjectSchema>(
if (Object.keys(variablesToConfigure).length === 0) return;
const choices: FormChoice[] = Object.entries(variablesToConfigure).map(([key, value]) => {
const zodType = schema[key];
const description = zodType?.description || key;
return { name: key, message: description, initial: String(value ?? '') };
});
const choices: FormChoice[] = Object.entries(variablesToConfigure).map(([key, value]) => ({
name: key,
message: promptLabel(schema[key], key),
initial: String(value ?? ''),
}));
const form = new (Enquirer as any).Form({
name: 'variables',
message: `[${cube.id}] ${cube.name}\n (↑↓ navigate, Enter to submit)`,
choices,
...terminalSize(),
});
try {
// Deliberately no catch — see `CubeSelection`. A cancelled form used to be
// swallowed here, leaving the cube short of values only the user could give
// and the run continuing as though the form had succeeded.
const result = await form.run();
const coercedResult: Record<string, any> = {};
for (const [key, value] of Object.entries(result)) {
@@ -237,7 +379,4 @@ export async function VariableAssignment<S extends AnyObjectSchema>(
coercedResult[key] = zodType ? coerceValue(value, zodType) : value;
}
variables.assign(cube.id, 'prompt', coercedResult);
} catch {
// User cancelled
}
}
+85 -7
View File
@@ -6,6 +6,7 @@
import fs from 'node:fs';
import path from 'node:path';
import type { TVariables } from './nopy.common.js';
import { NopyUsageError } from './nopy.errors.js';
/**
* Primitive value types that can be stored in session variables
@@ -31,7 +32,13 @@ export interface CubeSession {
* Authentication configuration for a session
*/
export interface AuthSession {
/** Authentication method */
/**
* Authentication method.
*
* `ssh` is not a third kind of credential — it means the connector owns
* authentication and nopy supplies none. It is what an `@vagrant/` or
* `@docker/` host gets, and nothing prompts for it.
*/
method: 'ssh-key' | 'password' | 'ssh';
/** Username for authentication (password auth only) */
username?: string;
@@ -40,8 +47,20 @@ export interface AuthSession {
/**
* Complete session configuration
*
* Everything but `cubes` and `auth` is optional, because a hand-written session
* is a first-class one — the loader requires exactly what it cannot work without.
* `version`, `timestamp` and `name` are stamped on every session nopy writes and
* never demanded of one it reads.
*/
export interface NopySession {
/**
* Format version of the file. Absent on every session written before this was
* stamped, and on most hand-written ones.
*/
version?: string;
/** ISO 8601 time the session was created */
timestamp?: string;
/** Optional session name */
name?: string;
/** Array of cube configurations */
@@ -54,6 +73,40 @@ export interface NopySession {
env?: TVariables;
}
/**
* The format version stamped into every session nopy writes.
*
* There is one, and nothing yet reads it to decide anything — it exists so that
* a future change to the shape can tell an old file from a new one, which is
* impossible after the fact.
*/
export const SESSION_VERSION = '1.0.0';
/**
* A one-line description of a session: `YYYY-MM-DD HH:mm - cubes → hosts`.
*
* Shared with the history list, which is where the format comes from — the two
* name the same thing and there is no reason for them to disagree.
*/
export function describeSession(session: NopySession, timestamp: string): string {
const dateStr = new Date(timestamp).toLocaleString('en-US', {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
hour12: false,
});
const cubeNames = session.cubes.map((c) => c.key).join(', ');
const truncatedCubes = cubeNames.length > 40 ? `${cubeNames.substring(0, 37)}...` : cubeNames;
const hosts = session.hosts?.join(', ') || 'no host';
const truncatedHosts = hosts.length > 20 ? `${hosts.substring(0, 17)}...` : hosts;
return `${dateStr} - ${truncatedCubes} → ${truncatedHosts}`;
}
/**
* Saves a session to a JSON file
*
@@ -129,7 +182,7 @@ function loadSessionFromJSON(filePath: string): NopySession {
*/
export async function loadSession(filePath: string): Promise<NopySession> {
if (!fs.existsSync(filePath)) {
throw new Error(`Session file not found: ${filePath}`);
throw new NopyUsageError(`Session file not found: ${filePath}`);
}
const ext = path.extname(filePath);
@@ -140,23 +193,44 @@ export async function loadSession(filePath: string): Promise<NopySession> {
} else if (ext === '.json') {
session = loadSessionFromJSON(filePath);
} else {
throw new Error(`Unsupported session file format: ${ext}. Use .json or .mjs`);
throw new NopyUsageError(`Unsupported session file format: ${ext}. Use .json or .mjs`);
}
// Validate required fields
if (!session.cubes || !Array.isArray(session.cubes)) {
throw new Error('Invalid session format: missing or invalid "cubes" field');
throw new NopyUsageError('Invalid session format: missing or invalid "cubes" field');
}
if (session.hosts && !Array.isArray(session.hosts)) {
throw new Error('Invalid session format: invalid "hosts" field');
throw new NopyUsageError('Invalid session format: invalid "hosts" field');
}
if (!session.auth) {
throw new Error('Invalid session format: missing "auth" field');
throw new NopyUsageError('Invalid session format: missing "auth" field');
}
// A version this build does not know is a warning, never a refusal: the file
// may well still load, and a session is often the only record of a deployment.
// A missing version says nothing at all — it predates the stamp.
if (session.version !== undefined && session.version !== SESSION_VERSION) {
console.error(
`Warning: session "${filePath}" declares version ${session.version}; ` +
`this build writes ${SESSION_VERSION}. Loading it anyway.`
);
}
return session;
}
/**
* Suffixes {@link listSessions} recognises.
*
* `.nopysession.*` is the documented name and the one the README's examples use;
* it was not matched at all, because `wild.nopysession.json` does not end in
* `.session.json` — the dot before `session` is part of the suffix. The shorter
* pair stays recognised: `saveSession` writes whatever path it is given, so
* files under the old name exist and there is no reason to stop finding them.
*/
const SESSION_SUFFIXES = ['.nopysession.json', '.nopysession.mjs', '.session.json', '.session.mjs'];
/**
* Lists all session files in a directory
*
@@ -170,7 +244,7 @@ export function listSessions(dirPath: string = process.cwd()): string[] {
const files = fs.readdirSync(dirPath);
return files
.filter((file) => file.endsWith('.session.json') || file.endsWith('.session.mjs'))
.filter((file) => SESSION_SUFFIXES.some((suffix) => file.endsWith(suffix)))
.map((file) => path.join(dirPath, file));
}
@@ -186,8 +260,12 @@ export function createSession(params: {
hosts: string[];
auth: AuthSession;
env?: TVariables;
/** Overrides the creation time; for tests, and for re-stamping a replay. */
timestamp?: string;
}): NopySession {
return {
version: SESSION_VERSION,
timestamp: params.timestamp ?? new Date().toISOString(),
name: params.name,
cubes: params.cubes,
hosts: params.hosts,
+15 -5
View File
@@ -35,8 +35,18 @@ export interface WorkflowResult {
username?: string;
/** Password if applicable */
password?: string;
/** Whether this is a session replay */
isReplay: boolean;
/**
* Where a replayed session came from, or `undefined` for a fresh interactive
* run.
*
* Was a boolean, which conflated two runs that need different treatment: a
* `-R`/`-H` replay is already in history and must not be recorded again, while
* a `--load-session` run is not in history at all — recording it is the only
* way `nopy history` and `-R` can see it afterwards. Everything that merely
* asks "am I replaying?" (reading values back off the session rather than
* prompting) takes `replaySource !== undefined`.
*/
replaySource?: 'file' | 'history';
}
/**
@@ -83,7 +93,7 @@ export async function runInteractiveWorkflow(
authMethod: authResult.authMethod,
username: authResult.username,
password: authResult.password,
isReplay: false,
replaySource: undefined,
};
}
@@ -141,7 +151,7 @@ export async function runReplayWorkflow(
authMethod,
username,
password,
isReplay: true,
replaySource: 'file',
};
}
@@ -196,7 +206,7 @@ export async function runSessionReplayWorkflow(
authMethod,
username,
password,
isReplay: true,
replaySource: 'history',
};
}
+398
View File
@@ -0,0 +1,398 @@
# NOPY.LLM.md — nopy for language models
This file was written by `nopy init` and is bundled with the nopy release that
wrote it. It is a working reference for AI assistants (and humans) operating in
a project that deploys with **nopy**. Read it before answering questions about
nopy, before writing or editing a cube, and before planning how to reach a
deployment goal. When this guide and the installed CLI disagree, the CLI wins —
check `nopy --help` and the package README.
## What nopy is
nopy is a CLI that wraps [pyinfra](https://docs.pyinfra.com/) — a Python
infrastructure-as-code tool — in an interactive workflow. Deployments are
organised into **cubes**: self-contained directories holding a JavaScript
manifest (declaring typed input variables, secrets, dependencies, and hooks) and
a plain pyinfra deploy script. nopy discovers cubes, prompts for a target host
and variable values, resolves dependencies into a topological order, and then
runs one `pyinfra` command per cube, sequentially. Every run is recorded and can
be replayed.
nopy does not vendor pyinfra. `pyinfra` must be on `PATH`
(`pipx install pyinfra`), and `docker` / `vagrant` too if those connectors are
used. Node ≥ 22 is required.
## Quick facts
| Thing | Value |
| --- | --- |
| Binary | `nopy` (default subcommand: `install`) |
| Config file | `.nopyrc.json` — cwd upward to `/`, plus `~/.nopyrc.json`, all merged |
| Cube | a directory with `manifest.mjs` + `deploy.py` |
| Session file | `*.nopysession.json` (`--save-session` / `--load-session`) |
| History | `.nopy.history.json` in the working directory — add it to `.gitignore` |
| Update cache | `~/.nopy/update-check.json` |
| Cube marker | a `.npcubes` file makes its directory a cube root |
| Authoring package | `@bitsquare/nopy-cubes` (imported by manifests) |
| Core cube bundle | `@bitsquare/nopy-cubes-core` |
## How to help — a decision guide
When asked to achieve a deployment goal, work through this order:
1. **Find an existing cube.** List the project's cube sources: `cubeDirs` and
`cubePackages` in the merged `.nopyrc.json`, plus any `.npcubes` marker
directories. The core bundle's cubes are listed at the end of this file.
Prefer configuring an existing cube over writing a new one.
2. **Compose cubes.** One run can select several cubes; each cube's declared
dependencies are pulled in automatically and deployed first. Do not
hand-order cubes that already declare their relationship.
3. **Configure, don't fork.** A cube's behaviour is steered by its schema
variables. Project-wide values belong under `env` in `.nopyrc.json`
(they override schema defaults); per-run values come from the prompts.
4. **Write a new cube** only when nothing covers the goal — see
[Authoring a cube](#authoring-a-cube). Keep it small, idempotent, and give
every variable a `.describe()` and (usually) a `.default()`.
5. **Make it repeatable.** For "run this again later": rely on history (`-R`,
`-H <id>`) or record a session file (`-s file.nopysession.json`). For
CI/unattended runs: `nopy install -D` plus values under `env` — see
[CI and unattended runs](#ci-and-unattended-runs).
## CLI reference
`nopy` with no subcommand runs `install`. Everything nopy says about itself
goes to **stderr**; stdout carries only deploy commands and pyinfra's own
output. Exit code is `1` if any cube failed, `0` otherwise.
```
nopy [install] interactive: pick cubes, host, auth, variables
nopy init write a starter .nopyrc.json and this guide (-f overwrites)
nopy create-cube [dir] scaffold a cube (manifest.mjs + deploy.py); prompts for
what --id and --name do not supply (-f overwrites)
nopy history list recorded sessions (--json for machine-readable)
nopy clear-history delete all recorded sessions
nopy self-update update nopy on its release channel (--dry-run, --force,
--channel <latest|next|main>, --registry <url>)
```
`install` flags:
| Flag | Effect |
| --- | --- |
| `-D, --use-defaults` | skip the variable form; values come from defaults, `env`, dependencies |
| `-K, --auth-method-key` | SSH key auth without asking |
| `-R, --repeat-last` | replay the newest history entry |
| `-H, --history <id>` | replay a specific history entry (`nopy history` shows ids) |
| `-s, --save-session <path>` | record the run to a session file |
| `-l, --load-session <path>` | replay a session file |
| `-n, --dry-run` | print the execution plan (commands + variables, secrets masked), run nothing |
| `-P, --print-only` | print only the deploy commands to stdout, run nothing |
| `-c, --continue-on-error` | keep deploying remaining cubes after a failure |
| `--no-save-history` | do not record this run |
Environment variables: `NOPY_DEBUG=1` prints full stack traces;
`NOPY_NO_UPDATE_CHECK=1` (or `CI` being set) disables the daily update check;
`NOPY_REGISTRY`, `NOPY_REGISTRY_TOKEN`, `NOPY_PACKAGE_MANAGER` steer
`self-update`.
## Configuration: `.nopyrc.json`
Every `.nopyrc.json` from the filesystem root down to the working directory,
plus `~/.nopyrc.json`, is merged root-first — the nearer file wins ties. Arrays
concatenate and dedupe, objects deep-merge; a child file can switch a property
to wholesale replacement with `"resolution": { "<property>": "override" }`.
Relative paths in `cubeDirs` resolve against the config file that wrote them,
and each `cubePackages` entry resolves from that file's directory too. If no
config file exists anywhere, `nopy install` refuses to run — `nopy init` fixes
that.
All properties, all optional:
```json
{
"hosts": ["web-01.example.com", "@docker/my-container", "@vagrant/default"],
"cubeDirs": ["./cubes"],
"cubePackages": ["@bitsquare/nopy-cubes-core"],
"env": { "KEY_DIR": "./keys" },
"secrets": ["DEPLOY_TOKEN"],
"log": { "verbosity": "info", "debug": false },
"history": { "maxSessions": 10, "autoSave": true },
"execution": { "continueOnError": false },
"resolution": { "hosts": "override" }
}
```
- **`hosts`** seeds the host picker (see [Hosts](#hosts-connectors-and-auth)).
- **`cubeDirs`** — directories scanned recursively for cubes.
- **`cubePackages`** — installed npm packages that ship cubes in a `cubes/`
directory (or wherever their `package.json` `nopy.cubes` points). Naming a
package that is missing or malformed is a hard error, never a silent skip.
- **`env`** — key/value pairs seeded onto **every** cube in the run, at a
priority above schema defaults. This is how a project pins values and how
`--use-defaults` runs are steered.
- **`secrets`** — `env` keys to treat as sensitive even though no manifest
declares them (masked, never recorded, delivered only to cubes whose schema
names them).
- **`log.verbosity`** — `silent` (default) | `info` (`-v`) | `verbose` (`-vv`)
| `trace` (`-vvv`); **`log.debug`** adds `--debug`. These become pyinfra
flags.
- **`history`** — `maxSessions` (default 10) and `autoSave` (default true).
- **`execution.continueOnError`** — project default for `-c`.
## Cubes
A cube is any directory holding both a manifest (`manifest.mjs` or
`*.manifest.mjs`) and a deploy script (`deploy.py` or `*.deploy.py`).
Discovery unions `cubeDirs`, the cube directories of every `cubePackages`
entry, and every ancestor directory containing a `.npcubes` marker file, then
scans recursively (skipping dot-directories and `node_modules`). Extra files in
a cube directory are ignored by the loader but reachable from the script — **the
deploy script runs with the cube directory as its working directory**.
Cube ids (e.g. `apt:install`, `net:tailscale`) are flat strings claimed
**globally** across all sources. Two cubes with one id abort the run with an
error naming both — there is no shadowing and no precedence. Prefix local cube
ids distinctly when a bundle is also installed. The id need not mirror the
path; it comes from `manifest.id`, falling back to an `[id]` prefix in
`manifest.name`, then the directory basename.
## Authoring a cube
`nopy create-cube --id myapp:caddy-site --name "Serve the app behind Caddy"`
scaffolds the layout below with a loadable example schema to replace — fully
non-interactive when both flags and the directory argument are given.
Layout:
```
cubes/
└── myapp/
└── caddy-site/
├── manifest.mjs
└── deploy.py
```
`manifest.mjs` — ESM, imports from `@bitsquare/nopy-cubes` (a local cube needs
no `node_modules` of its own: when normal resolution fails, nopy resolves
`@bitsquare/nopy-cubes` and `zod` from its own installation):
```javascript
import { Manifest } from '@bitsquare/nopy-cubes';
import { z } from 'zod';
export default Manifest({
id: 'myapp:caddy-site',
name: 'Serve the app behind Caddy',
dependencies: (vars) => ['caddy'], // runs before this cube
secrets: ['API_TOKEN'], // must be schema keys
schema: z.object({
DOMAIN: z.string().describe('Public domain for the site').default('example.com'),
PORT: z.number().describe('Upstream port').default(3000),
API_TOKEN: z.string().describe('Deploy token for the app'), // no default → required
}),
});
```
Schema rules:
- `.describe()` is the prompt label — set it on every field.
- `.default()` gives the field a value at the lowest priority. A field
**without** a default is required: interactive runs prompt for it, and a
`--use-defaults` run fails naming it unless `env` or a dependency supplies
it. Leave defaults off values that must not be guessed (a public key, a real
credential). Defaults may be functions (`.default(() => ...)`).
- `secrets` entries must name schema keys; anything else is a manifest error.
Secrets are masked in all output, never written to sessions or history,
re-prompted on replay, and delivered only to cubes whose schema declares
them. A `.default()` on a secret is plain text in the repo — use a
placeholder like `changeme` or none at all.
- `dependencies` is a function of the *collected* variables, so it can be
conditional. Each entry is an id or `[id, {VAR: value}]` to pass parameters;
passed parameters outrank everything, including the user's prompt answers.
- `before` / `after` are hook arrays: `(ctx, vars) => {}` where
`ctx.exec(id, vars)` schedules another cube (before or after this one).
Use dependencies for static requirements, hooks for conditional
orchestration and explicit parameter passing.
`deploy.py` — a plain pyinfra script. Every schema key is guaranteed present on
`host.data`:
```python
from pyinfra import host
from pyinfra.operations import apt, files, systemd
DOMAIN = str(host.data.DOMAIN)
PORT = host.data.PORT # arrives as int — pyinfra parses --data values
files.template(
name='Write Caddyfile site',
src='Caddyfile.j2', # relative to the cube directory (its cwd)
dest=f'/etc/caddy/sites/{DOMAIN}',
domain=DOMAIN, port=PORT,
_sudo=True,
)
systemd.service(name='Reload caddy', service='caddy', reloaded=True, _sudo=True)
```
**`--data` value coercion**: pyinfra parses values before the script sees them —
`"true"`/`"false"` become booleans, numeric strings become `int`, valid JSON
becomes the parsed structure, everything else stays a string. Wrap in `str()`
before string operations; pass booleans/ints straight through.
**pyinfra essentials**: operations live in `pyinfra.operations.*` (`apt`,
`server`, `files`, `systemd`, `git`, `python`, …) and are declarative — they
gather facts and no-op when the host already matches, so a well-written cube is
idempotent and safe to re-run. Global arguments like `_sudo=True`,
`_env={...}`, `_ignore_errors=True` work on every operation. Facts:
`host.get_fact(...)` from `pyinfra.facts.*`. Full reference:
<https://docs.pyinfra.com/>.
## Variables and precedence
A variable can be assigned from several places in one run; every assignment is
kept and tagged with an **origin**, and the highest-ranked origin wins:
| Rank | Origin | Set by |
| --- | --- | --- |
| 0 | `default` | the schema's `.default()` |
| 1 | `env` | the merged `env` block of `.nopyrc.json` |
| 2 | `session` | a replayed session file or history entry |
| 3 | `prompt` | what the user typed |
| 4 | `param` | a dependency spec or a hook's `exec()` |
Consequences worth knowing:
- `env` beats defaults, so `.nopyrc.json` steers `--use-defaults` runs.
- A recorded session beats current `env` and current defaults — replay is
faithful, not re-derived. Editing a default does not change what a replay
does; record a fresh session to pick it up.
- A key supplied by a dependency (`param`) is never prompted for and never
clobbered by a stale recording.
- Ordinary `env` values reach every cube (a cube may read keys its schema never
declared); declared secrets reach only cubes whose schema names them.
## Hosts, connectors, and auth
The host picker offers the configured `hosts`, a free-form `custom` entry, and
two connector shortcuts:
- **`@docker/<name-or-image>`** — a running container is mutated in place; an
image reference starts a throwaway container, applies the deploy, and commits
the result as a new image. Which one is meant is decided by the docker
connector (container match first).
- **`@vagrant/<machine>`** — deploys into a Vagrant machine.
Connector strings can be written directly into `hosts`. Auth methods: password
(prompts for user + password; becomes `--user <u> --password <p>`, masked in
output, never recorded), SSH key (`-K`; nopy passes nothing — pyinfra uses your
SSH config/agent), and `ssh` (session-recorded value meaning the connector owns
auth — what `@docker/` and `@vagrant/` hosts get, which is why replaying one
asks for nothing).
## Execution model
Per selected cube (dependencies first, post-order = topological order, cycles
reported by name), nopy builds and spawns — without a shell —
```
pyinfra <host> -y [-v|-vv|-vvv] [--debug] [--user U --password P] \
--data KEY=value ... --chdir <cubeDir> <cubeDir>/deploy.py
```
Commands run **sequentially** with inherited stdio, stopping at the first
failure unless `--continue-on-error`. There is no rollback: cubes that already
succeeded stay applied, cubes queued after the failure are skipped and not
reported as failed. A cube already emitted for the same (cube, host) pair is
not emitted twice.
## Sessions, history, and replay
Every completed run (including failed ones) is auto-recorded to
`.nopy.history.json` in the working directory — per-project, newest-first,
rotating at `history.maxSessions`. Not recorded: `--dry-run`, `--print-only`,
`--no-save-history`, empty selections, and `-R`/`-H` replays themselves.
A session records the **full snapshot**: selected cubes with every variable
value they settled on (whatever the origin), hosts, auth method and username.
Never recorded: the SSH password and any declared secret — both re-prompted on
replay. A replay also prompts for the host when none was recorded and for
required keys the schema gained since recording. `-D` combined with a replay
that would have to prompt fails naming the keys instead of deploying a
placeholder.
Session files (`-s` / `-l`) use the same JSON structure as history entries and
are the way to keep a run indefinitely — history rotates. `nopy history --json`
is how scripts find ids for `-H`.
## CI and unattended runs
```sh
nopy install --print-only > plan.txt # the commands, nothing else, stdout only
nopy install -D -K # no prompts: defaults + env, SSH key auth
nopy install -l ci.nopysession.json -D # replay a checked-in session
```
- stdout carries only deploy commands and pyinfra output; all nopy chatter is
stderr. The exit code is the verdict. There is deliberately no `--json` on
`install`.
- Values a `-D` run needs beyond schema defaults go under `env` in
`.nopyrc.json`; sensitive ones also under config `secrets` so they stay
masked and travel only to cubes that declare them.
- Secrets are still visible in the process table while pyinfra runs (`--data`
is argv) and in the prompt UI — `secrets` protects nopy's files and output,
nothing more.
## Troubleshooting
| Symptom | Cause / fix |
| --- | --- |
| `No .nopyrc.json found` | run `nopy init`, or create the file in the project or a parent |
| spawn failure on first deploy | `pyinfra` not on `PATH` — `pipx install pyinfra` |
| `Duplicate cube id '<id>' from 2 sources` | two sources claim one id; rename one or drop a source — there is no precedence |
| cube package errors at startup | a `cubePackages` entry is not installed, has no `cubes/` dir and no `nopy.cubes` override, or points outside itself — all hard errors |
| `cannot run with --use-defaults: <KEYS>` | required keys with no default; set them under `env`, pass from a dependency, or drop `-D` |
| replay aborts `Cube not found: <id>` | the cube was renamed/deleted since recording; the entry is unreplayable |
| replay asks for a value | it is a declared secret (never recorded) or a key added to the schema since the recording |
| variable arrives wrong-typed in Python | pyinfra parsed the `--data` value; `str()` it before string ops |
| error hides its stack | set `NOPY_DEBUG=1` |
## Core cube bundle
`@bitsquare/nopy-cubes-core` ships these cubes (snapshot — enumerate the
installed bundle's `cubes/` directory for the authoritative list). Add it with
`"cubePackages": ["@bitsquare/nopy-cubes-core"]` after installing it into the
project.
| Id | Purpose |
| --- | --- |
| `admin:cockpit` | Cockpit web admin console |
| `admin:hostname` | set the hostname |
| `admin:locale` | configure system locale |
| `apt:essentials` | baseline apt packages (git, curl, ufw, …) |
| `apt:install` | install arbitrary apt packages |
| `armor:fail2ban` | fail2ban hardening |
| `armor:ssh` | SSH daemon hardening |
| `armor:ufw` | UFW firewall rules |
| `caddy` | Caddy web server base install |
| `caddy:spa` | serve a single-page app via Caddy |
| `git:clone` | clone a repository |
| `net:tailscale` | install and authenticate Tailscale |
| `net:wifi:access-point` | configure a Wi-Fi access point |
| `net:wifi:connection` | join a Wi-Fi network |
| `runtime:docker` | install Docker |
| `runtime:nodevm` | install a Node.js runtime |
| `service:autostart` | systemd autostart unit for a command |
| `ssh:authorize` | authorize an SSH public key |
| `ssh:keygen` | generate SSH keys |
| `ssh:keyman` | deploy keys managed by keyman |
| `user:add` | create a user (shell, groups, authorized key) |
| `user:edit` | modify an existing user |
## Further reading
- Installed package README: full CLI walkthrough, secrets semantics, channels.
- `docs/HOOKS.md`, `docs/CUBE-BUNDLES.md`, `docs/SESSION_FORMAT.md`,
`docs/API.md` in the `@bitsquare/nopy` package.
- pyinfra: <https://docs.pyinfra.com/> (operations, facts, global arguments,
connectors).
@@ -0,0 +1,14 @@
# __CUBE_ID__ — __CUBE_NAME__
#
# Runs with the cube directory as its working directory. Every key in the
# manifest's schema arrives on host.data, already parsed by pyinfra — a
# boolean is a bool and a numeric string an int, not a string.
from pyinfra import host
from pyinfra.operations import server
GREETING = host.data.GREETING
server.shell(
name="Print the greeting",
commands=[f"echo '{GREETING}'"],
)
@@ -0,0 +1,15 @@
import { Manifest } from '@bitsquare/nopy-cubes';
import { z } from 'zod';
export default Manifest({
id: '__CUBE_ID__',
name: '__CUBE_NAME__',
// dependencies: () => ['apt:essentials'], // cubes to deploy first
// secrets: ['API_TOKEN'], // schema keys to mask and never persist
schema: z.object({
GREETING: z
.string()
.describe('Message the deploy prints on the host')
.default('hello from __CUBE_ID__'),
}),
});
+63
View File
@@ -200,4 +200,67 @@ describe('Variables secrets', () => {
expect(variables.persistable('cube-a')).toEqual({});
expect(variables.persistable('cube-b')).toEqual({ PASSWORD: 'b' });
});
it('excludes a declared secret from the env a session records', () => {
const variables = new Variables({ PASSWORD: 'hunter2', KEY_DIR: './vault' }, ['PASSWORD']);
expect(variables.persistableEnv()).toEqual({ KEY_DIR: './vault' });
});
it('records an env with no secrets in it whole', () => {
const variables = new Variables({ KEY_DIR: './vault' }, ['PASSWORD']);
expect(variables.persistableEnv()).toEqual({ KEY_DIR: './vault' });
});
});
describe('Variables globally declared secrets', () => {
/** `env` carrying a key that cube-a declares secret and cube-b knows nothing of. */
const withLeakyEnv = () => {
const variables = new Variables({ PASSWORD: 'wildpass123', KEY_DIR: '/vault' }, ['PASSWORD']);
variables.declareSecrets('cube-a', ['PASSWORD']);
variables.declareSchema('cube-a', ['USER', 'PASSWORD']);
variables.declareSchema('cube-b', ['PORT']);
return variables;
};
it('does not seed a secret onto a cube that does not declare it', () => {
const variables = withLeakyEnv();
variables.assign('cube-b', 'default', { PORT: 22 });
expect(variables.get('cube-b')).not.toHaveProperty('PASSWORD');
expect(variables.of('cube-b', 'PASSWORD')).toBeUndefined();
});
it('still seeds it onto a cube whose schema declares it', () => {
const variables = withLeakyEnv();
variables.assign('cube-a', 'default', {});
expect(variables.get('cube-a').PASSWORD).toBe('wildpass123');
expect(variables.of('cube-a', 'PASSWORD')?.origin).toBe('env');
expect(variables.persistable('cube-a')).not.toHaveProperty('PASSWORD');
});
it('keeps broadcasting an undeclared key that is not a secret', () => {
// ssh:keyman reads KEY_DIR off host.data without declaring it in its schema.
const variables = withLeakyEnv();
variables.assign('cube-b', 'default', {});
expect(variables.get('cube-b').KEY_DIR).toBe('/vault');
});
it('redacts a global secret on a cube whose own manifest forgot to list it', () => {
const variables = new Variables({}, ['PASSWORD']);
variables.assign('cube-b', 'prompt', { PASSWORD: 'typed' });
expect(variables.of('cube-b', 'PASSWORD')?.redacted).toBe(true);
expect(variables.persistable('cube-b')).toEqual({});
});
it('treats a cube that declared no schema as declaring nothing', () => {
const variables = new Variables({ PASSWORD: 'p' }, ['PASSWORD']);
variables.assign('cube-z', 'default', {});
expect(variables.get('cube-z')).toEqual({});
});
});
+235
View File
@@ -0,0 +1,235 @@
/**
* Tests for nopy.create-cube.
*
* The contract under test is not "two files appear" but "the loader accepts
* what the scaffold wrote": the round-trip through `loadCubes()` is what
* proves the templates and the loader agree.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { loadCubes } from '../src/cubes/index.js';
import {
assertCubeIdAvailable,
createCube,
cubeDirWarning,
DEPLOY_FILENAME,
formatCreateCubeResults,
MANIFEST_FILENAME,
suggestCubeDir,
validateCubeId,
} from '../src/nopy.create-cube.js';
import { NopyUsageError } from '../src/nopy.errors.js';
let tmpDir: string;
let originalCwd: string;
let originalHome: string | undefined;
beforeEach(() => {
originalCwd = process.cwd();
// realpath: os.tmpdir() is a symlink on macOS, and paths reported back by
// process.cwd() after a chdir are resolved — comparisons need one form.
tmpDir = fs.mkdtempSync(path.join(fs.realpathSync(os.tmpdir()), 'nopy-create-cube-'));
process.chdir(tmpDir);
// Point HOME at an empty directory so a developer's ~/.nopyrc.json cannot
// leak extra cube roots into the "no config anywhere" assertions.
originalHome = process.env.HOME;
process.env.HOME = tmpDir;
});
afterEach(() => {
process.chdir(originalCwd);
process.env.HOME = originalHome;
fs.rmSync(tmpDir, { recursive: true, force: true });
});
function writeConfig(config: Record<string, unknown>, dir = tmpDir): void {
fs.writeFileSync(path.join(dir, '.nopyrc.json'), JSON.stringify(config));
}
describe('validateCubeId', () => {
it('accepts the shapes the core bundle uses', () => {
for (const id of ['apt', 'net:tailscale', 'user:add', 'a1-b_c.d']) {
expect(validateCubeId(id)).toBeUndefined();
}
});
it('names the problem for ids the loader or shell would choke on', () => {
for (const id of ['', ' ', ':leading', 'has space', 'net/tailscale', '[bracketed]']) {
expect(validateCubeId(id)).toBeTypeOf('string');
}
});
});
describe('createCube', () => {
it('writes a manifest and deploy script with every token replaced', () => {
const dir = path.join(tmpDir, 'cubes', 'net', 'hello');
const results = createCube({ id: 'net:hello', name: 'Say hello', dir });
expect(results.map((r) => r.status)).toEqual(['created', 'created']);
expect(results.map((r) => r.file)).toEqual([MANIFEST_FILENAME, DEPLOY_FILENAME]);
for (const file of [MANIFEST_FILENAME, DEPLOY_FILENAME]) {
const content = fs.readFileSync(path.join(dir, file), 'utf-8');
expect(content).not.toContain('__CUBE_ID__');
expect(content).not.toContain('__CUBE_NAME__');
expect(content).toContain('net:hello');
}
expect(fs.readFileSync(path.join(dir, MANIFEST_FILENAME), 'utf-8')).toContain('Say hello');
});
it('rejects an invalid id and an empty name as usage errors', () => {
expect(() => createCube({ id: 'has space', name: 'x', dir: tmpDir })).toThrow(NopyUsageError);
expect(() => createCube({ id: 'ok', name: ' ', dir: tmpDir })).toThrow(NopyUsageError);
});
it('refuses a directory that is already a cube, naming the files', () => {
const dir = path.join(tmpDir, 'occupied');
fs.mkdirSync(dir);
fs.writeFileSync(path.join(dir, 'my.manifest.mjs'), 'export default {}');
fs.writeFileSync(path.join(dir, 'my.deploy.py'), '# deploy');
expect(() => createCube({ id: 'x', name: 'X', dir })).toThrow(/my\.manifest\.mjs/);
// A lone deploy script blocks too — scaffolding next to it would leave the
// loader with two deploy candidates and readdir order picking one.
const half = path.join(tmpDir, 'half');
fs.mkdirSync(half);
fs.writeFileSync(path.join(half, DEPLOY_FILENAME), '# deploy');
expect(() => createCube({ id: 'x', name: 'X', dir: half })).toThrow(NopyUsageError);
});
it('overwrites with force and reports it', () => {
const dir = path.join(tmpDir, 'again');
createCube({ id: 'again', name: 'First', dir });
const results = createCube({ id: 'again', name: 'Second', dir, force: true });
expect(results.map((r) => r.status)).toEqual(['overwritten', 'overwritten']);
expect(fs.readFileSync(path.join(dir, MANIFEST_FILENAME), 'utf-8')).toContain('Second');
});
});
describe('scaffolded cube', () => {
it('is discovered by the loader with the declared id, name and schema', async () => {
writeConfig({ cubeDirs: ['./cubes'] });
createCube({
id: 'net:hello',
// The apostrophe is the point: free text spliced into a single-quoted
// string literal must still parse.
name: "Bob's greeting",
dir: path.join(tmpDir, 'cubes', 'net', 'hello'),
});
const { cubes, errors } = await loadCubes();
expect(errors).toHaveLength(0);
const cube = cubes['net:hello'];
expect(cube).toBeDefined();
expect(cube.name).toBe("Bob's greeting");
expect(cube.schemaKeys()).toContain('GREETING');
expect(cube.getDefaults().GREETING).toContain('net:hello');
});
});
describe('suggestCubeDir', () => {
it('derives a path under ./cubes from the id when there is no config', () => {
expect(suggestCubeDir('net:tailscale')).toBe(path.join('cubes', 'net', 'tailscale'));
});
it('uses the first configured cube directory, relative to cwd when under it', () => {
const config = { cubeDirs: [path.join(tmpDir, 'deploy', 'cubes')] };
expect(suggestCubeDir('apt', config)).toBe(path.join('deploy', 'cubes', 'apt'));
});
it('stays absolute when the cube directory is outside cwd', () => {
const elsewhere = fs.mkdtempSync(path.join(fs.realpathSync(os.tmpdir()), 'nopy-elsewhere-'));
try {
const suggested = suggestCubeDir('apt', { cubeDirs: [elsewhere] });
expect(path.isAbsolute(suggested)).toBe(true);
expect(suggested).toBe(path.join(elsewhere, 'apt'));
} finally {
fs.rmSync(elsewhere, { recursive: true, force: true });
}
});
});
describe('cubeDirWarning', () => {
it('is silent without a config to consult', () => {
expect(cubeDirWarning(path.join(tmpDir, 'cubes', 'x'))).toBeUndefined();
});
it('is silent for a directory the loader will scan', () => {
writeConfig({ cubeDirs: ['./cubes'] });
expect(cubeDirWarning(path.join(tmpDir, 'cubes', 'net', 'x'))).toBeUndefined();
});
it('warns when the loader will never look there', () => {
writeConfig({ cubeDirs: ['./cubes'] });
const outside = path.join(tmpDir, 'elsewhere', 'x');
expect(cubeDirWarning(outside)).toContain('cubeDirs');
});
});
describe('assertCubeIdAvailable', () => {
it('resolves when there is no config to check against', async () => {
await expect(assertCubeIdAvailable('x', path.join(tmpDir, 'x'))).resolves.toBeUndefined();
});
it('resolves for an unclaimed id', async () => {
writeConfig({ cubeDirs: ['./cubes'] });
await expect(
assertCubeIdAvailable('free', path.join(tmpDir, 'cubes', 'free'))
).resolves.toBeUndefined();
});
it('rejects an id another directory already claims', async () => {
writeConfig({ cubeDirs: ['./cubes'] });
createCube({ id: 'taken', name: 'Taken', dir: path.join(tmpDir, 'cubes', 'taken') });
await expect(
assertCubeIdAvailable('taken', path.join(tmpDir, 'cubes', 'other'))
).rejects.toThrow(/already claimed/);
});
it('tolerates the claim coming from the target directory itself', async () => {
writeConfig({ cubeDirs: ['./cubes'] });
const dir = path.join(tmpDir, 'cubes', 'mine');
createCube({ id: 'mine', name: 'Mine', dir });
// The --force re-scaffold case: the id is "claimed", but by the very cube
// being recreated.
await expect(assertCubeIdAvailable('mine', dir)).resolves.toBeUndefined();
});
});
describe('formatCreateCubeResults', () => {
const results = [
{ file: MANIFEST_FILENAME, path: `/x/${MANIFEST_FILENAME}`, status: 'created' as const },
{ file: DEPLOY_FILENAME, path: `/x/${DEPLOY_FILENAME}`, status: 'created' as const },
];
it('reports the files and the next steps', () => {
const output = formatCreateCubeResults(results, { id: 'net:hello' });
expect(output).toContain(MANIFEST_FILENAME);
expect(output).toContain(DEPLOY_FILENAME);
expect(output).toContain('Next steps:');
expect(output).toContain('net:hello');
expect(output).not.toContain('Note:');
});
it('appends the discoverability warning when there is one', () => {
const warning = 'Note: /x is outside every configured cube directory';
expect(formatCreateCubeResults(results, { id: 'x', warning })).toContain(warning);
});
it('points skipped files at --force', () => {
const output = formatCreateCubeResults(
[{ file: MANIFEST_FILENAME, path: `/x/${MANIFEST_FILENAME}`, status: 'skipped' as const }],
{ id: 'x' }
);
expect(output).toContain('exists, skipped');
expect(output).toContain('--force');
});
});
@@ -8,6 +8,7 @@ import { z } from 'zod';
import { BuildContext } from '../src/cubes/dependencies.js';
import { Variables } from '../src/nopy.common.js';
import type { NopyConfig } from '../src/nopy.config.js';
import { NopyUsageError } from '../src/nopy.errors.js';
import type { NopySession } from '../src/nopy.session.js';
vi.mock('../src/nopy.prompts.js', async () => {
@@ -61,6 +62,143 @@ describe('BuildContext error handling', () => {
});
});
describe('BuildContext cycle detection', () => {
const depCube = (id: string, deps: string[]) =>
new Cube(
Manifest.create({ id, name: id, schema: z.object({}), dependencies: () => deps }),
`/test/${id}`,
'deploy.py'
);
const build = (cubes: Cube[]) =>
new BuildContext(
Object.fromEntries(cubes.map((c) => [c.id, c])),
new Variables(),
session(),
config,
{ method: 'ssh' },
{ useDefaults: true }
);
it('rejects a cube that depends on itself', async () => {
const context = build([depCube('cube-a', ['cube-a'])]);
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
'Circular dependency on host1: cube-a → cube-a'
);
});
it('names the whole path of a longer loop', async () => {
const context = build([
depCube('cube-a', ['cube-b']),
depCube('cube-b', ['cube-c']),
depCube('cube-c', ['cube-a']),
]);
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
'Circular dependency on host1: cube-a → cube-b → cube-c → cube-a'
);
});
it('reports the loop as usage rather than overflowing the stack', async () => {
const context = build([depCube('cube-a', ['cube-b']), depCube('cube-b', ['cube-a'])]);
// The point of the finding: before the guard this recursed until V8 gave up,
// and a RangeError names no cube and reads as a nopy crash rather than a
// manifest that says something impossible.
await expect(context.resolveCube('cube-a', 'host1')).rejects.toBeInstanceOf(NopyUsageError);
expect(context.deployCalls).toHaveLength(0);
});
it('catches a loop a hook closes rather than a dependency', async () => {
const a = new Cube(
Manifest.create({
id: 'cube-a',
name: 'A',
schema: z.object({}),
dependencies: () => ['cube-b'],
}),
'/test/cube-a',
'deploy.py'
);
const b = new Cube(
Manifest.create({
id: 'cube-b',
name: 'B',
schema: z.object({}),
before: [async (ctx) => ctx.exec('cube-a', {})],
}),
'/test/cube-b',
'deploy.py'
);
await expect(build([a, b]).resolveCube('cube-a', 'host1')).rejects.toThrow(
'Circular dependency on host1: cube-a → cube-b → cube-a'
);
});
it('allows a diamond, where the shared cube is entered twice but never nested', async () => {
const context = build([
depCube('top', ['left', 'right']),
depCube('left', ['shared']),
depCube('right', ['shared']),
depCube('shared', []),
]);
await context.resolveCube('top', 'host1');
expect(context.deployCalls.map((c) => c.cube)).toEqual(['shared', 'left', 'right', 'top']);
});
it('allows the same cube on a different host', async () => {
const context = build([depCube('cube-a', [])]);
await context.resolveCube('cube-a', 'host1');
await context.resolveCube('cube-a', 'host2');
expect(context.deployCalls.map((c) => c.host)).toEqual(['host1', 'host2']);
});
});
describe('BuildContext log configuration', () => {
const build = (log: NopyConfig['log']) =>
new BuildContext(
{ 'cube-a': testCube('cube-a') },
new Variables(),
session(),
{ env: {}, log } as NopyConfig,
{ method: 'ssh' },
{ useDefaults: true }
);
it('passes the configured verbosity and debug flags to pyinfra', async () => {
const context = build({ verbosity: 'verbose', debug: true });
await context.resolveCube('cube-a', 'host1');
expect(context.deployCalls[0].command.slice(0, 5)).toEqual([
'pyinfra',
'host1',
'-y',
'-vv',
'--debug',
]);
});
it('adds nothing when no log config is set', async () => {
const context = build(undefined);
await context.resolveCube('cube-a', 'host1');
expect(context.deployCalls[0].command.slice(0, 4)).toEqual([
'pyinfra',
'host1',
'-y',
'--chdir',
]);
});
});
describe('BuildContext session replay', () => {
it('takes variables from the session instead of prompting', async () => {
const cube = testCube('cube-a', z.object({ PORT: z.string().default('3000') }));
@@ -129,10 +267,15 @@ describe('BuildContext session replay', () => {
});
describe('BuildContext replay gaps', () => {
const replay = (cube: Cube, recorded: Record<string, string> = {}, options = {}) =>
const replay = (
cube: Cube,
recorded: Record<string, string> = {},
options = {},
variables = new Variables()
) =>
new BuildContext(
{ [cube.id]: cube },
new Variables(),
variables,
session([{ key: cube.id, variables: recorded }]),
config,
{ method: 'ssh' },
@@ -175,16 +318,18 @@ describe('BuildContext replay gaps', () => {
expect(VariableAssignment).not.toHaveBeenCalled();
});
it('refuses to deploy when the form was cancelled', async () => {
it('refuses to deploy when the form came back empty', async () => {
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
// The real VariableAssignment swallows a cancelled form, so the gap check
// has to run again afterwards or the cube ships without the variable.
// A form that resolves is not proof of an answer: enquirer renders
// `Math.min(limit, height)` fields, so a terminal misreporting its height
// submits `{}` without the user having seen a question. The gap check has
// to run again afterwards or the cube ships without the variable.
vi.mocked(VariableAssignment).mockResolvedValue(undefined);
const context = replay(cube);
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
'Cube "cube-a" is missing SSID and cannot be deployed.'
'Cube "cube-a" is missing SSID. Nothing supplied it'
);
expect(context.deployCalls).toHaveLength(0);
});
@@ -194,10 +339,36 @@ describe('BuildContext replay gaps', () => {
const context = replay(cube, {}, { useDefaults: true });
// A schema default is deliberately not good enough for a secret: it would
// deploy a different credential than the run being replayed.
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
/cannot be replayed with --use-defaults: PASSWORD/
/cannot be replayed with --use-defaults: PASSWORD would have to be entered\..*not accepted for a secret/s
);
});
it('accepts a secret supplied through config env under --use-defaults', async () => {
const cube = secretCube('cube-a', z.object({ PASSWORD: z.string().default('changeme') }));
const context = replay(
cube,
{},
{ useDefaults: true },
new Variables({ PASSWORD: 'from-env' }, ['PASSWORD'])
);
await context.resolveCube('cube-a', 'host1');
expect(VariableAssignment).not.toHaveBeenCalled();
expect(context.deployCalls[0].env.PASSWORD).toBe('from-env');
});
it('accepts a required variable a dependency passed under --use-defaults', async () => {
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
const context = replay(cube, {}, { useDefaults: true });
await context.resolveCube('cube-a', 'host1', { SSID: 'from-param' });
expect(context.deployCalls[0].env.SSID).toBe('from-param');
});
});
describe('BuildContext session recording', () => {
@@ -240,6 +411,50 @@ describe('BuildContext session recording', () => {
});
});
describe('BuildContext secret broadcast', () => {
// The field run put PASSWORD under `env` because the docs said to, and watched
// it appear unmasked on the command line of every cube that was not user:add.
const resolveBoth = async () => {
const declaring = secretCube('cube-a', z.object({ PASSWORD: z.string().default('changeme') }));
const innocent = testCube('cube-b', z.object({ PORT: z.string().default('22') }));
const context = new BuildContext(
{ 'cube-a': declaring, 'cube-b': innocent },
new Variables({ PASSWORD: 'wildpass123', KEY_DIR: '/vault' }, ['PASSWORD']),
session(),
config,
{ method: 'ssh' },
{ useDefaults: true }
);
await context.resolveCube('cube-a', 'host1');
await context.resolveCube('cube-b', 'host1');
return context;
};
it('never puts an env secret on a cube that does not declare it', async () => {
const context = await resolveBoth();
const [, forB] = context.deployCalls;
expect(forB.cube).toBe('cube-b');
expect(forB.env).not.toHaveProperty('PASSWORD');
expect(forB.command.join(' ')).not.toContain('wildpass123');
});
it('still delivers it to the cube that declares it', async () => {
const context = await resolveBoth();
const [forA] = context.deployCalls;
expect(forA.env.PASSWORD).toBe('wildpass123');
expect(forA.secrets).toEqual(['PASSWORD']);
});
it('leaves an ordinary env key broadcast to both', async () => {
const context = await resolveBoth();
expect(context.deployCalls.map((call) => call.env.KEY_DIR)).toEqual(['/vault', '/vault']);
});
});
describe('BuildContext --use-defaults', () => {
const withDefaults = (cube: Cube, variables = new Variables(), cfg = config) =>
new BuildContext(
@@ -267,7 +482,7 @@ describe('BuildContext --use-defaults', () => {
await context.resolveCube('cube-a', 'host1');
expect(context.deployCalls[0].command.join(' ')).toContain('--data "PORT=8080"');
expect(context.deployCalls[0].command).toContain('PORT=8080');
});
it('refuses to run a cube whose variable nothing can supply', async () => {
@@ -333,6 +548,38 @@ describe('BuildContext --use-defaults', () => {
});
});
describe('BuildContext interactive completeness', () => {
const interactive = (cube: Cube, variables = new Variables()) =>
new BuildContext({ [cube.id]: cube }, variables, session(), config, { method: 'ssh' });
it('refuses to deploy when the form submitted nothing', async () => {
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
// What a 0-row terminal does: the form renders no fields, the user sees no
// question, enquirer resolves `{}` and the run used to carry on and deploy
// the cube with SSID simply absent from `--data`.
vi.mocked(VariableAssignment).mockResolvedValue(undefined);
const context = interactive(cube);
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
/Cube "cube-a" is missing SSID\. Nothing supplied it/
);
expect(context.deployCalls).toHaveLength(0);
});
it('deploys when the form answered', async () => {
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
vi.mocked(VariableAssignment).mockImplementation(async (_cube, variables) => {
variables.assign('cube-a', 'prompt', { SSID: 'typed' });
});
const context = interactive(cube);
await context.resolveCube('cube-a', 'host1');
expect(context.deployCalls[0].env.SSID).toBe('typed');
});
});
describe('BuildContext command construction', () => {
const build = (auth: { method: string; username?: string; password?: string }) => {
const context = new BuildContext(
@@ -376,14 +623,30 @@ describe('BuildContext command construction', () => {
});
await context.resolveCube('cube-a', 'host1');
const command = context.deployCalls[0].command.join(' ');
const command = context.deployCalls[0].command;
expect(command).toContain('--data "PORT=3000"');
expect(command).toContain('--chdir /test/cube-a');
// argv, not a shell string: each flag and its value are separate elements,
// and nothing is pre-quoted.
expect(command).toContain('PORT=3000');
expect(command.join(' ')).toContain('--data PORT=3000');
expect(command.join(' ')).toContain('--chdir /test/cube-a');
expect(command).toContain('/test/cube-a/deploy.py');
expect(context.deployCalls[0].cwd).toBe('/test/cube-a');
});
it('keeps a value with shell metacharacters in one argv element', async () => {
// The whole point of dropping `shell: true`. Joined and handed to a shell,
// this value would have run `id` and swallowed the rest of the command.
const cube = testCube('cube-a', z.object({ MOTD: z.string().default('$(id); rm -rf /') }));
const context = new BuildContext({ 'cube-a': cube }, new Variables(), session(), config, {
method: 'ssh',
});
await context.resolveCube('cube-a', 'host1');
expect(context.deployCalls[0].command).toContain('MOTD=$(id); rm -rf /');
});
it('builds a separate call per host but records the cube session once', async () => {
const context = new BuildContext(
{ 'cube-a': testCube('cube-a') },
@@ -125,3 +125,53 @@ describe('BuildContext.resolveCube', () => {
expect(context.deployCalls.map((c) => c.cube)).toEqual(['cube-a', 'cube-b', 'cube-c']);
});
});
describe('deploy order across several selected cubes', () => {
// `nopy.main.ts` walks `workflow.selectedCubes` and calls `resolveCube` once
// per entry, so the order that list arrives in is the order the loop visits.
// Emission is post-order, though, so a declared edge is honoured whichever way
// round the two cubes were listed — the recursion *is* the topological sort,
// and these pin that rather than leaving it to be inferred from the one-root
// cases above.
async function resolveAll(cubes: Record<string, Cube>, selected: string[]): Promise<string[]> {
const context = new BuildContext(
cubes,
new Variables(),
{ cubes: [] } as any,
{
env: {},
} as any,
{ method: 'ssh' }
);
for (const id of selected) await context.resolveCube(id, 'host1');
return context.deployCalls.map((c) => c.cube);
}
it('emits a dependency first even when it is selected last', async () => {
const cubes = {
'cube-a': createTestCube('cube-a'),
'cube-b': createTestCube('cube-b', () => ['cube-a']),
};
// The list order is the inversion of the dependency: b depends on a, and a
// is named after it. Resolving b still drags a in ahead of itself, and the
// second visit is deduped rather than re-emitted at the tail.
expect(await resolveAll(cubes, ['cube-b', 'cube-a'])).toEqual(['cube-a', 'cube-b']);
expect(await resolveAll(cubes, ['cube-a', 'cube-b'])).toEqual(['cube-a', 'cube-b']);
});
it('interleaves an unrelated cube by list order and nothing else', async () => {
// With no edge between them there is nothing to sort on, so `cube-z` lands
// where the list put it. That is the whole of what selection order decides.
const cubes = {
'cube-a': createTestCube('cube-a'),
'cube-b': createTestCube('cube-b', () => ['cube-a']),
'cube-z': createTestCube('cube-z'),
};
expect(await resolveAll(cubes, ['cube-z', 'cube-b'])).toEqual(['cube-z', 'cube-a', 'cube-b']);
expect(await resolveAll(cubes, ['cube-b', 'cube-z'])).toEqual(['cube-a', 'cube-b', 'cube-z']);
});
});
+62
View File
@@ -0,0 +1,62 @@
/**
* Tests for nopy.errors — how a failed run is presented.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { NopyUsageError, reportError } from '../src/nopy.errors.js';
let out: ReturnType<typeof vi.spyOn>;
let err: ReturnType<typeof vi.spyOn>;
/** Everything written to stderr by the last call, as one string. */
const stderr = () => err.mock.calls.map((call) => String(call[0])).join('\n');
beforeEach(() => {
out = vi.spyOn(console, 'log').mockImplementation(() => {});
err = vi.spyOn(console, 'error').mockImplementation(() => {});
delete process.env.NOPY_DEBUG;
});
afterEach(() => {
vi.restoreAllMocks();
});
describe('reportError', () => {
it('prints a usage error as one line and points at the debug switch', () => {
reportError(new NopyUsageError('No .nopyrc.json found'));
expect(stderr()).toContain('Error: No .nopyrc.json found');
expect(stderr()).not.toContain('nopy.errors');
expect(stderr()).toContain('NOPY_DEBUG=1');
});
it('keeps the stack for anything unexpected', () => {
reportError(new TypeError('cannot read properties of undefined'));
expect(stderr()).toContain('Error: cannot read properties of undefined');
expect(stderr()).toContain('TypeError: cannot read properties of undefined\n at ');
expect(stderr()).not.toContain('NOPY_DEBUG=1');
});
it('prints the stack of a usage error under NOPY_DEBUG', () => {
process.env.NOPY_DEBUG = '1';
reportError(new NopyUsageError('No .nopyrc.json found'));
expect(stderr()).toContain('NopyUsageError: No .nopyrc.json found\n at ');
expect(stderr()).not.toContain('NOPY_DEBUG=1 for');
});
it('reports a thrown non-error', () => {
reportError('just a string');
expect(stderr()).toContain('Error: just a string');
expect(stderr()).toContain('NOPY_DEBUG=1');
});
it('says nothing on stdout', () => {
reportError(new NopyUsageError('No .nopyrc.json found'));
expect(out).not.toHaveBeenCalled();
});
});
+18 -7
View File
@@ -1,9 +1,9 @@
/**
* Tests for the executeDeployCalls path of nopy.executor.
*
* execa is mocked so no pyinfra process is ever spawned. Note the shape:
* the module calls execa({ shell: true })(command, opts), so the mock is a
* factory returning the runner.
* execa is mocked so no pyinfra process is ever spawned. The module calls
* `execa(file, args, opts)` directly — no shell, so no factory call to unwrap
* as there was while it went through `execa({ shell: true })`.
*/
import { beforeEach, describe, expect, it, vi } from 'vitest';
@@ -11,7 +11,7 @@ import { beforeEach, describe, expect, it, vi } from 'vitest';
const runner = vi.fn();
vi.mock('execa', () => ({
execa: vi.fn(() => runner),
execa: vi.fn((...args: unknown[]) => runner(...args)),
}));
import { execa } from 'execa';
@@ -50,16 +50,27 @@ describe('executeDeployCalls', () => {
logSpy.mockRestore();
});
it('runs the joined command in the call cwd with inherited stdio', async () => {
it('spawns the argv directly in the call cwd with inherited stdio', async () => {
await executeDeployCalls([call('cube-a')]);
expect(execa).toHaveBeenCalledWith({ shell: true });
expect(runner).toHaveBeenCalledWith('pyinfra web-1 -y cube-a.deploy.py', {
expect(execa).toHaveBeenCalledWith('pyinfra', ['web-1', '-y', 'cube-a.deploy.py'], {
cwd: '/cubes/cube-a',
stdio: 'inherit',
});
});
it('never asks execa for a shell', async () => {
// The regression that matters: with `shell: true` every `--data` value was
// shell syntax, so a password or a variable containing `;` or `$(…)` ran.
await executeDeployCalls([
{ ...call('cube-a'), command: ['pyinfra', 'web-1', '--data', 'MOTD=$(id); rm -rf /'] },
]);
const [, , options] = vi.mocked(execa).mock.calls[0] as unknown[];
expect(options).not.toHaveProperty('shell');
expect(vi.mocked(execa).mock.calls[0][1]).toContain('MOTD=$(id); rm -rf /');
});
it('reports success with a non-negative duration', async () => {
const [result] = await executeDeployCalls([call('cube-a')]);
+48 -25
View File
@@ -123,24 +123,10 @@ describe('outputExecutionPlan', () => {
expect(output).toContain('host1');
});
it('outputs JSON format when requested', () => {
const calls = [createTestCall('cube-a', 'host1')];
outputExecutionPlan(calls, true);
expect(consoleLogSpy).toHaveBeenCalledTimes(1);
const output = consoleLogSpy.mock.calls[0][0];
const parsed = JSON.parse(output);
expect(parsed.plan).toHaveLength(1);
expect(parsed.plan[0].cube).toBe('cube-a');
expect(parsed.plan[0].host).toBe('host1');
});
it('masks variables the manifest declared secret', () => {
const call: DeployCall = {
...createTestCall('cube-a', 'host1'),
command: ['pyinfra', 'host1', '-y', '--data "PASSWORD=hunter2"', '--data "OTHER=visible"'],
command: ['pyinfra', 'host1', '-y', '--data', 'PASSWORD=hunter2', '--data', 'OTHER=visible'],
env: { PASSWORD: 'hunter2', OTHER: 'visible' },
secrets: ['PASSWORD'],
};
@@ -200,34 +186,47 @@ describe('maskCommand', () => {
it('replaces the value of a declared secret', () => {
const masked = maskCommand(
call(['pyinfra', 'host1', '--data "PASSWORD=hunter2"'], ['PASSWORD'])
call(['pyinfra', 'host1', '--data', 'PASSWORD=hunter2'], ['PASSWORD'])
);
expect(masked).toBe('pyinfra host1 --data "PASSWORD=********"');
expect(masked).toBe('pyinfra host1 --data PASSWORD=********');
});
it('leaves other data alone', () => {
const masked = maskCommand(
call(['--data "SSID=home"', '--data "PASSWORD=hunter2"'], ['PASSWORD'])
call(['--data', 'SSID=home', '--data', 'PASSWORD=hunter2'], ['PASSWORD'])
);
expect(masked).toBe('--data "SSID=home" --data "PASSWORD=********"');
expect(masked).toBe('--data SSID=home --data PASSWORD=********');
});
it('masks a value containing spaces up to the closing quote', () => {
const masked = maskCommand(call(['--data "PASSWORD=two words"', '--chdir /x'], ['PASSWORD']));
it('masks a value containing spaces', () => {
const masked = maskCommand(
call(['--data', 'PASSWORD=two words', '--chdir', '/x'], ['PASSWORD'])
);
expect(masked).toBe('--data "PASSWORD=********" --chdir /x');
expect(masked).toBe('--data PASSWORD=******** --chdir /x');
});
it('masks an empty secret value', () => {
expect(maskCommand(call(['--data "PASSWORD="'], ['PASSWORD']))).toBe(
'--data "PASSWORD=********"'
expect(maskCommand(call(['--data', 'PASSWORD='], ['PASSWORD']))).toBe(
'--data PASSWORD=********'
);
});
it('masks a secret whose value contains a quote', () => {
// The old implementation bounded the value on the closing `"` the builder
// had written, so a value containing one leaked the rest of itself.
const masked = maskCommand(call(['--data', 'PASSWORD=he said "hi"'], ['PASSWORD']));
expect(masked).toBe('--data PASSWORD=********');
expect(masked).not.toContain('hi');
});
it('masks the ssh password whether or not the cube declares secrets', () => {
const masked = maskCommand(call(['pyinfra', 'host1', '--user bob --password s3cr3t', '-y']));
const masked = maskCommand(
call(['pyinfra', 'host1', '--user', 'bob', '--password', 's3cr3t', '-y'])
);
expect(masked).toBe('pyinfra host1 --user bob --password ******** -y');
});
@@ -235,4 +234,28 @@ describe('maskCommand', () => {
it('returns the command untouched when there is nothing to hide', () => {
expect(maskCommand(call(['pyinfra', 'host1', '-y']))).toBe('pyinfra host1 -y');
});
it('quotes an argument a shell would otherwise re-parse', () => {
// Display only — nothing runs through a shell — but `--print-only` output is
// meant to be pasteable, so it has to survive the round trip.
const masked = maskCommand(call(['--data', 'MOTD=$(id); rm -rf /']));
expect(masked).toBe(`--data 'MOTD=$(id); rm -rf /'`);
});
it('escapes an embedded single quote', () => {
expect(maskCommand(call(['--data', "NAME=o'brien"]))).toBe(`--data 'NAME=o'\\''brien'`);
});
it('quotes an empty argument rather than dropping it', () => {
expect(maskCommand(call(['pyinfra', '']))).toBe("pyinfra ''");
});
it('leaves a trailing --password with no value alone', () => {
expect(maskCommand(call(['pyinfra', '--password']))).toBe('pyinfra --password');
});
it('handles a --data argument with no equals sign', () => {
expect(maskCommand(call(['--data', 'BARE'], ['BARE']))).toBe('--data BARE=********');
});
});
+42
View File
@@ -0,0 +1,42 @@
/**
* Runs one real enquirer variable form and prints what it produced.
*
* Driven by `tests/prompts.pty.test.ts` under a pty of a chosen size. Nothing
* here is mocked — the point is the prompt library's own behaviour on a
* terminal that reports no size, which cannot be observed from inside a vitest
* worker because there is no TTY there to misreport.
*
* Prints one line, `NOPY_PROBE <json>`, holding the values the form assigned at
* the `prompt` origin. An empty object means the form submitted nothing.
*/
import { Cube, Manifest } from '@bitsquare/nopy-cubes';
import { z } from 'zod';
import { Variables } from '../../src/nopy.common.js';
import { VariableAssignment } from '../../src/nopy.prompts.js';
const KEYS = ['ALPHA', 'BETA'];
const cube = new Cube(
Manifest({
id: 'probe',
name: 'Zero-rows probe',
schema: z.object({
ALPHA: z.string().describe('First value').default(''),
BETA: z.string().describe('Second value').default(''),
}),
}),
'/cubes/probe',
'deploy.py'
);
const variables = new Variables();
await VariableAssignment(cube, variables);
const assigned: Record<string, unknown> = {};
for (const key of KEYS) {
const variable = variables.of('probe', key);
if (variable?.origin === 'prompt') assigned[key] = variable.value;
}
process.stdout.write(`\nNOPY_PROBE ${JSON.stringify(assigned)}\n`);

Some files were not shown because too many files have changed in this diff Show More