Compare commits
12
Commits
da84523a6d
...
eac44b637e
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
eac44b637e | ||
|
|
a28b75f522 | ||
|
|
41f4e49aa6 | ||
|
|
05f2d6aa56 | ||
|
|
b5702e423a | ||
|
|
89450cb7bc | ||
|
|
09554b6785 | ||
|
|
bb8b1bfa5c | ||
|
|
4daf27a3cd | ||
|
|
2019626618 | ||
|
|
0aa0be5542 | ||
|
|
ac7ea07e3c |
@@ -125,32 +125,14 @@ jobs:
|
|||||||
- name: Install
|
- name: Install
|
||||||
run: pnpm install --frozen-lockfile
|
run: pnpm install --frozen-lockfile
|
||||||
|
|
||||||
- name: Check the linked workspace packages are already released
|
# There used to be a *check linked deps are released* step here, refusing
|
||||||
env:
|
# to publish a package whose `workspace:` dependency was not yet on npmjs.
|
||||||
NAME: ${{ steps.target.outputs.name }}
|
# It was removed: `scripts/release.mjs` is what creates release tags now,
|
||||||
DIR: ${{ steps.target.outputs.dir }}
|
# and it already pushes them dependency-first and waits for each version to
|
||||||
run: |
|
# resolve on npmjs before pushing the next — so the ordering is enforced
|
||||||
set -euo pipefail
|
# before CI ever sees a tag, rather than after. `node scripts/linked-deps.mjs
|
||||||
# `pnpm publish` turns `workspace:*` into the version the linked
|
# <dir>` still prints what a package would bake in, if you want to check by
|
||||||
# package declares at this commit. If that version is not on the
|
# hand. A tag pushed some other way is no longer caught.
|
||||||
# registry yet, the release installs to a broken tree — and npmjs
|
|
||||||
# only lets you unpublish for 72 hours. Release the dependency first:
|
|
||||||
# nopy-cubes, then nopy, then any bundle.
|
|
||||||
#
|
|
||||||
# npmjs only: it is the irreversible one, and it needs no credentials
|
|
||||||
# to read, which this step does not have yet.
|
|
||||||
missing=0
|
|
||||||
for spec in $(node scripts/linked-deps.mjs "$DIR" | tr ' ' '@'); do
|
|
||||||
# Scoped, not `--registry`: `@scope:registry` outranks it, so a bare
|
|
||||||
# flag can be silently overridden by any project-level .npmrc.
|
|
||||||
if npm view "$spec" version --@bitsquare:registry="$NPMJS_REGISTRY" >/dev/null 2>&1; then
|
|
||||||
echo "${spec} is published"
|
|
||||||
else
|
|
||||||
echo "::error::${NAME} depends on ${spec}, which is not on npmjs. Release it first."
|
|
||||||
missing=1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
exit "$missing"
|
|
||||||
|
|
||||||
- name: Lint
|
- name: Lint
|
||||||
run: pnpm run lint:ci
|
run: pnpm run lint:ci
|
||||||
|
|||||||
@@ -4,6 +4,9 @@
|
|||||||
# anyway (see the Vagrantfile).
|
# anyway (see the Vagrantfile).
|
||||||
.vagrant-hostkeys
|
.vagrant-hostkeys
|
||||||
.python-version
|
.python-version
|
||||||
|
# The pty drivers under scripts/ import each other, so running one leaves a
|
||||||
|
# bytecode cache next to them.
|
||||||
|
__pycache__/
|
||||||
|
|
||||||
node_modules
|
node_modules
|
||||||
cache
|
cache
|
||||||
|
|||||||
@@ -2,6 +2,10 @@
|
|||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
- Do not create branches without being asked
|
||||||
|
|
||||||
## What this repo is
|
## What this repo is
|
||||||
|
|
||||||
A pnpm workspace holding two independently published CLIs, the authoring package
|
A pnpm workspace holding two independently published CLIs, the authoring package
|
||||||
@@ -142,13 +146,27 @@ One pass per invocation, `nopy.main.ts` orchestrating:
|
|||||||
variables (prompt, or read them back from the session on replay) → run
|
variables (prompt, or read them back from the session on replay) → run
|
||||||
`before` hooks → resolve `manifest.dependencies(vars)` (dynamic: it receives
|
`before` hooks → resolve `manifest.dependencies(vars)` (dynamic: it receives
|
||||||
the *collected* variables) → emit the deploy call → run `after` hooks. There
|
the *collected* variables) → emit the deploy call → run `after` hooks. There
|
||||||
is no separate topological sort; ordering falls out of the recursion, and a
|
is no separate topological sort; emission is post-order, so the ordering *is*
|
||||||
`${cubeId}:${host}` set makes emission idempotent. Hooks get a `HookContext`
|
topological without an algorithm computing it, and a `${cubeId}:${host}` set
|
||||||
|
makes emission idempotent. Hooks get a `HookContext`
|
||||||
whose `exec(id, vars)` re-enters `resolveCube`, so a hook can pull in a cube
|
whose `exec(id, vars)` re-enters `resolveCube`, so a hook can pull in a cube
|
||||||
that is not a declared dependency.
|
that is not a declared dependency.
|
||||||
6. **`nopy.executor.ts`** — runs the built `pyinfra <host> -y --data K=V ... --chdir <cubeDir> <script>`
|
Cycles are caught by a separate **resolution stack** — a (cube, host) pair
|
||||||
|
re-entered while still resolving raises with the whole path named. It has to
|
||||||
|
be separate from `resolvedCubes`, which is written *after* the descent and so
|
||||||
|
never sees a cycle at all, and which cannot be widened into a "seen" set
|
||||||
|
because re-entering a finished cube with different `param` overrides is
|
||||||
|
exactly what a dependency or a hook is for.
|
||||||
|
6. **`nopy.executor.ts`** — runs the built
|
||||||
|
`pyinfra <host> -y [-vv] [--debug] --data K=V ... --chdir <cubeDir> <script>`
|
||||||
commands through execa with inherited stdio, sequentially, stopping at the
|
commands through execa with inherited stdio, sequentially, stopping at the
|
||||||
first failure unless `continueOnError`.
|
first failure unless `continueOnError`. `DeployCall.command` is a true argv
|
||||||
|
and is spawned **without a shell**: it used to be joined into one string and
|
||||||
|
run through `execa({shell: true})`, which made every `--data` value shell
|
||||||
|
syntax — a password or a variable holding `;` or `$(…)` was executed. The only
|
||||||
|
thing that joins it back into a string is `maskCommand()`, for display, which
|
||||||
|
shell-quotes as it goes so `--print-only` output stays pasteable. The
|
||||||
|
verbosity/debug flags come from `config.log` through `logConfigToFlags()`.
|
||||||
|
|
||||||
### Variables
|
### Variables
|
||||||
|
|
||||||
@@ -277,6 +295,14 @@ another edge to the publish order. Extract it if a third CLI appears.
|
|||||||
|
|
||||||
Tag-driven, one package at a time; see `README.PUBLISH.md`.
|
Tag-driven, one package at a time; see `README.PUBLISH.md`.
|
||||||
|
|
||||||
|
`pnpm run release` (`scripts/release.mjs`, zx + enquirer + commander) is the
|
||||||
|
front door: pick packages, pick versions, write notes into `CHANGELOG.md`, run
|
||||||
|
the gate **against the bumped tree before committing** so a failure leaves
|
||||||
|
nothing to unpick, then commit, tag and push. Tags go out dependency-first and it
|
||||||
|
polls npmjs for each version before pushing the next — which is what replaced the
|
||||||
|
CI-side linked-deps guard. Tags are annotated (`-a -m`), not lightweight: a
|
||||||
|
lightweight tag is rejected outright under `tag.forceSignAnnotated`.
|
||||||
|
|
||||||
### Registry resolution
|
### Registry resolution
|
||||||
|
|
||||||
The repo commits a root `.npmrc` mapping `@bitsquare:registry` to the Gitea
|
The repo commits a root `.npmrc` mapping `@bitsquare:registry` to the Gitea
|
||||||
@@ -357,22 +383,28 @@ Three things the `workspace:*` links added, all of them non-obvious:
|
|||||||
`scripts/publish-order.mjs` topologically sorts over the `workspace:` edges;
|
`scripts/publish-order.mjs` topologically sorts over the `workspace:` edges;
|
||||||
the snapshot workflow stamps *every* version first and only then publishes in
|
the snapshot workflow stamps *every* version first and only then publishes in
|
||||||
that order, because `pnpm publish` reads the linked package's version at pack
|
that order, because `pnpm publish` reads the linked package's version at pack
|
||||||
time. `release.yml` additionally refuses to ship a package whose linked
|
time. `release.yml` used to additionally refuse to ship a package whose linked
|
||||||
dependency is not yet on npmjs (`scripts/linked-deps.mjs`) — npmjs is the
|
dependency was not yet on npmjs (`scripts/linked-deps.mjs`); that step is gone,
|
||||||
registry you cannot take a mistake back from.
|
and `scripts/release.mjs` enforces the same ordering earlier instead — it
|
||||||
|
pushes tags dependency-first and polls npmjs for each version before pushing
|
||||||
|
the next. `linked-deps.mjs` survives as a hand-check. A tag pushed some other
|
||||||
|
way is no longer caught, which is the accepted cost.
|
||||||
|
|
||||||
## Known drift
|
## Known drift
|
||||||
|
|
||||||
`logConfigToFlags()` is exported and tested but nothing feeds its output into the
|
`logConfigToFlags()` is now consumed by `buildDeployCall`, so `log.verbosity` /
|
||||||
built pyinfra command, so `log.verbosity` / `log.debug` in `.nopyrc.json`
|
`log.debug` in `.nopyrc.json` finally do what the README says. Note the
|
||||||
currently have no effect. Treat `docs/REFACTORING.md` as a plan, not a record.
|
consequence: `packages/nopy/.nopyrc.json` has always asked for
|
||||||
|
`"verbosity": "trace", "debug": true`, and a run from that directory now actually
|
||||||
|
gets `-vvv --debug`. Treat `docs/REFACTORING.md` as a plan, not a record.
|
||||||
|
|
||||||
The publish lane has now run against the Gitea registry: all four packages are
|
The publish lane has now run against the Gitea registry: all four packages are
|
||||||
there under `@main`, and `pnpm run try:snapshot` installs them into a throwaway
|
there under `@main`, and `pnpm run try:snapshot` installs them into a throwaway
|
||||||
project with npm and runs the binary. The npmjs lane has only ever published
|
project with npm and runs the binary. The npmjs lane has only ever published
|
||||||
`@bitsquare/nopy`; `keyman`, `nopy-cubes` and `nopy-cubes-core` have never been
|
`@bitsquare/nopy`; `keyman`, `nopy-cubes` and `nopy-cubes-core` have never been
|
||||||
released there, so the *check linked deps are released* guard in `release.yml`
|
released there. That used to be caught by the *check linked deps are released*
|
||||||
will stop the first `nopy` release until `nopy-cubes` ships.
|
guard in `release.yml`; now it is `pnpm run release` that holds `nopy`'s tag back
|
||||||
|
until `nopy-cubes` answers on npmjs.
|
||||||
|
|
||||||
Nothing checks that a bundle and the CLI reading it are compatible versions;
|
Nothing checks that a bundle and the CLI reading it are compatible versions;
|
||||||
`nopy.engines` was considered and deferred. `docs/CUBE-PACKAGES.md` is where all
|
`nopy.engines` was considered and deferred. `docs/CUBE-PACKAGES.md` is where all
|
||||||
@@ -383,10 +415,8 @@ from the plan.
|
|||||||
`src/index.ts` plus the authoring package; its *Known gaps* section is the short
|
`src/index.ts` plus the authoring package; its *Known gaps* section is the short
|
||||||
list of behaviour that surprises a reader (`--json` printing nothing on success,
|
list of behaviour that surprises a reader (`--json` printing nothing on success,
|
||||||
`DeployCall.dependencies` always empty, `ExecutionResult.stdout` never populated,
|
`DeployCall.dependencies` always empty, `ExecutionResult.stdout` never populated,
|
||||||
no cycle detection, and `self-update` reporting an empty dist-tag as an
|
and `self-update` reporting an empty dist-tag as an unreachable registry).
|
||||||
unreachable registry). `CubePackageRef` is referenced by the exported
|
`DOCS-AUDIT.md` tracks the drift in the remaining
|
||||||
`NopyConfig` but is not itself re-exported, so a consumer cannot name the type —
|
|
||||||
one line, not yet fixed. `DOCS-AUDIT.md` tracks the drift in the remaining
|
|
||||||
documents; §2.9 (the nopy README shipping yarn-workspace instructions to npmjs)
|
documents; §2.9 (the nopy README shipping yarn-workspace instructions to npmjs)
|
||||||
and §2.10 (the keyman README describing four of nine operations and inventing a
|
and §2.10 (the keyman README describing four of nine operations and inventing a
|
||||||
tenth) are both closed. The keyman README now quotes `helpText()` verbatim and a
|
tenth) are both closed. The keyman README now quotes `helpText()` verbatim and a
|
||||||
|
|||||||
+349
-54
@@ -15,21 +15,48 @@ Verified against the working tree at commit `fcc1817`. Line numbers are from tha
|
|||||||
state.
|
state.
|
||||||
|
|
||||||
Findings closed since are marked **✅ … fixed** and keep their original text as
|
Findings closed since are marked **✅ … fixed** and keep their original text as
|
||||||
the record of what was wrong. So far: §1.1 (`--use-defaults`), §2.2
|
the record of what was wrong. So far: §1.1 (`--use-defaults`), §1.3 (`log.*`),
|
||||||
(`getDefaults()`), §2.1 (precedence — the second half closed differently than
|
§1.5 (topological order, both halves), §2.2 (`getDefaults()`), §2.1 (precedence —
|
||||||
proposed), §3 in full (`docs/API.md`, regenerated), §4.2 (password on stdout —
|
the second half closed differently than proposed), §2.3 (the prompt label lost to
|
||||||
points 1 and 2 of 3), §4.3 (what a session records), §2.9 (the nopy README's
|
`.default()`), §3 in full (`docs/API.md`, regenerated), §4.2 (password on stdout,
|
||||||
yarn install instructions), and one bullet of §6.4.
|
all three points), §4.3 (what a session records), §2.9 (the nopy README's
|
||||||
|
yarn install instructions), §2.4 (`version` / `timestamp`, implemented rather
|
||||||
|
than deleted), §2.5 (`listSessions`' filename filter), §4.5 (`-s` on a replay,
|
||||||
|
plus `-l` and history), §5.1 and §6.1 (`service/autostart`, the README and the
|
||||||
|
script), §6.2 (`-H <id>` versus `--no-history`), §6.5 (cycle detection), §6.7 (a
|
||||||
|
secret written to the session in plaintext), and one bullet of §6.4.
|
||||||
|
|
||||||
Closing §3 also settled the documentation half of several findings elsewhere
|
Closing §3 also settled the documentation half of several findings elsewhere
|
||||||
without touching their underlying cause: §1.2, §1.3, §1.5, §2.3, §2.7, §4.4 and
|
without touching their underlying cause. Two of those are still in that state:
|
||||||
§6.5 are each now stated accurately in `docs/API.md`, but the code still behaves
|
§2.7 and §4.4 are stated accurately in `docs/API.md`, but the code still behaves
|
||||||
as those findings describe and they stay open.
|
as they describe and they stay open. The other four — §1.3, §1.5, §2.3 and §6.5 —
|
||||||
|
have since been closed in the code as well. §1.2 was closed outright by removing
|
||||||
|
the flag.
|
||||||
|
|
||||||
|
## Where the drift is
|
||||||
|
|
||||||
|
A field run against a fresh VM sorted the findings for us, and they cluster on
|
||||||
|
one seam. **Everything a human reads on screen matched the documentation.
|
||||||
|
Everything machine-facing had drifted** — `--json`, the session format, `-s` on a
|
||||||
|
replay, `-l` and history, `runtime:nodevm`'s parameters, the bundle's install
|
||||||
|
command.
|
||||||
|
|
||||||
|
That is not random rot. The interactive surface is maintained by daily use: a
|
||||||
|
wrong prompt label is noticed the next time someone runs the thing. The scripting
|
||||||
|
surface was documented from intent and then never exercised, so nothing pushed
|
||||||
|
back when it changed or was never built.
|
||||||
|
|
||||||
|
Corrections come in the two shapes that distinction implies. `--json` was
|
||||||
|
documented from intent and never built, so it was removed (§1.2). The rest was
|
||||||
|
built and then drifted, so it was fixed. Keeping it fixed means what remains of
|
||||||
|
the scripting surface — `--print-only`, sessions, history — needs tests that
|
||||||
|
assert on **stdout**, not prose.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Contents
|
## Contents
|
||||||
|
|
||||||
|
- [Where the drift is](#where-the-drift-is)
|
||||||
- [1. Documented features that do not exist](#1-documented-features-that-do-not-exist)
|
- [1. Documented features that do not exist](#1-documented-features-that-do-not-exist)
|
||||||
- [2. Documented behaviour that differs from the code](#2-documented-behaviour-that-differs-from-the-code)
|
- [2. Documented behaviour that differs from the code](#2-documented-behaviour-that-differs-from-the-code)
|
||||||
- [3. ✅ `docs/API.md` — systematic drift — fixed](#3--docsapimd--systematic-drift--fixed)
|
- [3. ✅ `docs/API.md` — systematic drift — fixed](#3--docsapimd--systematic-drift--fixed)
|
||||||
@@ -82,7 +109,7 @@ nopy.cli.ts:95 useDefaults: options... # passed
|
|||||||
cubes/dependencies.ts:35 useDefaults?: boolean; # declared — and that is all
|
cubes/dependencies.ts:35 useDefaults?: boolean; # declared — and that is all
|
||||||
```
|
```
|
||||||
|
|
||||||
### 1.2 🔴 `-j, --json` produces no output on success
|
### 1.2 🔴 `-j, --json` produces no output on success — **closed by removal**
|
||||||
|
|
||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
@@ -104,7 +131,30 @@ Related: `--dry-run --json` prints the **text** plan, not JSON.
|
|||||||
argument (`nopy.executor.ts:172`), even though the function supports it
|
argument (`nopy.executor.ts:172`), even though the function supports it
|
||||||
(`nopy.executor.ts:110`).
|
(`nopy.executor.ts:110`).
|
||||||
|
|
||||||
### 1.3 🟠 `log.verbosity` and `log.debug` have no effect
|
**Closed by deleting the flag, not by implementing it.** `executeDeployCalls`
|
||||||
|
runs pyinfra with *inherited* stdio, so during a run nopy does not own its own
|
||||||
|
stdout — pyinfra does. A JSON blob appended after an unbounded amount of another
|
||||||
|
process's output is not machine-readable by any definition a caller could rely
|
||||||
|
on; making it so means capturing pyinfra's output and giving up live progress.
|
||||||
|
Same root cause as `ExecutionResult.stdout` never being populated. What replaced
|
||||||
|
it is a promise a test can hold to: **stdout carries the deploy commands and
|
||||||
|
pyinfra's own output, everything nopy says about itself goes to stderr**, and the
|
||||||
|
exit code is the verdict. `nopy history --json` is a different flag, it works,
|
||||||
|
and it stays.
|
||||||
|
|
||||||
|
### 1.3 ✅ `log.verbosity` and `log.debug` have no effect — **fixed**
|
||||||
|
|
||||||
|
> **Closed by implementing it, not by deleting the documentation.** The two
|
||||||
|
> tables in the README were accurate about pyinfra's flags and about the mapping
|
||||||
|
> `logConfigToFlags()` already computed — the only missing step was one call.
|
||||||
|
> `buildDeployCall` now prefixes `logConfigToFlags(this.config.log)` onto the
|
||||||
|
> pyinfra argv, right after `-y`. Verified against a real `pyinfra` binary, which
|
||||||
|
> accepts `-vv --debug` in that position.
|
||||||
|
>
|
||||||
|
> Worth knowing rather than discovering: `packages/nopy/.nopyrc.json` asks for
|
||||||
|
> `"verbosity": "trace", "debug": true`, and it now gets `-vvv --debug`. The
|
||||||
|
> value was left alone — it says what its author meant, and it never did anything
|
||||||
|
> until now.
|
||||||
|
|
||||||
Pre-existing known drift, recorded in `CLAUDE.md`, but the README still presents
|
Pre-existing known drift, recorded in `CLAUDE.md`, but the README still presents
|
||||||
it as a working feature — two tables, a recommendation paragraph, and a slot in
|
it as a working feature — two tables, a recommendation paragraph, and a slot in
|
||||||
@@ -124,7 +174,25 @@ This is live in the repo's own config: `packages/nopy/.nopyrc.json` sets
|
|||||||
the current interface at `cubes/types.ts:43-56`, which has `id`, `name`,
|
the current interface at `cubes/types.ts:43-56`, which has `id`, `name`,
|
||||||
`schema`, `dependencies`, `before`, `after` and nothing else.
|
`schema`, `dependencies`, `before`, `after` and nothing else.
|
||||||
|
|
||||||
### 1.5 🟠 Topological sorting
|
### 1.5 ✅ Topological sorting — **closed, both halves**
|
||||||
|
|
||||||
|
> **The vocabulary complaint was answered by §6.9; the substantive one is now
|
||||||
|
> fixed in code.** There is still no sort pass, and there does not need to be:
|
||||||
|
> emission is post-order, so the output *is* a topological order of the graph
|
||||||
|
> (§6.9 records the measurement and two tests pin it). What the finding was right
|
||||||
|
> about is that a sort detects cycles and this did not.
|
||||||
|
>
|
||||||
|
> `BuildContext` now carries a **resolution stack** alongside `resolvedCubes`: a
|
||||||
|
> (cube, host) pair re-entered while it is still resolving raises a
|
||||||
|
> `NopyUsageError` naming the whole path — `Circular dependency on host1:
|
||||||
|
> a → b → c → a`. It has to be a separate structure. `resolvedCubes` is written
|
||||||
|
> *after* the descent, so a cycle never reaches it, and it cannot be widened into
|
||||||
|
> a "seen" set because re-entering a *finished* cube with different `param`
|
||||||
|
> overrides is precisely what a dependency or a hook is for. Six tests cover it,
|
||||||
|
> including a loop closed by a hook's `exec` rather than a `dependencies()`
|
||||||
|
> entry, and a diamond that must still be allowed.
|
||||||
|
>
|
||||||
|
> The README claims are reworded: "in dependency order, with cycle detection".
|
||||||
|
|
||||||
`README.md:11` ("Dependency resolution with **topological sorting**"),
|
`README.md:11` ("Dependency resolution with **topological sorting**"),
|
||||||
`README.md:25` ("Topologically sorts cubes based on dependencies") and
|
`README.md:25` ("Topologically sorts cubes based on dependencies") and
|
||||||
@@ -242,7 +310,20 @@ Three cubes in this repo are in that state today:
|
|||||||
The failure is silent — no error, no warning, just a pyinfra run with an empty
|
The failure is silent — no error, no warning, just a pyinfra run with an empty
|
||||||
data set.
|
data set.
|
||||||
|
|
||||||
### 2.3 🔴 `.describe()` before `.default()` loses the prompt label
|
### 2.3 ✅ `.describe()` before `.default()` loses the prompt label — **fixed**
|
||||||
|
|
||||||
|
> **Closed the one-line way, not by re-ordering 15 manifests.** `nopy.prompts.ts`
|
||||||
|
> has a `promptLabel()` that walks down through `default` / `optional` /
|
||||||
|
> `nullable` wrappers looking for a description, so both chaining orders now give
|
||||||
|
> the sentence and neither can regress. Discriminates on `zodKind`, not
|
||||||
|
> `instanceof`, for the reason recorded on that function.
|
||||||
|
>
|
||||||
|
> Proven twice over. The mocked test asserts all four shapes —
|
||||||
|
> `describe().default()`, `default().describe()`, a doubly-wrapped
|
||||||
|
> `describe().optional().default()`, and a field with no description at all. The
|
||||||
|
> **pty** test is the one that matters: its probe schema is written in the losing
|
||||||
|
> order and it now waits for `First value` on a real enquirer render, so removing
|
||||||
|
> the unwrapping fails a test that talks to an actual terminal.
|
||||||
|
|
||||||
`CLAUDE.md` and the cube contract state that each schema field is `.describe()`d
|
`CLAUDE.md` and the cube contract state that each schema field is `.describe()`d
|
||||||
and "the description is the prompt label". `nopy.prompts.ts:184-185` reads it as:
|
and "the description is the prompt label". `nopy.prompts.ts:184-185` reads it as:
|
||||||
@@ -275,7 +356,19 @@ the two documents disagree, and neither mentions that it matters.
|
|||||||
`net:tailscale` (all 4 fields), `runtime:nodevm` (all 4), `user:add` (all 4),
|
`net:tailscale` (all 4 fields), `runtime:nodevm` (all 4), `user:add` (all 4),
|
||||||
`ssh:keygen` (all 4) and `admin:locale` (all 4).
|
`ssh:keygen` (all 4) and `admin:locale` (all 4).
|
||||||
|
|
||||||
### 2.4 🟠 Session files claim `version` and `timestamp` fields
|
### 2.4 ✅ Session files claim `version` and `timestamp` fields — **fixed**
|
||||||
|
|
||||||
|
Closed by implementing them rather than deleting the claim. `createSession`
|
||||||
|
stamps `version: '1.0.0'` (exported as `SESSION_VERSION`) and an ISO
|
||||||
|
`timestamp`; `nopy()` fills in a default `name` at save time, using the same
|
||||||
|
`describeSession()` the history list uses — one implementation, so the two
|
||||||
|
cannot drift. `loadSession` still requires only `cubes` and `auth`, so every
|
||||||
|
session written before this, and every hand-written one, keeps loading; an
|
||||||
|
unrecognised `version` is a warning on stderr, never a refusal. The interface
|
||||||
|
and both documents now mark the three fields optional, which is what they are.
|
||||||
|
|
||||||
|
The original finding follows.
|
||||||
|
|
||||||
|
|
||||||
`README.md:179-181` shows a session with `"version": "1.0.0"` and
|
`README.md:179-181` shows a session with `"version": "1.0.0"` and
|
||||||
`"timestamp": "2025-10-13T10:30:00Z"`, and `docs/SESSION_FORMAT.md:305-306`
|
`"timestamp": "2025-10-13T10:30:00Z"`, and `docs/SESSION_FORMAT.md:305-306`
|
||||||
@@ -293,7 +386,14 @@ Consequence: a `version` field implies a compatibility check that does not exist
|
|||||||
Nothing reads it, so an incompatible old session fails later and more obscurely
|
Nothing reads it, so an incompatible old session fails later and more obscurely
|
||||||
than a version check would.
|
than a version check would.
|
||||||
|
|
||||||
### 2.5 🟠 Session filename convention does not match `listSessions()`
|
### 2.5 ✅ Session filename convention does not match `listSessions()` — **fixed**
|
||||||
|
|
||||||
|
`listSessions` now matches `*.nopysession.json` and `*.nopysession.mjs` as well
|
||||||
|
as the two shorter suffixes — `saveSession` writes whatever path it is handed,
|
||||||
|
so files under the old name exist and there was no reason to stop finding them.
|
||||||
|
|
||||||
|
The original finding follows.
|
||||||
|
|
||||||
|
|
||||||
The READMEs consistently use `*.nopysession.json` (`README.md:223`, `330`, `338`;
|
The READMEs consistently use `*.nopysession.json` (`README.md:223`, `330`, `338`;
|
||||||
`docs/DOCKER.md:54`; the shipped `example.nopysession.json`).
|
`docs/DOCKER.md:54`; the shipped `example.nopysession.json`).
|
||||||
@@ -310,7 +410,15 @@ file naming the README recommends is invisible to the function documented at
|
|||||||
`docs/API.md:430`. `loadSession` is unaffected (it switches on `.json`/`.mjs`),
|
`docs/API.md:430`. `loadSession` is unaffected (it switches on `.json`/`.mjs`),
|
||||||
so this only bites the listing API.
|
so this only bites the listing API.
|
||||||
|
|
||||||
### 2.6 🟠 `docs/DOCKER.md` container name contradicts the file it points at
|
### 2.6 ✅ `docs/DOCKER.md` container name contradicts the file it points at — **fixed**
|
||||||
|
|
||||||
|
> **Resolved.** `example.nopysession.json` now targets
|
||||||
|
> `@docker/nopy-test-container`, matching the guide, `packages/nopy/.nopyrc.json`
|
||||||
|
> and the session example in the nopy README. `nopy-test-ubuntu` was the outlier.
|
||||||
|
> Worth knowing why it silently "worked": an identifier with no matching
|
||||||
|
> container is read as an *image*, so the run built and committed a throwaway
|
||||||
|
> image instead of failing — the mode `docs/DOCKER.md` now documents at the end.
|
||||||
|
> The finding below is kept as the record of what was wrong.
|
||||||
|
|
||||||
`docs/DOCKER.md:35` and `:45`:
|
`docs/DOCKER.md:35` and `:45`:
|
||||||
|
|
||||||
@@ -372,7 +480,18 @@ this is what a reader sees on npmjs.com — build-from-monorepo instructions
|
|||||||
instead of `npm install -g @bitsquare/nopy`, which is what the root README and
|
instead of `npm install -g @bitsquare/nopy`, which is what the root README and
|
||||||
`README.PUBLISH.md:314` correctly tell people to run.
|
`README.PUBLISH.md:314` correctly tell people to run.
|
||||||
|
|
||||||
### 2.10 🟠 keyman README: two operations missing, one operation invented
|
### 2.10 ✅ keyman README: two operations missing, one operation invented — **fixed**
|
||||||
|
|
||||||
|
> **Resolved.** The README was rewritten against the code (`packages/keyman/docs/PLAN.md`
|
||||||
|
> Phase 9). All nine menu entries are documented, and a test asserts it contains
|
||||||
|
> every label `keyman.main.ts` offers, so a tenth cannot arrive undocumented.
|
||||||
|
> Encrypt is described as the union of `~/.ssh` and the tmp directory, which is
|
||||||
|
> what it does. The Quick Start now points at the Generate operation instead of
|
||||||
|
> `ssh-keygen`. Rotation stopped being an invention in Phase 10: it exists, in two
|
||||||
|
> halves (`keyman.rotate.ts`), and the README documents the sequence. The whole CLI
|
||||||
|
> surface is there too — `helpText()` quoted verbatim, with a test that fails if the
|
||||||
|
> two diverge — which was the other half of this, tracked as
|
||||||
|
> `packages/keyman/docs/AUDIT.md` §5.3. The finding below is kept as the record.
|
||||||
|
|
||||||
`packages/keyman/README.md:90-96` lists four menu entries: List, Encrypt,
|
`packages/keyman/README.md:90-96` lists four menu entries: List, Encrypt,
|
||||||
Decrypt, Quit. The menu (`keyman.main.ts:54-61`) has six:
|
Decrypt, Quit. The menu (`keyman.main.ts:54-61`) has six:
|
||||||
@@ -403,7 +522,9 @@ Root `README.md:57-58` describes "a hard **85 % branch** floor". Both
|
|||||||
the root README is the odd one out, and it is the file a new contributor reads
|
the root README is the odd one out, and it is the file a new contributor reads
|
||||||
first.
|
first.
|
||||||
|
|
||||||
### 2.12 🟡 `docs/DOCKER.md` relative link is broken
|
### 2.12 ✅ `docs/DOCKER.md` relative link is broken — **fixed**
|
||||||
|
|
||||||
|
> **Resolved.** The link is now `../README.md`.
|
||||||
|
|
||||||
`docs/DOCKER.md:8` links `[README.md](./README.md)`, which resolves to
|
`docs/DOCKER.md:8` links `[README.md](./README.md)`, which resolves to
|
||||||
`packages/nopy/docs/README.md` — nonexistent. It should be `../README.md`.
|
`packages/nopy/docs/README.md` — nonexistent. It should be `../README.md`.
|
||||||
@@ -422,8 +543,7 @@ first.
|
|||||||
>
|
>
|
||||||
> Three things were deliberately added rather than merely corrected. A
|
> Three things were deliberately added rather than merely corrected. A
|
||||||
> **Known gaps** section states the behaviour a reader would otherwise take on
|
> **Known gaps** section states the behaviour a reader would otherwise take on
|
||||||
> trust — `logConfigToFlags` being unconsumed (§1.3), `--json` printing nothing
|
> trust — `logConfigToFlags` being unconsumed (§1.3), the absent cycle detection (§1.5, §6.5), `DeployCall.dependencies`
|
||||||
> on success (§1.2), the absent cycle detection (§1.5, §6.5), `DeployCall.dependencies`
|
|
||||||
> always being `[]`, `ExecutionResult.stdout`/`stderr` never being populated, and
|
> always being `[]`, `ExecutionResult.stdout`/`stderr` never being populated, and
|
||||||
> hook variables not being schema-validated (§2.7). The `.describe()`/`.default()`
|
> hook variables not being schema-validated (§2.7). The `.describe()`/`.default()`
|
||||||
> ordering hazard (§2.3) is called out where the manifest example lives, with the
|
> ordering hazard (§2.3) is called out where the manifest example lives, with the
|
||||||
@@ -539,9 +659,30 @@ Because `loadCubes` turns each failure into an `errors` entry and `nopy.main.ts:
|
|||||||
aborts when `errors.length > 0`, a fresh clone cannot run a single cube. Neither
|
aborts when `errors.length > 0`, a fresh clone cannot run a single cube. Neither
|
||||||
README mentions a setup step.
|
README mentions a setup step.
|
||||||
|
|
||||||
### 4.2 🟠 The SSH password is printed in plaintext — **mostly fixed**
|
### 4.2 ✅ The SSH password is printed in plaintext — **fixed as far as it can be**
|
||||||
|
|
||||||
> **Points 1 and 2 resolved; point 3 stands.** `maskCommand()`
|
> **Point 3 is now closed too, and it was worse than the finding said.** The
|
||||||
|
> command is no longer a shell string. `buildDeployCall` emits a true argv — one
|
||||||
|
> element per argument, nothing pre-quoted — and `executeCall` spawns it as
|
||||||
|
> `execa(command[0], command.slice(1))` with no `shell` option at all.
|
||||||
|
>
|
||||||
|
> The finding called the quoting a vulnerability "in the password". It was not
|
||||||
|
> limited to the password: with `shell: true` the whole joined string was parsed
|
||||||
|
> by `sh`, and `--data` values were interpolated inside double quotes, so a `$(…)`
|
||||||
|
> or a backtick in *any* variable value was command substitution. Verified both
|
||||||
|
> ways against a real pyinfra: `--data 'MOTD=$(id); rm -rf /'` now arrives at
|
||||||
|
> `host.data.MOTD` verbatim.
|
||||||
|
>
|
||||||
|
> `maskCommand()` was rewritten to walk the argv by position rather than to
|
||||||
|
> pattern-match a joined string, which also fixes a leak the old version had — it
|
||||||
|
> bounded a secret's value on the closing `"` the builder had written, so a value
|
||||||
|
> containing a `"` leaked its own tail. It shell-quotes as it joins, so
|
||||||
|
> `--print-only` output is still pasteable.
|
||||||
|
>
|
||||||
|
> What remains is not fixable here: the value still reaches pyinfra on its
|
||||||
|
> command line and so is visible in `ps`. That is pyinfra's `--data` interface.
|
||||||
|
|
||||||
|
> **Points 1 and 2 resolved earlier.** `maskCommand()`
|
||||||
> (`nopy.executor.ts`) rewrites the SSH `--password` and every `--data` value the
|
> (`nopy.executor.ts`) rewrites the SSH `--password` and every `--data` value the
|
||||||
> manifest declared a secret, and it is applied at all three places the command
|
> manifest declared a secret, and it is applied at all three places the command
|
||||||
> string is printed: the debug log, the dry-run plan, and `--print-only`. The
|
> string is printed: the debug log, the dry-run plan, and `--print-only`. The
|
||||||
@@ -549,10 +690,8 @@ README mentions a setup step.
|
|||||||
> says which keys are sensitive, so `TOKEN`, `PSK` and `AUTH_KEY` are covered
|
> says which keys are sensitive, so `TOKEN`, `PSK` and `AUTH_KEY` are covered
|
||||||
> too, and it no longer matters that a key merely *looks* like a password.
|
> too, and it no longer matters that a key merely *looks* like a password.
|
||||||
>
|
>
|
||||||
> Point 3 is unchanged and now documented instead: the value still reaches
|
> (At the time: point 3 unchanged, documented rather than fixed. See
|
||||||
> pyinfra on its command line, so it is visible in `ps`. That is inherent to
|
> `docs/REFACTORING.md` item 7.)
|
||||||
> pyinfra's `--data` interface, not something nopy can mask. The shell-quoting
|
|
||||||
> concern in the same point is also still open. See `docs/REFACTORING.md` item 7.
|
|
||||||
|
|
||||||
Not stated in any document, and it sits directly against the security notes at
|
Not stated in any document, and it sits directly against the security notes at
|
||||||
`README.md:217` and `:325` (which are narrowly about *storage*, and are correct
|
`README.md:217` and `:325` (which are narrowly about *storage*, and are correct
|
||||||
@@ -612,7 +751,22 @@ Worth documenting alongside `--dry-run`, since the difference is not obvious:
|
|||||||
`--print-only` returns a `NopyResult` with `successful: 0` and skips execution
|
`--print-only` returns a `NopyResult` with `successful: 0` and skips execution
|
||||||
entirely, while `--dry-run` goes through the executor.
|
entirely, while `--dry-run` goes through the executor.
|
||||||
|
|
||||||
### 4.5 🟡 `--save-session` is ignored during a replay
|
### 4.5 ✅ `--save-session` is ignored during a replay — **fixed**
|
||||||
|
|
||||||
|
The guard is gone: the resolved cube set is exactly what the user asked to
|
||||||
|
capture, and a session written from a replay is no less valid than one written
|
||||||
|
from a fresh run. The README's "Recording a Session" examples now include the
|
||||||
|
replay form.
|
||||||
|
|
||||||
|
Fixed alongside it, from the same field run: a `--load-session` run was excluded
|
||||||
|
from history along with `-R`/`-H`, which was right for the latter two and wrong
|
||||||
|
for the first — a session file has never been in history, so `nopy history`
|
||||||
|
reported nothing afterwards and `-R` had nothing to repeat. `WorkflowResult`
|
||||||
|
now carries `replaySource: 'file' | 'history' | undefined` instead of a boolean,
|
||||||
|
which is the distinction the boolean could not express.
|
||||||
|
|
||||||
|
The original finding follows.
|
||||||
|
|
||||||
|
|
||||||
`nopy.main.ts:191` guards with `saveSessionPath && !workflow.isReplay`, so
|
`nopy.main.ts:191` guards with `saveSessionPath && !workflow.isReplay`, so
|
||||||
`nopy install -R -s out.json` writes nothing and says nothing. The
|
`nopy install -R -s out.json` writes nothing and says nothing. The
|
||||||
@@ -641,7 +795,15 @@ a project without `KEY_DIR` in their config gets `None`.
|
|||||||
Two cubes have **no README at all**: `cubes/admin/hostname` and `cubes/git/clone`
|
Two cubes have **no README at all**: `cubes/admin/hostname` and `cubes/git/clone`
|
||||||
(20 of 22 have one).
|
(20 of 22 have one).
|
||||||
|
|
||||||
### 5.1 🔴 `cubes/service/autostart/README.md` documents a different cube
|
### 5.1 ✅ `cubes/service/autostart/README.md` documents a different cube — **fixed**
|
||||||
|
|
||||||
|
> **Rewritten from the manifest and the (now working, see §6.1) deploy script.**
|
||||||
|
> Three parameters, `APP` / `SERVICE_NAME` / `AUTOSTART`, each described as what
|
||||||
|
> it actually does — including that `SERVICE_NAME` never reaches systemd and is
|
||||||
|
> a label only, so getting it wrong is cosmetic rather than a cube that manages
|
||||||
|
> the wrong unit. The new text also states the thing the old one obscured by
|
||||||
|
> describing a deploy pipeline: this cube does **not** create the unit file, it
|
||||||
|
> only enables and starts one that already exists.
|
||||||
|
|
||||||
The file is titled **"TypeStack Install Cube"** and describes cloning a git
|
The file is titled **"TypeStack Install Cube"** and describes cloning a git
|
||||||
repository, `yarn install`, `yarn build`, `docker compose up -d`, and PM2 process
|
repository, `yarn install`, `yarn build`, `docker compose up -d`, and PM2 process
|
||||||
@@ -705,7 +867,14 @@ have empty schemas.)
|
|||||||
|
|
||||||
Not documentation issues, but found while checking the docs and worth recording.
|
Not documentation issues, but found while checking the docs and worth recording.
|
||||||
|
|
||||||
### 6.1 🔴 `cubes/service/autostart/deploy.py` cannot run
|
### 6.1 ✅ `cubes/service/autostart/deploy.py` cannot run — **fixed**
|
||||||
|
|
||||||
|
> **Three lines.** `server` is imported alongside `systemd`, and `SERVICE_NAME`
|
||||||
|
> and `AUTOSTART` are read off `host.data` next to `APP`. The logic underneath
|
||||||
|
> was always right; nothing else changed. `python3 -m py_compile` passes.
|
||||||
|
>
|
||||||
|
> The "fails twice over" clause is stale: §2.2 closed with `-D`, so the cube does
|
||||||
|
> get its `--data` now.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from pyinfra.operations import systemd # `server` is never imported
|
from pyinfra.operations import systemd # `server` is never imported
|
||||||
@@ -722,7 +891,31 @@ if AUTOSTART: # NameError
|
|||||||
`host.data`; `server` is used but not imported. The script raises `NameError` on
|
`host.data`; `server` is used but not imported. The script raises `NameError` on
|
||||||
the `if`. Per §2.2 this cube also gets no `--data` at all, so it fails twice over.
|
the `if`. Per §2.2 this cube also gets no `--data` at all, so it fails twice over.
|
||||||
|
|
||||||
### 6.2 🔴 `-H <id>` and `--no-history` share one destination
|
### 6.2 ✅ `-H <id>` and `--no-history` share one destination — **fixed**
|
||||||
|
|
||||||
|
> **The boolean was renamed, not the replay flag.** `--no-history` is now
|
||||||
|
> `--no-save-history`, writing to `options.saveHistory`; `-H, --history <id>`
|
||||||
|
> keeps `options.history` and every documented invocation of it is unchanged.
|
||||||
|
> Renaming the boolean is the right way round twice over: `-H <id>` is what the
|
||||||
|
> help text, the README and `docs/API.md` all use, and "save history" is what the
|
||||||
|
> flag actually suppresses, next to `-s, --save-session`.
|
||||||
|
>
|
||||||
|
> Commander cannot be told to use a different destination — `attributeName()` is
|
||||||
|
> derived from the long flag with `no-` stripped — so separating the two meant
|
||||||
|
> changing one of the two spellings. The old spelling now fails loudly rather
|
||||||
|
> than silently, which is the point: `nopy install --no-history` prints
|
||||||
|
> `error: unknown option '--no-history'`.
|
||||||
|
>
|
||||||
|
> Verified by running the CLI, since `nopy.cli.ts` is argv wiring and excluded
|
||||||
|
> from coverage:
|
||||||
|
>
|
||||||
|
> ```
|
||||||
|
> install -H nonexistent-id -> Session not found: nonexistent-id
|
||||||
|
> install -H nonexistent-id --no-save-history -> Session not found: nonexistent-id
|
||||||
|
> install --no-history -> error: unknown option '--no-history'
|
||||||
|
> ```
|
||||||
|
>
|
||||||
|
> The middle line is the finding: the id used to be destroyed there.
|
||||||
|
|
||||||
Both options write to `options.history` (`nopy.cli.ts:57` and `:64`). Verified
|
Both options write to `options.history` (`nopy.cli.ts:57` and `:64`). Verified
|
||||||
with Commander:
|
with Commander:
|
||||||
@@ -752,13 +945,19 @@ the three has to give.
|
|||||||
with §4.2 this is a second path by which secrets reach stdout.~~ **Removed**
|
with §4.2 this is a second path by which secrets reach stdout.~~ **Removed**
|
||||||
alongside the `--use-defaults` work; it would have made an unattended run
|
alongside the `--use-defaults` work; it would have made an unattended run
|
||||||
unreadable. The two other paths in §4.2 are untouched.
|
unreadable. The two other paths in §4.2 are untouched.
|
||||||
- `keyman.encrypt.ts:19-20` — `console.log(tmpKeys); console.log(sshKeys);`
|
- ~~`keyman.encrypt.ts:19-20` — `console.log(tmpKeys); console.log(sshKeys);`
|
||||||
before the prompt.
|
before the prompt.~~ **Removed** in Phase 2 of the keyman remediation, along
|
||||||
|
with a third one nobody had noticed: a `console.log` *inside* a `filter`
|
||||||
|
callback in `keyman.decrypt.ts`, printing a line per vault directory.
|
||||||
|
|
||||||
### 6.5 🟡 No cycle detection
|
### 6.5 ✅ No cycle detection — **fixed**
|
||||||
|
|
||||||
Covered under §1.5. `docs/API.md:160` documents the error; there is no code that
|
Covered under §1.5, and closed there: the resolution stack raises a
|
||||||
raises it. Mutually dependent cubes recurse until the stack overflows.
|
`NopyUsageError` naming the whole path. `docs/API.md` documents the error again,
|
||||||
|
and this time something raises it.
|
||||||
|
|
||||||
|
~~`docs/API.md:160` documents the error; there is no code that raises it.
|
||||||
|
Mutually dependent cubes recurse until the stack overflows.~~
|
||||||
|
|
||||||
### 6.6 🟠 `ssh:keygen` depends on `user:add` but shares nothing with it
|
### 6.6 🟠 `ssh:keygen` depends on `user:add` but shares nothing with it
|
||||||
|
|
||||||
@@ -783,6 +982,88 @@ same shape as the `PASSWORD` default that was removed. It is recorded in the
|
|||||||
session, so replays are stable, but each fresh `-D` run still creates a
|
session, so replays are stable, but each fresh `-D` run still creates a
|
||||||
differently-named account.
|
differently-named account.
|
||||||
|
|
||||||
|
### 6.7 ✅ A declared secret was written to the session file in plaintext — **fixed**
|
||||||
|
|
||||||
|
Found in the acceptance run, not by reading. `README.md` promises that a session
|
||||||
|
holds no secret: "Passwords are never stored in session files. This covers both
|
||||||
|
the SSH password ... and any schema key a cube's manifest lists under `secrets`."
|
||||||
|
The `variables` block honoured that — `persistable()` leaves a declared secret
|
||||||
|
out entirely. The `env` block, one key higher in the same file, was a verbatim
|
||||||
|
copy of `.nopyrc.json`'s, so a credential declared there was written to the
|
||||||
|
session **and** to `.nopy.history.json` in plaintext.
|
||||||
|
|
||||||
|
Two of the fixes above widened the blast radius before it was noticed: §4.5 made
|
||||||
|
`--save-session` work on a replay, and §3.2 started recording `--load-session`
|
||||||
|
runs to history. Both write more files than before.
|
||||||
|
|
||||||
|
`Variables.persistableEnv()` applies the same rule to `env` that `persistable()`
|
||||||
|
applies to `variables`, and `nopy()` uses it instead of `config.env`. Verified in
|
||||||
|
the field: with `SSH_PASSWORD` declared under `secrets` and set in `env`, neither
|
||||||
|
the written session nor the history file contains the value.
|
||||||
|
|
||||||
|
### 6.8 ✅ `--print-only` was recorded in history — **fixed**
|
||||||
|
|
||||||
|
Also found in the acceptance run. `--dry-run` is excluded from history because it
|
||||||
|
deploys nothing; `--print-only`, which also deploys nothing, was not. Four
|
||||||
|
interactive runs against the VM produced four history entries, two of them from
|
||||||
|
`-P` passes that had only printed a command — and since `-R` repeats the head of
|
||||||
|
the list, the safe look-before-you-leap flag displaced the last real deployment
|
||||||
|
as the thing a bare `-R` would re-run.
|
||||||
|
|
||||||
|
One condition, `!printOnly`, alongside the `!dryRun` it belongs with. The
|
||||||
|
`README` list of "a run is *not* recorded when" and `docs/API.md` say so now.
|
||||||
|
|
||||||
|
### 6.9 ✅ Deploy order is the dependency tree; `CubeSelection` decides only the ties — **write-up corrected**
|
||||||
|
|
||||||
|
Found in the acceptance run, and the first write-up of it here was wrong. It
|
||||||
|
claimed a fix "has to decide what the right order even is — the order they were
|
||||||
|
picked in, or a topological one over `dependencies()`". Neither: the order is the
|
||||||
|
dependency tree, and that is already what nopy does. `resolveCube` resolves
|
||||||
|
`dependencies()` before emitting the cube itself, so emission is DFS post-order —
|
||||||
|
a topological order by construction. The recursion *is* the sort, which is what
|
||||||
|
`docs/API.md` means by "no separate topological sort", and `nopy.main.ts` walking
|
||||||
|
`selectedCubes` cannot break it: a cube listed ahead of its own dependency still
|
||||||
|
drags that dependency in first, and the second visit is deduped by `callKey`
|
||||||
|
rather than re-emitted at the tail. Pinned by *deploy order across several
|
||||||
|
selected cubes* in `tests/cubes.dependencies.test.ts`, both ways round.
|
||||||
|
|
||||||
|
This does not reopen §1.5, which stands: the *output* is a topological order but
|
||||||
|
there is no sort *algorithm*, and the price of that is still no cycle detection —
|
||||||
|
two mutually dependent cubes recurse until the stack overflows.
|
||||||
|
|
||||||
|
What list order does decide is where a cube with **no** edge lands, and that is
|
||||||
|
the whole of the real finding. `CubeSelection` returns enquirer's `selected`,
|
||||||
|
which is `choices.filter(enabled)` — display order, sorted by cube id, not the
|
||||||
|
order you ticked them. So picking `user:add` and `runtime:nodevm` yields
|
||||||
|
`['runtime:nodevm', 'user:add']`, and nothing reorders them because
|
||||||
|
`runtime:nodevm` declares `dependencies: () => []`. The acceptance run split them
|
||||||
|
into two invocations.
|
||||||
|
|
||||||
|
That missing edge is deliberate and stays missing: `user:add` *creates* a user,
|
||||||
|
so declaring it would make installing Node into an existing account silently
|
||||||
|
provision a new one. The prerequisite `SHELL=fish` really has is "fish and Oh My
|
||||||
|
Fish exist for `USER`", which no cube offers on its own — `user:add` only
|
||||||
|
provides it in passing. §5.3's `DeployError` is the answer for that, and it fires
|
||||||
|
before anything is changed. Ordering cannot fix an edge nobody can honestly
|
||||||
|
declare.
|
||||||
|
|
||||||
|
One residue, verified and left alone: an `after` hook's `exec(id)` runs after its
|
||||||
|
own cube is emitted, so it expresses "B after A" — but if B is also selected and
|
||||||
|
listed first, B is emitted first and the intent inverts. `after` hooks are not
|
||||||
|
the dependency graph and no cube in the bundle relies on this.
|
||||||
|
|
||||||
|
### 6.10 ✅ `runtime:nodevm` installed apt packages without refreshing the index — **fixed**
|
||||||
|
|
||||||
|
The same defect as §5.3 one operation earlier, and it only surfaced once §5.3 was
|
||||||
|
fixed and the cube could be run on a box where nothing else had. `apt.packages`
|
||||||
|
was called without `update`, alone among the six cubes in the bundle that install
|
||||||
|
packages. On a fresh `bento/ubuntu-24.04` the shipped index names .deb versions
|
||||||
|
the mirror has already superseded, so the fetch 404s and pyinfra reports
|
||||||
|
`executed 0 commands` before nvm is ever reached.
|
||||||
|
|
||||||
|
It passed on the first VM only because `user:add` had run there and pulled in
|
||||||
|
`apt:essentials`, which does pass `update`. Fixed with `update=True` on the call.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 7. Checked and accurate
|
## 7. Checked and accurate
|
||||||
@@ -813,35 +1094,49 @@ Recording what was verified and found correct, so a future pass need not redo it
|
|||||||
scan, dotted/`node_modules` skipping, the prefixed `*.manifest.mjs` fallback,
|
scan, dotted/`node_modules` skipping, the prefixed `*.manifest.mjs` fallback,
|
||||||
and the three-step id resolution match `cubes/loader.ts` exactly.
|
and the three-step id resolution match `cubes/loader.ts` exactly.
|
||||||
- **pyinfra `--data` type coercion** (`README.md:101`) — correct.
|
- **pyinfra `--data` type coercion** (`README.md:101`) — correct.
|
||||||
- **keyman config** — priority (`VAULT_ROOT` > file > defaults), the four default
|
- **keyman config** — priority (`VAULT_ROOT` > file > defaults) and the four
|
||||||
values, and the vault layout match `keyman.config.ts` and `keyman.encrypt.ts`.
|
default values match `keyman.config.ts`. The third clause of this entry used to
|
||||||
|
read "and the vault layout match[es] … `keyman.encrypt.ts`", which was true only
|
||||||
|
because `encrypt.ts` hardcoded `keys` and ignored the config — checking a
|
||||||
|
documented layout against the file that ignores the configuration is what kept
|
||||||
|
that defect invisible here. Both are honest now: the layout is configurable and
|
||||||
|
`encrypt` reads the configuration (`packages/keyman/docs/AUDIT.md` §1.1, §5.1).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Suggested order of attack
|
## Suggested order of attack
|
||||||
|
|
||||||
**1 — ~~Decide on the three phantom features.~~ Two left.** §1.1 (`-D`) is
|
**1 — ~~Decide on the three phantom features.~~ Done, three different ways.**
|
||||||
**done** — implemented, tested, and verified against every cube in `cubes/`.
|
§1.1 (`-D`) was implemented, tested, and verified against every cube in `cubes/`;
|
||||||
That closed §2.2 and half of §2.1 with it, since neither could be left standing
|
that closed §2.2 and half of §2.1 with it, since neither could be left standing
|
||||||
under a run that never prompts. §1.2 (`--json`) and §1.3 (`log.*`) are still
|
under a run that never prompts. §1.2 (`--json`) was removed rather than
|
||||||
"documented, wired up, never read": each is a small implementation or a small
|
implemented, for the reason recorded there. §1.3 (`log.*`) was implemented —
|
||||||
deletion, but neither can stay documented as working.
|
`logConfigToFlags()` finally has a caller, in `buildDeployCall`.
|
||||||
|
|
||||||
**4 — Decide the `.describe()`/`.default()` ordering (§2.3).** Either read
|
**4 — ~~Decide the `.describe()`/`.default()` ordering (§2.3).~~ Done, by
|
||||||
through the `ZodDefault` wrapper in `nopy.prompts.ts`, or fix the ordering in all
|
reading through the wrapper.** `promptLabel()` in `nopy.prompts.ts` walks
|
||||||
14 manifests and the README example. The first is one line and cannot regress.
|
`default`/`optional`/`nullable` down to the described schema, so both orders
|
||||||
|
work and the 15 manifests that had it "wrong" needed no edit. The alternative —
|
||||||
|
reordering every manifest — would have left the next one free to regress.
|
||||||
|
|
||||||
**5 — ~~Regenerate `docs/API.md` (§3).~~ Done.** Rewritten against the source
|
**5 — ~~Regenerate `docs/API.md` (§3).~~ Done.** Rewritten against the source
|
||||||
rather than patched, and extended to the exports that never had an entry
|
rather than patched, and extended to the exports that never had an entry
|
||||||
(variables, history, prompts, the authoring package). One new finding came out of
|
(variables, history, prompts, the authoring package). One new finding came out of
|
||||||
it: `CubePackageRef` is not re-exported from `src/index.ts` although `NopyConfig`
|
it — `CubePackageRef` was not re-exported from `src/index.ts` although
|
||||||
refers to it — a one-line fix, left for whoever next touches the export list.
|
`NopyConfig` refers to it — and that one line has since been added.
|
||||||
|
|
||||||
**6 — Cube docs (§5) and the two missing READMEs.** `service/autostart` is the
|
**6 — Cube docs (§5).** `service/autostart` was the worst and is **done**: its
|
||||||
worst — its README belongs to a different cube, and its `deploy.py` does not run
|
`deploy.py` now reads its three variables off `host.data` instead of raising
|
||||||
at all (§6.1).
|
`NameError` (§6.1), and its README describes that cube rather than a different
|
||||||
|
one (§5.1). Still open: §5.2 (four wrong parameters in
|
||||||
|
`network/wifi/access-point`), §5.3 (two cubes claiming to have no parameters) and
|
||||||
|
the missing READMEs.
|
||||||
|
|
||||||
**7 — Secrets on stdout (§4.2, §6.4).** The `console.log` in `Variables.assign`
|
**7 — ~~Secrets on stdout (§4.2, §6.4).~~ Done as far as it can be.** The
|
||||||
is gone. Still open: mask the password in the executor's debug line and in the
|
`console.log` in `Variables.assign` is gone; the password is masked in the
|
||||||
dry-run plan, and pass `--user`/`--password` as argv rather than interpolating
|
executor's debug line and in the dry-run plan; and the whole command is argv now,
|
||||||
into a shell string.
|
run without a shell, so nothing is interpolated into a string any shell will
|
||||||
|
re-parse. What is left is inherent: pyinfra takes `--data` on its own argv, so
|
||||||
|
the value is visible in `ps` on the machine running the deploy for the length of
|
||||||
|
the run. Fixing that means a change on pyinfra's side, not this one's. §6.4's
|
||||||
|
remaining bullet is unrelated debug output.
|
||||||
|
|||||||
+742
@@ -0,0 +1,742 @@
|
|||||||
|
# Field-report implementation plan
|
||||||
|
|
||||||
|
Turns the findings of the wild-run field report into work. Ordered by severity,
|
||||||
|
then by whether a fix unblocks a later one. Every phase is independently
|
||||||
|
shippable and ends at the existing gate (`lint:ci` → `typecheck` →
|
||||||
|
`test:coverage`).
|
||||||
|
|
||||||
|
Findings the field run confirmed that `DOCS-AUDIT.md` already tracks keep their
|
||||||
|
audit number, so the two documents stay in step: closing an item here closes it
|
||||||
|
there.
|
||||||
|
|
||||||
|
## Contents
|
||||||
|
|
||||||
|
- [0. Retractions](#0-retractions) — two findings were harness artefacts
|
||||||
|
- [1. The secret leak](#1-the-secret-leak) — `env` broadcasts a credential in the clear
|
||||||
|
- [2. Remove `--json`](#2-remove---json) — audit §1.2, closed by deletion
|
||||||
|
- [3. Replay and session correctness](#3-replay-and-session-correctness) — audit §2.4, §2.5, §4.5
|
||||||
|
- [4. The first five minutes](#4-the-first-five-minutes)
|
||||||
|
- [5. Cube defects](#5-cube-defects) — audit §5.3
|
||||||
|
- [6. Documentation sweep](#6-documentation-sweep)
|
||||||
|
- [7. Harness fix and acceptance run](#7-harness-fix-and-acceptance-run)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. Retractions
|
||||||
|
|
||||||
|
Two findings in the field report were caused by the PTY driver I used to script
|
||||||
|
the TUI, not by nopy. The driver called `pty.fork()` and never issued
|
||||||
|
`TIOCSWINSZ`, so the child saw a **0×0 terminal**.
|
||||||
|
|
||||||
|
`enquirer`'s `utils.height` (`lib/utils.js:80-86`) computes a sane fallback and
|
||||||
|
then throws it away:
|
||||||
|
|
||||||
|
```js
|
||||||
|
let rows = (stream && stream.rows) ? stream.rows : fallback; // fallback = 25
|
||||||
|
if (stream && typeof stream.getWindowSize === 'function') {
|
||||||
|
rows = stream.getWindowSize()[1]; // ← unconditional
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
A TTY always has `getWindowSize`, so `height` becomes `0`, and
|
||||||
|
`ArrayPrompt.limit` (`lib/types/array.js:604`) returns `Math.min(limit, 0)`.
|
||||||
|
`visible` is then empty for every array prompt.
|
||||||
|
|
||||||
|
Re-run with a 50×200 window, both work correctly:
|
||||||
|
|
||||||
|
| Field report | Actual |
|
||||||
|
| --- | --- |
|
||||||
|
| §3.6 multi-field forms never render their fields | All four fields render, accept input, and submit: `RESULT {"USER":"X","PASSWORD":"changeme","GROUPS":"","PUBKEY":""}` |
|
||||||
|
| §3.14 cube filter says "No matching choices" while matching fine | Filter renders correctly, highlights the matched substring, and returns `["user:add"]` |
|
||||||
|
|
||||||
|
**What survives, and it is worth fixing.** nopy has no defence against a
|
||||||
|
terminal that reports a degenerate size: the form silently submits `{}`, and the
|
||||||
|
run proceeds with every variable absent. That is [§4.4](#44-survive-a-terminal-that-reports-no-size)
|
||||||
|
and [§4.5](#45-never-deploy-a-cube-with-a-missing-required-variable). Field
|
||||||
|
report §3.7 (a required key dropped from the command) was reached through the
|
||||||
|
0×0 form, but the hole it exposed is real and independent: nothing on the
|
||||||
|
interactive path checks that a cube's required variables were actually filled.
|
||||||
|
|
||||||
|
Everything else in the field report stands.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The secret leak
|
||||||
|
|
||||||
|
Highest severity: following the documentation as written prints a credential in
|
||||||
|
plaintext, and the workaround it is prescribed for does not work either.
|
||||||
|
|
||||||
|
### 1.1 A declared secret must never be broadcast to cubes that do not declare it
|
||||||
|
|
||||||
|
**What happens.** `Variables.bucket()` (`nopy.common.ts:193-203`) seeds *every*
|
||||||
|
key of config `env` onto *every* cube as an `env`-origin assignment, and
|
||||||
|
`isSecret` (`nopy.common.ts:135-137`) is keyed per cube. So with `PASSWORD` under
|
||||||
|
`env`, a dry run prints:
|
||||||
|
|
||||||
|
```
|
||||||
|
Step 1: apt:essentials … --data "PASSWORD=wildpass123" ← unmasked
|
||||||
|
Step 2: user:add … --data "PASSWORD=********" ← masked
|
||||||
|
Step 3: runtime:nodevm … --data "PASSWORD=wildpass123" ← unmasked
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why the obvious fix is wrong.** "Seed `env` only onto cubes whose schema
|
||||||
|
declares the key" breaks a shipped cube: `ssh/keyman/deploy.py:28` reads
|
||||||
|
`host.data.get('KEY_DIR')`, a key its manifest does not declare and that exists
|
||||||
|
only in `.nopyrc.json` `env` (`packages/nopy/.nopyrc.json:5`). Broadcast is
|
||||||
|
load-bearing.
|
||||||
|
|
||||||
|
**Fix.** Narrow the rule to secrets only — broadcast stays, secrets stop
|
||||||
|
travelling:
|
||||||
|
|
||||||
|
1. In `nopy.main.ts`, after `loadCubes()`, collect the union of every loaded
|
||||||
|
manifest's `secrets` and hand it to `Variables`:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const declaredSecrets = new Set(Object.values(cubes).flatMap((c) => c.secrets));
|
||||||
|
const variables = new Variables(config.env, declaredSecrets);
|
||||||
|
```
|
||||||
|
|
||||||
|
Deterministic and ordering-free: it is computed before the first
|
||||||
|
`resolveCube`, so it does not depend on which cube resolves first.
|
||||||
|
|
||||||
|
2. In `bucket()`, skip seeding an `env` key that is in `declaredSecrets` unless
|
||||||
|
the cube itself declares that key in its schema. `Variables` needs the cube's
|
||||||
|
schema keys for this — add `declareSchema(cube, keys)`, called from
|
||||||
|
`BuildContext.resolveCube` immediately after `declareSecrets`
|
||||||
|
(`cubes/dependencies.ts:121`), before any assignment creates the bucket.
|
||||||
|
|
||||||
|
3. Mark globally, mask globally: a key in `declaredSecrets` is `redacted` on
|
||||||
|
whichever cube it does land on, even if that cube's own manifest forgot to
|
||||||
|
list it. Cheap defence against a manifest that declares `PASSWORD` in `schema`
|
||||||
|
and omits it from `secrets`.
|
||||||
|
|
||||||
|
4. New optional `.nopyrc.json` key, for an `env` secret no manifest declares
|
||||||
|
(an API token a hook uses, say):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "secrets": ["DEPLOY_TOKEN"], "env": { "DEPLOY_TOKEN": "…" } }
|
||||||
|
```
|
||||||
|
|
||||||
|
Unions into `declaredSecrets`. Validate it in `nopy.config.ts` alongside the
|
||||||
|
other properties.
|
||||||
|
|
||||||
|
**Verify.** New test in `tests/common.test.ts`: `env` carrying a key that cube A
|
||||||
|
declares secret and cube B does not → B's `get()` does not contain the key; A's
|
||||||
|
does and is redacted. New test in `tests/executor.test.ts`: the printed plan for
|
||||||
|
B contains no occurrence of the value.
|
||||||
|
|
||||||
|
### 1.2 `env` must satisfy the `--use-defaults` gap check
|
||||||
|
|
||||||
|
**What happens.** `fillSessionGaps` (`cubes/dependencies.ts:79-89`) builds
|
||||||
|
`gaps` as `missingRequired ∪ cube.secrets` — *unconditionally* including every
|
||||||
|
secret, regardless of whether anything supplied a value. Under `-D` it throws,
|
||||||
|
and the message tells you to do the thing you have already done:
|
||||||
|
|
||||||
|
```
|
||||||
|
Error: Cube "user:add" cannot be replayed with --use-defaults: PASSWORD would
|
||||||
|
have to be entered. … set the values under "env" in .nopyrc.json.
|
||||||
|
```
|
||||||
|
|
||||||
|
The value **was** read — without `-D` the prompt came pre-filled from `env`.
|
||||||
|
|
||||||
|
**Fix.** Under `useDefaults`, a gap is satisfied when something outside the
|
||||||
|
session supplied it deliberately:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const unsatisfied = gaps.filter((key) => {
|
||||||
|
const origin = this.variables.of(cube.id, key)?.origin;
|
||||||
|
return origin !== 'env' && origin !== 'param';
|
||||||
|
});
|
||||||
|
if (unsatisfied.length > 0) throw new Error(…);
|
||||||
|
```
|
||||||
|
|
||||||
|
`default` is deliberately **not** accepted for a secret. On a replay the
|
||||||
|
recorded value is gone by design, so falling through to a manifest default would
|
||||||
|
deploy a different credential than the run being replayed — silently. The
|
||||||
|
message says so, instead of repeating advice that already failed:
|
||||||
|
|
||||||
|
> `Cube "user:add" cannot be replayed with --use-defaults: PASSWORD is a secret
|
||||||
|
> and secrets are never recorded in a session. Set it under "env" in
|
||||||
|
> .nopyrc.json (a schema .default() is not accepted for a secret), pass it from
|
||||||
|
> a dependency, or replay without --use-defaults.`
|
||||||
|
|
||||||
|
**Verify.** `tests/cubes.dependencies.test.ts`: `-D` replay with the secret under
|
||||||
|
`env` succeeds and the value reaches the deploy call; with only a schema
|
||||||
|
`.default()` it throws and the message names the key. Both are new cases.
|
||||||
|
|
||||||
|
### 1.3 Documentation
|
||||||
|
|
||||||
|
`README.md:291` currently prescribes exactly the leak. After 1.1 and 1.2 the
|
||||||
|
advice becomes true; add one sentence under *Secrets* stating the new rule — a
|
||||||
|
declared secret in `env` reaches only the cubes that declare it — so the
|
||||||
|
interaction between the two features is written down once, in the place a reader
|
||||||
|
of either lands.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Remove `--json`
|
||||||
|
|
||||||
|
Audit §1.2, independently confirmed: `nopy install -R --json > j.out` →
|
||||||
|
`json.load()` raises; stdout carries seven ANSI-coloured log lines and no JSON.
|
||||||
|
**Closed by deletion, not by implementation.**
|
||||||
|
|
||||||
|
**Why removal is the right call and not just the cheap one.** `executeDeployCalls`
|
||||||
|
runs pyinfra through execa with *inherited* stdio (`nopy.executor.ts:117`), so
|
||||||
|
during a real run nopy does not own its own stdout — pyinfra does, and writes an
|
||||||
|
unbounded amount to it. A JSON blob appended after that is not machine-readable
|
||||||
|
by any definition a caller could rely on; making it so means capturing pyinfra's
|
||||||
|
output and giving up live progress, which is a real feature traded for a
|
||||||
|
speculative one. That is the same root cause as the documented gap that
|
||||||
|
`ExecutionResult.stdout` is never populated. The CI case the flag was for is
|
||||||
|
already covered: `--print-only` for the plan and the exit code for the verdict
|
||||||
|
(`nopy.cli.ts:138-140` exits 1 on any failure). Nothing can depend on the current
|
||||||
|
behaviour, because there is no current behaviour.
|
||||||
|
|
||||||
|
### 2.1 The flag, from the `install` command
|
||||||
|
|
||||||
|
- `nopy.cli.ts:91` — drop the `.option('-j, --json', …)` line.
|
||||||
|
- `:133` — drop `jsonOutput: options.json` from the `nopy()` call.
|
||||||
|
- `:147-160` — the `if (options.json)` error branch collapses to the single
|
||||||
|
`console.error`. This is the same statement [§4.3](#43-routine-errors-print-a-raw-node-stack-trace)
|
||||||
|
rewrites, so whichever phase lands first does both; the other just reads it.
|
||||||
|
|
||||||
|
### 2.2 `jsonOutput`, from the library
|
||||||
|
|
||||||
|
- `NopyOptions.jsonOutput` (`nopy.main.ts:110`) and its destructure (`:141`).
|
||||||
|
This is a **breaking change to an exported interface** — `docs/API.md:273`
|
||||||
|
documents it. It is a `0.x` minor bump, and an unknown property is a type error
|
||||||
|
rather than a silent behaviour change, so a consumer finds out at compile time.
|
||||||
|
- `:148` — the banner guard becomes `if (!replaySession && !loadSessionPath)`.
|
||||||
|
- `:158` — the JSON error dump goes; `log.error` on `:156-157` already reported
|
||||||
|
the same errors.
|
||||||
|
- `:227` — the `onProgress` callback loses its guard and always logs.
|
||||||
|
- `outputExecutionPlan(calls, asJson?)` (`nopy.executor.ts:148-158`) — drop the
|
||||||
|
parameter and the dead JSON branch. Exported and documented (`docs/API.md:588`);
|
||||||
|
nothing in `src/` passes the second argument, only a test does.
|
||||||
|
|
||||||
|
### 2.3 stdout hygiene — the one fix that survives, and now matters more
|
||||||
|
|
||||||
|
With `--json` gone, `--print-only` is the machine-readable surface, so it has to
|
||||||
|
be clean. Two writers currently pollute it, and `jsonOutput` was the only thing
|
||||||
|
holding either back:
|
||||||
|
|
||||||
|
- `configureLogtape`'s console sink uses `console.log` (`nopy.main.ts:34`),
|
||||||
|
against `README.md:409`, which promises stderr. Switch to `console.error`.
|
||||||
|
- `printActiveConfig` ends in `console.log` (`nopy.main.ts:95`) and is suppressed
|
||||||
|
today only by `jsonOutput` and by replay. Same switch.
|
||||||
|
|
||||||
|
Write the rule down once, in the README: **stdout carries the deploy commands and
|
||||||
|
pyinfra's own output; everything nopy says about itself goes to stderr.** That is
|
||||||
|
a promise a test can hold to, which the old `--json` claim never was.
|
||||||
|
|
||||||
|
Ripple worth knowing before starting: `tests/main.test.ts` spies on `console.log`
|
||||||
|
(`logSpy`) throughout, so moving logtape to `console.error` means moving those
|
||||||
|
spies. Mechanical, but it touches most of the file.
|
||||||
|
|
||||||
|
### 2.4 Documentation — most of the work
|
||||||
|
|
||||||
|
| File | Change |
|
||||||
|
| --- | --- |
|
||||||
|
| `packages/nopy/README.md:537-544` | Delete the *JSON output (for CI/CD)* block. Replace with the CI recipe that works: `--print-only` for the plan, exit code `1` for the verdict, `--continue-on-error` when you want every failure in one run. |
|
||||||
|
| `packages/nopy/README.md:409` | The stderr promise stays and is now load-bearing; reword its reason from `--json` to `--print-only` and piped stdout. |
|
||||||
|
| `docs/API.md:273` | Remove the `jsonOutput` row from the `NopyOptions` table. |
|
||||||
|
| `docs/API.md:588-596` | `outputExecutionPlan(calls, asJson?)` → `outputExecutionPlan(calls)`; the note that `--dry-run --json` prints the text plan goes with it. |
|
||||||
|
| `docs/API.md:1034` | Reword the stderr note the same way as `README.md:409`. |
|
||||||
|
| `docs/API.md:1170-1173` | *Known gaps*: the `--json` entry disappears — that is the point. `ExecutionResult.stdout` is never populated **stays**, and gains the reason (stdio is inherited), since that is now the honest answer to "how do I capture output?". |
|
||||||
|
| `docs/CUBE-PACKAGES.md:317` | Future-work line proposes surfacing a cube's source "in the interactive picker and in `--json` output"; drop the second half. |
|
||||||
|
| `nopy.exit.ts:77` | Comment cites `--json` and `--print-only` as the reason for the exit discipline; leave the discipline, drop the `--json` half. |
|
||||||
|
| `DOCS-AUDIT.md:85` | Mark §1.2 closed **by removal** and say so in one line — a reader of that document should not go looking for the fix. Also touch its back-references at `:446` and `:854`. |
|
||||||
|
|
||||||
|
### 2.5 `nopy history --json` is a different flag — keep it
|
||||||
|
|
||||||
|
`nopy.cli.ts:169-177` is a second, unrelated `-j, --json`, on the `history`
|
||||||
|
command, and it works: `JSON.stringify(listHistory())`. It was never part of
|
||||||
|
audit §1.2 — the field run used it successfully. Keep it. Nothing else writes to
|
||||||
|
stdout during `history`, so it has none of the problem above, it is three lines,
|
||||||
|
and it is how a script finds the id to pass to `-H`.
|
||||||
|
|
||||||
|
If the intent is that nopy has no JSON surface at all, removing it is
|
||||||
|
`nopy.cli.ts:169` plus `:173-177`, and `README.md:541` and `:573`. Flagging it
|
||||||
|
rather than deciding it: this one is a working feature, so deleting it is a
|
||||||
|
different kind of change from deleting one that never worked.
|
||||||
|
|
||||||
|
### 2.6 Tests
|
||||||
|
|
||||||
|
Delete, rather than adapt — they assert behaviour that no longer exists:
|
||||||
|
|
||||||
|
- `tests/main.test.ts:162-168` — *emits the errors as JSON when jsonOutput is set*
|
||||||
|
- `tests/main.test.ts:224-227` — *is suppressed for JSON output* (the sibling
|
||||||
|
`replaySession` / `loadSession` suppression cases stay and still cover `:148`)
|
||||||
|
- `tests/main.test.ts:366-373` — *stays silent on progress when jsonOutput is set*
|
||||||
|
- `tests/executor.test.ts:129` — the `outputExecutionPlan(calls, true)` case
|
||||||
|
|
||||||
|
Add one that holds the new rule: run with `printOnly` and assert `console.log`
|
||||||
|
received the command block and **nothing else** — banner and progress lines on
|
||||||
|
`console.error`. Deleting a covered branch moves coverage up, not down, so the
|
||||||
|
gate is not at risk here.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Replay and session correctness
|
||||||
|
|
||||||
|
### 3.1 `--save-session` no-ops on a replay (audit §4.5)
|
||||||
|
|
||||||
|
`nopy.main.ts:199` guards with `!workflow.isReplay`, so
|
||||||
|
`nopy install -R -s out.json` exits 0 and writes nothing. Drop the guard: the
|
||||||
|
resolved cube set is exactly what the user asked to capture, and a replay's
|
||||||
|
session is no less valid than a fresh run's.
|
||||||
|
|
||||||
|
### 3.2 A `--load-session` replay is not recorded
|
||||||
|
|
||||||
|
`nopy.main.ts:203` excludes every replay from history. For `-R` and `-H` that is
|
||||||
|
right and documented (`README.md:494`) — repeating must not push the original
|
||||||
|
out of the list. For `-l` it is wrong: the run is not already in history, so
|
||||||
|
after deploying from a session file `nopy history` says *"No sessions in
|
||||||
|
history"* and `-R` has nothing to repeat. That is what happened in the field run.
|
||||||
|
|
||||||
|
Record `-l` runs; keep `-R`/`-H` non-recording. `WorkflowResult` needs to
|
||||||
|
distinguish them — replace the boolean `isReplay` with
|
||||||
|
`replaySource: 'file' | 'history' | undefined`, or add a second flag. Then fix
|
||||||
|
`README.md:496-500`, whose explicit *"a run is not recorded when"* list omits
|
||||||
|
replays entirely and so contradicts `:494`.
|
||||||
|
|
||||||
|
### 3.3 The written session does not match the documented format (audit §2.4)
|
||||||
|
|
||||||
|
Documented (`README.md:224-254`, `docs/SESSION_FORMAT.md`) versus written:
|
||||||
|
|
||||||
|
| Field | Documented | Written |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `version` | `"1.0.0"` | absent |
|
||||||
|
| `name` | `"My Deployment Session"` | absent |
|
||||||
|
| `timestamp` | ISO 8601 | absent |
|
||||||
|
| `auth.method` | `"ssh-key"` | `"ssh"` |
|
||||||
|
| `auth.username` | `"root"` | absent |
|
||||||
|
|
||||||
|
This is what you consult in order to hand-write a session, which is what the
|
||||||
|
field run had to do.
|
||||||
|
|
||||||
|
Implement rather than delete — all three fields are cheap and two are useful:
|
||||||
|
|
||||||
|
- `createSession` (`nopy.session.ts:183-197`) stamps `version: '1.0.0'` and
|
||||||
|
`timestamp: new Date().toISOString()`, and derives a default `name` the way
|
||||||
|
`generateEntryName` already does for history (`nopy.history.ts:84-102`).
|
||||||
|
- `loadSession` (`:130-158`) keeps accepting sessions without them — every
|
||||||
|
existing file and every hand-written one must stay loadable. Warn on a
|
||||||
|
`version` it does not know; do not fail.
|
||||||
|
- `auth.method: 'ssh'` is real, not a bug: `runInteractiveWorkflow:64-67` uses it
|
||||||
|
for `@vagrant/` and `@docker/` hosts, where the connector owns authentication.
|
||||||
|
It is simply undocumented. Document the third value and when it appears.
|
||||||
|
|
||||||
|
### 3.4 `listSessions` does not match the documented filename (audit §2.5)
|
||||||
|
|
||||||
|
Docs say `.nopysession.json`; `listSessions` (`nopy.session.ts:166-175`) filters
|
||||||
|
for `.session.json` / `.session.mjs`, which `wild.nopysession.json` does not
|
||||||
|
match. Widen the filter to `.nopysession.json` / `.nopysession.mjs` and keep the
|
||||||
|
old suffixes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. The first five minutes
|
||||||
|
|
||||||
|
The four roughest edges a new user meets all sit before anything that works
|
||||||
|
well.
|
||||||
|
|
||||||
|
### 4.1 The documented install command 404s
|
||||||
|
|
||||||
|
`packages/nopy-cubes-core/README.md:9`, `packages/nopy/README.md:317` and `:344`
|
||||||
|
all open with:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm add -D @bitsquare/nopy-cubes-core
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
[ERR_PNPM_FETCH_404] GET https://registry.npmjs.org/@bitsquare%2Fnopy-cubes-core: Not Found
|
||||||
|
```
|
||||||
|
|
||||||
|
The bundle has never been published to npmjs, and an *untagged* Gitea install
|
||||||
|
resolves to nothing because Gitea publishes no `latest` tag. What rescued the
|
||||||
|
field run was pnpm's own error listing `main: 0.5.0-main.17.gda84523`.
|
||||||
|
|
||||||
|
Replace both snippets with the form that works, and say why:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm add -D @bitsquare/nopy-cubes-core@main \
|
||||||
|
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
|
||||||
|
```
|
||||||
|
|
||||||
|
`nopy-cubes-core`'s README does not mention Gitea at all; nopy's mentions it only
|
||||||
|
in a *Channels* section framed around installing the CLI. Both need the tag
|
||||||
|
requirement stated where the install command is, not two sections away. Revisit
|
||||||
|
when `release.yml` first ships the bundle to npmjs — the guard in that workflow
|
||||||
|
blocks the first `nopy` release until it does.
|
||||||
|
|
||||||
|
### 4.2 pyinfra is an unstated prerequisite
|
||||||
|
|
||||||
|
Nothing in the README says pyinfra must be installed separately and on `PATH`;
|
||||||
|
`nopy.executor.ts:117` spawns it directly. The field run only worked because it
|
||||||
|
happened to be there. Add a *Requirements* block next to the install command:
|
||||||
|
Node ≥ 22, `pyinfra` on `PATH` (`pipx install pyinfra`), plus whatever the chosen
|
||||||
|
connector needs (`vagrant`, `docker`). Optionally probe for it once at startup
|
||||||
|
and fail with one line instead of a spawn error.
|
||||||
|
|
||||||
|
### 4.3 Routine errors print a raw Node stack trace
|
||||||
|
|
||||||
|
`nopy.cli.ts:159` passes the error object as a third argument:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
console.error('Error:', error instanceof Error ? error.message : error, error);
|
||||||
|
```
|
||||||
|
|
||||||
|
so the message prints, then the whole error prints again with frames into
|
||||||
|
`dist/`. Running outside a project — the most likely first-run mistake — yields:
|
||||||
|
|
||||||
|
```
|
||||||
|
Error: No .nopyrc.json found. Create one in your project directory or any parent directory.
|
||||||
|
at loadConfig (…/dist/nopy.config.js:187:15)
|
||||||
|
at Command.<anonymous> (…/dist/nopy.cli.js:74:24)
|
||||||
|
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Drop the third argument; print the stack only under `NOPY_DEBUG`. Adopt keyman's
|
||||||
|
shape (`keyman.cli.ts` is the error boundary that turns a `UsageError` into one
|
||||||
|
line) so the two CLIs stay in step: a `NopyUsageError` for the errors that are
|
||||||
|
the user's to fix — no config, no cubes, missing required variable, unknown
|
||||||
|
session — and a stack for everything else.
|
||||||
|
|
||||||
|
### 4.4 Survive a terminal that reports no size
|
||||||
|
|
||||||
|
Per [§0](#0-retractions): with `stdout.rows === 0`, every enquirer array prompt
|
||||||
|
renders "No matching choices", the form submits `{}`, and nopy deploys with every
|
||||||
|
variable defaulted. Reachable outside a test harness — some CI pseudo-terminals,
|
||||||
|
`script -q`, and editor terminals during startup all report 0 rows.
|
||||||
|
|
||||||
|
Passing an explicit `limit` does **not** help (measured): enquirer clamps it with
|
||||||
|
`Math.min(limit, this.height)`. But `height` itself has an escape hatch one line
|
||||||
|
above the bug — `prompt.js:396`:
|
||||||
|
|
||||||
|
```js
|
||||||
|
get height() { return this.options.rows || utils.height(this.stdout, 25); }
|
||||||
|
```
|
||||||
|
|
||||||
|
`options.rows` short-circuits the broken function entirely, so the fix is to pass
|
||||||
|
a floored size rather than to fake a stdout:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const MIN_ROWS = 24, MIN_COLS = 80;
|
||||||
|
const terminalSize = (out = process.stdout) => ({
|
||||||
|
rows: Math.max(out.rows || 0, MIN_ROWS),
|
||||||
|
columns: Math.max(out.columns || 0, MIN_COLS),
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Measured, 2×2:
|
||||||
|
|
||||||
|
| PTY | without | with |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 0×0 | `RESULT {}` | `RESULT {"USER":"X","PASSWORD":"changeme","GROUPS":"","PUBKEY":""}` |
|
||||||
|
| 50×200 | full result | full result (`rows` passes through as 50) |
|
||||||
|
|
||||||
|
Apply to both enquirer call sites — `CubeSelection` (`nopy.prompts.ts:61-68`) and
|
||||||
|
`VariableAssignment` (`:238-243`) — and derive `pageSize` (`:55-56`) from the same
|
||||||
|
helper, where `process.stdout.rows || 24` already fails for `0` only to be clamped
|
||||||
|
away again.
|
||||||
|
|
||||||
|
*(An earlier draft of this section proposed a `Proxy` over `process.stdout`
|
||||||
|
reporting the floor. It works — also measured — but it fakes a stream object to
|
||||||
|
reach a value the prompt will take directly. `options.rows` is the same fix
|
||||||
|
without the impersonation.)*
|
||||||
|
|
||||||
|
enquirer 2.4.1 is the last release (2023) and this is its bug. Worth a comment at
|
||||||
|
the call site so nobody "simplifies" the sizes away later.
|
||||||
|
|
||||||
|
### 4.5 Never deploy a cube with a missing required variable
|
||||||
|
|
||||||
|
Field report §3.7. `README.md:99` guarantees *"Every key defined in the manifest
|
||||||
|
`schema` is guaranteed to be present on `host.data`"*, and the interactive path
|
||||||
|
does not enforce it: `resolveCube` calls `VariableAssignment`
|
||||||
|
(`cubes/dependencies.ts:138`) and goes straight to `buildDeployCall`.
|
||||||
|
`assertVariablesComplete` exists and runs **only** under `useDefaults` (`:136`).
|
||||||
|
`buildDeployCall` then emits `--data` for whatever variables exist
|
||||||
|
(`:186-189`), so a key nothing ever assigned is absent from the command
|
||||||
|
entirely and the deploy script reads `None`.
|
||||||
|
|
||||||
|
Two ways in, both real: a form that submits nothing (§4.4), and a form the user
|
||||||
|
cancels — `VariableAssignment`'s `catch {}` (`nopy.prompts.ts:252-254`) swallows
|
||||||
|
cancellation and returns as though it succeeded.
|
||||||
|
|
||||||
|
- Call the completeness check on the interactive path too, with a message that
|
||||||
|
fits: `Cube "user:add" is missing PUBKEY. It has no default value and nothing
|
||||||
|
supplied one.` (The replay path already does this at
|
||||||
|
`cubes/dependencies.ts:94-100`.)
|
||||||
|
- Distinguish cancel from error in `VariableAssignment` and route a cancel
|
||||||
|
through `nopy.exit.ts` like the other prompts, instead of continuing with a
|
||||||
|
half-filled cube.
|
||||||
|
|
||||||
|
**Verify.** `tests/cubes.dependencies.test.ts`: a cube with a required
|
||||||
|
no-default key, with the form stubbed to return `{}` → resolution throws and
|
||||||
|
names the key. This test fails today.
|
||||||
|
|
||||||
|
### 4.6 `self-update` prints a command that cannot work
|
||||||
|
|
||||||
|
From a project with no scope mapping in `.npmrc`:
|
||||||
|
|
||||||
|
```
|
||||||
|
Channel: main
|
||||||
|
Registry: https://registry.npmjs.org/
|
||||||
|
Available: unknown
|
||||||
|
Would run: npm install --global @bitsquare/nopy@main
|
||||||
|
```
|
||||||
|
|
||||||
|
`main` snapshots exist only on Gitea, and `buildSelfUpdateCommand`
|
||||||
|
(`nopy.update.ts:378-381`) deliberately omits the registry flag when the registry
|
||||||
|
*is* npmjs — correct in general, wrong for this combination. The channel is
|
||||||
|
derived from the running version, so nopy already knows the command is
|
||||||
|
unrunnable.
|
||||||
|
|
||||||
|
Detect `channel === 'main' && registry === NPMJS_REGISTRY` in the CLI action and
|
||||||
|
refuse with a line that fixes it:
|
||||||
|
|
||||||
|
> `You are running a main snapshot, which is published to Gitea only, but
|
||||||
|
> @bitsquare resolves to npmjs. Re-run with --registry <url>, or set it once:
|
||||||
|
> npm config set @bitsquare:registry <url>`
|
||||||
|
|
||||||
|
**Verify.** `tests/update.test.ts` already covers `buildSelfUpdateCommand`'s
|
||||||
|
registry logic; add the combination case.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Cube defects
|
||||||
|
|
||||||
|
### 5.1 `GLOBAL_PACKAGES` is accepted and then ignored
|
||||||
|
|
||||||
|
`runtime/nodevm/deploy.py:9` reads `GLOBAL_PACKAGES` off `host.data` and never
|
||||||
|
uses it; `:49` hardcodes the list. The field run passed
|
||||||
|
`GLOBAL_PACKAGES=npm-check-updates`, watched it appear in the plan and on the
|
||||||
|
command line, and found it absent from `npm ls -g`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
"npm install -g pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx"
|
||||||
|
```
|
||||||
|
|
||||||
|
Fix both halves so behaviour does not change for anyone who never set the
|
||||||
|
variable — use the parameter in the deploy, and make the manifest default the
|
||||||
|
list that is hardcoded today:
|
||||||
|
|
||||||
|
```python
|
||||||
|
f"npm install -g {GLOBAL_PACKAGES}"
|
||||||
|
```
|
||||||
|
|
||||||
|
```js
|
||||||
|
GLOBAL_PACKAGES: z.string()
|
||||||
|
.describe('Space-separated list of global npm packages to install')
|
||||||
|
.default('pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx'),
|
||||||
|
```
|
||||||
|
|
||||||
|
The current default (`npm-check-updates`) is not what the cube installs, so
|
||||||
|
today's default is wrong in both directions.
|
||||||
|
|
||||||
|
### 5.2 `runtime/nodevm/README.md` describes a different cube (audit §5.3)
|
||||||
|
|
||||||
|
| README says | Manifest / `deploy.py` |
|
||||||
|
| --- | --- |
|
||||||
|
| "This cube currently has no configurable parameters" (`:44`) | `VERSION`, `USER`, `ALIAS`, `GLOBAL_PACKAGES`, and `SHELL` after [§5.3](#53-runtimenodevm-has-an-undeclared-shell-dependency--add-a-shell-parameter) |
|
||||||
|
| "official NodeSource setup script" (`:22`) | `nvm` — `deploy.py:34-37` |
|
||||||
|
| "Installs the latest LTS version" (`:23`) | whatever `VERSION` says, default `v22.20.0` |
|
||||||
|
| "npm@11.1.0" in the global list (`:33`) | not installed |
|
||||||
|
| "Node.js is installed system-wide" (`:80`) | per-user under `~/.nvm` for `USER` |
|
||||||
|
|
||||||
|
Rewrite against the manifest. It is the only file that would tell a reader
|
||||||
|
`VERSION` or `USER` exist. `runtime/docker/README.md` makes the same
|
||||||
|
"no configurable parameters" claim with a `DISTRO` field — same fix, same commit.
|
||||||
|
|
||||||
|
### 5.3 `runtime:nodevm` has an undeclared shell dependency — add a `SHELL` parameter
|
||||||
|
|
||||||
|
`dependencies: () => []`, but `deploy.py` runs `omf install nvm` (`:35`) and sets
|
||||||
|
`_shell_executable='/usr/bin/fish'` (`:43`) — it needs fish **and** Oh My Fish
|
||||||
|
already installed for `USER`. In the field run it worked only because `user:add`
|
||||||
|
ran first and installs both. Declaring `user:add` as a dependency would be wrong:
|
||||||
|
it would create a user that is usually meant to already exist.
|
||||||
|
|
||||||
|
**Fix.** Make the shell a parameter — `SHELL: 'fish' | 'bash'` — so the cube can
|
||||||
|
be standalone, as its manifest already claims, without taking fish away from
|
||||||
|
anyone using it today.
|
||||||
|
|
||||||
|
```js
|
||||||
|
SHELL: z
|
||||||
|
.enum(['fish', 'bash'])
|
||||||
|
.describe('Login shell to install through. fish needs Oh My Fish; bash needs nothing')
|
||||||
|
.default('fish'),
|
||||||
|
```
|
||||||
|
|
||||||
|
**Default stays `fish`, deliberately.** nvm wires itself into whichever shell
|
||||||
|
installed it, so switching the default would leave an existing user — whose login
|
||||||
|
shell `user:add` set to fish — with node installed and invisible. Additive
|
||||||
|
change; the escape hatch for a fresh host is one variable.
|
||||||
|
|
||||||
|
Three places differ, and only three:
|
||||||
|
|
||||||
|
| | `fish` | `bash` |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `_shell_executable` | `/usr/bin/fish` | `/bin/bash` |
|
||||||
|
| Loading nvm | `omf install nvm` — the plugin defines `nvm` as a fish function that every login shell loads | `export NVM_DIR="$HOME/.nvm"; . "$NVM_DIR/nvm.sh"` |
|
||||||
|
| Making `npm` reachable | the plugin activates the `default` alias on load | `nvm use <ALIAS>` first |
|
||||||
|
|
||||||
|
The bash arm has one non-obvious constraint: **every entry in `commands` is its
|
||||||
|
own shell**, so sourcing `nvm.sh` and using `nvm` have to be a single entry.
|
||||||
|
Sourcing cannot be skipped either — nvm's installer appends to `~/.bashrc`, and
|
||||||
|
Ubuntu's `~/.bashrc` returns at line 1 for a non-interactive shell, so the hook
|
||||||
|
never runs under `su -c`. fish has no equivalent problem, which is presumably why
|
||||||
|
it was chosen.
|
||||||
|
|
||||||
|
That same constraint makes `set -gx NVM_DIR $HOME/.nvm` (`:38`) **dead today** —
|
||||||
|
its own shell, exported, exits. Drop it; the fish plugin sets `NVM_DIR` itself.
|
||||||
|
|
||||||
|
**Guard.** With `SHELL: 'fish'` on a host without fish, fail early and legibly
|
||||||
|
rather than inside `omf`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
if SHELL == 'fish' and not host.get_fact(Which, 'fish'):
|
||||||
|
raise DeployError(
|
||||||
|
f'runtime:nodevm: SHELL is "fish" but fish is not installed for {USER}. '
|
||||||
|
'Run user:add first, or set SHELL=bash.'
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Deliberately checks the binary only. Oh My Fish is a set of fish functions with
|
||||||
|
no binary to probe and no fixed path, so a check for it would be guesswork; if
|
||||||
|
fish is present and omf is not, `omf install nvm` fails with its own clear
|
||||||
|
message. Half a guard that is certain beats a whole one that is not.
|
||||||
|
|
||||||
|
**While in the file** — `deploy.py:1-6` imports `npm` and `python` and never uses
|
||||||
|
them, and assigns `hasNode = host.get_fact(Which, 'node')`, also unused. The
|
||||||
|
`Which` import stops being dead the moment the guard lands.
|
||||||
|
|
||||||
|
**Verify.** No test harness reaches a cube deploy script, so this is acceptance,
|
||||||
|
not unit: [§7.3](#7-harness-fix-and-acceptance-run) runs `runtime:nodevm` with
|
||||||
|
`SHELL=bash` on a **fresh** VM where `user:add` has not run, and confirms
|
||||||
|
`node -v` and the `GLOBAL_PACKAGES` list for `USER`. That is the case the cube has
|
||||||
|
never survived.
|
||||||
|
|
||||||
|
### 5.4 `VERSION` accepts `null` and would install `None`
|
||||||
|
|
||||||
|
`z.nullable(z.string()).default('v22.20.0')`, and `deploy.py:36` interpolates it
|
||||||
|
straight into `nvm install {VERSION}`. Either drop `nullable`, or handle `None`
|
||||||
|
as "latest LTS" — which is what the README claims the cube does anyway.
|
||||||
|
|
||||||
|
### 5.5 `user/add/README.md` — trim, do not rewrite
|
||||||
|
|
||||||
|
Every claim it makes was verified true in the field run, and its notes on *why*
|
||||||
|
`PUBKEY` has no default and *why* the generated password was removed are the best
|
||||||
|
documentation in the repo. Its last ~50 lines are generic Fish keybinding tips
|
||||||
|
(`Ctrl+L` → clear the terminal) unrelated to the cube. Move them somewhere they
|
||||||
|
belong or delete them; leave the rest alone.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Documentation sweep
|
||||||
|
|
||||||
|
Small, mechanical, no code.
|
||||||
|
|
||||||
|
| File | Change |
|
||||||
|
| --- | --- |
|
||||||
|
| `docs/VAGRANT.md` | Never states the `@vagrant/<name>` host syntax — the field run inferred it from an unrelated `@docker/` example. Add `"hosts": ["@vagrant/nopytestvm"]` and one line tying the VM name to it. Add `vagrant destroy -f` for cleanup. |
|
||||||
|
| `README.md` (root) | Lists a `cubes/` directory at the repo root that no longer exists (`:10`); describes `typecheck` as `tsc --build --noEmit` (`:31`), which TS rejects outright for a project with references. |
|
||||||
|
| `packages/nopy/README.md` | Top-level `--help` lists only `-V`/`-h`, then the *Examples* block uses `-R`, `-n`, `-P`, `-l`, `-s` — all of which live on `install`. Either add a "these are `install` options" line to the help text (`nopy.cli.ts:56-76`) or promote the common ones. |
|
||||||
|
| `packages/nopy/README.md` | The host picker offers `docker`, `vagrant`, `@vagrant/…`, `custom`; the first two appear in no document. Add the two connector shortcuts and what they prompt for. |
|
||||||
|
| `docs/SESSION_FORMAT.md` | Uses `.session.json` throughout while the README uses `.nopysession.json`. Pick one — `.nopysession.json` — and align both, together with [§3.4](#34-listsessions-does-not-match-the-documented-filename-audit-25). |
|
||||||
|
|
||||||
|
Also worth stating once, somewhere prominent: the accuracy failures cluster on
|
||||||
|
one seam. Everything a human reads on screen matched the docs; everything
|
||||||
|
machine-facing had drifted — `--json`, the session format, `-s` on replay, `-l`
|
||||||
|
and history, `nodevm`'s parameters, the install command. That is not random rot,
|
||||||
|
it is the interactive surface being maintained by daily use while the scripting
|
||||||
|
surface was documented from intent. Phases 2 and 3 are the correction, in the two
|
||||||
|
ways available: `--json` was documented from intent and never built, so it goes;
|
||||||
|
the rest was built and then drifted, so it gets fixed. Keeping it corrected means
|
||||||
|
what remains of the scripting surface — `--print-only`, sessions, history —
|
||||||
|
needs tests that assert on **stdout**, not prose.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Harness fix and acceptance run
|
||||||
|
|
||||||
|
**7.1** Fix the PTY driver before it lies again: `drive.py` and `expect.py` must
|
||||||
|
issue `TIOCSWINSZ` after `pty.fork()`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
fcntl.ioctl(fd, termios.TIOCSWINSZ, struct.pack('HHHH', rows, cols, 0, 0))
|
||||||
|
```
|
||||||
|
|
||||||
|
Worth keeping the drivers — they are the only way to test the TUI end to end —
|
||||||
|
so they belong in the repo under `scripts/`, not in a temp folder.
|
||||||
|
|
||||||
|
**7.2** Add a 0-rows regression test that exercises §4.4 directly: spawn the CLI
|
||||||
|
under a 0×0 PTY and assert the form still yields values. It has to be a real
|
||||||
|
child process for the same reason `cubes.resolve-hook.test.ts` does — inside a
|
||||||
|
vitest worker there is no TTY to misreport.
|
||||||
|
|
||||||
|
**7.3** Acceptance: re-run the field scenario from an empty directory —
|
||||||
|
`vagrant up`, install the bundle with the [§4.1](#41-the-documented-install-command-404s)
|
||||||
|
command, deploy `user:add` and `runtime:nodevm` **interactively** (not from a
|
||||||
|
hand-written session), then check:
|
||||||
|
|
||||||
|
- `npm ls -g` contains what `GLOBAL_PACKAGES` asked for (§5.1)
|
||||||
|
- on a **second, fresh** VM where `user:add` has *not* run, `runtime:nodevm` with
|
||||||
|
`SHELL=bash` installs node and the global packages; with `SHELL=fish` it fails
|
||||||
|
in one line naming the missing shell rather than inside `omf` (§5.3)
|
||||||
|
- `nopy install -P 2>/dev/null` prints the deploy commands and nothing else — no
|
||||||
|
banner, no progress lines, no update hint (§2.3)
|
||||||
|
- `nopy install --json` is rejected as an unknown option (§2.1)
|
||||||
|
- `nopy history` lists the `-l` run (§3.2)
|
||||||
|
- a dry run with a secret under `env` prints `********` on every cube (§1.1)
|
||||||
|
- `nopy` in an unconfigured directory prints one line (§4.3)
|
||||||
|
|
||||||
|
### What the run found
|
||||||
|
|
||||||
|
Every check above passed. One deviation and three findings.
|
||||||
|
|
||||||
|
**Deviation.** The bundle was installed from `pnpm pack` tarballs of the three
|
||||||
|
packages rather than from Gitea, because the cube fixes this plan makes are not
|
||||||
|
in any published snapshot and publishing one means pushing to `main`. The install
|
||||||
|
still goes through `cubePackages` → `node_modules` → `<root>/cubes`, which is the
|
||||||
|
part §4.1 is about; what it does *not* exercise is the registry and dist-tag half
|
||||||
|
of the documented command.
|
||||||
|
|
||||||
|
Two VMs, as specified: the first got `user:add` then `runtime:nodevm` with
|
||||||
|
`SHELL=fish`, the second (destroyed and recreated, no fish, no `user:add`) got
|
||||||
|
`runtime:nodevm` alone under both shells. Driven through `scripts/expect.py`, so
|
||||||
|
the interactive path is what was exercised.
|
||||||
|
|
||||||
|
**Findings**, all recorded in `DOCS-AUDIT.md`:
|
||||||
|
|
||||||
|
- §6.8 — `--print-only` was recorded in history where `--dry-run` was not, so a
|
||||||
|
`-P` pass displaced the last real deployment at the head of what `-R` repeats.
|
||||||
|
Fixed.
|
||||||
|
- §6.9 — picking `user:add` and `runtime:nodevm` together resolves nodevm first.
|
||||||
|
The first write-up blamed the ordering and was wrong: emission is already
|
||||||
|
post-order over `dependencies()`, so a declared edge wins over list order
|
||||||
|
whichever way round the two were listed, and a test now pins that. What list
|
||||||
|
order decides is where a cube with *no* edge lands — and `runtime:nodevm`
|
||||||
|
declares none, deliberately, because `user:add` creates a user. §5.3's
|
||||||
|
`DeployError` is the guard for that pair; the acceptance run used two
|
||||||
|
invocations.
|
||||||
|
- §6.10 — `runtime:nodevm` installed apt packages without refreshing the index,
|
||||||
|
which only surfaced once §5.3 let the cube run on a box where `apt:essentials`
|
||||||
|
had not. Fixed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Suggested order
|
||||||
|
|
||||||
|
1. **Phase 1** — the leak. Security, and the fix is contained.
|
||||||
|
2. **Phase 4.3–4.5** — the error boundary, the terminal proxy, and the
|
||||||
|
completeness check. Small, and they stop a silently wrong deployment.
|
||||||
|
3. **Phase 2** — remove `--json`. Mostly deletion, and it settles what the
|
||||||
|
scripting surface *is* before phase 6 documents it.
|
||||||
|
4. **Phase 6 + 4.1 + 4.2** — documentation. No code, immediate payoff for the
|
||||||
|
next new user.
|
||||||
|
5. **Phase 5** — cubes. Independent of everything above; ships with the bundle,
|
||||||
|
not the CLI.
|
||||||
|
6. **Phase 3** — session and replay. Largest surface, lowest severity.
|
||||||
|
7. **Phase 7** — harness and acceptance, last, so it exercises all of it.
|
||||||
+75
-17
@@ -131,7 +131,7 @@ tarball.
|
|||||||
|
|
||||||
```
|
```
|
||||||
checkout → resolve tag → check secrets
|
checkout → resolve tag → check secrets
|
||||||
→ pnpm → node → cache → install → check linked deps are released
|
→ pnpm → node → cache → install
|
||||||
→ lint:ci → typecheck → test:coverage → build → verify-pack
|
→ lint:ci → typecheck → test:coverage → build → verify-pack
|
||||||
→ publish to Gitea → publish to npmjs → delete .npmrc
|
→ publish to Gitea → publish to npmjs → delete .npmrc
|
||||||
→ create the Gitea release → step summary
|
→ create the Gitea release → step summary
|
||||||
@@ -141,13 +141,14 @@ Tag resolution and the secret check run **before** anything is installed or
|
|||||||
built, so a malformed tag or a missing token fails in seconds instead of after
|
built, so a malformed tag or a missing token fails in seconds instead of after
|
||||||
the whole gate.
|
the whole gate.
|
||||||
|
|
||||||
*Check linked deps are released* asks npmjs whether every `workspace:` dependency
|
There used to be a *check linked deps are released* step between install and
|
||||||
of the package being released already exists at the version pnpm is about to
|
lint, refusing to publish a package whose `workspace:` dependency was not yet on
|
||||||
bake in (`scripts/linked-deps.mjs` → `npm view`). Tagging `nopy-v1.3.0` while
|
npmjs. It is gone: [`scripts/release.mjs`](#cutting-a-release) is what creates
|
||||||
`@bitsquare/nopy-cubes@1.1.0` is still unpublished would otherwise ship a tarball
|
release tags now, and it pushes them dependency-first and waits for each version
|
||||||
nobody can install, and npmjs only lets you unpublish for 72 hours. The check is
|
to resolve on npmjs before pushing the next — so the ordering is enforced before
|
||||||
npmjs-only: it runs before any credentials are written, and npmjs is the registry
|
CI sees a tag rather than after. The trade is that a tag pushed by hand is no
|
||||||
where the mistake is permanent.
|
longer caught; `node scripts/linked-deps.mjs <dir>` still prints what a package
|
||||||
|
would bake in if you want to check yourself.
|
||||||
|
|
||||||
## The verification gate
|
## The verification gate
|
||||||
|
|
||||||
@@ -236,14 +237,60 @@ edit is discarded with the workspace and is never committed.
|
|||||||
|
|
||||||
## Cutting a release
|
## Cutting a release
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm run release
|
||||||
|
```
|
||||||
|
|
||||||
|
`scripts/release.mjs` does the whole sequence: pick the packages, pick each
|
||||||
|
version, write the release notes, run the gate, commit, tag and push. Everything
|
||||||
|
below describes what it does and how to do it by hand.
|
||||||
|
|
||||||
|
It runs in this order, and the order is the point:
|
||||||
|
|
||||||
|
1. **Preflight.** Refuses a dirty working tree (a release commit must contain the
|
||||||
|
bump and nothing else), warns if you are not on `main`, and refuses to run
|
||||||
|
when `main` is behind the remote — a tag on a stale commit ships a tree
|
||||||
|
nobody reviewed.
|
||||||
|
2. **Pick.** A checklist of the publishable packages, each annotated with its
|
||||||
|
local version and what npmjs already has. If you select a package that others
|
||||||
|
link to, it says so and offers to add them.
|
||||||
|
3. **Version.** `patch`/`minor`/`major`/`prerelease` computed from the manifest,
|
||||||
|
or type your own. Versions already on npmjs, and versions whose tag exists,
|
||||||
|
are shown struck out and cannot be chosen. A version that will not move
|
||||||
|
`latest` gets a warning rather than a refusal.
|
||||||
|
4. **Notes.** Opens `$EDITOR` seeded with the commits since the package's last
|
||||||
|
tag, and prepends the result to `packages/<pkg>/CHANGELOG.md` in the format
|
||||||
|
the release body parser expects.
|
||||||
|
5. **Verify.** `lint:ci → typecheck → test:coverage → build → verify-pack`,
|
||||||
|
against the bumped tree and **before** the commit, so a failure leaves nothing
|
||||||
|
to unpick — it offers to restore the tree instead.
|
||||||
|
6. **Commit, tag, push.** One commit, one annotated tag per package, then the
|
||||||
|
branch, then the tags **dependency-first**. After each tag it polls npmjs
|
||||||
|
until that exact version resolves before pushing the next.
|
||||||
|
|
||||||
|
Useful flags:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm run release -- --dry-run # print the plan, change nothing
|
||||||
|
pnpm run release -- -p nopy -v minor # skip the pickers
|
||||||
|
pnpm run release -- -p nopy-cubes nopy # several, ordered automatically
|
||||||
|
pnpm run release -- --no-verify # skip the gate (it still runs in CI)
|
||||||
|
pnpm run release -- --no-wait # push tags back to back
|
||||||
|
```
|
||||||
|
|
||||||
|
The push uses `SKIP_SIMPLE_GIT_HOOKS=1`, because the `pre-push` gate is the same
|
||||||
|
one step 5 just ran against the same tree.
|
||||||
|
|
||||||
|
### By hand
|
||||||
|
|
||||||
1. Bump `version` in `packages/<pkg>/package.json`.
|
1. Bump `version` in `packages/<pkg>/package.json`.
|
||||||
2. Add a changelog entry (see below).
|
2. Add a changelog entry (see below).
|
||||||
3. Commit, merge to `main`, and let the snapshot workflow go green.
|
3. Commit, merge to `main`, and let the snapshot workflow go green.
|
||||||
4. Tag that commit and push the tag:
|
4. Tag that commit and push the tag:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
git tag nopy-v1.2.0
|
git tag nopy-v<version>
|
||||||
git push origin nopy-v1.2.0
|
git push origin nopy-v<version>
|
||||||
```
|
```
|
||||||
|
|
||||||
The tag name is `<directory>-v<version>` — the directory under `packages/`, not
|
The tag name is `<directory>-v<version>` — the directory under `packages/`, not
|
||||||
@@ -267,10 +314,13 @@ waiting for each run to go green:
|
|||||||
nopy-cubes → nopy, nopy-cubes-core (these two are independent of each other)
|
nopy-cubes → nopy, nopy-cubes-core (these two are independent of each other)
|
||||||
```
|
```
|
||||||
|
|
||||||
Release `nopy` first and the run stops at the *check linked deps* step, telling
|
`pnpm run release` handles this for you — it sorts the selection over the
|
||||||
you the `nopy-cubes` version it wanted is not on npmjs. That is the guard working;
|
`workspace:` edges and will not push `nopy`'s tag until `nopy-cubes`'s new
|
||||||
release `nopy-cubes`, then re-tag. `node scripts/publish-order.mjs` prints the
|
version answers on npmjs. Releasing by hand, you own it: tag `nopy` first and its
|
||||||
order if you would rather not reason about it.
|
run publishes a tarball requiring a `nopy-cubes` version that does not exist, and
|
||||||
|
npmjs only lets you unpublish for 72 hours. `node scripts/publish-order.mjs`
|
||||||
|
prints the order, and `node scripts/linked-deps.mjs <dir>` prints exactly which
|
||||||
|
versions a package would bake in.
|
||||||
|
|
||||||
Bumping `nopy-cubes` means bumping the packages that depend on it in the same
|
Bumping `nopy-cubes` means bumping the packages that depend on it in the same
|
||||||
change — the `workspace:*` range resolves to whatever version is in the workspace
|
change — the `workspace:*` range resolves to whatever version is in the workspace
|
||||||
@@ -299,10 +349,12 @@ What a successful run leaves behind:
|
|||||||
|
|
||||||
## Changelogs and release notes
|
## Changelogs and release notes
|
||||||
|
|
||||||
Neither package has a `CHANGELOG.md` yet. Without one, the Gitea release body is
|
`pnpm run release` writes these for you — it opens `$EDITOR` seeded with the
|
||||||
just the install snippet — nothing fails.
|
commits since the package's last tag and prepends a `## <version> — <date>`
|
||||||
|
section, creating the file the first time. A package with no `CHANGELOG.md` is
|
||||||
|
fine: the Gitea release body degrades to the install snippet and nothing fails.
|
||||||
|
|
||||||
When you add one, `release.yml` extracts the section for the version being
|
`release.yml` extracts the section for the version being
|
||||||
released. The parser is deliberately dumb: it looks for the first `## ` heading
|
released. The parser is deliberately dumb: it looks for the first `## ` heading
|
||||||
whose text contains the version string, and takes every line until the next `## `
|
whose text contains the version string, and takes every line until the next `## `
|
||||||
heading. Any of these work:
|
heading. Any of these work:
|
||||||
@@ -644,6 +696,12 @@ node scripts/publish-order.mjs # the order to release in
|
|||||||
node scripts/linked-deps.mjs packages/nopy # what must be on the registry first
|
node scripts/linked-deps.mjs packages/nopy # what must be on the registry first
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Rehearse a release without touching anything:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm run release -- --dry-run
|
||||||
|
```
|
||||||
|
|
||||||
See what is on each registry, and which versions Gitea has that npmjs does not:
|
See what is on each registry, and which versions Gitea has that npmjs does not:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
|
|||||||
@@ -4,10 +4,11 @@ Infrastructure tooling monorepo: two published CLIs plus the pyinfra "cubes"
|
|||||||
they deploy.
|
they deploy.
|
||||||
|
|
||||||
| Path | Package | Binary | What it is |
|
| Path | Package | Binary | What it is |
|
||||||
| ----------------- | ------------------ | -------- | --------------------------------------------------- |
|
| -------------------------- | ---------------------------- | -------- | --------------------------------------------------- |
|
||||||
| `packages/nopy` | `@bitsquare/nopy` | `nopy` | interactive pyinfra script management and execution |
|
| `packages/nopy` | `@bitsquare/nopy` | `nopy` | interactive pyinfra script management and execution |
|
||||||
| `packages/keyman` | `@bitsquare/keyman` | `keyman` | SSH key management with `age` encryption |
|
| `packages/keyman` | `@bitsquare/keyman` | `keyman` | SSH key management with `age` encryption |
|
||||||
| `cubes/` | — | — | the deployment units `nopy` runs |
|
| `packages/nopy-cubes` | `@bitsquare/nopy-cubes` | — | the authoring surface a cube's `manifest.mjs` imports |
|
||||||
|
| `packages/nopy-cubes-core` | `@bitsquare/nopy-cubes-core` | — | the core bundle of deployment units `nopy` runs |
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
npm install -g @bitsquare/nopy @bitsquare/keyman
|
npm install -g @bitsquare/nopy @bitsquare/keyman
|
||||||
@@ -28,7 +29,7 @@ pnpm install
|
|||||||
| Command | Does |
|
| Command | Does |
|
||||||
| --------------------------- | --------------------------------------------------- |
|
| --------------------------- | --------------------------------------------------- |
|
||||||
| `pnpm run build` | compiles both packages with `tsc` |
|
| `pnpm run build` | compiles both packages with `tsc` |
|
||||||
| `pnpm run typecheck` | `tsc --build --noEmit` across the workspace |
|
| `pnpm run typecheck` | `tsc --build` across the workspace (see below) |
|
||||||
| `pnpm run lint` | Biome check |
|
| `pnpm run lint` | Biome check |
|
||||||
| `pnpm run lint:fix` | Biome check with fixes applied |
|
| `pnpm run lint:fix` | Biome check with fixes applied |
|
||||||
| `pnpm test` | vitest, both packages |
|
| `pnpm test` | vitest, both packages |
|
||||||
@@ -36,7 +37,11 @@ pnpm install
|
|||||||
| `pnpm run coverage:summary` | renders the last coverage run as a Markdown table |
|
| `pnpm run coverage:summary` | renders the last coverage run as a Markdown table |
|
||||||
|
|
||||||
`typescript` is on the 7.x native compiler, so `tsc` *is* the fast one — there is
|
`typescript` is on the 7.x native compiler, so `tsc` *is* the fast one — there is
|
||||||
no separate `tsgo` binary to keep in sync. Each package also has a dev-run script
|
no separate `tsgo` binary to keep in sync. `typecheck` is plain `tsc --build`,
|
||||||
|
not `--noEmit`: once a project has `references`, TypeScript rejects `--noEmit`
|
||||||
|
outright (TS6310), because a composite project has to emit the declarations its
|
||||||
|
dependents read. So the typecheck writes `dist` as a side effect — gitignored,
|
||||||
|
and it means the gate also proves the build works. Each package also has a dev-run script
|
||||||
(`pnpm --filter @bitsquare/nopy run nopy`) that executes the TypeScript sources
|
(`pnpm --filter @bitsquare/nopy run nopy`) that executes the TypeScript sources
|
||||||
directly through `tsx`.
|
directly through `tsx`.
|
||||||
|
|
||||||
|
|||||||
+6
-1
@@ -14,6 +14,7 @@
|
|||||||
"test:coverage": "pnpm -r run test:coverage",
|
"test:coverage": "pnpm -r run test:coverage",
|
||||||
"coverage:summary": "node scripts/coverage-summary.mjs",
|
"coverage:summary": "node scripts/coverage-summary.mjs",
|
||||||
"registry:status": "node scripts/registry-status.mjs",
|
"registry:status": "node scripts/registry-status.mjs",
|
||||||
|
"release": "node scripts/release.mjs",
|
||||||
"try:snapshot": "node scripts/try-snapshot.mjs",
|
"try:snapshot": "node scripts/try-snapshot.mjs",
|
||||||
"typecheck": "tsc --build",
|
"typecheck": "tsc --build",
|
||||||
"lint": "biome check .",
|
"lint": "biome check .",
|
||||||
@@ -31,7 +32,11 @@
|
|||||||
"@bitsquare/nopy-cubes-core": "workspace:*",
|
"@bitsquare/nopy-cubes-core": "workspace:*",
|
||||||
"@logtape/logtape": "^2.2.4",
|
"@logtape/logtape": "^2.2.4",
|
||||||
"@types/node": "^26.1.1",
|
"@types/node": "^26.1.1",
|
||||||
|
"commander": "^15.0.0",
|
||||||
|
"enquirer": "^2.4.1",
|
||||||
|
"semver": "^7.8.5",
|
||||||
"simple-git-hooks": "^2.13.1",
|
"simple-git-hooks": "^2.13.1",
|
||||||
"typescript": "^7.0.2"
|
"typescript": "^7.0.2",
|
||||||
|
"zx": "^8.8.5"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,9 +6,18 @@ base packages, users, SSH, firewalling, networking, web serving and runtimes.
|
|||||||
## Install
|
## Install
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
pnpm add -D @bitsquare/nopy-cubes-core
|
pnpm add -D @bitsquare/nopy-cubes-core@main \
|
||||||
|
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Both halves are required today.** This bundle has not been published to npmjs
|
||||||
|
yet, so it comes from the Gitea registry — and that registry publishes no
|
||||||
|
`latest` tag, so an *untagged* install resolves to nothing at all. Name `@main`
|
||||||
|
(a snapshot of every commit) or `@next` (a prerelease) explicitly. Point the
|
||||||
|
**scope** at Gitea rather than setting a bare `registry=`: Gitea serves
|
||||||
|
`@bitsquare` only and does not proxy npmjs, so everything else must keep
|
||||||
|
resolving from there. Reading needs no token while the repository is public.
|
||||||
|
|
||||||
Then name it in `.nopyrc.json`:
|
Then name it in `.nopyrc.json`:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
|
|||||||
@@ -35,7 +35,9 @@ Key benefits:
|
|||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
This cube currently has no configurable parameters.
|
| Variable | Default | What it does |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `DISTRO` | `ubuntu` | which of Docker's package repositories to add — `ubuntu` or `debian`. It selects the download path and nothing else; the release codename comes from the host's own `/etc/os-release`. |
|
||||||
|
|
||||||
## Dependencies
|
## Dependencies
|
||||||
|
|
||||||
|
|||||||
@@ -1,69 +1,79 @@
|
|||||||
# nodevm
|
# nodevm
|
||||||
|
|
||||||
**Install Node.js with essential global packages**
|
**Install Node.js through nvm, for one user, with global packages**
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
This cube installs the latest LTS (Long Term Support) version of Node.js along with essential global npm packages commonly needed for development and deployment.
|
Installs [nvm](https://github.com/nvm-sh/nvm) into a single user's home
|
||||||
|
directory, uses it to install one pinned Node.js version under an alias, and
|
||||||
|
installs a list of global npm packages for that user.
|
||||||
|
|
||||||
## What is Node.js?
|
Per-user, not system-wide. Nothing is placed on the system `PATH`, and another
|
||||||
|
user on the same host is unaffected — which is the point: version pinning belongs
|
||||||
Node.js is a JavaScript runtime built on Chrome's V8 engine that allows you to run JavaScript on the server. It's widely used for:
|
to whoever runs the app.
|
||||||
|
|
||||||
- Building web servers and APIs
|
|
||||||
- Command-line tools
|
|
||||||
- Build tools and task runners
|
|
||||||
- Real-time applications (chat, notifications)
|
|
||||||
- Microservices
|
|
||||||
|
|
||||||
## What This Cube Does
|
## What This Cube Does
|
||||||
|
|
||||||
1. **Installs Node.js LTS**
|
1. **Installs build dependencies** with apt, as root — `build-essential`,
|
||||||
- Downloads and runs the official NodeSource setup script
|
`libssl-dev`, `libtool`, `cmake`, and the cairo/pango/png/jpeg/vips/rsvg/pixman
|
||||||
- Installs the latest LTS version of Node.js
|
headers that native addons need. The package index is refreshed first: a box
|
||||||
- Includes npm (Node Package Manager)
|
nobody has updated lists .deb versions the mirror has already dropped.
|
||||||
|
2. **Installs nvm** for `USER` via the official install script, then
|
||||||
2. **Installs build dependencies**
|
`nvm install <VERSION>` and `nvm alias <ALIAS> <VERSION>`.
|
||||||
- `libssl-dev` - SSL/TLS libraries
|
3. **Installs `GLOBAL_PACKAGES`** with `npm install -g`, as `USER`.
|
||||||
- `libtool` - Library building tools
|
|
||||||
- `cmake` - Cross-platform build system
|
|
||||||
- `libpng-dev`, `libjpeg-dev`, `libvips-dev` - Image processing libraries
|
|
||||||
|
|
||||||
3. **Installs global npm packages**
|
|
||||||
- **npm@11.1.0** - Latest npm version
|
|
||||||
- **pm2** - Production process manager for Node.js apps
|
|
||||||
- **yarn** - Alternative package manager
|
|
||||||
- **local-web-server** - Local development web server
|
|
||||||
- **node-gyp** - Node.js native addon build tool
|
|
||||||
- **inquirer** - Interactive command-line prompts
|
|
||||||
- **execa** - Better child process execution
|
|
||||||
- **@dotenvx/dotenvx** - Environment variable management
|
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
This cube currently has no configurable parameters.
|
| Variable | Default | What it does |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `VERSION` | `v22.20.0` | the Node.js version nvm installs. A pin, not "latest LTS" — nvm's own version strings work, so `--lts` or `22` are accepted too. |
|
||||||
|
| `USER` | `vagrant` | the user nvm is installed **for**. Everything lands in that user's `~/.nvm`. |
|
||||||
|
| `ALIAS` | `nodelts` | the nvm alias pointing at `VERSION`, so later cubes and scripts can say `nvm use nodelts` without knowing the number. |
|
||||||
|
| `GLOBAL_PACKAGES` | `pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx` | space-separated, passed to one `npm install -g`. Setting it **replaces** the list rather than adding to it. |
|
||||||
|
| `SHELL` | `fish` | the login shell to install through — `fish` or `bash`. See below. |
|
||||||
|
|
||||||
|
### `SHELL`
|
||||||
|
|
||||||
|
nvm wires itself into whichever shell installed it, so this is not cosmetic.
|
||||||
|
|
||||||
|
- **`fish`** (default) additionally requires **Oh My Fish**, because loading nvm
|
||||||
|
goes through the `omf install nvm` plugin. `user:add` installs both, which is
|
||||||
|
the usual way a host arrives here. The cube fails with one line, before
|
||||||
|
changing anything, if `SHELL=fish` on a host with no fish.
|
||||||
|
- **`bash`** needs nothing beyond bash. Use it on a host where `user:add` has not
|
||||||
|
run.
|
||||||
|
|
||||||
|
The default stays `fish` so that an existing user — whose login shell `user:add`
|
||||||
|
set to fish — keeps getting a Node that their shell can actually see. Switching
|
||||||
|
would install it invisibly.
|
||||||
|
|
||||||
## Dependencies
|
## Dependencies
|
||||||
|
|
||||||
None - this cube can run standalone.
|
None declared: the cube runs standalone. With `SHELL=fish` it does have a real
|
||||||
|
prerequisite (fish + Oh My Fish, which `user:add` provides), but `user:add` is
|
||||||
|
deliberately not a declared dependency — it would *create* a user who is normally
|
||||||
|
meant to already exist. `SHELL=bash` is the standalone path.
|
||||||
|
|
||||||
## Post-Installation
|
## Post-Installation
|
||||||
|
|
||||||
Verify installation:
|
`node` is on `USER`'s `PATH` in a login shell, not in root's and not in a
|
||||||
|
non-interactive one. To check:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
node --version
|
su - <USER> -c 'node --version && npm --version'
|
||||||
npm --version
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Common commands:
|
From a bash script that is not a login shell, load nvm first:
|
||||||
- Run a Node.js app: `node app.js`
|
|
||||||
- Start with PM2: `pm2 start app.js`
|
|
||||||
- Install packages: `npm install <package>`
|
|
||||||
- Use yarn: `yarn add <package>`
|
|
||||||
|
|
||||||
## PM2 - Process Manager
|
```bash
|
||||||
|
export NVM_DIR="$HOME/.nvm"; . "$NVM_DIR/nvm.sh"
|
||||||
|
nvm use nodelts
|
||||||
|
```
|
||||||
|
|
||||||
PM2 is included for production deployments. Common PM2 commands:
|
## PM2 — process manager
|
||||||
|
|
||||||
|
`pm2` is in the default `GLOBAL_PACKAGES`, so it is installed unless you replaced
|
||||||
|
the list.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pm2 start app.js # Start application
|
pm2 start app.js # Start application
|
||||||
@@ -75,9 +85,11 @@ pm2 startup # Enable PM2 on boot
|
|||||||
pm2 save # Save current process list
|
pm2 save # Save current process list
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`pm2 startup` prints a `sudo` command to run; it does not enable itself.
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
- Node.js is installed system-wide
|
- Node.js is installed **per user**, under `~/.nvm` for `USER`.
|
||||||
- Global packages are accessible to all users
|
- Global packages belong to that user too, not to everyone on the host.
|
||||||
- npm cache is stored in `~/.npm`
|
- Run the cube again with a different `VERSION` and `ALIAS` to have several
|
||||||
- Use `nvm` if you need multiple Node.js versions
|
versions side by side; nvm is built for exactly that.
|
||||||
|
|||||||
@@ -1,13 +1,51 @@
|
|||||||
from pyinfra.operations import server, apt, npm, python
|
from pyinfra.operations import server, apt
|
||||||
from pyinfra import host
|
from pyinfra import host
|
||||||
from pyinfra.facts.files import Directory
|
|
||||||
from pyinfra.facts.server import Which
|
from pyinfra.facts.server import Which
|
||||||
|
from pyinfra.api.exceptions import DeployError
|
||||||
|
|
||||||
hasNode = host.get_fact(Which, 'node')
|
|
||||||
VERSION = host.data.VERSION
|
VERSION = host.data.VERSION
|
||||||
ALIAS = host.data.ALIAS
|
ALIAS = host.data.ALIAS
|
||||||
GLOBAL_PACKAGES = host.data.GLOBAL_PACKAGES
|
GLOBAL_PACKAGES = host.data.GLOBAL_PACKAGES
|
||||||
USER = host.data.USER
|
USER = host.data.USER
|
||||||
|
SHELL = host.data.SHELL
|
||||||
|
|
||||||
|
INSTALL_NVM = "curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash"
|
||||||
|
|
||||||
|
# Under bash, every entry in `commands` is its own shell, so loading nvm and
|
||||||
|
# using it have to be one entry. Loading cannot be skipped either: nvm's
|
||||||
|
# installer appends its hook to ~/.bashrc, and Ubuntu's ~/.bashrc returns at
|
||||||
|
# line 1 for a non-interactive shell, so under `su -c` the hook never runs.
|
||||||
|
LOAD_NVM = 'export NVM_DIR="$HOME/.nvm"; . "$NVM_DIR/nvm.sh"'
|
||||||
|
|
||||||
|
# The binary only. Oh My Fish is a set of fish functions with no binary and no
|
||||||
|
# fixed path, so probing for it would be guesswork — and if fish is there while
|
||||||
|
# omf is not, `omf install nvm` says so itself. Half a guard that is certain
|
||||||
|
# beats a whole one that is not.
|
||||||
|
if SHELL == 'fish' and not host.get_fact(Which, 'fish'):
|
||||||
|
raise DeployError(
|
||||||
|
f'runtime:nodevm: SHELL is "fish" but fish is not installed for {USER}. '
|
||||||
|
'Run user:add first, or set SHELL=bash.'
|
||||||
|
)
|
||||||
|
|
||||||
|
if SHELL == 'fish':
|
||||||
|
shell_executable = '/usr/bin/fish'
|
||||||
|
# The omf plugin defines `nvm` as a fish function that every login shell
|
||||||
|
# loads, and activates the `default` alias on load — so `nvm` and `npm` are
|
||||||
|
# both reachable in any later shell without setup, and NVM_DIR is set for us.
|
||||||
|
nvm_commands = [
|
||||||
|
INSTALL_NVM,
|
||||||
|
'omf install nvm',
|
||||||
|
f'nvm install {VERSION}',
|
||||||
|
f'nvm alias {ALIAS} {VERSION}',
|
||||||
|
]
|
||||||
|
npm_commands = [f'npm install -g {GLOBAL_PACKAGES}']
|
||||||
|
else:
|
||||||
|
shell_executable = '/bin/bash'
|
||||||
|
nvm_commands = [
|
||||||
|
INSTALL_NVM,
|
||||||
|
f'{LOAD_NVM}; nvm install {VERSION}; nvm alias {ALIAS} {VERSION}',
|
||||||
|
]
|
||||||
|
npm_commands = [f'{LOAD_NVM}; nvm use {ALIAS}; npm install -g {GLOBAL_PACKAGES}']
|
||||||
|
|
||||||
apt.packages(
|
apt.packages(
|
||||||
name=f'Install nodejs tools',
|
name=f'Install nodejs tools',
|
||||||
@@ -26,33 +64,30 @@ apt.packages(
|
|||||||
'librsvg2-dev',
|
'librsvg2-dev',
|
||||||
'libpixman-1-dev',
|
'libpixman-1-dev',
|
||||||
],
|
],
|
||||||
|
# Every other cube that installs packages refreshes the index first, and
|
||||||
|
# this one only got away without it while `user:add` ran ahead of it and
|
||||||
|
# dragged in `apt:essentials`. On a box nobody has updated, the index
|
||||||
|
# names .deb versions the mirror has already superseded and the fetch
|
||||||
|
# 404s — the same "assumes a predecessor cube ran" defect as the shell.
|
||||||
|
update=True,
|
||||||
_sudo = True,
|
_sudo = True,
|
||||||
)
|
)
|
||||||
|
|
||||||
server.shell(
|
server.shell(
|
||||||
commands=[
|
name=f'Install nvm and node {VERSION} for {USER}',
|
||||||
"curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash",
|
commands=nvm_commands,
|
||||||
"omf install nvm",
|
|
||||||
f"nvm install {VERSION}",
|
|
||||||
f"nvm alias {ALIAS} {VERSION}",
|
|
||||||
"set -gx NVM_DIR $HOME/.nvm",
|
|
||||||
],
|
|
||||||
_sudo=True,
|
_sudo=True,
|
||||||
_su_user=USER,
|
_su_user=USER,
|
||||||
_use_su_login=True,
|
_use_su_login=True,
|
||||||
_shell_executable='/usr/bin/fish'
|
_shell_executable=shell_executable,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
server.shell(
|
server.shell(
|
||||||
commands=[
|
name=f'Install global packages for {USER}',
|
||||||
"npm install -g pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx"
|
commands=npm_commands,
|
||||||
],
|
|
||||||
_sudo=True,
|
_sudo=True,
|
||||||
_su_user=USER,
|
_su_user=USER,
|
||||||
_use_su_login=True,
|
_use_su_login=True,
|
||||||
_shell_executable='/usr/bin/fish'
|
_shell_executable=shell_executable,
|
||||||
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -7,14 +7,24 @@ export default Manifest({
|
|||||||
dependencies: () => [],
|
dependencies: () => [],
|
||||||
schema: z.object({
|
schema: z.object({
|
||||||
VERSION: z
|
VERSION: z
|
||||||
.nullable(z.string())
|
.string()
|
||||||
.describe('Node.js version to install. It is recommended to use semver notation')
|
.describe('Node.js version to install. It is recommended to use semver notation')
|
||||||
.default('v22.20.0'),
|
.default('v22.20.0'),
|
||||||
USER: z.string().describe('Username for which to install nodejs').default('vagrant'),
|
USER: z.string().describe('Username for which to install nodejs').default('vagrant'),
|
||||||
ALIAS: z.string().describe('The alias for this node version').default('nodelts'),
|
ALIAS: z.string().describe('The alias for this node version').default('nodelts'),
|
||||||
|
// The list the deploy script used to hardcode. It is the default rather than
|
||||||
|
// a constant so that setting the variable adds to nothing and replaces
|
||||||
|
// everything — which is what "space-separated list" reads as.
|
||||||
GLOBAL_PACKAGES: z
|
GLOBAL_PACKAGES: z
|
||||||
.string()
|
.string()
|
||||||
.describe('Space-separated list of global npm packages to install')
|
.describe('Space-separated list of global npm packages to install')
|
||||||
.default('npm-check-updates'),
|
.default('pm2 yarn local-web-server node-gyp inquirer execa @dotenvx/dotenvx'),
|
||||||
|
// fish is the default because nvm wires itself into whichever shell installed
|
||||||
|
// it: switching would leave an existing user — whose login shell `user:add`
|
||||||
|
// set to fish — with node installed and invisible.
|
||||||
|
SHELL: z
|
||||||
|
.enum(['fish', 'bash'])
|
||||||
|
.describe('Login shell to install through. fish needs Oh My Fish; bash needs nothing')
|
||||||
|
.default('fish'),
|
||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,176 +1,61 @@
|
|||||||
# TypeStack Install Cube
|
# autostart
|
||||||
|
|
||||||
Deploys a Node.js/TypeScript application from a Git repository as a systemd service with Docker Compose and PM2 support.
|
**Enable and start an existing systemd service**
|
||||||
|
|
||||||
## Features
|
## Purpose
|
||||||
|
|
||||||
- Clones Git repository
|
Takes a systemd unit that is already installed on the host and decides whether
|
||||||
- Installs dependencies with Yarn
|
it runs: `systemctl enable` plus `systemctl start`, or neither.
|
||||||
- Builds the application
|
|
||||||
- Starts Docker Compose services
|
|
||||||
- Creates a systemd service for automatic startup
|
|
||||||
- Configures PM2 for process management
|
|
||||||
- Automatic restart on failure
|
|
||||||
|
|
||||||
## Requirements
|
It does **not** create the unit. Something else — a package, another cube, a
|
||||||
|
`files.template` — has to have put `<APP>.service` on the host first. This cube
|
||||||
- Git (for cloning repository)
|
is the switch, not the wiring.
|
||||||
- Yarn (for dependency management)
|
|
||||||
- Docker and Docker Compose
|
|
||||||
- PM2 (for process management)
|
|
||||||
- Node.js/NVM installed
|
|
||||||
- SSH key access to the repository (if using private repos)
|
|
||||||
|
|
||||||
## Configuration Parameters
|
|
||||||
|
|
||||||
### Required
|
|
||||||
|
|
||||||
- **USER**: System user to run the application (default: `teclabmin`)
|
|
||||||
- **REPO**: Git repository URL (default: `git@github.com:bennidi/teclab-flintstone.git`)
|
|
||||||
- **APP**: Application name/directory name (default: `flintstone`)
|
|
||||||
|
|
||||||
### Optional
|
|
||||||
|
|
||||||
- **ENV**: Application environment (default: `production`)
|
|
||||||
- **AUTOSTART**: Enable and start service immediately (default: `True`)
|
|
||||||
- **NODE_PATH**: Path to Node.js binaries (default: `/home/teclabmin/.nvm/versions/node/v21.7.3/bin`)
|
|
||||||
|
|
||||||
## Example Usage
|
|
||||||
|
|
||||||
### Basic Configuration
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"USER": "myuser",
|
|
||||||
"REPO": "git@github.com:myorg/myapp.git",
|
|
||||||
"APP": "myapp"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Advanced Configuration
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"USER": "appuser",
|
|
||||||
"REPO": "git@github.com:myorg/myapp.git",
|
|
||||||
"APP": "myapp",
|
|
||||||
"ENV": "staging",
|
|
||||||
"AUTOSTART": false,
|
|
||||||
"NODE_PATH": "/home/appuser/.nvm/versions/node/v20.0.0/bin"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## What This Cube Does
|
## What This Cube Does
|
||||||
|
|
||||||
1. **Clone Repository**: Clones the specified Git repository to `/home/<USER>/<APP>`
|
With `AUTOSTART=True` (the default), two `systemd.service` operations against
|
||||||
2. **Install Dependencies**: Runs `yarn install` to install all dependencies
|
`<APP>`: one setting `enabled=True` so the unit comes up on boot, one setting
|
||||||
3. **Build Application**: Runs `yarn build` to compile the application
|
`running=True` so it comes up now. Both are idempotent — a unit already enabled
|
||||||
4. **Start Docker Services**: Runs `docker compose up -d` to start containerized services
|
and running is left alone.
|
||||||
5. **Create Startup Script**: Creates `/home/<USER>/<APP>.service.sh` that:
|
|
||||||
- Starts Docker Compose services
|
|
||||||
- Starts PM2 with ecosystem.config.js
|
|
||||||
6. **Create Systemd Service**: Creates `/etc/systemd/system/<APP>.service` that:
|
|
||||||
- Runs after Docker service
|
|
||||||
- Uses the specified user
|
|
||||||
- Configures proper environment (HOME, PATH)
|
|
||||||
- Auto-restarts on failure
|
|
||||||
7. **Enable & Start Service**: Enables and starts the service (if AUTOSTART=True)
|
|
||||||
|
|
||||||
## Service Management
|
With `AUTOSTART=False`, nothing is changed. The cube prints the two commands you
|
||||||
|
would run by hand and exits, which is the point of the flag: install now, decide
|
||||||
|
later.
|
||||||
|
|
||||||
### Check service status
|
## Configuration
|
||||||
|
|
||||||
|
| Variable | Default | What it does |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `APP` | *(required)* | the systemd unit name, without the `.service` suffix — `flintstone` for `/etc/systemd/system/flintstone.service`. This is what `systemctl` is actually pointed at. |
|
||||||
|
| `SERVICE_NAME` | `Application` | a display name, used only in the operation labels pyinfra prints and in the `AUTOSTART=False` message. Changing it changes what you read, not what happens. |
|
||||||
|
| `AUTOSTART` | `true` | whether to enable and start the unit at all. |
|
||||||
|
|
||||||
|
`APP` has no default, so `nopy -D` (`--use-defaults`) fails by name rather than
|
||||||
|
guessing. Supply it under `env` in `.nopyrc.json`, from a dependency, or at the
|
||||||
|
prompt.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
None declared, and none implied beyond the unit file itself. `systemd.service`
|
||||||
|
is a pyinfra built-in; there is nothing to install.
|
||||||
|
|
||||||
|
## Post-Installation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo systemctl status <APP>
|
systemctl status <APP> # is it running?
|
||||||
|
systemctl is-enabled <APP> # will it come back after a reboot?
|
||||||
|
journalctl -u <APP> -f # follow its log
|
||||||
```
|
```
|
||||||
|
|
||||||
### Start the service
|
If the run fails with *Unit `<APP>.service` could not be found*, the unit was
|
||||||
|
never installed — see Purpose. `systemctl daemon-reload` is worth trying if the
|
||||||
```bash
|
file was written after systemd last read the directory.
|
||||||
sudo systemctl start <APP>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Stop the service
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl stop <APP>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Restart the service
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl restart <APP>
|
|
||||||
```
|
|
||||||
|
|
||||||
### View service logs
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo journalctl -u <APP> -f
|
|
||||||
```
|
|
||||||
|
|
||||||
### Disable autostart
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl disable <APP>
|
|
||||||
```
|
|
||||||
|
|
||||||
## File Structure
|
|
||||||
|
|
||||||
After deployment:
|
|
||||||
|
|
||||||
```
|
|
||||||
/home/<USER>/
|
|
||||||
├── <APP>/ # Application directory
|
|
||||||
│ ├── ecosystem.config.js # PM2 configuration
|
|
||||||
│ ├── docker-compose.yml # Docker services
|
|
||||||
│ └── ... # Application files
|
|
||||||
├── <APP>.service.sh # Startup script
|
|
||||||
/etc/systemd/system/
|
|
||||||
└── <APP>.service # Systemd service file
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Service fails to start
|
|
||||||
|
|
||||||
1. Check service logs:
|
|
||||||
```bash
|
|
||||||
sudo journalctl -u <APP> -n 50
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Verify Docker is running:
|
|
||||||
```bash
|
|
||||||
sudo systemctl status docker
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Check if Node.js path is correct:
|
|
||||||
```bash
|
|
||||||
which node
|
|
||||||
which pm2
|
|
||||||
```
|
|
||||||
|
|
||||||
### Repository clone fails
|
|
||||||
|
|
||||||
- Ensure SSH keys are properly configured for the user
|
|
||||||
- Test SSH access: `ssh -T git@github.com`
|
|
||||||
- Check repository URL is correct
|
|
||||||
|
|
||||||
### Docker Compose fails
|
|
||||||
|
|
||||||
- Verify Docker is installed and running
|
|
||||||
- Check docker-compose.yml exists in the application directory
|
|
||||||
- Ensure user has Docker permissions: `sudo usermod -aG docker <USER>`
|
|
||||||
|
|
||||||
### PM2 not starting
|
|
||||||
|
|
||||||
- Verify PM2 is installed: `pm2 --version`
|
|
||||||
- Check ecosystem.config.js exists
|
|
||||||
- Verify NODE_PATH includes PM2 binary location
|
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
- The service type is set to `forking` to support PM2's daemon mode
|
- Enabling and starting are separate systemd concepts and this cube always does
|
||||||
- Service will auto-restart on failure with a 5-second delay
|
both or neither. If you need one without the other, call `systemd.service`
|
||||||
- Maximum 5 restart attempts in the burst period
|
from your own deploy script.
|
||||||
- The service waits for Docker to be ready before starting
|
- `SERVICE_NAME` is deliberately not passed to systemd. The unit is identified
|
||||||
- Environment variables can be configured in the ecosystem.config.js file
|
by `APP` alone, so a wrong `SERVICE_NAME` is a cosmetic mistake rather than a
|
||||||
|
cube that manages the wrong service.
|
||||||
|
|||||||
@@ -1,8 +1,10 @@
|
|||||||
from pyinfra.operations import systemd
|
from pyinfra.operations import server, systemd
|
||||||
from pyinfra import host
|
from pyinfra import host
|
||||||
|
|
||||||
|
|
||||||
APP = host.data.APP
|
APP = host.data.APP
|
||||||
|
SERVICE_NAME = host.data.SERVICE_NAME
|
||||||
|
AUTOSTART = host.data.AUTOSTART
|
||||||
|
|
||||||
# Enable and start the service based on AUTOSTART flag
|
# Enable and start the service based on AUTOSTART flag
|
||||||
if AUTOSTART:
|
if AUTOSTART:
|
||||||
|
|||||||
@@ -92,52 +92,7 @@ After deployment:
|
|||||||
- Oh My Fish provides package management: `omf install <package>`
|
- Oh My Fish provides package management: `omf install <package>`
|
||||||
- To switch shells: `chsh -s /bin/bash` (or back to fish: `chsh -s /usr/bin/fish`)
|
- To switch shells: `chsh -s /bin/bash` (or back to fish: `chsh -s /usr/bin/fish`)
|
||||||
|
|
||||||
---
|
Fish's own key bindings and the plugins this cube installs are documented
|
||||||
|
upstream — `fish_key_reader` lists what is bound, and `omf help` what is
|
||||||
# 📌 Most Useful Fish Key Bindings (with Fisher Extensions)
|
installed. They used to be reproduced here at length, which is not something
|
||||||
|
this cube knows anything about.
|
||||||
## 🐟 Default Fish Key Bindings
|
|
||||||
|
|
||||||
- `Ctrl + C` → Cancel the current command
|
|
||||||
- `Ctrl + D` → Exit the shell (or logout if in SSH)
|
|
||||||
- `Ctrl + L` → Clear the terminal
|
|
||||||
- `Ctrl + R` → Search command history (enhanced by `fzf.fish`)
|
|
||||||
- `Ctrl + U` → Delete the entire command line
|
|
||||||
- `Ctrl + W` → Delete the last word
|
|
||||||
- `Alt + ← / →` → Move backward/forward by a word
|
|
||||||
|
|
||||||
## 🔍 Enhanced with `fzf.fish`
|
|
||||||
|
|
||||||
- `Ctrl + R` → **Fuzzy search command history**
|
|
||||||
- `Ctrl + T` → **Fuzzy search and insert file path**
|
|
||||||
- `Alt + C` → **Fuzzy search directories (`cd` with `z`)**
|
|
||||||
|
|
||||||
## 📂 Directory Navigation (with `z`)
|
|
||||||
|
|
||||||
- `z <dir>` → Jump to a frequently used directory
|
|
||||||
- `z -l` → List most-used directories
|
|
||||||
- `z -c` → Remove a directory from `z`'s database
|
|
||||||
|
|
||||||
## 🔄 Process & Job Management
|
|
||||||
|
|
||||||
- `Ctrl + Z` → Suspend the current process
|
|
||||||
- `fg` → Bring a suspended process back to foreground
|
|
||||||
- `jobs` → List background jobs
|
|
||||||
|
|
||||||
## 🎨 Other Handy Shortcuts
|
|
||||||
|
|
||||||
- `fish_vi_key_bindings` → Enable Vi mode (press `Esc` for normal mode)
|
|
||||||
- `Ctrl + G` → Show Git status (if using `fzf.fish`)
|
|
||||||
- `Ctrl + E` → Edit command line in `$EDITOR`
|
|
||||||
|
|
||||||
## ⚙️ Useful Commands for Key Binding
|
|
||||||
|
|
||||||
```fish
|
|
||||||
# Set Fish default key bindings
|
|
||||||
fish_default_key_bindings
|
|
||||||
|
|
||||||
# Enable Vi mode
|
|
||||||
fish_vi_key_bindings
|
|
||||||
|
|
||||||
# Rebind a custom key (Example: Ctrl + G for git status)
|
|
||||||
bind \cg 'git status'
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@bitsquare/nopy-cubes-core",
|
"name": "@bitsquare/nopy-cubes-core",
|
||||||
"version": "0.5.0",
|
"version": "1.0.1",
|
||||||
"description": "The core nopy cube bundle: apt, users, ssh, networking, services and runtimes.",
|
"description": "The core nopy cube bundle: apt, users, ssh, networking, services and runtimes.",
|
||||||
"keywords": [
|
"keywords": [
|
||||||
"nopy",
|
"nopy",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@bitsquare/nopy-cubes",
|
"name": "@bitsquare/nopy-cubes",
|
||||||
"version": "0.5.0",
|
"version": "1.0.1",
|
||||||
"description": "Authoring types for nopy cubes: the Manifest factory and the Cube contract.",
|
"description": "Authoring types for nopy cubes: the Manifest factory and the Cube contract.",
|
||||||
"keywords": [
|
"keywords": [
|
||||||
"nopy",
|
"nopy",
|
||||||
|
|||||||
@@ -192,6 +192,18 @@ export class Cube<Schema extends AnyObjectSchema = AnyObjectSchema> {
|
|||||||
return defaults as z.infer<Schema>;
|
return defaults as z.infer<Schema>;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Every key the schema declares, required or not.
|
||||||
|
*
|
||||||
|
* The question this answers is "does this cube claim to know about KEY", which
|
||||||
|
* is not the same as "does it have a value for it" — a cube can read a key off
|
||||||
|
* `host.data` that only the config `env` supplies. That distinction is what
|
||||||
|
* decides whether a secret is allowed to travel to it.
|
||||||
|
*/
|
||||||
|
schemaKeys(): string[] {
|
||||||
|
return Object.keys(this.manifest.schema.shape);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Schema keys that have to be supplied from somewhere: no `.default()`, and
|
* Schema keys that have to be supplied from somewhere: no `.default()`, and
|
||||||
* not optional. Nothing else can fill them in, so a run that cannot prompt
|
* not optional. Nothing else can fill them in, so a run that cannot prompt
|
||||||
|
|||||||
+119
-30
@@ -1,28 +1,30 @@
|
|||||||
# Nopy
|
# Nopy
|
||||||
|
|
||||||
A CLI tool that simplifies **pyinfra** script management and execution, providing an interactive workflow for deploying infrastructure configurations ("cubes") to remote hosts.
|
A CLI tool that simplifies **pyinfra** script management and execution, providing an interactive workflow for deploying infrastructure configurations `cubes` to remote hosts.
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Nopy wraps pyinfra with structure, validation, and an interactive experience for managing complex infrastructure deployments. It organizes deployments into self-contained "cubes" with dependency management, schema validation, and lifecycle hooks.
|
Nopy wraps [pyinfra](https://pyinfra.com/) in the javascript ecosystem to provide an interactive experience for managing repeatable infrastructure deployments. It organizes deployments into self-contained units - called `cubes` - adding support for transitive dependency management, user input validation, and different lifecycle hooks.
|
||||||
|
|
||||||
## Features
|
## Features in a Nutshell
|
||||||
|
|
||||||
- **Dependency resolution** with topological sorting
|
- **Manifest files** to support declarative description of user inputs and orchestration semantics per cube
|
||||||
- **Before/after hooks** for multi-cube orchestration
|
- **Dependency resolution** in dependency order, with cycle detection
|
||||||
|
- **Before/after hooks** for programmable, multi-cube orchestration
|
||||||
- **SSH key or password authentication**
|
- **SSH key or password authentication**
|
||||||
- **Default values** with optional customization via manifest `env`
|
- **Default values** with optional customization via manifest `env`
|
||||||
- **Schema validation** using Zod
|
- **Schema validation** and **type coercion** using Zod
|
||||||
- **Recursive cube directory discovery**
|
- **Recursive cube directory discovery**
|
||||||
- **Dry-run mode** for previewing deployments
|
- **Dry-run mode** for previewing deployment scenarios
|
||||||
- **JSON output** for CI/CD integration
|
- **Pipeable output** for CI/CD integration — the plan on stdout, everything else on stderr
|
||||||
- **Session history** with replay capability
|
- **Session history** for fast replay during development
|
||||||
|
- **Multi-layered** config files with natural discovery and deterministic parameter resolution
|
||||||
|
|
||||||
## Workflow
|
## Workflow
|
||||||
|
|
||||||
1. **Load cubes** - Discovers and validates cubes from configured directories
|
1. **Load cubes** - Discovers and validates cubes from configured directories
|
||||||
2. **Interactive prompts** - Select cubes, target host, and authentication method
|
2. **Interactive prompts** - Select cubes, target host, and authentication method
|
||||||
3. **Dependency resolution** - Topologically sorts cubes based on dependencies
|
3. **Dependency resolution** - Resolves each cube's dependencies before the cube itself, so the deploy order is a topological order of the graph; a cycle is reported by name rather than recursed into
|
||||||
4. **Variable assignment** - Validates and collects configuration with schema validation
|
4. **Variable assignment** - Validates and collects configuration with schema validation
|
||||||
5. **Execute hooks** - Runs before/after hooks for orchestration
|
5. **Execute hooks** - Runs before/after hooks for orchestration
|
||||||
6. **Deploy** - Sequentially executes pyinfra commands
|
6. **Deploy** - Sequentially executes pyinfra commands
|
||||||
@@ -33,23 +35,20 @@ Nopy wraps pyinfra with structure, validation, and an interactive experience for
|
|||||||
|
|
||||||
A cube is a **directory** containing two files:
|
A cube is a **directory** containing two files:
|
||||||
|
|
||||||
- **JavaScript manifest**: `manifest.mjs` defining schema, dependencies, defaults, secrets, and hooks
|
- **JavaScript manifest**: `manifest.mjs` defining schema, dependencies, defaults, secrets (encrypted only), and hooks
|
||||||
- **Python deployment script**: `deploy.py`, a plain pyinfra script
|
- **Python deployment script**: `deploy.py`, a plain pyinfra script
|
||||||
|
|
||||||
Configuration variables are declared in the manifest and validated with Zod schemas before the deployment script runs.
|
Configuration variables are declared in the manifest and validated with Zod schemas before the deployment script runs.
|
||||||
|
|
||||||
```
|
```
|
||||||
cubes/
|
cubes/
|
||||||
├── .npcubes
|
|
||||||
└── apt/
|
└── apt/
|
||||||
└── install/
|
└── install/
|
||||||
├── manifest.mjs
|
├── manifest.mjs
|
||||||
└── deploy.py
|
└── deploy.py
|
||||||
```
|
```
|
||||||
|
|
||||||
Any directory holding both files is treated as a cube, so cubes can be nested as deeply as you like to group them by topic. Discovery is recursive; directories starting with `.` and `node_modules` are skipped. Additional files in the cube directory (a `README.md`, config templates, and so on) are ignored by the loader and can be referenced from the deploy script — the script runs with its cube directory as the working directory.
|
Any directory holding both files is treated as a cube, so cubes can be nested and grouped by topic. Discovery is recursive; hidden directories starting with `.` and `node_modules` are skipped. Additional files in the cube directory (a `README.md`, config templates, and so on) are ignored by the loader but can be referenced from the deploy script — ** the pyinfra script runs with its cube directory as the working directory**.
|
||||||
|
|
||||||
The prefixed forms `<cube-name>.manifest.mjs` and `<cube-name>.deploy.py` are also still recognized, but plain `manifest.mjs` / `deploy.py` is the current convention.
|
|
||||||
|
|
||||||
A cube's identity comes from the manifest's `id` field (see below). If `id` is omitted, nopy falls back to an `[id]` prefix in the manifest `name`, and finally to the directory's own name. Note that the id does not have to mirror the folder path — `cubes/network/tailscale` declares `id: 'net:tailscale'`.
|
A cube's identity comes from the manifest's `id` field (see below). If `id` is omitted, nopy falls back to an `[id]` prefix in the manifest `name`, and finally to the directory's own name. Note that the id does not have to mirror the folder path — `cubes/network/tailscale` declares `id: 'net:tailscale'`.
|
||||||
|
|
||||||
@@ -65,7 +64,7 @@ export default cubes.Manifest({
|
|||||||
dependencies: () => [],
|
dependencies: () => [],
|
||||||
schema: z.object({
|
schema: z.object({
|
||||||
UPDATE: z.boolean().describe('Update package cache').default(false),
|
UPDATE: z.boolean().describe('Update package cache').default(false),
|
||||||
PACKAGES: z.string().describe('Space-separated list of packages').default('vim htop'),
|
PACKAGES: z.string().describe('Space-separated list of packages').default('curl htop'),
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
```
|
```
|
||||||
@@ -118,7 +117,7 @@ A variable can be set from several places in one run. Every assignment is kept,
|
|||||||
|
|
||||||
This allows cubes to ship with reasonable defaults while still allowing users to override them globally via `.nopyrc.json` or interactively during deployment. Because `env` outranks the schema, `.nopyrc.json` is also what steers a run started with `--use-defaults`, which never prompts.
|
This allows cubes to ship with reasonable defaults while still allowing users to override them globally via `.nopyrc.json` or interactively during deployment. Because `env` outranks the schema, `.nopyrc.json` is also what steers a run started with `--use-defaults`, which never prompts.
|
||||||
|
|
||||||
`prompt` and `param` rarely compete: a key a dependency supplies is left out of the prompt entirely, so the user is only ever asked about the keys nothing else has set.
|
`prompt` and `param` rarely compete: a key supplied by a dependency is left out of the user input prompt entirely.
|
||||||
|
|
||||||
Ranking by origin rather than by arrival order is what makes replay work: a recorded value is applied *before* the cube would be prompted for, and prompting can still override it, but a `--data` value pushed in by a dependency is never clobbered by a stale recording.
|
Ranking by origin rather than by arrival order is what makes replay work: a recorded value is applied *before* the cube would be prompted for, and prompting can still override it, but a `--data` value pushed in by a dependency is never clobbered by a stale recording.
|
||||||
|
|
||||||
@@ -142,13 +141,25 @@ export default cubes.Manifest({
|
|||||||
|
|
||||||
Every entry must be a key of `schema`; naming anything else is a manifest error and aborts the run, so a typo fails loudly instead of silently leaving a value unprotected.
|
Every entry must be a key of `schema`; naming anything else is a manifest error and aborts the run, so a typo fails loudly instead of silently leaving a value unprotected.
|
||||||
|
|
||||||
Declaring a key a secret changes three things:
|
Declaring a key a secret changes four things:
|
||||||
|
|
||||||
- **It is never written to a session file or to the history.** Everything else the run settled on is recorded — including values that came from a `.default()` — but declared secrets are left out.
|
- **It is never written to a session file or to the history.** Everything else the run settled on is recorded — including values that came from a `.default()` — but declared secrets are left out.
|
||||||
- **It is masked wherever a command or a plan is printed** — `--dry-run`, `--print-only`, and the debug log all show `********` in place of the value, in the variable list *and* in the `pyinfra` command line above it. The SSH password passed via `--password` is masked the same way, whether or not any cube declares secrets.
|
- **It is masked wherever a command or a plan is printed** — `--dry-run`, `--print-only`, and the debug log all show `********` in place of the value, in the variable list *and* in the `pyinfra` command line above it. The SSH password passed via `--password` is masked the same way, whether or not any cube declares secrets.
|
||||||
- **It is re-prompted on replay**, since there is nothing recorded to replay from (see [Session Recording and Replay](#session-recording-and-replay)).
|
- **It is re-prompted on replay**, since there is nothing recorded to replay from (see [Session Recording and Replay](#session-recording-and-replay)).
|
||||||
|
- **It stops travelling.** Ordinary `env` values are seeded onto every cube in the run, because a cube may read a key off `host.data` that its own schema never declared. A secret is the exception: it reaches only the cubes whose `schema` names it. Otherwise putting a password under `env` — which is what unattended replay asks you to do — would put it on the command line of every unrelated cube, where nothing masks it because that cube never called it a secret.
|
||||||
|
|
||||||
Nopy does not guess. A key called `PASSWORD` in a manifest that declares no `secrets` is treated as an ordinary variable — recorded, and printed in the clear.
|
Declaring is global, masking is global. A key any manifest calls a secret is masked and kept out of sessions on every cube it lands on, even one whose own manifest forgot to list it. What is *not* global is the guess: a key called `PASSWORD` that no manifest declares anywhere is an ordinary variable — broadcast, recorded, and printed in the clear.
|
||||||
|
|
||||||
|
For a sensitive `env` value that no cube declares at all — a token only a hook reads, say — name it in the config instead:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"secrets": ["DEPLOY_TOKEN"],
|
||||||
|
"env": { "DEPLOY_TOKEN": "..." }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Entries here behave exactly like a manifest's: masked, never recorded, and delivered only to cubes that declare them.
|
||||||
|
|
||||||
Three limits are worth knowing, because `secrets` keeps a value out of the files nopy writes and nothing more:
|
Three limits are worth knowing, because `secrets` keeps a value out of the files nopy writes and nothing more:
|
||||||
|
|
||||||
@@ -168,6 +179,7 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
|
|||||||
"env": {
|
"env": {
|
||||||
"SHARED_VAR": "value"
|
"SHARED_VAR": "value"
|
||||||
},
|
},
|
||||||
|
"secrets": ["DEPLOY_TOKEN"],
|
||||||
"log": {
|
"log": {
|
||||||
"verbosity": "info",
|
"verbosity": "info",
|
||||||
"debug": false
|
"debug": false
|
||||||
@@ -184,8 +196,36 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
|
|||||||
|
|
||||||
`history` controls automatic session recording (see [Deployment History](#deployment-history)), and `execution.continueOnError` sets the default for `--continue-on-error`.
|
`history` controls automatic session recording (see [Deployment History](#deployment-history)), and `execution.continueOnError` sets the default for `--continue-on-error`.
|
||||||
|
|
||||||
|
`secrets` names `env` keys to treat as sensitive that no manifest declares — it is the config-side half of a manifest's `secrets`, and behaves identically. See [Secrets](#secrets).
|
||||||
|
|
||||||
|
`hosts` seeds the target picker; see [Target hosts](#target-hosts) for what else that picker offers.
|
||||||
|
|
||||||
`cubeDirs` holds paths, `cubePackages` holds installed npm packages that ship cubes — see [Cube Discovery](#cube-discovery) below and [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md) for publishing your own. Both are additive, and both resolve relative to the config file that named them, not to the working directory: a `.nopyrc.json` two levels up may name a package that only exists in *its* `node_modules`.
|
`cubeDirs` holds paths, `cubePackages` holds installed npm packages that ship cubes — see [Cube Discovery](#cube-discovery) below and [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md) for publishing your own. Both are additive, and both resolve relative to the config file that named them, not to the working directory: a `.nopyrc.json` two levels up may name a package that only exists in *its* `node_modules`.
|
||||||
|
|
||||||
|
#### Target hosts
|
||||||
|
|
||||||
|
The host prompt offers more than the `hosts` array. Two entries at the top are
|
||||||
|
shortcuts for pyinfra's local connectors, each asking one follow-up question and
|
||||||
|
assembling the host string from the answer:
|
||||||
|
|
||||||
|
| Picked | Asks for | Becomes |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `docker` | a container name/id, **or** an image reference | `@docker/<answer>` |
|
||||||
|
| `vagrant` | the machine name (default `default`) | `@vagrant/<answer>` |
|
||||||
|
| *(a configured host)* | — | itself |
|
||||||
|
| `custom` | any address | itself |
|
||||||
|
|
||||||
|
The two connector forms can equally be written into `hosts` directly — a session
|
||||||
|
records whatever string the run used, so `"hosts": ["@vagrant/nopytestvm"]` and
|
||||||
|
picking `vagrant` are the same thing to everything downstream.
|
||||||
|
|
||||||
|
The docker answer is deliberately not validated as one kind or the other, because
|
||||||
|
the two mean very different things and only the connector can tell them apart (it
|
||||||
|
looks for a matching container first). A **container** is mutated in place and
|
||||||
|
left running; an **image** makes pyinfra start a throwaway container, apply the
|
||||||
|
deploy, commit the result as a new image and print its id. See
|
||||||
|
[DOCKER.md](docs/DOCKER.md) and [VAGRANT.md](docs/VAGRANT.md).
|
||||||
|
|
||||||
#### Logging Configuration
|
#### Logging Configuration
|
||||||
|
|
||||||
Control pyinfra output verbosity and debug information using the `log` configuration object:
|
Control pyinfra output verbosity and debug information using the `log` configuration object:
|
||||||
@@ -259,12 +299,17 @@ Sessions are stored in `.nopysession.json` files with the following structure:
|
|||||||
- **`env`**: The `env` block of `.nopyrc.json` as it stood at record time, kept for reference
|
- **`env`**: The `env` block of `.nopyrc.json` as it stood at record time, kept for reference
|
||||||
- **`hosts`**: Array of target hosts
|
- **`hosts`**: Array of target hosts
|
||||||
- **`auth`**: Authentication configuration (passwords are never stored)
|
- **`auth`**: Authentication configuration (passwords are never stored)
|
||||||
|
- **`version`**, **`timestamp`**, **`name`**: stamped on every session nopy writes — the format version, the ISO 8601 record time, and a one-line description in the same `date - cubes → hosts` form the history list uses
|
||||||
|
|
||||||
|
Only `cubes` and `auth` are required. A hand-written session may omit the rest, and one that predates the stamp still loads; a `version` this build does not recognise is a warning on stderr, never a refusal.
|
||||||
|
|
||||||
|
`auth.method` has a third value the picker never offers: **`ssh`**, meaning the connector owns authentication and nopy supplies none. It is what an `@vagrant/` or `@docker/` host gets, which is why replaying one asks for nothing.
|
||||||
|
|
||||||
**What is recorded:** every value each cube settled on, regardless of where it came from — a value the user typed, one inherited from `.nopyrc.json` `env`, one a dependency supplied, and one that fell through to the schema's `.default()` are all written out the same way. A session is therefore a full snapshot rather than a diff, and a `--use-defaults` run produces a session with real values in it instead of an empty one.
|
**What is recorded:** every value each cube settled on, regardless of where it came from — a value the user typed, one inherited from `.nopyrc.json` `env`, one a dependency supplied, and one that fell through to the schema's `.default()` are all written out the same way. A session is therefore a full snapshot rather than a diff, and a `--use-defaults` run produces a session with real values in it instead of an empty one.
|
||||||
|
|
||||||
The consequence is that replay is faithful rather than re-derived: the recorded value outranks the current `.nopyrc.json` `env` and the current schema default, so editing either one does not silently change what a replay does. To pick up a new default, record a fresh session.
|
The consequence is that replay is faithful rather than re-derived: the recorded value outranks the current `.nopyrc.json` `env` and the current schema default, so editing either one does not silently change what a replay does. To pick up a new default, record a fresh session.
|
||||||
|
|
||||||
**Security Note**: Passwords are never stored in session files. This covers both the SSH password — a session records the auth *method* and username, never the credential — and any schema key a cube's manifest lists under [`secrets`](#secrets). Both are re-prompted on replay.
|
**Security Note**: Passwords are never stored in session files. This covers both the SSH password — a session records the auth *method* and username, never the credential — and any schema key a cube's manifest lists under [`secrets`](#secrets). Both are re-prompted on replay. The rule applies to the session's `env` block as well as to each cube's `variables`, so a declared secret set in `.nopyrc.json` is left out of the recorded copy rather than written back out in plaintext.
|
||||||
|
|
||||||
#### Recording a Session
|
#### Recording a Session
|
||||||
|
|
||||||
@@ -274,6 +319,9 @@ nopy install --save-session my-deployment.nopysession.json
|
|||||||
|
|
||||||
# With defaults (no prompts for variables)
|
# With defaults (no prompts for variables)
|
||||||
nopy install -D --save-session automated-deployment.nopysession.json
|
nopy install -D --save-session automated-deployment.nopysession.json
|
||||||
|
|
||||||
|
# Also works on a replay — the resolved cube set is what you asked to capture
|
||||||
|
nopy install -R --save-session repeat-of-the-last-run.nopysession.json
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Replaying a Session
|
#### Replaying a Session
|
||||||
@@ -288,7 +336,9 @@ nopy install --load-session my-deployment.nopysession.json
|
|||||||
|
|
||||||
A replay runs straight through without asking anything, with three exceptions. Password authentication always re-prompts. A session with no recorded host falls back to the host picker. And a cube is re-prompted for its declared secrets, plus for any required variable the session has no value for — which happens when the cube's schema has gained a field since the session was written.
|
A replay runs straight through without asking anything, with three exceptions. Password authentication always re-prompts. A session with no recorded host falls back to the host picker. And a cube is re-prompted for its declared secrets, plus for any required variable the session has no value for — which happens when the cube's schema has gained a field since the session was written.
|
||||||
|
|
||||||
Those re-prompts are what a session cannot supply, so `--use-defaults` cannot paper over them: combining `-D` with a replay that needs either fails with a message naming the keys rather than deploying with a placeholder. Put the values under `env` in `.nopyrc.json` to make such a replay unattended.
|
Those re-prompts are what a session cannot supply, so `--use-defaults` cannot paper over them: combining `-D` with a replay that needs either fails with a message naming the keys rather than deploying with a placeholder. Put the values under `env` in `.nopyrc.json` — or pass them from a dependency — to make such a replay unattended. A secret supplied that way reaches only the cubes that declare it, so this does not broadcast it across the run; see [Secrets](#secrets).
|
||||||
|
|
||||||
|
A schema `.default()` is deliberately *not* accepted in its place. The recorded answer is gone on purpose, so falling back to the manifest would deploy a different credential than the run being replayed, and say nothing about it.
|
||||||
|
|
||||||
### Cube Discovery
|
### Cube Discovery
|
||||||
|
|
||||||
@@ -314,13 +364,17 @@ A cube package is an ordinary npm package that ships its cubes in a `cubes/` dir
|
|||||||
Install it and name it — nothing needs linking or copying:
|
Install it and name it — nothing needs linking or copying:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
pnpm add -D @bitsquare/nopy-cubes-core
|
pnpm add -D @bitsquare/nopy-cubes-core@main \
|
||||||
|
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
|
||||||
```
|
```
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "cubePackages": ["@bitsquare/nopy-cubes-core"] }
|
{ "cubePackages": ["@bitsquare/nopy-cubes-core"] }
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The tag and the registry flag are both required for this bundle today — see
|
||||||
|
[Installation](#installation).
|
||||||
|
|
||||||
Naming a package is a statement that cubes are expected from it, so anything wrong is an error that aborts the run rather than a silent skip: the package is not installed, it has neither a `cubes/` directory nor a `nopy.cubes` override, its `nopy.cubes` is malformed, or an entry points at a directory that does not exist or lies outside the package.
|
Naming a package is a statement that cubes are expected from it, so anything wrong is an error that aborts the run rather than a silent skip: the package is not installed, it has neither a `cubes/` directory nor a `nopy.cubes` override, its `nopy.cubes` is malformed, or an entry points at a directory that does not exist or lies outside the package.
|
||||||
|
|
||||||
#### Ids are claimed globally
|
#### Ids are claimed globally
|
||||||
@@ -331,6 +385,18 @@ Writing cubes to publish is covered in [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md).
|
|||||||
|
|
||||||
## Command Line Usage
|
## Command Line Usage
|
||||||
|
|
||||||
|
### Requirements
|
||||||
|
|
||||||
|
| | |
|
||||||
|
| --- | --- |
|
||||||
|
| **Node** | ≥ 22 |
|
||||||
|
| **pyinfra** | on `PATH` — `pipx install pyinfra` |
|
||||||
|
| **the connector** | `vagrant` or `docker` on `PATH`, if you deploy to one |
|
||||||
|
|
||||||
|
nopy builds pyinfra command lines and spawns them; it does not vendor pyinfra and
|
||||||
|
will not install it for you. A missing `pyinfra` surfaces as a spawn failure on
|
||||||
|
the first deploy, after every prompt has been answered.
|
||||||
|
|
||||||
### Installation
|
### Installation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -341,13 +407,21 @@ The cubes live in a separate bundle, installed into whichever project describes
|
|||||||
your infrastructure and named in its `.nopyrc.json`:
|
your infrastructure and named in its `.nopyrc.json`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pnpm add -D @bitsquare/nopy-cubes-core
|
pnpm add -D @bitsquare/nopy-cubes-core@main \
|
||||||
|
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
|
||||||
```
|
```
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "hosts": ["your-host"], "cubePackages": ["@bitsquare/nopy-cubes-core"] }
|
{ "hosts": ["your-host"], "cubePackages": ["@bitsquare/nopy-cubes-core"] }
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**The tag and the registry flag are both required for the bundle today.** It has
|
||||||
|
not been published to npmjs yet, and the Gitea registry publishes no `latest`
|
||||||
|
tag, so a plain `pnpm add -D @bitsquare/nopy-cubes-core` fails with a 404 against
|
||||||
|
npmjs and an untagged Gitea install resolves to nothing. Name `@main` or `@next`
|
||||||
|
explicitly. See [Channels](#channels) for what the tags mean and how to set the
|
||||||
|
scope persistently. The CLI itself is on npmjs and installs without either.
|
||||||
|
|
||||||
#### Channels
|
#### Channels
|
||||||
|
|
||||||
Three dist-tags are published, and the one you install from is the one you stay
|
Three dist-tags are published, and the one you install from is the one you stay
|
||||||
@@ -406,8 +480,8 @@ npm install -g @bitsquare/nopy@latest
|
|||||||
```
|
```
|
||||||
|
|
||||||
Once a day, `nopy` checks its channel in the background and prints a one-line
|
Once a day, `nopy` checks its channel in the background and prints a one-line
|
||||||
hint to **stderr** when a newer version exists — never to stdout, so `--json`
|
hint to **stderr** when a newer version exists — never to stdout, so a piped
|
||||||
and `--print-only` output stay clean. The answer is cached in
|
`--print-only` stays clean. The answer is cached in
|
||||||
`~/.nopy/update-check.json`; a registry that is slow or unreachable is given
|
`~/.nopy/update-check.json`; a registry that is slow or unreachable is given
|
||||||
1.5 seconds and then ignored.
|
1.5 seconds and then ignored.
|
||||||
|
|
||||||
@@ -493,11 +567,14 @@ Every deployment is automatically recorded to a `.nopy.history.json` file in the
|
|||||||
|
|
||||||
The recording happens before the deploy commands run, so a **failed** deployment is recorded too — `-R` is the quick way to retry one after fixing the cause. Replaying a session with `-R` or `-H` does not itself create a new entry, so repeating never pushes the original run out of the list.
|
The recording happens before the deploy commands run, so a **failed** deployment is recorded too — `-R` is the quick way to retry one after fixing the cause. Replaying a session with `-R` or `-H` does not itself create a new entry, so repeating never pushes the original run out of the list.
|
||||||
|
|
||||||
|
A `--load-session` run *is* recorded, and the distinction is the point: a session file has never been in history, so without the entry `nopy history` would report nothing afterwards and `-R` would have nothing to repeat.
|
||||||
|
|
||||||
A run is *not* recorded when:
|
A run is *not* recorded when:
|
||||||
|
|
||||||
- `--dry-run` or `--no-history` is passed
|
- `--dry-run`, `--print-only` or `--no-save-history` is passed — the first two deploy nothing, and history is what `-R` repeats
|
||||||
- No cubes were selected, so there was nothing to deploy
|
- No cubes were selected, so there was nothing to deploy
|
||||||
- `history.autoSave` is set to `false` in `.nopyrc.json`
|
- `history.autoSave` is set to `false` in `.nopyrc.json`
|
||||||
|
- it is a `-R` or `-H` replay, as above
|
||||||
|
|
||||||
Because the history file is resolved against the current working directory, each project keeps its own history — running nopy from a different directory will not find the previous run. As with session files, passwords are never stored and are re-prompted on replay.
|
Because the history file is resolved against the current working directory, each project keeps its own history — running nopy from a different directory will not find the previous run. As with session files, passwords are never stored and are re-prompted on replay.
|
||||||
|
|
||||||
@@ -534,14 +611,26 @@ nopy install --dry-run
|
|||||||
|
|
||||||
Shows the execution plan including commands, environment variables, and targets without running anything. Sensitive data is masked in output.
|
Shows the execution plan including commands, environment variables, and targets without running anything. Sensitive data is masked in output.
|
||||||
|
|
||||||
**JSON output (for CI/CD)**:
|
**CI/CD**:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
nopy install --json
|
nopy install --print-only > plan.txt # the commands, and nothing else
|
||||||
nopy history --json
|
nopy install -D # run it; exit code 1 if any cube failed
|
||||||
```
|
```
|
||||||
|
|
||||||
Machine-readable JSON output for scripting and CI/CD integration.
|
There is no `--json` on `install`, deliberately. A deploy runs pyinfra with
|
||||||
|
inherited stdio, so during a run nopy does not own its own stdout — pyinfra does,
|
||||||
|
and writes an unbounded amount to it. Anything nopy appended afterwards would not
|
||||||
|
be parseable by any definition a caller could rely on. Two things are guaranteed
|
||||||
|
instead:
|
||||||
|
|
||||||
|
- **stdout carries the deploy commands and pyinfra's own output. Everything nopy
|
||||||
|
says about itself — the config banner, progress lines, warnings, the update
|
||||||
|
hint, errors — goes to stderr.** So `--print-only` redirects cleanly.
|
||||||
|
- **The exit code is the verdict**: `1` if any cube failed, `0` otherwise.
|
||||||
|
|
||||||
|
`nopy history --json` is unaffected and is how a script finds the id to pass to
|
||||||
|
`-H`.
|
||||||
|
|
||||||
**Continue on error**:
|
**Continue on error**:
|
||||||
|
|
||||||
|
|||||||
+117
-62
@@ -137,6 +137,7 @@ class Cube<Schema extends AnyObjectSchema = AnyObjectSchema> {
|
|||||||
get secrets(): string[]; // manifest.secrets ?? []
|
get secrets(): string[]; // manifest.secrets ?? []
|
||||||
|
|
||||||
getDefaults(): z.infer<Schema>;
|
getDefaults(): z.infer<Schema>;
|
||||||
|
schemaKeys(): string[];
|
||||||
requiredKeys(): string[];
|
requiredKeys(): string[];
|
||||||
isSecret(key: string): boolean;
|
isSecret(key: string): boolean;
|
||||||
}
|
}
|
||||||
@@ -151,6 +152,11 @@ single required field used to leave the cube with no variables at all.
|
|||||||
and not optional. A `--use-defaults` run that cannot supply one aborts by name
|
and not optional. A `--use-defaults` run that cannot supply one aborts by name
|
||||||
rather than deploying the cube with the value missing.
|
rather than deploying the cube with the value missing.
|
||||||
|
|
||||||
|
`schemaKeys()` returns every declared key, required or not. It answers a
|
||||||
|
different question — whether the cube *claims to know about* a key, rather than
|
||||||
|
whether it has a value for one — and that is what decides whether a secret in the
|
||||||
|
config `env` is allowed to reach it.
|
||||||
|
|
||||||
### `CubeSource`
|
### `CubeSource`
|
||||||
|
|
||||||
Where a cube came from. Carried because a cube's directory does not say how it
|
Where a cube came from. Carried because a cube's directory does not say how it
|
||||||
@@ -264,13 +270,12 @@ const result = await nopy({ useDefaults: true, dryRun: true });
|
|||||||
|------|------|---------|-------------|
|
|------|------|---------|-------------|
|
||||||
| `useDefaults` | `boolean` | `false` | Skip the variable prompts. A cube with a required key nothing supplied aborts the run by name. |
|
| `useDefaults` | `boolean` | `false` | Skip the variable prompts. A cube with a required key nothing supplied aborts the run by name. |
|
||||||
| `useAuthKey` | `boolean` | `false` | Force SSH key auth, skipping the auth prompt. |
|
| `useAuthKey` | `boolean` | `false` | Force SSH key auth, skipping the auth prompt. |
|
||||||
| `saveSession` | `string` | – | Path to write the session to. **Ignored during a replay.** |
|
| `saveSession` | `string` | – | Path to write the session to. Honoured on a replay too. |
|
||||||
| `loadSession` | `string` | – | Path to a session file to replay. |
|
| `loadSession` | `string` | – | Path to a session file to replay. |
|
||||||
| `replaySession` | `NopySession` | – | A session object to replay, used by `-R` / `-H` from history. Takes precedence over `loadSession`. |
|
| `replaySession` | `NopySession` | – | A session object to replay, used by `-R` / `-H` from history. Takes precedence over `loadSession`. |
|
||||||
| `dryRun` | `boolean` | `false` | Print the execution plan instead of running it. |
|
| `dryRun` | `boolean` | `false` | Print the execution plan instead of running it. |
|
||||||
| `printOnly` | `boolean` | `false` | Print the built pyinfra commands and return; the executor is never reached. |
|
| `printOnly` | `boolean` | `false` | Print the built pyinfra commands and return; the executor is never reached. |
|
||||||
| `continueOnError` | `boolean` | `false` | Keep going after a cube fails. |
|
| `continueOnError` | `boolean` | `false` | Keep going after a cube fails. |
|
||||||
| `jsonOutput` | `boolean` | `false` | Suppress the config banner and progress lines. See [Known gaps](#known-gaps). |
|
|
||||||
| `saveToHistory` | `boolean` | `true` | Record the session in `.nopy.history.json`. |
|
| `saveToHistory` | `boolean` | `true` | Record the session in `.nopy.history.json`. |
|
||||||
|
|
||||||
**Returns:** `Promise<NopyResult | undefined>` — `undefined` when cube loading
|
**Returns:** `Promise<NopyResult | undefined>` — `undefined` when cube loading
|
||||||
@@ -434,21 +439,30 @@ Recursive, per (cube, host):
|
|||||||
5. emit the deploy call;
|
5. emit the deploy call;
|
||||||
6. run `after` hooks.
|
6. run `after` hooks.
|
||||||
|
|
||||||
There is no separate topological sort — the ordering falls out of the recursion,
|
There is no separate topological sort — emission is post-order, so a dependency
|
||||||
and a `${cubeId}:${host}` set makes emission idempotent. Consequently there is no
|
is always emitted ahead of its dependent and the ordering *is* topological
|
||||||
cycle detection either: two mutually dependent cubes recurse until the stack
|
without an algorithm computing it. A `${cubeId}:${host}` set makes emission
|
||||||
overflows.
|
idempotent.
|
||||||
|
|
||||||
**Throws** when the cube id is unknown, when `useDefaults` cannot fill a required
|
Cycles are detected by the resolution stack rather than by the sort that does not
|
||||||
key, when a replay would need a value only the user has (secrets are never
|
exist: a (cube, host) pair re-entered while it is still resolving raises with the
|
||||||
recorded), and when a cancelled prompt leaves a required key empty.
|
whole path named — `Circular dependency on host1: a → b → c → a`. The stack is
|
||||||
|
separate from the idempotence set on purpose, since re-entering a *finished* cube
|
||||||
|
with different `param` overrides is legitimate and a dependency or hook may do it.
|
||||||
|
|
||||||
The command it builds:
|
**Throws** when the cube id is unknown, when the dependency graph contains a
|
||||||
|
cycle, when `useDefaults` cannot fill a required key, when a replay would need a
|
||||||
|
value only the user has (secrets are never recorded), and when a cancelled prompt
|
||||||
|
leaves a required key empty.
|
||||||
|
|
||||||
|
The command it builds — an argv array, one element per argument, nothing quoted:
|
||||||
|
|
||||||
```
|
```
|
||||||
pyinfra <host> -y [--user U --password P] --data "K=V" … --chdir <cubeDir> <cubeDir>/<deployScript>
|
pyinfra <host> -y [-v|-vv|-vvv] [--debug] [--user U --password P] --data K=V … --chdir <cubeDir> <cubeDir>/<deployScript>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The verbosity and debug flags come from `config.log` via `logConfigToFlags()`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Variables Module
|
## Variables Module
|
||||||
@@ -507,9 +521,10 @@ displaced stays visible underneath. The trace is never persisted.
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
class Variables {
|
class Variables {
|
||||||
constructor(env?: TVariables);
|
constructor(env?: TVariables, globalSecrets?: Iterable<string>);
|
||||||
|
|
||||||
declareSecrets(cube: string, keys: readonly string[]): void;
|
declareSecrets(cube: string, keys: readonly string[]): void;
|
||||||
|
declareSchema(cube: string, keys: readonly string[]): void;
|
||||||
isSecret(cube: string, name: string): boolean;
|
isSecret(cube: string, name: string): boolean;
|
||||||
|
|
||||||
assign(cube: string, origin: Origin, values?: TVariables): void;
|
assign(cube: string, origin: Origin, values?: TVariables): void;
|
||||||
@@ -527,6 +542,25 @@ const MASK = '********';
|
|||||||
`declareSecrets()` is retroactive as well as prospective, so it does not matter
|
`declareSecrets()` is retroactive as well as prospective, so it does not matter
|
||||||
whether the caller declares before or after the values arrive.
|
whether the caller declares before or after the values arrive.
|
||||||
|
|
||||||
|
`globalSecrets` is every key *any* manifest declares secret, plus the config's
|
||||||
|
own `secrets` list. `nopy()` computes it once after `loadCubes()`, before the
|
||||||
|
first cube resolves, so resolution order cannot change whether a value is treated
|
||||||
|
as a credential. It does two things:
|
||||||
|
|
||||||
|
- `isSecret()` is true for such a key on **every** cube, so a manifest that lists
|
||||||
|
`PASSWORD` in `schema` and forgets it in `secrets` still gets masking and still
|
||||||
|
keeps the value out of the session.
|
||||||
|
- The config `env` stops being broadcast for it. Ordinary `env` keys are seeded
|
||||||
|
onto every cube — deliberately, since a cube may read a key off `host.data`
|
||||||
|
that it never declared — but a secret reaches only the cubes whose
|
||||||
|
`schemaKeys()` include it.
|
||||||
|
|
||||||
|
`declareSchema()` is what supplies those keys, and it has an ordering
|
||||||
|
requirement: call it before anything assigns to the cube, because the first
|
||||||
|
assignment is what seeds `env`. `BuildContext.resolveCube` calls it immediately
|
||||||
|
after `declareSecrets()`. It deliberately does not create the cube's bucket
|
||||||
|
itself.
|
||||||
|
|
||||||
`persistable()` leaves a secret out entirely rather than masking it, so a replay
|
`persistable()` leaves a secret out entirely rather than masking it, so a replay
|
||||||
sees it as absent and asks for it again. That is why replaying a session whose
|
sees it as absent and asks for it again. That is why replaying a session whose
|
||||||
cubes declare secrets is interactive even under `-D` — a `-D` replay that would
|
cubes declare secrets is interactive even under `-D` — a `-D` replay that would
|
||||||
@@ -574,9 +608,14 @@ interface ExecutionOptions {
|
|||||||
### `executeDeployCalls(calls, options?)`
|
### `executeDeployCalls(calls, options?)`
|
||||||
|
|
||||||
Runs the calls **sequentially**, in the order they were built, through
|
Runs the calls **sequentially**, in the order they were built, through
|
||||||
`execa({ shell: true })` with `stdio: 'inherit'` so pyinfra's output reaches the
|
`execa(command[0], command.slice(1))` with `stdio: 'inherit'` so pyinfra's output
|
||||||
terminal live. Stops at the first failure unless `continueOnError`. With
|
reaches the terminal live. Stops at the first failure unless `continueOnError`.
|
||||||
`dryRun`, prints the plan and returns `[]` without executing.
|
With `dryRun`, prints the plan and returns `[]` without executing.
|
||||||
|
|
||||||
|
**No shell.** It used to join `command` into one string and run it through
|
||||||
|
`execa({ shell: true })`, which made every `--data` value shell syntax: a
|
||||||
|
password or a variable containing `;`, a backtick or `$(…)` was executed rather
|
||||||
|
than passed along. Spawning the argv directly removes the parse step entirely.
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
const results = await executeDeployCalls(calls, {
|
const results = await executeDeployCalls(calls, {
|
||||||
@@ -585,15 +624,16 @@ const results = await executeDeployCalls(calls, {
|
|||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
### `outputExecutionPlan(calls, asJson?)`
|
### `outputExecutionPlan(calls)`
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
outputExecutionPlan(deployCalls); // text
|
outputExecutionPlan(deployCalls);
|
||||||
outputExecutionPlan(deployCalls, true); // JSON
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Both forms mask secrets. Note that `executeDeployCalls` calls this without the
|
Prints the plan a `--dry-run` shows, with secrets masked. Went from
|
||||||
second argument, so `--dry-run --json` prints the text plan.
|
`(calls, asJson?)` to `(calls)` when `--json` was removed; the JSON branch was
|
||||||
|
unreachable from the CLI, since `executeDeployCalls` never passed the second
|
||||||
|
argument.
|
||||||
|
|
||||||
### `maskCommand(call)` / `maskVariables(call)`
|
### `maskCommand(call)` / `maskVariables(call)`
|
||||||
|
|
||||||
@@ -604,9 +644,11 @@ maskVariables(call); // Record<string, string>
|
|||||||
|
|
||||||
pyinfra takes its data on the command line, so the real values have to be in
|
pyinfra takes its data on the command line, so the real values have to be in
|
||||||
`call.command`; these are the last point before they would reach a log, a
|
`call.command`; these are the last point before they would reach a log, a
|
||||||
`--print-only` dump or a dry-run plan. `maskCommand` replaces the SSH
|
`--print-only` dump or a dry-run plan. `maskCommand` walks the argv, replaces the
|
||||||
`--password` argument and every `--data "KEY=…"` whose key the manifest declared
|
element after `--password` and the value of every `--data KEY=…` whose key the
|
||||||
a secret.
|
manifest declared a secret, and shell-quotes the rest so `--print-only` output
|
||||||
|
stays pasteable. It is the only thing that joins `command` into a string —
|
||||||
|
nothing executes it that way.
|
||||||
|
|
||||||
This covers nopy's own output only. The value still reaches pyinfra on its
|
This covers nopy's own output only. The value still reaches pyinfra on its
|
||||||
command line, so it is visible in `ps` — inherent to pyinfra's `--data`
|
command line, so it is visible in `ps` — inherent to pyinfra's `--data`
|
||||||
@@ -641,7 +683,7 @@ interface WorkflowResult {
|
|||||||
authMethod: string;
|
authMethod: string;
|
||||||
username?: string;
|
username?: string;
|
||||||
password?: string;
|
password?: string;
|
||||||
isReplay: boolean;
|
replaySource?: 'file' | 'history'; // undefined on a fresh interactive run
|
||||||
}
|
}
|
||||||
|
|
||||||
interface WorkflowOptions {
|
interface WorkflowOptions {
|
||||||
@@ -673,10 +715,12 @@ The same, from a session object rather than a path — the `-R` / `-H` path.
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
interface NopySession {
|
interface NopySession {
|
||||||
|
cubes: CubeSession[]; // required
|
||||||
|
auth: AuthSession; // required
|
||||||
|
version?: string;
|
||||||
|
timestamp?: string; // ISO 8601
|
||||||
name?: string;
|
name?: string;
|
||||||
cubes: CubeSession[];
|
|
||||||
hosts?: string[];
|
hosts?: string[];
|
||||||
auth: AuthSession;
|
|
||||||
env?: TVariables;
|
env?: TVariables;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -692,7 +736,10 @@ interface AuthSession {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
There is no `version` or `timestamp` field, and nothing validates compatibility.
|
`version` and `timestamp` are stamped on every session nopy writes and demanded
|
||||||
|
of none it reads — an older file, or a hand-written one, simply lacks them.
|
||||||
|
Nothing validates compatibility beyond a warning on an unrecognised `version`;
|
||||||
|
the constant is exported as `SESSION_VERSION`.
|
||||||
|
|
||||||
A `CubeSession` records every value the cube settled on, whatever its origin —
|
A `CubeSession` records every value the cube settled on, whatever its origin —
|
||||||
not just the prompted ones — minus anything the manifest declared a secret. So a
|
not just the prompted ones — minus anything the manifest declared a secret. So a
|
||||||
@@ -702,8 +749,7 @@ and `env` happen to say later.
|
|||||||
|
|
||||||
### `saveSession(session, filePath)`
|
### `saveSession(session, filePath)`
|
||||||
|
|
||||||
Writes JSON, creating the directory if needed. Note that `nopy()` skips this
|
Writes JSON, creating the directory if needed.
|
||||||
during a replay.
|
|
||||||
|
|
||||||
### `loadSession(filePath)`
|
### `loadSession(filePath)`
|
||||||
|
|
||||||
@@ -713,7 +759,8 @@ const session = await loadSession('./deployment.session.mjs'); // default expor
|
|||||||
```
|
```
|
||||||
|
|
||||||
Dispatches on the extension; `.json` and `.mjs` only. Validates that `cubes` is
|
Dispatches on the extension; `.json` and `.mjs` only. Validates that `cubes` is
|
||||||
an array, that `hosts` (if present) is an array, and that `auth` exists.
|
an array, that `hosts` (if present) is an array, and that `auth` exists. A
|
||||||
|
`version` other than `SESSION_VERSION` warns on stderr and loads anyway.
|
||||||
|
|
||||||
### `createSession(params)`
|
### `createSession(params)`
|
||||||
|
|
||||||
@@ -725,18 +772,27 @@ const session = createSession({
|
|||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Stamps `version` and `timestamp`; pass `timestamp` to override the latter. It
|
||||||
|
does not derive a `name` — that needs the resolved cube list, which does not
|
||||||
|
exist yet at the point the session is created, so `nopy()` fills it in at save
|
||||||
|
time.
|
||||||
|
|
||||||
|
### `describeSession(session, timestamp)`
|
||||||
|
|
||||||
|
The one-line `date - cubes → hosts` description, shared with the history list so
|
||||||
|
that the two cannot drift.
|
||||||
|
|
||||||
### `listSessions(dirPath?)`
|
### `listSessions(dirPath?)`
|
||||||
|
|
||||||
Non-recursive; matches **`*.session.json`** and **`*.session.mjs`** only.
|
Non-recursive; matches `*.nopysession.json`, `*.nopysession.mjs`,
|
||||||
A file named `deploy.nopysession.json` will not be listed, though `loadSession`
|
`*.session.json` and `*.session.mjs`.
|
||||||
reads it fine.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## History Module
|
## History Module
|
||||||
|
|
||||||
Sessions are recorded automatically after a successful non-replay run, into
|
Sessions are recorded automatically, into `.nopy.history.json` in the working
|
||||||
`.nopy.history.json` in the working directory.
|
directory, before the deploy commands run — so a failed run is recorded too.
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
const HISTORY_FILE = '.nopy.history.json';
|
const HISTORY_FILE = '.nopy.history.json';
|
||||||
@@ -767,8 +823,10 @@ interface SessionHistory {
|
|||||||
| `removeFromHistory(id)` | `boolean` | `false` if the id was not found |
|
| `removeFromHistory(id)` | `boolean` | `false` if the id was not found |
|
||||||
| `formatHistoryList(entries)` | `string` | what `nopy history` prints |
|
| `formatHistoryList(entries)` | `string` | what `nopy history` prints |
|
||||||
|
|
||||||
Recording is suppressed for a dry run, a replay, a run that built no deploy
|
Recording is suppressed for a dry run, a print-only run, a `-R`/`-H` replay out
|
||||||
calls, `--no-history`, and `history.autoSave: false` in the config.
|
of history, a run that built no deploy calls, `--no-save-history`, and
|
||||||
|
`history.autoSave: false` in the config. A `--load-session` run **is** recorded: it is not in history already, and
|
||||||
|
without the entry `-R` would have nothing to repeat.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -785,6 +843,7 @@ interface NopyConfig {
|
|||||||
cubeDirs: string[];
|
cubeDirs: string[];
|
||||||
cubePackages: CubePackageRef[];
|
cubePackages: CubePackageRef[];
|
||||||
env: TVariables;
|
env: TVariables;
|
||||||
|
secrets?: string[]; // env keys to treat as sensitive that no manifest declares
|
||||||
log?: LogConfig;
|
log?: LogConfig;
|
||||||
history?: HistoryConfig;
|
history?: HistoryConfig;
|
||||||
execution?: ExecutionConfig;
|
execution?: ExecutionConfig;
|
||||||
@@ -827,9 +886,8 @@ config's `node_modules` rather than the working directory's. It is the same
|
|||||||
problem `PATH_PROPERTIES` solves for relative `cubeDirs`, with a different answer:
|
problem `PATH_PROPERTIES` solves for relative `cubeDirs`, with a different answer:
|
||||||
a reference to resolve later instead of a rewritten path.
|
a reference to resolve later instead of a rewritten path.
|
||||||
|
|
||||||
> The `CubePackageRef` name is currently not re-exported from the package root,
|
Re-exported from the package root alongside `NopyConfig`, which refers to it —
|
||||||
> though `NopyConfig` refers to it. Import it from `@bitsquare/nopy` and you get
|
it was not, until the regeneration of this document noticed.
|
||||||
> `NopyConfig` but not this type by name.
|
|
||||||
|
|
||||||
### `loadConfig()`
|
### `loadConfig()`
|
||||||
|
|
||||||
@@ -1031,7 +1089,7 @@ on `PATH`.
|
|||||||
**never throws** — it sits in front of every command the user actually asked
|
**never throws** — it sits in front of every command the user actually asked
|
||||||
for. Returns `null` immediately when `isUpdateCheckDisabled(env)`:
|
for. Returns `null` immediately when `isUpdateCheckDisabled(env)`:
|
||||||
`NOPY_NO_UPDATE_CHECK` set to anything but `0`/`false`, or `CI` set at all. The
|
`NOPY_NO_UPDATE_CHECK` set to anything but `0`/`false`, or `CI` set at all. The
|
||||||
CLI prints it to **stderr**, so `--json` and piped stdout stay clean.
|
CLI prints it to **stderr**, so a piped `--print-only` stays clean.
|
||||||
|
|
||||||
### `selfUpdate(options)` → `SelfUpdateResult`
|
### `selfUpdate(options)` → `SelfUpdateResult`
|
||||||
|
|
||||||
@@ -1062,8 +1120,7 @@ nopy install -l ./sess.json # replay a session file
|
|||||||
nopy install -n # dry run — print the plan, execute nothing
|
nopy install -n # dry run — print the plan, execute nothing
|
||||||
nopy install -P # print the built pyinfra commands and exit
|
nopy install -P # print the built pyinfra commands and exit
|
||||||
nopy install -c # continue after a failure
|
nopy install -c # continue after a failure
|
||||||
nopy install -j # JSON output
|
nopy install --no-save-history # do not record this run
|
||||||
nopy install --no-history # do not record this run
|
|
||||||
|
|
||||||
nopy history # list recorded sessions (alias: h; -j for JSON)
|
nopy history # list recorded sessions (alias: h; -j for JSON)
|
||||||
nopy clear-history # drop them all
|
nopy clear-history # drop them all
|
||||||
@@ -1083,8 +1140,10 @@ Exit code is 1 when any cube failed.
|
|||||||
"up to date", since an unanswerable check is not a negative answer. See
|
"up to date", since an unanswerable check is not a negative answer. See
|
||||||
[Known gaps](#known-gaps) for what that message conflates.
|
[Known gaps](#known-gaps) for what that message conflates.
|
||||||
|
|
||||||
> `-H <id>` and `--no-history` share one Commander destination, so passing both
|
> The suppression flag is `--no-save-history`, not `--no-history`. Commander
|
||||||
> discards the id and falls through to an interactive run.
|
> derives an option's destination from its long flag with `no-` stripped, so
|
||||||
|
> `--no-history` wrote to the same `options.history` that `-H <id>` does and
|
||||||
|
> `nopy install -H abc --no-history` silently discarded the id.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -1127,12 +1186,15 @@ export default Manifest({
|
|||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
> **Call `.default()` before `.describe()`.** In zod 4, `.default()` returns a
|
> **The order of `.default()` and `.describe()` does not matter.** It used to.
|
||||||
> `ZodDefault` wrapper that does not inherit `.description` from the type it
|
> In zod 4, `.default()` returns a `ZodDefault` wrapper that does not inherit
|
||||||
> wraps, and the prompt reads the description off the outer node. So
|
> `.description` from the type it wraps, so
|
||||||
> `z.boolean().describe('Update cache').default(false)` prompts with the bare key
|
> `z.boolean().describe('Update cache').default(false)` prompted with the bare
|
||||||
> `UPDATE`, while `z.boolean().default(false).describe('Update cache')` prompts
|
> key `UPDATE` while the other order prompted with the sentence — a difference
|
||||||
> with the sentence. Verified against zod 4.4.3.
|
> nothing announced, and one that 15 of the 22 core cubes were on the wrong side
|
||||||
|
> of. The prompt now unwraps `default`/`optional`/`nullable` looking for a
|
||||||
|
> description, so either chaining order gives the label. Verified against
|
||||||
|
> zod 4.4.3.
|
||||||
|
|
||||||
Every schema key reaches pyinfra as `--data KEY=value`, so `host.data.KEY` is
|
Every schema key reaches pyinfra as `--data KEY=value`, so `host.data.KEY` is
|
||||||
always defined. pyinfra parses the values itself: `"true"` arrives as a bool and
|
always defined. pyinfra parses the values itself: `"true"` arrives as a bool and
|
||||||
@@ -1164,20 +1226,13 @@ For packaging cubes as an installable npm bundle, see
|
|||||||
Real behaviour that a reader would otherwise take on trust. Tracked in
|
Real behaviour that a reader would otherwise take on trust. Tracked in
|
||||||
`DOCS-AUDIT.md` and summarised in `CLAUDE.md`.
|
`DOCS-AUDIT.md` and summarised in `CLAUDE.md`.
|
||||||
|
|
||||||
- **`logConfigToFlags()` is never consumed.** It is exported and unit-tested, but
|
|
||||||
nothing feeds its output into the built pyinfra command, so `log.verbosity` and
|
|
||||||
`log.debug` in `.nopyrc.json` have no effect today.
|
|
||||||
- **`--json` emits nothing on success.** `jsonOutput` suppresses the banner and
|
|
||||||
the progress lines, and prints `{success: false, errors}` when cube *loading*
|
|
||||||
fails. The success path returns `NopyResult` to the caller without printing it,
|
|
||||||
so a CI job gets pyinfra's inherited stdio and an exit code. `--dry-run --json`
|
|
||||||
prints the *text* plan.
|
|
||||||
- **No cycle detection.** Ordering is a side effect of recursion, not a
|
|
||||||
topological sort. Two mutually dependent cubes overflow the stack.
|
|
||||||
- **`DeployCall.dependencies` is always `[]`.** The field is populated nowhere;
|
- **`DeployCall.dependencies` is always `[]`.** The field is populated nowhere;
|
||||||
dependency information lives in the emission order.
|
dependency information lives in the emission order.
|
||||||
- **`ExecutionResult.stdout` / `.stderr` are always `undefined`,** because the
|
- **`ExecutionResult.stdout` / `.stderr` are always `undefined`,** because the
|
||||||
executor inherits stdio rather than capturing it.
|
executor inherits stdio rather than capturing it. This is also why `install`
|
||||||
|
has no `--json`: during a run nopy does not own its own stdout, so there is no
|
||||||
|
stream to put a machine-readable answer on. Use `--print-only` for the plan and
|
||||||
|
the exit code for the verdict.
|
||||||
- **Hook variables are not schema-validated.** The second argument to a hook is
|
- **Hook variables are not schema-validated.** The second argument to a hook is
|
||||||
the effective values as collected. `schema.parse()` runs in exactly one place —
|
the effective values as collected. `schema.parse()` runs in exactly one place —
|
||||||
`Cube.getDefaults()`, against `{}` — and prompt input is type-coerced, which is
|
`Cube.getDefaults()`, against `{}` — and prompt input is type-coerced, which is
|
||||||
|
|||||||
@@ -314,8 +314,8 @@ Rename one of them, or remove a source from .nopyrc.json.
|
|||||||
There is deliberately no override, alias or precedence rule. If two bundles ever
|
There is deliberately no override, alias or precedence rule. If two bundles ever
|
||||||
claim the same id they are mutually exclusive, and the fix is upstream.
|
claim the same id they are mutually exclusive, and the fix is upstream.
|
||||||
|
|
||||||
Surface the source in the interactive picker and in `--json` output so a user can
|
Surface the source in the interactive picker so a user can see where a cube came
|
||||||
see where a cube came from before running it.
|
from before running it.
|
||||||
|
|
||||||
## Phase 4 — `@bitsquare/nopy-cubes`, the authoring package — **done**
|
## Phase 4 — `@bitsquare/nopy-cubes`, the authoring package — **done**
|
||||||
|
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ This guide explains how to set up a local Docker container to test `nopy` deploy
|
|||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- Docker installed and running on your machine.
|
- Docker installed and running on your machine.
|
||||||
- `nopy` installed and linked (see [README.md](./README.md)).
|
- `nopy` installed and linked (see [README.md](../README.md)).
|
||||||
|
|
||||||
## 1. Setup SSH Key (Important)
|
## 1. Setup SSH Key (Important)
|
||||||
|
|
||||||
@@ -80,3 +80,48 @@ To stop and remove the container:
|
|||||||
```bash
|
```bash
|
||||||
docker rm -f nopy-test-container
|
docker rm -f nopy-test-container
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Building an image instead of targeting a container
|
||||||
|
|
||||||
|
The `@docker` connector reads its identifier two ways, and the difference is
|
||||||
|
the whole feature:
|
||||||
|
|
||||||
|
| Host | What pyinfra does |
|
||||||
|
| --------------------- | ------------------------------------------------------------------------------------- |
|
||||||
|
| `@docker/<container>` | runs against that container and leaves it running — the flow above |
|
||||||
|
| `@docker/<image>` | starts a throwaway container, deploys into it, `docker commit`s it, prints the new image ID, removes the container |
|
||||||
|
|
||||||
|
It looks for a matching container first, so nothing distinguishes the two at the
|
||||||
|
prompt: pick `docker` at host selection and enter either an existing container
|
||||||
|
or an image reference such as `ubuntu:24.04`.
|
||||||
|
|
||||||
|
```
|
||||||
|
$ nopy install
|
||||||
|
? Select host from inventory docker
|
||||||
|
? Specify docker container name/id, or an image to build from: ubuntu:24.04
|
||||||
|
...
|
||||||
|
--> docker build complete, image ID: 39b782da6859
|
||||||
|
$ docker tag 39b782da6859 myapp:1.0
|
||||||
|
```
|
||||||
|
|
||||||
|
Two things the connector cannot do, both worth knowing before treating this as a
|
||||||
|
Dockerfile replacement. The commit is untagged, so the image exists only as an
|
||||||
|
ID until you tag it; and it carries the base image's metadata unchanged —
|
||||||
|
`CMD`, `ENTRYPOINT`, `ENV`, `EXPOSE` have no equivalent in a cube. When either
|
||||||
|
matters, own the container yourself and commit deliberately:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cid=$(docker run -d ubuntu:24.04 sleep infinity)
|
||||||
|
nopy install # host: docker → paste $cid at the prompt
|
||||||
|
docker commit --change 'CMD ["/usr/sbin/sshd","-D"]' "$cid" myapp:1.0
|
||||||
|
docker rm -f "$cid"
|
||||||
|
```
|
||||||
|
|
||||||
|
There is no `--host` flag; for an unattended build put the identifier in a
|
||||||
|
session file's `hosts` array (as `example.nopysession.json` does) and replay it
|
||||||
|
with `nopy install -l <file>`.
|
||||||
|
|
||||||
|
Note also that an image target starts from a fresh container every run, so every
|
||||||
|
cube reports changes every time — idempotence only shows up when you re-run
|
||||||
|
against a container id. And there is no init system in a plain container, so
|
||||||
|
service-level cubes still need the `--privileged` systemd setup above.
|
||||||
|
|||||||
@@ -0,0 +1,348 @@
|
|||||||
|
# Requirements and facts
|
||||||
|
|
||||||
|
A proposal, not a record. Nothing here is built.
|
||||||
|
|
||||||
|
The question it answers: a cube should be able to say *"in order to run, a user
|
||||||
|
named xyz must exist, with fish as its shell, in these groups"* — and the engine
|
||||||
|
should check that against the real host rather than blindly running whatever
|
||||||
|
cube happens to create such a user.
|
||||||
|
|
||||||
|
Everything under [What pyinfra actually gives us](#what-pyinfra-actually-gives-us)
|
||||||
|
was measured against pyinfra **3.5.1**. Everything under
|
||||||
|
[The proposal](#the-proposal) is design.
|
||||||
|
|
||||||
|
## Contents
|
||||||
|
|
||||||
|
- [The problem with `dependencies`](#the-problem-with-dependencies)
|
||||||
|
- [What pyinfra actually gives us](#what-pyinfra-actually-gives-us)
|
||||||
|
- [The proposal](#the-proposal)
|
||||||
|
- [Why `facts.py` exposes a function, not a script](#why-factspy-exposes-a-function-not-a-script)
|
||||||
|
- [Execution model](#execution-model)
|
||||||
|
- [Sharp edges](#sharp-edges)
|
||||||
|
- [Measured vs. assumed](#measured-vs-assumed)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The problem with `dependencies`
|
||||||
|
|
||||||
|
`manifest.dependencies` conflates two things that are not the same:
|
||||||
|
|
||||||
|
- **a requirement** — "this cube needs a user xyz to exist"
|
||||||
|
- **a remedy** — "therefore run `user:add`"
|
||||||
|
|
||||||
|
Today only the remedy is expressible. `user:add` declares
|
||||||
|
`dependencies: () => ['apt:essentials']`, which means *always run
|
||||||
|
`apt:essentials`*, on every host, forever, regardless of whether anything it
|
||||||
|
installs is missing.
|
||||||
|
|
||||||
|
### What that actually costs
|
||||||
|
|
||||||
|
Be honest about this, because it is smaller than it first looks and it changes
|
||||||
|
what the feature is for.
|
||||||
|
|
||||||
|
pyinfra operations are **already fact-diffed**. `server.user(present=True,
|
||||||
|
shell=…)` gathers `server.Users` itself and no-ops when the state matches. The
|
||||||
|
current design is therefore not *incorrect* — it is idempotent. What it costs is:
|
||||||
|
|
||||||
|
1. **Prompts.** Resolving `user:add` drags `apt:essentials` into the plan *and
|
||||||
|
its variables into the prompt sequence*. The user is asked questions about a
|
||||||
|
cube they never chose. This is the real, visible tax.
|
||||||
|
2. **Time.** A no-op pyinfra run is still a connection, a fact gather, and an
|
||||||
|
operation build.
|
||||||
|
3. **Legibility.** The plan cannot say *"user xyz already exists, skipping"*.
|
||||||
|
|
||||||
|
This matters because the obvious objection to the whole feature is *"just write
|
||||||
|
`if not host.get_fact(…)` inside `deploy.py` — that is idiomatic pyinfra"*. The
|
||||||
|
answer is that nopy's layer is **planning and prompting**: deciding which cubes
|
||||||
|
enter the plan and which variables to ask for, before anything runs. A condition
|
||||||
|
inside `deploy.py` cannot help with either. That is the justification for lifting
|
||||||
|
facts into the Node runtime at all — and it means the feature buys **legibility
|
||||||
|
and prompt-avoidance, not correctness**.
|
||||||
|
|
||||||
|
## What pyinfra actually gives us
|
||||||
|
|
||||||
|
### The `fact` subcommand is not a machine interface
|
||||||
|
|
||||||
|
`pyinfra @local fact server.Groups` prints JSON — and prints it to **stderr**,
|
||||||
|
via `click.echo(jsonify(…), err=True)` in `pyinfra_cli/prints.py:print_fact`,
|
||||||
|
interleaved with `--> Loading config...` progress lines. Worse,
|
||||||
|
`_run_fact_operations` wraps each fact in `except PyinfraError: pass`, so the
|
||||||
|
command **exits 0 whether or not the fact resolved**.
|
||||||
|
|
||||||
|
Parsing that stream means fishing a JSON blob out of styled log output and
|
||||||
|
having no exit code to check. Rejected.
|
||||||
|
|
||||||
|
### A deploy file that prints to stdout is clean
|
||||||
|
|
||||||
|
pyinfra deploy files execute on the control machine at *build* time, and
|
||||||
|
`host.get_fact()` is available there — it is exactly how operations do their own
|
||||||
|
diffing. A file that gathers facts, prints JSON, and declares no operations
|
||||||
|
works:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import json, sys
|
||||||
|
from pyinfra import host
|
||||||
|
from pyinfra.facts.server import Users, Which
|
||||||
|
|
||||||
|
users = host.get_fact(Users)
|
||||||
|
u = users.get(host.data.USER)
|
||||||
|
print("###NOPY-FACTS###" + json.dumps({
|
||||||
|
"host": host.name,
|
||||||
|
"exists": u is not None,
|
||||||
|
"shell": (u or {}).get("shell"),
|
||||||
|
"groups": (u or {}).get("groups", []),
|
||||||
|
}), file=sys.stdout)
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
$ pyinfra @local -y --data USER=someone facts.py
|
||||||
|
EXIT=0
|
||||||
|
--- STDOUT ---
|
||||||
|
###NOPY-FACTS###{"host": "@local", "exists": false, "shell": null, "groups": []}
|
||||||
|
--- STDERR ---
|
||||||
|
--> Loading config...
|
||||||
|
…
|
||||||
|
--> Results:
|
||||||
|
Operation Hosts Success Error No Change
|
||||||
|
Grand total - - - -
|
||||||
|
```
|
||||||
|
|
||||||
|
Confirmed properties:
|
||||||
|
|
||||||
|
- **stdout is exclusively ours.** Every byte pyinfra emits goes to stderr, which
|
||||||
|
is the same reason nopy's own logging goes there (`configureLogtape`).
|
||||||
|
- **`--data` flows in identically** to a deploy script. Parameterising a probe
|
||||||
|
is free.
|
||||||
|
- **Zero operations is legal.** `Grand total -`, exit 0.
|
||||||
|
- **`host.name` is available**, so multi-host output is self-tagging.
|
||||||
|
- **A custom `FactBase` subclass in a sibling module imports fine** under
|
||||||
|
`--chdir`, so a cube can ship fact classes pyinfra does not have.
|
||||||
|
- **An exception exits 1** with a traceback on stderr.
|
||||||
|
|
||||||
|
### The trap
|
||||||
|
|
||||||
|
**An unreachable host exits 0 and prints nothing.**
|
||||||
|
|
||||||
|
```
|
||||||
|
$ pyinfra nonexistent.invalid.example -y probe.py
|
||||||
|
nonexistent.invalid.example is neither an inventory file, a (list of) hosts…
|
||||||
|
EXIT=0
|
||||||
|
--- STDOUT --- (empty)
|
||||||
|
```
|
||||||
|
|
||||||
|
Absence of a probe result must be a hard failure. It must never be read as
|
||||||
|
"this host has no requirements to check", which is the shape the bug would take.
|
||||||
|
The `###NOPY-FACTS###` sentinel exists for this: one line per host is *required*,
|
||||||
|
and a missing line is an error.
|
||||||
|
|
||||||
|
### `server.Users` already covers the motivating case
|
||||||
|
|
||||||
|
It returns `shell`, `groups`, `home`, `uid`, `gid`, `comment` and `password` per
|
||||||
|
user, keyed by name. `user:add` needs almost no custom fact code — see
|
||||||
|
[Sharp edges](#sharp-edges) for why `password` is a problem.
|
||||||
|
|
||||||
|
## The proposal
|
||||||
|
|
||||||
|
Split the one idea into two manifest fields, because there are two different
|
||||||
|
objects: what a cube can **report** about a host (owned by the cube responsible
|
||||||
|
for that state) and what a cube **demands** (owned by the consumer).
|
||||||
|
|
||||||
|
A single `facts:` field cannot be both.
|
||||||
|
|
||||||
|
### `provides` — on the cube that owns the state
|
||||||
|
|
||||||
|
```js
|
||||||
|
// cubes/user/add/manifest.mjs
|
||||||
|
export default Manifest({
|
||||||
|
id: 'user:add',
|
||||||
|
provides: {
|
||||||
|
/** What the probe returns. Validated on the way back in. */
|
||||||
|
schema: z.object({
|
||||||
|
exists: z.boolean(),
|
||||||
|
shell: z.string().nullable(),
|
||||||
|
groups: z.array(z.string()),
|
||||||
|
}),
|
||||||
|
/** Schema keys the probe needs in order to look anything up. */
|
||||||
|
params: ['USER'],
|
||||||
|
},
|
||||||
|
schema: z.object({ /* … unchanged … */ }),
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Plus a `facts.py` in the cube directory, discovered by convention exactly as
|
||||||
|
`deploy.py` is.
|
||||||
|
|
||||||
|
### `requires` — on the consumer
|
||||||
|
|
||||||
|
```js
|
||||||
|
requires: (vars) => [{
|
||||||
|
cube: 'user:add',
|
||||||
|
with: { USER: vars.DEPLOY_USER },
|
||||||
|
expect: z.object({
|
||||||
|
exists: z.literal(true),
|
||||||
|
shell: z.literal('/usr/bin/fish'),
|
||||||
|
groups: z.array(z.string()).refine((g) => g.includes('docker')),
|
||||||
|
}),
|
||||||
|
}],
|
||||||
|
```
|
||||||
|
|
||||||
|
Three deliberate choices:
|
||||||
|
|
||||||
|
**A requirement names its provider.** If `requires` stated only a predicate, the
|
||||||
|
engine would have to search the cube space for something that satisfies an
|
||||||
|
arbitrary Zod schema. That is a planner, and a planner is a research project.
|
||||||
|
Naming the cube keeps resolution linear and keeps the failure message legible.
|
||||||
|
|
||||||
|
**`requires` is `(vars) => …`, like `dependencies` already is.** A requirement
|
||||||
|
almost always depends on the consumer's own variables — you cannot know *which*
|
||||||
|
user to check for until the consumer has been asked.
|
||||||
|
|
||||||
|
**Zod is the predicate language.** It is already this repo's schema vocabulary,
|
||||||
|
`z.literal` and `.refine()` cover the cases, and `z.treeifyError()` produces the
|
||||||
|
failure report for free. The cost is that `.refine()` closures are not
|
||||||
|
serialisable, so a requirement can never be written into a session — only its
|
||||||
|
*result* can. See [Sharp edges](#sharp-edges).
|
||||||
|
|
||||||
|
### The engine's rule
|
||||||
|
|
||||||
|
This is the half of the design that "only run if requirements are fulfilled"
|
||||||
|
leaves out. When a requirement is **not** met, there are three possible answers:
|
||||||
|
|
||||||
|
| | |
|
||||||
|
| --- | --- |
|
||||||
|
| **Abort** | Honest, and useless. On a bare host nothing is fulfilled, so every fresh deploy fails. |
|
||||||
|
| **Skip the cube** | A silent no-op. Dangerous. |
|
||||||
|
| **Remedy** | Run the named cube. |
|
||||||
|
|
||||||
|
It has to be *remedy* — and remedy means "run the cube that provides it", which
|
||||||
|
is `dependencies` again. That is the actual insight here, and it is a much
|
||||||
|
smaller change than a parallel subsystem:
|
||||||
|
|
||||||
|
> **You do not need a new mechanism. You need dependencies to become conditional.**
|
||||||
|
|
||||||
|
So:
|
||||||
|
|
||||||
|
- requirement **met** → the provider is *not* scheduled and its variables are
|
||||||
|
*not* prompted for. This is where the prompt tax disappears.
|
||||||
|
- requirement **unmet** → the provider is scheduled with `with` applied as
|
||||||
|
overrides. `with` assigns at origin `param`, which already outranks every
|
||||||
|
other origin (`default < env < session < prompt < param`), so no new
|
||||||
|
precedence rule is needed.
|
||||||
|
|
||||||
|
## Why `facts.py` exposes a function, not a script
|
||||||
|
|
||||||
|
The obvious design is symmetry: `deploy.py` is a script, so `facts.py` is a
|
||||||
|
script. Reject it.
|
||||||
|
|
||||||
|
A recursive resolution over a dependency tree issues one probe per (cube, host,
|
||||||
|
params). At one pyinfra invocation each, that is one SSH connection each —
|
||||||
|
seconds apiece, multiplied by the tree. Unaffordable.
|
||||||
|
|
||||||
|
But probes are **pure reads with no ordering constraints between them**, which
|
||||||
|
means every probe for a given host can be gathered in a *single* pyinfra
|
||||||
|
invocation: one generated driver script that imports each cube's `facts.py`,
|
||||||
|
calls it with its params, and emits one JSON object per host. One connection per
|
||||||
|
host per resolution round, instead of one per cube.
|
||||||
|
|
||||||
|
A standalone script can only be run alone. A function can be batched:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# cubes/user/add/facts.py
|
||||||
|
from pyinfra.facts.server import Users
|
||||||
|
|
||||||
|
def gather(host, params):
|
||||||
|
u = host.get_fact(Users).get(params["USER"])
|
||||||
|
return {
|
||||||
|
"exists": u is not None,
|
||||||
|
"shell": (u or {}).get("shell"),
|
||||||
|
"groups": (u or {}).get("groups", []),
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Consequence: params arrive as one `--data NOPY_PROBES=<json>` blob rather than
|
||||||
|
per-cube `--data KEY=…`, since a batched run carries several cubes' params at
|
||||||
|
once. Probe results are memoised per (cube, host, params) in the `BuildContext`,
|
||||||
|
the same shape as the existing `resolvedCubes` set.
|
||||||
|
|
||||||
|
## Execution model
|
||||||
|
|
||||||
|
Probes run during **resolution**, before anything has deployed. That has a
|
||||||
|
consequence worth stating plainly rather than discovering later:
|
||||||
|
|
||||||
|
When the plan reads *"user absent → schedule `user:add` → then B"*, B's
|
||||||
|
requirement was never verified. It was **assumed** met because a remedy was
|
||||||
|
scheduled. Every tool in this space stops there.
|
||||||
|
|
||||||
|
Since the probe already exists, the loop can be closed for nearly nothing:
|
||||||
|
|
||||||
|
> **Re-run the provider's probe after it deploys, and fail loudly if the
|
||||||
|
> requirement still is not met.**
|
||||||
|
|
||||||
|
`facts.py` then serves as a post-condition test as well as a precondition check.
|
||||||
|
This is the single most valuable property of the design — it is strictly more
|
||||||
|
than Ansible's `when:` offers — and it should be built in from the start rather
|
||||||
|
than added as a later refinement.
|
||||||
|
|
||||||
|
Sketch of the resolution change in `BuildContext.resolveCube`, between steps 3
|
||||||
|
(`before` hooks) and 4 (dependencies):
|
||||||
|
|
||||||
|
1. Collect `manifest.requires?.(currentVars)`.
|
||||||
|
2. Batch every unmet-so-far probe for this host into one pyinfra run.
|
||||||
|
3. Parse each result against the provider's `provides.schema`, then against the
|
||||||
|
consumer's `expect`.
|
||||||
|
4. For each failure, `resolveCube(req.cube, host, req.with)`.
|
||||||
|
5. After that provider's deploy call executes, re-probe and assert.
|
||||||
|
|
||||||
|
Step 5 does not fit the current shape: `deployCalls` are all built first and
|
||||||
|
executed later, so a post-condition needs the executor to call back into
|
||||||
|
probing. That is the one structurally invasive part of this proposal.
|
||||||
|
|
||||||
|
## Sharp edges
|
||||||
|
|
||||||
|
**`--print-only` purity.** A dry run touches no host today. Probing breaks that
|
||||||
|
outright. Needs an explicit answer — either a probe-free plan that renders
|
||||||
|
requirements as unevaluated, or an opt-in flag. Silently connecting during what
|
||||||
|
the user believes is a dry run is not acceptable.
|
||||||
|
|
||||||
|
**Secret leakage.** `server.Users` returns the **encrypted password hash** in
|
||||||
|
its `password` field. A probe result that reaches history, a plan dump, or a log
|
||||||
|
line is a credential leak. The existing `secrets` machinery is per-*variable*
|
||||||
|
and does not apply — facts need their own redaction rule. Easy to miss, hard to
|
||||||
|
take back.
|
||||||
|
|
||||||
|
**Sessions must never persist facts.** Facts are host state at a moment in time;
|
||||||
|
a replay must re-probe, never restore. Recording them in *history* for
|
||||||
|
diagnostics is fine and useful. This is also forced by `.refine()` being
|
||||||
|
unserialisable.
|
||||||
|
|
||||||
|
**Sudo.** Many useful facts require `_sudo`, and a probe runs before the deploy
|
||||||
|
has made any of its own auth decisions. Unresolved — worth a spike.
|
||||||
|
|
||||||
|
**Cycles.** `requires` introduces a second edge type into a graph that still has
|
||||||
|
no cycle detection (see *Known drift* in `CLAUDE.md`). Conditional edges make
|
||||||
|
"A requires B, B requires A under different params" considerably more reachable
|
||||||
|
than the current unconditional ones do.
|
||||||
|
|
||||||
|
**Naming.** `provides` / `requires` over the original `facts:`, because the
|
||||||
|
field names should say which side of the relationship they belong to.
|
||||||
|
|
||||||
|
## Measured vs. assumed
|
||||||
|
|
||||||
|
Measured against pyinfra 3.5.1, on this machine:
|
||||||
|
|
||||||
|
- `fact` subcommand writes JSON to stderr and exits 0 on fact failure
|
||||||
|
- a deploy file's stdout is uncontaminated by pyinfra output
|
||||||
|
- `--data` reaches a probe as `host.data.KEY`
|
||||||
|
- a deploy file with zero operations succeeds
|
||||||
|
- a custom `FactBase` in a sibling module resolves under `--chdir`
|
||||||
|
- an exception in a deploy file exits 1
|
||||||
|
- **an unreachable host exits 0 with empty stdout**
|
||||||
|
|
||||||
|
Assumed, not yet tested:
|
||||||
|
|
||||||
|
- batching several cubes' probes into one driver script works and is meaningfully
|
||||||
|
cheaper than N invocations — this is the load-bearing cost assumption and
|
||||||
|
should be the first thing spiked
|
||||||
|
- `host.get_fact()` accepts `_sudo` from within a probe function
|
||||||
|
- the post-condition re-probe can be threaded through `executeDeployCalls`
|
||||||
|
without unpicking the build-then-execute split
|
||||||
@@ -2,9 +2,14 @@
|
|||||||
|
|
||||||
Nopy supports two session file formats: **JSON** and **MJS** (ES Module JavaScript).
|
Nopy supports two session file formats: **JSON** and **MJS** (ES Module JavaScript).
|
||||||
|
|
||||||
|
The extension is what picks the loader, so a session file has to end in `.json`
|
||||||
|
or `.mjs`; anything else is refused by name. The `.nopysession.*` names used
|
||||||
|
throughout are the convention `listSessions()` looks for — `-s` and `-l` accept
|
||||||
|
any path you give them.
|
||||||
|
|
||||||
## Supported Formats
|
## Supported Formats
|
||||||
|
|
||||||
### JSON Format (`.session.json`)
|
### JSON Format (`.nopysession.json`)
|
||||||
|
|
||||||
Traditional JSON format for session files:
|
Traditional JSON format for session files:
|
||||||
|
|
||||||
@@ -35,7 +40,7 @@ Traditional JSON format for session files:
|
|||||||
- Cannot use dynamic values or computation
|
- Cannot use dynamic values or computation
|
||||||
- No code reuse or imports
|
- No code reuse or imports
|
||||||
|
|
||||||
### MJS Format (`.session.mjs`) - **Recommended**
|
### MJS Format (`.nopysession.mjs`) - **Recommended**
|
||||||
|
|
||||||
JavaScript module format with full ES Module support:
|
JavaScript module format with full ES Module support:
|
||||||
|
|
||||||
@@ -172,7 +177,7 @@ export const commonCubes = [
|
|||||||
];
|
];
|
||||||
```
|
```
|
||||||
|
|
||||||
**my-session.session.mjs:**
|
**my-session.nopysession.mjs:**
|
||||||
```javascript
|
```javascript
|
||||||
import { productionHosts, commonCubes } from './common-config.mjs';
|
import { productionHosts, commonCubes } from './common-config.mjs';
|
||||||
|
|
||||||
@@ -232,7 +237,7 @@ function generateSession(config) {
|
|||||||
};
|
};
|
||||||
|
|
||||||
const content = `export default ${JSON.stringify(session, null, 2)};`;
|
const content = `export default ${JSON.stringify(session, null, 2)};`;
|
||||||
fs.writeFileSync('generated.session.mjs', content);
|
fs.writeFileSync('generated.nopysession.mjs', content);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Generate from external configuration
|
// Generate from external configuration
|
||||||
@@ -255,10 +260,10 @@ Both formats are loaded the same way:
|
|||||||
import { loadSession } from '@bitsquare/nopy';
|
import { loadSession } from '@bitsquare/nopy';
|
||||||
|
|
||||||
// Load JSON
|
// Load JSON
|
||||||
const jsonSession = await loadSession('./my-session.session.json');
|
const jsonSession = await loadSession('./my-session.nopysession.json');
|
||||||
|
|
||||||
// Load MJS
|
// Load MJS
|
||||||
const mjsSession = await loadSession('./my-session.session.mjs');
|
const mjsSession = await loadSession('./my-session.nopysession.mjs');
|
||||||
```
|
```
|
||||||
|
|
||||||
The file extension determines which loader to use.
|
The file extension determines which loader to use.
|
||||||
@@ -267,7 +272,7 @@ The file extension determines which loader to use.
|
|||||||
|
|
||||||
To convert an existing JSON session to MJS:
|
To convert an existing JSON session to MJS:
|
||||||
|
|
||||||
1. Rename the file from `.session.json` to `.session.mjs`
|
1. Rename the file from `.nopysession.json` to `.nopysession.mjs`
|
||||||
2. Add `export default` before the configuration object
|
2. Add `export default` before the configuration object
|
||||||
3. Remove quotes from property keys (optional)
|
3. Remove quotes from property keys (optional)
|
||||||
4. Add comments and dynamic values as needed
|
4. Add comments and dynamic values as needed
|
||||||
@@ -304,11 +309,12 @@ Both formats must export/contain an object with this structure:
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
interface NopySession {
|
interface NopySession {
|
||||||
version: string; // Session format version
|
cubes: CubeSession[]; // Array of cube configurations — required
|
||||||
timestamp: string; // ISO timestamp
|
auth: AuthSession; // Authentication configuration — required
|
||||||
cubes: CubeSession[]; // Array of cube configurations
|
version?: string; // Session format version, currently "1.0.0"
|
||||||
hosts: string[]; // Target hosts
|
timestamp?: string; // ISO timestamp
|
||||||
auth: AuthSession; // Authentication configuration
|
name?: string; // One-line description
|
||||||
|
hosts?: string[]; // Target hosts
|
||||||
env?: Record<string, any>; // Global environment variables
|
env?: Record<string, any>; // Global environment variables
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -323,6 +329,17 @@ interface AuthSession {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Only `cubes` and `auth` are demanded of a session being *read* — the loader
|
||||||
|
requires what it cannot work without and nothing else, so the sessions in these
|
||||||
|
examples are all valid, and one written before `version` existed still loads. A
|
||||||
|
session nopy *writes* always carries `version`, `timestamp` and `name`; a
|
||||||
|
`version` this build does not recognise produces a warning on stderr and loads
|
||||||
|
anyway.
|
||||||
|
|
||||||
|
`method: 'ssh'` is the third value and the one no prompt produces: it means the
|
||||||
|
connector handles authentication and nopy supplies no credential. Every
|
||||||
|
`@vagrant/` and `@docker/` host gets it.
|
||||||
|
|
||||||
A session nopy *writes* holds, per cube, every value that cube ran with — what
|
A session nopy *writes* holds, per cube, every value that cube ran with — what
|
||||||
was typed, what came from `.nopyrc.json`, what a dependency supplied, and what
|
was typed, what came from `.nopyrc.json`, what a dependency supplied, and what
|
||||||
fell through to the schema's `.default()`. Two things are deliberately absent and
|
fell through to the schema's `.default()`. Two things are deliberately absent and
|
||||||
|
|||||||
@@ -1,7 +1,9 @@
|
|||||||
# Vagrant
|
# Vagrant
|
||||||
|
|
||||||
`vagrant ssh-config` to find the SSH port of the machine
|
A Vagrant box is the cheapest way to run a cube against a real machine you can
|
||||||
`vagrant status --machine-readable` will be executed by pyinfra to get information about available VMs
|
throw away afterwards.
|
||||||
|
|
||||||
|
## The Vagrantfile
|
||||||
|
|
||||||
```ruby
|
```ruby
|
||||||
|
|
||||||
@@ -19,3 +21,34 @@ Vagrant.configure("2") do |config|
|
|||||||
end
|
end
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Naming the machine to nopy
|
||||||
|
|
||||||
|
The host string is `@vagrant/<name>`, where `<name>` is what `config.vm.define`
|
||||||
|
declared — `nopytestvm` above. It is pyinfra's connector syntax, not nopy's, and
|
||||||
|
it is the same shape as `@docker/<container-or-image>`.
|
||||||
|
|
||||||
|
Two ways to get there. Either pick `vagrant` in the host prompt and answer the
|
||||||
|
follow-up with the machine name, which is what builds the string for you, or put
|
||||||
|
it in `.nopyrc.json` so it appears in the list directly:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"hosts": ["@vagrant/nopytestvm"],
|
||||||
|
"cubePackages": ["@bitsquare/nopy-cubes-core"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
pyinfra runs `vagrant status --machine-readable` to find the available machines
|
||||||
|
and `vagrant ssh-config` for the SSH port, so `vagrant` has to be on `PATH` and
|
||||||
|
the box has to be `up` before a deploy.
|
||||||
|
|
||||||
|
## Cleaning up
|
||||||
|
|
||||||
|
```sh
|
||||||
|
vagrant halt # stop it, keep the disk
|
||||||
|
vagrant destroy -f # delete it — the next `vagrant up` is a fresh box
|
||||||
|
```
|
||||||
|
|
||||||
|
`destroy` is the one to use between test runs of a cube that is not idempotent:
|
||||||
|
re-running against a half-configured box tests something other than the cube.
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
{
|
{
|
||||||
|
"version": "1.0.0",
|
||||||
"name": "Example Deployment Session",
|
"name": "Example Deployment Session",
|
||||||
|
"timestamp": "2026-07-30T09:15:00.000Z",
|
||||||
"cubes": [
|
"cubes": [
|
||||||
{
|
{
|
||||||
"key": "apt:essentials",
|
"key": "apt:essentials",
|
||||||
@@ -8,7 +10,7 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"hosts": ["@docker/nopy-test-ubuntu"],
|
"hosts": ["@docker/nopy-test-container"],
|
||||||
"env": {
|
"env": {
|
||||||
"KEY_DIR": "../../vault/tmp"
|
"KEY_DIR": "../../vault/tmp"
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@bitsquare/nopy",
|
"name": "@bitsquare/nopy",
|
||||||
"version": "0.5.0",
|
"version": "1.0.1",
|
||||||
"description": "A system to simplify pyinfra script management and execution.",
|
"description": "A system to simplify pyinfra script management and execution.",
|
||||||
"keywords": [
|
"keywords": [
|
||||||
"pyinfra",
|
"pyinfra",
|
||||||
|
|||||||
@@ -7,6 +7,8 @@ import type { Cube, CubeVariables, HookContext } from '@bitsquare/nopy-cubes';
|
|||||||
import { getLogger } from '@logtape/logtape';
|
import { getLogger } from '@logtape/logtape';
|
||||||
import type { Variables } from '../nopy.common.js';
|
import type { Variables } from '../nopy.common.js';
|
||||||
import type { NopyConfig } from '../nopy.config.js';
|
import type { NopyConfig } from '../nopy.config.js';
|
||||||
|
import { logConfigToFlags } from '../nopy.config.js';
|
||||||
|
import { NopyUsageError } from '../nopy.errors.js';
|
||||||
import type { DeployCall } from '../nopy.executor.js';
|
import type { DeployCall } from '../nopy.executor.js';
|
||||||
import { VariableAssignment } from '../nopy.prompts.js';
|
import { VariableAssignment } from '../nopy.prompts.js';
|
||||||
import type { CubeSession, NopySession } from '../nopy.session.js';
|
import type { CubeSession, NopySession } from '../nopy.session.js';
|
||||||
@@ -21,6 +23,18 @@ export class BuildContext {
|
|||||||
public readonly cubeSessions: CubeSession[] = [];
|
public readonly cubeSessions: CubeSession[] = [];
|
||||||
private readonly resolvedCubes = new Set<string>();
|
private readonly resolvedCubes = new Set<string>();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The (cube, host) pairs currently being resolved, innermost last.
|
||||||
|
*
|
||||||
|
* `resolvedCubes` cannot serve here: it is written by `buildDeployCall`, which
|
||||||
|
* runs *after* the recursive descent, so a cycle never reaches it — the two
|
||||||
|
* cubes just recurse until the stack overflows. It cannot be widened into a
|
||||||
|
* "seen" set either, because re-entering a cube with different `param`
|
||||||
|
* overrides is a legitimate thing for a dependency or a hook to do. What is
|
||||||
|
* never legitimate is re-entering one that has not finished.
|
||||||
|
*/
|
||||||
|
private readonly resolving: { cubeId: string; host: string }[] = [];
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
public readonly allCubes: Record<string, Cube>,
|
public readonly allCubes: Record<string, Cube>,
|
||||||
public readonly variables: Variables,
|
public readonly variables: Variables,
|
||||||
@@ -45,21 +59,33 @@ export class BuildContext {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Fails a non-interactive run that cannot fill a required variable.
|
* Fails a run that cannot fill a required variable.
|
||||||
*
|
*
|
||||||
* Without this the cube would be deployed with the key simply absent from
|
* Without this the cube would be deployed with the key simply absent from
|
||||||
* `--data`, and the deploy script would read `None` off `host.data`.
|
* `--data`, and the deploy script would read `None` off `host.data` — against
|
||||||
|
* the documented guarantee that every schema key reaches it.
|
||||||
|
*
|
||||||
|
* Runs on the interactive path too, not only under `--use-defaults`. A prompt
|
||||||
|
* is not proof of an answer: a terminal that misreports its size renders an
|
||||||
|
* empty form and submits `{}` without the user seeing a field, which is
|
||||||
|
* exactly how this was found.
|
||||||
*/
|
*/
|
||||||
private assertVariablesComplete(cube: Cube): void {
|
private assertVariablesComplete(cube: Cube): void {
|
||||||
const missing = this.missingRequired(cube);
|
const missing = this.missingRequired(cube);
|
||||||
if (missing.length === 0) return;
|
if (missing.length === 0) return;
|
||||||
|
|
||||||
const [one, them] =
|
const list = missing.join(', ');
|
||||||
missing.length === 1 ? ['has no default value', 'it'] : ['have no default values', 'them'];
|
const them = missing.length === 1 ? 'it' : 'them';
|
||||||
throw new Error(
|
const have = missing.length === 1 ? 'has no default value' : 'have no default values';
|
||||||
`Cube "${cube.id}" cannot run with --use-defaults: ${missing.join(', ')} ${one}. ` +
|
|
||||||
|
throw new NopyUsageError(
|
||||||
|
this.options.useDefaults
|
||||||
|
? `Cube "${cube.id}" cannot run with --use-defaults: ${list} ${have}. ` +
|
||||||
`Set ${them} under "env" in .nopyrc.json, pass ${them} from a dependency, ` +
|
`Set ${them} under "env" in .nopyrc.json, pass ${them} from a dependency, ` +
|
||||||
'or drop --use-defaults to be prompted.'
|
'or drop --use-defaults to be prompted.'
|
||||||
|
: `Cube "${cube.id}" is missing ${list}. Nothing supplied ${them} — the form may have ` +
|
||||||
|
`been submitted empty. Re-run and fill ${them} in, or set ${them} under "env" ` +
|
||||||
|
'in .nopyrc.json.'
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -81,23 +107,58 @@ export class BuildContext {
|
|||||||
if (gaps.length === 0) return;
|
if (gaps.length === 0) return;
|
||||||
|
|
||||||
if (this.options.useDefaults) {
|
if (this.options.useDefaults) {
|
||||||
throw new Error(
|
// A gap is only a gap if nothing outside the session filled it. `env` and
|
||||||
`Cube "${cube.id}" cannot be replayed with --use-defaults: ${gaps.join(', ')} ` +
|
// `param` both say deliberately what the value is, which is exactly what
|
||||||
'would have to be entered. Secrets are never recorded in a session. ' +
|
// the old message told the user to do — and then failed anyway.
|
||||||
'Replay without --use-defaults, or set the values under "env" in .nopyrc.json.'
|
//
|
||||||
|
// `default` is not accepted here. The session dropped the secret on
|
||||||
|
// purpose, so falling through to a manifest default would deploy a
|
||||||
|
// different credential than the run being replayed, without saying so.
|
||||||
|
const unsatisfied = gaps.filter((key) => {
|
||||||
|
const origin = this.variables.of(cube.id, key)?.origin;
|
||||||
|
return origin !== 'env' && origin !== 'param';
|
||||||
|
});
|
||||||
|
|
||||||
|
if (unsatisfied.length > 0) {
|
||||||
|
const them = unsatisfied.length === 1 ? 'it' : 'them';
|
||||||
|
const secret = unsatisfied.some((key) => cube.secrets.includes(key));
|
||||||
|
throw new NopyUsageError(
|
||||||
|
`Cube "${cube.id}" cannot be replayed with --use-defaults: ` +
|
||||||
|
`${unsatisfied.join(', ')} would have to be entered. ` +
|
||||||
|
(secret ? 'Secrets are never recorded in a session. ' : '') +
|
||||||
|
`Set ${them} under "env" in .nopyrc.json` +
|
||||||
|
(secret ? ' (a schema default is not accepted for a secret)' : '') +
|
||||||
|
`, pass ${them} from a dependency, or replay without --use-defaults.`
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
log.debug('Filling session gaps', { cubeId: cube.id, gaps });
|
log.debug('Filling session gaps', { cubeId: cube.id, gaps });
|
||||||
await VariableAssignment(cube, this.variables, { keys: gaps });
|
await VariableAssignment(cube, this.variables, { keys: gaps });
|
||||||
|
|
||||||
// A cancelled form leaves the run short of a value it cannot invent.
|
// A form that resolved is not a form that was answered — same check, and
|
||||||
const stillMissing = this.missingRequired(cube);
|
// the same reason for it, as the interactive path.
|
||||||
if (stillMissing.length > 0) {
|
this.assertVariablesComplete(cube);
|
||||||
throw new Error(
|
|
||||||
`Cube "${cube.id}" is missing ${stillMissing.join(', ')} and cannot be deployed.`
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fails a run whose dependency graph loops back on itself.
|
||||||
|
*
|
||||||
|
* Names the whole path rather than just the repeated cube: with dependencies
|
||||||
|
* declared dynamically — and hooks free to `exec` anything at all — the edge
|
||||||
|
* that closed the loop is rarely the one you would guess from the two ends.
|
||||||
|
*/
|
||||||
|
private assertNoCycle(cubeId: string, host: string): void {
|
||||||
|
const at = this.resolving.findIndex((f) => f.cubeId === cubeId && f.host === host);
|
||||||
|
if (at === -1) return;
|
||||||
|
|
||||||
|
const path = [...this.resolving.slice(at).map((f) => f.cubeId), cubeId].join(' → ');
|
||||||
|
throw new NopyUsageError(
|
||||||
|
`Circular dependency on ${host}: ${path}. ` +
|
||||||
|
'A cube cannot depend on itself, directly or through a chain — check the ' +
|
||||||
|
'`dependencies()` of each cube named, and any `before`/`after` hook that calls `exec`.'
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -110,15 +171,35 @@ export class BuildContext {
|
|||||||
): Promise<void> {
|
): Promise<void> {
|
||||||
const cube = this.allCubes[cubeId];
|
const cube = this.allCubes[cubeId];
|
||||||
if (!cube) {
|
if (!cube) {
|
||||||
throw new Error(`Cube not found: ${cubeId}`);
|
throw new NopyUsageError(`Cube not found: ${cubeId}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
log.debug('Resolving cube', { cubeId, host });
|
log.debug('Resolving cube', { cubeId, host });
|
||||||
|
|
||||||
// 1. Declare secrets, then assign overrides and defaults. Declaring first
|
this.assertNoCycle(cubeId, host);
|
||||||
// means even the config `env` seeded on the cube's first assignment is
|
this.resolving.push({ cubeId, host });
|
||||||
// already marked, so nothing reaches a session or a log unredacted.
|
try {
|
||||||
|
await this.visitCube(cube, host, overrides);
|
||||||
|
} finally {
|
||||||
|
this.resolving.pop();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The body of {@link resolveCube}, once the cube is known and the cycle guard
|
||||||
|
* has admitted it.
|
||||||
|
*/
|
||||||
|
private async visitCube(cube: Cube, host: string, overrides: CubeVariables): Promise<void> {
|
||||||
|
const cubeId = cube.id;
|
||||||
|
|
||||||
|
// 1. Declare secrets and schema, then assign overrides and defaults. Both
|
||||||
|
// declarations have to come first: the cube's first assignment is what
|
||||||
|
// seeds the config `env` onto it, and by then it must already be known
|
||||||
|
// which of those keys are secret (so nothing reaches a session or a log
|
||||||
|
// unredacted) and which the cube actually declares (so a secret it does
|
||||||
|
// not declare is never seeded at all).
|
||||||
this.variables.declareSecrets(cubeId, cube.secrets);
|
this.variables.declareSecrets(cubeId, cube.secrets);
|
||||||
|
this.variables.declareSchema(cubeId, cube.schemaKeys());
|
||||||
if (Object.keys(overrides).length > 0) {
|
if (Object.keys(overrides).length > 0) {
|
||||||
this.variables.assign(cubeId, 'param', overrides);
|
this.variables.assign(cubeId, 'param', overrides);
|
||||||
}
|
}
|
||||||
@@ -136,6 +217,7 @@ export class BuildContext {
|
|||||||
this.assertVariablesComplete(cube);
|
this.assertVariablesComplete(cube);
|
||||||
} else {
|
} else {
|
||||||
await VariableAssignment(cube, this.variables);
|
await VariableAssignment(cube, this.variables);
|
||||||
|
this.assertVariablesComplete(cube);
|
||||||
}
|
}
|
||||||
|
|
||||||
const currentVars = this.variables.get(cubeId);
|
const currentVars = this.variables.get(cubeId);
|
||||||
@@ -171,6 +253,14 @@ export class BuildContext {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Builds and stores a deployment call for a resolved cube
|
* Builds and stores a deployment call for a resolved cube
|
||||||
|
*
|
||||||
|
* `command` is a true argv — one array element per argument, nothing
|
||||||
|
* pre-quoted. The executor spawns it without a shell, so a value containing a
|
||||||
|
* space, a quote or a `$(…)` is passed through verbatim instead of being
|
||||||
|
* re-parsed. It used to be a list of fragments joined into one shell string,
|
||||||
|
* which meant any variable value was shell syntax: an SSH password with a `;`
|
||||||
|
* in it ran whatever followed. {@link maskCommand} is the only thing that
|
||||||
|
* turns this back into a string, for display, and quotes as it goes.
|
||||||
*/
|
*/
|
||||||
private buildDeployCall(cube: Cube, host: string): void {
|
private buildDeployCall(cube: Cube, host: string): void {
|
||||||
const cubeId = cube.id;
|
const cubeId = cube.id;
|
||||||
@@ -178,17 +268,17 @@ export class BuildContext {
|
|||||||
|
|
||||||
if (this.resolvedCubes.has(callKey)) return;
|
if (this.resolvedCubes.has(callKey)) return;
|
||||||
|
|
||||||
const parts: string[] = [];
|
const parts: string[] = [...logConfigToFlags(this.config.log)];
|
||||||
if (this.auth.method === 'password' && this.auth.username && this.auth.password) {
|
if (this.auth.method === 'password' && this.auth.username && this.auth.password) {
|
||||||
parts.push(`--user ${this.auth.username} --password ${this.auth.password}`);
|
parts.push('--user', this.auth.username, '--password', this.auth.password);
|
||||||
}
|
}
|
||||||
|
|
||||||
const cubeVars = this.variables.get(cubeId);
|
const cubeVars = this.variables.get(cubeId);
|
||||||
Object.entries(cubeVars).forEach(([key, value]) => {
|
Object.entries(cubeVars).forEach(([key, value]) => {
|
||||||
parts.push(`--data "${key}=${value}"`);
|
parts.push('--data', `${key}=${value}`);
|
||||||
});
|
});
|
||||||
|
|
||||||
parts.push(`--chdir ${cube.dir}`);
|
parts.push('--chdir', cube.dir);
|
||||||
parts.push(`${cube.dir}/${cube.deployScript}`);
|
parts.push(`${cube.dir}/${cube.deployScript}`);
|
||||||
|
|
||||||
const command = ['pyinfra', host, '-y', ...parts];
|
const command = ['pyinfra', host, '-y', ...parts];
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ export type { Assignment, Origin, TVariables, Value } from './nopy.common.js';
|
|||||||
// Variables
|
// Variables
|
||||||
export { MASK, Variable, Variables } from './nopy.common.js';
|
export { MASK, Variable, Variables } from './nopy.common.js';
|
||||||
export type {
|
export type {
|
||||||
|
CubePackageRef,
|
||||||
ExecutionConfig,
|
ExecutionConfig,
|
||||||
HistoryConfig,
|
HistoryConfig,
|
||||||
LogConfig,
|
LogConfig,
|
||||||
|
|||||||
@@ -8,6 +8,7 @@
|
|||||||
import { createRequire } from 'node:module';
|
import { createRequire } from 'node:module';
|
||||||
import { Command } from 'commander';
|
import { Command } from 'commander';
|
||||||
import { loadConfig } from './nopy.config.js';
|
import { loadConfig } from './nopy.config.js';
|
||||||
|
import { reportError } from './nopy.errors.js';
|
||||||
import { exitWithFarewell, installGracefulExit, isCancellation } from './nopy.exit.js';
|
import { exitWithFarewell, installGracefulExit, isCancellation } from './nopy.exit.js';
|
||||||
import {
|
import {
|
||||||
clearHistory,
|
clearHistory,
|
||||||
@@ -33,8 +34,8 @@ const { version, buildInfo } = createRequire(import.meta.url)('../package.json')
|
|||||||
const versionLabel = buildInfo?.commit ? `${version} (${buildInfo.commit})` : version;
|
const versionLabel = buildInfo?.commit ? `${version} (${buildInfo.commit})` : version;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Prints the update hint to stderr, so it never lands in `--json` output or in
|
* Prints the update hint to stderr, so it never lands in a `--print-only`
|
||||||
* a `--print-only` command list being piped somewhere.
|
* command list being piped somewhere.
|
||||||
*/
|
*/
|
||||||
async function printUpdateNotice(): Promise<void> {
|
async function printUpdateNotice(): Promise<void> {
|
||||||
const notice = await updateNotice({ currentVersion: version });
|
const notice = await updateNotice({ currentVersion: version });
|
||||||
@@ -67,6 +68,9 @@ Examples:
|
|||||||
$ nopy history List all saved sessions
|
$ nopy history List all saved sessions
|
||||||
$ nopy clear-history Clear session history
|
$ nopy clear-history Clear session history
|
||||||
|
|
||||||
|
Every flag above belongs to 'install', the default command — 'nopy -R' is
|
||||||
|
'nopy install -R'. Run 'nopy install --help' for the full list.
|
||||||
|
|
||||||
Session Replay:
|
Session Replay:
|
||||||
Sessions are automatically saved to history after each deployment.
|
Sessions are automatically saved to history after each deployment.
|
||||||
Use 'nopy history' to see available sessions and their IDs.
|
Use 'nopy history' to see available sessions and their IDs.
|
||||||
@@ -88,16 +92,24 @@ program
|
|||||||
.option('-n, --dry-run', 'Show execution plan without running')
|
.option('-n, --dry-run', 'Show execution plan without running')
|
||||||
.option('-P, --print-only', 'Print deploy commands and exit (no execution)')
|
.option('-P, --print-only', 'Print deploy commands and exit (no execution)')
|
||||||
.option('-c, --continue-on-error', 'Continue executing after failures')
|
.option('-c, --continue-on-error', 'Continue executing after failures')
|
||||||
.option('-j, --json', 'Output results as JSON')
|
// `--no-save-history`, not `--no-history`: Commander derives the destination
|
||||||
.option('--no-history', 'Do not save this session to history')
|
// from the long flag with the `no-` stripped, so `--no-history` wrote to the
|
||||||
|
// same `options.history` that `-H <id>` does. `-H abc --no-history` set it to
|
||||||
|
// `false`, the id was silently discarded, and the run fell through to a full
|
||||||
|
// interactive session instead of replaying anything.
|
||||||
|
.option('--no-save-history', 'Do not save this session to history')
|
||||||
.action(async (options) => {
|
.action(async (options) => {
|
||||||
await printUpdateNotice();
|
await printUpdateNotice();
|
||||||
|
|
||||||
// Loaded lazily so that --help/--version work outside a configured project.
|
try {
|
||||||
|
// Loaded lazily so that --help/--version work outside a configured
|
||||||
|
// project — and inside the try, so that "no .nopyrc.json here" is
|
||||||
|
// reported by `reportError` rather than escaping as an unhandled
|
||||||
|
// rejection and printing node's own stack. It is the likeliest first-run
|
||||||
|
// mistake there is.
|
||||||
const execConfig = loadConfig().execution ?? {};
|
const execConfig = loadConfig().execution ?? {};
|
||||||
const continueOnError = options.continueOnError ?? execConfig.continueOnError ?? false;
|
const continueOnError = options.continueOnError ?? execConfig.continueOnError ?? false;
|
||||||
|
|
||||||
try {
|
|
||||||
// Handle session replay
|
// Handle session replay
|
||||||
const loadSessionPath = options.loadSession;
|
const loadSessionPath = options.loadSession;
|
||||||
let sessionToReplay: { session: import('./nopy.session.js').NopySession } | undefined;
|
let sessionToReplay: { session: import('./nopy.session.js').NopySession } | undefined;
|
||||||
@@ -109,7 +121,9 @@ program
|
|||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
sessionToReplay = lastEntry;
|
sessionToReplay = lastEntry;
|
||||||
console.log(`Repeating: ${lastEntry.name}\n`);
|
// stderr, like everything nopy says about itself — `-R --print-only` has
|
||||||
|
// to leave stdout to the commands.
|
||||||
|
console.error(`Repeating: ${lastEntry.name}\n`);
|
||||||
} else if (options.history) {
|
} else if (options.history) {
|
||||||
const entry = getSessionById(options.history);
|
const entry = getSessionById(options.history);
|
||||||
if (!entry) {
|
if (!entry) {
|
||||||
@@ -118,7 +132,7 @@ program
|
|||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
sessionToReplay = entry;
|
sessionToReplay = entry;
|
||||||
console.log(`Running: ${entry.name}\n`);
|
console.error(`Running: ${entry.name}\n`);
|
||||||
}
|
}
|
||||||
|
|
||||||
const result = await nopy({
|
const result = await nopy({
|
||||||
@@ -130,8 +144,7 @@ program
|
|||||||
dryRun: options.dryRun,
|
dryRun: options.dryRun,
|
||||||
printOnly: options.printOnly,
|
printOnly: options.printOnly,
|
||||||
continueOnError,
|
continueOnError,
|
||||||
jsonOutput: options.json,
|
saveToHistory: options.saveHistory !== false && !options.dryRun,
|
||||||
saveToHistory: options.history !== false && !options.dryRun,
|
|
||||||
});
|
});
|
||||||
|
|
||||||
// Exit with error code if deployment failed
|
// Exit with error code if deployment failed
|
||||||
@@ -144,20 +157,7 @@ program
|
|||||||
// the process-level handler.
|
// the process-level handler.
|
||||||
if (isCancellation(error)) exitWithFarewell();
|
if (isCancellation(error)) exitWithFarewell();
|
||||||
|
|
||||||
if (options.json) {
|
reportError(error);
|
||||||
console.log(
|
|
||||||
JSON.stringify(
|
|
||||||
{
|
|
||||||
success: false,
|
|
||||||
error: error instanceof Error ? error.message : String(error),
|
|
||||||
},
|
|
||||||
null,
|
|
||||||
2
|
|
||||||
)
|
|
||||||
);
|
|
||||||
} else {
|
|
||||||
console.error('Error:', error instanceof Error ? error.message : error, error);
|
|
||||||
}
|
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -114,8 +114,21 @@ export class Variable {
|
|||||||
export class Variables {
|
export class Variables {
|
||||||
private readonly store: Record<string, Record<string, Variable>> = {};
|
private readonly store: Record<string, Record<string, Variable>> = {};
|
||||||
private readonly secrets: Record<string, Set<string>> = {};
|
private readonly secrets: Record<string, Set<string>> = {};
|
||||||
|
private readonly schemas: Record<string, Set<string>> = {};
|
||||||
|
private readonly globalSecrets: Set<string>;
|
||||||
|
|
||||||
constructor(readonly env: TVariables = {}) {}
|
/**
|
||||||
|
* @param env - the `env` block of the merged config, seeded onto every cube
|
||||||
|
* @param globalSecrets - every key *any* manifest declares secret, plus the
|
||||||
|
* config's own `secrets` list. Known up front, before the first cube
|
||||||
|
* resolves, so it does not depend on resolution order.
|
||||||
|
*/
|
||||||
|
constructor(
|
||||||
|
readonly env: TVariables = {},
|
||||||
|
globalSecrets: Iterable<string> = []
|
||||||
|
) {
|
||||||
|
this.globalSecrets = new Set(globalSecrets);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Marks keys of one cube as holding secrets.
|
* Marks keys of one cube as holding secrets.
|
||||||
@@ -132,8 +145,30 @@ export class Variables {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Records which keys a cube's schema declares.
|
||||||
|
*
|
||||||
|
* Only {@link bucket} reads this, and only to decide whether a globally
|
||||||
|
* declared secret may be seeded from `env`. Call it before anything assigns to
|
||||||
|
* the cube — it deliberately does not create the bucket itself, because
|
||||||
|
* creating it is what seeds `env`.
|
||||||
|
*/
|
||||||
|
declareSchema(cube: string, keys: readonly string[]): void {
|
||||||
|
this.schemas[cube] ??= new Set<string>();
|
||||||
|
const declared = this.schemas[cube];
|
||||||
|
for (const key of keys) declared.add(key);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether a key is sensitive for a cube.
|
||||||
|
*
|
||||||
|
* True for a key the cube's own manifest declared, and also for one *another*
|
||||||
|
* manifest declared: a value that is a secret anywhere is a secret everywhere
|
||||||
|
* it lands. That covers the manifest that lists `PASSWORD` in `schema` and
|
||||||
|
* forgets it in `secrets`.
|
||||||
|
*/
|
||||||
isSecret(cube: string, name: string): boolean {
|
isSecret(cube: string, name: string): boolean {
|
||||||
return this.secrets[cube]?.has(name) ?? false;
|
return (this.secrets[cube]?.has(name) ?? false) || this.globalSecrets.has(name);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Records values for one cube, all at the same origin. */
|
/** Records values for one cube, all at the same origin. */
|
||||||
@@ -176,6 +211,24 @@ export class Variables {
|
|||||||
return values;
|
return values;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The config's `env` block minus anything declared secret — what a session's
|
||||||
|
* own `env` records.
|
||||||
|
*
|
||||||
|
* A session copies `env` verbatim for reference, which quietly undid
|
||||||
|
* {@link persistable}: a credential declared in `.nopyrc.json` was kept out of
|
||||||
|
* every cube's `variables` and then written to the same file one key higher up,
|
||||||
|
* in plaintext, along with a copy in `.nopy.history.json`. Same rule as
|
||||||
|
* `persistable`, applied to the same file.
|
||||||
|
*/
|
||||||
|
persistableEnv(): TVariables {
|
||||||
|
const values: TVariables = {};
|
||||||
|
for (const [name, value] of Object.entries(this.env)) {
|
||||||
|
if (!this.globalSecrets.has(name)) values[name] = value;
|
||||||
|
}
|
||||||
|
return values;
|
||||||
|
}
|
||||||
|
|
||||||
private create(cube: string, name: string, first: Assignment): Variable {
|
private create(cube: string, name: string, first: Assignment): Variable {
|
||||||
const variable = new Variable(cube, name, first);
|
const variable = new Variable(cube, name, first);
|
||||||
variable.redacted = this.isSecret(cube, name);
|
variable.redacted = this.isSecret(cube, name);
|
||||||
@@ -189,6 +242,12 @@ export class Variables {
|
|||||||
* rather than a parallel bag merged in at read time. That is what lets it
|
* rather than a parallel bag merged in at read time. That is what lets it
|
||||||
* carry an origin, show up in the trace, and lose to a prompt by the same rule
|
* carry an origin, show up in the trace, and lose to a prompt by the same rule
|
||||||
* as everything else.
|
* as everything else.
|
||||||
|
*
|
||||||
|
* One key is held back: a **secret**, on a cube whose schema does not mention
|
||||||
|
* it. Broadcasting is otherwise load-bearing — a cube may legitimately read a
|
||||||
|
* key off `host.data` that it never declared — but a credential does not
|
||||||
|
* belong on the command line of every unrelated cube in the run, where nothing
|
||||||
|
* masks it because that cube never declared it sensitive.
|
||||||
*/
|
*/
|
||||||
private bucket(cube: string): Record<string, Variable> {
|
private bucket(cube: string): Record<string, Variable> {
|
||||||
const existing = this.store[cube];
|
const existing = this.store[cube];
|
||||||
@@ -197,6 +256,7 @@ export class Variables {
|
|||||||
const bucket: Record<string, Variable> = {};
|
const bucket: Record<string, Variable> = {};
|
||||||
this.store[cube] = bucket;
|
this.store[cube] = bucket;
|
||||||
for (const [name, value] of Object.entries(this.env)) {
|
for (const [name, value] of Object.entries(this.env)) {
|
||||||
|
if (this.globalSecrets.has(name) && !this.schemas[cube]?.has(name)) continue;
|
||||||
bucket[name] = this.create(cube, name, { value, origin: 'env' });
|
bucket[name] = this.create(cube, name, { value, origin: 'env' });
|
||||||
}
|
}
|
||||||
return bucket;
|
return bucket;
|
||||||
|
|||||||
@@ -6,6 +6,7 @@
|
|||||||
import fs from 'node:fs';
|
import fs from 'node:fs';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import type { TVariables } from './nopy.common.js';
|
import type { TVariables } from './nopy.common.js';
|
||||||
|
import { NopyUsageError } from './nopy.errors.js';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Log verbosity levels for pyinfra output
|
* Log verbosity levels for pyinfra output
|
||||||
@@ -102,6 +103,14 @@ export interface NopyConfig {
|
|||||||
cubePackages: CubePackageRef[];
|
cubePackages: CubePackageRef[];
|
||||||
/** Global environment variables */
|
/** Global environment variables */
|
||||||
env: TVariables;
|
env: TVariables;
|
||||||
|
/**
|
||||||
|
* `env` keys to treat as sensitive even though no manifest says so.
|
||||||
|
*
|
||||||
|
* A manifest's own `secrets` list already covers the cubes that declare the
|
||||||
|
* key. This is for the value no cube declares at all — a token a hook reads,
|
||||||
|
* say — which would otherwise be broadcast and printed in the clear.
|
||||||
|
*/
|
||||||
|
secrets?: string[];
|
||||||
/** Logging configuration */
|
/** Logging configuration */
|
||||||
log?: LogConfig;
|
log?: LogConfig;
|
||||||
/** Session history configuration */
|
/** Session history configuration */
|
||||||
@@ -317,7 +326,7 @@ export function loadConfig(): NopyConfig {
|
|||||||
const configPaths = findConfigFiles();
|
const configPaths = findConfigFiles();
|
||||||
|
|
||||||
if (configPaths.length === 0) {
|
if (configPaths.length === 0) {
|
||||||
throw new Error(
|
throw new NopyUsageError(
|
||||||
`No ${CONFIG_FILENAME} found. Create one in your project directory or any parent directory.`
|
`No ${CONFIG_FILENAME} found. Create one in your project directory or any parent directory.`
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
@@ -334,7 +343,7 @@ export function loadConfig(): NopyConfig {
|
|||||||
config = mergeConfigs(config, resolvedConfig);
|
config = mergeConfigs(config, resolvedConfig);
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const message = err instanceof Error ? err.message : String(err);
|
const message = err instanceof Error ? err.message : String(err);
|
||||||
throw new Error(`Failed to load config ${configPath}: ${message}`);
|
throw new NopyUsageError(`Failed to load config ${configPath}: ${message}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,50 @@
|
|||||||
|
/**
|
||||||
|
* The errors that are the user's to fix.
|
||||||
|
* @module nopy.errors
|
||||||
|
*/
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A run that failed for a reason the user can act on: no config file, a cube
|
||||||
|
* that does not exist, a required variable nothing supplied, a session file
|
||||||
|
* that will not load.
|
||||||
|
*
|
||||||
|
* The point is the *presentation*, not the control flow — nothing catches this
|
||||||
|
* to recover. A stack trace through `dist/` says nothing useful about a missing
|
||||||
|
* `.nopyrc.json`, and printing one invites the reader to look for a bug in nopy
|
||||||
|
* instead of a typo in their project. The CLI prints the message alone and keeps
|
||||||
|
* the stack behind `NOPY_DEBUG`.
|
||||||
|
*
|
||||||
|
* Mirrors keyman's `UsageError` deliberately: the two CLIs are kept in step on
|
||||||
|
* how they fail for the same reason their update modules are duplicated rather
|
||||||
|
* than shared.
|
||||||
|
*/
|
||||||
|
export class NopyUsageError extends Error {
|
||||||
|
constructor(message: string) {
|
||||||
|
super(message);
|
||||||
|
this.name = 'NopyUsageError';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reports a failed run in as many lines as it deserves.
|
||||||
|
*
|
||||||
|
* A {@link NopyUsageError} prints as one line: it is something the reader can
|
||||||
|
* fix, and three frames into `dist/` say nothing about a missing `.nopyrc.json`
|
||||||
|
* except that it looks like a crash in nopy rather than a typo in the project.
|
||||||
|
* Everything else keeps its stack, because an unexpected failure is exactly when
|
||||||
|
* one is worth having. `NOPY_DEBUG` forces it for both.
|
||||||
|
*
|
||||||
|
* Lives here rather than in `nopy.cli.ts` because the CLI is excluded from
|
||||||
|
* coverage — it is argv wiring, and this is a decision.
|
||||||
|
*/
|
||||||
|
export function reportError(error: unknown): void {
|
||||||
|
const message = error instanceof Error ? error.message : String(error);
|
||||||
|
|
||||||
|
console.error(`Error: ${message}`);
|
||||||
|
|
||||||
|
const stack = error instanceof Error ? error.stack : undefined;
|
||||||
|
const wanted = process.env.NOPY_DEBUG || !(error instanceof NopyUsageError);
|
||||||
|
|
||||||
|
if (wanted && stack) console.error(stack);
|
||||||
|
else if (!process.env.NOPY_DEBUG) console.error('Set NOPY_DEBUG=1 for the full stack trace.');
|
||||||
|
}
|
||||||
@@ -20,7 +20,11 @@ export interface DeployCall {
|
|||||||
host: string;
|
host: string;
|
||||||
/** Working directory for execution */
|
/** Working directory for execution */
|
||||||
cwd: string;
|
cwd: string;
|
||||||
/** Full command array */
|
/**
|
||||||
|
* The command as argv — `command[0]` is the executable, the rest are its
|
||||||
|
* arguments, one element each and none of them quoted. Nothing joins this to
|
||||||
|
* run it; {@link maskCommand} joins it to *show* it.
|
||||||
|
*/
|
||||||
command: string[];
|
command: string[];
|
||||||
/** Environment variables for the cube */
|
/** Environment variables for the cube */
|
||||||
env: Record<string, unknown>;
|
env: Record<string, unknown>;
|
||||||
@@ -30,6 +34,18 @@ export interface DeployCall {
|
|||||||
dependencies: DependencySpec[];
|
dependencies: DependencySpec[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One argv element, quoted for a POSIX shell.
|
||||||
|
*
|
||||||
|
* Display only — nothing is executed through a shell any more. The point is
|
||||||
|
* that what `--print-only` writes can be pasted into a terminal and mean the
|
||||||
|
* same thing it meant here.
|
||||||
|
*/
|
||||||
|
function shellQuote(arg: string): string {
|
||||||
|
if (arg.length > 0 && /^[\w@%+=:,./-]+$/.test(arg)) return arg;
|
||||||
|
return `'${arg.replace(/'/g, `'\\''`)}'`;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The command as it is safe to show: the SSH password, and every `--data KEY=…`
|
* The command as it is safe to show: the SSH password, and every `--data KEY=…`
|
||||||
* whose key the manifest declared a secret, have their values replaced.
|
* whose key the manifest declared a secret, have their values replaced.
|
||||||
@@ -37,18 +53,37 @@ export interface DeployCall {
|
|||||||
* pyinfra takes its data on the command line, so the real values have to be in
|
* pyinfra takes its data on the command line, so the real values have to be in
|
||||||
* `call.command` — this is the last point before they would reach a log, a
|
* `call.command` — this is the last point before they would reach a log, a
|
||||||
* `--print-only` dump or a dry-run plan.
|
* `--print-only` dump or a dry-run plan.
|
||||||
|
*
|
||||||
|
* Walks the argv rather than pattern-matching a joined string. The old version
|
||||||
|
* bounded a `--data` value on the closing quote the builder had written, which
|
||||||
|
* tied masking to a quoting convention two modules apart; a value containing a
|
||||||
|
* `"` broke it, and it was the same brittleness that made the command a shell
|
||||||
|
* injection in the first place. Position is not guessable.
|
||||||
*/
|
*/
|
||||||
export function maskCommand(call: DeployCall): string {
|
export function maskCommand(call: DeployCall): string {
|
||||||
const command = call.command.join(' ');
|
const secrets = new Set(call.secrets ?? []);
|
||||||
const quoteMeta = (key: string) => key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
const argv = call.command;
|
||||||
|
const out: string[] = [];
|
||||||
|
|
||||||
// The builder always quotes a `--data` value, so the closing quote bounds it.
|
for (let i = 0; i < argv.length; i++) {
|
||||||
const masked = (call.secrets ?? []).reduce(
|
const arg = argv[i];
|
||||||
(acc, key) => acc.replace(new RegExp(`(--data "${quoteMeta(key)}=)[^"]*"`, 'g'), `$1${MASK}"`),
|
const next = argv[i + 1];
|
||||||
command
|
|
||||||
);
|
|
||||||
|
|
||||||
return masked.replace(/(--password )\S+/g, `$1${MASK}`);
|
if (arg === '--password' && next !== undefined) {
|
||||||
|
out.push(arg, MASK);
|
||||||
|
i++;
|
||||||
|
} else if (arg === '--data' && next !== undefined) {
|
||||||
|
// Split on the first `=` only: the key cannot contain one, the value can.
|
||||||
|
const eq = next.indexOf('=');
|
||||||
|
const key = eq === -1 ? next : next.slice(0, eq);
|
||||||
|
out.push(arg, secrets.has(key) ? `${key}=${MASK}` : shellQuote(next));
|
||||||
|
i++;
|
||||||
|
} else {
|
||||||
|
out.push(shellQuote(arg));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return out.join(' ');
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -107,14 +142,18 @@ export interface ExecutionOptions {
|
|||||||
*/
|
*/
|
||||||
async function executeCall(call: DeployCall): Promise<ExecutionResult> {
|
async function executeCall(call: DeployCall): Promise<ExecutionResult> {
|
||||||
const startTime = Date.now();
|
const startTime = Date.now();
|
||||||
const commandStr = call.command.join(' ');
|
const [file, ...args] = call.command;
|
||||||
|
|
||||||
try {
|
try {
|
||||||
log.info(`Executing: ${call.cube} -> ${call.host}`);
|
log.info(`Executing: ${call.cube} -> ${call.host}`);
|
||||||
log.debug(`Command: ${maskCommand(call)}`);
|
log.debug(`Command: ${maskCommand(call)}`);
|
||||||
|
|
||||||
// Inherit stdio for live output
|
// No shell. `execa({shell: true})` used to run the whole command as one
|
||||||
await execa({ shell: true })(commandStr, {
|
// string, which made every variable value shell syntax — a password or a
|
||||||
|
// `--data` value containing `;`, a backtick or `$(…)` was executed rather
|
||||||
|
// than passed along. Spawning the argv directly removes the parse step;
|
||||||
|
// pyinfra is still found on PATH, and stdio stays inherited for live output.
|
||||||
|
await execa(file, args, {
|
||||||
cwd: call.cwd,
|
cwd: call.cwd,
|
||||||
stdio: 'inherit',
|
stdio: 'inherit',
|
||||||
});
|
});
|
||||||
@@ -143,20 +182,8 @@ async function executeCall(call: DeployCall): Promise<ExecutionResult> {
|
|||||||
* Outputs the execution plan without running (dry run)
|
* Outputs the execution plan without running (dry run)
|
||||||
*
|
*
|
||||||
* @param calls - Array of deployment calls
|
* @param calls - Array of deployment calls
|
||||||
* @param asJson - Output as JSON instead of text
|
|
||||||
*/
|
*/
|
||||||
export function outputExecutionPlan(calls: DeployCall[], asJson?: boolean): void {
|
export function outputExecutionPlan(calls: DeployCall[]): void {
|
||||||
if (asJson) {
|
|
||||||
const plan = calls.map((call) => ({
|
|
||||||
cube: call.cube,
|
|
||||||
host: call.host,
|
|
||||||
command: maskCommand(call),
|
|
||||||
variables: maskVariables(call),
|
|
||||||
}));
|
|
||||||
console.log(JSON.stringify({ plan }, null, 2));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('\n=== Execution Plan (Dry Run) ===\n');
|
console.log('\n=== Execution Plan (Dry Run) ===\n');
|
||||||
|
|
||||||
for (let i = 0; i < calls.length; i++) {
|
for (let i = 0; i < calls.length; i++) {
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ export function restoreTerminal(): void {
|
|||||||
* Says goodbye and leaves.
|
* Says goodbye and leaves.
|
||||||
*
|
*
|
||||||
* The farewell goes to **stderr**, for the same reason the update hint does:
|
* The farewell goes to **stderr**, for the same reason the update hint does:
|
||||||
* `--json` and `--print-only` stay machine-readable no matter how the run ends.
|
* `--print-only` stays machine-readable no matter how the run ends.
|
||||||
*
|
*
|
||||||
* `process.exit` rather than letting the loop drain, because the prompt that
|
* `process.exit` rather than letting the loop drain, because the prompt that
|
||||||
* was cancelled is still holding stdin — after the teardown above threw, its
|
* was cancelled is still holding stdin — after the teardown above threw, its
|
||||||
|
|||||||
@@ -5,7 +5,7 @@
|
|||||||
|
|
||||||
import fs from 'node:fs';
|
import fs from 'node:fs';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import type { NopySession } from './nopy.session.js';
|
import { describeSession, type NopySession } from './nopy.session.js';
|
||||||
|
|
||||||
/** Default number of sessions to keep in history */
|
/** Default number of sessions to keep in history */
|
||||||
export const DEFAULT_HISTORY_SIZE = 10;
|
export const DEFAULT_HISTORY_SIZE = 10;
|
||||||
@@ -72,35 +72,6 @@ export function saveHistory(history: SessionHistory): void {
|
|||||||
fs.writeFileSync(historyPath, JSON.stringify(history, null, 2), 'utf-8');
|
fs.writeFileSync(historyPath, JSON.stringify(history, null, 2), 'utf-8');
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Generates a history entry name from session data
|
|
||||||
*
|
|
||||||
* Format: "YYYY-MM-DD HH:mm - cube1, cube2, ..."
|
|
||||||
*
|
|
||||||
* @param session - The session to name
|
|
||||||
* @param timestamp - ISO timestamp
|
|
||||||
* @returns Human-readable name
|
|
||||||
*/
|
|
||||||
function generateEntryName(session: NopySession, timestamp: string): string {
|
|
||||||
const date = new Date(timestamp);
|
|
||||||
const dateStr = date.toLocaleString('en-US', {
|
|
||||||
year: 'numeric',
|
|
||||||
month: '2-digit',
|
|
||||||
day: '2-digit',
|
|
||||||
hour: '2-digit',
|
|
||||||
minute: '2-digit',
|
|
||||||
hour12: false,
|
|
||||||
});
|
|
||||||
|
|
||||||
const cubeNames = session.cubes.map((c) => c.key).join(', ');
|
|
||||||
const truncatedCubes = cubeNames.length > 40 ? `${cubeNames.substring(0, 37)}...` : cubeNames;
|
|
||||||
|
|
||||||
const hosts = session.hosts?.join(', ') || 'no host';
|
|
||||||
const truncatedHosts = hosts.length > 20 ? `${hosts.substring(0, 17)}...` : hosts;
|
|
||||||
|
|
||||||
return `${dateStr} - ${truncatedCubes} → ${truncatedHosts}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Generates a unique ID for a history entry
|
* Generates a unique ID for a history entry
|
||||||
*/
|
*/
|
||||||
@@ -124,7 +95,7 @@ export function addToHistory(
|
|||||||
|
|
||||||
const entry: HistoryEntry = {
|
const entry: HistoryEntry = {
|
||||||
id: generateEntryId(),
|
id: generateEntryId(),
|
||||||
name: generateEntryName(session, timestamp),
|
name: describeSession(session, timestamp),
|
||||||
timestamp,
|
timestamp,
|
||||||
session,
|
session,
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -15,11 +15,16 @@ import {
|
|||||||
summarizeResults,
|
summarizeResults,
|
||||||
} from './nopy.executor.js';
|
} from './nopy.executor.js';
|
||||||
import { addToHistory, DEFAULT_HISTORY_SIZE } from './nopy.history.js';
|
import { addToHistory, DEFAULT_HISTORY_SIZE } from './nopy.history.js';
|
||||||
import { type NopySession, saveSession } from './nopy.session.js';
|
import { describeSession, type NopySession, SESSION_VERSION, saveSession } from './nopy.session.js';
|
||||||
import { runWorkflow } from './nopy.workflow.js';
|
import { runWorkflow } from './nopy.workflow.js';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Configures the logtape logger for console output
|
* Configures the logtape logger for console output.
|
||||||
|
*
|
||||||
|
* **stderr**, deliberately. stdout carries the deploy commands and pyinfra's own
|
||||||
|
* output; everything nopy says about itself goes to stderr, so `--print-only`
|
||||||
|
* can be piped somewhere. The sink used to write to stdout and was held back
|
||||||
|
* only by `--json`, which never worked and is gone.
|
||||||
*/
|
*/
|
||||||
function configureLogtape(): void {
|
function configureLogtape(): void {
|
||||||
configure({
|
configure({
|
||||||
@@ -31,7 +36,7 @@ function configureLogtape(): void {
|
|||||||
if (typeof formatted === 'string') {
|
if (typeof formatted === 'string') {
|
||||||
const msg = formatted.replace(/\r?\n$/, '');
|
const msg = formatted.replace(/\r?\n$/, '');
|
||||||
const props = record.properties as Record<string, unknown>;
|
const props = record.properties as Record<string, unknown>;
|
||||||
console.log(msg, ...Object.values(props));
|
console.error(msg, ...Object.values(props));
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
})(),
|
})(),
|
||||||
@@ -55,7 +60,7 @@ function configureLogtape(): void {
|
|||||||
configureLogtape();
|
configureLogtape();
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Prints the active configuration summary
|
* Prints the active configuration summary — to stderr, see {@link configureLogtape}.
|
||||||
*/
|
*/
|
||||||
function printActiveConfig(
|
function printActiveConfig(
|
||||||
config: import('./nopy.config.js').NopyConfig,
|
config: import('./nopy.config.js').NopyConfig,
|
||||||
@@ -92,7 +97,7 @@ function printActiveConfig(
|
|||||||
}
|
}
|
||||||
|
|
||||||
lines.push('');
|
lines.push('');
|
||||||
console.log(lines.join('\n'));
|
console.error(lines.join('\n'));
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -107,7 +112,6 @@ export interface NopyOptions {
|
|||||||
dryRun?: boolean;
|
dryRun?: boolean;
|
||||||
printOnly?: boolean;
|
printOnly?: boolean;
|
||||||
continueOnError?: boolean;
|
continueOnError?: boolean;
|
||||||
jsonOutput?: boolean;
|
|
||||||
saveToHistory?: boolean;
|
saveToHistory?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -138,24 +142,30 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
|
|||||||
dryRun = false,
|
dryRun = false,
|
||||||
printOnly = false,
|
printOnly = false,
|
||||||
continueOnError = false,
|
continueOnError = false,
|
||||||
jsonOutput = false,
|
|
||||||
saveToHistory = true,
|
saveToHistory = true,
|
||||||
} = opts;
|
} = opts;
|
||||||
|
|
||||||
const log = getLogger(['nopy']);
|
const log = getLogger(['nopy']);
|
||||||
const config = loadConfig();
|
const config = loadConfig();
|
||||||
|
|
||||||
if (!jsonOutput && !replaySession && !loadSessionPath) {
|
if (!replaySession && !loadSessionPath) {
|
||||||
printActiveConfig(config, { continueOnError });
|
printActiveConfig(config, { continueOnError });
|
||||||
}
|
}
|
||||||
|
|
||||||
const { cubes, errors } = await loadCubes();
|
const { cubes, errors } = await loadCubes();
|
||||||
const variables = new Variables(config.env);
|
|
||||||
|
// Every key any manifest calls a secret, plus the config's own list. Computed
|
||||||
|
// before the first cube resolves, so which cube happens to run first cannot
|
||||||
|
// change whether a credential is treated as one.
|
||||||
|
const declaredSecrets = new Set([
|
||||||
|
...Object.values(cubes).flatMap((cube) => cube.secrets),
|
||||||
|
...(config.secrets ?? []),
|
||||||
|
]);
|
||||||
|
const variables = new Variables(config.env, declaredSecrets);
|
||||||
|
|
||||||
if (errors.length > 0) {
|
if (errors.length > 0) {
|
||||||
log.error('Errors found during cube loading:');
|
log.error('Errors found during cube loading:');
|
||||||
for (const error of errors) log.error(error);
|
for (const error of errors) log.error(error);
|
||||||
if (jsonOutput) console.log(JSON.stringify({ success: false, errors }, null, 2));
|
|
||||||
return undefined;
|
return undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -180,7 +190,7 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
useDefaults,
|
useDefaults,
|
||||||
isSessionReplay: workflow.isReplay,
|
isSessionReplay: workflow.replaySource !== undefined,
|
||||||
}
|
}
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -190,17 +200,43 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The default name needs the resolved cube list, which does not exist until
|
||||||
|
// the build has run — so it is filled in here rather than in `createSession`,
|
||||||
|
// and only when nothing supplied one. `version` sits before the spread so that
|
||||||
|
// a replayed session keeps whatever its file declared; a hand-written session
|
||||||
|
// that declared none of the three gets all three.
|
||||||
|
const timestamp = workflow.session.timestamp ?? new Date().toISOString();
|
||||||
const sessionForSaving: NopySession = {
|
const sessionForSaving: NopySession = {
|
||||||
|
version: SESSION_VERSION,
|
||||||
...workflow.session,
|
...workflow.session,
|
||||||
|
timestamp,
|
||||||
cubes: context.cubeSessions,
|
cubes: context.cubeSessions,
|
||||||
env: config.env,
|
// Not `config.env` — a declared secret in there would be written to the
|
||||||
|
// session file in plaintext, one key above the `variables` it was carefully
|
||||||
|
// kept out of.
|
||||||
|
env: variables.persistableEnv(),
|
||||||
};
|
};
|
||||||
|
sessionForSaving.name ??= describeSession(sessionForSaving, timestamp);
|
||||||
|
|
||||||
if (saveSessionPath && !workflow.isReplay) {
|
// Saved on a replay too: the resolved cube set is exactly what was asked for,
|
||||||
|
// and a session written from a replay is no less valid than one written from a
|
||||||
|
// fresh run. The old `!isReplay` guard made `nopy install -R -s out.json` exit
|
||||||
|
// 0 having written nothing.
|
||||||
|
if (saveSessionPath) {
|
||||||
saveSession(sessionForSaving, saveSessionPath);
|
saveSession(sessionForSaving, saveSessionPath);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (saveToHistory && !dryRun && !workflow.isReplay && context.deployCalls.length > 0) {
|
// A `-R`/`-H` replay is already in history and re-recording it would push the
|
||||||
|
// original out of the list. A `--load-session` run is not in history at all,
|
||||||
|
// so unless it is recorded here, `nopy history` reports nothing afterwards and
|
||||||
|
// `-R` has nothing to repeat.
|
||||||
|
const recordable = workflow.replaySource !== 'history';
|
||||||
|
|
||||||
|
// `--print-only` is excluded for the same reason `--dry-run` is: neither
|
||||||
|
// deployed anything, and history is what `-R` repeats. Recording a run that
|
||||||
|
// never happened made `nopy install -P` — the safe look-before-you-leap flag —
|
||||||
|
// silently displace the last real deployment at the head of the list.
|
||||||
|
if (saveToHistory && !dryRun && !printOnly && recordable && context.deployCalls.length > 0) {
|
||||||
const historySize = config.history?.maxSessions ?? DEFAULT_HISTORY_SIZE;
|
const historySize = config.history?.maxSessions ?? DEFAULT_HISTORY_SIZE;
|
||||||
if (config.history?.autoSave !== false) {
|
if (config.history?.autoSave !== false) {
|
||||||
addToHistory(sessionForSaving, historySize);
|
addToHistory(sessionForSaving, historySize);
|
||||||
@@ -224,10 +260,8 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
|
|||||||
dryRun,
|
dryRun,
|
||||||
continueOnError,
|
continueOnError,
|
||||||
onProgress: (result, completed, total) => {
|
onProgress: (result, completed, total) => {
|
||||||
if (!jsonOutput) {
|
|
||||||
const status = result.success ? '✓' : '✗';
|
const status = result.success ? '✓' : '✗';
|
||||||
log.info(`[${completed}/${total}] ${status} ${result.cube} -> ${result.host}`);
|
log.info(`[${completed}/${total}] ${status} ${result.cube} -> ${result.host}`);
|
||||||
}
|
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -17,6 +17,45 @@ interface CubeChoice {
|
|||||||
message: string;
|
message: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Floor for a terminal that reports a size no prompt could render into. */
|
||||||
|
const MIN_ROWS = 24;
|
||||||
|
const MIN_COLS = 80;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The window size to hand an enquirer prompt, never smaller than {@link MIN_ROWS}.
|
||||||
|
*
|
||||||
|
* Load-bearing, not cosmetic. enquirer derives how many choices are visible from
|
||||||
|
* its height, and `utils.height` (`lib/utils.js:80`) computes a sane fallback and
|
||||||
|
* then throws it away:
|
||||||
|
*
|
||||||
|
* ```js
|
||||||
|
* let rows = (stream && stream.rows) ? stream.rows : fallback;
|
||||||
|
* if (stream && typeof stream.getWindowSize === 'function') {
|
||||||
|
* rows = stream.getWindowSize()[1]; // unconditional
|
||||||
|
* }
|
||||||
|
* ```
|
||||||
|
*
|
||||||
|
* A TTY always has `getWindowSize`, so a terminal reporting 0 rows — some CI
|
||||||
|
* pseudo-terminals, `script -q`, an editor terminal mid-startup — yields
|
||||||
|
* `height: 0`, `Math.min(limit, 0)` choices, and a form that renders nothing and
|
||||||
|
* submits `{}`. Passing `rows` bypasses that: `prompt.js:396` reads
|
||||||
|
* `this.options.rows || utils.height(...)`, so the broken function never runs.
|
||||||
|
*
|
||||||
|
* Measured on a 0×0 pty: without this the four-field form returns `{}`; with it,
|
||||||
|
* every field. No effect on a terminal that reports its size honestly.
|
||||||
|
* enquirer 2.4.1 is its final release, so the bug is not going to be fixed
|
||||||
|
* upstream.
|
||||||
|
*/
|
||||||
|
function terminalSize(out: NodeJS.WriteStream = process.stdout): {
|
||||||
|
rows: number;
|
||||||
|
columns: number;
|
||||||
|
} {
|
||||||
|
return {
|
||||||
|
rows: Math.max(out.rows || 0, MIN_ROWS),
|
||||||
|
columns: Math.max(out.columns || 0, MIN_COLS),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Fuzzy-filters the cube list against what the user has typed so far.
|
* Fuzzy-filters the cube list against what the user has typed so far.
|
||||||
*
|
*
|
||||||
@@ -52,8 +91,8 @@ export async function CubeSelection(
|
|||||||
// Clear terminal and move cursor to top
|
// Clear terminal and move cursor to top
|
||||||
process.stdout.write('\x1B[2J\x1B[0f');
|
process.stdout.write('\x1B[2J\x1B[0f');
|
||||||
|
|
||||||
const terminalHeight = process.stdout.rows || 24;
|
const size = terminalSize();
|
||||||
const pageSize = Math.max(10, terminalHeight - 5);
|
const pageSize = Math.max(10, size.rows - 5);
|
||||||
|
|
||||||
console.log('\n Cube Selection\n');
|
console.log('\n Cube Selection\n');
|
||||||
console.log(' Type to filter • Space to select • Enter to confirm\n');
|
console.log(' Type to filter • Space to select • Enter to confirm\n');
|
||||||
@@ -65,14 +104,15 @@ export async function CubeSelection(
|
|||||||
multiple: true,
|
multiple: true,
|
||||||
choices: cubeChoices,
|
choices: cubeChoices,
|
||||||
suggest: suggestCubes,
|
suggest: suggestCubes,
|
||||||
|
...size,
|
||||||
});
|
});
|
||||||
|
|
||||||
try {
|
// Deliberately no catch. Swallowing a cancellation here used to return an
|
||||||
|
// empty selection, which is indistinguishable from "the user picked nothing"
|
||||||
|
// and let the run carry on to deploy zero cubes. Both ways out now travel:
|
||||||
|
// a cancellation to `isCancellation` at the CLI boundary, anything else as
|
||||||
|
// the failure it is.
|
||||||
return { selectedCubes: await prompt.run() };
|
return { selectedCubes: await prompt.run() };
|
||||||
} catch {
|
|
||||||
// User cancelled
|
|
||||||
return { selectedCubes: [] };
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function AuthSelection(useAuthKey?: boolean): Promise<{
|
export async function AuthSelection(useAuthKey?: boolean): Promise<{
|
||||||
@@ -117,6 +157,17 @@ export async function PasswordSelection(username: string): Promise<string> {
|
|||||||
return password;
|
return password;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Prompts for the deployment target, normalising the built-ins into the host
|
||||||
|
* strings pyinfra's connectors expect.
|
||||||
|
*
|
||||||
|
* The docker branch takes either identifier the connector accepts, and they
|
||||||
|
* mean very different things: a **container** name or id is mutated in place
|
||||||
|
* and left running, while an **image** reference makes pyinfra start a
|
||||||
|
* throwaway container, apply the deploy, commit the result as a new image and
|
||||||
|
* print its id. Only the connector can tell the two apart — it looks for a
|
||||||
|
* matching container first — so the prompt does not try to.
|
||||||
|
*/
|
||||||
export async function HostSelection(hosts: string[]): Promise<string> {
|
export async function HostSelection(hosts: string[]): Promise<string> {
|
||||||
const selectedHost = await inquirer.prompt([
|
const selectedHost = await inquirer.prompt([
|
||||||
{
|
{
|
||||||
@@ -140,13 +191,14 @@ export async function HostSelection(hosts: string[]): Promise<string> {
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
type: 'input',
|
type: 'input',
|
||||||
name: 'dockerContainer',
|
name: 'dockerTarget',
|
||||||
message: 'Specify docker container name:',
|
message: 'Specify docker container name/id, or an image to build from:',
|
||||||
when: (answers) => answers.host === 'runtime:docker',
|
when: (answers) => answers.host === 'docker',
|
||||||
|
validate: (value: string) => value.trim().length > 0 || 'Required',
|
||||||
},
|
},
|
||||||
]);
|
]);
|
||||||
if (selectedHost.host === 'vagrant') return `@vagrant/${selectedHost.vagrantVM}`;
|
if (selectedHost.host === 'vagrant') return `@vagrant/${selectedHost.vagrantVM}`;
|
||||||
if (selectedHost.host === 'runtime:docker') return `@docker/${selectedHost.dockerContainer}`;
|
if (selectedHost.host === 'docker') return `@docker/${selectedHost.dockerTarget.trim()}`;
|
||||||
return selectedHost.customHost ?? selectedHost.host;
|
return selectedHost.customHost ?? selectedHost.host;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -184,6 +236,43 @@ interface FormChoice {
|
|||||||
initial: string;
|
initial: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The label to prompt a schema field with: its `.describe()`, read through the
|
||||||
|
* wrappers that hide it, falling back to the bare key.
|
||||||
|
*
|
||||||
|
* In zod 4 a description lives in `z.globalRegistry` keyed by the schema
|
||||||
|
* *instance*, and `.default()` returns a new `ZodDefault` around the described
|
||||||
|
* type rather than mutating it. So the wrapper carries no description of its
|
||||||
|
* own, and the chaining order used to decide whether the label survived:
|
||||||
|
*
|
||||||
|
* ```
|
||||||
|
* z.boolean().describe('Update package cache').default(false) → 'UPDATE'
|
||||||
|
* z.boolean().default(false).describe('Update package cache') → the sentence
|
||||||
|
* ```
|
||||||
|
*
|
||||||
|
* 15 of the 22 core cubes were written the first way, so most prompts showed a
|
||||||
|
* bare key. Unwrapping makes the two orders equivalent, which is the answer that
|
||||||
|
* cannot regress — the alternative was to re-order every manifest and hope the
|
||||||
|
* next one written gets it right.
|
||||||
|
*
|
||||||
|
* Discriminates on {@link zodKind}, not `instanceof`, for the reason given
|
||||||
|
* there. Falling open here only costs an ugly label, but there is no reason to.
|
||||||
|
*/
|
||||||
|
function promptLabel(zodType: unknown, key: string): string {
|
||||||
|
let current = zodType;
|
||||||
|
|
||||||
|
while (current) {
|
||||||
|
const description = (current as { description?: string }).description;
|
||||||
|
if (description) return description;
|
||||||
|
|
||||||
|
const kind = zodKind(current);
|
||||||
|
if (kind !== 'default' && kind !== 'optional' && kind !== 'nullable') break;
|
||||||
|
current = zodInner(current);
|
||||||
|
}
|
||||||
|
|
||||||
|
return key;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Asks the user for a cube's variables and records the answers.
|
* Asks the user for a cube's variables and records the answers.
|
||||||
*
|
*
|
||||||
@@ -217,19 +306,22 @@ export async function VariableAssignment<S extends AnyObjectSchema>(
|
|||||||
|
|
||||||
if (Object.keys(variablesToConfigure).length === 0) return;
|
if (Object.keys(variablesToConfigure).length === 0) return;
|
||||||
|
|
||||||
const choices: FormChoice[] = Object.entries(variablesToConfigure).map(([key, value]) => {
|
const choices: FormChoice[] = Object.entries(variablesToConfigure).map(([key, value]) => ({
|
||||||
const zodType = schema[key];
|
name: key,
|
||||||
const description = zodType?.description || key;
|
message: promptLabel(schema[key], key),
|
||||||
return { name: key, message: description, initial: String(value ?? '') };
|
initial: String(value ?? ''),
|
||||||
});
|
}));
|
||||||
|
|
||||||
const form = new (Enquirer as any).Form({
|
const form = new (Enquirer as any).Form({
|
||||||
name: 'variables',
|
name: 'variables',
|
||||||
message: `[${cube.id}] ${cube.name}\n (↑↓ navigate, Enter to submit)`,
|
message: `[${cube.id}] ${cube.name}\n (↑↓ navigate, Enter to submit)`,
|
||||||
choices,
|
choices,
|
||||||
|
...terminalSize(),
|
||||||
});
|
});
|
||||||
|
|
||||||
try {
|
// Deliberately no catch — see `CubeSelection`. A cancelled form used to be
|
||||||
|
// swallowed here, leaving the cube short of values only the user could give
|
||||||
|
// and the run continuing as though the form had succeeded.
|
||||||
const result = await form.run();
|
const result = await form.run();
|
||||||
const coercedResult: Record<string, any> = {};
|
const coercedResult: Record<string, any> = {};
|
||||||
for (const [key, value] of Object.entries(result)) {
|
for (const [key, value] of Object.entries(result)) {
|
||||||
@@ -237,7 +329,4 @@ export async function VariableAssignment<S extends AnyObjectSchema>(
|
|||||||
coercedResult[key] = zodType ? coerceValue(value, zodType) : value;
|
coercedResult[key] = zodType ? coerceValue(value, zodType) : value;
|
||||||
}
|
}
|
||||||
variables.assign(cube.id, 'prompt', coercedResult);
|
variables.assign(cube.id, 'prompt', coercedResult);
|
||||||
} catch {
|
|
||||||
// User cancelled
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,6 +6,7 @@
|
|||||||
import fs from 'node:fs';
|
import fs from 'node:fs';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import type { TVariables } from './nopy.common.js';
|
import type { TVariables } from './nopy.common.js';
|
||||||
|
import { NopyUsageError } from './nopy.errors.js';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Primitive value types that can be stored in session variables
|
* Primitive value types that can be stored in session variables
|
||||||
@@ -31,7 +32,13 @@ export interface CubeSession {
|
|||||||
* Authentication configuration for a session
|
* Authentication configuration for a session
|
||||||
*/
|
*/
|
||||||
export interface AuthSession {
|
export interface AuthSession {
|
||||||
/** Authentication method */
|
/**
|
||||||
|
* Authentication method.
|
||||||
|
*
|
||||||
|
* `ssh` is not a third kind of credential — it means the connector owns
|
||||||
|
* authentication and nopy supplies none. It is what an `@vagrant/` or
|
||||||
|
* `@docker/` host gets, and nothing prompts for it.
|
||||||
|
*/
|
||||||
method: 'ssh-key' | 'password' | 'ssh';
|
method: 'ssh-key' | 'password' | 'ssh';
|
||||||
/** Username for authentication (password auth only) */
|
/** Username for authentication (password auth only) */
|
||||||
username?: string;
|
username?: string;
|
||||||
@@ -40,8 +47,20 @@ export interface AuthSession {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Complete session configuration
|
* Complete session configuration
|
||||||
|
*
|
||||||
|
* Everything but `cubes` and `auth` is optional, because a hand-written session
|
||||||
|
* is a first-class one — the loader requires exactly what it cannot work without.
|
||||||
|
* `version`, `timestamp` and `name` are stamped on every session nopy writes and
|
||||||
|
* never demanded of one it reads.
|
||||||
*/
|
*/
|
||||||
export interface NopySession {
|
export interface NopySession {
|
||||||
|
/**
|
||||||
|
* Format version of the file. Absent on every session written before this was
|
||||||
|
* stamped, and on most hand-written ones.
|
||||||
|
*/
|
||||||
|
version?: string;
|
||||||
|
/** ISO 8601 time the session was created */
|
||||||
|
timestamp?: string;
|
||||||
/** Optional session name */
|
/** Optional session name */
|
||||||
name?: string;
|
name?: string;
|
||||||
/** Array of cube configurations */
|
/** Array of cube configurations */
|
||||||
@@ -54,6 +73,40 @@ export interface NopySession {
|
|||||||
env?: TVariables;
|
env?: TVariables;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The format version stamped into every session nopy writes.
|
||||||
|
*
|
||||||
|
* There is one, and nothing yet reads it to decide anything — it exists so that
|
||||||
|
* a future change to the shape can tell an old file from a new one, which is
|
||||||
|
* impossible after the fact.
|
||||||
|
*/
|
||||||
|
export const SESSION_VERSION = '1.0.0';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A one-line description of a session: `YYYY-MM-DD HH:mm - cubes → hosts`.
|
||||||
|
*
|
||||||
|
* Shared with the history list, which is where the format comes from — the two
|
||||||
|
* name the same thing and there is no reason for them to disagree.
|
||||||
|
*/
|
||||||
|
export function describeSession(session: NopySession, timestamp: string): string {
|
||||||
|
const dateStr = new Date(timestamp).toLocaleString('en-US', {
|
||||||
|
year: 'numeric',
|
||||||
|
month: '2-digit',
|
||||||
|
day: '2-digit',
|
||||||
|
hour: '2-digit',
|
||||||
|
minute: '2-digit',
|
||||||
|
hour12: false,
|
||||||
|
});
|
||||||
|
|
||||||
|
const cubeNames = session.cubes.map((c) => c.key).join(', ');
|
||||||
|
const truncatedCubes = cubeNames.length > 40 ? `${cubeNames.substring(0, 37)}...` : cubeNames;
|
||||||
|
|
||||||
|
const hosts = session.hosts?.join(', ') || 'no host';
|
||||||
|
const truncatedHosts = hosts.length > 20 ? `${hosts.substring(0, 17)}...` : hosts;
|
||||||
|
|
||||||
|
return `${dateStr} - ${truncatedCubes} → ${truncatedHosts}`;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Saves a session to a JSON file
|
* Saves a session to a JSON file
|
||||||
*
|
*
|
||||||
@@ -129,7 +182,7 @@ function loadSessionFromJSON(filePath: string): NopySession {
|
|||||||
*/
|
*/
|
||||||
export async function loadSession(filePath: string): Promise<NopySession> {
|
export async function loadSession(filePath: string): Promise<NopySession> {
|
||||||
if (!fs.existsSync(filePath)) {
|
if (!fs.existsSync(filePath)) {
|
||||||
throw new Error(`Session file not found: ${filePath}`);
|
throw new NopyUsageError(`Session file not found: ${filePath}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
const ext = path.extname(filePath);
|
const ext = path.extname(filePath);
|
||||||
@@ -140,23 +193,44 @@ export async function loadSession(filePath: string): Promise<NopySession> {
|
|||||||
} else if (ext === '.json') {
|
} else if (ext === '.json') {
|
||||||
session = loadSessionFromJSON(filePath);
|
session = loadSessionFromJSON(filePath);
|
||||||
} else {
|
} else {
|
||||||
throw new Error(`Unsupported session file format: ${ext}. Use .json or .mjs`);
|
throw new NopyUsageError(`Unsupported session file format: ${ext}. Use .json or .mjs`);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Validate required fields
|
// Validate required fields
|
||||||
if (!session.cubes || !Array.isArray(session.cubes)) {
|
if (!session.cubes || !Array.isArray(session.cubes)) {
|
||||||
throw new Error('Invalid session format: missing or invalid "cubes" field');
|
throw new NopyUsageError('Invalid session format: missing or invalid "cubes" field');
|
||||||
}
|
}
|
||||||
if (session.hosts && !Array.isArray(session.hosts)) {
|
if (session.hosts && !Array.isArray(session.hosts)) {
|
||||||
throw new Error('Invalid session format: invalid "hosts" field');
|
throw new NopyUsageError('Invalid session format: invalid "hosts" field');
|
||||||
}
|
}
|
||||||
if (!session.auth) {
|
if (!session.auth) {
|
||||||
throw new Error('Invalid session format: missing "auth" field');
|
throw new NopyUsageError('Invalid session format: missing "auth" field');
|
||||||
|
}
|
||||||
|
|
||||||
|
// A version this build does not know is a warning, never a refusal: the file
|
||||||
|
// may well still load, and a session is often the only record of a deployment.
|
||||||
|
// A missing version says nothing at all — it predates the stamp.
|
||||||
|
if (session.version !== undefined && session.version !== SESSION_VERSION) {
|
||||||
|
console.error(
|
||||||
|
`Warning: session "${filePath}" declares version ${session.version}; ` +
|
||||||
|
`this build writes ${SESSION_VERSION}. Loading it anyway.`
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
return session;
|
return session;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Suffixes {@link listSessions} recognises.
|
||||||
|
*
|
||||||
|
* `.nopysession.*` is the documented name and the one the README's examples use;
|
||||||
|
* it was not matched at all, because `wild.nopysession.json` does not end in
|
||||||
|
* `.session.json` — the dot before `session` is part of the suffix. The shorter
|
||||||
|
* pair stays recognised: `saveSession` writes whatever path it is given, so
|
||||||
|
* files under the old name exist and there is no reason to stop finding them.
|
||||||
|
*/
|
||||||
|
const SESSION_SUFFIXES = ['.nopysession.json', '.nopysession.mjs', '.session.json', '.session.mjs'];
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Lists all session files in a directory
|
* Lists all session files in a directory
|
||||||
*
|
*
|
||||||
@@ -170,7 +244,7 @@ export function listSessions(dirPath: string = process.cwd()): string[] {
|
|||||||
|
|
||||||
const files = fs.readdirSync(dirPath);
|
const files = fs.readdirSync(dirPath);
|
||||||
return files
|
return files
|
||||||
.filter((file) => file.endsWith('.session.json') || file.endsWith('.session.mjs'))
|
.filter((file) => SESSION_SUFFIXES.some((suffix) => file.endsWith(suffix)))
|
||||||
.map((file) => path.join(dirPath, file));
|
.map((file) => path.join(dirPath, file));
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -186,8 +260,12 @@ export function createSession(params: {
|
|||||||
hosts: string[];
|
hosts: string[];
|
||||||
auth: AuthSession;
|
auth: AuthSession;
|
||||||
env?: TVariables;
|
env?: TVariables;
|
||||||
|
/** Overrides the creation time; for tests, and for re-stamping a replay. */
|
||||||
|
timestamp?: string;
|
||||||
}): NopySession {
|
}): NopySession {
|
||||||
return {
|
return {
|
||||||
|
version: SESSION_VERSION,
|
||||||
|
timestamp: params.timestamp ?? new Date().toISOString(),
|
||||||
name: params.name,
|
name: params.name,
|
||||||
cubes: params.cubes,
|
cubes: params.cubes,
|
||||||
hosts: params.hosts,
|
hosts: params.hosts,
|
||||||
|
|||||||
@@ -35,8 +35,18 @@ export interface WorkflowResult {
|
|||||||
username?: string;
|
username?: string;
|
||||||
/** Password if applicable */
|
/** Password if applicable */
|
||||||
password?: string;
|
password?: string;
|
||||||
/** Whether this is a session replay */
|
/**
|
||||||
isReplay: boolean;
|
* Where a replayed session came from, or `undefined` for a fresh interactive
|
||||||
|
* run.
|
||||||
|
*
|
||||||
|
* Was a boolean, which conflated two runs that need different treatment: a
|
||||||
|
* `-R`/`-H` replay is already in history and must not be recorded again, while
|
||||||
|
* a `--load-session` run is not in history at all — recording it is the only
|
||||||
|
* way `nopy history` and `-R` can see it afterwards. Everything that merely
|
||||||
|
* asks "am I replaying?" (reading values back off the session rather than
|
||||||
|
* prompting) takes `replaySource !== undefined`.
|
||||||
|
*/
|
||||||
|
replaySource?: 'file' | 'history';
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -83,7 +93,7 @@ export async function runInteractiveWorkflow(
|
|||||||
authMethod: authResult.authMethod,
|
authMethod: authResult.authMethod,
|
||||||
username: authResult.username,
|
username: authResult.username,
|
||||||
password: authResult.password,
|
password: authResult.password,
|
||||||
isReplay: false,
|
replaySource: undefined,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -141,7 +151,7 @@ export async function runReplayWorkflow(
|
|||||||
authMethod,
|
authMethod,
|
||||||
username,
|
username,
|
||||||
password,
|
password,
|
||||||
isReplay: true,
|
replaySource: 'file',
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -196,7 +206,7 @@ export async function runSessionReplayWorkflow(
|
|||||||
authMethod,
|
authMethod,
|
||||||
username,
|
username,
|
||||||
password,
|
password,
|
||||||
isReplay: true,
|
replaySource: 'history',
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -200,4 +200,67 @@ describe('Variables secrets', () => {
|
|||||||
expect(variables.persistable('cube-a')).toEqual({});
|
expect(variables.persistable('cube-a')).toEqual({});
|
||||||
expect(variables.persistable('cube-b')).toEqual({ PASSWORD: 'b' });
|
expect(variables.persistable('cube-b')).toEqual({ PASSWORD: 'b' });
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('excludes a declared secret from the env a session records', () => {
|
||||||
|
const variables = new Variables({ PASSWORD: 'hunter2', KEY_DIR: './vault' }, ['PASSWORD']);
|
||||||
|
|
||||||
|
expect(variables.persistableEnv()).toEqual({ KEY_DIR: './vault' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('records an env with no secrets in it whole', () => {
|
||||||
|
const variables = new Variables({ KEY_DIR: './vault' }, ['PASSWORD']);
|
||||||
|
|
||||||
|
expect(variables.persistableEnv()).toEqual({ KEY_DIR: './vault' });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('Variables globally declared secrets', () => {
|
||||||
|
/** `env` carrying a key that cube-a declares secret and cube-b knows nothing of. */
|
||||||
|
const withLeakyEnv = () => {
|
||||||
|
const variables = new Variables({ PASSWORD: 'wildpass123', KEY_DIR: '/vault' }, ['PASSWORD']);
|
||||||
|
variables.declareSecrets('cube-a', ['PASSWORD']);
|
||||||
|
variables.declareSchema('cube-a', ['USER', 'PASSWORD']);
|
||||||
|
variables.declareSchema('cube-b', ['PORT']);
|
||||||
|
return variables;
|
||||||
|
};
|
||||||
|
|
||||||
|
it('does not seed a secret onto a cube that does not declare it', () => {
|
||||||
|
const variables = withLeakyEnv();
|
||||||
|
variables.assign('cube-b', 'default', { PORT: 22 });
|
||||||
|
|
||||||
|
expect(variables.get('cube-b')).not.toHaveProperty('PASSWORD');
|
||||||
|
expect(variables.of('cube-b', 'PASSWORD')).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('still seeds it onto a cube whose schema declares it', () => {
|
||||||
|
const variables = withLeakyEnv();
|
||||||
|
variables.assign('cube-a', 'default', {});
|
||||||
|
|
||||||
|
expect(variables.get('cube-a').PASSWORD).toBe('wildpass123');
|
||||||
|
expect(variables.of('cube-a', 'PASSWORD')?.origin).toBe('env');
|
||||||
|
expect(variables.persistable('cube-a')).not.toHaveProperty('PASSWORD');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps broadcasting an undeclared key that is not a secret', () => {
|
||||||
|
// ssh:keyman reads KEY_DIR off host.data without declaring it in its schema.
|
||||||
|
const variables = withLeakyEnv();
|
||||||
|
variables.assign('cube-b', 'default', {});
|
||||||
|
|
||||||
|
expect(variables.get('cube-b').KEY_DIR).toBe('/vault');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('redacts a global secret on a cube whose own manifest forgot to list it', () => {
|
||||||
|
const variables = new Variables({}, ['PASSWORD']);
|
||||||
|
variables.assign('cube-b', 'prompt', { PASSWORD: 'typed' });
|
||||||
|
|
||||||
|
expect(variables.of('cube-b', 'PASSWORD')?.redacted).toBe(true);
|
||||||
|
expect(variables.persistable('cube-b')).toEqual({});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('treats a cube that declared no schema as declaring nothing', () => {
|
||||||
|
const variables = new Variables({ PASSWORD: 'p' }, ['PASSWORD']);
|
||||||
|
variables.assign('cube-z', 'default', {});
|
||||||
|
|
||||||
|
expect(variables.get('cube-z')).toEqual({});
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import { z } from 'zod';
|
|||||||
import { BuildContext } from '../src/cubes/dependencies.js';
|
import { BuildContext } from '../src/cubes/dependencies.js';
|
||||||
import { Variables } from '../src/nopy.common.js';
|
import { Variables } from '../src/nopy.common.js';
|
||||||
import type { NopyConfig } from '../src/nopy.config.js';
|
import type { NopyConfig } from '../src/nopy.config.js';
|
||||||
|
import { NopyUsageError } from '../src/nopy.errors.js';
|
||||||
import type { NopySession } from '../src/nopy.session.js';
|
import type { NopySession } from '../src/nopy.session.js';
|
||||||
|
|
||||||
vi.mock('../src/nopy.prompts.js', async () => {
|
vi.mock('../src/nopy.prompts.js', async () => {
|
||||||
@@ -61,6 +62,143 @@ describe('BuildContext error handling', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('BuildContext cycle detection', () => {
|
||||||
|
const depCube = (id: string, deps: string[]) =>
|
||||||
|
new Cube(
|
||||||
|
Manifest.create({ id, name: id, schema: z.object({}), dependencies: () => deps }),
|
||||||
|
`/test/${id}`,
|
||||||
|
'deploy.py'
|
||||||
|
);
|
||||||
|
|
||||||
|
const build = (cubes: Cube[]) =>
|
||||||
|
new BuildContext(
|
||||||
|
Object.fromEntries(cubes.map((c) => [c.id, c])),
|
||||||
|
new Variables(),
|
||||||
|
session(),
|
||||||
|
config,
|
||||||
|
{ method: 'ssh' },
|
||||||
|
{ useDefaults: true }
|
||||||
|
);
|
||||||
|
|
||||||
|
it('rejects a cube that depends on itself', async () => {
|
||||||
|
const context = build([depCube('cube-a', ['cube-a'])]);
|
||||||
|
|
||||||
|
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
||||||
|
'Circular dependency on host1: cube-a → cube-a'
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('names the whole path of a longer loop', async () => {
|
||||||
|
const context = build([
|
||||||
|
depCube('cube-a', ['cube-b']),
|
||||||
|
depCube('cube-b', ['cube-c']),
|
||||||
|
depCube('cube-c', ['cube-a']),
|
||||||
|
]);
|
||||||
|
|
||||||
|
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
||||||
|
'Circular dependency on host1: cube-a → cube-b → cube-c → cube-a'
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('reports the loop as usage rather than overflowing the stack', async () => {
|
||||||
|
const context = build([depCube('cube-a', ['cube-b']), depCube('cube-b', ['cube-a'])]);
|
||||||
|
|
||||||
|
// The point of the finding: before the guard this recursed until V8 gave up,
|
||||||
|
// and a RangeError names no cube and reads as a nopy crash rather than a
|
||||||
|
// manifest that says something impossible.
|
||||||
|
await expect(context.resolveCube('cube-a', 'host1')).rejects.toBeInstanceOf(NopyUsageError);
|
||||||
|
expect(context.deployCalls).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('catches a loop a hook closes rather than a dependency', async () => {
|
||||||
|
const a = new Cube(
|
||||||
|
Manifest.create({
|
||||||
|
id: 'cube-a',
|
||||||
|
name: 'A',
|
||||||
|
schema: z.object({}),
|
||||||
|
dependencies: () => ['cube-b'],
|
||||||
|
}),
|
||||||
|
'/test/cube-a',
|
||||||
|
'deploy.py'
|
||||||
|
);
|
||||||
|
const b = new Cube(
|
||||||
|
Manifest.create({
|
||||||
|
id: 'cube-b',
|
||||||
|
name: 'B',
|
||||||
|
schema: z.object({}),
|
||||||
|
before: [async (ctx) => ctx.exec('cube-a', {})],
|
||||||
|
}),
|
||||||
|
'/test/cube-b',
|
||||||
|
'deploy.py'
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(build([a, b]).resolveCube('cube-a', 'host1')).rejects.toThrow(
|
||||||
|
'Circular dependency on host1: cube-a → cube-b → cube-a'
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('allows a diamond, where the shared cube is entered twice but never nested', async () => {
|
||||||
|
const context = build([
|
||||||
|
depCube('top', ['left', 'right']),
|
||||||
|
depCube('left', ['shared']),
|
||||||
|
depCube('right', ['shared']),
|
||||||
|
depCube('shared', []),
|
||||||
|
]);
|
||||||
|
|
||||||
|
await context.resolveCube('top', 'host1');
|
||||||
|
|
||||||
|
expect(context.deployCalls.map((c) => c.cube)).toEqual(['shared', 'left', 'right', 'top']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('allows the same cube on a different host', async () => {
|
||||||
|
const context = build([depCube('cube-a', [])]);
|
||||||
|
|
||||||
|
await context.resolveCube('cube-a', 'host1');
|
||||||
|
await context.resolveCube('cube-a', 'host2');
|
||||||
|
|
||||||
|
expect(context.deployCalls.map((c) => c.host)).toEqual(['host1', 'host2']);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('BuildContext log configuration', () => {
|
||||||
|
const build = (log: NopyConfig['log']) =>
|
||||||
|
new BuildContext(
|
||||||
|
{ 'cube-a': testCube('cube-a') },
|
||||||
|
new Variables(),
|
||||||
|
session(),
|
||||||
|
{ env: {}, log } as NopyConfig,
|
||||||
|
{ method: 'ssh' },
|
||||||
|
{ useDefaults: true }
|
||||||
|
);
|
||||||
|
|
||||||
|
it('passes the configured verbosity and debug flags to pyinfra', async () => {
|
||||||
|
const context = build({ verbosity: 'verbose', debug: true });
|
||||||
|
|
||||||
|
await context.resolveCube('cube-a', 'host1');
|
||||||
|
|
||||||
|
expect(context.deployCalls[0].command.slice(0, 5)).toEqual([
|
||||||
|
'pyinfra',
|
||||||
|
'host1',
|
||||||
|
'-y',
|
||||||
|
'-vv',
|
||||||
|
'--debug',
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('adds nothing when no log config is set', async () => {
|
||||||
|
const context = build(undefined);
|
||||||
|
|
||||||
|
await context.resolveCube('cube-a', 'host1');
|
||||||
|
|
||||||
|
expect(context.deployCalls[0].command.slice(0, 4)).toEqual([
|
||||||
|
'pyinfra',
|
||||||
|
'host1',
|
||||||
|
'-y',
|
||||||
|
'--chdir',
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('BuildContext session replay', () => {
|
describe('BuildContext session replay', () => {
|
||||||
it('takes variables from the session instead of prompting', async () => {
|
it('takes variables from the session instead of prompting', async () => {
|
||||||
const cube = testCube('cube-a', z.object({ PORT: z.string().default('3000') }));
|
const cube = testCube('cube-a', z.object({ PORT: z.string().default('3000') }));
|
||||||
@@ -129,10 +267,15 @@ describe('BuildContext session replay', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
describe('BuildContext replay gaps', () => {
|
describe('BuildContext replay gaps', () => {
|
||||||
const replay = (cube: Cube, recorded: Record<string, string> = {}, options = {}) =>
|
const replay = (
|
||||||
|
cube: Cube,
|
||||||
|
recorded: Record<string, string> = {},
|
||||||
|
options = {},
|
||||||
|
variables = new Variables()
|
||||||
|
) =>
|
||||||
new BuildContext(
|
new BuildContext(
|
||||||
{ [cube.id]: cube },
|
{ [cube.id]: cube },
|
||||||
new Variables(),
|
variables,
|
||||||
session([{ key: cube.id, variables: recorded }]),
|
session([{ key: cube.id, variables: recorded }]),
|
||||||
config,
|
config,
|
||||||
{ method: 'ssh' },
|
{ method: 'ssh' },
|
||||||
@@ -175,16 +318,18 @@ describe('BuildContext replay gaps', () => {
|
|||||||
expect(VariableAssignment).not.toHaveBeenCalled();
|
expect(VariableAssignment).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('refuses to deploy when the form was cancelled', async () => {
|
it('refuses to deploy when the form came back empty', async () => {
|
||||||
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
|
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
|
||||||
// The real VariableAssignment swallows a cancelled form, so the gap check
|
// A form that resolves is not proof of an answer: enquirer renders
|
||||||
// has to run again afterwards or the cube ships without the variable.
|
// `Math.min(limit, height)` fields, so a terminal misreporting its height
|
||||||
|
// submits `{}` without the user having seen a question. The gap check has
|
||||||
|
// to run again afterwards or the cube ships without the variable.
|
||||||
vi.mocked(VariableAssignment).mockResolvedValue(undefined);
|
vi.mocked(VariableAssignment).mockResolvedValue(undefined);
|
||||||
|
|
||||||
const context = replay(cube);
|
const context = replay(cube);
|
||||||
|
|
||||||
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
||||||
'Cube "cube-a" is missing SSID and cannot be deployed.'
|
'Cube "cube-a" is missing SSID. Nothing supplied it'
|
||||||
);
|
);
|
||||||
expect(context.deployCalls).toHaveLength(0);
|
expect(context.deployCalls).toHaveLength(0);
|
||||||
});
|
});
|
||||||
@@ -194,10 +339,36 @@ describe('BuildContext replay gaps', () => {
|
|||||||
|
|
||||||
const context = replay(cube, {}, { useDefaults: true });
|
const context = replay(cube, {}, { useDefaults: true });
|
||||||
|
|
||||||
|
// A schema default is deliberately not good enough for a secret: it would
|
||||||
|
// deploy a different credential than the run being replayed.
|
||||||
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
||||||
/cannot be replayed with --use-defaults: PASSWORD/
|
/cannot be replayed with --use-defaults: PASSWORD would have to be entered\..*not accepted for a secret/s
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('accepts a secret supplied through config env under --use-defaults', async () => {
|
||||||
|
const cube = secretCube('cube-a', z.object({ PASSWORD: z.string().default('changeme') }));
|
||||||
|
const context = replay(
|
||||||
|
cube,
|
||||||
|
{},
|
||||||
|
{ useDefaults: true },
|
||||||
|
new Variables({ PASSWORD: 'from-env' }, ['PASSWORD'])
|
||||||
|
);
|
||||||
|
|
||||||
|
await context.resolveCube('cube-a', 'host1');
|
||||||
|
|
||||||
|
expect(VariableAssignment).not.toHaveBeenCalled();
|
||||||
|
expect(context.deployCalls[0].env.PASSWORD).toBe('from-env');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('accepts a required variable a dependency passed under --use-defaults', async () => {
|
||||||
|
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
|
||||||
|
const context = replay(cube, {}, { useDefaults: true });
|
||||||
|
|
||||||
|
await context.resolveCube('cube-a', 'host1', { SSID: 'from-param' });
|
||||||
|
|
||||||
|
expect(context.deployCalls[0].env.SSID).toBe('from-param');
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('BuildContext session recording', () => {
|
describe('BuildContext session recording', () => {
|
||||||
@@ -240,6 +411,50 @@ describe('BuildContext session recording', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('BuildContext secret broadcast', () => {
|
||||||
|
// The field run put PASSWORD under `env` because the docs said to, and watched
|
||||||
|
// it appear unmasked on the command line of every cube that was not user:add.
|
||||||
|
const resolveBoth = async () => {
|
||||||
|
const declaring = secretCube('cube-a', z.object({ PASSWORD: z.string().default('changeme') }));
|
||||||
|
const innocent = testCube('cube-b', z.object({ PORT: z.string().default('22') }));
|
||||||
|
const context = new BuildContext(
|
||||||
|
{ 'cube-a': declaring, 'cube-b': innocent },
|
||||||
|
new Variables({ PASSWORD: 'wildpass123', KEY_DIR: '/vault' }, ['PASSWORD']),
|
||||||
|
session(),
|
||||||
|
config,
|
||||||
|
{ method: 'ssh' },
|
||||||
|
{ useDefaults: true }
|
||||||
|
);
|
||||||
|
|
||||||
|
await context.resolveCube('cube-a', 'host1');
|
||||||
|
await context.resolveCube('cube-b', 'host1');
|
||||||
|
return context;
|
||||||
|
};
|
||||||
|
|
||||||
|
it('never puts an env secret on a cube that does not declare it', async () => {
|
||||||
|
const context = await resolveBoth();
|
||||||
|
const [, forB] = context.deployCalls;
|
||||||
|
|
||||||
|
expect(forB.cube).toBe('cube-b');
|
||||||
|
expect(forB.env).not.toHaveProperty('PASSWORD');
|
||||||
|
expect(forB.command.join(' ')).not.toContain('wildpass123');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('still delivers it to the cube that declares it', async () => {
|
||||||
|
const context = await resolveBoth();
|
||||||
|
const [forA] = context.deployCalls;
|
||||||
|
|
||||||
|
expect(forA.env.PASSWORD).toBe('wildpass123');
|
||||||
|
expect(forA.secrets).toEqual(['PASSWORD']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('leaves an ordinary env key broadcast to both', async () => {
|
||||||
|
const context = await resolveBoth();
|
||||||
|
|
||||||
|
expect(context.deployCalls.map((call) => call.env.KEY_DIR)).toEqual(['/vault', '/vault']);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('BuildContext --use-defaults', () => {
|
describe('BuildContext --use-defaults', () => {
|
||||||
const withDefaults = (cube: Cube, variables = new Variables(), cfg = config) =>
|
const withDefaults = (cube: Cube, variables = new Variables(), cfg = config) =>
|
||||||
new BuildContext(
|
new BuildContext(
|
||||||
@@ -267,7 +482,7 @@ describe('BuildContext --use-defaults', () => {
|
|||||||
|
|
||||||
await context.resolveCube('cube-a', 'host1');
|
await context.resolveCube('cube-a', 'host1');
|
||||||
|
|
||||||
expect(context.deployCalls[0].command.join(' ')).toContain('--data "PORT=8080"');
|
expect(context.deployCalls[0].command).toContain('PORT=8080');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('refuses to run a cube whose variable nothing can supply', async () => {
|
it('refuses to run a cube whose variable nothing can supply', async () => {
|
||||||
@@ -333,6 +548,38 @@ describe('BuildContext --use-defaults', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('BuildContext interactive completeness', () => {
|
||||||
|
const interactive = (cube: Cube, variables = new Variables()) =>
|
||||||
|
new BuildContext({ [cube.id]: cube }, variables, session(), config, { method: 'ssh' });
|
||||||
|
|
||||||
|
it('refuses to deploy when the form submitted nothing', async () => {
|
||||||
|
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
|
||||||
|
// What a 0-row terminal does: the form renders no fields, the user sees no
|
||||||
|
// question, enquirer resolves `{}` and the run used to carry on and deploy
|
||||||
|
// the cube with SSID simply absent from `--data`.
|
||||||
|
vi.mocked(VariableAssignment).mockResolvedValue(undefined);
|
||||||
|
|
||||||
|
const context = interactive(cube);
|
||||||
|
|
||||||
|
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
||||||
|
/Cube "cube-a" is missing SSID\. Nothing supplied it/
|
||||||
|
);
|
||||||
|
expect(context.deployCalls).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('deploys when the form answered', async () => {
|
||||||
|
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
|
||||||
|
vi.mocked(VariableAssignment).mockImplementation(async (_cube, variables) => {
|
||||||
|
variables.assign('cube-a', 'prompt', { SSID: 'typed' });
|
||||||
|
});
|
||||||
|
|
||||||
|
const context = interactive(cube);
|
||||||
|
await context.resolveCube('cube-a', 'host1');
|
||||||
|
|
||||||
|
expect(context.deployCalls[0].env.SSID).toBe('typed');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('BuildContext command construction', () => {
|
describe('BuildContext command construction', () => {
|
||||||
const build = (auth: { method: string; username?: string; password?: string }) => {
|
const build = (auth: { method: string; username?: string; password?: string }) => {
|
||||||
const context = new BuildContext(
|
const context = new BuildContext(
|
||||||
@@ -376,14 +623,30 @@ describe('BuildContext command construction', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
await context.resolveCube('cube-a', 'host1');
|
await context.resolveCube('cube-a', 'host1');
|
||||||
const command = context.deployCalls[0].command.join(' ');
|
const command = context.deployCalls[0].command;
|
||||||
|
|
||||||
expect(command).toContain('--data "PORT=3000"');
|
// argv, not a shell string: each flag and its value are separate elements,
|
||||||
expect(command).toContain('--chdir /test/cube-a');
|
// and nothing is pre-quoted.
|
||||||
|
expect(command).toContain('PORT=3000');
|
||||||
|
expect(command.join(' ')).toContain('--data PORT=3000');
|
||||||
|
expect(command.join(' ')).toContain('--chdir /test/cube-a');
|
||||||
expect(command).toContain('/test/cube-a/deploy.py');
|
expect(command).toContain('/test/cube-a/deploy.py');
|
||||||
expect(context.deployCalls[0].cwd).toBe('/test/cube-a');
|
expect(context.deployCalls[0].cwd).toBe('/test/cube-a');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('keeps a value with shell metacharacters in one argv element', async () => {
|
||||||
|
// The whole point of dropping `shell: true`. Joined and handed to a shell,
|
||||||
|
// this value would have run `id` and swallowed the rest of the command.
|
||||||
|
const cube = testCube('cube-a', z.object({ MOTD: z.string().default('$(id); rm -rf /') }));
|
||||||
|
const context = new BuildContext({ 'cube-a': cube }, new Variables(), session(), config, {
|
||||||
|
method: 'ssh',
|
||||||
|
});
|
||||||
|
|
||||||
|
await context.resolveCube('cube-a', 'host1');
|
||||||
|
|
||||||
|
expect(context.deployCalls[0].command).toContain('MOTD=$(id); rm -rf /');
|
||||||
|
});
|
||||||
|
|
||||||
it('builds a separate call per host but records the cube session once', async () => {
|
it('builds a separate call per host but records the cube session once', async () => {
|
||||||
const context = new BuildContext(
|
const context = new BuildContext(
|
||||||
{ 'cube-a': testCube('cube-a') },
|
{ 'cube-a': testCube('cube-a') },
|
||||||
|
|||||||
@@ -125,3 +125,53 @@ describe('BuildContext.resolveCube', () => {
|
|||||||
expect(context.deployCalls.map((c) => c.cube)).toEqual(['cube-a', 'cube-b', 'cube-c']);
|
expect(context.deployCalls.map((c) => c.cube)).toEqual(['cube-a', 'cube-b', 'cube-c']);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('deploy order across several selected cubes', () => {
|
||||||
|
// `nopy.main.ts` walks `workflow.selectedCubes` and calls `resolveCube` once
|
||||||
|
// per entry, so the order that list arrives in is the order the loop visits.
|
||||||
|
// Emission is post-order, though, so a declared edge is honoured whichever way
|
||||||
|
// round the two cubes were listed — the recursion *is* the topological sort,
|
||||||
|
// and these pin that rather than leaving it to be inferred from the one-root
|
||||||
|
// cases above.
|
||||||
|
async function resolveAll(cubes: Record<string, Cube>, selected: string[]): Promise<string[]> {
|
||||||
|
const context = new BuildContext(
|
||||||
|
cubes,
|
||||||
|
new Variables(),
|
||||||
|
{ cubes: [] } as any,
|
||||||
|
{
|
||||||
|
env: {},
|
||||||
|
} as any,
|
||||||
|
{ method: 'ssh' }
|
||||||
|
);
|
||||||
|
|
||||||
|
for (const id of selected) await context.resolveCube(id, 'host1');
|
||||||
|
|
||||||
|
return context.deployCalls.map((c) => c.cube);
|
||||||
|
}
|
||||||
|
|
||||||
|
it('emits a dependency first even when it is selected last', async () => {
|
||||||
|
const cubes = {
|
||||||
|
'cube-a': createTestCube('cube-a'),
|
||||||
|
'cube-b': createTestCube('cube-b', () => ['cube-a']),
|
||||||
|
};
|
||||||
|
|
||||||
|
// The list order is the inversion of the dependency: b depends on a, and a
|
||||||
|
// is named after it. Resolving b still drags a in ahead of itself, and the
|
||||||
|
// second visit is deduped rather than re-emitted at the tail.
|
||||||
|
expect(await resolveAll(cubes, ['cube-b', 'cube-a'])).toEqual(['cube-a', 'cube-b']);
|
||||||
|
expect(await resolveAll(cubes, ['cube-a', 'cube-b'])).toEqual(['cube-a', 'cube-b']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('interleaves an unrelated cube by list order and nothing else', async () => {
|
||||||
|
// With no edge between them there is nothing to sort on, so `cube-z` lands
|
||||||
|
// where the list put it. That is the whole of what selection order decides.
|
||||||
|
const cubes = {
|
||||||
|
'cube-a': createTestCube('cube-a'),
|
||||||
|
'cube-b': createTestCube('cube-b', () => ['cube-a']),
|
||||||
|
'cube-z': createTestCube('cube-z'),
|
||||||
|
};
|
||||||
|
|
||||||
|
expect(await resolveAll(cubes, ['cube-z', 'cube-b'])).toEqual(['cube-z', 'cube-a', 'cube-b']);
|
||||||
|
expect(await resolveAll(cubes, ['cube-b', 'cube-z'])).toEqual(['cube-a', 'cube-b', 'cube-z']);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
/**
|
||||||
|
* Tests for nopy.errors — how a failed run is presented.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
|
import { NopyUsageError, reportError } from '../src/nopy.errors.js';
|
||||||
|
|
||||||
|
let out: ReturnType<typeof vi.spyOn>;
|
||||||
|
let err: ReturnType<typeof vi.spyOn>;
|
||||||
|
|
||||||
|
/** Everything written to stderr by the last call, as one string. */
|
||||||
|
const stderr = () => err.mock.calls.map((call) => String(call[0])).join('\n');
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
out = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||||
|
err = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||||
|
delete process.env.NOPY_DEBUG;
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
vi.restoreAllMocks();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('reportError', () => {
|
||||||
|
it('prints a usage error as one line and points at the debug switch', () => {
|
||||||
|
reportError(new NopyUsageError('No .nopyrc.json found'));
|
||||||
|
|
||||||
|
expect(stderr()).toContain('Error: No .nopyrc.json found');
|
||||||
|
expect(stderr()).not.toContain('nopy.errors');
|
||||||
|
expect(stderr()).toContain('NOPY_DEBUG=1');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps the stack for anything unexpected', () => {
|
||||||
|
reportError(new TypeError('cannot read properties of undefined'));
|
||||||
|
|
||||||
|
expect(stderr()).toContain('Error: cannot read properties of undefined');
|
||||||
|
expect(stderr()).toContain('TypeError: cannot read properties of undefined\n at ');
|
||||||
|
expect(stderr()).not.toContain('NOPY_DEBUG=1');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('prints the stack of a usage error under NOPY_DEBUG', () => {
|
||||||
|
process.env.NOPY_DEBUG = '1';
|
||||||
|
|
||||||
|
reportError(new NopyUsageError('No .nopyrc.json found'));
|
||||||
|
|
||||||
|
expect(stderr()).toContain('NopyUsageError: No .nopyrc.json found\n at ');
|
||||||
|
expect(stderr()).not.toContain('NOPY_DEBUG=1 for');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('reports a thrown non-error', () => {
|
||||||
|
reportError('just a string');
|
||||||
|
|
||||||
|
expect(stderr()).toContain('Error: just a string');
|
||||||
|
expect(stderr()).toContain('NOPY_DEBUG=1');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('says nothing on stdout', () => {
|
||||||
|
reportError(new NopyUsageError('No .nopyrc.json found'));
|
||||||
|
|
||||||
|
expect(out).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
/**
|
/**
|
||||||
* Tests for the executeDeployCalls path of nopy.executor.
|
* Tests for the executeDeployCalls path of nopy.executor.
|
||||||
*
|
*
|
||||||
* execa is mocked so no pyinfra process is ever spawned. Note the shape:
|
* execa is mocked so no pyinfra process is ever spawned. The module calls
|
||||||
* the module calls execa({ shell: true })(command, opts), so the mock is a
|
* `execa(file, args, opts)` directly — no shell, so no factory call to unwrap
|
||||||
* factory returning the runner.
|
* as there was while it went through `execa({ shell: true })`.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
@@ -11,7 +11,7 @@ import { beforeEach, describe, expect, it, vi } from 'vitest';
|
|||||||
const runner = vi.fn();
|
const runner = vi.fn();
|
||||||
|
|
||||||
vi.mock('execa', () => ({
|
vi.mock('execa', () => ({
|
||||||
execa: vi.fn(() => runner),
|
execa: vi.fn((...args: unknown[]) => runner(...args)),
|
||||||
}));
|
}));
|
||||||
|
|
||||||
import { execa } from 'execa';
|
import { execa } from 'execa';
|
||||||
@@ -50,16 +50,27 @@ describe('executeDeployCalls', () => {
|
|||||||
logSpy.mockRestore();
|
logSpy.mockRestore();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('runs the joined command in the call cwd with inherited stdio', async () => {
|
it('spawns the argv directly in the call cwd with inherited stdio', async () => {
|
||||||
await executeDeployCalls([call('cube-a')]);
|
await executeDeployCalls([call('cube-a')]);
|
||||||
|
|
||||||
expect(execa).toHaveBeenCalledWith({ shell: true });
|
expect(execa).toHaveBeenCalledWith('pyinfra', ['web-1', '-y', 'cube-a.deploy.py'], {
|
||||||
expect(runner).toHaveBeenCalledWith('pyinfra web-1 -y cube-a.deploy.py', {
|
|
||||||
cwd: '/cubes/cube-a',
|
cwd: '/cubes/cube-a',
|
||||||
stdio: 'inherit',
|
stdio: 'inherit',
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('never asks execa for a shell', async () => {
|
||||||
|
// The regression that matters: with `shell: true` every `--data` value was
|
||||||
|
// shell syntax, so a password or a variable containing `;` or `$(…)` ran.
|
||||||
|
await executeDeployCalls([
|
||||||
|
{ ...call('cube-a'), command: ['pyinfra', 'web-1', '--data', 'MOTD=$(id); rm -rf /'] },
|
||||||
|
]);
|
||||||
|
|
||||||
|
const [, , options] = vi.mocked(execa).mock.calls[0] as unknown[];
|
||||||
|
expect(options).not.toHaveProperty('shell');
|
||||||
|
expect(vi.mocked(execa).mock.calls[0][1]).toContain('MOTD=$(id); rm -rf /');
|
||||||
|
});
|
||||||
|
|
||||||
it('reports success with a non-negative duration', async () => {
|
it('reports success with a non-negative duration', async () => {
|
||||||
const [result] = await executeDeployCalls([call('cube-a')]);
|
const [result] = await executeDeployCalls([call('cube-a')]);
|
||||||
|
|
||||||
|
|||||||
@@ -123,24 +123,10 @@ describe('outputExecutionPlan', () => {
|
|||||||
expect(output).toContain('host1');
|
expect(output).toContain('host1');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('outputs JSON format when requested', () => {
|
|
||||||
const calls = [createTestCall('cube-a', 'host1')];
|
|
||||||
|
|
||||||
outputExecutionPlan(calls, true);
|
|
||||||
|
|
||||||
expect(consoleLogSpy).toHaveBeenCalledTimes(1);
|
|
||||||
const output = consoleLogSpy.mock.calls[0][0];
|
|
||||||
const parsed = JSON.parse(output);
|
|
||||||
|
|
||||||
expect(parsed.plan).toHaveLength(1);
|
|
||||||
expect(parsed.plan[0].cube).toBe('cube-a');
|
|
||||||
expect(parsed.plan[0].host).toBe('host1');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('masks variables the manifest declared secret', () => {
|
it('masks variables the manifest declared secret', () => {
|
||||||
const call: DeployCall = {
|
const call: DeployCall = {
|
||||||
...createTestCall('cube-a', 'host1'),
|
...createTestCall('cube-a', 'host1'),
|
||||||
command: ['pyinfra', 'host1', '-y', '--data "PASSWORD=hunter2"', '--data "OTHER=visible"'],
|
command: ['pyinfra', 'host1', '-y', '--data', 'PASSWORD=hunter2', '--data', 'OTHER=visible'],
|
||||||
env: { PASSWORD: 'hunter2', OTHER: 'visible' },
|
env: { PASSWORD: 'hunter2', OTHER: 'visible' },
|
||||||
secrets: ['PASSWORD'],
|
secrets: ['PASSWORD'],
|
||||||
};
|
};
|
||||||
@@ -200,34 +186,47 @@ describe('maskCommand', () => {
|
|||||||
|
|
||||||
it('replaces the value of a declared secret', () => {
|
it('replaces the value of a declared secret', () => {
|
||||||
const masked = maskCommand(
|
const masked = maskCommand(
|
||||||
call(['pyinfra', 'host1', '--data "PASSWORD=hunter2"'], ['PASSWORD'])
|
call(['pyinfra', 'host1', '--data', 'PASSWORD=hunter2'], ['PASSWORD'])
|
||||||
);
|
);
|
||||||
|
|
||||||
expect(masked).toBe('pyinfra host1 --data "PASSWORD=********"');
|
expect(masked).toBe('pyinfra host1 --data PASSWORD=********');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('leaves other data alone', () => {
|
it('leaves other data alone', () => {
|
||||||
const masked = maskCommand(
|
const masked = maskCommand(
|
||||||
call(['--data "SSID=home"', '--data "PASSWORD=hunter2"'], ['PASSWORD'])
|
call(['--data', 'SSID=home', '--data', 'PASSWORD=hunter2'], ['PASSWORD'])
|
||||||
);
|
);
|
||||||
|
|
||||||
expect(masked).toBe('--data "SSID=home" --data "PASSWORD=********"');
|
expect(masked).toBe('--data SSID=home --data PASSWORD=********');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('masks a value containing spaces up to the closing quote', () => {
|
it('masks a value containing spaces', () => {
|
||||||
const masked = maskCommand(call(['--data "PASSWORD=two words"', '--chdir /x'], ['PASSWORD']));
|
const masked = maskCommand(
|
||||||
|
call(['--data', 'PASSWORD=two words', '--chdir', '/x'], ['PASSWORD'])
|
||||||
|
);
|
||||||
|
|
||||||
expect(masked).toBe('--data "PASSWORD=********" --chdir /x');
|
expect(masked).toBe('--data PASSWORD=******** --chdir /x');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('masks an empty secret value', () => {
|
it('masks an empty secret value', () => {
|
||||||
expect(maskCommand(call(['--data "PASSWORD="'], ['PASSWORD']))).toBe(
|
expect(maskCommand(call(['--data', 'PASSWORD='], ['PASSWORD']))).toBe(
|
||||||
'--data "PASSWORD=********"'
|
'--data PASSWORD=********'
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('masks a secret whose value contains a quote', () => {
|
||||||
|
// The old implementation bounded the value on the closing `"` the builder
|
||||||
|
// had written, so a value containing one leaked the rest of itself.
|
||||||
|
const masked = maskCommand(call(['--data', 'PASSWORD=he said "hi"'], ['PASSWORD']));
|
||||||
|
|
||||||
|
expect(masked).toBe('--data PASSWORD=********');
|
||||||
|
expect(masked).not.toContain('hi');
|
||||||
|
});
|
||||||
|
|
||||||
it('masks the ssh password whether or not the cube declares secrets', () => {
|
it('masks the ssh password whether or not the cube declares secrets', () => {
|
||||||
const masked = maskCommand(call(['pyinfra', 'host1', '--user bob --password s3cr3t', '-y']));
|
const masked = maskCommand(
|
||||||
|
call(['pyinfra', 'host1', '--user', 'bob', '--password', 's3cr3t', '-y'])
|
||||||
|
);
|
||||||
|
|
||||||
expect(masked).toBe('pyinfra host1 --user bob --password ******** -y');
|
expect(masked).toBe('pyinfra host1 --user bob --password ******** -y');
|
||||||
});
|
});
|
||||||
@@ -235,4 +234,28 @@ describe('maskCommand', () => {
|
|||||||
it('returns the command untouched when there is nothing to hide', () => {
|
it('returns the command untouched when there is nothing to hide', () => {
|
||||||
expect(maskCommand(call(['pyinfra', 'host1', '-y']))).toBe('pyinfra host1 -y');
|
expect(maskCommand(call(['pyinfra', 'host1', '-y']))).toBe('pyinfra host1 -y');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('quotes an argument a shell would otherwise re-parse', () => {
|
||||||
|
// Display only — nothing runs through a shell — but `--print-only` output is
|
||||||
|
// meant to be pasteable, so it has to survive the round trip.
|
||||||
|
const masked = maskCommand(call(['--data', 'MOTD=$(id); rm -rf /']));
|
||||||
|
|
||||||
|
expect(masked).toBe(`--data 'MOTD=$(id); rm -rf /'`);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('escapes an embedded single quote', () => {
|
||||||
|
expect(maskCommand(call(['--data', "NAME=o'brien"]))).toBe(`--data 'NAME=o'\\''brien'`);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('quotes an empty argument rather than dropping it', () => {
|
||||||
|
expect(maskCommand(call(['pyinfra', '']))).toBe("pyinfra ''");
|
||||||
|
});
|
||||||
|
|
||||||
|
it('leaves a trailing --password with no value alone', () => {
|
||||||
|
expect(maskCommand(call(['pyinfra', '--password']))).toBe('pyinfra --password');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('handles a --data argument with no equals sign', () => {
|
||||||
|
expect(maskCommand(call(['--data', 'BARE'], ['BARE']))).toBe('--data BARE=********');
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
+42
@@ -0,0 +1,42 @@
|
|||||||
|
/**
|
||||||
|
* Runs one real enquirer variable form and prints what it produced.
|
||||||
|
*
|
||||||
|
* Driven by `tests/prompts.pty.test.ts` under a pty of a chosen size. Nothing
|
||||||
|
* here is mocked — the point is the prompt library's own behaviour on a
|
||||||
|
* terminal that reports no size, which cannot be observed from inside a vitest
|
||||||
|
* worker because there is no TTY there to misreport.
|
||||||
|
*
|
||||||
|
* Prints one line, `NOPY_PROBE <json>`, holding the values the form assigned at
|
||||||
|
* the `prompt` origin. An empty object means the form submitted nothing.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { Cube, Manifest } from '@bitsquare/nopy-cubes';
|
||||||
|
import { z } from 'zod';
|
||||||
|
import { Variables } from '../../src/nopy.common.js';
|
||||||
|
import { VariableAssignment } from '../../src/nopy.prompts.js';
|
||||||
|
|
||||||
|
const KEYS = ['ALPHA', 'BETA'];
|
||||||
|
|
||||||
|
const cube = new Cube(
|
||||||
|
Manifest({
|
||||||
|
id: 'probe',
|
||||||
|
name: 'Zero-rows probe',
|
||||||
|
schema: z.object({
|
||||||
|
ALPHA: z.string().describe('First value').default(''),
|
||||||
|
BETA: z.string().describe('Second value').default(''),
|
||||||
|
}),
|
||||||
|
}),
|
||||||
|
'/cubes/probe',
|
||||||
|
'deploy.py'
|
||||||
|
);
|
||||||
|
|
||||||
|
const variables = new Variables();
|
||||||
|
await VariableAssignment(cube, variables);
|
||||||
|
|
||||||
|
const assigned: Record<string, unknown> = {};
|
||||||
|
for (const key of KEYS) {
|
||||||
|
const variable = variables.of('probe', key);
|
||||||
|
if (variable?.origin === 'prompt') assigned[key] = variable.value;
|
||||||
|
}
|
||||||
|
|
||||||
|
process.stdout.write(`\nNOPY_PROBE ${JSON.stringify(assigned)}\n`);
|
||||||
@@ -48,7 +48,12 @@ vi.mock('../src/cubes/index.js', () => ({ loadCubes }));
|
|||||||
vi.mock('../src/nopy.config.js', () => ({ loadConfig, getConfigPaths }));
|
vi.mock('../src/nopy.config.js', () => ({ loadConfig, getConfigPaths }));
|
||||||
vi.mock('../src/nopy.workflow.js', () => ({ runWorkflow }));
|
vi.mock('../src/nopy.workflow.js', () => ({ runWorkflow }));
|
||||||
vi.mock('../src/nopy.history.js', () => ({ addToHistory, DEFAULT_HISTORY_SIZE: 10 }));
|
vi.mock('../src/nopy.history.js', () => ({ addToHistory, DEFAULT_HISTORY_SIZE: 10 }));
|
||||||
vi.mock('../src/nopy.session.js', () => ({ saveSession }));
|
// Only the writer is a spy — `describeSession` and the version constant are pure
|
||||||
|
// and the assertions below are about what nopy() actually stamps.
|
||||||
|
vi.mock('../src/nopy.session.js', async (importOriginal) => ({
|
||||||
|
...(await importOriginal<typeof import('../src/nopy.session.js')>()),
|
||||||
|
saveSession,
|
||||||
|
}));
|
||||||
vi.mock('../src/cubes/dependencies.js', () => ({
|
vi.mock('../src/cubes/dependencies.js', () => ({
|
||||||
BuildContext: class {
|
BuildContext: class {
|
||||||
resolveCube = resolveCube;
|
resolveCube = resolveCube;
|
||||||
@@ -67,16 +72,17 @@ vi.mock('../src/nopy.executor.js', async (importOriginal) => {
|
|||||||
|
|
||||||
import { nopy } from '../src/nopy.main.js';
|
import { nopy } from '../src/nopy.main.js';
|
||||||
|
|
||||||
const session = (): NopySession =>
|
/**
|
||||||
({
|
* A session as bare as the loader will accept one — no `version`, `timestamp`
|
||||||
version: '1.0',
|
* or `name`, which is exactly what a hand-written file looks like and what
|
||||||
name: 'test',
|
* `nopy()` has to fill in.
|
||||||
createdAt: '2026-01-01T00:00:00.000Z',
|
*/
|
||||||
|
const session = (): NopySession => ({
|
||||||
cubes: [],
|
cubes: [],
|
||||||
hosts: ['web-1'],
|
hosts: ['web-1'],
|
||||||
auth: { method: 'ssh-key' },
|
auth: { method: 'ssh-key' },
|
||||||
env: {},
|
env: {},
|
||||||
}) as NopySession;
|
});
|
||||||
|
|
||||||
const call = (cube: string): DeployCall => ({
|
const call = (cube: string): DeployCall => ({
|
||||||
cube,
|
cube,
|
||||||
@@ -88,10 +94,12 @@ const call = (cube: string): DeployCall => ({
|
|||||||
});
|
});
|
||||||
|
|
||||||
let logSpy: ReturnType<typeof vi.spyOn>;
|
let logSpy: ReturnType<typeof vi.spyOn>;
|
||||||
|
let errSpy: ReturnType<typeof vi.spyOn>;
|
||||||
|
|
||||||
beforeEach(() => {
|
beforeEach(() => {
|
||||||
vi.clearAllMocks();
|
vi.clearAllMocks();
|
||||||
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
|
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||||
|
errSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||||
|
|
||||||
state.config = { hosts: ['web-1'], cubeDirs: [], cubePackages: [], env: {} };
|
state.config = { hosts: ['web-1'], cubeDirs: [], cubePackages: [], env: {} };
|
||||||
state.loadResult = { cubes: { 'cube-a': {} }, errors: [] };
|
state.loadResult = { cubes: { 'cube-a': {} }, errors: [] };
|
||||||
@@ -102,14 +110,19 @@ beforeEach(() => {
|
|||||||
session: session(),
|
session: session(),
|
||||||
selectedCubes: ['cube-a'],
|
selectedCubes: ['cube-a'],
|
||||||
authMethod: 'ssh-key',
|
authMethod: 'ssh-key',
|
||||||
isReplay: false,
|
replaySource: undefined,
|
||||||
});
|
});
|
||||||
executeDeployCalls.mockResolvedValue([
|
executeDeployCalls.mockResolvedValue([
|
||||||
{ cube: 'cube-a', host: 'web-1', success: true, duration: 10 },
|
{ cube: 'cube-a', host: 'web-1', success: true, duration: 10 },
|
||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
const output = () => logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
|
/**
|
||||||
|
* The two streams, kept apart on purpose: stdout carries the deploy commands
|
||||||
|
* and pyinfra's own output, everything nopy says about itself goes to stderr.
|
||||||
|
*/
|
||||||
|
const stdout = () => logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
|
||||||
|
const stderr = () => errSpy.mock.calls.map((c) => c.join(' ')).join('\n');
|
||||||
|
|
||||||
describe('nopy', () => {
|
describe('nopy', () => {
|
||||||
it('runs the happy path and reports success', async () => {
|
it('runs the happy path and reports success', async () => {
|
||||||
@@ -141,7 +154,7 @@ describe('nopy', () => {
|
|||||||
session: { ...session(), hosts: ['web-1', 'web-2'] },
|
session: { ...session(), hosts: ['web-1', 'web-2'] },
|
||||||
selectedCubes: ['cube-a', 'cube-b'],
|
selectedCubes: ['cube-a', 'cube-b'],
|
||||||
authMethod: 'ssh-key',
|
authMethod: 'ssh-key',
|
||||||
isReplay: false,
|
replaySource: undefined,
|
||||||
});
|
});
|
||||||
|
|
||||||
await nopy();
|
await nopy();
|
||||||
@@ -159,13 +172,13 @@ describe('nopy', () => {
|
|||||||
expect(runWorkflow).not.toHaveBeenCalled();
|
expect(runWorkflow).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('emits the errors as JSON when jsonOutput is set', async () => {
|
it('reports them on stderr', async () => {
|
||||||
state.loadResult = { cubes: {}, errors: ['bad manifest'] };
|
state.loadResult = { cubes: {}, errors: ['bad manifest'] };
|
||||||
|
|
||||||
await nopy({ jsonOutput: true });
|
await nopy();
|
||||||
|
|
||||||
const payload = JSON.parse(logSpy.mock.calls.at(-1)?.[0] as string);
|
expect(stderr()).toContain('bad manifest');
|
||||||
expect(payload).toEqual({ success: false, errors: ['bad manifest'] });
|
expect(stdout()).toBe('');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -180,7 +193,7 @@ describe('nopy', () => {
|
|||||||
|
|
||||||
await nopy({ continueOnError: true });
|
await nopy({ continueOnError: true });
|
||||||
|
|
||||||
const text = output();
|
const text = stderr();
|
||||||
expect(text).toContain('Configuration');
|
expect(text).toContain('Configuration');
|
||||||
expect(text).toContain('Hosts:');
|
expect(text).toContain('Hosts:');
|
||||||
expect(text).toContain('Cube dirs:');
|
expect(text).toContain('Cube dirs:');
|
||||||
@@ -198,7 +211,7 @@ describe('nopy', () => {
|
|||||||
|
|
||||||
await nopy();
|
await nopy();
|
||||||
|
|
||||||
const text = output();
|
const text = stderr();
|
||||||
expect(text).toContain('Configuration');
|
expect(text).toContain('Configuration');
|
||||||
expect(text).not.toContain('Hosts:');
|
expect(text).not.toContain('Hosts:');
|
||||||
expect(text).not.toContain('Cube dirs:');
|
expect(text).not.toContain('Cube dirs:');
|
||||||
@@ -215,25 +228,20 @@ describe('nopy', () => {
|
|||||||
|
|
||||||
await nopy();
|
await nopy();
|
||||||
|
|
||||||
const text = output();
|
const text = stderr();
|
||||||
expect(text).toContain('~/.nopyrc.json');
|
expect(text).toContain('~/.nopyrc.json');
|
||||||
expect(text).toContain('./.nopyrc.json');
|
expect(text).toContain('./.nopyrc.json');
|
||||||
expect(text).toContain('/etc/nopy/.nopyrc.json');
|
expect(text).toContain('/etc/nopy/.nopyrc.json');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('is suppressed for JSON output', async () => {
|
|
||||||
await nopy({ jsonOutput: true });
|
|
||||||
expect(output()).not.toContain('Configuration');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('is suppressed when replaying a session object', async () => {
|
it('is suppressed when replaying a session object', async () => {
|
||||||
await nopy({ replaySession: session() });
|
await nopy({ replaySession: session() });
|
||||||
expect(output()).not.toContain('Configuration');
|
expect(stderr()).not.toContain('Configuration');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('is suppressed when replaying a session file', async () => {
|
it('is suppressed when replaying a session file', async () => {
|
||||||
await nopy({ loadSession: '/tmp/s.json' });
|
await nopy({ loadSession: '/tmp/s.json' });
|
||||||
expect(output()).not.toContain('Configuration');
|
expect(stderr()).not.toContain('Configuration');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -247,17 +255,57 @@ describe('nopy', () => {
|
|||||||
expect(written.cubes).toEqual(state.cubeSessions);
|
expect(written.cubes).toEqual(state.cubeSessions);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('does not save a replayed session back to file', async () => {
|
it('leaves a declared secret out of the recorded env', async () => {
|
||||||
|
// The session's `env` is a copy of the config's, and used to be copied
|
||||||
|
// verbatim — writing to disk, in plaintext, the credential that was kept
|
||||||
|
// out of every cube's `variables` one key below.
|
||||||
|
state.config = {
|
||||||
|
...state.config,
|
||||||
|
env: { PASSWORD: 'hunter2', KEY_DIR: './vault' },
|
||||||
|
secrets: ['PASSWORD'],
|
||||||
|
};
|
||||||
|
|
||||||
|
await nopy({ saveSession: '/tmp/out.json' });
|
||||||
|
|
||||||
|
expect(saveSession.mock.calls[0][0].env).toEqual({ KEY_DIR: './vault' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('saves a replayed session too', async () => {
|
||||||
runWorkflow.mockResolvedValue({
|
runWorkflow.mockResolvedValue({
|
||||||
session: session(),
|
session: session(),
|
||||||
selectedCubes: ['cube-a'],
|
selectedCubes: ['cube-a'],
|
||||||
authMethod: 'ssh-key',
|
authMethod: 'ssh-key',
|
||||||
isReplay: true,
|
replaySource: 'history',
|
||||||
});
|
});
|
||||||
|
|
||||||
await nopy({ saveSession: '/tmp/out.json' });
|
await nopy({ saveSession: '/tmp/out.json' });
|
||||||
|
|
||||||
expect(saveSession).not.toHaveBeenCalled();
|
expect(saveSession).toHaveBeenCalledTimes(1);
|
||||||
|
expect(saveSession.mock.calls[0][1]).toBe('/tmp/out.json');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('stamps version, timestamp and a derived name', async () => {
|
||||||
|
await nopy({ saveSession: '/tmp/out.json' });
|
||||||
|
|
||||||
|
const [written] = saveSession.mock.calls[0];
|
||||||
|
expect(written.version).toBe('1.0.0');
|
||||||
|
expect(written.timestamp).toMatch(/^\d{4}-\d{2}-\d{2}T/);
|
||||||
|
expect(written.name).toContain('cube-a');
|
||||||
|
expect(written.name).toContain('web-1');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps the name and version a replayed session already carried', async () => {
|
||||||
|
runWorkflow.mockResolvedValue({
|
||||||
|
session: { ...session(), version: '0.9.0', name: 'hand-written', timestamp: 'then' },
|
||||||
|
selectedCubes: ['cube-a'],
|
||||||
|
authMethod: 'ssh-key',
|
||||||
|
replaySource: 'file',
|
||||||
|
});
|
||||||
|
|
||||||
|
await nopy({ saveSession: '/tmp/out.json' });
|
||||||
|
|
||||||
|
const [written] = saveSession.mock.calls[0];
|
||||||
|
expect(written).toMatchObject({ version: '0.9.0', name: 'hand-written', timestamp: 'then' });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('does not save when no path is given', async () => {
|
it('does not save when no path is given', async () => {
|
||||||
@@ -300,12 +348,12 @@ describe('nopy', () => {
|
|||||||
expect(addToHistory).not.toHaveBeenCalled();
|
expect(addToHistory).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('skips history for a replay', async () => {
|
it('skips history for a replay out of history', async () => {
|
||||||
runWorkflow.mockResolvedValue({
|
runWorkflow.mockResolvedValue({
|
||||||
session: session(),
|
session: session(),
|
||||||
selectedCubes: ['cube-a'],
|
selectedCubes: ['cube-a'],
|
||||||
authMethod: 'ssh-key',
|
authMethod: 'ssh-key',
|
||||||
isReplay: true,
|
replaySource: 'history',
|
||||||
});
|
});
|
||||||
|
|
||||||
await nopy();
|
await nopy();
|
||||||
@@ -313,6 +361,19 @@ describe('nopy', () => {
|
|||||||
expect(addToHistory).not.toHaveBeenCalled();
|
expect(addToHistory).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('records a replay out of a session file', async () => {
|
||||||
|
runWorkflow.mockResolvedValue({
|
||||||
|
session: session(),
|
||||||
|
selectedCubes: ['cube-a'],
|
||||||
|
authMethod: 'ssh-key',
|
||||||
|
replaySource: 'file',
|
||||||
|
});
|
||||||
|
|
||||||
|
await nopy();
|
||||||
|
|
||||||
|
expect(addToHistory).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
|
||||||
it('skips history when nothing would be deployed', async () => {
|
it('skips history when nothing would be deployed', async () => {
|
||||||
state.deployCalls = [];
|
state.deployCalls = [];
|
||||||
|
|
||||||
@@ -320,19 +381,36 @@ describe('nopy', () => {
|
|||||||
|
|
||||||
expect(addToHistory).not.toHaveBeenCalled();
|
expect(addToHistory).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('skips history for a print-only run', async () => {
|
||||||
|
// Same rule as `--dry-run`: nothing was deployed, so nothing belongs at
|
||||||
|
// the head of the list `-R` repeats.
|
||||||
|
await nopy({ printOnly: true });
|
||||||
|
|
||||||
|
expect(addToHistory).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('printOnly', () => {
|
describe('printOnly', () => {
|
||||||
it('prints commands and never executes', async () => {
|
it('prints commands and never executes', async () => {
|
||||||
await nopy({ printOnly: true });
|
await nopy({ printOnly: true });
|
||||||
|
|
||||||
const text = output();
|
const text = stdout();
|
||||||
expect(text).toContain('Deploy Commands');
|
expect(text).toContain('Deploy Commands');
|
||||||
expect(text).toContain('# cube-a -> web-1');
|
expect(text).toContain('# cube-a -> web-1');
|
||||||
expect(text).toContain('pyinfra web-1 -y cube-a.deploy.py');
|
expect(text).toContain('pyinfra web-1 -y cube-a.deploy.py');
|
||||||
expect(executeDeployCalls).not.toHaveBeenCalled();
|
expect(executeDeployCalls).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('keeps stdout to the commands and nothing else', async () => {
|
||||||
|
// The whole point of the split: `nopy -P > plan.txt` has to be the plan.
|
||||||
|
// The config banner and every log line are on the other stream.
|
||||||
|
await nopy({ printOnly: true });
|
||||||
|
|
||||||
|
expect(stdout()).not.toContain('Configuration');
|
||||||
|
expect(stderr()).toContain('Configuration');
|
||||||
|
});
|
||||||
|
|
||||||
it('reports the command count as the summary total', async () => {
|
it('reports the command count as the summary total', async () => {
|
||||||
const result = await nopy({ printOnly: true });
|
const result = await nopy({ printOnly: true });
|
||||||
|
|
||||||
@@ -359,17 +437,9 @@ describe('nopy', () => {
|
|||||||
const [, options] = executeDeployCalls.mock.calls[0];
|
const [, options] = executeDeployCalls.mock.calls[0];
|
||||||
options.onProgress({ cube: 'cube-a', host: 'web-1', success: true }, 1, 1);
|
options.onProgress({ cube: 'cube-a', host: 'web-1', success: true }, 1, 1);
|
||||||
options.onProgress({ cube: 'cube-b', host: 'web-1', success: false }, 1, 1);
|
options.onProgress({ cube: 'cube-b', host: 'web-1', success: false }, 1, 1);
|
||||||
// Exercises both the ✓ and ✗ branches; logtape writes via console.log.
|
// Exercises both the ✓ and ✗ branches; logtape writes via console.error.
|
||||||
expect(logSpy).toHaveBeenCalled();
|
expect(stderr()).toContain('cube-a');
|
||||||
});
|
expect(stderr()).toContain('cube-b');
|
||||||
|
|
||||||
it('stays silent on progress when jsonOutput is set', async () => {
|
|
||||||
await nopy({ jsonOutput: true });
|
|
||||||
|
|
||||||
const [, options] = executeDeployCalls.mock.calls[0];
|
|
||||||
const before = logSpy.mock.calls.length;
|
|
||||||
options.onProgress({ cube: 'cube-a', host: 'web-1', success: true }, 1, 1);
|
|
||||||
expect(logSpy.mock.calls.length).toBe(before);
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,81 @@
|
|||||||
|
/**
|
||||||
|
* The variable form, on a terminal that reports no size.
|
||||||
|
*
|
||||||
|
* This is the one case that cannot be tested from inside a vitest worker: there
|
||||||
|
* is no TTY there for enquirer to misread, so the mocked tests in
|
||||||
|
* `prompts.test.ts` prove only that `rows` is *passed*, never that passing it
|
||||||
|
* matters. Here a real pty is opened at 0x0 — `pty.fork()`'s own default, and
|
||||||
|
* what `script -q` and some CI terminals report — and a real form is answered
|
||||||
|
* through it.
|
||||||
|
*
|
||||||
|
* Measured both ways while writing this: with `terminalSize()` removed from
|
||||||
|
* `nopy.prompts.ts`, the form never renders and the driver times out with
|
||||||
|
* nothing on the wire.
|
||||||
|
*
|
||||||
|
* Needs `python3` for the pty; skipped, loudly, where there is none.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { execFileSync, spawnSync } from 'node:child_process';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import os from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||||
|
|
||||||
|
const EXPECT_PY = fileURLToPath(new URL('../../../scripts/expect.py', import.meta.url));
|
||||||
|
const TSX = fileURLToPath(new URL('../node_modules/.bin/tsx', import.meta.url));
|
||||||
|
const PROBE = fileURLToPath(new URL('./fixtures/form-probe.ts', import.meta.url));
|
||||||
|
|
||||||
|
const hasPython = spawnSync('python3', ['--version']).status === 0;
|
||||||
|
|
||||||
|
/** Down arrow — how the form moves from one field to the next. */
|
||||||
|
const DOWN = '\u001b[B';
|
||||||
|
|
||||||
|
// Waits for the *description*, not the key. The probe schema is written
|
||||||
|
// `.describe(…).default(…)` — the order that used to lose the label — so this
|
||||||
|
// step also witnesses the zod-wrapper unwrapping through a real enquirer render
|
||||||
|
// rather than through the mocked Form in `prompts.test.ts`.
|
||||||
|
const STEPS = [
|
||||||
|
{ expect: 'First value', send: 'alpha-typed', settle: 0.6 },
|
||||||
|
{ send: DOWN, settle: 0.4 },
|
||||||
|
{ send: 'beta-typed', settle: 0.4 },
|
||||||
|
{ send: '\r', settle: 1.2 },
|
||||||
|
];
|
||||||
|
|
||||||
|
describe.skipIf(!hasPython)('variable form over a pty', () => {
|
||||||
|
let tmpDir: string;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'nopy-pty-'));
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
/** Answers the probe form on a pty of the given size; returns what it assigned. */
|
||||||
|
const answerForm = (rows: number, cols: number) => {
|
||||||
|
const stepsPath = path.join(tmpDir, 'steps.json');
|
||||||
|
const logPath = path.join(tmpDir, 'session.log');
|
||||||
|
fs.writeFileSync(stepsPath, JSON.stringify(STEPS));
|
||||||
|
|
||||||
|
execFileSync('python3', [EXPECT_PY, stepsPath, '--', TSX, PROBE], {
|
||||||
|
env: {
|
||||||
|
...process.env,
|
||||||
|
PTY_ROWS: String(rows),
|
||||||
|
PTY_COLS: String(cols),
|
||||||
|
EXPECT_TIMEOUT: '60',
|
||||||
|
EXPECT_LOG: logPath,
|
||||||
|
},
|
||||||
|
stdio: 'pipe',
|
||||||
|
});
|
||||||
|
|
||||||
|
const transcript = fs.readFileSync(logPath, 'utf-8').replace(/\r/g, '');
|
||||||
|
const line = transcript.split('\n').find((l) => l.startsWith('NOPY_PROBE '));
|
||||||
|
return line ? JSON.parse(line.slice('NOPY_PROBE '.length)) : undefined;
|
||||||
|
};
|
||||||
|
|
||||||
|
it('collects every field on a terminal reporting 0x0', () => {
|
||||||
|
expect(answerForm(0, 0)).toEqual({ ALPHA: 'alpha-typed', BETA: 'beta-typed' });
|
||||||
|
}, 90_000);
|
||||||
|
});
|
||||||
@@ -55,11 +55,27 @@ const question = (name: string) => questions().find((q) => q.name === name);
|
|||||||
/** Grabs the options the last enquirer AutoComplete prompt was constructed with. */
|
/** Grabs the options the last enquirer AutoComplete prompt was constructed with. */
|
||||||
const autoComplete = () => autoCompleteCtor.mock.calls.at(-1)?.[0] as Record<string, any>;
|
const autoComplete = () => autoCompleteCtor.mock.calls.at(-1)?.[0] as Record<string, any>;
|
||||||
|
|
||||||
|
/** Grabs the options the last enquirer Form prompt was constructed with. */
|
||||||
|
const formOptions = () => formCtor.mock.calls.at(-1)?.[0] as Record<string, any>;
|
||||||
|
|
||||||
/** Grabs the choices the last enquirer Form prompt was constructed with. */
|
/** Grabs the choices the last enquirer Form prompt was constructed with. */
|
||||||
const formChoices = () => {
|
const formChoices = () => formOptions().choices as Record<string, any>[];
|
||||||
const options = formCtor.mock.calls.at(-1)?.[0] as { choices: Record<string, any>[] };
|
|
||||||
return options.choices;
|
/** Runs `body` with the terminal reporting the given size, then puts it back. */
|
||||||
};
|
async function withTerminal(
|
||||||
|
size: { rows: number; columns: number },
|
||||||
|
body: () => Promise<void>
|
||||||
|
): Promise<void> {
|
||||||
|
const was = { rows: process.stdout.rows, columns: process.stdout.columns };
|
||||||
|
Object.defineProperty(process.stdout, 'rows', { value: size.rows, configurable: true });
|
||||||
|
Object.defineProperty(process.stdout, 'columns', { value: size.columns, configurable: true });
|
||||||
|
try {
|
||||||
|
await body();
|
||||||
|
} finally {
|
||||||
|
Object.defineProperty(process.stdout, 'rows', { value: was.rows, configurable: true });
|
||||||
|
Object.defineProperty(process.stdout, 'columns', { value: was.columns, configurable: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
const cube = (id: string, name: string, schema = z.object({})) =>
|
const cube = (id: string, name: string, schema = z.object({})) =>
|
||||||
new Cube(Manifest({ id, name, schema }), `/cubes/${id}`, 'deploy.py');
|
new Cube(Manifest({ id, name, schema }), `/cubes/${id}`, 'deploy.py');
|
||||||
@@ -113,24 +129,42 @@ describe('CubeSelection', () => {
|
|||||||
|
|
||||||
it('derives page size from the terminal height', async () => {
|
it('derives page size from the terminal height', async () => {
|
||||||
autoCompleteRun.mockResolvedValue([]);
|
autoCompleteRun.mockResolvedValue([]);
|
||||||
const rows = process.stdout.rows;
|
|
||||||
|
|
||||||
Object.defineProperty(process.stdout, 'rows', { value: 40, configurable: true });
|
await withTerminal({ rows: 40, columns: 200 }, async () => {
|
||||||
await CubeSelection(cubes);
|
await CubeSelection(cubes);
|
||||||
expect(autoComplete().limit).toBe(35);
|
expect(autoComplete().limit).toBe(35);
|
||||||
|
|
||||||
// Falls back to a floor of 10 on a short (or unknown) terminal.
|
|
||||||
Object.defineProperty(process.stdout, 'rows', { value: 0, configurable: true });
|
|
||||||
await CubeSelection(cubes);
|
|
||||||
expect(autoComplete().limit).toBe(19);
|
|
||||||
|
|
||||||
Object.defineProperty(process.stdout, 'rows', { value: rows, configurable: true });
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('selects nothing when the user cancels', async () => {
|
// A terminal reporting nothing is floored, not believed.
|
||||||
|
await withTerminal({ rows: 0, columns: 0 }, async () => {
|
||||||
|
await CubeSelection(cubes);
|
||||||
|
expect(autoComplete().limit).toBe(19);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('hands the prompt a window size it can render into', async () => {
|
||||||
|
autoCompleteRun.mockResolvedValue([]);
|
||||||
|
|
||||||
|
// Passing `rows` is what keeps enquirer away from its own `utils.height`,
|
||||||
|
// which overwrites a good fallback with `getWindowSize()[1]` — zero here.
|
||||||
|
await withTerminal({ rows: 0, columns: 0 }, async () => {
|
||||||
|
await CubeSelection(cubes);
|
||||||
|
expect(autoComplete()).toMatchObject({ rows: 24, columns: 80 });
|
||||||
|
});
|
||||||
|
|
||||||
|
await withTerminal({ rows: 50, columns: 200 }, async () => {
|
||||||
|
await CubeSelection(cubes);
|
||||||
|
expect(autoComplete()).toMatchObject({ rows: 50, columns: 200 });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('lets a cancellation travel instead of returning an empty selection', async () => {
|
||||||
|
// An empty selection is a legitimate answer, so swallowing the rejection
|
||||||
|
// here made "the user backed out" and "the user picked nothing" the same
|
||||||
|
// event and let the run continue to deploy zero cubes.
|
||||||
autoCompleteRun.mockRejectedValue(new Error('cancelled'));
|
autoCompleteRun.mockRejectedValue(new Error('cancelled'));
|
||||||
|
|
||||||
await expect(CubeSelection(cubes)).resolves.toEqual({ selectedCubes: [] });
|
await expect(CubeSelection(cubes)).rejects.toThrow('cancelled');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -199,11 +233,32 @@ describe('HostSelection', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it('prefixes a docker container', async () => {
|
it('prefixes a docker container', async () => {
|
||||||
inquirerPrompt.mockResolvedValue({ host: 'runtime:docker', dockerContainer: 'box' });
|
inquirerPrompt.mockResolvedValue({ host: 'docker', dockerTarget: 'box' });
|
||||||
|
|
||||||
await expect(HostSelection([])).resolves.toBe('@docker/box');
|
await expect(HostSelection([])).resolves.toBe('@docker/box');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('prefixes a docker image reference the same way', async () => {
|
||||||
|
inquirerPrompt.mockResolvedValue({ host: 'docker', dockerTarget: 'ubuntu:24.04' });
|
||||||
|
|
||||||
|
await expect(HostSelection([])).resolves.toBe('@docker/ubuntu:24.04');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('trims the docker identifier', async () => {
|
||||||
|
inquirerPrompt.mockResolvedValue({ host: 'docker', dockerTarget: ' ubuntu:24.04 ' });
|
||||||
|
|
||||||
|
await expect(HostSelection([])).resolves.toBe('@docker/ubuntu:24.04');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects an empty docker identifier', async () => {
|
||||||
|
inquirerPrompt.mockResolvedValue({ host: 'web-1' });
|
||||||
|
|
||||||
|
await HostSelection([]);
|
||||||
|
|
||||||
|
expect(question('dockerTarget')?.validate(' ')).toBe('Required');
|
||||||
|
expect(question('dockerTarget')?.validate('ubuntu:24.04')).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
it('gates the follow-up questions on the chosen host', async () => {
|
it('gates the follow-up questions on the chosen host', async () => {
|
||||||
inquirerPrompt.mockResolvedValue({ host: 'web-1' });
|
inquirerPrompt.mockResolvedValue({ host: 'web-1' });
|
||||||
|
|
||||||
@@ -213,8 +268,12 @@ describe('HostSelection', () => {
|
|||||||
expect(question('customHost')?.when({ host: 'web-1' })).toBe(false);
|
expect(question('customHost')?.when({ host: 'web-1' })).toBe(false);
|
||||||
expect(question('vagrantVM')?.when({ host: 'vagrant' })).toBe(true);
|
expect(question('vagrantVM')?.when({ host: 'vagrant' })).toBe(true);
|
||||||
expect(question('vagrantVM')?.when({ host: 'web-1' })).toBe(false);
|
expect(question('vagrantVM')?.when({ host: 'web-1' })).toBe(false);
|
||||||
expect(question('dockerContainer')?.when({ host: 'runtime:docker' })).toBe(true);
|
// The regression: the gate compared against the `runtime:docker` *cube id*,
|
||||||
expect(question('dockerContainer')?.when({ host: 'web-1' })).toBe(false);
|
// so picking `docker` from the list skipped this question entirely and the
|
||||||
|
// host came back as the literal string `docker`.
|
||||||
|
expect(question('dockerTarget')?.when({ host: 'docker' })).toBe(true);
|
||||||
|
expect(question('dockerTarget')?.when({ host: 'runtime:docker' })).toBe(false);
|
||||||
|
expect(question('dockerTarget')?.when({ host: 'web-1' })).toBe(false);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -297,6 +356,34 @@ describe('VariableAssignment', () => {
|
|||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('finds the label whichever side of .default() it was declared on', async () => {
|
||||||
|
// zod 4 keys a description to the schema *instance* and `.default()` returns
|
||||||
|
// a new wrapper, so `describe().default()` used to prompt with the bare key.
|
||||||
|
// 15 of the 22 core cubes were written that way round.
|
||||||
|
const both = cube(
|
||||||
|
'both',
|
||||||
|
'Both',
|
||||||
|
z.object({
|
||||||
|
BEFORE: z.boolean().describe('Update package cache').default(false),
|
||||||
|
AFTER: z.boolean().default(false).describe('Restart afterwards'),
|
||||||
|
WRAPPED: z.string().describe('Optional note').optional().default('n/a'),
|
||||||
|
NEITHER: z.string().default('x'),
|
||||||
|
})
|
||||||
|
);
|
||||||
|
const variables = new Variables();
|
||||||
|
variables.assign('both', 'default', both.getDefaults());
|
||||||
|
formRun.mockResolvedValue({});
|
||||||
|
|
||||||
|
await VariableAssignment(both, variables);
|
||||||
|
|
||||||
|
expect(formChoices().map((c) => c.message)).toEqual([
|
||||||
|
'Update package cache',
|
||||||
|
'Restart afterwards',
|
||||||
|
'Optional note',
|
||||||
|
'NEITHER',
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
it('offers the value the run would use, not the bare schema default', async () => {
|
it('offers the value the run would use, not the bare schema default', async () => {
|
||||||
const svc = cube('svc', 'Service', schema);
|
const svc = cube('svc', 'Service', schema);
|
||||||
const variables = new Variables({ port: 2222 });
|
const variables = new Variables({ port: 2222 });
|
||||||
@@ -393,13 +480,31 @@ describe('VariableAssignment', () => {
|
|||||||
expect(variables.get('svc').extra).toBe('kept');
|
expect(variables.get('svc').extra).toBe('kept');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('assigns nothing when the user cancels the form', async () => {
|
it('hands the form a window size it can render into', async () => {
|
||||||
|
const variables = new Variables();
|
||||||
|
formRun.mockResolvedValue({});
|
||||||
|
|
||||||
|
// The form is where a zero height actually costs something: enquirer
|
||||||
|
// renders `Math.min(limit, height)` fields, so a 0-row terminal shows none
|
||||||
|
// of them and submits `{}` without the user ever seeing the questions.
|
||||||
|
await withTerminal({ rows: 0, columns: 0 }, async () => {
|
||||||
|
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||||
|
expect(formOptions()).toMatchObject({ rows: 24, columns: 80 });
|
||||||
|
});
|
||||||
|
|
||||||
|
await withTerminal({ rows: 50, columns: 200 }, async () => {
|
||||||
|
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||||
|
expect(formOptions()).toMatchObject({ rows: 50, columns: 200 });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('lets a cancelled form travel rather than assigning nothing', async () => {
|
||||||
const variables = new Variables();
|
const variables = new Variables();
|
||||||
formRun.mockRejectedValue(new Error('cancelled'));
|
formRun.mockRejectedValue(new Error('cancelled'));
|
||||||
|
|
||||||
await expect(
|
await expect(VariableAssignment(cube('svc', 'Service', schema), variables)).rejects.toThrow(
|
||||||
VariableAssignment(cube('svc', 'Service', schema), variables)
|
'cancelled'
|
||||||
).resolves.toBeUndefined();
|
);
|
||||||
expect(variables.get('svc')).toEqual({});
|
expect(variables.get('svc')).toEqual({});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -5,12 +5,14 @@
|
|||||||
import fs from 'node:fs';
|
import fs from 'node:fs';
|
||||||
import os from 'node:os';
|
import os from 'node:os';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
import {
|
import {
|
||||||
createSession,
|
createSession,
|
||||||
|
describeSession,
|
||||||
listSessions,
|
listSessions,
|
||||||
loadSession,
|
loadSession,
|
||||||
type NopySession,
|
type NopySession,
|
||||||
|
SESSION_VERSION,
|
||||||
saveSession,
|
saveSession,
|
||||||
} from '../src/nopy.session.js';
|
} from '../src/nopy.session.js';
|
||||||
|
|
||||||
@@ -48,6 +50,62 @@ describe('createSession', () => {
|
|||||||
|
|
||||||
expect(session.env).toEqual({ KEY: 'value' });
|
expect(session.env).toEqual({ KEY: 'value' });
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('stamps the format version and a creation time', () => {
|
||||||
|
const session = createSession({ cubes: [], hosts: ['localhost'], auth: { method: 'ssh' } });
|
||||||
|
|
||||||
|
expect(session.version).toBe(SESSION_VERSION);
|
||||||
|
expect(new Date(session.timestamp!).toISOString()).toBe(session.timestamp);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('lets the caller supply the timestamp', () => {
|
||||||
|
const session = createSession({
|
||||||
|
cubes: [],
|
||||||
|
hosts: ['localhost'],
|
||||||
|
auth: { method: 'ssh' },
|
||||||
|
timestamp: '2026-01-01T00:00:00.000Z',
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(session.timestamp).toBe('2026-01-01T00:00:00.000Z');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('describeSession', () => {
|
||||||
|
const at = '2026-01-01T12:30:00.000Z';
|
||||||
|
|
||||||
|
it('names the cubes and the hosts', () => {
|
||||||
|
const name = describeSession(
|
||||||
|
{
|
||||||
|
cubes: [{ key: 'apt:essentials', variables: {} }],
|
||||||
|
hosts: ['web-1'],
|
||||||
|
auth: { method: 'ssh' },
|
||||||
|
},
|
||||||
|
at
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(name).toContain('apt:essentials');
|
||||||
|
expect(name).toContain('web-1');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('says so when there is no host', () => {
|
||||||
|
const name = describeSession({ cubes: [], auth: { method: 'ssh' } }, at);
|
||||||
|
|
||||||
|
expect(name).toContain('no host');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('truncates a long cube list and a long host list', () => {
|
||||||
|
const name = describeSession(
|
||||||
|
{
|
||||||
|
cubes: Array.from({ length: 10 }, (_, i) => ({ key: `cube-${i}`, variables: {} })),
|
||||||
|
hosts: Array.from({ length: 10 }, (_, i) => `host-${i}`),
|
||||||
|
auth: { method: 'ssh' },
|
||||||
|
},
|
||||||
|
at
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(name).toContain('...');
|
||||||
|
expect(name.split('→')[1]).toContain('...');
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('saveSession and loadSession', () => {
|
describe('saveSession and loadSession', () => {
|
||||||
@@ -116,6 +174,51 @@ describe('saveSession and loadSession', () => {
|
|||||||
|
|
||||||
await expect(loadSession(sessionPath)).rejects.toThrow('auth');
|
await expect(loadSession(sessionPath)).rejects.toThrow('auth');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Half of SESSION_FORMAT.md is about the MJS form, and nothing exercised it.
|
||||||
|
// Each test needs its own filename: `import()` caches by URL, so a second
|
||||||
|
// module written to the same path would never be read.
|
||||||
|
it('loads a session from an MJS default export', async () => {
|
||||||
|
const mjsPath = path.join(tempDir, 'ok.session.mjs');
|
||||||
|
fs.writeFileSync(
|
||||||
|
mjsPath,
|
||||||
|
'export default { cubes: [{ key: "apt:essentials", variables: {} }], auth: { method: "ssh" } };'
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(loadSession(mjsPath)).resolves.toMatchObject({
|
||||||
|
cubes: [{ key: 'apt:essentials', variables: {} }],
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects an MJS session with no default export', async () => {
|
||||||
|
const mjsPath = path.join(tempDir, 'no-default.session.mjs');
|
||||||
|
fs.writeFileSync(mjsPath, 'export const session = {};');
|
||||||
|
|
||||||
|
await expect(loadSession(mjsPath)).rejects.toThrow('must export a default object');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('loads a session with no version at all', async () => {
|
||||||
|
fs.writeFileSync(sessionPath, JSON.stringify({ cubes: [], auth: { method: 'ssh' } }));
|
||||||
|
const warn = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||||
|
|
||||||
|
await expect(loadSession(sessionPath)).resolves.toMatchObject({ cubes: [] });
|
||||||
|
expect(warn).not.toHaveBeenCalled();
|
||||||
|
|
||||||
|
warn.mockRestore();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('warns about an unknown version but still loads it', async () => {
|
||||||
|
fs.writeFileSync(
|
||||||
|
sessionPath,
|
||||||
|
JSON.stringify({ version: '9.9.9', cubes: [], auth: { method: 'ssh' } })
|
||||||
|
);
|
||||||
|
const warn = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||||
|
|
||||||
|
await expect(loadSession(sessionPath)).resolves.toMatchObject({ version: '9.9.9' });
|
||||||
|
expect(warn.mock.calls[0][0]).toContain('9.9.9');
|
||||||
|
|
||||||
|
warn.mockRestore();
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('listSessions', () => {
|
describe('listSessions', () => {
|
||||||
@@ -154,4 +257,18 @@ describe('listSessions', () => {
|
|||||||
expect(result).toHaveLength(1);
|
expect(result).toHaveLength(1);
|
||||||
expect(result[0].endsWith('test.session.mjs')).toBe(true);
|
expect(result[0].endsWith('test.session.mjs')).toBe(true);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('finds the documented .nopysession.* files', () => {
|
||||||
|
// The name every example in the README uses, and the one this missed:
|
||||||
|
// `wild.nopysession.json` does not end in `.session.json`.
|
||||||
|
fs.writeFileSync(path.join(tempDir, 'wild.nopysession.json'), '{}');
|
||||||
|
fs.writeFileSync(path.join(tempDir, 'wild.nopysession.mjs'), 'export default {}');
|
||||||
|
fs.writeFileSync(path.join(tempDir, 'nopysession.json'), '{}');
|
||||||
|
|
||||||
|
const result = listSessions(tempDir);
|
||||||
|
|
||||||
|
expect(result).toHaveLength(2);
|
||||||
|
expect(result.some((p) => p.endsWith('wild.nopysession.json'))).toBe(true);
|
||||||
|
expect(result.some((p) => p.endsWith('wild.nopysession.mjs'))).toBe(true);
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -78,7 +78,7 @@ describe('runInteractiveWorkflow', () => {
|
|||||||
|
|
||||||
expect(result.selectedCubes).toEqual(['cube-a']);
|
expect(result.selectedCubes).toEqual(['cube-a']);
|
||||||
expect(result.authMethod).toBe('ssh-key');
|
expect(result.authMethod).toBe('ssh-key');
|
||||||
expect(result.isReplay).toBe(false);
|
expect(result.replaySource).toBeUndefined();
|
||||||
expect(result.session.hosts).toEqual(['web-1']);
|
expect(result.session.hosts).toEqual(['web-1']);
|
||||||
expect(result.session.env).toEqual({ GLOBAL: 'value' });
|
expect(result.session.env).toEqual({ GLOBAL: 'value' });
|
||||||
expect(mockHostSelection).toHaveBeenCalledWith(config.hosts);
|
expect(mockHostSelection).toHaveBeenCalledWith(config.hosts);
|
||||||
@@ -150,7 +150,7 @@ describe('runReplayWorkflow', () => {
|
|||||||
const result = await runReplayWorkflow('/tmp/s.json', cubes, config);
|
const result = await runReplayWorkflow('/tmp/s.json', cubes, config);
|
||||||
|
|
||||||
expect(mockLoadSession).toHaveBeenCalledWith('/tmp/s.json');
|
expect(mockLoadSession).toHaveBeenCalledWith('/tmp/s.json');
|
||||||
expect(result.isReplay).toBe(true);
|
expect(result.replaySource).toBe('file');
|
||||||
expect(result.selectedCubes).toEqual(['cube-a']);
|
expect(result.selectedCubes).toEqual(['cube-a']);
|
||||||
expect(mockHostSelection).not.toHaveBeenCalled();
|
expect(mockHostSelection).not.toHaveBeenCalled();
|
||||||
expect(mockPasswordSelection).not.toHaveBeenCalled();
|
expect(mockPasswordSelection).not.toHaveBeenCalled();
|
||||||
@@ -225,7 +225,7 @@ describe('runSessionReplayWorkflow', () => {
|
|||||||
it('replays an in-memory session without prompting', async () => {
|
it('replays an in-memory session without prompting', async () => {
|
||||||
const result = await runSessionReplayWorkflow(session(), cubes, config);
|
const result = await runSessionReplayWorkflow(session(), cubes, config);
|
||||||
|
|
||||||
expect(result.isReplay).toBe(true);
|
expect(result.replaySource).toBe('history');
|
||||||
expect(result.selectedCubes).toEqual(['cube-a']);
|
expect(result.selectedCubes).toEqual(['cube-a']);
|
||||||
expect(mockLoadSession).not.toHaveBeenCalled();
|
expect(mockLoadSession).not.toHaveBeenCalled();
|
||||||
expect(mockHostSelection).not.toHaveBeenCalled();
|
expect(mockHostSelection).not.toHaveBeenCalled();
|
||||||
@@ -287,7 +287,7 @@ describe('runWorkflow dispatch', () => {
|
|||||||
it('prefers an in-memory replay session over everything else', async () => {
|
it('prefers an in-memory replay session over everything else', async () => {
|
||||||
const result = await runWorkflow('/tmp/s.json', cubes, config, {}, session());
|
const result = await runWorkflow('/tmp/s.json', cubes, config, {}, session());
|
||||||
|
|
||||||
expect(result.isReplay).toBe(true);
|
expect(result.replaySource).toBe('history');
|
||||||
expect(mockLoadSession).not.toHaveBeenCalled();
|
expect(mockLoadSession).not.toHaveBeenCalled();
|
||||||
expect(mockCubeSelection).not.toHaveBeenCalled();
|
expect(mockCubeSelection).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
@@ -298,14 +298,14 @@ describe('runWorkflow dispatch', () => {
|
|||||||
const result = await runWorkflow('/tmp/s.json', cubes, config);
|
const result = await runWorkflow('/tmp/s.json', cubes, config);
|
||||||
|
|
||||||
expect(mockLoadSession).toHaveBeenCalledWith('/tmp/s.json');
|
expect(mockLoadSession).toHaveBeenCalledWith('/tmp/s.json');
|
||||||
expect(result.isReplay).toBe(true);
|
expect(result.replaySource).toBe('file');
|
||||||
expect(mockCubeSelection).not.toHaveBeenCalled();
|
expect(mockCubeSelection).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('falls back to the interactive workflow', async () => {
|
it('falls back to the interactive workflow', async () => {
|
||||||
const result = await runWorkflow(undefined, cubes, config, { useAuthKey: true });
|
const result = await runWorkflow(undefined, cubes, config, { useAuthKey: true });
|
||||||
|
|
||||||
expect(result.isReplay).toBe(false);
|
expect(result.replaySource).toBeUndefined();
|
||||||
expect(mockCubeSelection).toHaveBeenCalled();
|
expect(mockCubeSelection).toHaveBeenCalled();
|
||||||
expect(mockAuthSelection).toHaveBeenCalledWith(true);
|
expect(mockAuthSelection).toHaveBeenCalledWith(true);
|
||||||
});
|
});
|
||||||
|
|||||||
Generated
+12
@@ -26,12 +26,24 @@ importers:
|
|||||||
'@types/node':
|
'@types/node':
|
||||||
specifier: ^26.1.1
|
specifier: ^26.1.1
|
||||||
version: 26.1.1
|
version: 26.1.1
|
||||||
|
commander:
|
||||||
|
specifier: ^15.0.0
|
||||||
|
version: 15.0.0
|
||||||
|
enquirer:
|
||||||
|
specifier: ^2.4.1
|
||||||
|
version: 2.4.1
|
||||||
|
semver:
|
||||||
|
specifier: ^7.8.5
|
||||||
|
version: 7.8.5
|
||||||
simple-git-hooks:
|
simple-git-hooks:
|
||||||
specifier: ^2.13.1
|
specifier: ^2.13.1
|
||||||
version: 2.13.1
|
version: 2.13.1
|
||||||
typescript:
|
typescript:
|
||||||
specifier: ^7.0.2
|
specifier: ^7.0.2
|
||||||
version: 7.0.2
|
version: 7.0.2
|
||||||
|
zx:
|
||||||
|
specifier: ^8.8.5
|
||||||
|
version: 8.8.5
|
||||||
|
|
||||||
packages/keyman:
|
packages/keyman:
|
||||||
dependencies:
|
dependencies:
|
||||||
|
|||||||
Executable
+57
@@ -0,0 +1,57 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Drive an interactive CLI through a pty on a fixed schedule.
|
||||||
|
|
||||||
|
scripts/drive.py <script.json> -- <cmd> [args...]
|
||||||
|
|
||||||
|
`script.json` is a list of `[seconds_since_start, "text to send"]` pairs.
|
||||||
|
Everything the child prints is echoed to this process's stdout.
|
||||||
|
|
||||||
|
Timing-based, so it is the blunt one — good for a quick manual poke at the TUI,
|
||||||
|
bad for anything that has to be reliable. Prefer `expect.py`, which waits for the
|
||||||
|
prompt instead of guessing when it will appear.
|
||||||
|
|
||||||
|
Environment: `PTY_ROWS` / `PTY_COLS` (default 50x200), `DRIVE_TIMEOUT` seconds.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import select
|
||||||
|
import signal
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
from ptysize import size_from_env, spawn # noqa: E402
|
||||||
|
|
||||||
|
sep = sys.argv.index("--")
|
||||||
|
steps = json.loads(open(sys.argv[1]).read())
|
||||||
|
cmd = sys.argv[sep + 1 :]
|
||||||
|
|
||||||
|
rows, cols = size_from_env()
|
||||||
|
pid, fd = spawn(cmd, rows, cols)
|
||||||
|
|
||||||
|
start = time.time()
|
||||||
|
pending = list(steps)
|
||||||
|
deadline = start + float(os.environ.get("DRIVE_TIMEOUT", "3600"))
|
||||||
|
|
||||||
|
while True:
|
||||||
|
if pending and time.time() - start >= pending[0][0]:
|
||||||
|
_, text = pending.pop(0)
|
||||||
|
os.write(fd, text.encode())
|
||||||
|
r, _, _ = select.select([fd], [], [], 0.2)
|
||||||
|
if r:
|
||||||
|
try:
|
||||||
|
data = os.read(fd, 65536)
|
||||||
|
except OSError:
|
||||||
|
break
|
||||||
|
if not data:
|
||||||
|
break
|
||||||
|
sys.stdout.buffer.write(data)
|
||||||
|
sys.stdout.buffer.flush()
|
||||||
|
if time.time() > deadline:
|
||||||
|
os.kill(pid, signal.SIGKILL)
|
||||||
|
break
|
||||||
|
|
||||||
|
_, status = os.waitpid(pid, 0)
|
||||||
|
sys.stderr.write("\n[drive.py] exit status: %d\n" % (status >> 8))
|
||||||
|
sys.exit(status >> 8)
|
||||||
Executable
+110
@@ -0,0 +1,110 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Expect-style pty driver — waits for each prompt before answering it.
|
||||||
|
|
||||||
|
scripts/expect.py <script.json> -- <cmd> [args...]
|
||||||
|
|
||||||
|
`script.json` is a list of steps:
|
||||||
|
|
||||||
|
{"expect": "<regex>", "send": "<text>", "settle": 0.4}
|
||||||
|
{"send": "<text>"} -- send immediately
|
||||||
|
|
||||||
|
Each regex is matched against the ANSI-stripped output accumulated *since the
|
||||||
|
previous step completed*, not against everything seen so far, so the same prompt
|
||||||
|
text can be awaited twice in one run — which the nopy variable form does, once
|
||||||
|
per cube.
|
||||||
|
|
||||||
|
Environment: `PTY_ROWS` / `PTY_COLS` (default 50x200), `EXPECT_TIMEOUT` seconds,
|
||||||
|
`EXPECT_LOG` for the transcript path (default `expect.log`).
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import select
|
||||||
|
import signal
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
from ptysize import size_from_env, spawn # noqa: E402
|
||||||
|
|
||||||
|
ANSI = re.compile(rb"\x1b\[[0-9;?]*[a-zA-Z]|\x1b\][^\x07]*\x07|\r")
|
||||||
|
|
||||||
|
sep = sys.argv.index("--")
|
||||||
|
steps = json.loads(open(sys.argv[1]).read())
|
||||||
|
cmd = sys.argv[sep + 1 :]
|
||||||
|
TIMEOUT = float(os.environ.get("EXPECT_TIMEOUT", "900"))
|
||||||
|
|
||||||
|
rows, cols = size_from_env()
|
||||||
|
pid, fd = spawn(cmd, rows, cols)
|
||||||
|
|
||||||
|
log = open(os.environ.get("EXPECT_LOG", "expect.log"), "wb")
|
||||||
|
start = time.time()
|
||||||
|
window = b"" # output since the last completed step
|
||||||
|
alive = True
|
||||||
|
|
||||||
|
|
||||||
|
def pump(seconds):
|
||||||
|
"""Read child output for `seconds`, appending to `window`."""
|
||||||
|
global window, alive
|
||||||
|
end = time.time() + seconds
|
||||||
|
while time.time() < end:
|
||||||
|
r, _, _ = select.select([fd], [], [], min(0.2, max(0.01, end - time.time())))
|
||||||
|
if not r:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
data = os.read(fd, 65536)
|
||||||
|
except OSError:
|
||||||
|
alive = False
|
||||||
|
return
|
||||||
|
if not data:
|
||||||
|
alive = False
|
||||||
|
return
|
||||||
|
window += data
|
||||||
|
log.write(data)
|
||||||
|
log.flush()
|
||||||
|
|
||||||
|
|
||||||
|
failed = False
|
||||||
|
|
||||||
|
for i, step in enumerate(steps):
|
||||||
|
pattern = step.get("expect")
|
||||||
|
if pattern:
|
||||||
|
rx = re.compile(pattern.encode())
|
||||||
|
found = False
|
||||||
|
while time.time() - start < TIMEOUT:
|
||||||
|
if rx.search(ANSI.sub(b"", window)):
|
||||||
|
found = True
|
||||||
|
break
|
||||||
|
if not alive:
|
||||||
|
break
|
||||||
|
pump(0.3)
|
||||||
|
if not found:
|
||||||
|
sys.stderr.write(
|
||||||
|
"\n[expect] step %d timed out waiting for %r\n" % (i, pattern)
|
||||||
|
)
|
||||||
|
failed = True
|
||||||
|
os.kill(pid, signal.SIGKILL)
|
||||||
|
break
|
||||||
|
sys.stderr.write("[expect] step %d matched %r\n" % (i, pattern))
|
||||||
|
pump(step.get("settle", 0.5))
|
||||||
|
window = b""
|
||||||
|
text = step.get("send")
|
||||||
|
if text:
|
||||||
|
os.write(fd, text.encode())
|
||||||
|
|
||||||
|
# drain until the child exits
|
||||||
|
while alive and time.time() - start < TIMEOUT:
|
||||||
|
pump(1.0)
|
||||||
|
if alive:
|
||||||
|
sys.stderr.write("\n[expect] overall timeout, killing child\n")
|
||||||
|
failed = True
|
||||||
|
os.kill(pid, signal.SIGKILL)
|
||||||
|
|
||||||
|
_, status = os.waitpid(pid, 0)
|
||||||
|
log.close()
|
||||||
|
code = status >> 8
|
||||||
|
sys.stderr.write("\n[expect] exit status: %d\n" % code)
|
||||||
|
# A driver that gave up must not report the child's exit code as its own — a
|
||||||
|
# SIGKILLed child can still look like a clean 0 to a caller reading $?.
|
||||||
|
sys.exit(1 if failed else code)
|
||||||
@@ -5,10 +5,15 @@
|
|||||||
*
|
*
|
||||||
* The version is the one the linked package declares *right now*, which is
|
* The version is the one the linked package declares *right now*, which is
|
||||||
* exactly what `pnpm publish` will substitute for `workspace:*` when it packs.
|
* exactly what `pnpm publish` will substitute for `workspace:*` when it packs.
|
||||||
* A release can therefore check that each of them is already on the registry
|
* So this is the list of versions that must already be on the registry before
|
||||||
* before shipping a manifest that points at a version nobody can install.
|
* shipping the package, or the tarball points at something nobody can install.
|
||||||
*
|
*
|
||||||
* node scripts/linked-deps.mjs packages/nopy
|
* node scripts/linked-deps.mjs packages/nopy
|
||||||
|
*
|
||||||
|
* `release.yml` no longer runs this as a gate. `scripts/release.mjs` enforces
|
||||||
|
* the same ordering earlier and more cheaply, by pushing tags dependency-first
|
||||||
|
* and waiting for each version to appear on npmjs before pushing the next. This
|
||||||
|
* stays as the hand-check for when you want to see the list yourself.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import fs from 'node:fs';
|
import fs from 'node:fs';
|
||||||
|
|||||||
@@ -0,0 +1,50 @@
|
|||||||
|
"""Spawn a child on a pty whose window size is what you asked for.
|
||||||
|
|
||||||
|
`pty.fork()` leaves the new terminal at **0 rows by 0 columns**, and nothing
|
||||||
|
fixes that afterwards: `COLUMNS`/`LINES` in the environment are a shell
|
||||||
|
convention that `ioctl(TIOCGWINSZ)` has never heard of, so `process.stdout.rows`
|
||||||
|
in the child stays 0 however they are set.
|
||||||
|
|
||||||
|
That is not a detail. A driver that forgets the ioctl is testing a terminal no
|
||||||
|
user has, and it lied to us once already — the enquirer `Form` under it returned
|
||||||
|
`{}` for reasons that had nothing to do with the code under test. See the
|
||||||
|
`terminalSize` comment in `packages/nopy/src/nopy.prompts.ts`.
|
||||||
|
|
||||||
|
So the size is always set explicitly here, including when it is set to 0: the
|
||||||
|
degenerate terminal is worth testing, but only on purpose.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import fcntl
|
||||||
|
import os
|
||||||
|
import pty
|
||||||
|
import struct
|
||||||
|
import termios
|
||||||
|
|
||||||
|
|
||||||
|
def set_winsize(fd, rows, cols):
|
||||||
|
fcntl.ioctl(fd, termios.TIOCSWINSZ, struct.pack("HHHH", rows, cols, 0, 0))
|
||||||
|
|
||||||
|
|
||||||
|
def spawn(cmd, rows, cols, env=None):
|
||||||
|
"""Fork `cmd` onto a pty sized `rows` x `cols`. Returns (pid, fd)."""
|
||||||
|
pid, fd = pty.fork()
|
||||||
|
if pid == 0:
|
||||||
|
os.environ["TERM"] = "xterm-256color"
|
||||||
|
# Kept in step with the ioctl so that a program reading either one gets
|
||||||
|
# the same answer. The ioctl is what actually matters.
|
||||||
|
os.environ["COLUMNS"] = str(cols)
|
||||||
|
os.environ["LINES"] = str(rows)
|
||||||
|
for key, value in (env or {}).items():
|
||||||
|
os.environ[key] = value
|
||||||
|
os.execvp(cmd[0], cmd)
|
||||||
|
|
||||||
|
set_winsize(fd, rows, cols)
|
||||||
|
return pid, fd
|
||||||
|
|
||||||
|
|
||||||
|
def size_from_env(default_rows=50, default_cols=200):
|
||||||
|
"""`PTY_ROWS` / `PTY_COLS`, so a caller can ask for the 0x0 case."""
|
||||||
|
return (
|
||||||
|
int(os.environ.get("PTY_ROWS", default_rows)),
|
||||||
|
int(os.environ.get("PTY_COLS", default_cols)),
|
||||||
|
)
|
||||||
@@ -0,0 +1,866 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Interactive release client: pick packages, choose versions, hand tags to CI.
|
||||||
|
*
|
||||||
|
* `release.yml` is tag-driven and deliberately dumb — the tag names the package,
|
||||||
|
* `package.json` names the version, and the run fails if they disagree. Getting
|
||||||
|
* to a good tag is the fiddly part, and it is all manual today: bump the right
|
||||||
|
* manifest, write a changelog the release body can quote, run the gate, tag with
|
||||||
|
* the `<directory>-v<version>` spelling, and push the tags dependency-first
|
||||||
|
* because `pnpm publish` bakes a linked package's *current* version into its
|
||||||
|
* dependent's tarball at pack time. This script does that sequence.
|
||||||
|
*
|
||||||
|
* pnpm run release # pick packages and versions
|
||||||
|
* pnpm run release -- -p nopy -v minor # non-interactive version choice
|
||||||
|
* pnpm run release -- --dry-run # print the plan, change nothing
|
||||||
|
*
|
||||||
|
* Order of operations is load-bearing: versions and changelogs are written to
|
||||||
|
* the working tree, the gate runs against *that* tree, and only then is anything
|
||||||
|
* committed. A failing gate therefore leaves no commit to unpick — the script
|
||||||
|
* offers to restore the tree instead.
|
||||||
|
*
|
||||||
|
* After each tag is pushed it polls npmjs until that exact version resolves.
|
||||||
|
* That is both the ordering barrier (a dependent must not be tagged until the
|
||||||
|
* version pnpm will bake into it exists on the registry) and the final proof
|
||||||
|
* that the release actually landed, rather than that CI accepted the tag.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import os from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { Command } from 'commander';
|
||||||
|
import Enquirer from 'enquirer';
|
||||||
|
import semver from 'semver';
|
||||||
|
import { $, chalk, fs } from 'zx';
|
||||||
|
|
||||||
|
const PACKAGES_DIR = 'packages';
|
||||||
|
const RANGE_FIELDS = ['dependencies', 'peerDependencies', 'optionalDependencies'];
|
||||||
|
const NPMJS_REGISTRY = 'https://registry.npmjs.org/';
|
||||||
|
const FETCH_TIMEOUT_MS = 15_000;
|
||||||
|
const POLL_INTERVAL_MS = 15_000;
|
||||||
|
|
||||||
|
// Terminals that report no size make enquirer render zero choices and submit
|
||||||
|
// silently — see the long note on `terminalSize` in nopy.prompts.ts.
|
||||||
|
const MIN_ROWS = 24;
|
||||||
|
const MIN_COLS = 80;
|
||||||
|
|
||||||
|
$.verbose = false;
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Workspace
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
const readManifest = (dir) => JSON.parse(fs.readFileSync(path.join(dir, 'package.json'), 'utf-8'));
|
||||||
|
|
||||||
|
/** Every publishable package, alphabetically by directory. */
|
||||||
|
function readWorkspace() {
|
||||||
|
return fs
|
||||||
|
.readdirSync(PACKAGES_DIR)
|
||||||
|
.map((name) => path.join(PACKAGES_DIR, name))
|
||||||
|
.filter((dir) => fs.existsSync(path.join(dir, 'package.json')))
|
||||||
|
.map((dir) => ({ dir, slug: path.basename(dir), manifest: readManifest(dir) }))
|
||||||
|
.filter(({ manifest }) => !manifest.private)
|
||||||
|
.sort((a, b) => a.dir.localeCompare(b.dir));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The other workspace packages this one links to, as directories. */
|
||||||
|
function linkedDirs(pkg, byName) {
|
||||||
|
return RANGE_FIELDS.flatMap((field) => Object.entries(pkg.manifest[field] ?? {}))
|
||||||
|
.filter(([, range]) => range.startsWith('workspace:'))
|
||||||
|
.map(([name]) => byName.get(name)?.dir)
|
||||||
|
.filter((dir) => dir !== undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Topological order over the `workspace:` edges — dependencies first.
|
||||||
|
*
|
||||||
|
* The same ordering `scripts/publish-order.mjs` prints, recomputed here rather
|
||||||
|
* than shelled out to because this needs the subset being released, not all of
|
||||||
|
* `packages/`. Alphabetical among packages the graph does not separate, so a
|
||||||
|
* plan printed twice reads the same both times.
|
||||||
|
*/
|
||||||
|
function dependencyOrder(packages) {
|
||||||
|
const byName = new Map(packages.map((pkg) => [pkg.manifest.name, pkg]));
|
||||||
|
const byDir = new Map(packages.map((pkg) => [pkg.dir, pkg]));
|
||||||
|
const ordered = [];
|
||||||
|
const emitted = new Set();
|
||||||
|
const visiting = new Set();
|
||||||
|
|
||||||
|
const visit = (pkg) => {
|
||||||
|
if (emitted.has(pkg.dir)) return;
|
||||||
|
if (visiting.has(pkg.dir)) throw new Error(`Dependency cycle in the workspace, at ${pkg.dir}`);
|
||||||
|
visiting.add(pkg.dir);
|
||||||
|
for (const dir of linkedDirs(pkg, byName)) {
|
||||||
|
const dependency = byDir.get(dir);
|
||||||
|
if (dependency) visit(dependency);
|
||||||
|
}
|
||||||
|
visiting.delete(pkg.dir);
|
||||||
|
emitted.add(pkg.dir);
|
||||||
|
ordered.push(pkg);
|
||||||
|
};
|
||||||
|
|
||||||
|
for (const pkg of [...packages].sort((a, b) => a.dir.localeCompare(b.dir))) visit(pkg);
|
||||||
|
return ordered;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// npmjs
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetches a packument from npmjs, normalising every failure into a printable
|
||||||
|
* shape rather than throwing — an unreachable registry should soften the picker,
|
||||||
|
* not abort the release.
|
||||||
|
*/
|
||||||
|
async function packument(name) {
|
||||||
|
let response;
|
||||||
|
try {
|
||||||
|
response = await fetch(`${NPMJS_REGISTRY}${encodeURIComponent(name)}`, {
|
||||||
|
headers: { accept: 'application/vnd.npm.install-v1+json, application/json' },
|
||||||
|
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
return { reachable: false, note: error.name === 'TimeoutError' ? 'timed out' : 'unreachable' };
|
||||||
|
}
|
||||||
|
|
||||||
|
if (response.status === 404) return { reachable: true, published: false, versions: [], tags: {} };
|
||||||
|
if (!response.ok) return { reachable: true, note: `HTTP ${response.status}` };
|
||||||
|
|
||||||
|
let body;
|
||||||
|
try {
|
||||||
|
body = await response.json();
|
||||||
|
} catch {
|
||||||
|
return { reachable: true, note: 'unparseable response' };
|
||||||
|
}
|
||||||
|
if (body.error) return { reachable: true, published: false, versions: [], tags: {} };
|
||||||
|
|
||||||
|
return {
|
||||||
|
reachable: true,
|
||||||
|
published: true,
|
||||||
|
tags: body['dist-tags'] ?? {},
|
||||||
|
versions: Object.keys(body.versions ?? {}),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether one exact version resolves on npmjs.
|
||||||
|
*
|
||||||
|
* The per-version endpoint rather than the packument: it 404s until the publish
|
||||||
|
* lands, where a cached packument can answer without the new version in it and
|
||||||
|
* read as "not yet" long after it is there.
|
||||||
|
*/
|
||||||
|
async function versionExists(name, version) {
|
||||||
|
try {
|
||||||
|
const response = await fetch(
|
||||||
|
`${NPMJS_REGISTRY}${encodeURIComponent(name)}/${encodeURIComponent(version)}`,
|
||||||
|
{
|
||||||
|
headers: { accept: 'application/json', 'cache-control': 'no-cache' },
|
||||||
|
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
|
||||||
|
}
|
||||||
|
);
|
||||||
|
return response.ok;
|
||||||
|
} catch {
|
||||||
|
// A blip mid-poll is not an answer; the next tick asks again.
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Polls until `name@version` resolves on npmjs, or the deadline passes. */
|
||||||
|
async function waitForRelease(name, version, timeoutMs) {
|
||||||
|
const started = Date.now();
|
||||||
|
const label = `${name}@${version}`;
|
||||||
|
process.stdout.write(` waiting for ${label} on npmjs `);
|
||||||
|
|
||||||
|
for (;;) {
|
||||||
|
if (await versionExists(name, version)) {
|
||||||
|
const seconds = Math.round((Date.now() - started) / 1000);
|
||||||
|
console.log(chalk.green(` published after ${seconds}s`));
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (Date.now() - started > timeoutMs) {
|
||||||
|
console.log(chalk.red(' timed out'));
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
process.stdout.write('.');
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// git
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
const git = async (...args) => (await $`git ${args}`).stdout.trim();
|
||||||
|
|
||||||
|
async function currentBranch() {
|
||||||
|
return await git('rev-parse', '--abbrev-ref', 'HEAD');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function isClean() {
|
||||||
|
return (await git('status', '--porcelain')) === '';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Local tags plus whatever the remote has, so a re-tag is caught either way. */
|
||||||
|
async function existingTags(remote) {
|
||||||
|
const local = (await git('tag', '--list')).split('\n').filter(Boolean);
|
||||||
|
let remoteTags = [];
|
||||||
|
try {
|
||||||
|
const output = await git('ls-remote', '--tags', remote);
|
||||||
|
remoteTags = output
|
||||||
|
.split('\n')
|
||||||
|
.filter(Boolean)
|
||||||
|
.map((line) => line.split('refs/tags/')[1])
|
||||||
|
.filter((tag) => tag && !tag.endsWith('^{}'));
|
||||||
|
} catch {
|
||||||
|
// Offline, or no such remote. The local list still catches the common case
|
||||||
|
// and `git push` will reject a duplicate anyway.
|
||||||
|
}
|
||||||
|
return new Set([...local, ...remoteTags]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The newest `<slug>-v*` tag by semver, or undefined if the package is unreleased. */
|
||||||
|
function lastTagFor(slug, tags) {
|
||||||
|
const prefix = `${slug}-v`;
|
||||||
|
return [...tags]
|
||||||
|
.filter((tag) => tag.startsWith(prefix) && semver.valid(tag.slice(prefix.length)))
|
||||||
|
.sort((a, b) => semver.rcompare(a.slice(prefix.length), b.slice(prefix.length)))[0];
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Prompts
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
const terminalSize = () => ({
|
||||||
|
rows: Math.max(process.stdout.rows || 0, MIN_ROWS),
|
||||||
|
columns: Math.max(process.stdout.columns || 0, MIN_COLS),
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs an enquirer prompt, refusing outright when there is no terminal to run it
|
||||||
|
* in.
|
||||||
|
*
|
||||||
|
* Without the guard a piped or CI invocation hangs on a prompt nobody can
|
||||||
|
* answer, and node reports it as `unsettled top-level await` — which says
|
||||||
|
* nothing about the missing flag that would have avoided the prompt.
|
||||||
|
*/
|
||||||
|
function ask(Kind, options) {
|
||||||
|
if (!process.stdin.isTTY) {
|
||||||
|
throw new Error(
|
||||||
|
`'${options.message}' needs an answer, but stdin is not a terminal.\n` +
|
||||||
|
'Supply --package / --version (and --no-changelog) to run non-interactively.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return new Enquirer[Kind]({ ...options, ...terminalSize() }).run();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A yes/no question. Unlike {@link ask} this has a defensible answer without a
|
||||||
|
* terminal — the default — because every caller is an optional extra step.
|
||||||
|
*/
|
||||||
|
async function confirm(message, initial = false) {
|
||||||
|
if (!process.stdin.isTTY) {
|
||||||
|
console.log(chalk.dim(` ${message} → ${initial ? 'yes' : 'no'} (not a terminal)`));
|
||||||
|
return initial;
|
||||||
|
}
|
||||||
|
return await new Enquirer.Confirm({ name: 'ok', message, initial, ...terminalSize() }).run();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One line per package, so the picker shows what npmjs already has. */
|
||||||
|
function describeForPicker(pkg, registry) {
|
||||||
|
const local = `local ${pkg.manifest.version}`;
|
||||||
|
if (!registry.reachable) return `${local} · npmjs ${registry.note}`;
|
||||||
|
if (registry.note) return `${local} · npmjs ${registry.note}`;
|
||||||
|
if (!registry.published) return `${local} · not on npmjs`;
|
||||||
|
const latest = registry.tags?.latest;
|
||||||
|
return `${local} · npmjs latest ${latest ?? '(none)'}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function selectPackages(candidates, registries) {
|
||||||
|
const selected = await ask('MultiSelect', {
|
||||||
|
name: 'packages',
|
||||||
|
message: 'Which packages are you releasing?',
|
||||||
|
hint: '(space to select, enter to confirm)',
|
||||||
|
choices: candidates.map((pkg) => ({
|
||||||
|
name: pkg.slug,
|
||||||
|
message: `${pkg.manifest.name.padEnd(28)} ${describeForPicker(pkg, registries.get(pkg.manifest.name))}`,
|
||||||
|
})),
|
||||||
|
validate: (value) => (value.length > 0 ? true : 'Select at least one package.'),
|
||||||
|
});
|
||||||
|
return selected;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resolves a version for one package, either from `--version` or interactively.
|
||||||
|
*
|
||||||
|
* Bumps are computed from the manifest, not from npmjs: the manifest is the
|
||||||
|
* repo's source of truth and the thing `release.yml` compares the tag against.
|
||||||
|
* Where npmjs is ahead — which it is for every package still carrying the old
|
||||||
|
* `1.0.0-alpha5` — that is surfaced as a warning rather than a different base,
|
||||||
|
* because silently jumping the local version is how you lose a bump.
|
||||||
|
*/
|
||||||
|
async function chooseVersion(pkg, registry, requested, tags) {
|
||||||
|
const current = pkg.manifest.version;
|
||||||
|
const taken = new Set(registry.versions ?? []);
|
||||||
|
const KEYWORDS = ['patch', 'minor', 'major', 'prerelease'];
|
||||||
|
|
||||||
|
const check = (version) => {
|
||||||
|
if (!semver.valid(version)) return `'${version}' is not a valid semver version.`;
|
||||||
|
if (taken.has(version)) return `${pkg.manifest.name}@${version} is already on npmjs.`;
|
||||||
|
if (tags.has(`${pkg.slug}-v${version}`)) return `Tag ${pkg.slug}-v${version} already exists.`;
|
||||||
|
return true;
|
||||||
|
};
|
||||||
|
|
||||||
|
if (requested) {
|
||||||
|
const version = KEYWORDS.includes(requested)
|
||||||
|
? semver.inc(current, requested, requested === 'prerelease' ? 'rc' : undefined)
|
||||||
|
: requested;
|
||||||
|
const problem = check(version);
|
||||||
|
if (problem !== true) throw new Error(problem);
|
||||||
|
return version;
|
||||||
|
}
|
||||||
|
|
||||||
|
const bump = (release, identifier) => semver.inc(current, release, identifier);
|
||||||
|
const choices = [
|
||||||
|
{ name: bump('patch'), message: `patch ${current} → ${bump('patch')}` },
|
||||||
|
{ name: bump('minor'), message: `minor ${current} → ${bump('minor')}` },
|
||||||
|
{ name: bump('major'), message: `major ${current} → ${bump('major')}` },
|
||||||
|
{
|
||||||
|
name: bump('prerelease', 'rc'),
|
||||||
|
// Publishes under `next`, never `latest` — the rule in release.yml is
|
||||||
|
// purely "does the version contain a `-`".
|
||||||
|
message: `prerelease ${current} → ${bump('prerelease', 'rc')} (dist-tag: next)`,
|
||||||
|
},
|
||||||
|
{ name: 'custom', message: 'custom…' },
|
||||||
|
].map((choice) => {
|
||||||
|
const problem = choice.name === 'custom' ? true : check(choice.name);
|
||||||
|
return problem === true
|
||||||
|
? choice
|
||||||
|
: { ...choice, message: `${choice.message} ${problem}`, disabled: true };
|
||||||
|
});
|
||||||
|
|
||||||
|
const picked = await ask('Select', {
|
||||||
|
name: 'version',
|
||||||
|
message: `Version for ${pkg.manifest.name} (currently ${current})`,
|
||||||
|
choices,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (picked !== 'custom') {
|
||||||
|
const problem = check(picked);
|
||||||
|
if (problem !== true) throw new Error(problem);
|
||||||
|
return picked;
|
||||||
|
}
|
||||||
|
|
||||||
|
return await ask('Input', {
|
||||||
|
name: 'version',
|
||||||
|
message: `Version for ${pkg.manifest.name}`,
|
||||||
|
initial: bump('patch'),
|
||||||
|
validate: check,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Changelog
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
const CHANGELOG_HEADER = '# Changelog\n';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Collects release notes in `$EDITOR`, seeded with the commits since the
|
||||||
|
* package's last tag.
|
||||||
|
*
|
||||||
|
* The seed is the point: `release.yml` quotes this section verbatim into the
|
||||||
|
* Gitea release body, and a summary written next to the actual commit list is a
|
||||||
|
* better one than a summary written from memory. Lines starting with `#` are
|
||||||
|
* stripped, so the seed can carry instructions without them leaking into the
|
||||||
|
* release.
|
||||||
|
*/
|
||||||
|
async function collectNotes(pkg, version, tags) {
|
||||||
|
const editor = process.env.VISUAL || process.env.EDITOR;
|
||||||
|
const lastTag = lastTagFor(pkg.slug, tags);
|
||||||
|
|
||||||
|
let log = '';
|
||||||
|
try {
|
||||||
|
const range = lastTag ? `${lastTag}..HEAD` : 'HEAD';
|
||||||
|
log = await git('log', range, '--oneline', '--no-decorate', '-n', '40', '--', pkg.dir);
|
||||||
|
} catch {
|
||||||
|
// A shallow clone or a brand new package — the template still works.
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!process.stdin.isTTY) {
|
||||||
|
// Nothing here can prompt, and an editor would have no terminal to draw on.
|
||||||
|
// Missing notes are survivable — the release body degrades to the install
|
||||||
|
// snippet — so say so and carry on rather than failing the release.
|
||||||
|
console.log(chalk.yellow(` no TTY — skipping release notes for ${pkg.manifest.name}.`));
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!editor) {
|
||||||
|
// One line beats nothing when there is no editor to fall back on.
|
||||||
|
const summary = await ask('Input', {
|
||||||
|
name: 'notes',
|
||||||
|
message: `Release notes for ${pkg.manifest.name}@${version} (blank to skip, $EDITOR unset)`,
|
||||||
|
});
|
||||||
|
return summary.trim() ? `- ${summary.trim()}` : '';
|
||||||
|
}
|
||||||
|
|
||||||
|
const seed = [
|
||||||
|
'',
|
||||||
|
`# Release notes for ${pkg.manifest.name}@${version}.`,
|
||||||
|
'# Lines starting with "#" are ignored. Leave the file empty to skip.',
|
||||||
|
'#',
|
||||||
|
`# Commits since ${lastTag ?? 'the beginning'} touching ${pkg.dir}:`,
|
||||||
|
...(log ? log.split('\n').map((line) => `# ${line}`) : ['# (none)']),
|
||||||
|
'',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const file = path.join(
|
||||||
|
fs.mkdtempSync(path.join(os.tmpdir(), 'release-notes-')),
|
||||||
|
`${pkg.slug}-${version}.md`
|
||||||
|
);
|
||||||
|
fs.writeFileSync(file, seed);
|
||||||
|
|
||||||
|
try {
|
||||||
|
// The editor owns the terminal — `stdio: inherit` is what makes a full-screen
|
||||||
|
// vim usable here rather than a scrambled buffer.
|
||||||
|
await $({ stdio: 'inherit' })`${editor} ${file}`;
|
||||||
|
return fs
|
||||||
|
.readFileSync(file, 'utf-8')
|
||||||
|
.split('\n')
|
||||||
|
.filter((line) => !line.startsWith('#'))
|
||||||
|
.join('\n')
|
||||||
|
.trim();
|
||||||
|
} finally {
|
||||||
|
fs.rmSync(path.dirname(file), { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Prepends a section for `version`, creating the file if the package has none. */
|
||||||
|
function writeChangelog(pkg, version, notes, today) {
|
||||||
|
const file = path.join(pkg.dir, 'CHANGELOG.md');
|
||||||
|
const existed = fs.existsSync(file);
|
||||||
|
const section = `## ${version} — ${today}\n\n${notes}\n`;
|
||||||
|
|
||||||
|
if (!existed) {
|
||||||
|
fs.writeFileSync(file, `${CHANGELOG_HEADER}\n${section}`);
|
||||||
|
return { file, created: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Insert above the newest existing section, so the file stays newest-first and
|
||||||
|
// the new entry lands *below* any `# Changelog` title rather than above it.
|
||||||
|
const body = fs.readFileSync(file, 'utf-8');
|
||||||
|
const at = body.startsWith('## ') ? 0 : body.indexOf('\n## ') + 1;
|
||||||
|
const updated =
|
||||||
|
at === 0 && !body.startsWith('## ')
|
||||||
|
? `${body.replace(/\n*$/, '\n')}\n${section}`
|
||||||
|
: `${body.slice(0, at)}${section}\n${body.slice(at)}`;
|
||||||
|
|
||||||
|
fs.writeFileSync(file, updated);
|
||||||
|
return { file, created: false };
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Steps
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
async function preflight(remote) {
|
||||||
|
const root = await git('rev-parse', '--show-toplevel');
|
||||||
|
if (path.resolve(root) !== path.resolve(process.cwd())) {
|
||||||
|
throw new Error(`Run this from the repository root (${root}).`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!(await isClean())) {
|
||||||
|
throw new Error(
|
||||||
|
'The working tree has uncommitted changes.\n' +
|
||||||
|
'A release commit must contain the version bump and nothing else — commit or stash first.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const branch = await currentBranch();
|
||||||
|
|
||||||
|
console.log(chalk.dim(` fetching ${remote}…`));
|
||||||
|
try {
|
||||||
|
await $`git fetch ${remote} --tags --quiet`;
|
||||||
|
} catch {
|
||||||
|
console.log(chalk.yellow(` could not reach ${remote} — continuing with local refs only.`));
|
||||||
|
return { branch };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Behind the remote is the dangerous one: the tag would point at a commit that
|
||||||
|
// is not what `main` will look like, and the release would ship a tree nobody
|
||||||
|
// reviewed. Ahead is merely unpushed, and this script pushes.
|
||||||
|
const behind = await git('rev-list', '--count', `HEAD..${remote}/${branch}`).catch(() => '0');
|
||||||
|
if (behind !== '0') {
|
||||||
|
throw new Error(
|
||||||
|
`${branch} is ${behind} commit(s) behind ${remote}/${branch}. Pull before releasing.`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
return { branch };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The gate, run against the bumped working tree. Same three commands as CI. */
|
||||||
|
async function verify() {
|
||||||
|
const steps = [
|
||||||
|
['lint', ['run', 'lint:ci']],
|
||||||
|
['typecheck', ['run', 'typecheck']],
|
||||||
|
['test', ['run', 'test:coverage']],
|
||||||
|
['build', ['run', 'build']],
|
||||||
|
];
|
||||||
|
|
||||||
|
for (const [label, args] of steps) {
|
||||||
|
console.log(chalk.bold(`\n ▸ pnpm ${args.join(' ')}`));
|
||||||
|
const result = await $({ stdio: 'inherit', nothrow: true })`pnpm ${args}`;
|
||||||
|
if (result.exitCode !== 0) return label;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(chalk.bold('\n ▸ node scripts/verify-pack.mjs'));
|
||||||
|
const packed = await $({ stdio: 'inherit', nothrow: true })`node scripts/verify-pack.mjs`;
|
||||||
|
return packed.exitCode === 0 ? null : 'verify-pack';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Puts the tree back the way preflight found it — which was clean, by contract. */
|
||||||
|
async function restore(touched) {
|
||||||
|
for (const { file, created } of touched) {
|
||||||
|
if (created) fs.rmSync(file, { force: true });
|
||||||
|
}
|
||||||
|
const tracked = touched.filter((entry) => !entry.created).map((entry) => entry.file);
|
||||||
|
if (tracked.length > 0) await $`git checkout -- ${tracked}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Main
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
async function release(options) {
|
||||||
|
const workspace = readWorkspace();
|
||||||
|
const byName = new Map(workspace.map((pkg) => [pkg.manifest.name, pkg]));
|
||||||
|
const bySlug = new Map(workspace.map((pkg) => [pkg.slug, pkg]));
|
||||||
|
|
||||||
|
console.log(chalk.bold('\nRelease\n'));
|
||||||
|
const { branch } = await preflight(options.remote);
|
||||||
|
|
||||||
|
if (branch !== 'main') {
|
||||||
|
console.log(
|
||||||
|
chalk.yellow(
|
||||||
|
` You are on '${branch}', not 'main'. Tags release from any branch, but the\n` +
|
||||||
|
' snapshot lane and the review flow both assume main.'
|
||||||
|
)
|
||||||
|
);
|
||||||
|
if (!options.yes && !(await confirm(`Release from '${branch}' anyway?`))) return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
const tags = await existingTags(options.remote);
|
||||||
|
|
||||||
|
console.log(chalk.dim(' reading npmjs…\n'));
|
||||||
|
const registries = new Map(
|
||||||
|
await Promise.all(
|
||||||
|
workspace.map(async (pkg) => [pkg.manifest.name, await packument(pkg.manifest.name)])
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
// --- pick packages -------------------------------------------------------
|
||||||
|
|
||||||
|
let slugs;
|
||||||
|
if (options.package?.length) {
|
||||||
|
slugs = options.package.map((given) => {
|
||||||
|
const pkg = bySlug.get(given) ?? byName.get(given);
|
||||||
|
if (!pkg) {
|
||||||
|
throw new Error(
|
||||||
|
`Unknown package '${given}'. Known: ${workspace.map((p) => p.slug).join(', ')}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return pkg.slug;
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
slugs = await selectPackages(workspace, registries);
|
||||||
|
}
|
||||||
|
|
||||||
|
let selected = dependencyOrder(slugs.map((slug) => bySlug.get(slug)));
|
||||||
|
|
||||||
|
if (options.version && selected.length !== 1) {
|
||||||
|
throw new Error('--version applies to a single package; select one with --package.');
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- warn about dependents ----------------------------------------------
|
||||||
|
|
||||||
|
// `pnpm publish` resolves `workspace:*` to whatever the linked package
|
||||||
|
// declares at pack time, so bumping a library silently changes what its
|
||||||
|
// dependents' *next* release will require. Better to decide that here than to
|
||||||
|
// discover it in a tarball.
|
||||||
|
const selectedDirs = new Set(selected.map((pkg) => pkg.dir));
|
||||||
|
const dependents = workspace.filter(
|
||||||
|
(pkg) =>
|
||||||
|
!selectedDirs.has(pkg.dir) && linkedDirs(pkg, byName).some((dir) => selectedDirs.has(dir))
|
||||||
|
);
|
||||||
|
|
||||||
|
if (dependents.length > 0) {
|
||||||
|
const names = dependents.map((p) => p.manifest.name).join(', ');
|
||||||
|
console.log(
|
||||||
|
chalk.yellow(
|
||||||
|
`\n ${names} ${dependents.length === 1 ? 'links' : 'link'} to a package you are\n` +
|
||||||
|
' releasing. It is not in this release, but its next one will require the new\n' +
|
||||||
|
' version whether or not you meant it to.\n'
|
||||||
|
)
|
||||||
|
);
|
||||||
|
// Not offered under --version (that flag names one version, and applying it
|
||||||
|
// to a package the user did not ask for is worse than making them re-run)
|
||||||
|
// and not under --dry-run, which must not change the plan it is printing.
|
||||||
|
const mayAdd = !options.yes && !options.version && !options.dryRun;
|
||||||
|
if (mayAdd && (await confirm('Add them to this release?'))) {
|
||||||
|
selected = dependencyOrder([...selected, ...dependents]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- versions ------------------------------------------------------------
|
||||||
|
|
||||||
|
const plan = [];
|
||||||
|
for (const pkg of selected) {
|
||||||
|
const version = await chooseVersion(
|
||||||
|
pkg,
|
||||||
|
registries.get(pkg.manifest.name),
|
||||||
|
options.version,
|
||||||
|
tags
|
||||||
|
);
|
||||||
|
|
||||||
|
const latest = registries.get(pkg.manifest.name)?.tags?.latest;
|
||||||
|
if (latest && semver.valid(latest) && semver.lte(version, latest)) {
|
||||||
|
console.log(
|
||||||
|
chalk.yellow(
|
||||||
|
` ! ${version} is not above npmjs latest (${latest}) — 'latest' will not move to it.`
|
||||||
|
)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
plan.push({ pkg, version, tag: `${pkg.slug}-v${version}` });
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- notes ---------------------------------------------------------------
|
||||||
|
|
||||||
|
// Not under --dry-run: a preview of the plan should not make you sit down and
|
||||||
|
// write release notes for a release you have not committed to yet.
|
||||||
|
if (options.changelog && !options.dryRun) {
|
||||||
|
for (const entry of plan) {
|
||||||
|
entry.notes = await collectNotes(entry.pkg, entry.version, tags);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- plan ----------------------------------------------------------------
|
||||||
|
|
||||||
|
const distTag = (version) => (version.includes('-') ? 'next' : 'latest');
|
||||||
|
const message = `release: ${plan.map((e) => `${e.pkg.manifest.name}@${e.version}`).join(', ')}`;
|
||||||
|
|
||||||
|
console.log(chalk.bold('\n Plan\n'));
|
||||||
|
for (const entry of plan) {
|
||||||
|
const { pkg, version, tag } = entry;
|
||||||
|
console.log(` ${chalk.bold(pkg.manifest.name)} ${pkg.manifest.version} → ${version}`);
|
||||||
|
console.log(` tag ${tag}`);
|
||||||
|
console.log(` dist-tag ${distTag(version)}`);
|
||||||
|
const notes = entry.notes ? `${entry.notes.split('\n').length} line(s)` : '—';
|
||||||
|
console.log(
|
||||||
|
` changelog ${options.dryRun && options.changelog ? '(prompted later)' : notes}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
console.log(`\n commit ${message}`);
|
||||||
|
console.log(` push ${options.remote} ${branch}, then tags in the order above`);
|
||||||
|
console.log(
|
||||||
|
options.wait
|
||||||
|
? ` wait for each version on npmjs (up to ${options.waitTimeout}s each)\n`
|
||||||
|
: ' wait no — tags are pushed back to back\n'
|
||||||
|
);
|
||||||
|
|
||||||
|
if (options.dryRun) {
|
||||||
|
console.log(chalk.dim(' --dry-run: nothing was changed.\n'));
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!options.yes && !(await confirm('Proceed?', true))) {
|
||||||
|
console.log(' Nothing was changed.');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- write ---------------------------------------------------------------
|
||||||
|
|
||||||
|
const today = new Date().toISOString().slice(0, 10);
|
||||||
|
const touched = [];
|
||||||
|
|
||||||
|
for (const entry of plan) {
|
||||||
|
// `npm pkg set`, the same edit the workflows make, rather than a hand-rolled
|
||||||
|
// rewrite of the manifest.
|
||||||
|
await $({ cwd: entry.pkg.dir })`npm pkg set version=${entry.version}`;
|
||||||
|
touched.push({ file: path.join(entry.pkg.dir, 'package.json'), created: false });
|
||||||
|
|
||||||
|
if (entry.notes) {
|
||||||
|
touched.push(writeChangelog(entry.pkg, entry.version, entry.notes, today));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
console.log(chalk.green(`\n wrote ${touched.length} file(s)`));
|
||||||
|
|
||||||
|
// --- verify --------------------------------------------------------------
|
||||||
|
|
||||||
|
if (options.verify) {
|
||||||
|
const failed = await verify();
|
||||||
|
if (failed) {
|
||||||
|
console.log(chalk.red(`\n ${failed} failed. Nothing has been committed.`));
|
||||||
|
if (options.yes || (await confirm('Restore the version and changelog edits?', true))) {
|
||||||
|
await restore(touched);
|
||||||
|
console.log(' Tree restored.');
|
||||||
|
} else {
|
||||||
|
console.log(' Edits left in place for inspection.');
|
||||||
|
}
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
console.log(chalk.green('\n gate passed'));
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- commit and tag ------------------------------------------------------
|
||||||
|
|
||||||
|
await $`git add -- ${touched.map((entry) => entry.file)}`;
|
||||||
|
await $`git commit -m ${message}`;
|
||||||
|
console.log(chalk.green(` committed ${await git('rev-parse', '--short=7', 'HEAD')}`));
|
||||||
|
|
||||||
|
for (const entry of plan) {
|
||||||
|
// Annotated (`-a -m`), never lightweight. A lightweight tag is rejected
|
||||||
|
// outright under `tag.forceSignAnnotated`/`tag.gpgSign`, which is a common
|
||||||
|
// enough setting to have hit this on the first real run; and the annotation
|
||||||
|
// is worth having anyway, since it puts the notes in `git show <tag>`.
|
||||||
|
const body = [`${entry.pkg.manifest.name}@${entry.version}`, entry.notes]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join('\n\n');
|
||||||
|
await $`git tag -a ${entry.tag} -m ${body}`;
|
||||||
|
}
|
||||||
|
console.log(chalk.green(` tagged ${plan.map((e) => e.tag).join(', ')}`));
|
||||||
|
|
||||||
|
// --- push ----------------------------------------------------------------
|
||||||
|
|
||||||
|
// The gate just ran against this exact tree, so the pre-push hook would only
|
||||||
|
// run it a second time. `quiet` is spelled out because a `$({…})` instance
|
||||||
|
// does not inherit the module-level `$.verbose`, and git writes push progress
|
||||||
|
// to stderr — this script reports each push itself.
|
||||||
|
const push = $({ env: { ...process.env, SKIP_SIMPLE_GIT_HOOKS: '1' }, quiet: true });
|
||||||
|
|
||||||
|
await push`git push ${options.remote} HEAD:${branch}`;
|
||||||
|
console.log(chalk.green(` pushed ${branch}`));
|
||||||
|
|
||||||
|
const pending = [];
|
||||||
|
for (const entry of plan) {
|
||||||
|
await push`git push ${options.remote} ${entry.tag}`;
|
||||||
|
console.log(chalk.green(` pushed ${entry.tag}`));
|
||||||
|
|
||||||
|
if (!options.wait) continue;
|
||||||
|
|
||||||
|
const landed = await waitForRelease(
|
||||||
|
entry.pkg.manifest.name,
|
||||||
|
entry.version,
|
||||||
|
options.waitTimeout * 1000
|
||||||
|
);
|
||||||
|
if (!landed) {
|
||||||
|
pending.push(entry);
|
||||||
|
console.log(
|
||||||
|
chalk.red(
|
||||||
|
` ${entry.pkg.manifest.name}@${entry.version} has not appeared on npmjs.\n` +
|
||||||
|
' Check the Release run before pushing anything that depends on it.'
|
||||||
|
)
|
||||||
|
);
|
||||||
|
// Deliberately no rollback: the tag is pushed and CI may still be mid-run.
|
||||||
|
// Deleting it here would race the publish it is waiting for.
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- report --------------------------------------------------------------
|
||||||
|
|
||||||
|
const blocked = pending[0];
|
||||||
|
const shipped = blocked ? plan.slice(0, plan.indexOf(blocked)) : plan;
|
||||||
|
|
||||||
|
console.log(chalk.bold('\n Done\n'));
|
||||||
|
for (const entry of shipped) {
|
||||||
|
console.log(` ${entry.pkg.manifest.name}@${entry.version} (${distTag(entry.version)})`);
|
||||||
|
console.log(chalk.dim(` npm install -g ${entry.pkg.manifest.name}@${entry.version}`));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (blocked) {
|
||||||
|
// Every tag was created locally before the first push, so the ones after the
|
||||||
|
// blockage exist here and simply have not gone out.
|
||||||
|
const unpushed = plan.slice(plan.indexOf(blocked) + 1);
|
||||||
|
console.log(chalk.yellow(`\n Blocked on ${blocked.pkg.manifest.name}@${blocked.version}.`));
|
||||||
|
if (unpushed.length > 0) {
|
||||||
|
console.log(
|
||||||
|
chalk.yellow(
|
||||||
|
` Tagged locally but not pushed: ${unpushed.map((e) => e.tag).join(', ')}\n` +
|
||||||
|
` Push them once the blocked release lands:\n` +
|
||||||
|
unpushed.map((e) => ` git push ${options.remote} ${e.tag}`).join('\n')
|
||||||
|
)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
console.log('');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// CLI
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
const program = new Command();
|
||||||
|
|
||||||
|
program
|
||||||
|
.name('release')
|
||||||
|
.description('Interactively version, verify, tag and push a release.')
|
||||||
|
.option('-p, --package <name...>', 'Packages to release (directory or npm name)')
|
||||||
|
.option('-v, --version <spec>', 'Version or bump (patch|minor|major|prerelease), one package')
|
||||||
|
.option('-n, --dry-run', 'Print the plan and change nothing')
|
||||||
|
.option('-y, --yes', 'Skip confirmations (assumes yes)')
|
||||||
|
.option('--remote <name>', 'Git remote to push to', 'origin')
|
||||||
|
.option('--no-verify', 'Skip the lint/typecheck/test/build gate')
|
||||||
|
.option('--no-changelog', 'Do not prompt for release notes')
|
||||||
|
.option('--no-wait', 'Do not poll npmjs between tag pushes')
|
||||||
|
.option('--wait-timeout <seconds>', 'How long to wait for each version', Number, 1200)
|
||||||
|
.addHelpText(
|
||||||
|
'after',
|
||||||
|
`
|
||||||
|
Examples:
|
||||||
|
$ pnpm run release Pick packages and versions
|
||||||
|
$ pnpm run release -- -p nopy -v minor Bump nopy's minor, no picker
|
||||||
|
$ pnpm run release -- -p nopy-cubes nopy Release both, dependency-first
|
||||||
|
$ pnpm run release -- --dry-run Show the plan only
|
||||||
|
|
||||||
|
The tag is '<directory>-v<version>' — 'nopy-v1.2.0', not the npm name. Tags are
|
||||||
|
pushed dependency-first and each version is confirmed on npmjs before the next
|
||||||
|
tag goes out, because pnpm bakes a linked package's version into its dependent's
|
||||||
|
tarball at pack time.
|
||||||
|
`
|
||||||
|
)
|
||||||
|
.action(async (options) => {
|
||||||
|
try {
|
||||||
|
process.exitCode = await release(options);
|
||||||
|
} catch (error) {
|
||||||
|
// A cancelled enquirer prompt rejects with '' — that is a user backing
|
||||||
|
// out, not a failure worth a stack trace.
|
||||||
|
if (error === '' || error === undefined) {
|
||||||
|
console.log('\n Cancelled. Nothing was changed.\n');
|
||||||
|
process.exitCode = 1;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Indented per line, not just the first — several of these messages are
|
||||||
|
// two or three lines and the continuation used to hang off the margin.
|
||||||
|
const text = String(error.message ?? error)
|
||||||
|
.split('\n')
|
||||||
|
.map((line) => ` ${line}`)
|
||||||
|
.join('\n');
|
||||||
|
console.error(chalk.red(`\n${text}\n`));
|
||||||
|
process.exitCode = 1;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// `pnpm run release -- --dry-run` forwards the `--` itself rather than eating it
|
||||||
|
// (measured on pnpm 11), and commander reads a bare `--` as "the rest are
|
||||||
|
// positionals" — so the habitual spelling died with *too many arguments*. This
|
||||||
|
// script takes no positional arguments at all, so a `--` can only ever be that
|
||||||
|
// artefact; dropping it makes both spellings work.
|
||||||
|
await program.parseAsync(
|
||||||
|
process.argv.slice(2).filter((argument) => argument !== '--'),
|
||||||
|
{ from: 'user' }
|
||||||
|
);
|
||||||
Reference in New Issue
Block a user