hardening and bugfixing prior to stable release

This commit is contained in:
Benjamin Diedrichsen
2026-07-31 18:21:43 +02:00
parent ac7ea07e3c
commit 0aa0be5542
44 changed files with 3016 additions and 436 deletions
+104 -14
View File
@@ -15,7 +15,7 @@ Nopy wraps pyinfra with structure, validation, and an interactive experience for
- **Schema validation** using Zod
- **Recursive cube directory discovery**
- **Dry-run mode** for previewing deployments
- **JSON output** for CI/CD integration
- **Pipeable output** for CI/CD integration — the plan on stdout, everything else on stderr
- **Session history** with replay capability
## Workflow
@@ -142,13 +142,25 @@ export default cubes.Manifest({
Every entry must be a key of `schema`; naming anything else is a manifest error and aborts the run, so a typo fails loudly instead of silently leaving a value unprotected.
Declaring a key a secret changes three things:
Declaring a key a secret changes four things:
- **It is never written to a session file or to the history.** Everything else the run settled on is recorded — including values that came from a `.default()` — but declared secrets are left out.
- **It is masked wherever a command or a plan is printed** — `--dry-run`, `--print-only`, and the debug log all show `********` in place of the value, in the variable list *and* in the `pyinfra` command line above it. The SSH password passed via `--password` is masked the same way, whether or not any cube declares secrets.
- **It is re-prompted on replay**, since there is nothing recorded to replay from (see [Session Recording and Replay](#session-recording-and-replay)).
- **It stops travelling.** Ordinary `env` values are seeded onto every cube in the run, because a cube may read a key off `host.data` that its own schema never declared. A secret is the exception: it reaches only the cubes whose `schema` names it. Otherwise putting a password under `env` — which is what unattended replay asks you to do — would put it on the command line of every unrelated cube, where nothing masks it because that cube never called it a secret.
Nopy does not guess. A key called `PASSWORD` in a manifest that declares no `secrets` is treated as an ordinary variable — recorded, and printed in the clear.
Declaring is global, masking is global. A key any manifest calls a secret is masked and kept out of sessions on every cube it lands on, even one whose own manifest forgot to list it. What is *not* global is the guess: a key called `PASSWORD` that no manifest declares anywhere is an ordinary variable — broadcast, recorded, and printed in the clear.
For a sensitive `env` value that no cube declares at all — a token only a hook reads, say — name it in the config instead:
```json
{
"secrets": ["DEPLOY_TOKEN"],
"env": { "DEPLOY_TOKEN": "..." }
}
```
Entries here behave exactly like a manifest's: masked, never recorded, and delivered only to cubes that declare them.
Three limits are worth knowing, because `secrets` keeps a value out of the files nopy writes and nothing more:
@@ -168,6 +180,7 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
"env": {
"SHARED_VAR": "value"
},
"secrets": ["DEPLOY_TOKEN"],
"log": {
"verbosity": "info",
"debug": false
@@ -184,8 +197,36 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
`history` controls automatic session recording (see [Deployment History](#deployment-history)), and `execution.continueOnError` sets the default for `--continue-on-error`.
`secrets` names `env` keys to treat as sensitive that no manifest declares — it is the config-side half of a manifest's `secrets`, and behaves identically. See [Secrets](#secrets).
`hosts` seeds the target picker; see [Target hosts](#target-hosts) for what else that picker offers.
`cubeDirs` holds paths, `cubePackages` holds installed npm packages that ship cubes — see [Cube Discovery](#cube-discovery) below and [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md) for publishing your own. Both are additive, and both resolve relative to the config file that named them, not to the working directory: a `.nopyrc.json` two levels up may name a package that only exists in *its* `node_modules`.
#### Target hosts
The host prompt offers more than the `hosts` array. Two entries at the top are
shortcuts for pyinfra's local connectors, each asking one follow-up question and
assembling the host string from the answer:
| Picked | Asks for | Becomes |
| --- | --- | --- |
| `docker` | a container name/id, **or** an image reference | `@docker/<answer>` |
| `vagrant` | the machine name (default `default`) | `@vagrant/<answer>` |
| *(a configured host)* | — | itself |
| `custom` | any address | itself |
The two connector forms can equally be written into `hosts` directly — a session
records whatever string the run used, so `"hosts": ["@vagrant/nopytestvm"]` and
picking `vagrant` are the same thing to everything downstream.
The docker answer is deliberately not validated as one kind or the other, because
the two mean very different things and only the connector can tell them apart (it
looks for a matching container first). A **container** is mutated in place and
left running; an **image** makes pyinfra start a throwaway container, apply the
deploy, commit the result as a new image and print its id. See
[DOCKER.md](docs/DOCKER.md) and [VAGRANT.md](docs/VAGRANT.md).
#### Logging Configuration
Control pyinfra output verbosity and debug information using the `log` configuration object:
@@ -259,12 +300,17 @@ Sessions are stored in `.nopysession.json` files with the following structure:
- **`env`**: The `env` block of `.nopyrc.json` as it stood at record time, kept for reference
- **`hosts`**: Array of target hosts
- **`auth`**: Authentication configuration (passwords are never stored)
- **`version`**, **`timestamp`**, **`name`**: stamped on every session nopy writes — the format version, the ISO 8601 record time, and a one-line description in the same `date - cubes → hosts` form the history list uses
Only `cubes` and `auth` are required. A hand-written session may omit the rest, and one that predates the stamp still loads; a `version` this build does not recognise is a warning on stderr, never a refusal.
`auth.method` has a third value the picker never offers: **`ssh`**, meaning the connector owns authentication and nopy supplies none. It is what an `@vagrant/` or `@docker/` host gets, which is why replaying one asks for nothing.
**What is recorded:** every value each cube settled on, regardless of where it came from — a value the user typed, one inherited from `.nopyrc.json` `env`, one a dependency supplied, and one that fell through to the schema's `.default()` are all written out the same way. A session is therefore a full snapshot rather than a diff, and a `--use-defaults` run produces a session with real values in it instead of an empty one.
The consequence is that replay is faithful rather than re-derived: the recorded value outranks the current `.nopyrc.json` `env` and the current schema default, so editing either one does not silently change what a replay does. To pick up a new default, record a fresh session.
**Security Note**: Passwords are never stored in session files. This covers both the SSH password — a session records the auth *method* and username, never the credential — and any schema key a cube's manifest lists under [`secrets`](#secrets). Both are re-prompted on replay.
**Security Note**: Passwords are never stored in session files. This covers both the SSH password — a session records the auth *method* and username, never the credential — and any schema key a cube's manifest lists under [`secrets`](#secrets). Both are re-prompted on replay. The rule applies to the session's `env` block as well as to each cube's `variables`, so a declared secret set in `.nopyrc.json` is left out of the recorded copy rather than written back out in plaintext.
#### Recording a Session
@@ -274,6 +320,9 @@ nopy install --save-session my-deployment.nopysession.json
# With defaults (no prompts for variables)
nopy install -D --save-session automated-deployment.nopysession.json
# Also works on a replay — the resolved cube set is what you asked to capture
nopy install -R --save-session repeat-of-the-last-run.nopysession.json
```
#### Replaying a Session
@@ -288,7 +337,9 @@ nopy install --load-session my-deployment.nopysession.json
A replay runs straight through without asking anything, with three exceptions. Password authentication always re-prompts. A session with no recorded host falls back to the host picker. And a cube is re-prompted for its declared secrets, plus for any required variable the session has no value for — which happens when the cube's schema has gained a field since the session was written.
Those re-prompts are what a session cannot supply, so `--use-defaults` cannot paper over them: combining `-D` with a replay that needs either fails with a message naming the keys rather than deploying with a placeholder. Put the values under `env` in `.nopyrc.json` to make such a replay unattended.
Those re-prompts are what a session cannot supply, so `--use-defaults` cannot paper over them: combining `-D` with a replay that needs either fails with a message naming the keys rather than deploying with a placeholder. Put the values under `env` in `.nopyrc.json` — or pass them from a dependency — to make such a replay unattended. A secret supplied that way reaches only the cubes that declare it, so this does not broadcast it across the run; see [Secrets](#secrets).
A schema `.default()` is deliberately *not* accepted in its place. The recorded answer is gone on purpose, so falling back to the manifest would deploy a different credential than the run being replayed, and say nothing about it.
### Cube Discovery
@@ -314,13 +365,17 @@ A cube package is an ordinary npm package that ships its cubes in a `cubes/` dir
Install it and name it — nothing needs linking or copying:
```sh
pnpm add -D @bitsquare/nopy-cubes-core
pnpm add -D @bitsquare/nopy-cubes-core@main \
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
```
```json
{ "cubePackages": ["@bitsquare/nopy-cubes-core"] }
```
The tag and the registry flag are both required for this bundle today — see
[Installation](#installation).
Naming a package is a statement that cubes are expected from it, so anything wrong is an error that aborts the run rather than a silent skip: the package is not installed, it has neither a `cubes/` directory nor a `nopy.cubes` override, its `nopy.cubes` is malformed, or an entry points at a directory that does not exist or lies outside the package.
#### Ids are claimed globally
@@ -331,6 +386,18 @@ Writing cubes to publish is covered in [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md).
## Command Line Usage
### Requirements
| | |
| --- | --- |
| **Node** | ≥ 22 |
| **pyinfra** | on `PATH` — `pipx install pyinfra` |
| **the connector** | `vagrant` or `docker` on `PATH`, if you deploy to one |
nopy builds pyinfra command lines and spawns them; it does not vendor pyinfra and
will not install it for you. A missing `pyinfra` surfaces as a spawn failure on
the first deploy, after every prompt has been answered.
### Installation
```bash
@@ -341,13 +408,21 @@ The cubes live in a separate bundle, installed into whichever project describes
your infrastructure and named in its `.nopyrc.json`:
```bash
pnpm add -D @bitsquare/nopy-cubes-core
pnpm add -D @bitsquare/nopy-cubes-core@main \
--@bitsquare:registry=https://gitea.bitsquare.dev/api/packages/BitSquare/npm/
```
```json
{ "hosts": ["your-host"], "cubePackages": ["@bitsquare/nopy-cubes-core"] }
```
**The tag and the registry flag are both required for the bundle today.** It has
not been published to npmjs yet, and the Gitea registry publishes no `latest`
tag, so a plain `pnpm add -D @bitsquare/nopy-cubes-core` fails with a 404 against
npmjs and an untagged Gitea install resolves to nothing. Name `@main` or `@next`
explicitly. See [Channels](#channels) for what the tags mean and how to set the
scope persistently. The CLI itself is on npmjs and installs without either.
#### Channels
Three dist-tags are published, and the one you install from is the one you stay
@@ -406,8 +481,8 @@ npm install -g @bitsquare/nopy@latest
```
Once a day, `nopy` checks its channel in the background and prints a one-line
hint to **stderr** when a newer version exists — never to stdout, so `--json`
and `--print-only` output stay clean. The answer is cached in
hint to **stderr** when a newer version exists — never to stdout, so a piped
`--print-only` stays clean. The answer is cached in
`~/.nopy/update-check.json`; a registry that is slow or unreachable is given
1.5 seconds and then ignored.
@@ -493,11 +568,14 @@ Every deployment is automatically recorded to a `.nopy.history.json` file in the
The recording happens before the deploy commands run, so a **failed** deployment is recorded too — `-R` is the quick way to retry one after fixing the cause. Replaying a session with `-R` or `-H` does not itself create a new entry, so repeating never pushes the original run out of the list.
A `--load-session` run *is* recorded, and the distinction is the point: a session file has never been in history, so without the entry `nopy history` would report nothing afterwards and `-R` would have nothing to repeat.
A run is *not* recorded when:
- `--dry-run` or `--no-history` is passed
- `--dry-run`, `--print-only` or `--no-history` is passed — the first two deploy nothing, and history is what `-R` repeats
- No cubes were selected, so there was nothing to deploy
- `history.autoSave` is set to `false` in `.nopyrc.json`
- it is a `-R` or `-H` replay, as above
Because the history file is resolved against the current working directory, each project keeps its own history — running nopy from a different directory will not find the previous run. As with session files, passwords are never stored and are re-prompted on replay.
@@ -534,14 +612,26 @@ nopy install --dry-run
Shows the execution plan including commands, environment variables, and targets without running anything. Sensitive data is masked in output.
**JSON output (for CI/CD)**:
**CI/CD**:
```bash
nopy install --json
nopy history --json
nopy install --print-only > plan.txt # the commands, and nothing else
nopy install -D # run it; exit code 1 if any cube failed
```
Machine-readable JSON output for scripting and CI/CD integration.
There is no `--json` on `install`, deliberately. A deploy runs pyinfra with
inherited stdio, so during a run nopy does not own its own stdout — pyinfra does,
and writes an unbounded amount to it. Anything nopy appended afterwards would not
be parseable by any definition a caller could rely on. Two things are guaranteed
instead:
- **stdout carries the deploy commands and pyinfra's own output. Everything nopy
says about itself — the config banner, progress lines, warnings, the update
hint, errors — goes to stderr.** So `--print-only` redirects cleanly.
- **The exit code is the verdict**: `1` if any cube failed, `0` otherwise.
`nopy history --json` is unaffected and is how a script finds the id to pass to
`-H`.
**Continue on error**:
+70 -30
View File
@@ -137,6 +137,7 @@ class Cube<Schema extends AnyObjectSchema = AnyObjectSchema> {
get secrets(): string[]; // manifest.secrets ?? []
getDefaults(): z.infer<Schema>;
schemaKeys(): string[];
requiredKeys(): string[];
isSecret(key: string): boolean;
}
@@ -151,6 +152,11 @@ single required field used to leave the cube with no variables at all.
and not optional. A `--use-defaults` run that cannot supply one aborts by name
rather than deploying the cube with the value missing.
`schemaKeys()` returns every declared key, required or not. It answers a
different question — whether the cube *claims to know about* a key, rather than
whether it has a value for one — and that is what decides whether a secret in the
config `env` is allowed to reach it.
### `CubeSource`
Where a cube came from. Carried because a cube's directory does not say how it
@@ -264,13 +270,12 @@ const result = await nopy({ useDefaults: true, dryRun: true });
|------|------|---------|-------------|
| `useDefaults` | `boolean` | `false` | Skip the variable prompts. A cube with a required key nothing supplied aborts the run by name. |
| `useAuthKey` | `boolean` | `false` | Force SSH key auth, skipping the auth prompt. |
| `saveSession` | `string` | – | Path to write the session to. **Ignored during a replay.** |
| `saveSession` | `string` | – | Path to write the session to. Honoured on a replay too. |
| `loadSession` | `string` | – | Path to a session file to replay. |
| `replaySession` | `NopySession` | – | A session object to replay, used by `-R` / `-H` from history. Takes precedence over `loadSession`. |
| `dryRun` | `boolean` | `false` | Print the execution plan instead of running it. |
| `printOnly` | `boolean` | `false` | Print the built pyinfra commands and return; the executor is never reached. |
| `continueOnError` | `boolean` | `false` | Keep going after a cube fails. |
| `jsonOutput` | `boolean` | `false` | Suppress the config banner and progress lines. See [Known gaps](#known-gaps). |
| `saveToHistory` | `boolean` | `true` | Record the session in `.nopy.history.json`. |
**Returns:** `Promise<NopyResult | undefined>` — `undefined` when cube loading
@@ -507,9 +512,10 @@ displaced stays visible underneath. The trace is never persisted.
```typescript
class Variables {
constructor(env?: TVariables);
constructor(env?: TVariables, globalSecrets?: Iterable<string>);
declareSecrets(cube: string, keys: readonly string[]): void;
declareSchema(cube: string, keys: readonly string[]): void;
isSecret(cube: string, name: string): boolean;
assign(cube: string, origin: Origin, values?: TVariables): void;
@@ -527,6 +533,25 @@ const MASK = '********';
`declareSecrets()` is retroactive as well as prospective, so it does not matter
whether the caller declares before or after the values arrive.
`globalSecrets` is every key *any* manifest declares secret, plus the config's
own `secrets` list. `nopy()` computes it once after `loadCubes()`, before the
first cube resolves, so resolution order cannot change whether a value is treated
as a credential. It does two things:
- `isSecret()` is true for such a key on **every** cube, so a manifest that lists
`PASSWORD` in `schema` and forgets it in `secrets` still gets masking and still
keeps the value out of the session.
- The config `env` stops being broadcast for it. Ordinary `env` keys are seeded
onto every cube — deliberately, since a cube may read a key off `host.data`
that it never declared — but a secret reaches only the cubes whose
`schemaKeys()` include it.
`declareSchema()` is what supplies those keys, and it has an ordering
requirement: call it before anything assigns to the cube, because the first
assignment is what seeds `env`. `BuildContext.resolveCube` calls it immediately
after `declareSecrets()`. It deliberately does not create the cube's bucket
itself.
`persistable()` leaves a secret out entirely rather than masking it, so a replay
sees it as absent and asks for it again. That is why replaying a session whose
cubes declare secrets is interactive even under `-D` — a `-D` replay that would
@@ -585,15 +610,16 @@ const results = await executeDeployCalls(calls, {
});
```
### `outputExecutionPlan(calls, asJson?)`
### `outputExecutionPlan(calls)`
```typescript
outputExecutionPlan(deployCalls); // text
outputExecutionPlan(deployCalls, true); // JSON
outputExecutionPlan(deployCalls);
```
Both forms mask secrets. Note that `executeDeployCalls` calls this without the
second argument, so `--dry-run --json` prints the text plan.
Prints the plan a `--dry-run` shows, with secrets masked. Went from
`(calls, asJson?)` to `(calls)` when `--json` was removed; the JSON branch was
unreachable from the CLI, since `executeDeployCalls` never passed the second
argument.
### `maskCommand(call)` / `maskVariables(call)`
@@ -641,7 +667,7 @@ interface WorkflowResult {
authMethod: string;
username?: string;
password?: string;
isReplay: boolean;
replaySource?: 'file' | 'history'; // undefined on a fresh interactive run
}
interface WorkflowOptions {
@@ -673,10 +699,12 @@ The same, from a session object rather than a path — the `-R` / `-H` path.
```typescript
interface NopySession {
cubes: CubeSession[]; // required
auth: AuthSession; // required
version?: string;
timestamp?: string; // ISO 8601
name?: string;
cubes: CubeSession[];
hosts?: string[];
auth: AuthSession;
env?: TVariables;
}
@@ -692,7 +720,10 @@ interface AuthSession {
}
```
There is no `version` or `timestamp` field, and nothing validates compatibility.
`version` and `timestamp` are stamped on every session nopy writes and demanded
of none it reads — an older file, or a hand-written one, simply lacks them.
Nothing validates compatibility beyond a warning on an unrecognised `version`;
the constant is exported as `SESSION_VERSION`.
A `CubeSession` records every value the cube settled on, whatever its origin —
not just the prompted ones — minus anything the manifest declared a secret. So a
@@ -702,8 +733,7 @@ and `env` happen to say later.
### `saveSession(session, filePath)`
Writes JSON, creating the directory if needed. Note that `nopy()` skips this
during a replay.
Writes JSON, creating the directory if needed.
### `loadSession(filePath)`
@@ -713,7 +743,8 @@ const session = await loadSession('./deployment.session.mjs'); // default expor
```
Dispatches on the extension; `.json` and `.mjs` only. Validates that `cubes` is
an array, that `hosts` (if present) is an array, and that `auth` exists.
an array, that `hosts` (if present) is an array, and that `auth` exists. A
`version` other than `SESSION_VERSION` warns on stderr and loads anyway.
### `createSession(params)`
@@ -725,18 +756,27 @@ const session = createSession({
});
```
Stamps `version` and `timestamp`; pass `timestamp` to override the latter. It
does not derive a `name` — that needs the resolved cube list, which does not
exist yet at the point the session is created, so `nopy()` fills it in at save
time.
### `describeSession(session, timestamp)`
The one-line `date - cubes → hosts` description, shared with the history list so
that the two cannot drift.
### `listSessions(dirPath?)`
Non-recursive; matches **`*.session.json`** and **`*.session.mjs`** only.
A file named `deploy.nopysession.json` will not be listed, though `loadSession`
reads it fine.
Non-recursive; matches `*.nopysession.json`, `*.nopysession.mjs`,
`*.session.json` and `*.session.mjs`.
---
## History Module
Sessions are recorded automatically after a successful non-replay run, into
`.nopy.history.json` in the working directory.
Sessions are recorded automatically, into `.nopy.history.json` in the working
directory, before the deploy commands run — so a failed run is recorded too.
```typescript
const HISTORY_FILE = '.nopy.history.json';
@@ -767,8 +807,10 @@ interface SessionHistory {
| `removeFromHistory(id)` | `boolean` | `false` if the id was not found |
| `formatHistoryList(entries)` | `string` | what `nopy history` prints |
Recording is suppressed for a dry run, a replay, a run that built no deploy
calls, `--no-history`, and `history.autoSave: false` in the config.
Recording is suppressed for a dry run, a print-only run, a `-R`/`-H` replay out
of history, a run that built no deploy calls, `--no-history`, and
`history.autoSave: false` in the config. A `--load-session` run **is** recorded: it is not in history already, and
without the entry `-R` would have nothing to repeat.
---
@@ -785,6 +827,7 @@ interface NopyConfig {
cubeDirs: string[];
cubePackages: CubePackageRef[];
env: TVariables;
secrets?: string[]; // env keys to treat as sensitive that no manifest declares
log?: LogConfig;
history?: HistoryConfig;
execution?: ExecutionConfig;
@@ -1031,7 +1074,7 @@ on `PATH`.
**never throws** — it sits in front of every command the user actually asked
for. Returns `null` immediately when `isUpdateCheckDisabled(env)`:
`NOPY_NO_UPDATE_CHECK` set to anything but `0`/`false`, or `CI` set at all. The
CLI prints it to **stderr**, so `--json` and piped stdout stay clean.
CLI prints it to **stderr**, so a piped `--print-only` stays clean.
### `selfUpdate(options)` → `SelfUpdateResult`
@@ -1062,7 +1105,6 @@ nopy install -l ./sess.json # replay a session file
nopy install -n # dry run — print the plan, execute nothing
nopy install -P # print the built pyinfra commands and exit
nopy install -c # continue after a failure
nopy install -j # JSON output
nopy install --no-history # do not record this run
nopy history # list recorded sessions (alias: h; -j for JSON)
@@ -1167,17 +1209,15 @@ Real behaviour that a reader would otherwise take on trust. Tracked in
- **`logConfigToFlags()` is never consumed.** It is exported and unit-tested, but
nothing feeds its output into the built pyinfra command, so `log.verbosity` and
`log.debug` in `.nopyrc.json` have no effect today.
- **`--json` emits nothing on success.** `jsonOutput` suppresses the banner and
the progress lines, and prints `{success: false, errors}` when cube *loading*
fails. The success path returns `NopyResult` to the caller without printing it,
so a CI job gets pyinfra's inherited stdio and an exit code. `--dry-run --json`
prints the *text* plan.
- **No cycle detection.** Ordering is a side effect of recursion, not a
topological sort. Two mutually dependent cubes overflow the stack.
- **`DeployCall.dependencies` is always `[]`.** The field is populated nowhere;
dependency information lives in the emission order.
- **`ExecutionResult.stdout` / `.stderr` are always `undefined`,** because the
executor inherits stdio rather than capturing it.
executor inherits stdio rather than capturing it. This is also why `install`
has no `--json`: during a run nopy does not own its own stdout, so there is no
stream to put a machine-readable answer on. Use `--print-only` for the plan and
the exit code for the verdict.
- **Hook variables are not schema-validated.** The second argument to a hook is
the effective values as collected. `schema.parse()` runs in exactly one place —
`Cube.getDefaults()`, against `{}` — and prompt input is type-coerced, which is
+2 -2
View File
@@ -314,8 +314,8 @@ Rename one of them, or remove a source from .nopyrc.json.
There is deliberately no override, alias or precedence rule. If two bundles ever
claim the same id they are mutually exclusive, and the fix is upstream.
Surface the source in the interactive picker and in `--json` output so a user can
see where a cube came from before running it.
Surface the source in the interactive picker so a user can see where a cube came
from before running it.
## Phase 4 — `@bitsquare/nopy-cubes`, the authoring package — **done**
+348
View File
@@ -0,0 +1,348 @@
# Requirements and facts
A proposal, not a record. Nothing here is built.
The question it answers: a cube should be able to say *"in order to run, a user
named xyz must exist, with fish as its shell, in these groups"* — and the engine
should check that against the real host rather than blindly running whatever
cube happens to create such a user.
Everything under [What pyinfra actually gives us](#what-pyinfra-actually-gives-us)
was measured against pyinfra **3.5.1**. Everything under
[The proposal](#the-proposal) is design.
## Contents
- [The problem with `dependencies`](#the-problem-with-dependencies)
- [What pyinfra actually gives us](#what-pyinfra-actually-gives-us)
- [The proposal](#the-proposal)
- [Why `facts.py` exposes a function, not a script](#why-factspy-exposes-a-function-not-a-script)
- [Execution model](#execution-model)
- [Sharp edges](#sharp-edges)
- [Measured vs. assumed](#measured-vs-assumed)
---
## The problem with `dependencies`
`manifest.dependencies` conflates two things that are not the same:
- **a requirement** — "this cube needs a user xyz to exist"
- **a remedy** — "therefore run `user:add`"
Today only the remedy is expressible. `user:add` declares
`dependencies: () => ['apt:essentials']`, which means *always run
`apt:essentials`*, on every host, forever, regardless of whether anything it
installs is missing.
### What that actually costs
Be honest about this, because it is smaller than it first looks and it changes
what the feature is for.
pyinfra operations are **already fact-diffed**. `server.user(present=True,
shell=…)` gathers `server.Users` itself and no-ops when the state matches. The
current design is therefore not *incorrect* — it is idempotent. What it costs is:
1. **Prompts.** Resolving `user:add` drags `apt:essentials` into the plan *and
its variables into the prompt sequence*. The user is asked questions about a
cube they never chose. This is the real, visible tax.
2. **Time.** A no-op pyinfra run is still a connection, a fact gather, and an
operation build.
3. **Legibility.** The plan cannot say *"user xyz already exists, skipping"*.
This matters because the obvious objection to the whole feature is *"just write
`if not host.get_fact(…)` inside `deploy.py` — that is idiomatic pyinfra"*. The
answer is that nopy's layer is **planning and prompting**: deciding which cubes
enter the plan and which variables to ask for, before anything runs. A condition
inside `deploy.py` cannot help with either. That is the justification for lifting
facts into the Node runtime at all — and it means the feature buys **legibility
and prompt-avoidance, not correctness**.
## What pyinfra actually gives us
### The `fact` subcommand is not a machine interface
`pyinfra @local fact server.Groups` prints JSON — and prints it to **stderr**,
via `click.echo(jsonify(…), err=True)` in `pyinfra_cli/prints.py:print_fact`,
interleaved with `--> Loading config...` progress lines. Worse,
`_run_fact_operations` wraps each fact in `except PyinfraError: pass`, so the
command **exits 0 whether or not the fact resolved**.
Parsing that stream means fishing a JSON blob out of styled log output and
having no exit code to check. Rejected.
### A deploy file that prints to stdout is clean
pyinfra deploy files execute on the control machine at *build* time, and
`host.get_fact()` is available there — it is exactly how operations do their own
diffing. A file that gathers facts, prints JSON, and declares no operations
works:
```python
import json, sys
from pyinfra import host
from pyinfra.facts.server import Users, Which
users = host.get_fact(Users)
u = users.get(host.data.USER)
print("###NOPY-FACTS###" + json.dumps({
"host": host.name,
"exists": u is not None,
"shell": (u or {}).get("shell"),
"groups": (u or {}).get("groups", []),
}), file=sys.stdout)
```
```
$ pyinfra @local -y --data USER=someone facts.py
EXIT=0
--- STDOUT ---
###NOPY-FACTS###{"host": "@local", "exists": false, "shell": null, "groups": []}
--- STDERR ---
--> Loading config...
…
--> Results:
Operation Hosts Success Error No Change
Grand total - - - -
```
Confirmed properties:
- **stdout is exclusively ours.** Every byte pyinfra emits goes to stderr, which
is the same reason nopy's own logging goes there (`configureLogtape`).
- **`--data` flows in identically** to a deploy script. Parameterising a probe
is free.
- **Zero operations is legal.** `Grand total -`, exit 0.
- **`host.name` is available**, so multi-host output is self-tagging.
- **A custom `FactBase` subclass in a sibling module imports fine** under
`--chdir`, so a cube can ship fact classes pyinfra does not have.
- **An exception exits 1** with a traceback on stderr.
### The trap
**An unreachable host exits 0 and prints nothing.**
```
$ pyinfra nonexistent.invalid.example -y probe.py
nonexistent.invalid.example is neither an inventory file, a (list of) hosts…
EXIT=0
--- STDOUT --- (empty)
```
Absence of a probe result must be a hard failure. It must never be read as
"this host has no requirements to check", which is the shape the bug would take.
The `###NOPY-FACTS###` sentinel exists for this: one line per host is *required*,
and a missing line is an error.
### `server.Users` already covers the motivating case
It returns `shell`, `groups`, `home`, `uid`, `gid`, `comment` and `password` per
user, keyed by name. `user:add` needs almost no custom fact code — see
[Sharp edges](#sharp-edges) for why `password` is a problem.
## The proposal
Split the one idea into two manifest fields, because there are two different
objects: what a cube can **report** about a host (owned by the cube responsible
for that state) and what a cube **demands** (owned by the consumer).
A single `facts:` field cannot be both.
### `provides` — on the cube that owns the state
```js
// cubes/user/add/manifest.mjs
export default Manifest({
id: 'user:add',
provides: {
/** What the probe returns. Validated on the way back in. */
schema: z.object({
exists: z.boolean(),
shell: z.string().nullable(),
groups: z.array(z.string()),
}),
/** Schema keys the probe needs in order to look anything up. */
params: ['USER'],
},
schema: z.object({ /* … unchanged … */ }),
});
```
Plus a `facts.py` in the cube directory, discovered by convention exactly as
`deploy.py` is.
### `requires` — on the consumer
```js
requires: (vars) => [{
cube: 'user:add',
with: { USER: vars.DEPLOY_USER },
expect: z.object({
exists: z.literal(true),
shell: z.literal('/usr/bin/fish'),
groups: z.array(z.string()).refine((g) => g.includes('docker')),
}),
}],
```
Three deliberate choices:
**A requirement names its provider.** If `requires` stated only a predicate, the
engine would have to search the cube space for something that satisfies an
arbitrary Zod schema. That is a planner, and a planner is a research project.
Naming the cube keeps resolution linear and keeps the failure message legible.
**`requires` is `(vars) => …`, like `dependencies` already is.** A requirement
almost always depends on the consumer's own variables — you cannot know *which*
user to check for until the consumer has been asked.
**Zod is the predicate language.** It is already this repo's schema vocabulary,
`z.literal` and `.refine()` cover the cases, and `z.treeifyError()` produces the
failure report for free. The cost is that `.refine()` closures are not
serialisable, so a requirement can never be written into a session — only its
*result* can. See [Sharp edges](#sharp-edges).
### The engine's rule
This is the half of the design that "only run if requirements are fulfilled"
leaves out. When a requirement is **not** met, there are three possible answers:
| | |
| --- | --- |
| **Abort** | Honest, and useless. On a bare host nothing is fulfilled, so every fresh deploy fails. |
| **Skip the cube** | A silent no-op. Dangerous. |
| **Remedy** | Run the named cube. |
It has to be *remedy* — and remedy means "run the cube that provides it", which
is `dependencies` again. That is the actual insight here, and it is a much
smaller change than a parallel subsystem:
> **You do not need a new mechanism. You need dependencies to become conditional.**
So:
- requirement **met** → the provider is *not* scheduled and its variables are
*not* prompted for. This is where the prompt tax disappears.
- requirement **unmet** → the provider is scheduled with `with` applied as
overrides. `with` assigns at origin `param`, which already outranks every
other origin (`default < env < session < prompt < param`), so no new
precedence rule is needed.
## Why `facts.py` exposes a function, not a script
The obvious design is symmetry: `deploy.py` is a script, so `facts.py` is a
script. Reject it.
A recursive resolution over a dependency tree issues one probe per (cube, host,
params). At one pyinfra invocation each, that is one SSH connection each —
seconds apiece, multiplied by the tree. Unaffordable.
But probes are **pure reads with no ordering constraints between them**, which
means every probe for a given host can be gathered in a *single* pyinfra
invocation: one generated driver script that imports each cube's `facts.py`,
calls it with its params, and emits one JSON object per host. One connection per
host per resolution round, instead of one per cube.
A standalone script can only be run alone. A function can be batched:
```python
# cubes/user/add/facts.py
from pyinfra.facts.server import Users
def gather(host, params):
u = host.get_fact(Users).get(params["USER"])
return {
"exists": u is not None,
"shell": (u or {}).get("shell"),
"groups": (u or {}).get("groups", []),
}
```
Consequence: params arrive as one `--data NOPY_PROBES=<json>` blob rather than
per-cube `--data KEY=…`, since a batched run carries several cubes' params at
once. Probe results are memoised per (cube, host, params) in the `BuildContext`,
the same shape as the existing `resolvedCubes` set.
## Execution model
Probes run during **resolution**, before anything has deployed. That has a
consequence worth stating plainly rather than discovering later:
When the plan reads *"user absent → schedule `user:add` → then B"*, B's
requirement was never verified. It was **assumed** met because a remedy was
scheduled. Every tool in this space stops there.
Since the probe already exists, the loop can be closed for nearly nothing:
> **Re-run the provider's probe after it deploys, and fail loudly if the
> requirement still is not met.**
`facts.py` then serves as a post-condition test as well as a precondition check.
This is the single most valuable property of the design — it is strictly more
than Ansible's `when:` offers — and it should be built in from the start rather
than added as a later refinement.
Sketch of the resolution change in `BuildContext.resolveCube`, between steps 3
(`before` hooks) and 4 (dependencies):
1. Collect `manifest.requires?.(currentVars)`.
2. Batch every unmet-so-far probe for this host into one pyinfra run.
3. Parse each result against the provider's `provides.schema`, then against the
consumer's `expect`.
4. For each failure, `resolveCube(req.cube, host, req.with)`.
5. After that provider's deploy call executes, re-probe and assert.
Step 5 does not fit the current shape: `deployCalls` are all built first and
executed later, so a post-condition needs the executor to call back into
probing. That is the one structurally invasive part of this proposal.
## Sharp edges
**`--print-only` purity.** A dry run touches no host today. Probing breaks that
outright. Needs an explicit answer — either a probe-free plan that renders
requirements as unevaluated, or an opt-in flag. Silently connecting during what
the user believes is a dry run is not acceptable.
**Secret leakage.** `server.Users` returns the **encrypted password hash** in
its `password` field. A probe result that reaches history, a plan dump, or a log
line is a credential leak. The existing `secrets` machinery is per-*variable*
and does not apply — facts need their own redaction rule. Easy to miss, hard to
take back.
**Sessions must never persist facts.** Facts are host state at a moment in time;
a replay must re-probe, never restore. Recording them in *history* for
diagnostics is fine and useful. This is also forced by `.refine()` being
unserialisable.
**Sudo.** Many useful facts require `_sudo`, and a probe runs before the deploy
has made any of its own auth decisions. Unresolved — worth a spike.
**Cycles.** `requires` introduces a second edge type into a graph that still has
no cycle detection (see *Known drift* in `CLAUDE.md`). Conditional edges make
"A requires B, B requires A under different params" considerably more reachable
than the current unconditional ones do.
**Naming.** `provides` / `requires` over the original `facts:`, because the
field names should say which side of the relationship they belong to.
## Measured vs. assumed
Measured against pyinfra 3.5.1, on this machine:
- `fact` subcommand writes JSON to stderr and exits 0 on fact failure
- a deploy file's stdout is uncontaminated by pyinfra output
- `--data` reaches a probe as `host.data.KEY`
- a deploy file with zero operations succeeds
- a custom `FactBase` in a sibling module resolves under `--chdir`
- an exception in a deploy file exits 1
- **an unreachable host exits 0 with empty stdout**
Assumed, not yet tested:
- batching several cubes' probes into one driver script works and is meaningfully
cheaper than N invocations — this is the load-bearing cost assumption and
should be the first thing spiked
- `host.get_fact()` accepts `_sudo` from within a probe function
- the post-condition re-probe can be threaded through `executeDeployCalls`
without unpicking the build-then-execute split
+29 -12
View File
@@ -2,9 +2,14 @@
Nopy supports two session file formats: **JSON** and **MJS** (ES Module JavaScript).
The extension is what picks the loader, so a session file has to end in `.json`
or `.mjs`; anything else is refused by name. The `.nopysession.*` names used
throughout are the convention `listSessions()` looks for — `-s` and `-l` accept
any path you give them.
## Supported Formats
### JSON Format (`.session.json`)
### JSON Format (`.nopysession.json`)
Traditional JSON format for session files:
@@ -35,7 +40,7 @@ Traditional JSON format for session files:
- Cannot use dynamic values or computation
- No code reuse or imports
### MJS Format (`.session.mjs`) - **Recommended**
### MJS Format (`.nopysession.mjs`) - **Recommended**
JavaScript module format with full ES Module support:
@@ -172,7 +177,7 @@ export const commonCubes = [
];
```
**my-session.session.mjs:**
**my-session.nopysession.mjs:**
```javascript
import { productionHosts, commonCubes } from './common-config.mjs';
@@ -232,7 +237,7 @@ function generateSession(config) {
};
const content = `export default ${JSON.stringify(session, null, 2)};`;
fs.writeFileSync('generated.session.mjs', content);
fs.writeFileSync('generated.nopysession.mjs', content);
}
// Generate from external configuration
@@ -255,10 +260,10 @@ Both formats are loaded the same way:
import { loadSession } from '@bitsquare/nopy';
// Load JSON
const jsonSession = await loadSession('./my-session.session.json');
const jsonSession = await loadSession('./my-session.nopysession.json');
// Load MJS
const mjsSession = await loadSession('./my-session.session.mjs');
const mjsSession = await loadSession('./my-session.nopysession.mjs');
```
The file extension determines which loader to use.
@@ -267,7 +272,7 @@ The file extension determines which loader to use.
To convert an existing JSON session to MJS:
1. Rename the file from `.session.json` to `.session.mjs`
1. Rename the file from `.nopysession.json` to `.nopysession.mjs`
2. Add `export default` before the configuration object
3. Remove quotes from property keys (optional)
4. Add comments and dynamic values as needed
@@ -304,11 +309,12 @@ Both formats must export/contain an object with this structure:
```typescript
interface NopySession {
version: string; // Session format version
timestamp: string; // ISO timestamp
cubes: CubeSession[]; // Array of cube configurations
hosts: string[]; // Target hosts
auth: AuthSession; // Authentication configuration
cubes: CubeSession[]; // Array of cube configurations — required
auth: AuthSession; // Authentication configuration — required
version?: string; // Session format version, currently "1.0.0"
timestamp?: string; // ISO timestamp
name?: string; // One-line description
hosts?: string[]; // Target hosts
env?: Record<string, any>; // Global environment variables
}
@@ -323,6 +329,17 @@ interface AuthSession {
}
```
Only `cubes` and `auth` are demanded of a session being *read* — the loader
requires what it cannot work without and nothing else, so the sessions in these
examples are all valid, and one written before `version` existed still loads. A
session nopy *writes* always carries `version`, `timestamp` and `name`; a
`version` this build does not recognise produces a warning on stderr and loads
anyway.
`method: 'ssh'` is the third value and the one no prompt produces: it means the
connector handles authentication and nopy supplies no credential. Every
`@vagrant/` and `@docker/` host gets it.
A session nopy *writes* holds, per cube, every value that cube ran with — what
was typed, what came from `.nopyrc.json`, what a dependency supplied, and what
fell through to the schema's `.default()`. Two things are deliberately absent and
+35 -2
View File
@@ -1,7 +1,9 @@
# Vagrant
`vagrant ssh-config` to find the SSH port of the machine
`vagrant status --machine-readable` will be executed by pyinfra to get information about available VMs
A Vagrant box is the cheapest way to run a cube against a real machine you can
throw away afterwards.
## The Vagrantfile
```ruby
@@ -19,3 +21,34 @@ Vagrant.configure("2") do |config|
end
```
## Naming the machine to nopy
The host string is `@vagrant/<name>`, where `<name>` is what `config.vm.define`
declared — `nopytestvm` above. It is pyinfra's connector syntax, not nopy's, and
it is the same shape as `@docker/<container-or-image>`.
Two ways to get there. Either pick `vagrant` in the host prompt and answer the
follow-up with the machine name, which is what builds the string for you, or put
it in `.nopyrc.json` so it appears in the list directly:
```json
{
"hosts": ["@vagrant/nopytestvm"],
"cubePackages": ["@bitsquare/nopy-cubes-core"]
}
```
pyinfra runs `vagrant status --machine-readable` to find the available machines
and `vagrant ssh-config` for the SSH port, so `vagrant` has to be on `PATH` and
the box has to be `up` before a deploy.
## Cleaning up
```sh
vagrant halt # stop it, keep the disk
vagrant destroy -f # delete it — the next `vagrant up` is a fresh box
```
`destroy` is the one to use between test runs of a cube that is not idempotent:
re-running against a half-configured box tests something other than the cube.
+2
View File
@@ -1,5 +1,7 @@
{
"version": "1.0.0",
"name": "Example Deployment Session",
"timestamp": "2026-07-30T09:15:00.000Z",
"cubes": [
{
"key": "apt:essentials",
+58 -24
View File
@@ -7,6 +7,7 @@ import type { Cube, CubeVariables, HookContext } from '@bitsquare/nopy-cubes';
import { getLogger } from '@logtape/logtape';
import type { Variables } from '../nopy.common.js';
import type { NopyConfig } from '../nopy.config.js';
import { NopyUsageError } from '../nopy.errors.js';
import type { DeployCall } from '../nopy.executor.js';
import { VariableAssignment } from '../nopy.prompts.js';
import type { CubeSession, NopySession } from '../nopy.session.js';
@@ -45,21 +46,33 @@ export class BuildContext {
}
/**
* Fails a non-interactive run that cannot fill a required variable.
* Fails a run that cannot fill a required variable.
*
* Without this the cube would be deployed with the key simply absent from
* `--data`, and the deploy script would read `None` off `host.data`.
* `--data`, and the deploy script would read `None` off `host.data` — against
* the documented guarantee that every schema key reaches it.
*
* Runs on the interactive path too, not only under `--use-defaults`. A prompt
* is not proof of an answer: a terminal that misreports its size renders an
* empty form and submits `{}` without the user seeing a field, which is
* exactly how this was found.
*/
private assertVariablesComplete(cube: Cube): void {
const missing = this.missingRequired(cube);
if (missing.length === 0) return;
const [one, them] =
missing.length === 1 ? ['has no default value', 'it'] : ['have no default values', 'them'];
throw new Error(
`Cube "${cube.id}" cannot run with --use-defaults: ${missing.join(', ')} ${one}. ` +
`Set ${them} under "env" in .nopyrc.json, pass ${them} from a dependency, ` +
'or drop --use-defaults to be prompted.'
const list = missing.join(', ');
const them = missing.length === 1 ? 'it' : 'them';
const have = missing.length === 1 ? 'has no default value' : 'have no default values';
throw new NopyUsageError(
this.options.useDefaults
? `Cube "${cube.id}" cannot run with --use-defaults: ${list} ${have}. ` +
`Set ${them} under "env" in .nopyrc.json, pass ${them} from a dependency, ` +
'or drop --use-defaults to be prompted.'
: `Cube "${cube.id}" is missing ${list}. Nothing supplied ${them} — the form may have ` +
`been submitted empty. Re-run and fill ${them} in, or set ${them} under "env" ` +
'in .nopyrc.json.'
);
}
@@ -81,23 +94,39 @@ export class BuildContext {
if (gaps.length === 0) return;
if (this.options.useDefaults) {
throw new Error(
`Cube "${cube.id}" cannot be replayed with --use-defaults: ${gaps.join(', ')} ` +
'would have to be entered. Secrets are never recorded in a session. ' +
'Replay without --use-defaults, or set the values under "env" in .nopyrc.json.'
);
// A gap is only a gap if nothing outside the session filled it. `env` and
// `param` both say deliberately what the value is, which is exactly what
// the old message told the user to do — and then failed anyway.
//
// `default` is not accepted here. The session dropped the secret on
// purpose, so falling through to a manifest default would deploy a
// different credential than the run being replayed, without saying so.
const unsatisfied = gaps.filter((key) => {
const origin = this.variables.of(cube.id, key)?.origin;
return origin !== 'env' && origin !== 'param';
});
if (unsatisfied.length > 0) {
const them = unsatisfied.length === 1 ? 'it' : 'them';
const secret = unsatisfied.some((key) => cube.secrets.includes(key));
throw new NopyUsageError(
`Cube "${cube.id}" cannot be replayed with --use-defaults: ` +
`${unsatisfied.join(', ')} would have to be entered. ` +
(secret ? 'Secrets are never recorded in a session. ' : '') +
`Set ${them} under "env" in .nopyrc.json` +
(secret ? ' (a schema default is not accepted for a secret)' : '') +
`, pass ${them} from a dependency, or replay without --use-defaults.`
);
}
return;
}
log.debug('Filling session gaps', { cubeId: cube.id, gaps });
await VariableAssignment(cube, this.variables, { keys: gaps });
// A cancelled form leaves the run short of a value it cannot invent.
const stillMissing = this.missingRequired(cube);
if (stillMissing.length > 0) {
throw new Error(
`Cube "${cube.id}" is missing ${stillMissing.join(', ')} and cannot be deployed.`
);
}
// A form that resolved is not a form that was answered — same check, and
// the same reason for it, as the interactive path.
this.assertVariablesComplete(cube);
}
/**
@@ -110,15 +139,19 @@ export class BuildContext {
): Promise<void> {
const cube = this.allCubes[cubeId];
if (!cube) {
throw new Error(`Cube not found: ${cubeId}`);
throw new NopyUsageError(`Cube not found: ${cubeId}`);
}
log.debug('Resolving cube', { cubeId, host });
// 1. Declare secrets, then assign overrides and defaults. Declaring first
// means even the config `env` seeded on the cube's first assignment is
// already marked, so nothing reaches a session or a log unredacted.
// 1. Declare secrets and schema, then assign overrides and defaults. Both
// declarations have to come first: the cube's first assignment is what
// seeds the config `env` onto it, and by then it must already be known
// which of those keys are secret (so nothing reaches a session or a log
// unredacted) and which the cube actually declares (so a secret it does
// not declare is never seeded at all).
this.variables.declareSecrets(cubeId, cube.secrets);
this.variables.declareSchema(cubeId, cube.schemaKeys());
if (Object.keys(overrides).length > 0) {
this.variables.assign(cubeId, 'param', overrides);
}
@@ -136,6 +169,7 @@ export class BuildContext {
this.assertVariablesComplete(cube);
} else {
await VariableAssignment(cube, this.variables);
this.assertVariablesComplete(cube);
}
const currentVars = this.variables.get(cubeId);
+19 -24
View File
@@ -8,6 +8,7 @@
import { createRequire } from 'node:module';
import { Command } from 'commander';
import { loadConfig } from './nopy.config.js';
import { reportError } from './nopy.errors.js';
import { exitWithFarewell, installGracefulExit, isCancellation } from './nopy.exit.js';
import {
clearHistory,
@@ -33,8 +34,8 @@ const { version, buildInfo } = createRequire(import.meta.url)('../package.json')
const versionLabel = buildInfo?.commit ? `${version} (${buildInfo.commit})` : version;
/**
* Prints the update hint to stderr, so it never lands in `--json` output or in
* a `--print-only` command list being piped somewhere.
* Prints the update hint to stderr, so it never lands in a `--print-only`
* command list being piped somewhere.
*/
async function printUpdateNotice(): Promise<void> {
const notice = await updateNotice({ currentVersion: version });
@@ -67,6 +68,9 @@ Examples:
$ nopy history List all saved sessions
$ nopy clear-history Clear session history
Every flag above belongs to 'install', the default command — 'nopy -R' is
'nopy install -R'. Run 'nopy install --help' for the full list.
Session Replay:
Sessions are automatically saved to history after each deployment.
Use 'nopy history' to see available sessions and their IDs.
@@ -88,16 +92,19 @@ program
.option('-n, --dry-run', 'Show execution plan without running')
.option('-P, --print-only', 'Print deploy commands and exit (no execution)')
.option('-c, --continue-on-error', 'Continue executing after failures')
.option('-j, --json', 'Output results as JSON')
.option('--no-history', 'Do not save this session to history')
.action(async (options) => {
await printUpdateNotice();
// Loaded lazily so that --help/--version work outside a configured project.
const execConfig = loadConfig().execution ?? {};
const continueOnError = options.continueOnError ?? execConfig.continueOnError ?? false;
try {
// Loaded lazily so that --help/--version work outside a configured
// project — and inside the try, so that "no .nopyrc.json here" is
// reported by `reportError` rather than escaping as an unhandled
// rejection and printing node's own stack. It is the likeliest first-run
// mistake there is.
const execConfig = loadConfig().execution ?? {};
const continueOnError = options.continueOnError ?? execConfig.continueOnError ?? false;
// Handle session replay
const loadSessionPath = options.loadSession;
let sessionToReplay: { session: import('./nopy.session.js').NopySession } | undefined;
@@ -109,7 +116,9 @@ program
process.exit(1);
}
sessionToReplay = lastEntry;
console.log(`Repeating: ${lastEntry.name}\n`);
// stderr, like everything nopy says about itself — `-R --print-only` has
// to leave stdout to the commands.
console.error(`Repeating: ${lastEntry.name}\n`);
} else if (options.history) {
const entry = getSessionById(options.history);
if (!entry) {
@@ -118,7 +127,7 @@ program
process.exit(1);
}
sessionToReplay = entry;
console.log(`Running: ${entry.name}\n`);
console.error(`Running: ${entry.name}\n`);
}
const result = await nopy({
@@ -130,7 +139,6 @@ program
dryRun: options.dryRun,
printOnly: options.printOnly,
continueOnError,
jsonOutput: options.json,
saveToHistory: options.history !== false && !options.dryRun,
});
@@ -144,20 +152,7 @@ program
// the process-level handler.
if (isCancellation(error)) exitWithFarewell();
if (options.json) {
console.log(
JSON.stringify(
{
success: false,
error: error instanceof Error ? error.message : String(error),
},
null,
2
)
);
} else {
console.error('Error:', error instanceof Error ? error.message : error, error);
}
reportError(error);
process.exit(1);
}
});
+62 -2
View File
@@ -114,8 +114,21 @@ export class Variable {
export class Variables {
private readonly store: Record<string, Record<string, Variable>> = {};
private readonly secrets: Record<string, Set<string>> = {};
private readonly schemas: Record<string, Set<string>> = {};
private readonly globalSecrets: Set<string>;
constructor(readonly env: TVariables = {}) {}
/**
* @param env - the `env` block of the merged config, seeded onto every cube
* @param globalSecrets - every key *any* manifest declares secret, plus the
* config's own `secrets` list. Known up front, before the first cube
* resolves, so it does not depend on resolution order.
*/
constructor(
readonly env: TVariables = {},
globalSecrets: Iterable<string> = []
) {
this.globalSecrets = new Set(globalSecrets);
}
/**
* Marks keys of one cube as holding secrets.
@@ -132,8 +145,30 @@ export class Variables {
}
}
/**
* Records which keys a cube's schema declares.
*
* Only {@link bucket} reads this, and only to decide whether a globally
* declared secret may be seeded from `env`. Call it before anything assigns to
* the cube — it deliberately does not create the bucket itself, because
* creating it is what seeds `env`.
*/
declareSchema(cube: string, keys: readonly string[]): void {
this.schemas[cube] ??= new Set<string>();
const declared = this.schemas[cube];
for (const key of keys) declared.add(key);
}
/**
* Whether a key is sensitive for a cube.
*
* True for a key the cube's own manifest declared, and also for one *another*
* manifest declared: a value that is a secret anywhere is a secret everywhere
* it lands. That covers the manifest that lists `PASSWORD` in `schema` and
* forgets it in `secrets`.
*/
isSecret(cube: string, name: string): boolean {
return this.secrets[cube]?.has(name) ?? false;
return (this.secrets[cube]?.has(name) ?? false) || this.globalSecrets.has(name);
}
/** Records values for one cube, all at the same origin. */
@@ -176,6 +211,24 @@ export class Variables {
return values;
}
/**
* The config's `env` block minus anything declared secret — what a session's
* own `env` records.
*
* A session copies `env` verbatim for reference, which quietly undid
* {@link persistable}: a credential declared in `.nopyrc.json` was kept out of
* every cube's `variables` and then written to the same file one key higher up,
* in plaintext, along with a copy in `.nopy.history.json`. Same rule as
* `persistable`, applied to the same file.
*/
persistableEnv(): TVariables {
const values: TVariables = {};
for (const [name, value] of Object.entries(this.env)) {
if (!this.globalSecrets.has(name)) values[name] = value;
}
return values;
}
private create(cube: string, name: string, first: Assignment): Variable {
const variable = new Variable(cube, name, first);
variable.redacted = this.isSecret(cube, name);
@@ -189,6 +242,12 @@ export class Variables {
* rather than a parallel bag merged in at read time. That is what lets it
* carry an origin, show up in the trace, and lose to a prompt by the same rule
* as everything else.
*
* One key is held back: a **secret**, on a cube whose schema does not mention
* it. Broadcasting is otherwise load-bearing — a cube may legitimately read a
* key off `host.data` that it never declared — but a credential does not
* belong on the command line of every unrelated cube in the run, where nothing
* masks it because that cube never declared it sensitive.
*/
private bucket(cube: string): Record<string, Variable> {
const existing = this.store[cube];
@@ -197,6 +256,7 @@ export class Variables {
const bucket: Record<string, Variable> = {};
this.store[cube] = bucket;
for (const [name, value] of Object.entries(this.env)) {
if (this.globalSecrets.has(name) && !this.schemas[cube]?.has(name)) continue;
bucket[name] = this.create(cube, name, { value, origin: 'env' });
}
return bucket;
+11 -2
View File
@@ -6,6 +6,7 @@
import fs from 'node:fs';
import path from 'node:path';
import type { TVariables } from './nopy.common.js';
import { NopyUsageError } from './nopy.errors.js';
/**
* Log verbosity levels for pyinfra output
@@ -102,6 +103,14 @@ export interface NopyConfig {
cubePackages: CubePackageRef[];
/** Global environment variables */
env: TVariables;
/**
* `env` keys to treat as sensitive even though no manifest says so.
*
* A manifest's own `secrets` list already covers the cubes that declare the
* key. This is for the value no cube declares at all — a token a hook reads,
* say — which would otherwise be broadcast and printed in the clear.
*/
secrets?: string[];
/** Logging configuration */
log?: LogConfig;
/** Session history configuration */
@@ -317,7 +326,7 @@ export function loadConfig(): NopyConfig {
const configPaths = findConfigFiles();
if (configPaths.length === 0) {
throw new Error(
throw new NopyUsageError(
`No ${CONFIG_FILENAME} found. Create one in your project directory or any parent directory.`
);
}
@@ -334,7 +343,7 @@ export function loadConfig(): NopyConfig {
config = mergeConfigs(config, resolvedConfig);
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
throw new Error(`Failed to load config ${configPath}: ${message}`);
throw new NopyUsageError(`Failed to load config ${configPath}: ${message}`);
}
}
+50
View File
@@ -0,0 +1,50 @@
/**
* The errors that are the user's to fix.
* @module nopy.errors
*/
/**
* A run that failed for a reason the user can act on: no config file, a cube
* that does not exist, a required variable nothing supplied, a session file
* that will not load.
*
* The point is the *presentation*, not the control flow — nothing catches this
* to recover. A stack trace through `dist/` says nothing useful about a missing
* `.nopyrc.json`, and printing one invites the reader to look for a bug in nopy
* instead of a typo in their project. The CLI prints the message alone and keeps
* the stack behind `NOPY_DEBUG`.
*
* Mirrors keyman's `UsageError` deliberately: the two CLIs are kept in step on
* how they fail for the same reason their update modules are duplicated rather
* than shared.
*/
export class NopyUsageError extends Error {
constructor(message: string) {
super(message);
this.name = 'NopyUsageError';
}
}
/**
* Reports a failed run in as many lines as it deserves.
*
* A {@link NopyUsageError} prints as one line: it is something the reader can
* fix, and three frames into `dist/` say nothing about a missing `.nopyrc.json`
* except that it looks like a crash in nopy rather than a typo in the project.
* Everything else keeps its stack, because an unexpected failure is exactly when
* one is worth having. `NOPY_DEBUG` forces it for both.
*
* Lives here rather than in `nopy.cli.ts` because the CLI is excluded from
* coverage — it is argv wiring, and this is a decision.
*/
export function reportError(error: unknown): void {
const message = error instanceof Error ? error.message : String(error);
console.error(`Error: ${message}`);
const stack = error instanceof Error ? error.stack : undefined;
const wanted = process.env.NOPY_DEBUG || !(error instanceof NopyUsageError);
if (wanted && stack) console.error(stack);
else if (!process.env.NOPY_DEBUG) console.error('Set NOPY_DEBUG=1 for the full stack trace.');
}
+1 -13
View File
@@ -143,20 +143,8 @@ async function executeCall(call: DeployCall): Promise<ExecutionResult> {
* Outputs the execution plan without running (dry run)
*
* @param calls - Array of deployment calls
* @param asJson - Output as JSON instead of text
*/
export function outputExecutionPlan(calls: DeployCall[], asJson?: boolean): void {
if (asJson) {
const plan = calls.map((call) => ({
cube: call.cube,
host: call.host,
command: maskCommand(call),
variables: maskVariables(call),
}));
console.log(JSON.stringify({ plan }, null, 2));
return;
}
export function outputExecutionPlan(calls: DeployCall[]): void {
console.log('\n=== Execution Plan (Dry Run) ===\n');
for (let i = 0; i < calls.length; i++) {
+1 -1
View File
@@ -74,7 +74,7 @@ export function restoreTerminal(): void {
* Says goodbye and leaves.
*
* The farewell goes to **stderr**, for the same reason the update hint does:
* `--json` and `--print-only` stay machine-readable no matter how the run ends.
* `--print-only` stays machine-readable no matter how the run ends.
*
* `process.exit` rather than letting the loop drain, because the prompt that
* was cancelled is still holding stdin — after the teardown above threw, its
+2 -31
View File
@@ -5,7 +5,7 @@
import fs from 'node:fs';
import path from 'node:path';
import type { NopySession } from './nopy.session.js';
import { describeSession, type NopySession } from './nopy.session.js';
/** Default number of sessions to keep in history */
export const DEFAULT_HISTORY_SIZE = 10;
@@ -72,35 +72,6 @@ export function saveHistory(history: SessionHistory): void {
fs.writeFileSync(historyPath, JSON.stringify(history, null, 2), 'utf-8');
}
/**
* Generates a history entry name from session data
*
* Format: "YYYY-MM-DD HH:mm - cube1, cube2, ..."
*
* @param session - The session to name
* @param timestamp - ISO timestamp
* @returns Human-readable name
*/
function generateEntryName(session: NopySession, timestamp: string): string {
const date = new Date(timestamp);
const dateStr = date.toLocaleString('en-US', {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
hour12: false,
});
const cubeNames = session.cubes.map((c) => c.key).join(', ');
const truncatedCubes = cubeNames.length > 40 ? `${cubeNames.substring(0, 37)}...` : cubeNames;
const hosts = session.hosts?.join(', ') || 'no host';
const truncatedHosts = hosts.length > 20 ? `${hosts.substring(0, 17)}...` : hosts;
return `${dateStr} - ${truncatedCubes} → ${truncatedHosts}`;
}
/**
* Generates a unique ID for a history entry
*/
@@ -124,7 +95,7 @@ export function addToHistory(
const entry: HistoryEntry = {
id: generateEntryId(),
name: generateEntryName(session, timestamp),
name: describeSession(session, timestamp),
timestamp,
session,
};
+52 -18
View File
@@ -15,11 +15,16 @@ import {
summarizeResults,
} from './nopy.executor.js';
import { addToHistory, DEFAULT_HISTORY_SIZE } from './nopy.history.js';
import { type NopySession, saveSession } from './nopy.session.js';
import { describeSession, type NopySession, SESSION_VERSION, saveSession } from './nopy.session.js';
import { runWorkflow } from './nopy.workflow.js';
/**
* Configures the logtape logger for console output
* Configures the logtape logger for console output.
*
* **stderr**, deliberately. stdout carries the deploy commands and pyinfra's own
* output; everything nopy says about itself goes to stderr, so `--print-only`
* can be piped somewhere. The sink used to write to stdout and was held back
* only by `--json`, which never worked and is gone.
*/
function configureLogtape(): void {
configure({
@@ -31,7 +36,7 @@ function configureLogtape(): void {
if (typeof formatted === 'string') {
const msg = formatted.replace(/\r?\n$/, '');
const props = record.properties as Record<string, unknown>;
console.log(msg, ...Object.values(props));
console.error(msg, ...Object.values(props));
}
};
})(),
@@ -55,7 +60,7 @@ function configureLogtape(): void {
configureLogtape();
/**
* Prints the active configuration summary
* Prints the active configuration summary — to stderr, see {@link configureLogtape}.
*/
function printActiveConfig(
config: import('./nopy.config.js').NopyConfig,
@@ -92,7 +97,7 @@ function printActiveConfig(
}
lines.push('');
console.log(lines.join('\n'));
console.error(lines.join('\n'));
}
/**
@@ -107,7 +112,6 @@ export interface NopyOptions {
dryRun?: boolean;
printOnly?: boolean;
continueOnError?: boolean;
jsonOutput?: boolean;
saveToHistory?: boolean;
}
@@ -138,24 +142,30 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
dryRun = false,
printOnly = false,
continueOnError = false,
jsonOutput = false,
saveToHistory = true,
} = opts;
const log = getLogger(['nopy']);
const config = loadConfig();
if (!jsonOutput && !replaySession && !loadSessionPath) {
if (!replaySession && !loadSessionPath) {
printActiveConfig(config, { continueOnError });
}
const { cubes, errors } = await loadCubes();
const variables = new Variables(config.env);
// Every key any manifest calls a secret, plus the config's own list. Computed
// before the first cube resolves, so which cube happens to run first cannot
// change whether a credential is treated as one.
const declaredSecrets = new Set([
...Object.values(cubes).flatMap((cube) => cube.secrets),
...(config.secrets ?? []),
]);
const variables = new Variables(config.env, declaredSecrets);
if (errors.length > 0) {
log.error('Errors found during cube loading:');
for (const error of errors) log.error(error);
if (jsonOutput) console.log(JSON.stringify({ success: false, errors }, null, 2));
return undefined;
}
@@ -180,7 +190,7 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
},
{
useDefaults,
isSessionReplay: workflow.isReplay,
isSessionReplay: workflow.replaySource !== undefined,
}
);
@@ -190,17 +200,43 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
}
}
// The default name needs the resolved cube list, which does not exist until
// the build has run — so it is filled in here rather than in `createSession`,
// and only when nothing supplied one. `version` sits before the spread so that
// a replayed session keeps whatever its file declared; a hand-written session
// that declared none of the three gets all three.
const timestamp = workflow.session.timestamp ?? new Date().toISOString();
const sessionForSaving: NopySession = {
version: SESSION_VERSION,
...workflow.session,
timestamp,
cubes: context.cubeSessions,
env: config.env,
// Not `config.env` — a declared secret in there would be written to the
// session file in plaintext, one key above the `variables` it was carefully
// kept out of.
env: variables.persistableEnv(),
};
sessionForSaving.name ??= describeSession(sessionForSaving, timestamp);
if (saveSessionPath && !workflow.isReplay) {
// Saved on a replay too: the resolved cube set is exactly what was asked for,
// and a session written from a replay is no less valid than one written from a
// fresh run. The old `!isReplay` guard made `nopy install -R -s out.json` exit
// 0 having written nothing.
if (saveSessionPath) {
saveSession(sessionForSaving, saveSessionPath);
}
if (saveToHistory && !dryRun && !workflow.isReplay && context.deployCalls.length > 0) {
// A `-R`/`-H` replay is already in history and re-recording it would push the
// original out of the list. A `--load-session` run is not in history at all,
// so unless it is recorded here, `nopy history` reports nothing afterwards and
// `-R` has nothing to repeat.
const recordable = workflow.replaySource !== 'history';
// `--print-only` is excluded for the same reason `--dry-run` is: neither
// deployed anything, and history is what `-R` repeats. Recording a run that
// never happened made `nopy install -P` — the safe look-before-you-leap flag —
// silently displace the last real deployment at the head of the list.
if (saveToHistory && !dryRun && !printOnly && recordable && context.deployCalls.length > 0) {
const historySize = config.history?.maxSessions ?? DEFAULT_HISTORY_SIZE;
if (config.history?.autoSave !== false) {
addToHistory(sessionForSaving, historySize);
@@ -224,10 +260,8 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
dryRun,
continueOnError,
onProgress: (result, completed, total) => {
if (!jsonOutput) {
const status = result.success ? '✓' : '✗';
log.info(`[${completed}/${total}] ${status} ${result.cube} -> ${result.host}`);
}
const status = result.success ? '✓' : '✗';
log.info(`[${completed}/${total}] ${status} ${result.cube} -> ${result.host}`);
},
});
+58 -18
View File
@@ -17,6 +17,45 @@ interface CubeChoice {
message: string;
}
/** Floor for a terminal that reports a size no prompt could render into. */
const MIN_ROWS = 24;
const MIN_COLS = 80;
/**
* The window size to hand an enquirer prompt, never smaller than {@link MIN_ROWS}.
*
* Load-bearing, not cosmetic. enquirer derives how many choices are visible from
* its height, and `utils.height` (`lib/utils.js:80`) computes a sane fallback and
* then throws it away:
*
* ```js
* let rows = (stream && stream.rows) ? stream.rows : fallback;
* if (stream && typeof stream.getWindowSize === 'function') {
* rows = stream.getWindowSize()[1]; // unconditional
* }
* ```
*
* A TTY always has `getWindowSize`, so a terminal reporting 0 rows — some CI
* pseudo-terminals, `script -q`, an editor terminal mid-startup — yields
* `height: 0`, `Math.min(limit, 0)` choices, and a form that renders nothing and
* submits `{}`. Passing `rows` bypasses that: `prompt.js:396` reads
* `this.options.rows || utils.height(...)`, so the broken function never runs.
*
* Measured on a 0×0 pty: without this the four-field form returns `{}`; with it,
* every field. No effect on a terminal that reports its size honestly.
* enquirer 2.4.1 is its final release, so the bug is not going to be fixed
* upstream.
*/
function terminalSize(out: NodeJS.WriteStream = process.stdout): {
rows: number;
columns: number;
} {
return {
rows: Math.max(out.rows || 0, MIN_ROWS),
columns: Math.max(out.columns || 0, MIN_COLS),
};
}
/**
* Fuzzy-filters the cube list against what the user has typed so far.
*
@@ -52,8 +91,8 @@ export async function CubeSelection(
// Clear terminal and move cursor to top
process.stdout.write('\x1B[2J\x1B[0f');
const terminalHeight = process.stdout.rows || 24;
const pageSize = Math.max(10, terminalHeight - 5);
const size = terminalSize();
const pageSize = Math.max(10, size.rows - 5);
console.log('\n Cube Selection\n');
console.log(' Type to filter • Space to select • Enter to confirm\n');
@@ -65,14 +104,15 @@ export async function CubeSelection(
multiple: true,
choices: cubeChoices,
suggest: suggestCubes,
...size,
});
try {
return { selectedCubes: await prompt.run() };
} catch {
// User cancelled
return { selectedCubes: [] };
}
// Deliberately no catch. Swallowing a cancellation here used to return an
// empty selection, which is indistinguishable from "the user picked nothing"
// and let the run carry on to deploy zero cubes. Both ways out now travel:
// a cancellation to `isCancellation` at the CLI boundary, anything else as
// the failure it is.
return { selectedCubes: await prompt.run() };
}
export async function AuthSelection(useAuthKey?: boolean): Promise<{
@@ -239,17 +279,17 @@ export async function VariableAssignment<S extends AnyObjectSchema>(
name: 'variables',
message: `[${cube.id}] ${cube.name}\n (↑↓ navigate, Enter to submit)`,
choices,
...terminalSize(),
});
try {
const result = await form.run();
const coercedResult: Record<string, any> = {};
for (const [key, value] of Object.entries(result)) {
const zodType = schema[key];
coercedResult[key] = zodType ? coerceValue(value, zodType) : value;
}
variables.assign(cube.id, 'prompt', coercedResult);
} catch {
// User cancelled
// Deliberately no catch — see `CubeSelection`. A cancelled form used to be
// swallowed here, leaving the cube short of values only the user could give
// and the run continuing as though the form had succeeded.
const result = await form.run();
const coercedResult: Record<string, any> = {};
for (const [key, value] of Object.entries(result)) {
const zodType = schema[key];
coercedResult[key] = zodType ? coerceValue(value, zodType) : value;
}
variables.assign(cube.id, 'prompt', coercedResult);
}
+85 -7
View File
@@ -6,6 +6,7 @@
import fs from 'node:fs';
import path from 'node:path';
import type { TVariables } from './nopy.common.js';
import { NopyUsageError } from './nopy.errors.js';
/**
* Primitive value types that can be stored in session variables
@@ -31,7 +32,13 @@ export interface CubeSession {
* Authentication configuration for a session
*/
export interface AuthSession {
/** Authentication method */
/**
* Authentication method.
*
* `ssh` is not a third kind of credential — it means the connector owns
* authentication and nopy supplies none. It is what an `@vagrant/` or
* `@docker/` host gets, and nothing prompts for it.
*/
method: 'ssh-key' | 'password' | 'ssh';
/** Username for authentication (password auth only) */
username?: string;
@@ -40,8 +47,20 @@ export interface AuthSession {
/**
* Complete session configuration
*
* Everything but `cubes` and `auth` is optional, because a hand-written session
* is a first-class one — the loader requires exactly what it cannot work without.
* `version`, `timestamp` and `name` are stamped on every session nopy writes and
* never demanded of one it reads.
*/
export interface NopySession {
/**
* Format version of the file. Absent on every session written before this was
* stamped, and on most hand-written ones.
*/
version?: string;
/** ISO 8601 time the session was created */
timestamp?: string;
/** Optional session name */
name?: string;
/** Array of cube configurations */
@@ -54,6 +73,40 @@ export interface NopySession {
env?: TVariables;
}
/**
* The format version stamped into every session nopy writes.
*
* There is one, and nothing yet reads it to decide anything — it exists so that
* a future change to the shape can tell an old file from a new one, which is
* impossible after the fact.
*/
export const SESSION_VERSION = '1.0.0';
/**
* A one-line description of a session: `YYYY-MM-DD HH:mm - cubes → hosts`.
*
* Shared with the history list, which is where the format comes from — the two
* name the same thing and there is no reason for them to disagree.
*/
export function describeSession(session: NopySession, timestamp: string): string {
const dateStr = new Date(timestamp).toLocaleString('en-US', {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
hour12: false,
});
const cubeNames = session.cubes.map((c) => c.key).join(', ');
const truncatedCubes = cubeNames.length > 40 ? `${cubeNames.substring(0, 37)}...` : cubeNames;
const hosts = session.hosts?.join(', ') || 'no host';
const truncatedHosts = hosts.length > 20 ? `${hosts.substring(0, 17)}...` : hosts;
return `${dateStr} - ${truncatedCubes} → ${truncatedHosts}`;
}
/**
* Saves a session to a JSON file
*
@@ -129,7 +182,7 @@ function loadSessionFromJSON(filePath: string): NopySession {
*/
export async function loadSession(filePath: string): Promise<NopySession> {
if (!fs.existsSync(filePath)) {
throw new Error(`Session file not found: ${filePath}`);
throw new NopyUsageError(`Session file not found: ${filePath}`);
}
const ext = path.extname(filePath);
@@ -140,23 +193,44 @@ export async function loadSession(filePath: string): Promise<NopySession> {
} else if (ext === '.json') {
session = loadSessionFromJSON(filePath);
} else {
throw new Error(`Unsupported session file format: ${ext}. Use .json or .mjs`);
throw new NopyUsageError(`Unsupported session file format: ${ext}. Use .json or .mjs`);
}
// Validate required fields
if (!session.cubes || !Array.isArray(session.cubes)) {
throw new Error('Invalid session format: missing or invalid "cubes" field');
throw new NopyUsageError('Invalid session format: missing or invalid "cubes" field');
}
if (session.hosts && !Array.isArray(session.hosts)) {
throw new Error('Invalid session format: invalid "hosts" field');
throw new NopyUsageError('Invalid session format: invalid "hosts" field');
}
if (!session.auth) {
throw new Error('Invalid session format: missing "auth" field');
throw new NopyUsageError('Invalid session format: missing "auth" field');
}
// A version this build does not know is a warning, never a refusal: the file
// may well still load, and a session is often the only record of a deployment.
// A missing version says nothing at all — it predates the stamp.
if (session.version !== undefined && session.version !== SESSION_VERSION) {
console.error(
`Warning: session "${filePath}" declares version ${session.version}; ` +
`this build writes ${SESSION_VERSION}. Loading it anyway.`
);
}
return session;
}
/**
* Suffixes {@link listSessions} recognises.
*
* `.nopysession.*` is the documented name and the one the README's examples use;
* it was not matched at all, because `wild.nopysession.json` does not end in
* `.session.json` — the dot before `session` is part of the suffix. The shorter
* pair stays recognised: `saveSession` writes whatever path it is given, so
* files under the old name exist and there is no reason to stop finding them.
*/
const SESSION_SUFFIXES = ['.nopysession.json', '.nopysession.mjs', '.session.json', '.session.mjs'];
/**
* Lists all session files in a directory
*
@@ -170,7 +244,7 @@ export function listSessions(dirPath: string = process.cwd()): string[] {
const files = fs.readdirSync(dirPath);
return files
.filter((file) => file.endsWith('.session.json') || file.endsWith('.session.mjs'))
.filter((file) => SESSION_SUFFIXES.some((suffix) => file.endsWith(suffix)))
.map((file) => path.join(dirPath, file));
}
@@ -186,8 +260,12 @@ export function createSession(params: {
hosts: string[];
auth: AuthSession;
env?: TVariables;
/** Overrides the creation time; for tests, and for re-stamping a replay. */
timestamp?: string;
}): NopySession {
return {
version: SESSION_VERSION,
timestamp: params.timestamp ?? new Date().toISOString(),
name: params.name,
cubes: params.cubes,
hosts: params.hosts,
+15 -5
View File
@@ -35,8 +35,18 @@ export interface WorkflowResult {
username?: string;
/** Password if applicable */
password?: string;
/** Whether this is a session replay */
isReplay: boolean;
/**
* Where a replayed session came from, or `undefined` for a fresh interactive
* run.
*
* Was a boolean, which conflated two runs that need different treatment: a
* `-R`/`-H` replay is already in history and must not be recorded again, while
* a `--load-session` run is not in history at all — recording it is the only
* way `nopy history` and `-R` can see it afterwards. Everything that merely
* asks "am I replaying?" (reading values back off the session rather than
* prompting) takes `replaySource !== undefined`.
*/
replaySource?: 'file' | 'history';
}
/**
@@ -83,7 +93,7 @@ export async function runInteractiveWorkflow(
authMethod: authResult.authMethod,
username: authResult.username,
password: authResult.password,
isReplay: false,
replaySource: undefined,
};
}
@@ -141,7 +151,7 @@ export async function runReplayWorkflow(
authMethod,
username,
password,
isReplay: true,
replaySource: 'file',
};
}
@@ -196,7 +206,7 @@ export async function runSessionReplayWorkflow(
authMethod,
username,
password,
isReplay: true,
replaySource: 'history',
};
}
+63
View File
@@ -200,4 +200,67 @@ describe('Variables secrets', () => {
expect(variables.persistable('cube-a')).toEqual({});
expect(variables.persistable('cube-b')).toEqual({ PASSWORD: 'b' });
});
it('excludes a declared secret from the env a session records', () => {
const variables = new Variables({ PASSWORD: 'hunter2', KEY_DIR: './vault' }, ['PASSWORD']);
expect(variables.persistableEnv()).toEqual({ KEY_DIR: './vault' });
});
it('records an env with no secrets in it whole', () => {
const variables = new Variables({ KEY_DIR: './vault' }, ['PASSWORD']);
expect(variables.persistableEnv()).toEqual({ KEY_DIR: './vault' });
});
});
describe('Variables globally declared secrets', () => {
/** `env` carrying a key that cube-a declares secret and cube-b knows nothing of. */
const withLeakyEnv = () => {
const variables = new Variables({ PASSWORD: 'wildpass123', KEY_DIR: '/vault' }, ['PASSWORD']);
variables.declareSecrets('cube-a', ['PASSWORD']);
variables.declareSchema('cube-a', ['USER', 'PASSWORD']);
variables.declareSchema('cube-b', ['PORT']);
return variables;
};
it('does not seed a secret onto a cube that does not declare it', () => {
const variables = withLeakyEnv();
variables.assign('cube-b', 'default', { PORT: 22 });
expect(variables.get('cube-b')).not.toHaveProperty('PASSWORD');
expect(variables.of('cube-b', 'PASSWORD')).toBeUndefined();
});
it('still seeds it onto a cube whose schema declares it', () => {
const variables = withLeakyEnv();
variables.assign('cube-a', 'default', {});
expect(variables.get('cube-a').PASSWORD).toBe('wildpass123');
expect(variables.of('cube-a', 'PASSWORD')?.origin).toBe('env');
expect(variables.persistable('cube-a')).not.toHaveProperty('PASSWORD');
});
it('keeps broadcasting an undeclared key that is not a secret', () => {
// ssh:keyman reads KEY_DIR off host.data without declaring it in its schema.
const variables = withLeakyEnv();
variables.assign('cube-b', 'default', {});
expect(variables.get('cube-b').KEY_DIR).toBe('/vault');
});
it('redacts a global secret on a cube whose own manifest forgot to list it', () => {
const variables = new Variables({}, ['PASSWORD']);
variables.assign('cube-b', 'prompt', { PASSWORD: 'typed' });
expect(variables.of('cube-b', 'PASSWORD')?.redacted).toBe(true);
expect(variables.persistable('cube-b')).toEqual({});
});
it('treats a cube that declared no schema as declaring nothing', () => {
const variables = new Variables({ PASSWORD: 'p' }, ['PASSWORD']);
variables.assign('cube-z', 'default', {});
expect(variables.get('cube-z')).toEqual({});
});
});
@@ -129,10 +129,15 @@ describe('BuildContext session replay', () => {
});
describe('BuildContext replay gaps', () => {
const replay = (cube: Cube, recorded: Record<string, string> = {}, options = {}) =>
const replay = (
cube: Cube,
recorded: Record<string, string> = {},
options = {},
variables = new Variables()
) =>
new BuildContext(
{ [cube.id]: cube },
new Variables(),
variables,
session([{ key: cube.id, variables: recorded }]),
config,
{ method: 'ssh' },
@@ -175,16 +180,18 @@ describe('BuildContext replay gaps', () => {
expect(VariableAssignment).not.toHaveBeenCalled();
});
it('refuses to deploy when the form was cancelled', async () => {
it('refuses to deploy when the form came back empty', async () => {
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
// The real VariableAssignment swallows a cancelled form, so the gap check
// has to run again afterwards or the cube ships without the variable.
// A form that resolves is not proof of an answer: enquirer renders
// `Math.min(limit, height)` fields, so a terminal misreporting its height
// submits `{}` without the user having seen a question. The gap check has
// to run again afterwards or the cube ships without the variable.
vi.mocked(VariableAssignment).mockResolvedValue(undefined);
const context = replay(cube);
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
'Cube "cube-a" is missing SSID and cannot be deployed.'
'Cube "cube-a" is missing SSID. Nothing supplied it'
);
expect(context.deployCalls).toHaveLength(0);
});
@@ -194,10 +201,36 @@ describe('BuildContext replay gaps', () => {
const context = replay(cube, {}, { useDefaults: true });
// A schema default is deliberately not good enough for a secret: it would
// deploy a different credential than the run being replayed.
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
/cannot be replayed with --use-defaults: PASSWORD/
/cannot be replayed with --use-defaults: PASSWORD would have to be entered\..*not accepted for a secret/s
);
});
it('accepts a secret supplied through config env under --use-defaults', async () => {
const cube = secretCube('cube-a', z.object({ PASSWORD: z.string().default('changeme') }));
const context = replay(
cube,
{},
{ useDefaults: true },
new Variables({ PASSWORD: 'from-env' }, ['PASSWORD'])
);
await context.resolveCube('cube-a', 'host1');
expect(VariableAssignment).not.toHaveBeenCalled();
expect(context.deployCalls[0].env.PASSWORD).toBe('from-env');
});
it('accepts a required variable a dependency passed under --use-defaults', async () => {
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
const context = replay(cube, {}, { useDefaults: true });
await context.resolveCube('cube-a', 'host1', { SSID: 'from-param' });
expect(context.deployCalls[0].env.SSID).toBe('from-param');
});
});
describe('BuildContext session recording', () => {
@@ -240,6 +273,50 @@ describe('BuildContext session recording', () => {
});
});
describe('BuildContext secret broadcast', () => {
// The field run put PASSWORD under `env` because the docs said to, and watched
// it appear unmasked on the command line of every cube that was not user:add.
const resolveBoth = async () => {
const declaring = secretCube('cube-a', z.object({ PASSWORD: z.string().default('changeme') }));
const innocent = testCube('cube-b', z.object({ PORT: z.string().default('22') }));
const context = new BuildContext(
{ 'cube-a': declaring, 'cube-b': innocent },
new Variables({ PASSWORD: 'wildpass123', KEY_DIR: '/vault' }, ['PASSWORD']),
session(),
config,
{ method: 'ssh' },
{ useDefaults: true }
);
await context.resolveCube('cube-a', 'host1');
await context.resolveCube('cube-b', 'host1');
return context;
};
it('never puts an env secret on a cube that does not declare it', async () => {
const context = await resolveBoth();
const [, forB] = context.deployCalls;
expect(forB.cube).toBe('cube-b');
expect(forB.env).not.toHaveProperty('PASSWORD');
expect(forB.command.join(' ')).not.toContain('wildpass123');
});
it('still delivers it to the cube that declares it', async () => {
const context = await resolveBoth();
const [forA] = context.deployCalls;
expect(forA.env.PASSWORD).toBe('wildpass123');
expect(forA.secrets).toEqual(['PASSWORD']);
});
it('leaves an ordinary env key broadcast to both', async () => {
const context = await resolveBoth();
expect(context.deployCalls.map((call) => call.env.KEY_DIR)).toEqual(['/vault', '/vault']);
});
});
describe('BuildContext --use-defaults', () => {
const withDefaults = (cube: Cube, variables = new Variables(), cfg = config) =>
new BuildContext(
@@ -333,6 +410,38 @@ describe('BuildContext --use-defaults', () => {
});
});
describe('BuildContext interactive completeness', () => {
const interactive = (cube: Cube, variables = new Variables()) =>
new BuildContext({ [cube.id]: cube }, variables, session(), config, { method: 'ssh' });
it('refuses to deploy when the form submitted nothing', async () => {
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
// What a 0-row terminal does: the form renders no fields, the user sees no
// question, enquirer resolves `{}` and the run used to carry on and deploy
// the cube with SSID simply absent from `--data`.
vi.mocked(VariableAssignment).mockResolvedValue(undefined);
const context = interactive(cube);
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
/Cube "cube-a" is missing SSID\. Nothing supplied it/
);
expect(context.deployCalls).toHaveLength(0);
});
it('deploys when the form answered', async () => {
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
vi.mocked(VariableAssignment).mockImplementation(async (_cube, variables) => {
variables.assign('cube-a', 'prompt', { SSID: 'typed' });
});
const context = interactive(cube);
await context.resolveCube('cube-a', 'host1');
expect(context.deployCalls[0].env.SSID).toBe('typed');
});
});
describe('BuildContext command construction', () => {
const build = (auth: { method: string; username?: string; password?: string }) => {
const context = new BuildContext(
@@ -125,3 +125,53 @@ describe('BuildContext.resolveCube', () => {
expect(context.deployCalls.map((c) => c.cube)).toEqual(['cube-a', 'cube-b', 'cube-c']);
});
});
describe('deploy order across several selected cubes', () => {
// `nopy.main.ts` walks `workflow.selectedCubes` and calls `resolveCube` once
// per entry, so the order that list arrives in is the order the loop visits.
// Emission is post-order, though, so a declared edge is honoured whichever way
// round the two cubes were listed — the recursion *is* the topological sort,
// and these pin that rather than leaving it to be inferred from the one-root
// cases above.
async function resolveAll(cubes: Record<string, Cube>, selected: string[]): Promise<string[]> {
const context = new BuildContext(
cubes,
new Variables(),
{ cubes: [] } as any,
{
env: {},
} as any,
{ method: 'ssh' }
);
for (const id of selected) await context.resolveCube(id, 'host1');
return context.deployCalls.map((c) => c.cube);
}
it('emits a dependency first even when it is selected last', async () => {
const cubes = {
'cube-a': createTestCube('cube-a'),
'cube-b': createTestCube('cube-b', () => ['cube-a']),
};
// The list order is the inversion of the dependency: b depends on a, and a
// is named after it. Resolving b still drags a in ahead of itself, and the
// second visit is deduped rather than re-emitted at the tail.
expect(await resolveAll(cubes, ['cube-b', 'cube-a'])).toEqual(['cube-a', 'cube-b']);
expect(await resolveAll(cubes, ['cube-a', 'cube-b'])).toEqual(['cube-a', 'cube-b']);
});
it('interleaves an unrelated cube by list order and nothing else', async () => {
// With no edge between them there is nothing to sort on, so `cube-z` lands
// where the list put it. That is the whole of what selection order decides.
const cubes = {
'cube-a': createTestCube('cube-a'),
'cube-b': createTestCube('cube-b', () => ['cube-a']),
'cube-z': createTestCube('cube-z'),
};
expect(await resolveAll(cubes, ['cube-z', 'cube-b'])).toEqual(['cube-z', 'cube-a', 'cube-b']);
expect(await resolveAll(cubes, ['cube-b', 'cube-z'])).toEqual(['cube-a', 'cube-b', 'cube-z']);
});
});
+62
View File
@@ -0,0 +1,62 @@
/**
* Tests for nopy.errors — how a failed run is presented.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { NopyUsageError, reportError } from '../src/nopy.errors.js';
let out: ReturnType<typeof vi.spyOn>;
let err: ReturnType<typeof vi.spyOn>;
/** Everything written to stderr by the last call, as one string. */
const stderr = () => err.mock.calls.map((call) => String(call[0])).join('\n');
beforeEach(() => {
out = vi.spyOn(console, 'log').mockImplementation(() => {});
err = vi.spyOn(console, 'error').mockImplementation(() => {});
delete process.env.NOPY_DEBUG;
});
afterEach(() => {
vi.restoreAllMocks();
});
describe('reportError', () => {
it('prints a usage error as one line and points at the debug switch', () => {
reportError(new NopyUsageError('No .nopyrc.json found'));
expect(stderr()).toContain('Error: No .nopyrc.json found');
expect(stderr()).not.toContain('nopy.errors');
expect(stderr()).toContain('NOPY_DEBUG=1');
});
it('keeps the stack for anything unexpected', () => {
reportError(new TypeError('cannot read properties of undefined'));
expect(stderr()).toContain('Error: cannot read properties of undefined');
expect(stderr()).toContain('TypeError: cannot read properties of undefined\n at ');
expect(stderr()).not.toContain('NOPY_DEBUG=1');
});
it('prints the stack of a usage error under NOPY_DEBUG', () => {
process.env.NOPY_DEBUG = '1';
reportError(new NopyUsageError('No .nopyrc.json found'));
expect(stderr()).toContain('NopyUsageError: No .nopyrc.json found\n at ');
expect(stderr()).not.toContain('NOPY_DEBUG=1 for');
});
it('reports a thrown non-error', () => {
reportError('just a string');
expect(stderr()).toContain('Error: just a string');
expect(stderr()).toContain('NOPY_DEBUG=1');
});
it('says nothing on stdout', () => {
reportError(new NopyUsageError('No .nopyrc.json found'));
expect(out).not.toHaveBeenCalled();
});
});
-14
View File
@@ -123,20 +123,6 @@ describe('outputExecutionPlan', () => {
expect(output).toContain('host1');
});
it('outputs JSON format when requested', () => {
const calls = [createTestCall('cube-a', 'host1')];
outputExecutionPlan(calls, true);
expect(consoleLogSpy).toHaveBeenCalledTimes(1);
const output = consoleLogSpy.mock.calls[0][0];
const parsed = JSON.parse(output);
expect(parsed.plan).toHaveLength(1);
expect(parsed.plan[0].cube).toBe('cube-a');
expect(parsed.plan[0].host).toBe('host1');
});
it('masks variables the manifest declared secret', () => {
const call: DeployCall = {
...createTestCall('cube-a', 'host1'),
+42
View File
@@ -0,0 +1,42 @@
/**
* Runs one real enquirer variable form and prints what it produced.
*
* Driven by `tests/prompts.pty.test.ts` under a pty of a chosen size. Nothing
* here is mocked — the point is the prompt library's own behaviour on a
* terminal that reports no size, which cannot be observed from inside a vitest
* worker because there is no TTY there to misreport.
*
* Prints one line, `NOPY_PROBE <json>`, holding the values the form assigned at
* the `prompt` origin. An empty object means the form submitted nothing.
*/
import { Cube, Manifest } from '@bitsquare/nopy-cubes';
import { z } from 'zod';
import { Variables } from '../../src/nopy.common.js';
import { VariableAssignment } from '../../src/nopy.prompts.js';
const KEYS = ['ALPHA', 'BETA'];
const cube = new Cube(
Manifest({
id: 'probe',
name: 'Zero-rows probe',
schema: z.object({
ALPHA: z.string().describe('First value').default(''),
BETA: z.string().describe('Second value').default(''),
}),
}),
'/cubes/probe',
'deploy.py'
);
const variables = new Variables();
await VariableAssignment(cube, variables);
const assigned: Record<string, unknown> = {};
for (const key of KEYS) {
const variable = variables.of('probe', key);
if (variable?.origin === 'prompt') assigned[key] = variable.value;
}
process.stdout.write(`\nNOPY_PROBE ${JSON.stringify(assigned)}\n`);
+115 -45
View File
@@ -48,7 +48,12 @@ vi.mock('../src/cubes/index.js', () => ({ loadCubes }));
vi.mock('../src/nopy.config.js', () => ({ loadConfig, getConfigPaths }));
vi.mock('../src/nopy.workflow.js', () => ({ runWorkflow }));
vi.mock('../src/nopy.history.js', () => ({ addToHistory, DEFAULT_HISTORY_SIZE: 10 }));
vi.mock('../src/nopy.session.js', () => ({ saveSession }));
// Only the writer is a spy — `describeSession` and the version constant are pure
// and the assertions below are about what nopy() actually stamps.
vi.mock('../src/nopy.session.js', async (importOriginal) => ({
...(await importOriginal<typeof import('../src/nopy.session.js')>()),
saveSession,
}));
vi.mock('../src/cubes/dependencies.js', () => ({
BuildContext: class {
resolveCube = resolveCube;
@@ -67,16 +72,17 @@ vi.mock('../src/nopy.executor.js', async (importOriginal) => {
import { nopy } from '../src/nopy.main.js';
const session = (): NopySession =>
({
version: '1.0',
name: 'test',
createdAt: '2026-01-01T00:00:00.000Z',
cubes: [],
hosts: ['web-1'],
auth: { method: 'ssh-key' },
env: {},
}) as NopySession;
/**
* A session as bare as the loader will accept one — no `version`, `timestamp`
* or `name`, which is exactly what a hand-written file looks like and what
* `nopy()` has to fill in.
*/
const session = (): NopySession => ({
cubes: [],
hosts: ['web-1'],
auth: { method: 'ssh-key' },
env: {},
});
const call = (cube: string): DeployCall => ({
cube,
@@ -88,10 +94,12 @@ const call = (cube: string): DeployCall => ({
});
let logSpy: ReturnType<typeof vi.spyOn>;
let errSpy: ReturnType<typeof vi.spyOn>;
beforeEach(() => {
vi.clearAllMocks();
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
errSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
state.config = { hosts: ['web-1'], cubeDirs: [], cubePackages: [], env: {} };
state.loadResult = { cubes: { 'cube-a': {} }, errors: [] };
@@ -102,14 +110,19 @@ beforeEach(() => {
session: session(),
selectedCubes: ['cube-a'],
authMethod: 'ssh-key',
isReplay: false,
replaySource: undefined,
});
executeDeployCalls.mockResolvedValue([
{ cube: 'cube-a', host: 'web-1', success: true, duration: 10 },
]);
});
const output = () => logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
/**
* The two streams, kept apart on purpose: stdout carries the deploy commands
* and pyinfra's own output, everything nopy says about itself goes to stderr.
*/
const stdout = () => logSpy.mock.calls.map((c) => c.join(' ')).join('\n');
const stderr = () => errSpy.mock.calls.map((c) => c.join(' ')).join('\n');
describe('nopy', () => {
it('runs the happy path and reports success', async () => {
@@ -141,7 +154,7 @@ describe('nopy', () => {
session: { ...session(), hosts: ['web-1', 'web-2'] },
selectedCubes: ['cube-a', 'cube-b'],
authMethod: 'ssh-key',
isReplay: false,
replaySource: undefined,
});
await nopy();
@@ -159,13 +172,13 @@ describe('nopy', () => {
expect(runWorkflow).not.toHaveBeenCalled();
});
it('emits the errors as JSON when jsonOutput is set', async () => {
it('reports them on stderr', async () => {
state.loadResult = { cubes: {}, errors: ['bad manifest'] };
await nopy({ jsonOutput: true });
await nopy();
const payload = JSON.parse(logSpy.mock.calls.at(-1)?.[0] as string);
expect(payload).toEqual({ success: false, errors: ['bad manifest'] });
expect(stderr()).toContain('bad manifest');
expect(stdout()).toBe('');
});
});
@@ -180,7 +193,7 @@ describe('nopy', () => {
await nopy({ continueOnError: true });
const text = output();
const text = stderr();
expect(text).toContain('Configuration');
expect(text).toContain('Hosts:');
expect(text).toContain('Cube dirs:');
@@ -198,7 +211,7 @@ describe('nopy', () => {
await nopy();
const text = output();
const text = stderr();
expect(text).toContain('Configuration');
expect(text).not.toContain('Hosts:');
expect(text).not.toContain('Cube dirs:');
@@ -215,25 +228,20 @@ describe('nopy', () => {
await nopy();
const text = output();
const text = stderr();
expect(text).toContain('~/.nopyrc.json');
expect(text).toContain('./.nopyrc.json');
expect(text).toContain('/etc/nopy/.nopyrc.json');
});
it('is suppressed for JSON output', async () => {
await nopy({ jsonOutput: true });
expect(output()).not.toContain('Configuration');
});
it('is suppressed when replaying a session object', async () => {
await nopy({ replaySession: session() });
expect(output()).not.toContain('Configuration');
expect(stderr()).not.toContain('Configuration');
});
it('is suppressed when replaying a session file', async () => {
await nopy({ loadSession: '/tmp/s.json' });
expect(output()).not.toContain('Configuration');
expect(stderr()).not.toContain('Configuration');
});
});
@@ -247,17 +255,57 @@ describe('nopy', () => {
expect(written.cubes).toEqual(state.cubeSessions);
});
it('does not save a replayed session back to file', async () => {
it('leaves a declared secret out of the recorded env', async () => {
// The session's `env` is a copy of the config's, and used to be copied
// verbatim — writing to disk, in plaintext, the credential that was kept
// out of every cube's `variables` one key below.
state.config = {
...state.config,
env: { PASSWORD: 'hunter2', KEY_DIR: './vault' },
secrets: ['PASSWORD'],
};
await nopy({ saveSession: '/tmp/out.json' });
expect(saveSession.mock.calls[0][0].env).toEqual({ KEY_DIR: './vault' });
});
it('saves a replayed session too', async () => {
runWorkflow.mockResolvedValue({
session: session(),
selectedCubes: ['cube-a'],
authMethod: 'ssh-key',
isReplay: true,
replaySource: 'history',
});
await nopy({ saveSession: '/tmp/out.json' });
expect(saveSession).not.toHaveBeenCalled();
expect(saveSession).toHaveBeenCalledTimes(1);
expect(saveSession.mock.calls[0][1]).toBe('/tmp/out.json');
});
it('stamps version, timestamp and a derived name', async () => {
await nopy({ saveSession: '/tmp/out.json' });
const [written] = saveSession.mock.calls[0];
expect(written.version).toBe('1.0.0');
expect(written.timestamp).toMatch(/^\d{4}-\d{2}-\d{2}T/);
expect(written.name).toContain('cube-a');
expect(written.name).toContain('web-1');
});
it('keeps the name and version a replayed session already carried', async () => {
runWorkflow.mockResolvedValue({
session: { ...session(), version: '0.9.0', name: 'hand-written', timestamp: 'then' },
selectedCubes: ['cube-a'],
authMethod: 'ssh-key',
replaySource: 'file',
});
await nopy({ saveSession: '/tmp/out.json' });
const [written] = saveSession.mock.calls[0];
expect(written).toMatchObject({ version: '0.9.0', name: 'hand-written', timestamp: 'then' });
});
it('does not save when no path is given', async () => {
@@ -300,12 +348,12 @@ describe('nopy', () => {
expect(addToHistory).not.toHaveBeenCalled();
});
it('skips history for a replay', async () => {
it('skips history for a replay out of history', async () => {
runWorkflow.mockResolvedValue({
session: session(),
selectedCubes: ['cube-a'],
authMethod: 'ssh-key',
isReplay: true,
replaySource: 'history',
});
await nopy();
@@ -313,6 +361,19 @@ describe('nopy', () => {
expect(addToHistory).not.toHaveBeenCalled();
});
it('records a replay out of a session file', async () => {
runWorkflow.mockResolvedValue({
session: session(),
selectedCubes: ['cube-a'],
authMethod: 'ssh-key',
replaySource: 'file',
});
await nopy();
expect(addToHistory).toHaveBeenCalledTimes(1);
});
it('skips history when nothing would be deployed', async () => {
state.deployCalls = [];
@@ -320,19 +381,36 @@ describe('nopy', () => {
expect(addToHistory).not.toHaveBeenCalled();
});
it('skips history for a print-only run', async () => {
// Same rule as `--dry-run`: nothing was deployed, so nothing belongs at
// the head of the list `-R` repeats.
await nopy({ printOnly: true });
expect(addToHistory).not.toHaveBeenCalled();
});
});
describe('printOnly', () => {
it('prints commands and never executes', async () => {
await nopy({ printOnly: true });
const text = output();
const text = stdout();
expect(text).toContain('Deploy Commands');
expect(text).toContain('# cube-a -> web-1');
expect(text).toContain('pyinfra web-1 -y cube-a.deploy.py');
expect(executeDeployCalls).not.toHaveBeenCalled();
});
it('keeps stdout to the commands and nothing else', async () => {
// The whole point of the split: `nopy -P > plan.txt` has to be the plan.
// The config banner and every log line are on the other stream.
await nopy({ printOnly: true });
expect(stdout()).not.toContain('Configuration');
expect(stderr()).toContain('Configuration');
});
it('reports the command count as the summary total', async () => {
const result = await nopy({ printOnly: true });
@@ -359,17 +437,9 @@ describe('nopy', () => {
const [, options] = executeDeployCalls.mock.calls[0];
options.onProgress({ cube: 'cube-a', host: 'web-1', success: true }, 1, 1);
options.onProgress({ cube: 'cube-b', host: 'web-1', success: false }, 1, 1);
// Exercises both the ✓ and ✗ branches; logtape writes via console.log.
expect(logSpy).toHaveBeenCalled();
});
it('stays silent on progress when jsonOutput is set', async () => {
await nopy({ jsonOutput: true });
const [, options] = executeDeployCalls.mock.calls[0];
const before = logSpy.mock.calls.length;
options.onProgress({ cube: 'cube-a', host: 'web-1', success: true }, 1, 1);
expect(logSpy.mock.calls.length).toBe(before);
// Exercises both the ✓ and ✗ branches; logtape writes via console.error.
expect(stderr()).toContain('cube-a');
expect(stderr()).toContain('cube-b');
});
});
});
+77
View File
@@ -0,0 +1,77 @@
/**
* The variable form, on a terminal that reports no size.
*
* This is the one case that cannot be tested from inside a vitest worker: there
* is no TTY there for enquirer to misread, so the mocked tests in
* `prompts.test.ts` prove only that `rows` is *passed*, never that passing it
* matters. Here a real pty is opened at 0x0 — `pty.fork()`'s own default, and
* what `script -q` and some CI terminals report — and a real form is answered
* through it.
*
* Measured both ways while writing this: with `terminalSize()` removed from
* `nopy.prompts.ts`, the form never renders and the driver times out with
* nothing on the wire.
*
* Needs `python3` for the pty; skipped, loudly, where there is none.
*/
import { execFileSync, spawnSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
const EXPECT_PY = fileURLToPath(new URL('../../../scripts/expect.py', import.meta.url));
const TSX = fileURLToPath(new URL('../node_modules/.bin/tsx', import.meta.url));
const PROBE = fileURLToPath(new URL('./fixtures/form-probe.ts', import.meta.url));
const hasPython = spawnSync('python3', ['--version']).status === 0;
/** Down arrow — how the form moves from one field to the next. */
const DOWN = '\u001b[B';
const STEPS = [
{ expect: 'ALPHA', send: 'alpha-typed', settle: 0.6 },
{ send: DOWN, settle: 0.4 },
{ send: 'beta-typed', settle: 0.4 },
{ send: '\r', settle: 1.2 },
];
describe.skipIf(!hasPython)('variable form over a pty', () => {
let tmpDir: string;
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'nopy-pty-'));
});
afterEach(() => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
/** Answers the probe form on a pty of the given size; returns what it assigned. */
const answerForm = (rows: number, cols: number) => {
const stepsPath = path.join(tmpDir, 'steps.json');
const logPath = path.join(tmpDir, 'session.log');
fs.writeFileSync(stepsPath, JSON.stringify(STEPS));
execFileSync('python3', [EXPECT_PY, stepsPath, '--', TSX, PROBE], {
env: {
...process.env,
PTY_ROWS: String(rows),
PTY_COLS: String(cols),
EXPECT_TIMEOUT: '60',
EXPECT_LOG: logPath,
},
stdio: 'pipe',
});
const transcript = fs.readFileSync(logPath, 'utf-8').replace(/\r/g, '');
const line = transcript.split('\n').find((l) => l.startsWith('NOPY_PROBE '));
return line ? JSON.parse(line.slice('NOPY_PROBE '.length)) : undefined;
};
it('collects every field on a terminal reporting 0x0', () => {
expect(answerForm(0, 0)).toEqual({ ALPHA: 'alpha-typed', BETA: 'beta-typed' });
}, 90_000);
});
+72 -20
View File
@@ -55,11 +55,27 @@ const question = (name: string) => questions().find((q) => q.name === name);
/** Grabs the options the last enquirer AutoComplete prompt was constructed with. */
const autoComplete = () => autoCompleteCtor.mock.calls.at(-1)?.[0] as Record<string, any>;
/** Grabs the options the last enquirer Form prompt was constructed with. */
const formOptions = () => formCtor.mock.calls.at(-1)?.[0] as Record<string, any>;
/** Grabs the choices the last enquirer Form prompt was constructed with. */
const formChoices = () => {
const options = formCtor.mock.calls.at(-1)?.[0] as { choices: Record<string, any>[] };
return options.choices;
};
const formChoices = () => formOptions().choices as Record<string, any>[];
/** Runs `body` with the terminal reporting the given size, then puts it back. */
async function withTerminal(
size: { rows: number; columns: number },
body: () => Promise<void>
): Promise<void> {
const was = { rows: process.stdout.rows, columns: process.stdout.columns };
Object.defineProperty(process.stdout, 'rows', { value: size.rows, configurable: true });
Object.defineProperty(process.stdout, 'columns', { value: size.columns, configurable: true });
try {
await body();
} finally {
Object.defineProperty(process.stdout, 'rows', { value: was.rows, configurable: true });
Object.defineProperty(process.stdout, 'columns', { value: was.columns, configurable: true });
}
}
const cube = (id: string, name: string, schema = z.object({})) =>
new Cube(Manifest({ id, name, schema }), `/cubes/${id}`, 'deploy.py');
@@ -113,24 +129,42 @@ describe('CubeSelection', () => {
it('derives page size from the terminal height', async () => {
autoCompleteRun.mockResolvedValue([]);
const rows = process.stdout.rows;
Object.defineProperty(process.stdout, 'rows', { value: 40, configurable: true });
await CubeSelection(cubes);
expect(autoComplete().limit).toBe(35);
await withTerminal({ rows: 40, columns: 200 }, async () => {
await CubeSelection(cubes);
expect(autoComplete().limit).toBe(35);
});
// Falls back to a floor of 10 on a short (or unknown) terminal.
Object.defineProperty(process.stdout, 'rows', { value: 0, configurable: true });
await CubeSelection(cubes);
expect(autoComplete().limit).toBe(19);
Object.defineProperty(process.stdout, 'rows', { value: rows, configurable: true });
// A terminal reporting nothing is floored, not believed.
await withTerminal({ rows: 0, columns: 0 }, async () => {
await CubeSelection(cubes);
expect(autoComplete().limit).toBe(19);
});
});
it('selects nothing when the user cancels', async () => {
it('hands the prompt a window size it can render into', async () => {
autoCompleteRun.mockResolvedValue([]);
// Passing `rows` is what keeps enquirer away from its own `utils.height`,
// which overwrites a good fallback with `getWindowSize()[1]` — zero here.
await withTerminal({ rows: 0, columns: 0 }, async () => {
await CubeSelection(cubes);
expect(autoComplete()).toMatchObject({ rows: 24, columns: 80 });
});
await withTerminal({ rows: 50, columns: 200 }, async () => {
await CubeSelection(cubes);
expect(autoComplete()).toMatchObject({ rows: 50, columns: 200 });
});
});
it('lets a cancellation travel instead of returning an empty selection', async () => {
// An empty selection is a legitimate answer, so swallowing the rejection
// here made "the user backed out" and "the user picked nothing" the same
// event and let the run continue to deploy zero cubes.
autoCompleteRun.mockRejectedValue(new Error('cancelled'));
await expect(CubeSelection(cubes)).resolves.toEqual({ selectedCubes: [] });
await expect(CubeSelection(cubes)).rejects.toThrow('cancelled');
});
});
@@ -418,13 +452,31 @@ describe('VariableAssignment', () => {
expect(variables.get('svc').extra).toBe('kept');
});
it('assigns nothing when the user cancels the form', async () => {
it('hands the form a window size it can render into', async () => {
const variables = new Variables();
formRun.mockResolvedValue({});
// The form is where a zero height actually costs something: enquirer
// renders `Math.min(limit, height)` fields, so a 0-row terminal shows none
// of them and submits `{}` without the user ever seeing the questions.
await withTerminal({ rows: 0, columns: 0 }, async () => {
await VariableAssignment(cube('svc', 'Service', schema), variables);
expect(formOptions()).toMatchObject({ rows: 24, columns: 80 });
});
await withTerminal({ rows: 50, columns: 200 }, async () => {
await VariableAssignment(cube('svc', 'Service', schema), variables);
expect(formOptions()).toMatchObject({ rows: 50, columns: 200 });
});
});
it('lets a cancelled form travel rather than assigning nothing', async () => {
const variables = new Variables();
formRun.mockRejectedValue(new Error('cancelled'));
await expect(
VariableAssignment(cube('svc', 'Service', schema), variables)
).resolves.toBeUndefined();
await expect(VariableAssignment(cube('svc', 'Service', schema), variables)).rejects.toThrow(
'cancelled'
);
expect(variables.get('svc')).toEqual({});
});
+118 -1
View File
@@ -5,12 +5,14 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import {
createSession,
describeSession,
listSessions,
loadSession,
type NopySession,
SESSION_VERSION,
saveSession,
} from '../src/nopy.session.js';
@@ -48,6 +50,62 @@ describe('createSession', () => {
expect(session.env).toEqual({ KEY: 'value' });
});
it('stamps the format version and a creation time', () => {
const session = createSession({ cubes: [], hosts: ['localhost'], auth: { method: 'ssh' } });
expect(session.version).toBe(SESSION_VERSION);
expect(new Date(session.timestamp!).toISOString()).toBe(session.timestamp);
});
it('lets the caller supply the timestamp', () => {
const session = createSession({
cubes: [],
hosts: ['localhost'],
auth: { method: 'ssh' },
timestamp: '2026-01-01T00:00:00.000Z',
});
expect(session.timestamp).toBe('2026-01-01T00:00:00.000Z');
});
});
describe('describeSession', () => {
const at = '2026-01-01T12:30:00.000Z';
it('names the cubes and the hosts', () => {
const name = describeSession(
{
cubes: [{ key: 'apt:essentials', variables: {} }],
hosts: ['web-1'],
auth: { method: 'ssh' },
},
at
);
expect(name).toContain('apt:essentials');
expect(name).toContain('web-1');
});
it('says so when there is no host', () => {
const name = describeSession({ cubes: [], auth: { method: 'ssh' } }, at);
expect(name).toContain('no host');
});
it('truncates a long cube list and a long host list', () => {
const name = describeSession(
{
cubes: Array.from({ length: 10 }, (_, i) => ({ key: `cube-${i}`, variables: {} })),
hosts: Array.from({ length: 10 }, (_, i) => `host-${i}`),
auth: { method: 'ssh' },
},
at
);
expect(name).toContain('...');
expect(name.split('→')[1]).toContain('...');
});
});
describe('saveSession and loadSession', () => {
@@ -116,6 +174,51 @@ describe('saveSession and loadSession', () => {
await expect(loadSession(sessionPath)).rejects.toThrow('auth');
});
// Half of SESSION_FORMAT.md is about the MJS form, and nothing exercised it.
// Each test needs its own filename: `import()` caches by URL, so a second
// module written to the same path would never be read.
it('loads a session from an MJS default export', async () => {
const mjsPath = path.join(tempDir, 'ok.session.mjs');
fs.writeFileSync(
mjsPath,
'export default { cubes: [{ key: "apt:essentials", variables: {} }], auth: { method: "ssh" } };'
);
await expect(loadSession(mjsPath)).resolves.toMatchObject({
cubes: [{ key: 'apt:essentials', variables: {} }],
});
});
it('rejects an MJS session with no default export', async () => {
const mjsPath = path.join(tempDir, 'no-default.session.mjs');
fs.writeFileSync(mjsPath, 'export const session = {};');
await expect(loadSession(mjsPath)).rejects.toThrow('must export a default object');
});
it('loads a session with no version at all', async () => {
fs.writeFileSync(sessionPath, JSON.stringify({ cubes: [], auth: { method: 'ssh' } }));
const warn = vi.spyOn(console, 'error').mockImplementation(() => {});
await expect(loadSession(sessionPath)).resolves.toMatchObject({ cubes: [] });
expect(warn).not.toHaveBeenCalled();
warn.mockRestore();
});
it('warns about an unknown version but still loads it', async () => {
fs.writeFileSync(
sessionPath,
JSON.stringify({ version: '9.9.9', cubes: [], auth: { method: 'ssh' } })
);
const warn = vi.spyOn(console, 'error').mockImplementation(() => {});
await expect(loadSession(sessionPath)).resolves.toMatchObject({ version: '9.9.9' });
expect(warn.mock.calls[0][0]).toContain('9.9.9');
warn.mockRestore();
});
});
describe('listSessions', () => {
@@ -154,4 +257,18 @@ describe('listSessions', () => {
expect(result).toHaveLength(1);
expect(result[0].endsWith('test.session.mjs')).toBe(true);
});
it('finds the documented .nopysession.* files', () => {
// The name every example in the README uses, and the one this missed:
// `wild.nopysession.json` does not end in `.session.json`.
fs.writeFileSync(path.join(tempDir, 'wild.nopysession.json'), '{}');
fs.writeFileSync(path.join(tempDir, 'wild.nopysession.mjs'), 'export default {}');
fs.writeFileSync(path.join(tempDir, 'nopysession.json'), '{}');
const result = listSessions(tempDir);
expect(result).toHaveLength(2);
expect(result.some((p) => p.endsWith('wild.nopysession.json'))).toBe(true);
expect(result.some((p) => p.endsWith('wild.nopysession.mjs'))).toBe(true);
});
});
+6 -6
View File
@@ -78,7 +78,7 @@ describe('runInteractiveWorkflow', () => {
expect(result.selectedCubes).toEqual(['cube-a']);
expect(result.authMethod).toBe('ssh-key');
expect(result.isReplay).toBe(false);
expect(result.replaySource).toBeUndefined();
expect(result.session.hosts).toEqual(['web-1']);
expect(result.session.env).toEqual({ GLOBAL: 'value' });
expect(mockHostSelection).toHaveBeenCalledWith(config.hosts);
@@ -150,7 +150,7 @@ describe('runReplayWorkflow', () => {
const result = await runReplayWorkflow('/tmp/s.json', cubes, config);
expect(mockLoadSession).toHaveBeenCalledWith('/tmp/s.json');
expect(result.isReplay).toBe(true);
expect(result.replaySource).toBe('file');
expect(result.selectedCubes).toEqual(['cube-a']);
expect(mockHostSelection).not.toHaveBeenCalled();
expect(mockPasswordSelection).not.toHaveBeenCalled();
@@ -225,7 +225,7 @@ describe('runSessionReplayWorkflow', () => {
it('replays an in-memory session without prompting', async () => {
const result = await runSessionReplayWorkflow(session(), cubes, config);
expect(result.isReplay).toBe(true);
expect(result.replaySource).toBe('history');
expect(result.selectedCubes).toEqual(['cube-a']);
expect(mockLoadSession).not.toHaveBeenCalled();
expect(mockHostSelection).not.toHaveBeenCalled();
@@ -287,7 +287,7 @@ describe('runWorkflow dispatch', () => {
it('prefers an in-memory replay session over everything else', async () => {
const result = await runWorkflow('/tmp/s.json', cubes, config, {}, session());
expect(result.isReplay).toBe(true);
expect(result.replaySource).toBe('history');
expect(mockLoadSession).not.toHaveBeenCalled();
expect(mockCubeSelection).not.toHaveBeenCalled();
});
@@ -298,14 +298,14 @@ describe('runWorkflow dispatch', () => {
const result = await runWorkflow('/tmp/s.json', cubes, config);
expect(mockLoadSession).toHaveBeenCalledWith('/tmp/s.json');
expect(result.isReplay).toBe(true);
expect(result.replaySource).toBe('file');
expect(mockCubeSelection).not.toHaveBeenCalled();
});
it('falls back to the interactive workflow', async () => {
const result = await runWorkflow(undefined, cubes, config, { useAuthKey: true });
expect(result.isReplay).toBe(false);
expect(result.replaySource).toBeUndefined();
expect(mockCubeSelection).toHaveBeenCalled();
expect(mockAuthSelection).toHaveBeenCalledWith(true);
});