improving documentation consistency. auditing documentation drifts. planning cube packaging
This commit is contained in:
Benjamin Diedrichsen
@@ -0,0 +1,152 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## What this repo is
|
||||
|
||||
A pnpm workspace holding two independently published CLIs plus the pyinfra
|
||||
deployment units one of them runs:
|
||||
|
||||
| Path | Package | Binary | Role |
|
||||
| ----------------- | ------------------- | -------- | -------------------------------------------------------- |
|
||||
| `packages/nopy` | `@bitsquare/nopy` | `nopy` | interactive pyinfra script management and execution |
|
||||
| `packages/keyman` | `@bitsquare/keyman` | `keyman` | SSH key management, shelling out to `age` / `ssh-keygen` |
|
||||
| `cubes/` | — | — | the deployment units `nopy` runs (not published) |
|
||||
|
||||
The root package is private; only `packages/*` ship. The two packages do not
|
||||
depend on each other.
|
||||
|
||||
## Commands
|
||||
|
||||
```sh
|
||||
pnpm install # also installs the git hooks via simple-git-hooks
|
||||
pnpm run build # tsc --build across both packages (project references)
|
||||
pnpm run typecheck # tsc --build --noEmit
|
||||
pnpm run lint # biome check . (lint:fix / lint:ci variants)
|
||||
pnpm test # vitest run, both packages
|
||||
pnpm run test:coverage # vitest with the coverage gate
|
||||
pnpm run coverage:summary # renders the last coverage run as a Markdown table
|
||||
```
|
||||
|
||||
Single package / single test:
|
||||
|
||||
```sh
|
||||
pnpm --filter @bitsquare/nopy run test tests/config.test.ts # one file
|
||||
pnpm --filter @bitsquare/nopy run test -t "merges configs" # by test name
|
||||
pnpm --filter @bitsquare/nopy run test:watch
|
||||
pnpm --filter @bitsquare/nopy run nopy # run the CLI from source via tsx
|
||||
pnpm --filter @bitsquare/keyman run keyman
|
||||
```
|
||||
|
||||
`typescript` is the 7.x native compiler, so `tsc` *is* the fast one — there is no
|
||||
separate `tsgo` binary.
|
||||
|
||||
## Verification gate
|
||||
|
||||
`lint:ci` → `typecheck` → `test:coverage` is one gate, run in three places: the
|
||||
`pre-push` hook, `ci.yml` (non-`main`), and `publish-snapshot.yml` (`main`).
|
||||
`pre-commit` runs Biome with fixes on staged files only. Bypass with
|
||||
`SKIP_SIMPLE_GIT_HOOKS=1`; re-install after editing the hook config with
|
||||
`pnpm exec simple-git-hooks`.
|
||||
|
||||
Coverage thresholds live in each package's `vitest.config.ts` (85 % branches and
|
||||
functions, 80 % lines and statements), not in a CI flag — they fail identically
|
||||
locally and on the runner. Barrel files (`src/index.ts`, `src/cubes/index.ts`,
|
||||
`src/nopy.cubes.ts`) and the Commander argv wiring (`src/*.cli.ts`) are excluded;
|
||||
adding logic to those files means moving it somewhere covered.
|
||||
|
||||
Both packages set `pool: 'forks'` because tests use `process.chdir()` — most
|
||||
loader/config tests build a throwaway tree under `os.tmpdir()` and chdir into it,
|
||||
since discovery is driven entirely by the working directory.
|
||||
|
||||
## nopy architecture
|
||||
|
||||
One pass per invocation, `nopy.main.ts` orchestrating:
|
||||
|
||||
1. **`nopy.config.ts`** — `loadConfig()` walks up from `process.cwd()` collecting
|
||||
every `.nopyrc.json` plus `~/.nopyrc.json`, then merges them root-first.
|
||||
Per-property strategy comes from the child's `resolution` block (`merge` is
|
||||
the default: arrays concatenate and dedupe, objects deep-merge; `override`
|
||||
replaces). Only properties listed in `PATH_PROPERTIES` (`cubeDirs`) get
|
||||
relative paths resolved against their own config file's directory. **Throws**
|
||||
if no config file exists anywhere — which is why `nopy.cli.ts` calls it lazily
|
||||
inside the action, so `--help`/`--version` work outside a project.
|
||||
2. **`cubes/loader.ts`** — `findCubeDirectories()` unions `config.cubeDirs` with
|
||||
every ancestor directory holding a `.npcubes` marker, then scans each
|
||||
recursively (skipping dotted dirs and `node_modules`). A directory is a cube
|
||||
when it holds both a manifest (`manifest.mjs` or `*.manifest.mjs`) and a
|
||||
deploy script (`deploy.py` or `*.deploy.py`); manifests are loaded by dynamic
|
||||
`import()`. Cube id = `manifest.id` → a `[id]` prefix in `manifest.name` →
|
||||
the directory basename. Ids are flat and need not mirror the path
|
||||
(`cubes/network/tailscale` declares `net:tailscale`). Duplicate ids and bad
|
||||
manifests become entries in `errors`, which aborts the run.
|
||||
3. **`nopy.workflow.ts`** — picks interactive, file-replay, or history-replay and
|
||||
normalises all three into a `WorkflowResult`. Replays never re-prompt except
|
||||
for passwords (never persisted) and a missing host.
|
||||
4. **`cubes/dependencies.ts` → `BuildContext.resolveCube()`** — the core.
|
||||
Recursive, per (cube, host): assign params and schema defaults → collect
|
||||
variables (prompt, or read them back from the session on replay) → run
|
||||
`before` hooks → resolve `manifest.dependencies(vars)` (dynamic: it receives
|
||||
the *collected* variables) → emit the deploy call → run `after` hooks. There
|
||||
is no separate topological sort; ordering falls out of the recursion, and a
|
||||
`${cubeId}:${host}` set makes emission idempotent. Hooks get a `HookContext`
|
||||
whose `exec(id, vars)` re-enters `resolveCube`, so a hook can pull in a cube
|
||||
that is not a declared dependency.
|
||||
5. **`nopy.executor.ts`** — runs the built `pyinfra <host> -y --data K=V ... --chdir <cubeDir> <script>`
|
||||
commands through execa with inherited stdio, sequentially, stopping at the
|
||||
first failure unless `continueOnError`.
|
||||
|
||||
### Variables
|
||||
|
||||
`Variables` (`nopy.common.ts`) keeps three per-cube scopes plus one global bag.
|
||||
`get(id)` merges them lowest-to-highest: `global` (from config `env`) →
|
||||
`defaults` (Zod `.default()`, and recorded session values on replay) → `prompts`
|
||||
(what the user typed) → `params` (values passed by a dependency spec or hook).
|
||||
Every schema key is guaranteed present on the pyinfra side; pyinfra parses
|
||||
`--data` values itself, so `"true"` arrives as a bool and numeric strings as ints.
|
||||
|
||||
### Cube contract
|
||||
|
||||
A cube directory holds `manifest.mjs` + `deploy.py`; anything else in it is
|
||||
ignored by the loader but reachable from the script, which runs with the cube
|
||||
directory as its cwd. Manifests are ESM, import `cubes.Manifest` from
|
||||
`@bitsquare/nopy`, and declare `id`, `name`, a Zod `schema` (each field
|
||||
`.describe()`d — the description is the prompt label — and `.default()`ed), plus
|
||||
optional `dependencies`/`before`/`after`.
|
||||
|
||||
**Gotcha:** those manifests resolve `@bitsquare/nopy` through ordinary Node
|
||||
resolution from the manifest's own directory. Nothing in this repo links the
|
||||
package into `cubes/` or `packages/nopy/cubes/`, so loading them fails with
|
||||
`ERR_MODULE_NOT_FOUND` until you link it (`pnpm --filter @bitsquare/nopy run
|
||||
link:local`, then `npm link @bitsquare/nopy` where you run from).
|
||||
|
||||
## keyman architecture
|
||||
|
||||
Much smaller: `keyman.cli.ts` (argv, plus a `--print-config` escape hatch) →
|
||||
`keyman.main.ts`, an inquirer menu loop dispatching to one module per operation
|
||||
(`list`/`copy`/`generate`/`encrypt`/`decrypt`). `keyman.config.ts` mirrors nopy's
|
||||
upward-traversal + `resolution` merge for `.keymanrc.json`, but validates the
|
||||
result with Zod and falls back to defaults instead of throwing. `VAULT_ROOT` in
|
||||
the environment beats the config file. Encryption shells out to `age` /
|
||||
`age-keygen` / `ssh-keygen`, which must be on `PATH`.
|
||||
|
||||
## Releasing
|
||||
|
||||
Tag-driven, one package at a time; see `README.PUBLISH.md`.
|
||||
|
||||
- Push to `main` → `publish-snapshot.yml` publishes both packages to the Gitea
|
||||
registry as `<version>-main.<run>.g<sha>` under the `main` dist-tag. The
|
||||
version is set on the runner with `npm pkg set` and never committed.
|
||||
- `git tag <dir>-v<version>` (e.g. `nopy-v1.2.0` — the directory under
|
||||
`packages/`, not the npm name) → `release.yml` publishes to Gitea *and* npmjs.
|
||||
The tag chooses the package, `package.json` supplies the version, and the run
|
||||
fails if they disagree. A prerelease version goes out as `next`, otherwise
|
||||
`latest`.
|
||||
|
||||
So: bump `packages/<pkg>/package.json`, land it on `main`, then tag that commit.
|
||||
|
||||
## Known drift
|
||||
|
||||
`logConfigToFlags()` is exported and tested but nothing feeds its output into the
|
||||
built pyinfra command, so `log.verbosity` / `log.debug` in `.nopyrc.json`
|
||||
currently have no effect. Treat `docs/REFACTORING.md` as a plan, not a record.
|
||||
+714
@@ -0,0 +1,714 @@
|
||||
# Documentation audit
|
||||
|
||||
Every claim in the repository's Markdown was checked against the source it
|
||||
describes. Findings are grouped by *kind of divergence*, because the fix differs:
|
||||
a phantom feature needs a decision (build it or delete the docs), a wrong claim
|
||||
needs an edit, a gap needs prose.
|
||||
|
||||
Severity is about what it costs a reader:
|
||||
|
||||
- **🔴 broken** — following the documentation produces a wrong result or a crash.
|
||||
- **🟠 misleading** — the documentation states something the code does not do.
|
||||
- **🟡 gap** — the code does something real that no document mentions.
|
||||
|
||||
Verified against the working tree at commit `fcc1817`. Line numbers are from that
|
||||
state.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
|
||||
- [1. Documented features that do not exist](#1-documented-features-that-do-not-exist)
|
||||
- [2. Documented behaviour that differs from the code](#2-documented-behaviour-that-differs-from-the-code)
|
||||
- [3. `docs/API.md` — systematic drift](#3-docsapimd--systematic-drift)
|
||||
- [4. Undocumented behaviour](#4-undocumented-behaviour)
|
||||
- [5. Cube documentation](#5-cube-documentation)
|
||||
- [6. Defects found while verifying](#6-defects-found-while-verifying)
|
||||
- [7. Checked and accurate](#7-checked-and-accurate)
|
||||
- [Suggested order of attack](#suggested-order-of-attack)
|
||||
|
||||
---
|
||||
|
||||
## 1. Documented features that do not exist
|
||||
|
||||
These are the same class of problem as the `--parallel` flag that was removed
|
||||
earlier: documented in detail, absent from the source.
|
||||
|
||||
### 1.1 🔴 `-D, --use-defaults` does nothing
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Docs say** | `README.md:291` "Install with defaults (no prompts for customization)"; `docs/API.md:39` "Skip variable prompts, use defaults"; `nopy.cli.ts:54` "Run cubes with default values without prompts" |
|
||||
| **Code does** | Nothing. |
|
||||
|
||||
The option is threaded through four layers and then dropped. `nopy.cli.ts:95` →
|
||||
`nopy.main.ts:125` → `nopy.main.ts:174` → `BuildContext.options.useDefaults`
|
||||
(`cubes/dependencies.ts:35`), where it is **never read**. The only branch that
|
||||
skips prompting is `isSessionReplay` (`cubes/dependencies.ts:62`).
|
||||
`runInteractiveWorkflow` destructures only `useAuthKey` and ignores its
|
||||
`useDefaults` too (`nopy.workflow.ts:50`).
|
||||
|
||||
Every documented `-D` invocation — including
|
||||
`nopy install -D --save-session automated-deployment.nopysession.json`
|
||||
(`README.md:226`), which is presented as the way to do an unattended run —
|
||||
prompts for every variable of every cube.
|
||||
|
||||
```
|
||||
$ grep -rn "useDefaults" packages/nopy/src/
|
||||
nopy.workflow.ts:19 useDefaults?: boolean; # declared
|
||||
nopy.main.ts:94 useDefaults?: boolean; # declared
|
||||
nopy.main.ts:125 useDefaults = false, # defaulted
|
||||
nopy.main.ts:158 { useDefaults, useAuthKey },# passed
|
||||
nopy.main.ts:174 useDefaults, # passed
|
||||
nopy.cli.ts:95 useDefaults: options... # passed
|
||||
cubes/dependencies.ts:35 useDefaults?: boolean; # declared — and that is all
|
||||
```
|
||||
|
||||
### 1.2 🔴 `-j, --json` produces no output on success
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Docs say** | `README.md:18` "**JSON output** for CI/CD integration"; `README.md:360-367` "Machine-readable JSON output for scripting and CI/CD integration"; `docs/API.md:573` |
|
||||
| **Code does** | Emits JSON **only** on failure. |
|
||||
|
||||
`jsonOutput` reaches three places in `nopy.main.ts` (140, 150, 219) and none of
|
||||
them prints a result. It suppresses the config banner, prints
|
||||
`{success: false, errors}` when cube *loading* fails, and suppresses progress
|
||||
lines. The success path at `nopy.main.ts:226-236` returns the `NopyResult` object
|
||||
to the caller, and `nopy.cli.ts:107-110` inspects `result.success` without
|
||||
printing it.
|
||||
|
||||
A CI job running `nopy install --json` gets pyinfra's inherited stdio and nothing
|
||||
machine-readable. The exit code is the only usable signal.
|
||||
|
||||
Related: `--dry-run --json` prints the **text** plan, not JSON.
|
||||
`executeDeployCalls` calls `outputExecutionPlan(calls)` without the `asJson`
|
||||
argument (`nopy.executor.ts:172`), even though the function supports it
|
||||
(`nopy.executor.ts:110`).
|
||||
|
||||
### 1.3 🟠 `log.verbosity` and `log.debug` have no effect
|
||||
|
||||
Pre-existing known drift, recorded in `CLAUDE.md`, but the README still presents
|
||||
it as a working feature — two tables, a recommendation paragraph, and a slot in
|
||||
the config example (`README.md:127-130`, `143-163`).
|
||||
|
||||
`logConfigToFlags()` (`nopy.config.ts:352`) is exported and has 8 unit tests, but
|
||||
nothing calls it. `buildDeployCall` (`cubes/dependencies.ts:105-124`) constructs
|
||||
the pyinfra argv without consulting `config.log` at all.
|
||||
|
||||
This is live in the repo's own config: `packages/nopy/.nopyrc.json` sets
|
||||
`"verbosity": "trace", "debug": true` and gets neither.
|
||||
|
||||
### 1.4 🟠 Manifest `env` property
|
||||
|
||||
`README.md:14` lists "**Default values** with optional customization via manifest
|
||||
`env`". `env` was removed from `Manifest` — see `docs/REFACTORING.md` item 3, and
|
||||
the current interface at `cubes/types.ts:43-56`, which has `id`, `name`,
|
||||
`schema`, `dependencies`, `before`, `after` and nothing else.
|
||||
|
||||
### 1.5 🟠 Topological sorting
|
||||
|
||||
`README.md:11` ("Dependency resolution with **topological sorting**"),
|
||||
`README.md:25` ("Topologically sorts cubes based on dependencies") and
|
||||
`README.md:381` ("because cubes are topologically sorted") describe an algorithm
|
||||
that does not exist.
|
||||
|
||||
There is no sort. `BuildContext.resolveCube` recurses depth-first and pushes each
|
||||
cube after its dependencies, with a `${cubeId}:${host}` set for idempotence
|
||||
(`cubes/dependencies.ts:43-100`, `105-143`). The ordering is a side effect of the
|
||||
recursion order.
|
||||
|
||||
This matters beyond vocabulary: a topological sort detects cycles, and this does
|
||||
not. Two cubes that depend on each other recurse until the stack overflows —
|
||||
`resolvedCubes` is only consulted in `buildDeployCall`, which runs *after* the
|
||||
recursive call. `docs/API.md:160` still promises `Error` "if ... circular
|
||||
dependency detected".
|
||||
|
||||
---
|
||||
|
||||
## 2. Documented behaviour that differs from the code
|
||||
|
||||
### 2.1 🔴 Variable precedence is wrong in both directions
|
||||
|
||||
`README.md:107-113` states:
|
||||
|
||||
> **Priority order (lowest to highest):**
|
||||
> 1. Zod schema `.default()` values
|
||||
> 2. Global `env` from `.nopyrc.json`
|
||||
> 3. Accumulated variables from dependencies
|
||||
> 4. User prompts / session replay
|
||||
|
||||
`Variables.get()` (`nopy.common.ts:33-39`) does:
|
||||
|
||||
```typescript
|
||||
return {
|
||||
...this.global, // 1. config env (lowest)
|
||||
...this.defaults[id], // 2. Zod defaults
|
||||
...this.prompts[id], // 3. what the user typed
|
||||
...this.params[id], // 4. dependency / hook (highest)
|
||||
};
|
||||
```
|
||||
|
||||
Two pairs are inverted, and both have consequences:
|
||||
|
||||
- **Zod defaults beat `env`, not the other way round.** So `README.md:114` —
|
||||
"allowing users to override them globally via `.nopyrc.json`" — is backwards.
|
||||
Setting `"env": {"UPDATE": false}` cannot override a cube declaring
|
||||
`.default(true)`; the `env` value is only visible for keys the schema does not
|
||||
define. This is why `KEY_DIR` works in `packages/nopy/.nopyrc.json` (no cube
|
||||
declares it) and why anything else would not.
|
||||
- **Dependency/hook parameters beat user prompts.** A value passed as
|
||||
`[['user:add', {USER: 'deploy'}]]` silently overrides what the operator just
|
||||
typed at the form. The docs promise the opposite.
|
||||
|
||||
### 2.2 🔴 "Every key ... is guaranteed to be present on `host.data`" — not when a field lacks `.default()`
|
||||
|
||||
`README.md:99` states it outright; `README.md:105` claims defaults ensure "every
|
||||
cube has a predictable starting state".
|
||||
|
||||
`Cube.getDefaults()` (`cubes/types.ts:106-112`) is:
|
||||
|
||||
```typescript
|
||||
try {
|
||||
return this.manifest.schema.parse({});
|
||||
} catch {
|
||||
return {} as z.infer<Schema>;
|
||||
}
|
||||
```
|
||||
|
||||
One field without a `.default()` makes `parse({})` throw, and the `catch`
|
||||
discards the defaults of **every other field in the cube**. `VariableAssignment`
|
||||
then iterates over that empty object and returns before prompting
|
||||
(`nopy.prompts.ts:175-181`), so the user is never asked. The cube deploys with no
|
||||
`--data` flags at all and every `host.data.X` is `None`.
|
||||
|
||||
Three cubes in this repo are in that state today:
|
||||
|
||||
| Cube | Field(s) without `.default()` | Result |
|
||||
|---|---|---|
|
||||
| `net:wifi:connection` | `SSID`, `PASSWORD` | no prompt, no `--data`, all 4 vars lost |
|
||||
| `service:autostart` | `APP` | no prompt, no `--data`, all 3 vars lost |
|
||||
| `user:edit` | `USER` | no prompt, no `--data`, all 4 vars lost |
|
||||
|
||||
The failure is silent — no error, no warning, just a pyinfra run with an empty
|
||||
data set.
|
||||
|
||||
### 2.3 🔴 `.describe()` before `.default()` loses the prompt label
|
||||
|
||||
`CLAUDE.md` and the cube contract state that each schema field is `.describe()`d
|
||||
and "the description is the prompt label". `nopy.prompts.ts:184-185` reads it as:
|
||||
|
||||
```typescript
|
||||
const zodType = schema[key];
|
||||
const description = zodType?.description || key;
|
||||
```
|
||||
|
||||
In zod 4, `.default()` returns a `ZodDefault` **wrapper** that does not inherit
|
||||
`.description` from the type it wraps. Ordering therefore decides whether the
|
||||
label survives:
|
||||
|
||||
```
|
||||
z.boolean().describe('Update package cache').default(false) → description undefined
|
||||
z.boolean().default(false).describe('Update package cache') → description preserved
|
||||
```
|
||||
|
||||
The README's own manifest example (`README.md:67-68`) uses the losing order, so
|
||||
anyone copying it gets bare `UPDATE` / `PACKAGES` keys as prompt labels instead
|
||||
of the sentences they wrote. `docs/API.md:610` happens to use the working order —
|
||||
the two documents disagree, and neither mentions that it matters.
|
||||
|
||||
14 of the 21 cubes in `cubes/` are affected; among them
|
||||
`net:tailscale` (all 4 fields), `runtime:nodevm` (all 4), `user:add` (all 4),
|
||||
`ssh:keygen` (all 4) and `admin:locale` (all 4).
|
||||
|
||||
### 2.4 🟠 Session files claim `version` and `timestamp` fields
|
||||
|
||||
`README.md:179-181` shows a session with `"version": "1.0.0"` and
|
||||
`"timestamp": "2025-10-13T10:30:00Z"`, and `docs/SESSION_FORMAT.md:305-306`
|
||||
declares both **required** in the `NopySession` interface. Every MJS example in
|
||||
that file sets them.
|
||||
|
||||
`NopySession` (`nopy.session.ts:44-55`) has neither. `createSession`
|
||||
(`nopy.session.ts:183-197`) does not add them, `saveSession` writes the object
|
||||
verbatim (`nopy.session.ts:68-79`), and `loadSession`'s validation
|
||||
(`nopy.session.ts:146-155`) checks only `cubes`, `hosts` and `auth`. The
|
||||
repository's own `packages/nopy/example.nopysession.json` omits both — it does
|
||||
not match the format its own documentation prescribes.
|
||||
|
||||
Consequence: a `version` field implies a compatibility check that does not exist.
|
||||
Nothing reads it, so an incompatible old session fails later and more obscurely
|
||||
than a version check would.
|
||||
|
||||
### 2.5 🟠 Session filename convention does not match `listSessions()`
|
||||
|
||||
The READMEs consistently use `*.nopysession.json` (`README.md:223`, `330`, `338`;
|
||||
`docs/DOCKER.md:54`; the shipped `example.nopysession.json`).
|
||||
`docs/SESSION_FORMAT.md` consistently uses `*.session.json` / `*.session.mjs`.
|
||||
|
||||
`listSessions()` (`nopy.session.ts:173`) matches only the second form:
|
||||
|
||||
```typescript
|
||||
.filter((file) => file.endsWith('.session.json') || file.endsWith('.session.mjs'))
|
||||
```
|
||||
|
||||
`"my-deployment.nopysession.json"` does not end in `".session.json"`, so the
|
||||
file naming the README recommends is invisible to the function documented at
|
||||
`docs/API.md:430`. `loadSession` is unaffected (it switches on `.json`/`.mjs`),
|
||||
so this only bites the listing API.
|
||||
|
||||
### 2.6 🟠 `docs/DOCKER.md` container name contradicts the file it points at
|
||||
|
||||
`docs/DOCKER.md:35` and `:45`:
|
||||
|
||||
> We explicitly name it `nopy-test-container` because the
|
||||
> `example.nopysession.json` is configured to target this specific container name.
|
||||
|
||||
`packages/nopy/example.nopysession.json` targets `@docker/nopy-test-ubuntu` —
|
||||
the *image* tag from the build step, not the container name. Following the guide
|
||||
exactly produces a pyinfra run against a container that does not exist.
|
||||
|
||||
(`packages/nopy/.nopyrc.json` does list `@docker/nopy-test-container` in `hosts`,
|
||||
so the guide was probably written against the config rather than the session.)
|
||||
|
||||
### 2.7 🟠 `docs/HOOKS.md` — hook parameters are not validated
|
||||
|
||||
`docs/HOOKS.md:48` describes the hook's second argument as "The final,
|
||||
**validated** variables for the current cube".
|
||||
|
||||
`cubes/dependencies.ts:71` passes `this.variables.get(cubeId)` — a plain merge of
|
||||
the four scopes. `schema.parse()` is called in exactly one place,
|
||||
`Cube.getDefaults()` on an empty object, to extract defaults. Values from
|
||||
prompts, `env`, dependencies and hooks are never validated against the schema at
|
||||
any point in the pipeline. `coerceValue` (`nopy.prompts.ts:145-159`) type-coerces
|
||||
prompt input, which is not the same as validation and does not apply to the other
|
||||
three scopes.
|
||||
|
||||
### 2.8 🟠 `docs/HOOKS.md` — dependencies can pass variables too
|
||||
|
||||
The comparison table at `docs/HOOKS.md:83` says variable passing is
|
||||
"Inherited from env" for dependencies versus "Explicitly passed via `exec()`" for
|
||||
hooks, presenting explicit parameters as a hook-only capability.
|
||||
|
||||
`DependencySpec` is `string | [id, variables?]` (`cubes/types.ts:23`) and
|
||||
`cubes/dependencies.ts:86-88` unpacks the tuple and forwards it into the same
|
||||
`params` scope that `exec()` writes to. The two mechanisms are identical in this
|
||||
respect; per §2.1 both outrank user prompts.
|
||||
|
||||
### 2.9 🟠 nopy README installation section describes the wrong package manager
|
||||
|
||||
`README.md:248-279` says "This package is part of a **yarn** workspace monorepo",
|
||||
then gives `yarn install`, `yarn workspace @bitsquare/nopy build`,
|
||||
`yarn workspace @bitsquare/nopy nopy`, and `yarn nopy`.
|
||||
|
||||
The repo is a **pnpm** workspace: `packageManager: "pnpm@11.17.0"` in the root
|
||||
manifest, `pnpm-workspace.yaml`, a `pnpm-lock.yaml`, and every other document
|
||||
(root `README.md`, `README.PUBLISH.md`, `CLAUDE.md`) uses pnpm. There is no
|
||||
`yarn.lock`.
|
||||
|
||||
The section is also the wrong content for the file. `README.md` is one of three
|
||||
files shipped in the npm tarball (`files: ["dist", "README.md", "LICENSE"]`), so
|
||||
this is what a reader sees on npmjs.com — build-from-monorepo instructions
|
||||
instead of `npm install -g @bitsquare/nopy`, which is what the root README and
|
||||
`README.PUBLISH.md:314` correctly tell people to run.
|
||||
|
||||
### 2.10 🟠 keyman README: two operations missing, one operation invented
|
||||
|
||||
`packages/keyman/README.md:90-96` lists four menu entries: List, Encrypt,
|
||||
Decrypt, Quit. The menu (`keyman.main.ts:54-61`) has six:
|
||||
|
||||
```
|
||||
📋 List keys 📝 Copy public key 🆕 Generate key
|
||||
🔒 Encrypt keys 🔓 Decrypt keys ❌ Quit
|
||||
```
|
||||
|
||||
`Copy public key` and `Generate key` are undocumented — the latter being the only
|
||||
way to create a key inside the tool, which is why the Quick Start
|
||||
(`packages/keyman/README.md:33`) tells the user to shell out to `ssh-keygen`
|
||||
manually.
|
||||
|
||||
Conversely `packages/keyman/README.md:11` advertises "🔄 Support for key
|
||||
rotation". There is no rotation anywhere: `grep -rn "rotat" packages/keyman/src/`
|
||||
returns nothing.
|
||||
|
||||
`packages/keyman/README.md:93` also says encrypt takes keys "from `vault/tmp/`".
|
||||
`encryptKeys` (`keyman.encrypt.ts:12-22`) unions `~/.ssh` and `vault/tmp`, and
|
||||
offers both in the checkbox.
|
||||
|
||||
### 2.11 🟡 Root README understates the coverage gate
|
||||
|
||||
Root `README.md:57-58` describes "a hard **85 % branch** floor". Both
|
||||
`vitest.config.ts` files set four thresholds: branches 85, functions 85, lines
|
||||
80, statements 80. `README.PUBLISH.md:135` and `CLAUDE.md` both state all four —
|
||||
the root README is the odd one out, and it is the file a new contributor reads
|
||||
first.
|
||||
|
||||
### 2.12 🟡 `docs/DOCKER.md` relative link is broken
|
||||
|
||||
`docs/DOCKER.md:8` links `[README.md](./README.md)`, which resolves to
|
||||
`packages/nopy/docs/README.md` — nonexistent. It should be `../README.md`.
|
||||
|
||||
---
|
||||
|
||||
## 3. `docs/API.md` — systematic drift
|
||||
|
||||
`docs/API.md` documents an earlier architecture. It is not a matter of
|
||||
individual stale lines: the two central type definitions, one whole module, and
|
||||
two of the documented functions describe code that no longer exists. Anyone
|
||||
building against this file writes code that will not compile.
|
||||
|
||||
Recommendation: regenerate rather than patch.
|
||||
|
||||
### 3.1 🔴 Functions that do not exist
|
||||
|
||||
| Documented | Reality |
|
||||
|---|---|
|
||||
| `resolveDependencies(cubes, selectedCubeNames)` (`API.md:142-160`) | No such export. Resolution is `BuildContext.resolveCube` and returns nothing — it accumulates into `deployCalls`. |
|
||||
| `buildDeployCalls(cubeNames, hosts, context)` (`API.md:286-313`) | No such export. The entire "Builder Module" section, and its `BuildResult` interface, describes code replaced by `BuildContext` (`docs/REFACTORING.md` item 2). Still listed in the table of contents at `API.md:12`. |
|
||||
|
||||
### 3.2 🔴 `Cube<Schema>` — wrong shape entirely
|
||||
|
||||
`API.md:75-84` documents an interface with `key`, `dependencies: string[]`,
|
||||
`schema`, `defaults()`, `before`, `after`.
|
||||
|
||||
`Cube` (`cubes/types.ts:88-113`) is a **class**: constructor `(manifest, dir,
|
||||
deployScript)`, getters `id` and `name`, method `getDefaults()`. Everything else
|
||||
lives behind `.manifest`. Not one documented member name is correct — `key` is
|
||||
`id`, `defaults()` is `getDefaults()`, and `dependencies`/`schema`/`before`/
|
||||
`after` are on `cube.manifest`, not on `cube`.
|
||||
|
||||
### 3.3 🔴 `Manifest<Schema>` — wrong shape
|
||||
|
||||
`API.md:92-100` documents `key`, `dependencies: string[]`, `defaults: () => ...`.
|
||||
|
||||
`Manifest` (`cubes/types.ts:43-56`) has `id` (not `key`), no `defaults` member at
|
||||
all, and `dependencies` is a **function of the collected variables**:
|
||||
|
||||
```typescript
|
||||
dependencies?: (variables: z.infer<Schema>) => DependencySpec[];
|
||||
```
|
||||
|
||||
That signature change is the headline of `docs/REFACTORING.md` item 2. The
|
||||
`API.md` example at `:171` does use the function form, so the file contradicts
|
||||
itself two paragraphs apart. Both `Cube` and `Manifest` are additionally shown
|
||||
as generic over `z.AnyZodObject`, which zod 4 removed; the codebase defines
|
||||
`AnyObjectSchema` for exactly this reason (`cubes/types.ts:13`).
|
||||
|
||||
### 3.4 🟠 Incorrect signatures and examples
|
||||
|
||||
| Location | Documented | Actual |
|
||||
|---|---|---|
|
||||
| `API.md:486-492` | `saveConfig(data, local?)`, example passes `false` | `saveConfig(data, configPath?: string)` (`nopy.config.ts:323`). Passing `false` writes nothing useful. |
|
||||
| `API.md:481-484` | Search order `./nopyrc.json` then `~/.nopyrc.json` | Filename is `.nopyrc.json` (leading dot). Home is applied **first** (lowest priority, `nopy.config.ts:117-120`), all ancestors are collected and merged root-first, and the function **throws** when none is found (`nopy.config.ts:285-289`). The `resolution` merge strategy is not mentioned. |
|
||||
| `API.md:534-540` | `VariableAssignment(cube, env)` returning vars | `VariableAssignment(cube, variables: Variables)` returns `Promise<void>` and mutates the `Variables` instance (`nopy.prompts.ts:167`). The example's return value is always `undefined`. |
|
||||
| `API.md:321-331` | `runWorkflow(sessionPath, cubes, config, options?)` | Takes a fifth parameter `replaySession?: NopySession` (`nopy.workflow.ts:206-212`) — the entire history-replay path. |
|
||||
| `API.md:337-344` | `WorkflowResult.cubesWithDependencies` | Field is `selectedCubes` (`nopy.workflow.ts:31`). |
|
||||
| `API.md:202-209` | `DeployCall.dependencies: string[]` | `DependencySpec[]` (`nopy.executor.ts:27`) — and always `[]` in practice (`cubes/dependencies.ts:132`). |
|
||||
| `API.md:452-457` | `NopyConfig` with 4 fields | Missing `history` and `execution` (`nopy.config.ts:68-81`). |
|
||||
| `API.md:37-45` | 7 `NopyOptions` parameters | Missing `printOnly`, `replaySession`, `saveToHistory` (`nopy.main.ts:93-104`). |
|
||||
| `API.md:169-175` | `cubes.Manifest` example | Omits `id`, the field that determines the cube's identity. |
|
||||
|
||||
### 3.5 🟡 Exported and undocumented
|
||||
|
||||
Public API in `src/index.ts` with no `API.md` entry: the entire history module
|
||||
(`addToHistory`, `listHistory`, `getLastSession`, `getSessionById`,
|
||||
`clearHistory`, `removeFromHistory`, `loadHistory`, `saveHistory`,
|
||||
`getHistoryPath`, `formatHistoryList`, `HISTORY_FILE`, `DEFAULT_HISTORY_SIZE`,
|
||||
plus `HistoryEntry` / `SessionHistory`), `BuildContext`,
|
||||
`runSessionReplayWorkflow`, `getConfigPaths`, `findCubeDirectories`, `getCube`,
|
||||
`filterInternalVariables`, `separateEnvAndCubeVariables`.
|
||||
|
||||
The CLI cheat-sheet (`API.md:554-578`) omits `-R`, `-H`, `-P`, `--no-history`,
|
||||
and the `history` / `clear-history` commands.
|
||||
|
||||
`ManifestFactory` (`cubes/factories.ts:28`) is marked `@deprecated` but is not
|
||||
re-exported from `cubes/index.ts`, so it is unreachable dead code.
|
||||
|
||||
---
|
||||
|
||||
## 4. Undocumented behaviour
|
||||
|
||||
### 4.1 🔴 Cubes in this repo cannot be loaded
|
||||
|
||||
`CLAUDE.md` records the `@bitsquare/nopy` linking gotcha; the package README does
|
||||
not mention it at all, and the gotcha is incomplete.
|
||||
|
||||
Cube manifests are loaded by dynamic `import()` from their own directory
|
||||
(`cubes/loader.ts:75`), so they resolve their imports through ordinary Node
|
||||
resolution from `cubes/…`. Nothing in the tree provides either dependency:
|
||||
|
||||
```
|
||||
node_modules/@bitsquare/ → absent
|
||||
packages/nopy/node_modules/@bitsquare/ → absent
|
||||
cubes/node_modules/ → absent
|
||||
```
|
||||
|
||||
`@bitsquare/nopy` is the documented half. **`zod` is the other half** — 19 of 21
|
||||
manifests `import { z } from 'zod'`, and that fails independently of the nopy
|
||||
link. Verified by loading every manifest with a resolver hook: with only
|
||||
`@bitsquare/nopy` mapped, 19 of 21 fail `ERR_MODULE_NOT_FOUND: zod`.
|
||||
|
||||
Because `loadCubes` turns each failure into an `errors` entry and `nopy.main.ts:147-152`
|
||||
aborts when `errors.length > 0`, a fresh clone cannot run a single cube. Neither
|
||||
README mentions a setup step.
|
||||
|
||||
### 4.2 🟠 The SSH password is printed in plaintext
|
||||
|
||||
Not stated in any document, and it sits directly against the security notes at
|
||||
`README.md:217` and `:325` (which are narrowly about *storage*, and are correct
|
||||
as far as they go).
|
||||
|
||||
`buildDeployCall` embeds the password in the command string
|
||||
(`cubes/dependencies.ts:112-113`):
|
||||
|
||||
```typescript
|
||||
parts.push(`--user ${this.auth.username} --password ${this.auth.password}`);
|
||||
```
|
||||
|
||||
That string is then:
|
||||
|
||||
1. logged at debug level — `log.debug(\`Command: ${commandStr}\`)`
|
||||
(`nopy.executor.ts:76`) — and the `nopy` logger is configured with
|
||||
`lowestLevel: 'debug'` (`nopy.main.ts:41-44`), so it **prints to the console
|
||||
on every deployment**;
|
||||
2. printed unmasked by `--dry-run` — `outputExecutionPlan` masks values whose key
|
||||
contains "password" in the `--data` section (`nopy.executor.ts:134`) but prints
|
||||
`call.command.join(' ')` verbatim one line earlier (`nopy.executor.ts:127`);
|
||||
3. passed through `execa({shell: true})` (`nopy.executor.ts:79`), making it
|
||||
visible in the process list and, without quoting, vulnerable to shell
|
||||
metacharacters in the password.
|
||||
|
||||
### 4.3 🟠 History and session files record only prompted values
|
||||
|
||||
`README.md:414` says an entry records "the variable values that were answered at
|
||||
the prompts" — accurate, but the consequence is not drawn out.
|
||||
`buildDeployCall` records `this.variables.get(cubeId, 'prompts')`
|
||||
(`cubes/dependencies.ts:138`) — the prompts scope alone. Values that came from a
|
||||
dependency spec, a hook's `exec()`, `env`, or a schema default are **not** in the
|
||||
entry.
|
||||
|
||||
Combined with §2.1 (dependency params outrank prompts) this means a replay can
|
||||
legitimately produce a different command than the run it replays, if the
|
||||
dependency graph resolved differently.
|
||||
|
||||
### 4.4 🟡 `-P, --print-only` is undocumented
|
||||
|
||||
Implemented (`nopy.cli.ts:61`, `nopy.main.ts:202-213`), listed in the CLI's own
|
||||
help examples (`nopy.cli.ts:38`), and absent from `README.md` and `docs/API.md`.
|
||||
It prints the built pyinfra commands grouped by cube and exits.
|
||||
|
||||
Worth documenting alongside `--dry-run`, since the difference is not obvious:
|
||||
`--print-only` returns a `NopyResult` with `successful: 0` and skips execution
|
||||
entirely, while `--dry-run` goes through the executor.
|
||||
|
||||
### 4.5 🟡 `--save-session` is ignored during a replay
|
||||
|
||||
`nopy.main.ts:191` guards with `saveSessionPath && !workflow.isReplay`, so
|
||||
`nopy install -R -s out.json` writes nothing and says nothing. The
|
||||
"Recording a Session" section (`README.md:219-227`) does not mention it.
|
||||
|
||||
### 4.6 🟡 `.npcubes` is documented but unused in this repo
|
||||
|
||||
`README.md:43` shows a `cubes/.npcubes` marker in the layout diagram and
|
||||
`README.md:244` documents the mechanism. `find . -name .npcubes` returns nothing
|
||||
— discovery here runs entirely off `cubeDirs` in `.nopyrc.json`. The feature
|
||||
exists in `findCubeDirectories` (`cubes/loader.ts:26-30`); the diagram just shows
|
||||
a file that no reader will find if they go looking.
|
||||
|
||||
### 4.7 🟡 `ssh:keyman` depends on a global `env` value
|
||||
|
||||
`cubes/ssh/keyman/deploy.py` reads `host.data.get('KEY_DIR')`, which no manifest
|
||||
declares — it comes from `env` in `.nopyrc.json`. This works (§2.1: `env` is
|
||||
visible for keys the schema does not define) and it is the only cube relying on
|
||||
the mechanism, but nothing documents the coupling. Anyone running that cube from
|
||||
a project without `KEY_DIR` in their config gets `None`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Cube documentation
|
||||
|
||||
Two cubes have **no README at all**: `cubes/admin/hostname` and `cubes/git/clone`
|
||||
(19 of 21 have one).
|
||||
|
||||
### 5.1 🔴 `cubes/service/autostart/README.md` documents a different cube
|
||||
|
||||
The file is titled **"TypeStack Install Cube"** and describes cloning a git
|
||||
repository, `yarn install`, `yarn build`, `docker compose up -d`, and PM2 process
|
||||
management. The manifest (`service:autostart`, "Manage systemd service
|
||||
autostart") does none of that — it has three fields and calls `systemd.service`.
|
||||
|
||||
| README documents | In the schema? |
|
||||
|---|---|
|
||||
| `USER` | ❌ |
|
||||
| `REPO` | ❌ |
|
||||
| `ENV` | ❌ |
|
||||
| `NODE_PATH` | ❌ |
|
||||
| `APP` | ✅ |
|
||||
| `AUTOSTART` | ✅ |
|
||||
| — | `SERVICE_NAME` (undocumented) |
|
||||
|
||||
Every "Requirements" entry (Git, Yarn, Docker, PM2, NVM, SSH keys) is inapplicable.
|
||||
It reads as a leftover from a cube that was split or renamed.
|
||||
|
||||
### 5.2 🟠 `cubes/network/wifi/access-point/README.md` — four wrong parameters
|
||||
|
||||
| README | Manifest |
|
||||
|---|---|
|
||||
| `NETWORK_DEVICE` (default `wlan0`) | does not exist |
|
||||
| `CHANNEL` | does not exist — **but `deploy.py` reads it** (see §6.3) |
|
||||
| `IP_ADDRESS` (default `192.168.50.1`) | field is `AP_IP`, default `192.168.4.1` |
|
||||
| `CONNECTION_NAME` default `net:wifi:ap` | default is `pi-point` |
|
||||
| SSID / PASSWORD listed as "Required" | both have defaults (`PiPoint` / `1223334444`) |
|
||||
|
||||
Both worked examples set keys that will be ignored.
|
||||
|
||||
### 5.3 🟠 Two cubes claim to have no parameters
|
||||
|
||||
| Cube | README says | Schema has |
|
||||
|---|---|---|
|
||||
| `cubes/runtime/docker` | "This cube currently has no configurable parameters." | `DISTRO` (`ubuntu` \| `debian`) |
|
||||
| `cubes/runtime/nodevm` | "This cube currently has no configurable parameters." | `VERSION`, `USER`, `ALIAS`, `GLOBAL_PACKAGES` |
|
||||
|
||||
`nodevm`'s README also describes installing "the latest LTS via the official
|
||||
NodeSource setup script", while the manifest is named "Install nvm and nodejs
|
||||
with global packages" and takes an explicit `VERSION`.
|
||||
|
||||
(`admin/cockpit` and `armor/fail2ban` make the same claim and are correct — both
|
||||
have empty schemas.)
|
||||
|
||||
### 5.4 🟡 Cube id conventions are inconsistent
|
||||
|
||||
- `cubes/caddy/base` declares `id: 'caddy'` while its sibling declares
|
||||
`caddy:spa`. Every other nested cube uses the `group:name` form.
|
||||
- `net:wifi:connection` sets `name: 'network:wifi:connection - Connect to a WiFi
|
||||
network'` and `user:edit` sets `name: 'user:edit - Modify an existing user
|
||||
account'` — the id is baked into the display name. Since the picker renders
|
||||
`${cube.id} - ${cube.name}` (`nopy.prompts.ts:43`), these show as
|
||||
`net:wifi:connection - network:wifi:connection - Connect to a WiFi network`.
|
||||
`README.md:54` documents `[id]`-in-name as a *fallback* for a missing `id`
|
||||
field, not as a prefix to carry alongside one.
|
||||
|
||||
---
|
||||
|
||||
## 6. Defects found while verifying
|
||||
|
||||
Not documentation issues, but found while checking the docs and worth recording.
|
||||
|
||||
### 6.1 🔴 `cubes/service/autostart/deploy.py` cannot run
|
||||
|
||||
```python
|
||||
from pyinfra.operations import systemd # `server` is never imported
|
||||
from pyinfra import host
|
||||
|
||||
APP = host.data.APP # AUTOSTART and SERVICE_NAME never read
|
||||
|
||||
if AUTOSTART: # NameError
|
||||
...
|
||||
server.shell(...) # NameError, in the else branch
|
||||
```
|
||||
|
||||
`AUTOSTART` and `SERVICE_NAME` are declared in the manifest and never pulled off
|
||||
`host.data`; `server` is used but not imported. The script raises `NameError` on
|
||||
the `if`. Per §2.2 this cube also gets no `--data` at all, so it fails twice over.
|
||||
|
||||
### 6.2 🔴 `-H <id>` and `--no-history` share one destination
|
||||
|
||||
Both options write to `options.history` (`nopy.cli.ts:57` and `:64`). Verified
|
||||
with Commander:
|
||||
|
||||
```
|
||||
argv=[] -> {} # undefined → saves
|
||||
argv=["--no-history"] -> {"history":false} # correct
|
||||
argv=["-H","abc123"] -> {"history":"abc123"} # correct
|
||||
argv=["-H","abc123","--no-history"] -> {"history":false} # id destroyed
|
||||
```
|
||||
|
||||
In the last case the `-H` argument is silently discarded and nopy falls through
|
||||
to a full interactive run instead of replaying. `saveToHistory: options.history
|
||||
!== false` (`nopy.cli.ts:104`) works only because the two meanings happen not to
|
||||
collide in the common cases.
|
||||
|
||||
### 6.3 🟠 `access-point/deploy.py` reads an undeclared variable
|
||||
|
||||
`host.data.get('CHANNEL')` — no manifest declares `CHANNEL`, so it is always
|
||||
`None`. The README documents it as a supported optional parameter (§5.2). One of
|
||||
the three has to give.
|
||||
|
||||
### 6.4 🟡 Debug output left in the shipped code
|
||||
|
||||
- `nopy.common.ts:22` — `console.log('Assigning', artefactId, scope, values)`
|
||||
fires on every variable assignment, printing values to the console. Combined
|
||||
with §4.2 this is a second path by which secrets reach stdout.
|
||||
- `keyman.encrypt.ts:19-20` — `console.log(tmpKeys); console.log(sshKeys);`
|
||||
before the prompt.
|
||||
|
||||
### 6.5 🟡 No cycle detection
|
||||
|
||||
Covered under §1.5. `docs/API.md:160` documents the error; there is no code that
|
||||
raises it. Mutually dependent cubes recurse until the stack overflows.
|
||||
|
||||
---
|
||||
|
||||
## 7. Checked and accurate
|
||||
|
||||
Recording what was verified and found correct, so a future pass need not redo it.
|
||||
|
||||
- **`README.PUBLISH.md`** — checked against `.gitea/workflows/*.yml` and both
|
||||
manifests. Workflow triggers, the `files` array, dist-tag rules, the snapshot
|
||||
version format, `upload-artifact@v3`, `cache@v4`, the `npm pack --dry-run`
|
||||
step, `retention-days: 7`, and all four coverage thresholds are right. The
|
||||
only file that states the coverage gate completely.
|
||||
- **Root `README.md`** — pnpm/corepack, Node ≥ 22 with `.nvmrc` pinning 24, the
|
||||
script table, and both git hooks match the root `package.json`. Only the
|
||||
coverage line is incomplete (§2.11).
|
||||
- **`CLAUDE.md`** — accurate throughout, including the `logConfigToFlags` drift
|
||||
note and the resolution/merge description. Two additions worth making: `zod` is
|
||||
missing from the cube-linking gotcha (§4.1), and `-D` being a no-op (§1.1)
|
||||
belongs under "Known drift".
|
||||
- **`README.md` history section** (`:392-429`) — the recording rules, the
|
||||
fail-then-`-R` flow, replay-does-not-re-record, the four suppression cases, the
|
||||
`defaults`-layer replay semantics, and the `Cube not found` failure mode all
|
||||
check out against `nopy.history.ts`, `nopy.main.ts:195-200` and
|
||||
`cubes/dependencies.ts:62-69`.
|
||||
- **`README.md` continue-on-error section** (`:369-390`) — fail-fast, no
|
||||
rollback, skipped-cubes-absent-from-results, exit code 1, and CLI-over-config
|
||||
precedence match `nopy.executor.ts:180-193` and `nopy.cli.ts:67-68`.
|
||||
- **`README.md` cube layout and discovery** — the directory-pair rule, recursive
|
||||
scan, dotted/`node_modules` skipping, the prefixed `*.manifest.mjs` fallback,
|
||||
and the three-step id resolution match `cubes/loader.ts` exactly.
|
||||
- **pyinfra `--data` type coercion** (`README.md:101`) — correct.
|
||||
- **keyman config** — priority (`VAULT_ROOT` > file > defaults), the four default
|
||||
values, and the vault layout match `keyman.config.ts` and `keyman.encrypt.ts`.
|
||||
|
||||
---
|
||||
|
||||
## Suggested order of attack
|
||||
|
||||
**1 — Decide on the three phantom features.** §1.1 (`-D`), §1.2 (`--json`) and
|
||||
§1.3 (`log.*`) are all "documented, wired up, never read". Each is a small
|
||||
implementation or a small deletion; what they cannot stay is documented as
|
||||
working. `-D` is the most damaging, since it is advertised as the unattended path.
|
||||
|
||||
**2 — Fix `getDefaults()` swallowing the schema (§2.2).** One `catch` block
|
||||
silently disables three cubes today and will disable any cube whose author
|
||||
forgets a `.default()`. Falling back per-field, or surfacing the parse error as a
|
||||
load-time `errors` entry, both beat returning `{}`.
|
||||
|
||||
**3 — Correct the precedence table (§2.1).** Four lines of README, but it
|
||||
documents the system backwards in both directions, and the `env`-override promise
|
||||
is the one people will build on.
|
||||
|
||||
**4 — Decide the `.describe()`/`.default()` ordering (§2.3).** Either read
|
||||
through the `ZodDefault` wrapper in `nopy.prompts.ts`, or fix the ordering in all
|
||||
14 manifests and the README example. The first is one line and cannot regress.
|
||||
|
||||
**5 — Regenerate `docs/API.md` (§3).** Too far gone to patch: two core types,
|
||||
one whole module, and two functions describe code that no longer exists.
|
||||
|
||||
**6 — Cube docs (§5) and the two missing READMEs.** `service/autostart` is the
|
||||
worst — its README belongs to a different cube, and its `deploy.py` does not run
|
||||
at all (§6.1).
|
||||
|
||||
**7 — Secrets on stdout (§4.2, §6.4).** Drop the `console.log` in
|
||||
`Variables.assign`, mask the password in the executor's debug line and in the
|
||||
dry-run plan, and pass `--user`/`--password` as argv rather than interpolating
|
||||
into a shell string.
|
||||
+141
-69
@@ -6,15 +6,52 @@ A CLI tool that simplifies **pyinfra** script management and execution, providin
|
||||
|
||||
Nopy wraps pyinfra with structure, validation, and an interactive experience for managing complex infrastructure deployments. It organizes deployments into self-contained "cubes" with dependency management, schema validation, and lifecycle hooks.
|
||||
|
||||
## Features
|
||||
|
||||
- **Dependency resolution** with topological sorting
|
||||
- **Before/after hooks** for multi-cube orchestration
|
||||
- **SSH key or password authentication**
|
||||
- **Default values** with optional customization via manifest `env`
|
||||
- **Schema validation** using Zod
|
||||
- **Recursive cube directory discovery**
|
||||
- **Dry-run mode** for previewing deployments
|
||||
- **JSON output** for CI/CD integration
|
||||
- **Session history** with replay capability
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Load cubes** - Discovers and validates cubes from configured directories
|
||||
2. **Interactive prompts** - Select cubes, target host, and authentication method
|
||||
3. **Dependency resolution** - Topologically sorts cubes based on dependencies
|
||||
4. **Variable assignment** - Validates and collects configuration with schema validation
|
||||
5. **Execute hooks** - Runs before/after hooks for orchestration
|
||||
6. **Deploy** - Sequentially executes pyinfra commands
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Cubes
|
||||
|
||||
Self-contained deployment units consisting of:
|
||||
A cube is a **directory** containing two files:
|
||||
|
||||
- **Python deployment script**: `<cube-name>.deploy.py`
|
||||
- **JavaScript manifest**: `<cube-name>.manifest.mjs` defining schema, dependencies, defaults, and hooks
|
||||
- **Configuration variables**: Validated with Zod schemas
|
||||
- **JavaScript manifest**: `manifest.mjs` defining schema, dependencies, defaults, and hooks
|
||||
- **Python deployment script**: `deploy.py`, a plain pyinfra script
|
||||
|
||||
Configuration variables are declared in the manifest and validated with Zod schemas before the deployment script runs.
|
||||
|
||||
```
|
||||
cubes/
|
||||
├── .npcubes
|
||||
└── apt/
|
||||
└── install/
|
||||
├── manifest.mjs
|
||||
└── deploy.py
|
||||
```
|
||||
|
||||
Any directory holding both files is treated as a cube, so cubes can be nested as deeply as you like to group them by topic. Discovery is recursive; directories starting with `.` and `node_modules` are skipped. Additional files in the cube directory (a `README.md`, config templates, and so on) are ignored by the loader and can be referenced from the deploy script — the script runs with its cube directory as the working directory.
|
||||
|
||||
The prefixed forms `<cube-name>.manifest.mjs` and `<cube-name>.deploy.py` are also still recognized, but plain `manifest.mjs` / `deploy.py` is the current convention.
|
||||
|
||||
A cube's identity comes from the manifest's `id` field (see below). If `id` is omitted, nopy falls back to an `[id]` prefix in the manifest `name`, and finally to the directory's own name. Note that the id does not have to mirror the folder path — `cubes/network/tailscale` declares `id: 'net:tailscale'`.
|
||||
|
||||
#### Cube Manifest
|
||||
|
||||
@@ -33,6 +70,36 @@ export default cubes.Manifest({
|
||||
})
|
||||
```
|
||||
|
||||
#### Deployment Script
|
||||
|
||||
The matching `deploy.py` is a plain pyinfra script. Nopy passes each schema variable to pyinfra as `--data KEY=value`, so they are available on `host.data`:
|
||||
|
||||
```python
|
||||
from pyinfra import host
|
||||
from pyinfra.operations import apt
|
||||
|
||||
UPDATE = host.data.UPDATE
|
||||
PACKAGES = str(host.data.PACKAGES).split(' ')
|
||||
|
||||
apt.packages(
|
||||
name='Install essential packages',
|
||||
packages=['ca-certificates', 'gnupg', 'lsb-release'],
|
||||
update=UPDATE,
|
||||
_sudo=True,
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name='Install custom packages',
|
||||
packages=[p.strip() for p in PACKAGES if p],
|
||||
update=UPDATE,
|
||||
_sudo=True,
|
||||
)
|
||||
```
|
||||
|
||||
Every key defined in the manifest `schema` is guaranteed to be present on `host.data` — either from the Zod `.default()`, from `.nopyrc.json`, from a dependency, or from a user prompt.
|
||||
|
||||
**Value types**: pyinfra parses `--data` values before your script sees them. `"true"` / `"false"` become booleans, numeric strings become `int`, valid JSON becomes the parsed structure, and everything else stays a string. This is why `UPDATE` can be handed straight to pyinfra's `update=` argument, while `PACKAGES` is wrapped in `str(...)` before splitting.
|
||||
|
||||
#### Variable Defaults
|
||||
|
||||
Variable defaults are defined directly in the Zod schema using `.default()`. This ensures that every cube has a predictable starting state and provides type-safe default values.
|
||||
@@ -60,10 +127,19 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
|
||||
"log": {
|
||||
"verbosity": "info",
|
||||
"debug": false
|
||||
},
|
||||
"history": {
|
||||
"maxSessions": 10,
|
||||
"autoSave": true
|
||||
},
|
||||
"execution": {
|
||||
"continueOnError": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`history` controls automatic session recording (see [Deployment History](#deployment-history)), and `execution.continueOnError` sets the default for `--continue-on-error`.
|
||||
|
||||
#### Logging Configuration
|
||||
|
||||
Control pyinfra output verbosity and debug information using the `log` configuration object:
|
||||
@@ -84,39 +160,6 @@ Control pyinfra output verbosity and debug information using the `log` configura
|
||||
| `false` | (none) | No debug logs (default) | Normal operation |
|
||||
| `true` | `--debug` | Enable pyinfra debug logs | Deep debugging of pyinfra internals |
|
||||
|
||||
**Examples:**
|
||||
|
||||
Basic troubleshooting:
|
||||
|
||||
```json
|
||||
{
|
||||
"log": {
|
||||
"verbosity": "info"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Debug command failures:
|
||||
|
||||
```json
|
||||
{
|
||||
"log": {
|
||||
"verbosity": "trace"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Deep debugging with pyinfra internals:
|
||||
|
||||
```json
|
||||
{
|
||||
"log": {
|
||||
"verbosity": "trace",
|
||||
"debug": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Recommendation:** Start with `"info"` for typical troubleshooting, use `"trace"` when investigating command failures, and enable `debug: true` only when debugging pyinfra itself.
|
||||
|
||||
### Session Recording and Replay
|
||||
@@ -264,11 +307,23 @@ nopy install -K
|
||||
**Repeat last run**:
|
||||
|
||||
```bash
|
||||
nopy install --repeat-last-run
|
||||
nopy install --repeat-last
|
||||
# or
|
||||
nopy install -R
|
||||
```
|
||||
|
||||
Every deployment is automatically recorded to a `.nopy.history.json` file in the current working directory, so the last run is always available to `-R` without having to pass `--save-session` first. The default retention is the 10 most recent sessions (configurable via `history.maxSessions`); use `nopy history` to list them and `nopy install -H <id>` to replay any one of them — see [Deployment History](#deployment-history).
|
||||
|
||||
The recording happens before the deploy commands run, so a **failed** deployment is recorded too — `-R` is the quick way to retry one after fixing the cause. Replaying a session with `-R` or `-H` does not itself create a new entry, so repeating never pushes the original run out of the list.
|
||||
|
||||
A run is *not* recorded when:
|
||||
|
||||
- `--dry-run` or `--no-history` is passed
|
||||
- No cubes were selected, so there was nothing to deploy
|
||||
- `history.autoSave` is set to `false` in `.nopyrc.json`
|
||||
|
||||
Because the history file is resolved against the current working directory, each project keeps its own history — running nopy from a different directory will not find the previous run. As with session files, passwords are never stored and are re-prompted on replay.
|
||||
|
||||
**Save session for replay**:
|
||||
|
||||
```bash
|
||||
@@ -302,14 +357,6 @@ nopy install --dry-run
|
||||
|
||||
Shows the execution plan including commands, environment variables, and targets without running anything. Sensitive data is masked in output.
|
||||
|
||||
**Parallel execution**:
|
||||
|
||||
```bash
|
||||
nopy install --parallel
|
||||
```
|
||||
|
||||
Executes independent cubes in parallel using a dependency graph. Cubes are grouped into execution stages, with a default concurrency limit of 4.
|
||||
|
||||
**JSON output (for CI/CD)**:
|
||||
|
||||
```bash
|
||||
@@ -323,17 +370,64 @@ Machine-readable JSON output for scripting and CI/CD integration.
|
||||
|
||||
```bash
|
||||
nopy install --continue-on-error
|
||||
# or
|
||||
nopy install -c
|
||||
```
|
||||
|
||||
Continue deploying remaining cubes even if one fails.
|
||||
|
||||
**View deployment history**:
|
||||
**Default behaviour (fail-fast)**: without this flag, nopy stops at the first cube that fails. Cubes are deployed sequentially in dependency order, so the failing cube's output is the last thing you see — every cube still queued behind it is skipped entirely and is never attempted.
|
||||
|
||||
This is deliberate: because cubes are topologically sorted, a cube that fails is often a dependency of the ones after it, and continuing would deploy them onto a half-configured host.
|
||||
|
||||
Two consequences worth knowing:
|
||||
|
||||
- **Cubes that already succeeded are not rolled back.** The host is left in a partial state — the cubes before the failure are applied, the rest are not. Fix the cause and re-run; well-written cubes are idempotent, so re-applying the earlier ones is normally harmless.
|
||||
- **Skipped cubes are not reported as failed.** They are simply absent from the results, so a summary of "3 successful, 1 failed" out of 6 cubes means the remaining 2 were never run.
|
||||
|
||||
Either way, the command exits with code `1` if any cube failed, which is what CI picks up. Use `--continue-on-error` when your cubes are genuinely independent and you would rather collect every failure in one run than stop at the first.
|
||||
|
||||
The default can be flipped for a project by setting `execution.continueOnError` in `.nopyrc.json`; the CLI flag takes precedence over it.
|
||||
|
||||
#### Deployment History
|
||||
|
||||
```bash
|
||||
nopy history # List recent deployments
|
||||
nopy history --json # Same list as JSON, including each recorded session
|
||||
nopy install -H <id> # Replay a specific deployment by ID
|
||||
nopy clear-history # Delete all recorded sessions
|
||||
```
|
||||
|
||||
History is what makes [Repeat last run](#basic-commands) work, but it holds more than just the last deployment: every recorded run stays replayable until newer runs push it out. `nopy history` (alias `nopy h`) lists them newest first, with `→` marking the entry that `-R` would replay:
|
||||
|
||||
```
|
||||
Session History:
|
||||
|
||||
→ [1] 07/26/2026, 14:32 - apt:install, net:tailscale → root@web-01
|
||||
ID: mdk3n1qx4a2fh
|
||||
[2] 07/26/2026, 09:05 - apt:install → root@web-01
|
||||
ID: mdk0zzp8b71cq
|
||||
|
||||
Total: 2 session(s)
|
||||
```
|
||||
|
||||
Each entry records the selected cubes together with the variable values that were answered at the prompts, the target hosts, the authentication method, and the username — never the password. Pass an ID to `-H` to run that exact combination again:
|
||||
|
||||
```bash
|
||||
nopy install -H mdk0zzp8b71cq
|
||||
```
|
||||
|
||||
A replay is non-interactive: cube selection, host, and variable values all come from the entry, so nopy runs straight through without asking anything. The two exceptions are password authentication, which always re-prompts, and an entry with no recorded host, which falls back to the host picker.
|
||||
|
||||
Two things are worth knowing before relying on an older entry:
|
||||
|
||||
- **Recorded values are applied as defaults, not as a frozen snapshot.** If a cube's schema has gained a variable since the run was recorded, the replay neither prompts for it nor fails — the new variable quietly takes its Zod `.default()`. Global `env` values are likewise read from the *current* `.nopyrc.json` rather than from the entry.
|
||||
- **A replay fails if a cube no longer exists.** Renaming or deleting a cube id makes every history entry that referenced it unreplayable: nopy logs `Cube from session not found` and then aborts with `Cube not found: <id>`.
|
||||
|
||||
The history lives in `.nopy.history.json` in the working directory and uses the same structure as a session file, so trimming the array by hand is a perfectly good way to prune it. It does contain the variable values that were entered, which is why it is listed in this repository's `.gitignore` — treat it like any other file holding deployment configuration. A corrupt or unreadable history file is treated as empty rather than raising an error, which looks exactly like a project that has never been deployed from.
|
||||
|
||||
For a run you want to keep indefinitely, don't rely on history — it rotates. Use `--save-session` to write it to a file you control (see [Session Recording and Replay](#session-recording-and-replay)).
|
||||
|
||||
### Development
|
||||
|
||||
**Run without building**:
|
||||
@@ -348,28 +442,6 @@ npm run nopy
|
||||
npm run debug
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Load cubes** - Discovers and validates cubes from configured directories
|
||||
2. **Interactive prompts** - Select cubes, target host, and authentication method
|
||||
3. **Dependency resolution** - Topologically sorts cubes based on dependencies
|
||||
4. **Variable assignment** - Validates and collects configuration with schema validation
|
||||
5. **Execute hooks** - Runs before/after hooks for orchestration
|
||||
6. **Deploy** - Sequentially executes pyinfra commands
|
||||
|
||||
## Features
|
||||
|
||||
- **Dependency resolution** with topological sorting
|
||||
- **Parallel execution** of independent cubes in stages
|
||||
- **Before/after hooks** for multi-cube orchestration
|
||||
- **SSH key or password authentication**
|
||||
- **Default values** with optional customization via manifest `env`
|
||||
- **Schema validation** using Zod
|
||||
- **Recursive cube directory discovery**
|
||||
- **Dry-run mode** for previewing deployments
|
||||
- **JSON output** for CI/CD integration
|
||||
- **Session history** with replay capability
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Cube Hooks](docs/HOOKS.md) - Lifecycle hooks for dynamic orchestration
|
||||
|
||||
+21
-29
@@ -41,7 +41,6 @@ const result = await nopy({
|
||||
| `saveSession` | `string` | - | Path to save session file |
|
||||
| `loadSession` | `string` | - | Path to load session for replay |
|
||||
| `dryRun` | `boolean` | `false` | Show execution plan without running |
|
||||
| `parallel` | `boolean` | `false` | Execute independent cubes in parallel |
|
||||
| `continueOnError` | `boolean` | `false` | Continue after failures |
|
||||
| `jsonOutput` | `boolean` | `false` | Output results as JSON |
|
||||
|
||||
@@ -87,7 +86,7 @@ interface Cube<Schema extends z.AnyZodObject = z.AnyZodObject> {
|
||||
|
||||
#### `Manifest<Schema>`
|
||||
|
||||
Cube manifest (used in `*.manifest.mjs` files).
|
||||
Cube manifest (used in `manifest.mjs` files).
|
||||
|
||||
```typescript
|
||||
interface Manifest<Schema extends z.AnyZodObject = z.AnyZodObject> {
|
||||
@@ -160,23 +159,14 @@ const order = resolveDependencies(cubes, ['apt-all']);
|
||||
|
||||
**Throws:** `Error` if cube not found or circular dependency detected
|
||||
|
||||
#### `buildExecutionStages(cubes, selectedCubeNames)`
|
||||
|
||||
Groups cubes into stages for parallel execution.
|
||||
|
||||
```typescript
|
||||
const stages = buildExecutionStages(cubes, ['apt-all', 'docker']);
|
||||
// Returns: [['apt:essentials'], ['apt-more', 'docker'], ['apt-all']]
|
||||
```
|
||||
|
||||
**Returns:** `string[][]` - Array of stages
|
||||
|
||||
#### `createManifest(options)`
|
||||
#### `cubes.Manifest(options)`
|
||||
|
||||
Factory function for creating cube manifests.
|
||||
|
||||
```typescript
|
||||
export default createManifest({
|
||||
import { cubes } from '@bitsquare/nopy';
|
||||
|
||||
export default cubes.Manifest({
|
||||
name: 'My Cube',
|
||||
dependencies: () => [['apt:essentials']],
|
||||
schema: z.object({
|
||||
@@ -185,6 +175,8 @@ export default createManifest({
|
||||
});
|
||||
```
|
||||
|
||||
`createManifest` and `manifest` are exported as equivalent aliases; `cubes.Manifest` is the documented form.
|
||||
|
||||
#### `uniqid(length?)`
|
||||
|
||||
Generates a random alphanumeric string.
|
||||
@@ -239,8 +231,6 @@ Options for deployment execution.
|
||||
|
||||
```typescript
|
||||
interface ExecutionOptions {
|
||||
parallel?: boolean;
|
||||
concurrency?: number;
|
||||
continueOnError?: boolean;
|
||||
dryRun?: boolean;
|
||||
onProgress?: (result: ExecutionResult, completed: number, total: number) => void;
|
||||
@@ -252,12 +242,11 @@ interface ExecutionOptions {
|
||||
|
||||
#### `executeDeployCalls(calls, options?)`
|
||||
|
||||
Executes an array of deployment calls.
|
||||
Executes an array of deployment calls sequentially, in the order they were built.
|
||||
|
||||
```typescript
|
||||
const results = await executeDeployCalls(calls, {
|
||||
parallel: true,
|
||||
concurrency: 4,
|
||||
continueOnError: false,
|
||||
onProgress: (result, completed, total) => {
|
||||
console.log(`${completed}/${total}`);
|
||||
},
|
||||
@@ -581,9 +570,6 @@ nopy install -l ./my-session.json
|
||||
# Dry run
|
||||
nopy install -n
|
||||
|
||||
# Parallel execution
|
||||
nopy install -p
|
||||
|
||||
# JSON output
|
||||
nopy install -j
|
||||
|
||||
@@ -597,21 +583,27 @@ nopy install -c
|
||||
|
||||
### File Structure
|
||||
|
||||
A cube is a directory containing both a `manifest.mjs` and a `deploy.py`:
|
||||
|
||||
```
|
||||
cubes/
|
||||
└── my-cube/
|
||||
├── my-cube.manifest.mjs
|
||||
└── my-cube.deploy.py
|
||||
├── manifest.mjs
|
||||
└── deploy.py
|
||||
```
|
||||
|
||||
Cube directories may be nested for grouping (`cubes/apt/install/`), and any extra files alongside the pair are available to the deploy script via relative paths.
|
||||
|
||||
The prefixed forms `<cube-name>.manifest.mjs` and `<cube-name>.deploy.py` are still recognized for backwards compatibility.
|
||||
|
||||
### Manifest Example
|
||||
|
||||
```javascript
|
||||
// my-cube.manifest.mjs
|
||||
import { createManifest } from '@bitsquare/nopy';
|
||||
// manifest.mjs
|
||||
import { cubes } from '@bitsquare/nopy';
|
||||
import { z } from 'zod';
|
||||
|
||||
export default createManifest({
|
||||
export default cubes.Manifest({
|
||||
name: 'My Cube',
|
||||
dependencies: () => [['apt:essentials']],
|
||||
schema: z.object({
|
||||
@@ -634,7 +626,7 @@ export default createManifest({
|
||||
### Deploy Script Example
|
||||
|
||||
```python
|
||||
# my-cube.deploy.py
|
||||
# deploy.py
|
||||
from pyinfra import host
|
||||
from pyinfra.operations import apt, server
|
||||
|
||||
|
||||
@@ -0,0 +1,529 @@
|
||||
# Cube bundles as npm packages
|
||||
|
||||
Status: **plan, not a record.** Nothing here is implemented yet.
|
||||
|
||||
Distributing cubes as npm packages so a project can `pnpm add @acme/cubes-net`
|
||||
and have its cubes show up in `nopy` alongside local ones.
|
||||
|
||||
## Goals
|
||||
|
||||
- A cube bundle is an ordinary npm package, publishable to npmjs or Gitea
|
||||
through the existing release lanes.
|
||||
- A consuming project opts into a bundle explicitly, by name, in `.nopyrc.json`.
|
||||
- Existing manifests, dependency specs (`dependencies: () => ['apt:essentials']`)
|
||||
and stored session history keep working untouched.
|
||||
- The in-repo `cubes/` tree becomes the first published bundle, proving the path.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Automatic discovery of bundles from the dependency tree. Cubes run privileged
|
||||
deploy scripts against real hosts; a transitive dependency contributing one
|
||||
silently is a supply-chain hole. Opt-in per package, always.
|
||||
- Namespacing or id rewriting. Ids stay flat and global (see *Decisions*).
|
||||
- Version compatibility checks between a bundle and the `nopy` running it.
|
||||
Noted as a risk, deferred.
|
||||
|
||||
## Decisions
|
||||
|
||||
| Question | Decision |
|
||||
| --- | --- |
|
||||
| Duplicate cube ids across sources | **Hard error.** No precedence, no shadowing. Mitigation is a good error message, not a fallback. |
|
||||
| Id format | Unchanged, flat. The id is the session key (`dependencies.ts:135`); changing it breaks `--repeat-last` and `--history`. |
|
||||
| Discovery | Explicit `cubePackages` list in `.nopyrc.json`. |
|
||||
| Migrate in-repo `cubes/` | Yes — `packages/cubes-core`, as the proof of concept. |
|
||||
| Split an authoring package (`@bitsquare/nopy-cube`) | **Yes.** Bundles take a regular dependency on it; `@bitsquare/nopy` re-exports it for backwards compatibility. See *Phase 4*. |
|
||||
|
||||
## Current state
|
||||
|
||||
What already works, unchanged:
|
||||
|
||||
- `--chdir <cubeDir>` (`nopy.executor.ts`) means a `deploy.py` under
|
||||
`node_modules` runs fine; pyinfra only needs the path.
|
||||
- `scanDirectory` skips `node_modules` when *descending* (`loader.ts:101`), not
|
||||
for the root it is handed. So `"cubeDirs": ["./node_modules/@acme/cubes-net/cubes"]`
|
||||
works today. That is the escape hatch until this lands, and it stays working
|
||||
afterwards.
|
||||
|
||||
What blocks a clean story:
|
||||
|
||||
1. **Manifest imports.** `manifest.mjs` does `import { cubes } from '@bitsquare/nopy'`,
|
||||
resolved by ordinary Node resolution from the manifest's own directory. From
|
||||
inside `node_modules/@acme/cubes-net/`, that resolves upward into the
|
||||
consumer's `node_modules` — fine if the consumer installed `@bitsquare/nopy`,
|
||||
`ERR_MODULE_NOT_FOUND` if `nopy` is only installed globally. Same gotcha
|
||||
CLAUDE.md already documents for the local `cubes/` tree.
|
||||
2. **No way to name a package** in config, only paths.
|
||||
3. **Recursively scanning `node_modules` is not a workaround.** pnpm symlinks
|
||||
direct deps, and `readdir(withFileTypes)` reports a symlink as
|
||||
`isSymbolicLink()`, not `isDirectory()` — the scan would skip every package.
|
||||
Package roots must be resolved explicitly.
|
||||
|
||||
## Phase 0 — fixes that land first
|
||||
|
||||
Independent of packaging, and the duplicate-id work depends on them.
|
||||
|
||||
**0.1 `scanDirectory` drops subtrees on duplicates.** `loader.ts:84-87` pushes
|
||||
the error and `return`s, which exits before the recursive descent at line 100.
|
||||
Cubes nested below a duplicate never get scanned, so the error report is
|
||||
incomplete: you fix one collision, re-run, find the next. Should record the
|
||||
duplicate and keep descending.
|
||||
|
||||
**0.2 Duplicate detection is order-dependent.** `loadCubes()` runs
|
||||
`Promise.all` over folders into a shared `cubes` object, so which source is
|
||||
"first" and which is "the duplicate" varies run to run. Restructure: the scan
|
||||
emits a flat list of candidates, then a single grouping pass builds `cubes` and
|
||||
the error list. Makes the hard-error path deterministic, which the tests need.
|
||||
|
||||
**0.3 `apt:essentials` is already declared twice.** `cubes/apt/essentials`
|
||||
declares it via `id`; `packages/nopy/cubes/apt/essentials` declares it via the
|
||||
`[apt:essentials]` prefix in `name`. `cubeDirs` merges root-first, so running
|
||||
`nopy` from `packages/nopy` already collects both and errors. Rename the
|
||||
`packages/nopy/cubes` fixtures (`[test:apt-essentials]`, `[test:apt-all]`,
|
||||
`[test:apt-more]`) — they are dev fixtures, not real cubes, and the migration in
|
||||
Phase 5 makes the collision permanent otherwise.
|
||||
|
||||
**0.4 `coerceValue` breaks if zod is ever duplicated.** `nopy.prompts.ts:147-154`
|
||||
discriminates with `instanceof z.ZodDefault`, `z.ZodBoolean`, `z.ZodNumber` and
|
||||
friends — checks against the *running CLI's* zod instance. The moment a bundle
|
||||
resolves its own copy of zod (entirely possible once manifests arrive from
|
||||
`node_modules`; see Phase 4), every check returns false and `coerceValue` falls
|
||||
through to the raw string, silently. Booleans stop being booleans.
|
||||
|
||||
Rewrite against the string discriminant, which is instance-agnostic. Verified on
|
||||
the installed zod 4.4.3:
|
||||
|
||||
```
|
||||
z.boolean().default(false).def.type → 'default'
|
||||
z.boolean().default(false).def.innerType → { def: { type: 'boolean' } }
|
||||
z.number().def.type → 'number'
|
||||
```
|
||||
|
||||
Do this before anything else in Phase 4 lands, and it stops being a footgun for
|
||||
the local `cubes/` tree too.
|
||||
|
||||
## Phase 1 — the bundle contract
|
||||
|
||||
A cube bundle is an npm package with a `nopy` field:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "@acme/cubes-net",
|
||||
"version": "1.0.0",
|
||||
"type": "module",
|
||||
"nopy": { "cubes": ["./cubes"] },
|
||||
"files": ["cubes", "README.md", "LICENSE"],
|
||||
"keywords": ["nopy", "nopy-cubes", "pyinfra"],
|
||||
"dependencies": {
|
||||
"@bitsquare/nopy-cube": "^1.0.0",
|
||||
"zod": "^4.4.3"
|
||||
},
|
||||
"publishConfig": { "access": "public" }
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `nopy.cubes` — directories relative to the package root, scanned exactly like
|
||||
`cubeDirs` entries. Required; a package listed in `cubePackages` without a
|
||||
`nopy` field is an error, not a silent skip. Listing it means the user expects
|
||||
cubes from it.
|
||||
- Both dependencies are **regular dependencies, not peers**, and both are
|
||||
load-bearing: a manifest imports `Manifest` from `@bitsquare/nopy-cube` and `z`
|
||||
from `zod`. `@bitsquare/nopy-cube` peer-depends on zod, so the bundle's copy is
|
||||
the one everybody uses — see Phase 4.
|
||||
- The package needs no `exports` entry for this to work — resolution reads
|
||||
`package.json` off disk (Phase 2), so the `exports` map is irrelevant.
|
||||
- **A bundle's directory is read-only at runtime.** Under pnpm, `node_modules`
|
||||
content is hardlinked into the global store; a cube writing next to its own
|
||||
`deploy.py` corrupts that store for every project on the machine. Cubes must
|
||||
write to `/tmp` or the remote host, never their own dir.
|
||||
- A bundle must not ship a `.nopyrc.json`. Config discovery walks up from
|
||||
`process.cwd()`, never from cube directories, so it would never be read.
|
||||
|
||||
## Phase 2 — resolution
|
||||
|
||||
### Config surface
|
||||
|
||||
```json
|
||||
{
|
||||
"cubePackages": ["@acme/cubes-net", "@acme/cubes-caddy"]
|
||||
}
|
||||
```
|
||||
|
||||
Merges through the existing `resolution` machinery for free — arrays concat and
|
||||
dedupe — so a parent config supplies the org baseline and a child adds to it.
|
||||
|
||||
**Resolution origin.** A package must be resolved from *the directory of the
|
||||
config file that declared it*, not from `process.cwd()`. Otherwise a bundle
|
||||
listed in `~/.nopyrc.json` cannot resolve unless every project happens to depend
|
||||
on it. This is the same problem `PATH_PROPERTIES` solves for `cubeDirs`, but the
|
||||
output is a tagged reference rather than a rewritten string:
|
||||
|
||||
```ts
|
||||
export interface CubePackageRef {
|
||||
spec: string; // '@acme/cubes-net'
|
||||
from: string; // dirname of the .nopyrc.json that declared it
|
||||
}
|
||||
```
|
||||
|
||||
So the file format and the loaded format diverge for this one key:
|
||||
|
||||
```ts
|
||||
interface NopyConfigFile extends Omit<Partial<NopyConfig>, 'cubePackages'> {
|
||||
cubePackages?: string[];
|
||||
resolution?: ResolutionConfig;
|
||||
}
|
||||
|
||||
interface NopyConfig {
|
||||
cubePackages: CubePackageRef[];
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
`resolveConfigPaths()` performs the `string → CubePackageRef` conversion, next to
|
||||
where it resolves `PATH_PROPERTIES`. Two consequences to handle:
|
||||
|
||||
- `mergeValue`'s array dedupe only fires when every element is a primitive
|
||||
(`config.ts:142`), so refs fall through to plain concat. Dedupe by `spec` in
|
||||
the resolver instead.
|
||||
- Dedupe is **last-wins**: merge order is root-first, so the last occurrence is
|
||||
the most specific config, and its `from` is the right resolution origin.
|
||||
|
||||
### Resolver
|
||||
|
||||
New file `packages/nopy/src/cubes/packages.ts`:
|
||||
|
||||
```ts
|
||||
export interface CubePackage {
|
||||
name: string;
|
||||
root: string;
|
||||
dirs: string[]; // absolute, from nopy.cubes
|
||||
}
|
||||
|
||||
export function resolveCubePackages(
|
||||
refs: CubePackageRef[]
|
||||
): { packages: CubePackage[]; errors: string[] };
|
||||
```
|
||||
|
||||
Locate the package root without going through `exports` and without tripping on
|
||||
pnpm symlinks:
|
||||
|
||||
```ts
|
||||
const req = createRequire(path.join(ref.from, 'noop.js'));
|
||||
for (const dir of req.resolve.paths(ref.spec) ?? []) {
|
||||
const manifest = path.join(dir, ref.spec, 'package.json');
|
||||
if (fs.existsSync(manifest)) return path.dirname(manifest);
|
||||
}
|
||||
```
|
||||
|
||||
`resolve.paths()` walks the `node_modules` chain upward from `ref.from` plus the
|
||||
global paths; `existsSync` follows symlinks, so pnpm's
|
||||
`node_modules/@acme/cubes-net → ../.pnpm/…` resolves correctly.
|
||||
|
||||
Errors (each aborts the run, consistent with the existing `errors` contract):
|
||||
|
||||
- package not found on any candidate path
|
||||
- `package.json` unparseable
|
||||
- no `nopy.cubes`, or it is not a non-empty array of strings
|
||||
- a `nopy.cubes` entry escapes the package root, or does not exist
|
||||
|
||||
### Wiring
|
||||
|
||||
`findCubeDirectories()` currently returns `string[]`. It becomes the union of
|
||||
three sources, each tagged so the loader can attribute a cube to it:
|
||||
|
||||
```ts
|
||||
export type CubeRoot =
|
||||
| { type: 'dir'; dir: string } // cubeDirs, .npcubes markers
|
||||
| { type: 'package'; dir: string; packageName: string }; // cubePackages
|
||||
|
||||
export function findCubeRoots(): { roots: CubeRoot[]; errors: string[] };
|
||||
```
|
||||
|
||||
Keep `findCubeDirectories()` as a thin wrapper returning `roots.map(r => r.dir)` —
|
||||
it is exported from `src/cubes/index.ts` and covered by tests. The
|
||||
`node_modules` skip inside `scanDirectory` stays and is now *correct*: a
|
||||
bundle's own `node_modules` should not be scanned.
|
||||
|
||||
## Phase 3 — hard errors with attribution
|
||||
|
||||
`Cube` gains a source, as an optional fourth constructor parameter so the public
|
||||
signature stays backwards compatible:
|
||||
|
||||
```ts
|
||||
export type CubeSource =
|
||||
| { type: 'dir'; dir: string }
|
||||
| { type: 'package'; packageName: string; dir: string };
|
||||
|
||||
class Cube {
|
||||
constructor(
|
||||
manifest: Manifest<Schema>,
|
||||
dir: string,
|
||||
deployScript: string,
|
||||
source: CubeSource = { type: 'dir', dir }
|
||||
) {}
|
||||
}
|
||||
```
|
||||
|
||||
The duplicate error carries both sources and is order-independent (Phase 0.2):
|
||||
|
||||
```
|
||||
Duplicate cube id 'apt:essentials' from 2 sources:
|
||||
package @bitsquare/cubes-core /…/node_modules/@bitsquare/cubes-core/cubes/apt/essentials
|
||||
directory /repo/packages/nopy/cubes/apt/essentials
|
||||
Rename one of them, or remove a source from .nopyrc.json.
|
||||
```
|
||||
|
||||
There is deliberately no override, alias or precedence rule. If two bundles ever
|
||||
claim the same id they are mutually exclusive, and the fix is upstream.
|
||||
|
||||
Surface the source in the interactive picker and in `--json` output so a user can
|
||||
see where a cube came from before running it.
|
||||
|
||||
## Phase 4 — `@bitsquare/nopy-cube`, the authoring package
|
||||
|
||||
The problem: a manifest does `import { cubes } from '@bitsquare/nopy'`, resolved
|
||||
by ordinary Node resolution from the manifest's own directory. From inside
|
||||
`node_modules/@acme/cubes-net/`, that only resolves if the consumer installed
|
||||
`@bitsquare/nopy` locally — a globally-installed CLI leaves nothing to find.
|
||||
|
||||
The fix is to give bundles something they can depend on *normally*, so resolution
|
||||
is plain, boring, spec-compliant Node with no loader tricks in the critical path.
|
||||
|
||||
### The package
|
||||
|
||||
`packages/nopy-cube` — the `Manifest` factory, the `Cube` class, and the types
|
||||
from `cubes/types.ts`. No CLI, no `execa`, `inquirer`, `enquirer`, `zx`, or
|
||||
`commander`. Today a cube manifest — a file that ships nothing but data — drags
|
||||
the entire CLI in as a transitive dependency; this makes the authoring surface
|
||||
honest about how small it is, and gives the *contract* a version number that
|
||||
moves independently of the CLI's.
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "@bitsquare/nopy-cube",
|
||||
"version": "1.0.0-alpha0",
|
||||
"type": "module",
|
||||
"exports": { ".": { "types": "./dist/index.d.ts", "default": "./dist/index.js" } },
|
||||
"peerDependencies": { "zod": "^4.4.3" },
|
||||
"files": ["dist", "README.md", "LICENSE"]
|
||||
}
|
||||
```
|
||||
|
||||
**zod is a peer, deliberately.** Bundles declare zod as a regular dependency, so
|
||||
exactly one zod instance serves the manifest, the schema it builds, and the
|
||||
`Manifest` factory. Phase 0.4 removes the CLI's `instanceof` dependence on that
|
||||
being the *same* copy the CLI uses, but keeping the bundle side single-instance
|
||||
is still the right default.
|
||||
|
||||
### Moving `cubes/types.ts`
|
||||
|
||||
`@bitsquare/nopy` re-exports everything from `@bitsquare/nopy-cube` — through
|
||||
`src/cubes/index.ts` and the `cubes` namespace in `src/nopy.cubes.ts`, both
|
||||
already coverage-excluded barrels — so `import { cubes } from '@bitsquare/nopy'`
|
||||
in every existing manifest keeps working unchanged. Nothing in `cubes/` has to be
|
||||
touched at migration time.
|
||||
|
||||
Repo plumbing this requires:
|
||||
|
||||
- `tsconfig.base.json`: add `"@bitsquare/nopy-cube": ["./packages/nopy-cube/src"]`
|
||||
to `paths`.
|
||||
- Root `tsconfig.json`: add the project reference.
|
||||
- `packages/nopy/tsconfig.json`: `references` is currently `[]` — add
|
||||
`{ "path": "../nopy-cube" }`. This is the first reference edge in the repo, so
|
||||
`tsc --build` ordering starts mattering.
|
||||
- `packages/nopy/package.json`: `"@bitsquare/nopy-cube": "workspace:*"`.
|
||||
- A `vitest.config.ts` for the new package with the same thresholds. `Manifest()`,
|
||||
`Manifest.create()` and `Cube.getDefaults()` all carry logic, so the relevant
|
||||
cases move over from `tests/cubes.factories.test.ts`.
|
||||
|
||||
### The release lane needs fixing first
|
||||
|
||||
This is the part that is easy to miss. `link-workspace-packages` is unset and
|
||||
pnpm 10+ defaults it to `false`, so a plain semver range would resolve
|
||||
`@bitsquare/nopy-cube` from the registry instead of linking the workspace copy —
|
||||
the dependency has to use `workspace:*`.
|
||||
|
||||
But **both workflows publish with `npm publish`**, and npm does not understand
|
||||
the `workspace:` protocol. `@bitsquare/nopy` would ship a manifest carrying
|
||||
`"@bitsquare/nopy-cube": "workspace:*"`, which fails on install with
|
||||
`EUNSUPPORTEDPROTOCOL`. This has never mattered because the two current packages
|
||||
do not depend on each other; `nopy → nopy-cube` is the first edge, and the PoC
|
||||
bundle in Phase 5 adds a second.
|
||||
|
||||
Pick one before publishing anything:
|
||||
|
||||
- **Switch to `pnpm publish --no-git-checks`**, which rewrites the protocol to a
|
||||
concrete version on pack. Cleanest, but changes the publish step in both
|
||||
workflows and pulls in pnpm's own lifecycle behaviour.
|
||||
- **Rewrite the range with `npm pkg set` before publishing**, extending the
|
||||
pattern `publish-snapshot.yml` already uses for `version`. In `release.yml` one
|
||||
package ships at a time, so it pins to whatever version `packages/nopy-cube/package.json`
|
||||
declares at that commit. In `publish-snapshot.yml` the loop needs to become two
|
||||
passes — compute every snapshot version first, then publish — so `nopy` can pin
|
||||
the exact `nopy-cube` snapshot from the same run.
|
||||
|
||||
Recommendation: `pnpm publish`, and verify against the Gitea registry with a
|
||||
throwaway version before the first real release.
|
||||
|
||||
### Also: the resolve hook
|
||||
|
||||
Independent of the split, and worth building anyway — it retires the
|
||||
`ERR_MODULE_NOT_FOUND` gotcha CLAUDE.md documents for the local `cubes/` tree,
|
||||
where manifests import `@bitsquare/nopy` from a directory that has no link to it.
|
||||
|
||||
With the split, the hook is a convenience rather than load-bearing: bundles
|
||||
resolve `@bitsquare/nopy-cube` through their own `node_modules` and never reach
|
||||
it.
|
||||
|
||||
New `packages/nopy/src/nopy.resolve-hook.mjs`, registered once from `loadCubes()`
|
||||
before the first `import(manifestPath)`:
|
||||
|
||||
```ts
|
||||
module.register('./nopy.resolve-hook.mjs', import.meta.url, {
|
||||
data: { fallback: import.meta.resolve('./index.js') },
|
||||
});
|
||||
```
|
||||
|
||||
The hook tries `next(specifier, ctx)` **first** and only falls back to the
|
||||
running CLI's own copy on failure. That ordering matters: a consumer that has its
|
||||
own `@bitsquare/nopy` installed keeps using it, so the hook never silently
|
||||
introduces version skew.
|
||||
|
||||
Constraints:
|
||||
|
||||
- `module.register()` is process-global and cannot be undone. Install it once,
|
||||
behind a module-level guard.
|
||||
- The hook file runs on a separate thread; the `data` payload must be
|
||||
structured-cloneable (a string URL is).
|
||||
- The `.mjs` must ship in `dist` and be listed in `files` — it already is, via
|
||||
the `dist` entry.
|
||||
- It resolves `@bitsquare/nopy`, not `@bitsquare/nopy-cube`. Bundles never depend
|
||||
on the hook; only the in-repo `cubes/` tree and hand-written local cubes do.
|
||||
|
||||
## Phase 5 — proof of concept: `packages/cubes-core`
|
||||
|
||||
Depends on Phase 4 shipping first — the bundle cannot declare
|
||||
`@bitsquare/nopy-cube` as a dependency until it exists, and the publish-lane fix
|
||||
has to be in place before either package is published.
|
||||
|
||||
1. `git mv cubes packages/cubes-core/cubes` — preserves per-file history.
|
||||
2. Add `packages/cubes-core/package.json` per the Phase 1 contract. Version
|
||||
`1.0.0-alpha0`, tracking the current alpha train. Not private. Its
|
||||
`@bitsquare/nopy-cube` dependency uses `workspace:*` in the repo, which is
|
||||
exactly the case the Phase 4 publish fix has to handle.
|
||||
Migrating the manifests' `import { cubes } from '@bitsquare/nopy'` to
|
||||
`import { Manifest } from '@bitsquare/nopy-cube'` is optional — the re-export
|
||||
keeps the old form working — but doing it here is what proves the bundle
|
||||
resolves without the CLI present at all.
|
||||
3. Root `.nopyrc.json`: **replace** `"cubeDirs": ["./cubes"]` with
|
||||
`"cubePackages": ["@bitsquare/cubes-core"]`. Replace, not add — keeping both
|
||||
means every id resolves from two sources and the hard error fires on every
|
||||
run.
|
||||
4. Root `package.json`: add `"@bitsquare/cubes-core": "workspace:*"` to
|
||||
`devDependencies`, so pnpm symlinks it into the root `node_modules`. This is
|
||||
what makes the PoC exercise the real pnpm symlink resolution path rather than
|
||||
a plain directory.
|
||||
5. `packages/nopy/.nopyrc.json` keeps `"cubeDirs": ["./cubes"]` for its fixtures.
|
||||
Config merges root-first, so running from `packages/nopy` now pulls in
|
||||
`@bitsquare/cubes-core` *and* the fixtures — which is exactly the collision
|
||||
Phase 0.3 renames away.
|
||||
6. Workflow changes are limited to the publish-lane fix from Phase 4.
|
||||
`publish-snapshot.yml` loops `for dir in packages/*/` and picks both new
|
||||
packages up automatically; `release.yml` resolves `packages/<pkg>` from the
|
||||
tag, so `cubes-core-v1.0.0` and `nopy-cube-v1.0.0` work as-is. Verify on the
|
||||
first snapshot run that a package with no `build` script is skipped cleanly by
|
||||
`pnpm -r run build` (it is) and that publishing is happy with no lifecycle
|
||||
scripts.
|
||||
7. No `tsconfig` reference for `cubes-core` — the bundle has no TypeScript. (The
|
||||
`nopy-cube` references from Phase 4 are separate.)
|
||||
8. Biome already lints `cubes/**/*.mjs` from the root; only the path changes.
|
||||
|
||||
### Verifying the PoC
|
||||
|
||||
- **In-workspace:** `pnpm --filter @bitsquare/nopy run nopy -P` from the repo
|
||||
root lists `net:tailscale`, `apt:install`, … and prints deploy commands whose
|
||||
`--chdir` points into `node_modules/@bitsquare/cubes-core/cubes/…`.
|
||||
- **Out-of-workspace (the real test):** `npm pack` the bundle, install the
|
||||
tarball into a throwaway directory with a `.nopyrc.json` naming it, install
|
||||
`nopy` *globally*, and run `nopy -P`. This is what actually exercises Phase 4 —
|
||||
a manifest resolving its import from a `node_modules` tree that has no
|
||||
`@bitsquare/nopy` in it. Check the installed tarball's `package.json` really
|
||||
carries a concrete `@bitsquare/nopy-cube` range and not `workspace:*`.
|
||||
|
||||
## Phase 6 — documentation
|
||||
|
||||
- `CLAUDE.md`: the repo table gains two rows (`packages/nopy-cube`,
|
||||
`packages/cubes-core`) and loses the `cubes/` one; "The two packages do not
|
||||
depend on each other" is no longer true; the loader section in *nopy
|
||||
architecture*; and the *Gotcha* paragraph, which the resolve hook retires.
|
||||
- `packages/nopy/docs/CUBE-BUNDLES.md` (new): authoring guide — package shape,
|
||||
read-only constraint, id collision policy, publishing.
|
||||
- `packages/nopy/docs/API.md` + `README.md`: `cubePackages`.
|
||||
- `README.PUBLISH.md`: `nopy-cube-v*` and `cubes-core-v*` as new tag prefixes,
|
||||
plus the ordering constraint — `nopy-cube` releases before anything that
|
||||
depends on it.
|
||||
|
||||
## Testing
|
||||
|
||||
The coverage gate (85 % branches/functions, 80 % lines/statements, per package)
|
||||
is not a CI flag — new modules without tests fail the gate locally and on the
|
||||
runner alike.
|
||||
|
||||
`tests/cubes.packages.test.ts` (new) — build a fake `node_modules` tree under
|
||||
`os.tmpdir()` and `chdir` into it, as the existing loader/config tests do:
|
||||
|
||||
- resolves a scoped and an unscoped package
|
||||
- resolves through a symlinked package directory (mimicking pnpm)
|
||||
- resolves from the declaring config's directory, not `cwd`
|
||||
- missing package → error naming the spec
|
||||
- package without `nopy.cubes` → error
|
||||
- `nopy.cubes` entry that does not exist, and one that escapes the root → errors
|
||||
- last-wins dedupe when parent and child config both name a package
|
||||
|
||||
`tests/cubes.loader.test.ts` — package-sourced cubes load; `source` attribution
|
||||
is correct for all three root types.
|
||||
|
||||
`tests/cubes.loader.edge.test.ts` — duplicate across a dir and a package errors
|
||||
and names both; cubes nested below a duplicate still get scanned (Phase 0.1);
|
||||
the error is identical regardless of scan order (Phase 0.2).
|
||||
|
||||
`tests/config.test.ts` — `cubePackages` merge, `override` resolution strategy,
|
||||
`CubePackageRef` provenance.
|
||||
|
||||
`tests/prompts.test.ts` — `coerceValue` against schemas built by a *different*
|
||||
zod instance, so Phase 0.4 cannot silently regress to `instanceof`.
|
||||
|
||||
`packages/nopy-cube/` — its own `vitest.config.ts` at the same thresholds. The
|
||||
`Manifest()` / `Manifest.create()` / `Cube.getDefaults()` cases move over from
|
||||
`tests/cubes.factories.test.ts`; what stays behind is whatever tests the
|
||||
re-export surface.
|
||||
|
||||
Resolve hook — `module.register()` is process-global, so this cannot be unit
|
||||
tested in-process. Add an integration test that spawns the CLI as a child process
|
||||
against a fixture tree, under the existing `test:integration` script.
|
||||
|
||||
## Risks
|
||||
|
||||
1. **`module.register()` is irreversible and process-wide.** It affects
|
||||
everything loaded afterwards, including the CLI's own lazy imports. Guarded
|
||||
single install, `next()`-first ordering.
|
||||
2. **Hard-error duplicates have no escape hatch.** Two bundles claiming one id
|
||||
cannot be used together, full stop. If that bites in practice the follow-up is
|
||||
a `cubeAliases` map or a per-package id prefix — explicitly out of scope here.
|
||||
3. **Store corruption.** A bundled cube writing to its own directory damages the
|
||||
pnpm global store for every project on the machine. Documented in Phase 1;
|
||||
a runtime warning is a possible follow-up.
|
||||
4. **No version compatibility check.** A bundle authored against a future `nopy`
|
||||
loaded by an older one fails at manifest-import time with a confusing error.
|
||||
A `nopy.engines` field checked at resolution time would fix it. Deferred.
|
||||
5. **Bundles vendoring cubes in their own `node_modules`** will not be found, by
|
||||
design.
|
||||
6. **`workspace:*` escaping into a published manifest.** The failure is silent at
|
||||
publish time and only shows up when someone installs the package. Phase 4
|
||||
fixes the lane; a `postpack` assertion that no dependency range starts with
|
||||
`workspace:` would make it impossible to regress.
|
||||
7. **Three packages, three version lines.** `nopy-cube` is the contract, so a
|
||||
breaking change there ripples to every published bundle in the wild — which is
|
||||
the point of versioning it separately, but it means the compatibility question
|
||||
from risk 4 gets more pressing, not less.
|
||||
@@ -51,19 +51,15 @@ Hooks can be synchronous or asynchronous (returning a `Promise`).
|
||||
|
||||
## Mechanics
|
||||
|
||||
### Sequential Execution
|
||||
### Execution Order
|
||||
|
||||
In sequential execution mode (the default), cubes added via hooks will follow the order in which they were pushed to the deployment plan:
|
||||
Cubes are always deployed sequentially, in the order they were pushed to the deployment plan. For a cube with hooks that means:
|
||||
|
||||
1. Cubes from `before` hooks.
|
||||
2. The current cube itself.
|
||||
3. Cubes from `after` hooks.
|
||||
|
||||
### Parallel Execution
|
||||
|
||||
In parallel execution mode, cubes added via hooks **do not automatically inherit dependencies**.
|
||||
|
||||
If a `before` hook calls `exec('setup-cube')`, it ensures that `setup-cube` is placed earlier in the deployment plan, but for parallel execution, you should still ensure that dependencies are correctly specified if one cube relies on another's completion.
|
||||
Because a `before` hook only places its cube *earlier in the plan*, it guarantees ordering but not much else — if the relationship is a real dependency rather than a one-off ordering nudge, declare it in `dependencies` so it is resolved and deduplicated like any other.
|
||||
|
||||
### Variable Passing
|
||||
|
||||
|
||||
@@ -7,10 +7,12 @@ This document tracks the major refactoring of the `nopy` package.
|
||||
### 1. Remove parallel execution
|
||||
- **Status**: ✅ Completed
|
||||
- **Goal**: Remove all logic supporting parallel execution of cubes to simplify the execution flow and improve reliability.
|
||||
- **Rationale**: The feature never shipped. Concurrent pyinfra processes interleave their output, which made deployment logs unreadable — a cost that outweighed the wall-clock saving. Do not reintroduce it without first solving per-cube output buffering.
|
||||
- **Context**:
|
||||
- Parallelism removed from `NopyConfig`, `NopyOptions`, and `executeDeployCalls`.
|
||||
- `buildExecutionStages` deleted.
|
||||
- CLI flags `--parallel` and `--concurrency` removed.
|
||||
- Documentation caught up later: `README.md`, `docs/API.md`, and `docs/HOOKS.md` had all continued to describe the feature as if it existed.
|
||||
- **Proposed Solution**: (Done)
|
||||
|
||||
### 2. Rework cube building process & Dependency Resolution
|
||||
|
||||
Reference in New Issue
Block a user