[fix] default parameter run records parameters in session for replay[fix] remove default parameters for several cubes
This commit is contained in:
+112
-17
@@ -33,7 +33,7 @@ Nopy wraps pyinfra with structure, validation, and an interactive experience for
|
||||
|
||||
A cube is a **directory** containing two files:
|
||||
|
||||
- **JavaScript manifest**: `manifest.mjs` defining schema, dependencies, defaults, and hooks
|
||||
- **JavaScript manifest**: `manifest.mjs` defining schema, dependencies, defaults, secrets, and hooks
|
||||
- **Python deployment script**: `deploy.py`, a plain pyinfra script
|
||||
|
||||
Configuration variables are declared in the manifest and validated with Zod schemas before the deployment script runs.
|
||||
@@ -96,7 +96,7 @@ apt.packages(
|
||||
)
|
||||
```
|
||||
|
||||
Every key defined in the manifest `schema` is guaranteed to be present on `host.data` — either from the Zod `.default()`, from `.nopyrc.json`, from a dependency, or from a user prompt.
|
||||
Every key defined in the manifest `schema` is guaranteed to be present on `host.data` — either from the Zod `.default()`, from `.nopyrc.json`, from a recorded session, from a dependency, or from a user prompt.
|
||||
|
||||
**Value types**: pyinfra parses `--data` values before your script sees them. `"true"` / `"false"` become booleans, numeric strings become `int`, valid JSON becomes the parsed structure, and everything else stays a string. This is why `UPDATE` can be handed straight to pyinfra's `update=` argument, while `PACKAGES` is wrapped in `str(...)` before splitting.
|
||||
|
||||
@@ -104,18 +104,57 @@ Every key defined in the manifest `schema` is guaranteed to be present on `host.
|
||||
|
||||
Variable defaults are defined directly in the Zod schema using `.default()`. This ensures that every cube has a predictable starting state and provides type-safe default values.
|
||||
|
||||
**Priority order (lowest to highest):**
|
||||
A variable can be set from several places in one run. Every assignment is kept, tagged with where it came from — its **origin** — and the highest-ranked origin wins.
|
||||
|
||||
1. Zod schema `.default()` values
|
||||
2. Global `env` from `.nopyrc.json`
|
||||
3. User prompts, or the recorded answers on session replay
|
||||
4. Variables passed in by a dependency or a hook
|
||||
**Origins, lowest to highest:**
|
||||
|
||||
| Origin | Set by |
|
||||
| --------- | ------------------------------------------------------- |
|
||||
| `default` | the Zod schema's `.default()` |
|
||||
| `env` | the `env` block of `.nopyrc.json` |
|
||||
| `session` | a value recorded in a session file or history entry |
|
||||
| `prompt` | what the user typed |
|
||||
| `param` | a dependency spec or a `before`/`after` hook |
|
||||
|
||||
This allows cubes to ship with reasonable defaults while still allowing users to override them globally via `.nopyrc.json` or interactively during deployment. Because `env` outranks the schema, `.nopyrc.json` is also what steers a run started with `--use-defaults`, which never prompts.
|
||||
|
||||
3 and 4 rarely compete: a key a dependency supplies is left out of the prompt entirely, so the user is only ever asked about the keys nothing else has set.
|
||||
`prompt` and `param` rarely compete: a key a dependency supplies is left out of the prompt entirely, so the user is only ever asked about the keys nothing else has set.
|
||||
|
||||
A field declared without `.default()` has none of sources 1 and 2 to fall back on. It is prompted for like any other, with an empty initial value — but a run that cannot prompt (`--use-defaults`) fails on it unless `env` or a dependency provides it.
|
||||
Ranking by origin rather than by arrival order is what makes replay work: a recorded value is applied *before* the cube would be prompted for, and prompting can still override it, but a `--data` value pushed in by a dependency is never clobbered by a stale recording.
|
||||
|
||||
A field declared without `.default()` has no `default` origin to fall back on. It is prompted for like any other, with an empty initial value — but a run that cannot prompt (`--use-defaults`) fails on it unless `env` or a dependency provides it.
|
||||
|
||||
#### Secrets
|
||||
|
||||
A manifest can name schema keys that hold sensitive values:
|
||||
|
||||
```javascript
|
||||
export default cubes.Manifest({
|
||||
id: 'user:add',
|
||||
name: 'Add a user account',
|
||||
secrets: ['PASSWORD'],
|
||||
schema: z.object({
|
||||
USERNAME: z.string().describe('Username for the new account').default('deploy'),
|
||||
PASSWORD: z.string().describe('Password for the new user account').default('changeme'),
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
Every entry must be a key of `schema`; naming anything else is a manifest error and aborts the run, so a typo fails loudly instead of silently leaving a value unprotected.
|
||||
|
||||
Declaring a key a secret changes three things:
|
||||
|
||||
- **It is never written to a session file or to the history.** Everything else the run settled on is recorded — including values that came from a `.default()` — but declared secrets are left out.
|
||||
- **It is masked wherever a command or a plan is printed** — `--dry-run`, `--print-only`, and the debug log all show `********` in place of the value, in the variable list *and* in the `pyinfra` command line above it. The SSH password passed via `--password` is masked the same way, whether or not any cube declares secrets.
|
||||
- **It is re-prompted on replay**, since there is nothing recorded to replay from (see [Session Recording and Replay](#session-recording-and-replay)).
|
||||
|
||||
Nopy does not guess. A key called `PASSWORD` in a manifest that declares no `secrets` is treated as an ordinary variable — recorded, and printed in the clear.
|
||||
|
||||
Three limits are worth knowing, because `secrets` keeps a value out of the files nopy writes and nothing more:
|
||||
|
||||
- **It is on the command line.** pyinfra takes its data as `--data KEY=value`, so the real value is visible in `ps` for as long as the deployment runs. Masking covers nopy's own output, not the process table.
|
||||
- **The prompt shows it.** The variable form displays and pre-fills what it is asking about, so a secret is on screen while it is being entered or confirmed.
|
||||
- **A `.default()` is not protected.** A default lives in the manifest, in plain text, wherever the manifest is checked in. Give a secret a placeholder default like `changeme` if it needs one at all, never a real credential.
|
||||
|
||||
### Configuration
|
||||
|
||||
@@ -125,6 +164,7 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
|
||||
{
|
||||
"hosts": ["host1.example.com", "host2.example.com"],
|
||||
"cubeDirs": ["./cubes", "../shared-cubes"],
|
||||
"cubePackages": ["@bitsquare/cubes-core"],
|
||||
"env": {
|
||||
"SHARED_VAR": "value"
|
||||
},
|
||||
@@ -144,6 +184,8 @@ Uses `.nopyrc.json` files (project-level or home directory) containing:
|
||||
|
||||
`history` controls automatic session recording (see [Deployment History](#deployment-history)), and `execution.continueOnError` sets the default for `--continue-on-error`.
|
||||
|
||||
`cubeDirs` holds paths, `cubePackages` holds installed npm packages that ship cubes — see [Cube Discovery](#cube-discovery) below and [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md) for publishing your own. Both are additive, and both resolve relative to the config file that named them, not to the working directory: a `.nopyrc.json` two levels up may name a package that only exists in *its* `node_modules`.
|
||||
|
||||
#### Logging Configuration
|
||||
|
||||
Control pyinfra output verbosity and debug information using the `log` configuration object:
|
||||
@@ -213,12 +255,16 @@ Sessions are stored in `.nopysession.json` files with the following structure:
|
||||
|
||||
**Structure Details:**
|
||||
|
||||
- **`cubes`**: Array of cubes with only cube-specific variables (not global env vars)
|
||||
- **`env`**: Global environment variables shared across cubes (like in `.nopyrc.json`)
|
||||
- **`cubes`**: Array of cubes with the variable values that cube ran with
|
||||
- **`env`**: The `env` block of `.nopyrc.json` as it stood at record time, kept for reference
|
||||
- **`hosts`**: Array of target hosts
|
||||
- **`auth`**: Authentication configuration (passwords are never stored)
|
||||
|
||||
**Security Note**: Passwords are never stored in session files. If a session uses password authentication, you'll be prompted for the password during replay.
|
||||
**What is recorded:** every value each cube settled on, regardless of where it came from — a value the user typed, one inherited from `.nopyrc.json` `env`, one a dependency supplied, and one that fell through to the schema's `.default()` are all written out the same way. A session is therefore a full snapshot rather than a diff, and a `--use-defaults` run produces a session with real values in it instead of an empty one.
|
||||
|
||||
The consequence is that replay is faithful rather than re-derived: the recorded value outranks the current `.nopyrc.json` `env` and the current schema default, so editing either one does not silently change what a replay does. To pick up a new default, record a fresh session.
|
||||
|
||||
**Security Note**: Passwords are never stored in session files. This covers both the SSH password — a session records the auth *method* and username, never the credential — and any schema key a cube's manifest lists under [`secrets`](#secrets). Both are re-prompted on replay.
|
||||
|
||||
#### Recording a Session
|
||||
|
||||
@@ -240,12 +286,48 @@ nopy install --load-session my-deployment.nopysession.json
|
||||
# Only password authentication will prompt for credentials
|
||||
```
|
||||
|
||||
A replay runs straight through without asking anything, with three exceptions. Password authentication always re-prompts. A session with no recorded host falls back to the host picker. And a cube is re-prompted for its declared secrets, plus for any required variable the session has no value for — which happens when the cube's schema has gained a field since the session was written.
|
||||
|
||||
Those re-prompts are what a session cannot supply, so `--use-defaults` cannot paper over them: combining `-D` with a replay that needs either fails with a message naming the keys rather than deploying with a placeholder. Put the values under `env` in `.nopyrc.json` to make such a replay unattended.
|
||||
|
||||
### Cube Discovery
|
||||
|
||||
Nopy searches for cubes in:
|
||||
|
||||
1. Directories specified in `.nopyrc.json` `cubeDirs`
|
||||
2. Directories containing a `.npcubes` marker file (searching upwards from current directory)
|
||||
2. Cube directories of every package listed in `.nopyrc.json` `cubePackages`
|
||||
3. Directories containing a `.npcubes` marker file (searching upwards from current directory)
|
||||
|
||||
All three are unioned and scanned the same way. A directory is a cube when it holds both a manifest (`manifest.mjs` or `*.manifest.mjs`) and a deploy script (`deploy.py` or `*.deploy.py`); dotted directories and `node_modules` are skipped during the scan.
|
||||
|
||||
#### Cube packages
|
||||
|
||||
A cube package is an ordinary npm package that ships cube directories and points at them from its own `package.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "@bitsquare/cubes-core",
|
||||
"nopy": { "cubes": ["./cubes"] }
|
||||
}
|
||||
```
|
||||
|
||||
Install it and name it — nothing needs linking or copying:
|
||||
|
||||
```sh
|
||||
pnpm add -D @bitsquare/cubes-core
|
||||
```
|
||||
|
||||
```json
|
||||
{ "cubePackages": ["@bitsquare/cubes-core"] }
|
||||
```
|
||||
|
||||
Naming a package is a statement that cubes are expected from it, so anything wrong is an error that aborts the run rather than a silent skip: the package is not installed, it declares no `nopy.cubes`, or an entry points at a directory that does not exist or lies outside the package.
|
||||
|
||||
#### Ids are claimed globally
|
||||
|
||||
A cube id such as `apt:essentials` is claimed across every source at once, not per directory or per package. Two cubes with the same id abort the run with an error naming both and where each came from. There is no precedence rule and no shadowing — a local cube does not quietly win over a packaged one, in either direction. Prefix your own cubes distinctly if you point `cubeDirs` at a local tree alongside an installed bundle.
|
||||
|
||||
Writing cubes to publish is covered in [CUBE-BUNDLES.md](docs/CUBE-BUNDLES.md).
|
||||
|
||||
## Command Line Usage
|
||||
|
||||
@@ -319,6 +401,17 @@ have no default values. Set them under "env" in .nopyrc.json, pass them from a
|
||||
dependency, or drop --use-defaults to be prompted.
|
||||
```
|
||||
|
||||
Pairing `-D` with a replay fails the same way when the replay would have to ask
|
||||
something — a declared [secret](#secrets), which is never recorded, or a required
|
||||
variable the session has no value for. Both are the sources `-D` has no substitute
|
||||
for, so it stops rather than deploying a placeholder:
|
||||
|
||||
```
|
||||
Error: Cube "user:add" cannot be replayed with --use-defaults: PASSWORD would
|
||||
have to be entered. Secrets are never recorded in a session. Replay without
|
||||
--use-defaults, or set the values under "env" in .nopyrc.json.
|
||||
```
|
||||
|
||||
**Use SSH key authentication**:
|
||||
|
||||
```bash
|
||||
@@ -434,20 +527,20 @@ Session History:
|
||||
Total: 2 session(s)
|
||||
```
|
||||
|
||||
Each entry records the selected cubes together with the variable values that were answered at the prompts, the target hosts, the authentication method, and the username — never the password. Pass an ID to `-H` to run that exact combination again:
|
||||
Each entry records the selected cubes together with every variable value they ran with, the target hosts, the authentication method, and the username — never the password, and never a key the manifest declared a [secret](#secrets). Pass an ID to `-H` to run that exact combination again:
|
||||
|
||||
```bash
|
||||
nopy install -H mdk0zzp8b71cq
|
||||
```
|
||||
|
||||
A replay is non-interactive: cube selection, host, and variable values all come from the entry, so nopy runs straight through without asking anything. The two exceptions are password authentication, which always re-prompts, and an entry with no recorded host, which falls back to the host picker.
|
||||
A replay is non-interactive: cube selection, host, and variable values all come from the entry, so nopy runs straight through without asking anything. It asks only for what the entry cannot hold — the password under password authentication, and any declared secret — plus the host picker when the entry recorded none.
|
||||
|
||||
Two things are worth knowing before relying on an older entry:
|
||||
|
||||
- **Recorded values are applied as defaults, not as a frozen snapshot.** If a cube's schema has gained a variable since the run was recorded, the replay neither prompts for it nor fails — the new variable quietly takes its Zod `.default()`. Global `env` values are likewise read from the *current* `.nopyrc.json` rather than from the entry.
|
||||
- **Recorded values win over the current configuration.** The entry is a snapshot of everything the run settled on, so editing a cube's `.default()` or the `env` block of `.nopyrc.json` afterwards does not change what the replay does. A variable the schema has gained *since* the entry was written has nothing recorded: if it has a `.default()` the replay quietly takes it, and if it is required the replay prompts for it.
|
||||
- **A replay fails if a cube no longer exists.** Renaming or deleting a cube id makes every history entry that referenced it unreplayable: nopy logs `Cube from session not found` and then aborts with `Cube not found: <id>`.
|
||||
|
||||
The history lives in `.nopy.history.json` in the working directory and uses the same structure as a session file, so trimming the array by hand is a perfectly good way to prune it. It does contain the variable values that were entered, which is why it is listed in this repository's `.gitignore` — treat it like any other file holding deployment configuration. A corrupt or unreadable history file is treated as empty rather than raising an error, which looks exactly like a project that has never been deployed from.
|
||||
The history lives in `.nopy.history.json` in the working directory and uses the same structure as a session file, so trimming the array by hand is a perfectly good way to prune it. It does contain the variable values a run used, which is why it is listed in this repository's `.gitignore` — treat it like any other file holding deployment configuration. A corrupt or unreadable history file is treated as empty rather than raising an error, which looks exactly like a project that has never been deployed from.
|
||||
|
||||
For a run you want to keep indefinitely, don't rely on history — it rotates. Use `--save-session` to write it to a file you control (see [Session Recording and Replay](#session-recording-and-replay)).
|
||||
|
||||
@@ -468,7 +561,9 @@ npm run debug
|
||||
## Documentation
|
||||
|
||||
- [Cube Hooks](docs/HOOKS.md) - Lifecycle hooks for dynamic orchestration
|
||||
- [Cube Bundles](docs/CUBE-BUNDLES.md) - Distributing cubes as npm packages
|
||||
- [Session Format](docs/SESSION_FORMAT.md) - Internal JSON/MJS session structure
|
||||
- [API Reference](docs/API.md) - Types and exported functions
|
||||
|
||||
## Resources
|
||||
|
||||
|
||||
@@ -65,6 +65,20 @@ interface NopyResult {
|
||||
|
||||
The cubes module provides types and functions for working with deployment units.
|
||||
|
||||
The authoring half of it — `Manifest`, `Cube`, `Hook`, `uniqid` and the rest —
|
||||
actually lives in **[`@bitsquare/nopy-cube`](../../nopy-cube)**, a package with
|
||||
no CLI and no dependency other than zod. `@bitsquare/nopy` re-exports all of it,
|
||||
so both of these work:
|
||||
|
||||
```javascript
|
||||
import { Manifest } from '@bitsquare/nopy-cube'; // in a manifest.mjs — prefer this
|
||||
import { cubes } from '@bitsquare/nopy'; // cubes.Manifest — still supported
|
||||
```
|
||||
|
||||
Import from `nopy-cube` in a cube bundle you intend to publish: it lets the
|
||||
bundle depend on the authoring types without pulling the whole CLI in as a
|
||||
dependency. See [CUBE-BUNDLES.md](CUBE-BUNDLES.md).
|
||||
|
||||
### Types
|
||||
|
||||
#### `Cube<Schema>`
|
||||
@@ -76,6 +90,7 @@ interface Cube<Schema extends z.AnyZodObject = z.AnyZodObject> {
|
||||
key: string; // Unique identifier
|
||||
name: string; // Human-readable name
|
||||
dir: string; // Absolute path to cube directory
|
||||
source: CubeSource; // Where it was discovered
|
||||
dependencies: string[];
|
||||
schema: Schema;
|
||||
defaults: () => z.infer<Schema>;
|
||||
@@ -84,6 +99,18 @@ interface Cube<Schema extends z.AnyZodObject = z.AnyZodObject> {
|
||||
}
|
||||
```
|
||||
|
||||
#### `CubeSource`
|
||||
|
||||
Where a cube came from. Carried so that a duplicate-id error can name the origin
|
||||
of each claimant, which is the difference between a usable error message and a
|
||||
puzzle when the collision is between a local tree and an installed bundle.
|
||||
|
||||
```typescript
|
||||
type CubeSource =
|
||||
| { type: 'dir'; dir: string }
|
||||
| { type: 'package'; packageName: string; dir: string };
|
||||
```
|
||||
|
||||
#### `Manifest<Schema>`
|
||||
|
||||
Cube manifest (used in `manifest.mjs` files).
|
||||
@@ -124,7 +151,9 @@ interface HookContext {
|
||||
|
||||
#### `loadCubes()`
|
||||
|
||||
Loads all cubes from discovered cube directories.
|
||||
Loads all cubes from discovered cube directories — `cubeDirs`, the directories
|
||||
declared by every package in `cubePackages`, and any ancestor directory holding a
|
||||
`.npcubes` marker.
|
||||
|
||||
```typescript
|
||||
const { cubes, errors } = await loadCubes();
|
||||
@@ -139,6 +168,27 @@ interface LoadResult {
|
||||
}
|
||||
```
|
||||
|
||||
`errors` is non-empty for a duplicate id, a manifest that fails to load, a
|
||||
package in `cubePackages` that is not installed or declares no cubes, and a
|
||||
`nopy.cubes` entry that is missing or points outside its package. Any of them
|
||||
aborts the run — none is a silent skip.
|
||||
|
||||
#### `resolveCubePackages(refs)`
|
||||
|
||||
Resolves `CubePackageRef[]` to installed packages and their cube directories.
|
||||
Called by `loadCubes()`; exported because the resolution failures are worth
|
||||
testing on their own.
|
||||
|
||||
```typescript
|
||||
const { packages, errors } = resolveCubePackages(config.cubePackages);
|
||||
|
||||
interface CubePackage {
|
||||
name: string; // the name it was requested under
|
||||
root: string; // absolute path to the package root
|
||||
dirs: string[]; // absolute paths from its `nopy.cubes` field
|
||||
}
|
||||
```
|
||||
|
||||
#### `resolveDependencies(cubes, selectedCubeNames)`
|
||||
|
||||
Resolves all transitive dependencies for selected cubes.
|
||||
@@ -452,11 +502,31 @@ Configuration file structure.
|
||||
interface NopyConfig {
|
||||
hosts: string[];
|
||||
cubeDirs: string[];
|
||||
cubePackages: CubePackageRef[];
|
||||
env: EnvConfig;
|
||||
log?: LogConfig;
|
||||
}
|
||||
```
|
||||
|
||||
#### `CubePackageRef`
|
||||
|
||||
A package named in `cubePackages`, paired with where it was named. In the config
|
||||
file an entry is just a string (`"@bitsquare/cubes-core"`); `loadConfig()`
|
||||
normalises it.
|
||||
|
||||
```typescript
|
||||
interface CubePackageRef {
|
||||
/** The package name, as written in the config. */
|
||||
spec: string;
|
||||
/** Directory of the config file that named it — resolution starts here. */
|
||||
from: string;
|
||||
}
|
||||
```
|
||||
|
||||
`from` is what makes a package named in a parent config resolve against *that*
|
||||
config's `node_modules`, not the working directory's. It is the same problem
|
||||
`PATH_PROPERTIES` solves for relative `cubeDirs`.
|
||||
|
||||
#### `LogConfig`
|
||||
|
||||
Logging configuration.
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
# Cube bundles
|
||||
|
||||
How to package cubes as an npm package so other projects can install them, and
|
||||
what changes once a cube lives in `node_modules` instead of in your own tree.
|
||||
|
||||
If you only want to *use* a published bundle, you need one line of config:
|
||||
|
||||
```json
|
||||
{ "cubePackages": ["@bitsquare/cubes-core"] }
|
||||
```
|
||||
|
||||
The rest of this document is for writing one.
|
||||
|
||||
- [What a bundle is](#what-a-bundle-is)
|
||||
- [The package manifest](#the-package-manifest)
|
||||
- [Writing the cubes](#writing-the-cubes)
|
||||
- [Ids are claimed globally](#ids-are-claimed-globally)
|
||||
- [An installed bundle is read-only](#an-installed-bundle-is-read-only)
|
||||
- [How resolution actually works](#how-resolution-actually-works)
|
||||
- [Publishing](#publishing)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
## What a bundle is
|
||||
|
||||
An ordinary npm package that ships cube directories and points at them from its
|
||||
own `package.json`. There is no build step, no plugin API and no entry point —
|
||||
nopy reads the directories off disk and imports each `manifest.mjs` directly.
|
||||
|
||||
```
|
||||
@acme/cubes-web
|
||||
├── package.json nopy.cubes → ["./cubes"]
|
||||
├── README.md
|
||||
└── cubes/
|
||||
├── nginx/
|
||||
│ ├── manifest.mjs
|
||||
│ └── deploy.py
|
||||
└── certbot/
|
||||
├── manifest.mjs
|
||||
└── deploy.py
|
||||
```
|
||||
|
||||
`@bitsquare/cubes-core` in this repository is the worked example, and is consumed
|
||||
by this repository through exactly the mechanism described here — it is not
|
||||
special-cased.
|
||||
|
||||
## The package manifest
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "@acme/cubes-web",
|
||||
"version": "1.0.0",
|
||||
"type": "module",
|
||||
"nopy": { "cubes": ["./cubes"] },
|
||||
"files": ["cubes", "!cubes/**/*.log", "README.md", "LICENSE"],
|
||||
"publishConfig": { "access": "public" },
|
||||
"dependencies": {
|
||||
"@bitsquare/nopy-cube": "^1.0.0",
|
||||
"zod": "^4.4.3"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**`nopy.cubes`** is the only field nopy requires. It is an array of directories,
|
||||
relative to the package root, each scanned recursively for cubes. Several
|
||||
entries are fine; a single `["./cubes"]` is the norm. Every entry must exist and
|
||||
must stay inside the package — a path escaping the root is refused, not resolved.
|
||||
|
||||
**`type: "module"`** matters: manifests are ESM. Without it a `manifest.mjs` still
|
||||
loads (the extension carries the day), but anything it imports relatively will
|
||||
not behave the way you expect.
|
||||
|
||||
**`files`** decides the tarball. Note the negation: a cube that has been run
|
||||
leaves a `pyinfra-debug.log` next to its `deploy.py`, and `.gitignore` has no
|
||||
effect on what npm packs. Check with `npm pack --dry-run` before publishing.
|
||||
|
||||
**Dependencies** are `@bitsquare/nopy-cube` and `zod`, both real dependencies
|
||||
rather than peers — a bundle is a leaf, and the copies it gets are the copies its
|
||||
manifests use. Do **not** depend on `@bitsquare/nopy`: the CLI is what installs
|
||||
your bundle, not the other way round, and depending on it invites two copies of
|
||||
the same code into one tree.
|
||||
|
||||
## Writing the cubes
|
||||
|
||||
A cube directory holds a manifest (`manifest.mjs` or `*.manifest.mjs`) and a
|
||||
deploy script (`deploy.py` or `*.deploy.py`). Anything else in the directory is
|
||||
invisible to the loader but readable from the script, which runs with the cube
|
||||
directory as its working directory.
|
||||
|
||||
```javascript
|
||||
// cubes/nginx/manifest.mjs
|
||||
import { Manifest } from '@bitsquare/nopy-cube';
|
||||
import { z } from 'zod';
|
||||
|
||||
export default Manifest({
|
||||
id: 'web:nginx',
|
||||
name: 'Install and configure nginx',
|
||||
dependencies: () => ['apt:essentials'],
|
||||
secrets: ['TLS_KEY'],
|
||||
schema: z.object({
|
||||
SERVER_NAME: z.string().describe('Server name').default('example.com'),
|
||||
TLS_KEY: z.string().describe('TLS private key (PEM)').default(''),
|
||||
HTTP2: z.boolean().describe('Enable HTTP/2').default(true),
|
||||
}),
|
||||
});
|
||||
```
|
||||
|
||||
```python
|
||||
# cubes/nginx/deploy.py
|
||||
from pyinfra import host
|
||||
from pyinfra.operations import apt, files
|
||||
|
||||
SERVER_NAME = host.data.SERVER_NAME
|
||||
|
||||
apt.packages(name='Install nginx', packages=['nginx'], _sudo=True)
|
||||
```
|
||||
|
||||
Import **`@bitsquare/nopy-cube`**, not `@bitsquare/nopy`. It is types and a
|
||||
factory with zod as its only peer — no CLI, no prompts, no process spawning — so
|
||||
your bundle stays a leaf. (`@bitsquare/nopy` re-exports the same surface as
|
||||
`cubes.Manifest`, which is what older manifests use. It still works; it just
|
||||
drags the CLI into your dependency graph if you declare it.)
|
||||
|
||||
Four things the schema is load-bearing for:
|
||||
|
||||
- **`.describe()` is the prompt label.** A field without one prompts with its raw
|
||||
key.
|
||||
- **`.default()` makes the field optional.** A field with no default is required,
|
||||
and is re-prompted on replay if a session has no value for it.
|
||||
- **Every schema key reaches pyinfra** as `--data KEY=value`, so `host.data.KEY`
|
||||
is always defined. pyinfra parses the values itself: `"true"` arrives as a
|
||||
bool, `"8080"` as an int.
|
||||
- **`secrets` names keys whose values must not be persisted.** They are excluded
|
||||
from session files and history, masked wherever a command is printed, and
|
||||
re-prompted on replay. Naming a key that is not in the schema is a load error.
|
||||
A secret is still visible in `ps` while pyinfra runs — masking covers nopy's
|
||||
own output, not the process table — so treat it as protection against writing
|
||||
credentials to disk, not as protection against a shared host.
|
||||
|
||||
`dependencies` is a function of the *collected* variables, so it can branch on
|
||||
what the user actually answered, and it may pass parameters:
|
||||
|
||||
```javascript
|
||||
dependencies: (v) => (v.HTTP2 ? ['apt:essentials', ['web:tls', { MODE: 'strict' }]] : []),
|
||||
```
|
||||
|
||||
`before` / `after` hooks get a context whose `exec(id, vars)` pulls in any cube
|
||||
by id, declared dependency or not. See [HOOKS.md](HOOKS.md).
|
||||
|
||||
## Ids are claimed globally
|
||||
|
||||
An id is claimed across every source at once — `cubeDirs`, `.npcubes` trees and
|
||||
every installed bundle share one flat namespace. Two cubes claiming the same id
|
||||
abort the run with an error naming both and where each came from.
|
||||
|
||||
There is no precedence and no shadowing, deliberately, in either direction: a
|
||||
local cube does not quietly win over a packaged one, and installing a second
|
||||
bundle cannot silently change what an existing id deploys. Overriding a cube from
|
||||
a bundle is not a supported operation; fork the cube under your own id instead.
|
||||
|
||||
So prefix distinctly. `@acme/cubes-web` claiming `nginx` is asking for trouble the
|
||||
first time someone installs a second bundle; `web:nginx` is not. Ids need not
|
||||
mirror the directory layout — `cubes/network/tailscale` declares `net:tailscale`
|
||||
— so the prefix is free.
|
||||
|
||||
An id is also the session key. Renaming one silently invalidates every recorded
|
||||
session that used it, so treat a rename as a breaking change of the bundle.
|
||||
|
||||
## An installed bundle is read-only
|
||||
|
||||
Under pnpm, installed files are **hardlinked into a global store shared by every
|
||||
project on the machine**. A cube that writes next to its own `deploy.py` does not
|
||||
just dirty one `node_modules` — it corrupts that store for every other project.
|
||||
|
||||
Write to `/tmp`, to a path the user configured, or to the remote host. Never to
|
||||
the cube's own directory. Files the cube needs to *read* (templates, config
|
||||
fragments, systemd units) are fine and are exactly what the cube directory is for
|
||||
— `deploy.py` runs with it as the working directory, so `files.template('nginx.conf.j2', ...)`
|
||||
resolves.
|
||||
|
||||
This is the one constraint that does not exist while the cubes live in your own
|
||||
repo, which makes it the one most likely to be discovered late. Test against an
|
||||
installed copy, not a linked one.
|
||||
|
||||
## How resolution actually works
|
||||
|
||||
Worth knowing, because two of the failure modes are otherwise baffling.
|
||||
|
||||
**Where a package is looked up from.** Each `cubePackages` entry is resolved from
|
||||
the directory of the config file that named it, not from the working directory.
|
||||
Configs merge upward, so a `.nopyrc.json` two levels up can name a bundle that
|
||||
only exists in *its* `node_modules`, and it resolves. The lookup reads
|
||||
`package.json` off disk via `createRequire(...).resolve.paths()` rather than
|
||||
going through `exports` — a bundle ships directories and has no entry point to
|
||||
declare.
|
||||
|
||||
**Why the loader does not simply scan `node_modules`.** It cannot: pnpm plants a
|
||||
symlink at `node_modules/<name>`, and `readdir` reports it as a symlink, not a
|
||||
directory, so a recursive scan skips every package silently. Naming packages
|
||||
explicitly is the fix, and it is also the reason `node_modules` is skipped during
|
||||
the cube scan itself.
|
||||
|
||||
**How a manifest finds its imports.** Ordinary Node resolution, from the
|
||||
manifest's own directory. An installed bundle has its own `node_modules` with
|
||||
`@bitsquare/nopy-cube` and `zod` in it, so this just works. A hand-written cube
|
||||
sitting in a directory with no `node_modules` would historically fail with
|
||||
`ERR_MODULE_NOT_FOUND`; nopy now registers a resolve hook that catches exactly
|
||||
that case and falls back to resolving `@bitsquare/nopy-cube`, `@bitsquare/nopy`
|
||||
and `zod` from the running CLI. Normal resolution is always tried first, so a
|
||||
cube that ships its own zod keeps it. Treat the hook as a convenience for local
|
||||
cubes — a published bundle must declare its dependencies properly.
|
||||
|
||||
**Two copies of zod is a real hazard.** `instanceof` comparisons fail across
|
||||
copies, which is why nopy inspects schemas structurally (`schema.def.type`) and
|
||||
why `secrets` is a plain array rather than `.meta()` metadata — zod's metadata
|
||||
registry is per-copy, and a marker written into one copy's registry is invisible
|
||||
to another. Keep your zod range compatible with the CLI's (`^4.4.3`) and the
|
||||
package manager will usually give you one copy.
|
||||
|
||||
## Publishing
|
||||
|
||||
Nothing bundle-specific: `npm publish` (or `pnpm publish`) with a version bump.
|
||||
Some things worth deciding once:
|
||||
|
||||
- **Version the bundle independently of nopy.** There is no compatibility check
|
||||
between the two — the loader reads whatever `nopy.cubes` points at. Document
|
||||
the nopy version you test against in your README.
|
||||
- **Renaming or removing an id is breaking.** It invalidates recorded sessions
|
||||
and breaks any manifest listing it as a dependency, including manifests in
|
||||
other people's bundles.
|
||||
- **Changing a schema key is breaking** in the same way; adding one with a
|
||||
`.default()` is not.
|
||||
- **Test the installed shape, not the linked one.** `npm pack`, install the
|
||||
tarball into a throwaway directory with a `.nopyrc.json` naming it, and deploy
|
||||
from it. This is what catches a missing file, a cube that writes to its own
|
||||
directory, and an undeclared dependency — none of which show up while the
|
||||
package is symlinked into the repo that wrote it.
|
||||
|
||||
For an unattended check, replay a session file rather than reaching for `-P`
|
||||
alone, which still opens the interactive picker:
|
||||
|
||||
```sh
|
||||
nopy install -l session.json -P -D
|
||||
```
|
||||
|
||||
Note that a replay re-prompts for anything a manifest lists in `secrets` —
|
||||
those are never written to a session — so pick a cube without them, or put the
|
||||
values under `env` in `.nopyrc.json`.
|
||||
|
||||
For how this repository releases its own packages, see
|
||||
[README.PUBLISH.md](../../../README.PUBLISH.md).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause |
|
||||
| --- | --- |
|
||||
| `Cube package 'X' is not installed (looked up from …)` | Not installed, or installed somewhere other than the config that named it. The path in the message is where the lookup started. |
|
||||
| `Cube package 'X' declares no cubes` | Missing or malformed `nopy.cubes` in the package's `package.json`. It must be a non-empty array of strings. |
|
||||
| `'./cubes' does not exist in …` | The directory was not packed. Check `files` and `npm pack --dry-run`. |
|
||||
| `'…' points outside the package` | A `nopy.cubes` entry escaping the package root. Not allowed. |
|
||||
| `Duplicate cube id 'X' from N sources:` | Two or more cubes claiming one id; the message lists each source. Rename one — there is no precedence rule to lean on. |
|
||||
| `ERR_MODULE_NOT_FOUND` for `zod` or `@bitsquare/nopy-cube` | The bundle did not declare them as dependencies. The resolve-hook fallback covers loose local cubes, not published packages. |
|
||||
| `Invalid manifest in …: 'secrets' names X, which is not in the schema` | A `secrets` entry with no matching schema key — usually a typo or a renamed field. |
|
||||
| Cubes work linked, fail installed | Almost always a write into the cube's own directory, or a file missing from `files`. |
|
||||
@@ -1,6 +1,11 @@
|
||||
# Cube bundles as npm packages
|
||||
|
||||
Status: **Phase 0 has landed; Phases 1–6 are still a plan, not a record.**
|
||||
Status: **All six phases have landed. This document is now a record, not a plan.**
|
||||
The one thing still unproven is the publish lane against a real registry — see
|
||||
*Risks*.
|
||||
`cubePackages` resolves and loads end to end, `@bitsquare/nopy-cube` exists and
|
||||
the publish lane can ship a linked package. What is missing is a bundle to point
|
||||
`cubePackages` at.
|
||||
|
||||
Distributing cubes as npm packages so a project can `pnpm add @acme/cubes-net`
|
||||
and have its cubes show up in `nopy` alongside local ones.
|
||||
@@ -166,7 +171,7 @@ Rules:
|
||||
- A bundle must not ship a `.nopyrc.json`. Config discovery walks up from
|
||||
`process.cwd()`, never from cube directories, so it would never be read.
|
||||
|
||||
## Phase 2 — resolution
|
||||
## Phase 2 — resolution — **done**
|
||||
|
||||
### Config surface
|
||||
|
||||
@@ -271,7 +276,7 @@ it is exported from `src/cubes/index.ts` and covered by tests. The
|
||||
`node_modules` skip inside `scanDirectory` stays and is now *correct*: a
|
||||
bundle's own `node_modules` should not be scanned.
|
||||
|
||||
## Phase 3 — hard errors with attribution
|
||||
## Phase 3 — hard errors with attribution — **done**
|
||||
|
||||
`Cube` gains a source, as an optional fourth constructor parameter so the public
|
||||
signature stays backwards compatible:
|
||||
@@ -306,7 +311,7 @@ claim the same id they are mutually exclusive, and the fix is upstream.
|
||||
Surface the source in the interactive picker and in `--json` output so a user can
|
||||
see where a cube came from before running it.
|
||||
|
||||
## Phase 4 — `@bitsquare/nopy-cube`, the authoring package
|
||||
## Phase 4 — `@bitsquare/nopy-cube`, the authoring package — **done**
|
||||
|
||||
The problem: a manifest does `import { cubes } from '@bitsquare/nopy'`, resolved
|
||||
by ordinary Node resolution from the manifest's own directory. From inside
|
||||
@@ -350,20 +355,33 @@ already coverage-excluded barrels — so `import { cubes } from '@bitsquare/nopy
|
||||
in every existing manifest keeps working unchanged. Nothing in `cubes/` has to be
|
||||
touched at migration time.
|
||||
|
||||
Repo plumbing this requires:
|
||||
`cubes/types.ts` and `cubes/factories.ts` moved wholesale, with
|
||||
`tests/cubes.types.test.ts` and `tests/cubes.factories.test.ts` behind them.
|
||||
`tests/helpers/foreign-zod.ts` is duplicated rather than shared — fifteen lines,
|
||||
and the alternative is a test-only dependency edge between the packages.
|
||||
|
||||
- `tsconfig.base.json`: add `"@bitsquare/nopy-cube": ["./packages/nopy-cube/src"]`
|
||||
to `paths`.
|
||||
- Root `tsconfig.json`: add the project reference.
|
||||
- `packages/nopy/tsconfig.json`: `references` is currently `[]` — add
|
||||
`{ "path": "../nopy-cube" }`. This is the first reference edge in the repo, so
|
||||
`tsc --build` ordering starts mattering.
|
||||
Repo plumbing it took:
|
||||
|
||||
- `tsconfig.base.json`: `"@bitsquare/nopy-cube": ["./packages/nopy-cube/src"]`.
|
||||
- Root `tsconfig.json` and `packages/nopy/tsconfig.json`: the project reference.
|
||||
This is the first reference edge in the repo, and it broke the gate
|
||||
immediately: **`tsc --build --noEmit` is not legal once a project has
|
||||
references** — TS6310, "referenced project may not disable emit", because a
|
||||
composite project has to emit the declarations its dependents read. The root
|
||||
`typecheck` script is now plain `tsc --build`. It still fails on a type error,
|
||||
and it now also proves the build works; the cost is that it writes `dist`,
|
||||
which is gitignored.
|
||||
- `packages/nopy/package.json`: `"@bitsquare/nopy-cube": "workspace:*"`.
|
||||
- A `vitest.config.ts` for the new package with the same thresholds. `Manifest()`,
|
||||
`Manifest.create()` and `Cube.getDefaults()` all carry logic, so the relevant
|
||||
cases move over from `tests/cubes.factories.test.ts`.
|
||||
- `packages/nopy/vitest.config.ts`: a `resolve.alias` for `@bitsquare/nopy-cube`
|
||||
pointing at `../nopy-cube/src/index.ts`. Without it the workspace link
|
||||
resolves through `exports` to `dist`, so `pnpm test` on a clean checkout would
|
||||
fail until something had built it, and a stale `dist` would silently be what
|
||||
the tests ran against. The same config excludes `**/nopy-cube/**` from
|
||||
coverage — the aliased files were being counted against nopy's thresholds.
|
||||
- A `vitest.config.ts` for the new package with the same thresholds. It sits at
|
||||
100 % statements/functions/lines, 91 % branches.
|
||||
|
||||
### The release lane needs fixing first
|
||||
### The release lane needed fixing first
|
||||
|
||||
This is the part that is easy to miss. `link-workspace-packages` is unset and
|
||||
pnpm 10+ defaults it to `false`, so a plain semver range would resolve
|
||||
@@ -389,10 +407,40 @@ Pick one before publishing anything:
|
||||
passes — compute every snapshot version first, then publish — so `nopy` can pin
|
||||
the exact `nopy-cube` snapshot from the same run.
|
||||
|
||||
Recommendation: `pnpm publish`, and verify against the Gitea registry with a
|
||||
**Measured, both directions.** `npm pack` in `packages/nopy` produces a tarball
|
||||
whose manifest still reads `"@bitsquare/nopy-cube": "workspace:*"`; `pnpm pack`
|
||||
produces one that reads `"1.0.0-alpha0"`. So the failure was real and the fix
|
||||
works.
|
||||
|
||||
Went with `pnpm publish --ignore-scripts --no-git-checks` in both workflows.
|
||||
`--no-git-checks` is not optional in either: `release.yml` runs on a detached
|
||||
HEAD, and `publish-snapshot.yml` dirties the tree by stamping versions.
|
||||
(`pnpm pack` has no `--ignore-scripts`, only `pnpm publish` does.)
|
||||
|
||||
Three small scripts carry the parts that are easy to get wrong, all runnable
|
||||
locally:
|
||||
|
||||
- **`scripts/verify-pack.mjs`** — packs every publishable package and fails if a
|
||||
`workspace:` range survived into the tarball. Runs between build and publish
|
||||
in both workflows. Turns "npm would have shipped a broken manifest" from an
|
||||
install-time surprise into a red run.
|
||||
- **`scripts/publish-order.mjs`** — topologically sorts the publishable packages.
|
||||
`packages/*/` alphabetically puts `nopy` ahead of the `nopy-cube` it depends
|
||||
on; the snapshot workflow now iterates this instead.
|
||||
- **`scripts/linked-deps.mjs`** — lists a package's workspace links as
|
||||
`<name> <version>`, resolved by package name rather than by directory.
|
||||
`release.yml` uses it to refuse a release whose linked dependency is not on
|
||||
npmjs yet, which is the one mistake that cannot be taken back after 72 hours.
|
||||
|
||||
`publish-snapshot.yml` also became two passes over the packages: stamp every
|
||||
version first, then publish. `pnpm publish` substitutes the version the linked
|
||||
package declares *at pack time*, so `nopy-cube` has to be carrying its snapshot
|
||||
version before `nopy` is packed.
|
||||
|
||||
Still unverified: none of this has run against the Gitea registry. Worth a
|
||||
throwaway version before the first real release.
|
||||
|
||||
### Also: the resolve hook
|
||||
### Also: the resolve hook — built
|
||||
|
||||
Independent of the split, and worth building anyway — it retires the
|
||||
`ERR_MODULE_NOT_FOUND` gotcha CLAUDE.md documents for the local `cubes/` tree,
|
||||
@@ -414,33 +462,48 @@ Falling back for `zod` hands local cubes the *CLI's* zod instance, so no
|
||||
duplication arises there. Bundles are the case that duplicates it, and Phase 0.4
|
||||
is what makes that safe.
|
||||
|
||||
New `packages/nopy/src/nopy.resolve-hook.mjs`, registered once from `loadCubes()`
|
||||
`packages/nopy/src/cubes/resolve-hook.mjs`, registered once from `loadCubes()`
|
||||
before the first `import(manifestPath)`:
|
||||
|
||||
```ts
|
||||
module.register('./nopy.resolve-hook.mjs', import.meta.url, {
|
||||
data: { fallback: import.meta.resolve('./index.js') },
|
||||
});
|
||||
module.register('./resolve-hook.mjs', import.meta.url, { data: { from: import.meta.url } });
|
||||
```
|
||||
|
||||
The hook tries `next(specifier, ctx)` **first** and only falls back to the
|
||||
running CLI's own copy on failure. That ordering matters: a consumer that has its
|
||||
own `@bitsquare/nopy` installed keeps using it, so the hook never silently
|
||||
introduces version skew.
|
||||
`from` is a URL inside the running CLI's own package; the hook thread builds a
|
||||
`createRequire` from it and resolves the fallbacks out of the CLI's own
|
||||
dependencies.
|
||||
|
||||
Constraints:
|
||||
The hook tries `next(specifier, ctx)` **first** and only falls back on failure.
|
||||
That ordering matters: a consumer that has its own copy installed keeps using
|
||||
it, so the hook never silently introduces version skew. There is a test for
|
||||
exactly that — a stub `zod` beside the cube wins over the CLI's real one.
|
||||
|
||||
- `module.register()` is process-global and cannot be undone. Install it once,
|
||||
behind a module-level guard.
|
||||
Constraints, as built:
|
||||
|
||||
- `module.register()` is process-global and cannot be undone. Installed once,
|
||||
behind a module-level guard, and wrapped in a `try` — the hook is a
|
||||
convenience, so a registration failure must not abort a run.
|
||||
- The hook file runs on a separate thread; the `data` payload must be
|
||||
structured-cloneable (a string URL is).
|
||||
- The `.mjs` must ship in `dist` and be listed in `files` — it already is, via
|
||||
the `dist` entry.
|
||||
- It resolves `@bitsquare/nopy` and `zod`, not `@bitsquare/nopy-cube`. Bundles
|
||||
never depend on the hook; only the in-repo `cubes/` tree and hand-written local
|
||||
cubes do.
|
||||
- The `.mjs` has to reach `dist`, and `tsc` does not copy it: nopy's `build` is
|
||||
now `tsc && cp src/cubes/*.mjs dist/cubes/`. `files` already covers it via the
|
||||
`dist` entry.
|
||||
- It covers **three** specifiers, not the two the plan named: `zod`,
|
||||
`@bitsquare/nopy`, and `@bitsquare/nopy-cube` — a hand-written local cube is
|
||||
as entitled to the new authoring package as to the old one. Subpaths count
|
||||
(`@bitsquare/nopy/package.json`), anything else stays a hard failure.
|
||||
|
||||
## Phase 5 — proof of concept: `packages/cubes-core`
|
||||
**The tests have to spawn a real `node`.** Written inside the vitest worker they
|
||||
pass whether or not the hook is installed: vite resolves the dynamic import
|
||||
itself and finds `zod` from the project root. `tests/cubes.resolve-hook.test.ts`
|
||||
therefore runs each case in a child process, and the first case asserts the
|
||||
*failure* without the hook so the rest cannot silently stop proving anything.
|
||||
|
||||
**Verified end to end.** From a plain `node` at the repo root, with nothing
|
||||
linked, the built loader reads all 22 cubes under `cubes/` with zero errors. The
|
||||
`ERR_MODULE_NOT_FOUND` gotcha in `CLAUDE.md` is retired.
|
||||
|
||||
## Phase 5 — proof of concept: `packages/cubes-core` — **done**
|
||||
|
||||
Depends on Phase 4 shipping first — the bundle cannot declare
|
||||
`@bitsquare/nopy-cube` as a dependency until it exists, and the publish-lane fix
|
||||
@@ -478,19 +541,50 @@ has to be in place before either package is published.
|
||||
`nopy-cube` references from Phase 4 are separate.)
|
||||
8. Biome already lints `cubes/**/*.mjs` from the root; only the path changes.
|
||||
|
||||
### Verifying the PoC
|
||||
### What differed from the plan
|
||||
|
||||
- **In-workspace:** `pnpm --filter @bitsquare/nopy run nopy -P` from the repo
|
||||
root lists `net:tailscale`, `apt:install`, … and prints deploy commands whose
|
||||
`--chdir` points into `node_modules/@bitsquare/cubes-core/cubes/…`.
|
||||
- **Out-of-workspace (the real test):** `npm pack` the bundle, install the
|
||||
tarball into a throwaway directory with a `.nopyrc.json` naming it, install
|
||||
`nopy` *globally*, and run `nopy -P`. This is what actually exercises Phase 4 —
|
||||
a manifest resolving its import from a `node_modules` tree that has no
|
||||
`@bitsquare/nopy` in it. Check the installed tarball's `package.json` really
|
||||
carries a concrete `@bitsquare/nopy-cube` range and not `workspace:*`.
|
||||
- **Step 2's optional migration was done.** All 22 manifests now import
|
||||
`{ Manifest }` from `@bitsquare/nopy-cube`, not `{ cubes }` from
|
||||
`@bitsquare/nopy`. Optional for correctness, but it is the only version of the
|
||||
PoC that proves anything: leaving the old import in place would have resolved
|
||||
through the CLI that happens to sit in the same tree.
|
||||
- **`uniqid` had to move too.** Two manifests use it (`admin:hostname` bare,
|
||||
`user:add` via `cubes.uniqid`), so `src/cubes/utils.ts` and its test went to
|
||||
`nopy-cube` alongside `types.ts`, and `uniqid` joined the authoring barrel.
|
||||
Otherwise one migrated manifest would still have been importing the CLI.
|
||||
- **`files` needs a log exclusion.** Cubes that have been run leave a gitignored
|
||||
`pyinfra-debug.log` next to `deploy.py`; gitignore does not filter an npm
|
||||
tarball. `"files": ["cubes", "!cubes/**/*.log", …]` does. Verified: 22
|
||||
manifests, 22 deploy scripts, 0 logs in the packed artefact.
|
||||
- **`verify-pack.mjs` picks the bundle up for free** — it walks every non-private
|
||||
`packages/*`, so `cubes-core`'s `workspace:*` edge is checked like nopy's.
|
||||
|
||||
## Phase 6 — documentation
|
||||
### Verifying the PoC — done
|
||||
|
||||
- **In-workspace:** the built loader, run from the repo root against the new
|
||||
root `.nopyrc.json`, reads 22 cubes with 0 errors and reports
|
||||
`source: { type: 'package', packageName: '@bitsquare/cubes-core', dir:
|
||||
'…/node_modules/@bitsquare/cubes-core/cubes' }` — the pnpm symlink path, not a
|
||||
plain directory.
|
||||
- **Out-of-workspace (the real test):** `pnpm pack` for `nopy-cube`, `nopy` and
|
||||
`cubes-core`, then **`npm install`** of all three tarballs into a throwaway
|
||||
directory with a `.nopyrc.json` naming only the bundle. npm is the strict test
|
||||
here — it does not understand `workspace:`, so a leaked range fails the install
|
||||
outright. It installed clean, and the installed
|
||||
`@bitsquare/nopy/package.json` carries `"@bitsquare/nopy-cube":
|
||||
"1.0.0-alpha0"`. `nopy install -l session.json -P -D` then resolved
|
||||
`apt:essentials` and printed a `--chdir` into
|
||||
`node_modules/@bitsquare/cubes-core/cubes/apt/essentials`. Since the loader
|
||||
aborts on any manifest error and this run did not, all 22 manifests imported
|
||||
`@bitsquare/nopy-cube` and `zod` successfully from a tree containing no
|
||||
workspace links.
|
||||
|
||||
Note for anyone repeating this: `-P` on its own is interactive, and a replay
|
||||
still prompts for anything a manifest declares in `secrets` (they are never
|
||||
persisted to a session) — `net:tailscale` will sit there waiting. Use a
|
||||
session file with a cube that has no secrets, or answer the prompt.
|
||||
|
||||
## Phase 6 — documentation — **done**
|
||||
|
||||
- `CLAUDE.md`: the repo table gains two rows (`packages/nopy-cube`,
|
||||
`packages/cubes-core`) and loses the `cubes/` one; "The two packages do not
|
||||
@@ -503,6 +597,21 @@ has to be in place before either package is published.
|
||||
plus the ordering constraint — `nopy-cube` releases before anything that
|
||||
depends on it.
|
||||
|
||||
Beyond the list: `CLAUDE.md` also needed the `typecheck` command corrected
|
||||
(`tsc --build`, not `--noEmit` — see Phase 4), a note on the vitest source alias
|
||||
and the coverage exclusion, and three entries under *Known drift*. `README.PUBLISH.md`
|
||||
absorbed the whole publish-lane rework, not just the tag prefixes: `pnpm publish`
|
||||
over `npm publish` and why, the two-pass version stamping, `verify-pack.mjs`,
|
||||
`publish-order.mjs`, `linked-deps.mjs`, and a local rehearsal recipe that uses
|
||||
**npm** to install the tarballs precisely because npm is the one that rejects a
|
||||
leaked `workspace:` range.
|
||||
|
||||
One workflow change came out of writing this up: `ci.yml` now runs
|
||||
`verify-pack.mjs` too. It was only in the two publish workflows, which means a
|
||||
leaked range would have failed the release rather than the pull request that
|
||||
introduced it — the wrong end of the process for a mistake that is free to catch
|
||||
early.
|
||||
|
||||
## Testing
|
||||
|
||||
The coverage gate (85 % branches/functions, 80 % lines/statements, per package)
|
||||
|
||||
@@ -54,3 +54,29 @@ This document tracks the major refactoring of the `nopy` package.
|
||||
- `VariableAssignment` offers every schema key, not only the ones carrying a default, and shows the value the run would actually use as the initial.
|
||||
- **Proposed Solution**: (Done)
|
||||
|
||||
|
||||
### 6. Make variable assignment a first-class concept
|
||||
- **Status**: ✅ Completed
|
||||
- **Goal**: Give a variable an identity and a provenance, instead of inferring both from which bag it happened to sit in.
|
||||
- **Rationale**: Item 5 left precedence encoded as the field order of an object literal inside `Variables.get()` — `defaults`, then `global`, then `prompts`, then `params`. Nothing named the ranking, nothing could be asked where a value came from, and a replay had to be smuggled into the `prompts` bag because there was no origin that meant "recorded". Every question that followed — what should a session record, which values are safe to print — needed provenance to answer.
|
||||
- **Context**:
|
||||
- `Assignment { value, origin }` and an `Origin` ranked `default(0) < env(1) < session(2) < prompt(3) < param(4)`. Precedence is now data, not the order lines appear in.
|
||||
- `Variable` is a class over an assignment list. `assignments` is the true history, newest first and never reordered; `ordered` is a *stable* sort of it by origin rank, and `value`/`origin` read the head of that. Stability is what makes the two views coexist: same-origin ties keep the newest in front while the value it displaced stays visible.
|
||||
- The `global` bag is gone. Config `env` is seeded per cube as a real assignment at origin `env`, so `variables.get('global')` — a cube id that was never a cube — is no longer a thing.
|
||||
- Replay assigns at origin `session`, which outranks `env` and `default` on its own. The `prompts`-bag workaround is deleted.
|
||||
- A session records `Variables.persistable()` — every effective value, not just prompted ones. A `-D` run used to record nothing and replay by re-deriving from whatever the defaults said at replay time.
|
||||
- **Trade-off accepted**: recorded values now outrank the current `.nopyrc.json` `env` and the current schema defaults, so editing either no longer leaks into an existing session's replay. That is the point of a snapshot, but it does mean picking up a new default requires re-recording.
|
||||
- **Proposed Solution**: (Done)
|
||||
|
||||
### 7. Manifest-declared secrets
|
||||
- **Status**: ✅ Completed
|
||||
- **Goal**: Let a manifest say which schema keys hold sensitive values, and act on it.
|
||||
- **Rationale**: Item 6 made sessions record everything, which forced the question of what must *not* be recorded. The codebase already had an answer of sorts — `outputExecutionPlan` masked any variable whose name contained "password" — that missed `TOKEN`, `PSK` and `AUTH_KEY`, and was defeated anyway by the unmasked command printed one line above it.
|
||||
- **Context**:
|
||||
- `Manifest.secrets?: string[]`, validated at load: an entry that is not a key of `schema` is a manifest error and aborts the run, so a typo cannot silently leave a value unprotected.
|
||||
- Deliberately a plain array, not zod metadata. `.meta()` and `.describe()` store into `z.globalRegistry`, which is per-copy — a manifest built by a different zod copy would look up empty. Fail-open is fine for a missing prompt label and unacceptable for a secret marker.
|
||||
- `maskCommand()` replaces declared `--data` values and the SSH `--password` in the command string itself, and is wired into `--print-only`, the dry-run plan and the debug log. The `nopy` logger runs at `lowestLevel: 'debug'`, so that last one was printing credentials on every run.
|
||||
- Secrets are excluded from `persistable()`, so a replay has a gap where one used to be. `fillSessionGaps` prompts for `requiredKeys() ∪ secrets`; under `-D` it fails naming them, consistent with item 5's fail-fast.
|
||||
- **Scope limit**: `secrets` keeps a value out of what nopy writes. The value is still on pyinfra's command line (visible in `ps`), still echoed by the variable form, and a `.default()` is still plain text in the manifest. Documented rather than fixed — the first is inherent to pyinfra's interface.
|
||||
- **Bug fixed along the way**: `cubes/user/add` generated a random password as its schema `.default()`. Because the key had a default it was never in `requiredKeys()`, and because a generated default is re-evaluated on every read, an unattended run created an account with a credential nobody had seen and a replay created a different one again. It is now the literal `changeme`.
|
||||
- **Proposed Solution**: (Done)
|
||||
|
||||
@@ -294,8 +294,9 @@ export default {
|
||||
2. **Document your cubes** - Add comments explaining what each cube does
|
||||
3. **Use environment variables** - Make sessions reusable across environments
|
||||
4. **Extract common config** - Share configuration across multiple sessions
|
||||
5. **Version control** - Both formats work well with git
|
||||
5. **Version control** - Both formats work well with git, but a recorded session holds every value its run used; read one before committing it
|
||||
6. **Validate at runtime** - The loader validates the structure regardless of format
|
||||
7. **Leave secrets out** - Declare them in the manifest instead, and let the replay ask
|
||||
|
||||
## Session Schema
|
||||
|
||||
@@ -321,3 +322,16 @@ interface AuthSession {
|
||||
username?: string;
|
||||
}
|
||||
```
|
||||
|
||||
A session nopy *writes* holds, per cube, every value that cube ran with — what
|
||||
was typed, what came from `.nopyrc.json`, what a dependency supplied, and what
|
||||
fell through to the schema's `.default()`. Two things are deliberately absent and
|
||||
are asked for again on replay: the SSH password, and any key the cube's manifest
|
||||
listed under `secrets`.
|
||||
|
||||
A session you write by hand is under no such obligation — `variables` may hold as
|
||||
few keys as you like, and anything missing resolves the usual way. Note that a
|
||||
key declared a secret is prompted for whether or not the session carries a value:
|
||||
writing one in only pre-fills the prompt, it does not skip it. The variable form
|
||||
shows what it is editing, so a secret you put in a session file appears on screen
|
||||
as well as on disk.
|
||||
|
||||
@@ -43,7 +43,7 @@
|
||||
},
|
||||
"scripts": {
|
||||
"clean": "rm -rf dist .tsbuildinfo",
|
||||
"build": "tsc",
|
||||
"build": "tsc && cp src/cubes/*.mjs dist/cubes/",
|
||||
"prepack": "pnpm run build",
|
||||
"link:local": "pnpm run build && npm link",
|
||||
"nopy": "tsx src/nopy.cli.ts",
|
||||
@@ -54,6 +54,7 @@
|
||||
"test:watch": "vitest"
|
||||
},
|
||||
"dependencies": {
|
||||
"@bitsquare/nopy-cube": "workspace:*",
|
||||
"@logtape/logtape": "^2.2.4",
|
||||
"commander": "^15.0.0",
|
||||
"enquirer": "^2.4.1",
|
||||
|
||||
@@ -3,13 +3,13 @@
|
||||
* @module cubes/dependencies
|
||||
*/
|
||||
|
||||
import type { Cube, CubeVariables, HookContext } from '@bitsquare/nopy-cube';
|
||||
import { getLogger } from '@logtape/logtape';
|
||||
import type { Variables } from '../nopy.common.js';
|
||||
import type { NopyConfig } from '../nopy.config.js';
|
||||
import type { DeployCall } from '../nopy.executor.js';
|
||||
import { VariableAssignment } from '../nopy.prompts.js';
|
||||
import type { CubeSession, NopySession } from '../nopy.session.js';
|
||||
import type { Cube, CubeVariables, HookContext } from './types.js';
|
||||
|
||||
const log = getLogger(['nopy', 'resolution']);
|
||||
|
||||
@@ -38,6 +38,12 @@ export class BuildContext {
|
||||
} = {}
|
||||
) {}
|
||||
|
||||
/** Required schema keys that nothing has supplied a value for. */
|
||||
private missingRequired(cube: Cube): string[] {
|
||||
const resolved = this.variables.get(cube.id);
|
||||
return cube.requiredKeys().filter((key) => resolved[key] === undefined);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fails a non-interactive run that cannot fill a required variable.
|
||||
*
|
||||
@@ -45,8 +51,7 @@ export class BuildContext {
|
||||
* `--data`, and the deploy script would read `None` off `host.data`.
|
||||
*/
|
||||
private assertVariablesComplete(cube: Cube): void {
|
||||
const resolved = this.variables.get(cube.id);
|
||||
const missing = cube.requiredKeys().filter((key) => resolved[key] === undefined);
|
||||
const missing = this.missingRequired(cube);
|
||||
if (missing.length === 0) return;
|
||||
|
||||
const [one, them] =
|
||||
@@ -58,6 +63,43 @@ export class BuildContext {
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Asks for the variables a replay cannot supply on its own.
|
||||
*
|
||||
* Two kinds. Required keys can be absent because the session predates them or
|
||||
* was recorded by a `--use-defaults` run. Secrets are absent by design: they
|
||||
* are never written to a session, so replaying without asking would deploy a
|
||||
* cube with the key missing — or, for a secret carrying a default, with a
|
||||
* value silently different from the run being replayed.
|
||||
*
|
||||
* Secrets are asked for even when a default did fill them in, which is why
|
||||
* this cannot key off "has no value": the whole point is that the recorded
|
||||
* answer is gone and only the user knows what it was.
|
||||
*/
|
||||
private async fillSessionGaps(cube: Cube): Promise<void> {
|
||||
const gaps = [...new Set([...this.missingRequired(cube), ...cube.secrets])];
|
||||
if (gaps.length === 0) return;
|
||||
|
||||
if (this.options.useDefaults) {
|
||||
throw new Error(
|
||||
`Cube "${cube.id}" cannot be replayed with --use-defaults: ${gaps.join(', ')} ` +
|
||||
'would have to be entered. Secrets are never recorded in a session. ' +
|
||||
'Replay without --use-defaults, or set the values under "env" in .nopyrc.json.'
|
||||
);
|
||||
}
|
||||
|
||||
log.debug('Filling session gaps', { cubeId: cube.id, gaps });
|
||||
await VariableAssignment(cube, this.variables, { keys: gaps });
|
||||
|
||||
// A cancelled form leaves the run short of a value it cannot invent.
|
||||
const stillMissing = this.missingRequired(cube);
|
||||
if (stillMissing.length > 0) {
|
||||
throw new Error(
|
||||
`Cube "${cube.id}" is missing ${stillMissing.join(', ')} and cannot be deployed.`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves a cube, its dependencies, and hooks recursively
|
||||
*/
|
||||
@@ -73,20 +115,22 @@ export class BuildContext {
|
||||
|
||||
log.debug('Resolving cube', { cubeId, host });
|
||||
|
||||
// 1. Assign overrides and defaults
|
||||
// 1. Declare secrets, then assign overrides and defaults. Declaring first
|
||||
// means even the config `env` seeded on the cube's first assignment is
|
||||
// already marked, so nothing reaches a session or a log unredacted.
|
||||
this.variables.declareSecrets(cubeId, cube.secrets);
|
||||
if (Object.keys(overrides).length > 0) {
|
||||
this.variables.assign(cubeId, 'params', overrides);
|
||||
this.variables.assign(cubeId, 'param', overrides);
|
||||
}
|
||||
this.variables.assign(cubeId, 'defaults', cube.getDefaults());
|
||||
this.variables.assign(cubeId, 'default', cube.getDefaults());
|
||||
|
||||
// 2. Variable collection
|
||||
if (this.options.isSessionReplay) {
|
||||
// Recorded answers go back into the scope they came from, so a replay
|
||||
// reproduces them even when `env` sets the same key to something else.
|
||||
const sessionCube = this.session.cubes.find((c) => c.key === cubeId);
|
||||
if (sessionCube) {
|
||||
this.variables.assign(cubeId, 'prompts', sessionCube.variables);
|
||||
this.variables.assign(cubeId, 'session', sessionCube.variables);
|
||||
}
|
||||
await this.fillSessionGaps(cube);
|
||||
} else if (this.options.useDefaults) {
|
||||
log.debug('Skipping prompts, using defaults', { cubeId });
|
||||
this.assertVariablesComplete(cube);
|
||||
@@ -155,13 +199,18 @@ export class BuildContext {
|
||||
cwd: cube.dir,
|
||||
command,
|
||||
env: cubeVars,
|
||||
secrets: cube.secrets,
|
||||
dependencies: [],
|
||||
});
|
||||
|
||||
if (!this.cubeSessions.some((s) => s.key === cubeId)) {
|
||||
// Every value the run settled on, not just the prompted ones — otherwise a
|
||||
// `--use-defaults` run records nothing and replaying it re-derives from
|
||||
// whatever the defaults and `env` happen to say now. Secrets are the one
|
||||
// exclusion; a replay asks for those again.
|
||||
this.cubeSessions.push({
|
||||
key: cubeId,
|
||||
variables: this.variables.get(cubeId, 'prompts'),
|
||||
variables: this.variables.persistable(cubeId),
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
/**
|
||||
* Factory functions for creating cube configurations
|
||||
* @module cubes/factories
|
||||
*/
|
||||
|
||||
import { type AnyObjectSchema, Manifest } from './types.js';
|
||||
|
||||
/**
|
||||
* Creates a manifest configuration for a cube
|
||||
*
|
||||
* @param opts - Manifest options including name, schema, dependencies, and hooks
|
||||
* @returns Manifest configuration object
|
||||
*/
|
||||
export function createManifest<Schema extends AnyObjectSchema>(
|
||||
opts: Pick<Manifest<Schema>, 'name'> & Partial<Omit<Manifest<Schema>, 'name'>>
|
||||
): Manifest<Schema> {
|
||||
return Manifest(opts);
|
||||
}
|
||||
|
||||
/**
|
||||
* Alias for createManifest - for backwards compatibility with existing manifests
|
||||
*/
|
||||
export const manifest = createManifest;
|
||||
|
||||
/**
|
||||
* @deprecated Use createManifest or manifest instead
|
||||
*/
|
||||
export const ManifestFactory = createManifest;
|
||||
|
||||
export { Manifest } from './types.js';
|
||||
@@ -6,34 +6,37 @@
|
||||
* @module cubes
|
||||
*/
|
||||
|
||||
// Dependencies
|
||||
export { BuildContext } from './dependencies.js';
|
||||
// Factory functions
|
||||
export {
|
||||
createManifest,
|
||||
manifest,
|
||||
} from './factories.js';
|
||||
// Loader
|
||||
export {
|
||||
findCubeDirectories,
|
||||
getCube,
|
||||
loadCubes,
|
||||
} from './loader.js';
|
||||
// The authoring surface lives in its own package so that a cube bundle can
|
||||
// depend on it without pulling the CLI in. Re-exported here so that
|
||||
// `import { cubes } from '@bitsquare/nopy'` in a manifest keeps working.
|
||||
export type {
|
||||
AnyObjectSchema,
|
||||
CubeSource,
|
||||
CubeVariables,
|
||||
DependencySpec,
|
||||
Hook,
|
||||
HookContext,
|
||||
LoadResult,
|
||||
} from './types.js';
|
||||
// Types
|
||||
} from '@bitsquare/nopy-cube';
|
||||
export {
|
||||
Cube,
|
||||
createManifest,
|
||||
Manifest,
|
||||
manifest,
|
||||
uniqid,
|
||||
zodInner,
|
||||
zodKind,
|
||||
} from './types.js';
|
||||
|
||||
// Utilities
|
||||
export { uniqid } from './utils.js';
|
||||
} from '@bitsquare/nopy-cube';
|
||||
// Dependencies
|
||||
export { BuildContext } from './dependencies.js';
|
||||
// Loader
|
||||
export type { CubeRoot } from './loader.js';
|
||||
export {
|
||||
findCubeDirectories,
|
||||
findCubeRoots,
|
||||
getCube,
|
||||
loadCubes,
|
||||
} from './loader.js';
|
||||
// Packages
|
||||
export type { CubePackage } from './packages.js';
|
||||
export { resolveCubePackages } from './packages.js';
|
||||
|
||||
@@ -3,21 +3,55 @@
|
||||
* @module cubes/loader
|
||||
*/
|
||||
|
||||
import module from 'node:module';
|
||||
import path from 'node:path';
|
||||
import { Cube, type CubeSource, type LoadResult, type Manifest } from '@bitsquare/nopy-cube';
|
||||
import { z } from 'zod';
|
||||
import { fs } from 'zx';
|
||||
import { loadConfig } from '../nopy.config.js';
|
||||
import { Cube, type LoadResult, type Manifest } from './types.js';
|
||||
import { resolveCubePackages } from './packages.js';
|
||||
|
||||
let hookRegistered = false;
|
||||
|
||||
/**
|
||||
* Traverses upwards from the current working directory to the root
|
||||
* and collects all directories that contain a `.npcubes` marker file.
|
||||
* Installs the fallback resolver that lets a manifest in a bare directory
|
||||
* import `@bitsquare/nopy-cube` or `zod` — see `resolve-hook.mjs`.
|
||||
*
|
||||
* Also includes directories specified in the `.nopyrc.json` configuration.
|
||||
*
|
||||
* @returns Array of absolute paths to directories containing cubes
|
||||
* `module.register()` is process-global and cannot be undone, so this runs once
|
||||
* and only when cubes are about to be imported. Registration failing is not
|
||||
* worth aborting a run over: without the hook, a cube that needed it fails on
|
||||
* its own import with a message that names the file.
|
||||
*/
|
||||
export function findCubeDirectories(): string[] {
|
||||
function registerResolveHook(): void {
|
||||
if (hookRegistered) return;
|
||||
hookRegistered = true;
|
||||
|
||||
try {
|
||||
// `from` is a URL inside this package, so the hook thread resolves the
|
||||
// fallbacks out of the running CLI's own dependencies.
|
||||
module.register('./resolve-hook.mjs', import.meta.url, { data: { from: import.meta.url } });
|
||||
} catch {
|
||||
// Nothing to do: the hook is a convenience, never load-bearing.
|
||||
}
|
||||
}
|
||||
|
||||
/** A directory to scan, and what put it in the list. */
|
||||
export interface CubeRoot {
|
||||
dir: string;
|
||||
source: CubeSource;
|
||||
}
|
||||
|
||||
/**
|
||||
* Collects every root to scan for cubes:
|
||||
*
|
||||
* - `cubeDirs` from the merged configuration,
|
||||
* - every ancestor of the working directory holding a `.npcubes` marker file,
|
||||
* - the cube directories of every package named in `cubePackages`.
|
||||
*
|
||||
* Only the last of those can fail — a missing directory is ignored, a missing
|
||||
* package is not (see `resolveCubePackages`).
|
||||
*/
|
||||
export function findCubeRoots(): { roots: CubeRoot[]; errors: string[] } {
|
||||
let currentDir = process.cwd();
|
||||
const config = loadConfig();
|
||||
const dirSet = new Set<string>(config.cubeDirs.map((dir) => path.resolve(process.cwd(), dir)));
|
||||
@@ -38,7 +72,29 @@ export function findCubeDirectories(): string[] {
|
||||
currentDir = parentDir;
|
||||
}
|
||||
|
||||
return [...dirSet];
|
||||
const roots: CubeRoot[] = [...dirSet].map((dir) => ({ dir, source: { type: 'dir', dir } }));
|
||||
|
||||
const { packages, errors } = resolveCubePackages(config.cubePackages);
|
||||
for (const pkg of packages) {
|
||||
for (const dir of pkg.dirs) {
|
||||
roots.push({ dir, source: { type: 'package', packageName: pkg.name, dir } });
|
||||
}
|
||||
}
|
||||
|
||||
return { roots, errors };
|
||||
}
|
||||
|
||||
/**
|
||||
* The directories {@link findCubeRoots} would scan.
|
||||
*
|
||||
* Kept for callers that only want the paths; anything that needs to attribute
|
||||
* a cube to where it came from should use `findCubeRoots` instead, which also
|
||||
* reports the errors this one drops.
|
||||
*
|
||||
* @returns Array of absolute paths to directories containing cubes
|
||||
*/
|
||||
export function findCubeDirectories(): string[] {
|
||||
return findCubeRoots().roots.map((root) => root.dir);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -56,10 +112,12 @@ interface CubeCandidate {
|
||||
manifest: Manifest;
|
||||
dir: string;
|
||||
deployScript: string;
|
||||
source: CubeSource;
|
||||
}
|
||||
|
||||
/** What one root directory contributed. */
|
||||
interface ScanResult {
|
||||
root: CubeRoot;
|
||||
candidates: CubeCandidate[];
|
||||
errors: string[];
|
||||
}
|
||||
@@ -99,11 +157,23 @@ async function scanDirectory(currentDir: string, result: ScanResult): Promise<vo
|
||||
manifest.id = cubeId;
|
||||
manifest.schema = manifest.schema ?? z.object({});
|
||||
|
||||
// A `secrets` entry naming a key that is not in the schema protects
|
||||
// nothing, and a typo in one is invisible at runtime — the value would
|
||||
// just be persisted. Cheaper to refuse the cube than to ship the leak.
|
||||
const unknown = (manifest.secrets ?? []).filter((key) => !(key in manifest.schema.shape));
|
||||
if (unknown.length > 0) {
|
||||
result.errors.push(
|
||||
`Invalid manifest in ${manifestPath}: 'secrets' names ${unknown.join(', ')}, ` +
|
||||
`which ${unknown.length === 1 ? 'is' : 'are'} not in the schema`
|
||||
);
|
||||
}
|
||||
|
||||
result.candidates.push({
|
||||
id: cubeId,
|
||||
manifest,
|
||||
dir: currentDir,
|
||||
deployScript: deployFile.name,
|
||||
source: result.root.source,
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
@@ -118,9 +188,23 @@ async function scanDirectory(currentDir: string, result: ScanResult): Promise<vo
|
||||
}
|
||||
}
|
||||
|
||||
/** The message a duplicate id produces. Aborts the run — see `nopy.main.ts`. */
|
||||
/**
|
||||
* The message a duplicate id produces. Aborts the run — see `nopy.main.ts`.
|
||||
*
|
||||
* There is deliberately no precedence rule to fall back on: two cubes claiming
|
||||
* one id are mutually exclusive, and the fix belongs upstream. So the message
|
||||
* has to carry everything needed to go and make it, which means naming every
|
||||
* claimant and how each got into the run.
|
||||
*/
|
||||
function duplicateError(id: string, group: CubeCandidate[]): string {
|
||||
const where = group.map((c) => ` ${c.dir}`).join('\n');
|
||||
const label = (candidate: CubeCandidate) =>
|
||||
candidate.source.type === 'package' ? `package ${candidate.source.packageName}` : 'directory';
|
||||
const width = Math.max(...group.map((candidate) => label(candidate).length));
|
||||
|
||||
const where = group
|
||||
.map((candidate) => ` ${label(candidate).padEnd(width)} ${candidate.dir}`)
|
||||
.join('\n');
|
||||
|
||||
return (
|
||||
`Duplicate cube id '${id}' from ${group.length} sources:\n${where}\n` +
|
||||
`Rename one of them, or remove a source from .nopyrc.json.`
|
||||
@@ -137,20 +221,21 @@ function duplicateError(id: string, group: CubeCandidate[]): string {
|
||||
* run, which is what makes the hard error testable.
|
||||
*/
|
||||
export async function loadCubes(): Promise<LoadResult> {
|
||||
const cubesFolders = findCubeDirectories();
|
||||
const { roots, errors: rootErrors } = findCubeRoots();
|
||||
registerResolveHook();
|
||||
|
||||
const scans = await Promise.all(
|
||||
cubesFolders.map(async (folder) => {
|
||||
const result: ScanResult = { candidates: [], errors: [] };
|
||||
if (fs.existsSync(folder)) {
|
||||
await scanDirectory(folder, result);
|
||||
roots.map(async (root) => {
|
||||
const result: ScanResult = { root, candidates: [], errors: [] };
|
||||
if (fs.existsSync(root.dir)) {
|
||||
await scanDirectory(root.dir, result);
|
||||
}
|
||||
return result;
|
||||
})
|
||||
);
|
||||
|
||||
// Promise.all preserves input order regardless of completion order.
|
||||
const errors = scans.flatMap((scan) => scan.errors);
|
||||
const errors = [...rootErrors, ...scans.flatMap((scan) => scan.errors)];
|
||||
|
||||
// One directory reachable from two roots (a `cubeDirs` entry nested under a
|
||||
// `.npcubes` marker, say) is one cube seen twice, not a collision.
|
||||
@@ -172,7 +257,7 @@ export async function loadCubes(): Promise<LoadResult> {
|
||||
// The map is still populated for the callers that only report; a duplicate
|
||||
// is fatal, so which candidate landed here never reaches a deploy.
|
||||
const [first] = group;
|
||||
cubes[id] = new Cube(first.manifest, first.dir, first.deployScript);
|
||||
cubes[id] = new Cube(first.manifest, first.dir, first.deployScript, first.source);
|
||||
}
|
||||
|
||||
return { cubes, errors };
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
/**
|
||||
* Resolving cube packages named in `cubePackages` to directories on disk.
|
||||
* @module cubes/packages
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import { createRequire } from 'node:module';
|
||||
import path from 'node:path';
|
||||
import type { CubePackageRef } from '../nopy.config.js';
|
||||
|
||||
/** An installed cube package, located and validated. */
|
||||
export interface CubePackage {
|
||||
/** The name it was requested under. */
|
||||
name: string;
|
||||
/** Absolute path to the package root. */
|
||||
root: string;
|
||||
/** Absolute paths to its cube directories, from `nopy.cubes`. */
|
||||
dirs: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Finds a package root without going through its `exports` map.
|
||||
*
|
||||
* `exports` is deliberately bypassed: a cube bundle ships directories, not an
|
||||
* entry point, and requiring it to declare one would make the contract heavier
|
||||
* for no gain. Reading `package.json` off disk also sidesteps pnpm's layout —
|
||||
* `existsSync` follows the symlink pnpm plants at `node_modules/<name>`, which
|
||||
* a directory scan would skip (`readdir` reports it as a symlink, not a
|
||||
* directory).
|
||||
*/
|
||||
function findPackageRoot(ref: CubePackageRef): string | undefined {
|
||||
// createRequire needs a file path, not a directory; the file need not exist.
|
||||
const req = createRequire(path.join(ref.from, 'noop.js'));
|
||||
for (const dir of req.resolve.paths(ref.spec) ?? []) {
|
||||
if (fs.existsSync(path.join(dir, ref.spec, 'package.json'))) {
|
||||
return path.join(dir, ref.spec);
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves every named package to its cube directories.
|
||||
*
|
||||
* Anything wrong is an error rather than a silent skip: naming a package in
|
||||
* `cubePackages` is a statement that cubes are expected from it, and errors
|
||||
* abort the run (see `nopy.main.ts`).
|
||||
*/
|
||||
export function resolveCubePackages(refs: CubePackageRef[]): {
|
||||
packages: CubePackage[];
|
||||
errors: string[];
|
||||
} {
|
||||
const packages: CubePackage[] = [];
|
||||
const errors: string[] = [];
|
||||
|
||||
// `mergeValue` only de-duplicates arrays of primitives, and these are
|
||||
// objects, so the same package named by a parent and a child config arrives
|
||||
// twice. Last wins: configs merge root-first, so the last occurrence came
|
||||
// from the most specific config and carries the right resolution origin.
|
||||
const unique = new Map<string, CubePackageRef>();
|
||||
for (const ref of refs) unique.set(ref.spec, ref);
|
||||
|
||||
for (const ref of unique.values()) {
|
||||
const root = findPackageRoot(ref);
|
||||
if (!root) {
|
||||
errors.push(`Cube package '${ref.spec}' is not installed (looked up from ${ref.from}).`);
|
||||
continue;
|
||||
}
|
||||
|
||||
let manifest: { nopy?: { cubes?: unknown } };
|
||||
try {
|
||||
manifest = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf-8'));
|
||||
} catch (err) {
|
||||
errors.push(`Cube package '${ref.spec}': cannot read ${root}/package.json: ${err}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const declared = manifest.nopy?.cubes;
|
||||
if (
|
||||
!Array.isArray(declared) ||
|
||||
declared.length === 0 ||
|
||||
!declared.every((entry) => typeof entry === 'string')
|
||||
) {
|
||||
errors.push(
|
||||
`Cube package '${ref.spec}' declares no cubes. ` +
|
||||
`Expected "nopy": { "cubes": ["./cubes"] } in ${root}/package.json.`
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
const dirs: string[] = [];
|
||||
for (const entry of declared as string[]) {
|
||||
const dir = path.resolve(root, entry);
|
||||
|
||||
if (dir !== root && !dir.startsWith(root + path.sep)) {
|
||||
errors.push(`Cube package '${ref.spec}': '${entry}' points outside the package.`);
|
||||
} else if (!fs.existsSync(dir)) {
|
||||
errors.push(`Cube package '${ref.spec}': '${entry}' does not exist in ${root}.`);
|
||||
} else {
|
||||
dirs.push(dir);
|
||||
}
|
||||
}
|
||||
|
||||
if (dirs.length > 0) packages.push({ name: ref.spec, root, dirs });
|
||||
}
|
||||
|
||||
return { packages, errors };
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
/**
|
||||
* A module resolve hook that lets a hand-written cube import the packages nopy
|
||||
* itself already has.
|
||||
*
|
||||
* A manifest is loaded with `import(manifestPath)`, so its imports resolve from
|
||||
* its own directory. A cube sitting in an arbitrary `cubeDirs` entry — no
|
||||
* package.json above it, no node_modules beside it — therefore cannot import
|
||||
* `@bitsquare/nopy-cube` or `zod` at all, and the run dies on
|
||||
* ERR_MODULE_NOT_FOUND before a single deploy is built.
|
||||
*
|
||||
* A published cube bundle never reaches this: it declares its own dependencies
|
||||
* and Node resolves them normally. This is for the local tree.
|
||||
*
|
||||
* Plain `.mjs` rather than TypeScript because the hook runs on its own thread,
|
||||
* loaded by Node directly from `dist` — there is no compile step in that path.
|
||||
*
|
||||
* @module cubes/resolve-hook
|
||||
*/
|
||||
|
||||
import { createRequire } from 'node:module';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
|
||||
/**
|
||||
* The specifiers worth rescuing: what a manifest legitimately needs and cannot
|
||||
* be expected to install for itself. Anything else stays a hard failure — a
|
||||
* cube that wants a library should depend on it.
|
||||
*/
|
||||
const FALLBACK_ROOTS = ['@bitsquare/nopy-cube', '@bitsquare/nopy', 'zod'];
|
||||
|
||||
/** @type {NodeRequire | undefined} */
|
||||
let fallbackRequire;
|
||||
|
||||
/**
|
||||
* @param {{ from: string }} data - a URL inside the running CLI's own package,
|
||||
* which is where the fallback resolution starts from.
|
||||
*/
|
||||
export function initialize(data) {
|
||||
fallbackRequire = createRequire(data.from);
|
||||
}
|
||||
|
||||
/** True for `zod` and for subpaths like `zod/v4` or `@bitsquare/nopy/package.json`. */
|
||||
function isCovered(specifier) {
|
||||
return FALLBACK_ROOTS.some((root) => specifier === root || specifier.startsWith(`${root}/`));
|
||||
}
|
||||
|
||||
export async function resolve(specifier, context, next) {
|
||||
try {
|
||||
// Normal resolution first, always. A consumer that has its own copy
|
||||
// installed keeps using it, so the hook can never introduce version skew —
|
||||
// it only fills in for a lookup that was going to fail.
|
||||
return await next(specifier, context);
|
||||
} catch (error) {
|
||||
if (!fallbackRequire || !isCovered(specifier)) throw error;
|
||||
try {
|
||||
return { url: pathToFileURL(fallbackRequire.resolve(specifier)).href, shortCircuit: true };
|
||||
} catch {
|
||||
// The CLI cannot see it either. Report the original failure, which names
|
||||
// the importer rather than the CLI.
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,202 +0,0 @@
|
||||
/**
|
||||
* Type definitions for Nopy cubes
|
||||
* @module cubes/types
|
||||
*/
|
||||
|
||||
import { z } from 'zod';
|
||||
|
||||
/**
|
||||
* Any object schema, whatever its shape.
|
||||
*
|
||||
* Stands in for zod 3's `z.AnyZodObject`, which zod 4 removed.
|
||||
*/
|
||||
export type AnyObjectSchema = z.ZodObject<Record<string, z.ZodType<any>>>;
|
||||
|
||||
/**
|
||||
* Variables that can be passed to a cube
|
||||
*/
|
||||
export type CubeVariables = Record<string, string | number | boolean>;
|
||||
|
||||
/**
|
||||
* A dependency specification
|
||||
*/
|
||||
export type DependencySpec = string | [id: string, variables?: CubeVariables];
|
||||
|
||||
/**
|
||||
* Context passed to cube hooks for executing other cubes
|
||||
*/
|
||||
export interface HookContext {
|
||||
exec: (key: string, variables: CubeVariables) => Promise<void> | void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Hook function type for before/after cube execution
|
||||
*/
|
||||
export type Hook<Schema extends AnyObjectSchema = AnyObjectSchema> = (
|
||||
ctx: HookContext,
|
||||
variables: z.infer<Schema>
|
||||
) => void | Promise<void>;
|
||||
|
||||
/**
|
||||
* User-defined specification for a cube
|
||||
*/
|
||||
export interface Manifest<Schema extends AnyObjectSchema = AnyObjectSchema> {
|
||||
/** Unique identifier for the cube (used for dependency references) */
|
||||
id: string;
|
||||
/** Human-readable name of the cube */
|
||||
name: string;
|
||||
/** Zod schema for validating cube variables */
|
||||
schema: Schema;
|
||||
/** Dynamic dependency resolver based on collected variables */
|
||||
dependencies?: (variables: z.infer<Schema>) => DependencySpec[];
|
||||
/** Hooks to run before cube execution */
|
||||
before?: Hook<Schema>[];
|
||||
/** Hooks to run after cube execution */
|
||||
after?: Hook<Schema>[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Factory function and namespace for Manifest
|
||||
*/
|
||||
export function Manifest<Schema extends AnyObjectSchema>(
|
||||
opts: Pick<Manifest<Schema>, 'name'> & Partial<Omit<Manifest<Schema>, 'name'>>
|
||||
): Manifest<Schema> {
|
||||
return {
|
||||
id: opts.id ?? '',
|
||||
name: opts.name,
|
||||
schema: opts.schema ?? (z.object({}) as unknown as Schema),
|
||||
dependencies: opts.dependencies,
|
||||
before: opts.before ?? [],
|
||||
after: opts.after ?? [],
|
||||
};
|
||||
}
|
||||
|
||||
export namespace Manifest {
|
||||
/**
|
||||
* Internal create helper
|
||||
*/
|
||||
export function create<Schema extends AnyObjectSchema>(
|
||||
opts: Pick<Manifest<Schema>, 'name'> & Partial<Omit<Manifest<Schema>, 'name'>>
|
||||
): Manifest<Schema> {
|
||||
return Manifest(opts);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* zod's runtime discriminant for a schema node, as a plain string.
|
||||
*
|
||||
* `instanceof z.ZodDefault` compares against the *running* copy of zod. A cube
|
||||
* manifest is free to build its schema with a different copy — its own
|
||||
* dependency, or one shipped inside a bundle — and then every `instanceof`
|
||||
* quietly returns false and the caller falls through to a wrong answer instead
|
||||
* of failing. `def.type` holds across instances, so nothing here may go back to
|
||||
* `instanceof`.
|
||||
*/
|
||||
export function zodKind(zodType: unknown): string {
|
||||
return (zodType as { def: { type: string } }).def.type;
|
||||
}
|
||||
|
||||
/**
|
||||
* The type a wrapper wraps — `.default()`, `.optional()`, `.nullable()`.
|
||||
* Only call this for a node whose {@link zodKind} is one of those.
|
||||
*/
|
||||
export function zodInner(zodType: unknown): z.ZodType {
|
||||
return (zodType as { def: { innerType: z.ZodType } }).def.innerType;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads the `.default()` off a schema field, unwrapping the wrappers that may
|
||||
* sit above it (`.default().optional()`, `.default().nullable()`).
|
||||
*
|
||||
* Returns `undefined` for a field that declares no default — which is also how
|
||||
* `requiredKeys()` recognises a field the user has to supply.
|
||||
*/
|
||||
function defaultValueOf(zodType: z.ZodType): unknown {
|
||||
const kind = zodKind(zodType);
|
||||
if (kind === 'default') {
|
||||
// zod 4 exposes `defaultValue` as a getter that already invokes a lazily
|
||||
// declared default; the function branch is insurance against that changing.
|
||||
const { defaultValue } = (zodType as unknown as { def: { defaultValue: unknown } }).def;
|
||||
return typeof defaultValue === 'function' ? defaultValue() : defaultValue;
|
||||
}
|
||||
if (kind === 'optional' || kind === 'nullable') {
|
||||
return defaultValueOf(zodInner(zodType));
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Where a cube was discovered.
|
||||
*
|
||||
* Worth carrying because a cube's own directory does not say how it got into
|
||||
* the run: `/…/node_modules/@acme/cubes-net/cubes/x` could equally have come
|
||||
* from a `cubeDirs` entry pointing straight at it.
|
||||
*/
|
||||
export type CubeSource =
|
||||
/** Found under a `cubeDirs` entry or a `.npcubes` marker, at `dir`. */
|
||||
| { type: 'dir'; dir: string }
|
||||
/** Contributed by a package named in `cubePackages`. */
|
||||
| { type: 'package'; packageName: string; dir: string };
|
||||
|
||||
/**
|
||||
* A fully loaded cube with its filesystem location and runtime state
|
||||
*/
|
||||
export class Cube<Schema extends AnyObjectSchema = AnyObjectSchema> {
|
||||
constructor(
|
||||
public readonly manifest: Manifest<Schema>,
|
||||
public readonly dir: string,
|
||||
public readonly deployScript: string,
|
||||
/** Defaults to the cube's own directory, for cubes built by hand. */
|
||||
public readonly source: CubeSource = { type: 'dir', dir }
|
||||
) {}
|
||||
|
||||
get id(): string {
|
||||
return this.manifest.id;
|
||||
}
|
||||
|
||||
get name(): string {
|
||||
return this.manifest.name;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns default values for the cube's schema.
|
||||
*
|
||||
* Parsing an empty object resolves every default in one go, but it fails
|
||||
* outright as soon as one field has no `.default()`. Falling back to a
|
||||
* per-field read keeps the defaults that *are* declared instead of dropping
|
||||
* the whole set — a single required field used to leave the cube with no
|
||||
* variables at all.
|
||||
*/
|
||||
getDefaults(): z.infer<Schema> {
|
||||
const parsed = this.manifest.schema.safeParse({});
|
||||
if (parsed.success) return parsed.data as z.infer<Schema>;
|
||||
|
||||
const defaults: Record<string, unknown> = {};
|
||||
for (const [key, zodType] of Object.entries(this.manifest.schema.shape)) {
|
||||
const value = defaultValueOf(zodType);
|
||||
if (value !== undefined) defaults[key] = value;
|
||||
}
|
||||
return defaults as z.infer<Schema>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Schema keys that have to be supplied from somewhere: no `.default()`, and
|
||||
* not optional. Nothing else can fill them in, so a run that cannot prompt
|
||||
* has to fail rather than deploy a cube with the value missing.
|
||||
*/
|
||||
requiredKeys(): string[] {
|
||||
return Object.entries(this.manifest.schema.shape)
|
||||
.filter(([, zodType]) => !zodType.safeParse(undefined).success)
|
||||
.map(([key]) => key);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Result of loading cubes from the filesystem
|
||||
*/
|
||||
export interface LoadResult {
|
||||
/** Map of cube key to Cube object */
|
||||
cubes: Record<string, Cube>;
|
||||
/** List of errors encountered during loading */
|
||||
errors: string[];
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
/**
|
||||
* Utility functions for cubes
|
||||
* @module cubes/utils
|
||||
*/
|
||||
|
||||
/**
|
||||
* Generates a random string of the specified length using the current nanotime as a seed.
|
||||
*
|
||||
* Uses a simple Linear Congruential Generator (LCG) seeded with high-resolution time.
|
||||
* Suitable for generating unique identifiers, not for cryptographic purposes.
|
||||
*
|
||||
* @param length - The desired length of the random string (default: 5)
|
||||
* @returns A random alphanumeric string of the specified length
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const id = uniqid(); // e.g., "Kx7Pm"
|
||||
* const longId = uniqid(10); // e.g., "Kx7PmQr2Yw"
|
||||
* ```
|
||||
*/
|
||||
export function uniqid(length = 5): string {
|
||||
const charset = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
|
||||
const charsetLength = charset.length;
|
||||
|
||||
// Use process.hrtime.bigint() for high-resolution time in nanoseconds
|
||||
let seed = Number(process.hrtime.bigint() % BigInt(Number.MAX_SAFE_INTEGER));
|
||||
|
||||
const randomString: string[] = [];
|
||||
|
||||
for (let i = 0; i < length; i++) {
|
||||
// Simple linear congruential generator (LCG) for pseudo-randomness
|
||||
seed = (seed * 48271) % 2147483647;
|
||||
const index = seed % charsetLength;
|
||||
randomString.push(charset[index]);
|
||||
}
|
||||
|
||||
return randomString.join('');
|
||||
}
|
||||
@@ -6,6 +6,9 @@
|
||||
|
||||
// Cubes module
|
||||
export * from './cubes/index.js';
|
||||
export type { Assignment, Origin, TVariables, Value } from './nopy.common.js';
|
||||
// Variables
|
||||
export { MASK, Variable, Variables } from './nopy.common.js';
|
||||
export type {
|
||||
ExecutionConfig,
|
||||
HistoryConfig,
|
||||
@@ -28,6 +31,8 @@ export type {
|
||||
// Executor
|
||||
export {
|
||||
executeDeployCalls,
|
||||
maskCommand,
|
||||
maskVariables,
|
||||
outputExecutionPlan,
|
||||
summarizeResults,
|
||||
} from './nopy.executor.js';
|
||||
@@ -60,14 +65,7 @@ export {
|
||||
} from './nopy.prompts.js';
|
||||
export type { AuthSession, CubeSession, NopySession } from './nopy.session.js';
|
||||
// Session management
|
||||
export {
|
||||
createSession,
|
||||
filterInternalVariables,
|
||||
listSessions,
|
||||
loadSession,
|
||||
saveSession,
|
||||
separateEnvAndCubeVariables,
|
||||
} from './nopy.session.js';
|
||||
export { createSession, listSessions, loadSession, saveSession } from './nopy.session.js';
|
||||
export type { WorkflowOptions, WorkflowResult } from './nopy.workflow.js';
|
||||
// Workflow
|
||||
export {
|
||||
|
||||
@@ -1,49 +1,204 @@
|
||||
/**
|
||||
* Environment variable configuration
|
||||
* Variable assignment and provenance
|
||||
* @module nopy.common
|
||||
*/
|
||||
export type TVariables = Record<string, string | number | boolean>;
|
||||
|
||||
export namespace Variables {
|
||||
export type ArtefactId = string;
|
||||
export type Scope = 'defaults' | 'prompts' | 'params';
|
||||
/** What a cube variable can hold — the value types `--data KEY=VALUE` can carry. */
|
||||
export type Value = string | number | boolean;
|
||||
|
||||
/** A flat bag of variable values, keyed by name. */
|
||||
export type TVariables = Record<string, Value>;
|
||||
|
||||
/**
|
||||
* Where a value came from, in ascending precedence.
|
||||
*
|
||||
* The order is the point. It used to be implied by the field order of an object
|
||||
* literal inside `Variables.get()` — load-bearing, invisible, and one careless
|
||||
* reformat away from silently changing which value wins. Here it is stated once,
|
||||
* in {@link RANK}, and everything else derives from it.
|
||||
*
|
||||
* - `default` — a `.default()` on the cube's schema
|
||||
* - `env` — the `env` block of `.nopyrc.json`
|
||||
* - `session` — read back from a recorded session on replay
|
||||
* - `prompt` — what the user typed
|
||||
* - `param` — handed over by a dependency spec or a hook's `exec()`
|
||||
*/
|
||||
export type Origin = 'default' | 'env' | 'session' | 'prompt' | 'param';
|
||||
|
||||
const RANK: Record<Origin, number> = {
|
||||
default: 0,
|
||||
env: 1,
|
||||
session: 2,
|
||||
prompt: 3,
|
||||
param: 4,
|
||||
};
|
||||
|
||||
/** One value handed to a variable, and where it came from. */
|
||||
export interface Assignment {
|
||||
value: Value;
|
||||
origin: Origin;
|
||||
}
|
||||
|
||||
export class Variables {
|
||||
/** @summary env as configured in cube or session script */
|
||||
defaults: Record<Variables.ArtefactId, TVariables> = {};
|
||||
/** @summary env as configured via prompts */
|
||||
prompts: Record<Variables.ArtefactId, TVariables> = {};
|
||||
/** @summary env as handed via params (on hook calls) */
|
||||
params: Record<Variables.ArtefactId, TVariables> = {};
|
||||
/** What a secret shows as wherever a value would otherwise be printed. */
|
||||
export const MASK = '********';
|
||||
|
||||
constructor(readonly global: TVariables = {}) {}
|
||||
/**
|
||||
* One variable of one cube, and every value it has ever been given.
|
||||
*
|
||||
* Two orderings are kept, deliberately: {@link assignments} is the raw trace in
|
||||
* the order things happened, and {@link ordered} re-ranks it by origin. The
|
||||
* first answers "how did we get here", the second answers "what wins".
|
||||
*/
|
||||
export class Variable {
|
||||
/** Every assignment received, newest first. Never reordered. */
|
||||
readonly assignments: Assignment[] = [];
|
||||
|
||||
assign(artefactId: Variables.ArtefactId, scope: Variables.Scope, values: TVariables = {}) {
|
||||
if (!this[scope][artefactId]) {
|
||||
this[scope][artefactId] = values;
|
||||
} else {
|
||||
Object.assign(this[scope][artefactId], values);
|
||||
}
|
||||
/**
|
||||
* Declared a secret by the cube's manifest: kept out of saved sessions and
|
||||
* masked wherever the value would otherwise be printed.
|
||||
*/
|
||||
redacted = false;
|
||||
|
||||
constructor(
|
||||
readonly cube: string,
|
||||
readonly name: string,
|
||||
first: Assignment
|
||||
) {
|
||||
this.assign(first);
|
||||
}
|
||||
|
||||
assign(assignment: Assignment): void {
|
||||
this.assignments.unshift(assignment);
|
||||
}
|
||||
|
||||
/**
|
||||
* Merges the scopes for one cube, lowest precedence first:
|
||||
* schema defaults → global `env` → prompts (or replayed session values) →
|
||||
* params handed over by a dependency or a hook.
|
||||
* The trace re-ranked by origin, winner first.
|
||||
*
|
||||
* Defaults sit at the bottom so `env` in `.nopyrc.json` can steer a run that
|
||||
* never prompts (`--use-defaults`); a key that a dependency supplies is never
|
||||
* prompted for, so prompts and params do not compete in practice.
|
||||
* Stability is load-bearing here. The trace is newest-first and
|
||||
* `Array.prototype.sort` is stable per spec, so two assignments sharing an
|
||||
* origin keep their relative order and the newer one stays in front: the
|
||||
* second dependency to pass a param wins, and the one it displaced is still
|
||||
* visible underneath instead of being overwritten out of existence.
|
||||
*/
|
||||
get(artefactId: Variables.ArtefactId, scope?: Variables.Scope): TVariables {
|
||||
if (scope) {
|
||||
return this[scope][artefactId] || {};
|
||||
}
|
||||
get ordered(): Assignment[] {
|
||||
return [...this.assignments].sort((a, b) => RANK[b.origin] - RANK[a.origin]);
|
||||
}
|
||||
|
||||
/** The assignment that wins. Never undefined — a Variable is born with one. */
|
||||
get effective(): Assignment {
|
||||
return this.ordered[0];
|
||||
}
|
||||
|
||||
get value(): Value {
|
||||
return this.effective.value;
|
||||
}
|
||||
|
||||
get origin(): Origin {
|
||||
return this.effective.origin;
|
||||
}
|
||||
|
||||
/** Safe to log: a redacted variable never yields its value. */
|
||||
toJSON(): { cube: string; name: string; value: Value; origin: Origin } {
|
||||
return {
|
||||
...this.defaults[artefactId],
|
||||
...this.global,
|
||||
...this.prompts[artefactId],
|
||||
...this.params[artefactId],
|
||||
cube: this.cube,
|
||||
name: this.name,
|
||||
value: this.redacted ? MASK : this.value,
|
||||
origin: this.origin,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Every variable of every cube in one run, with its provenance.
|
||||
*/
|
||||
export class Variables {
|
||||
private readonly store: Record<string, Record<string, Variable>> = {};
|
||||
private readonly secrets: Record<string, Set<string>> = {};
|
||||
|
||||
constructor(readonly env: TVariables = {}) {}
|
||||
|
||||
/**
|
||||
* Marks keys of one cube as holding secrets.
|
||||
*
|
||||
* Retroactive as well as prospective, so it does not matter whether the
|
||||
* caller declares before or after the values arrive.
|
||||
*/
|
||||
declareSecrets(cube: string, keys: readonly string[]): void {
|
||||
this.secrets[cube] ??= new Set<string>();
|
||||
const declared = this.secrets[cube];
|
||||
for (const key of keys) declared.add(key);
|
||||
for (const variable of this.all(cube)) {
|
||||
if (declared.has(variable.name)) variable.redacted = true;
|
||||
}
|
||||
}
|
||||
|
||||
isSecret(cube: string, name: string): boolean {
|
||||
return this.secrets[cube]?.has(name) ?? false;
|
||||
}
|
||||
|
||||
/** Records values for one cube, all at the same origin. */
|
||||
assign(cube: string, origin: Origin, values: TVariables = {}): void {
|
||||
const bucket = this.bucket(cube);
|
||||
for (const [name, value] of Object.entries(values)) {
|
||||
const existing = bucket[name];
|
||||
if (existing) existing.assign({ value, origin });
|
||||
else bucket[name] = this.create(cube, name, { value, origin });
|
||||
}
|
||||
}
|
||||
|
||||
/** Every variable known for one cube. */
|
||||
all(cube: string): Variable[] {
|
||||
return Object.values(this.store[cube] ?? {});
|
||||
}
|
||||
|
||||
/** One variable, or `undefined` if nothing has ever assigned to it. */
|
||||
of(cube: string, name: string): Variable | undefined {
|
||||
return this.store[cube]?.[name];
|
||||
}
|
||||
|
||||
/** The effective values for one cube — what goes on the pyinfra command line. */
|
||||
get(cube: string): TVariables {
|
||||
const values: TVariables = {};
|
||||
for (const variable of this.all(cube)) values[variable.name] = variable.value;
|
||||
return values;
|
||||
}
|
||||
|
||||
/**
|
||||
* The effective values minus anything declared secret — what a session
|
||||
* records. A secret is left out entirely rather than masked, so a replay sees
|
||||
* it as absent and asks for it again.
|
||||
*/
|
||||
persistable(cube: string): TVariables {
|
||||
const values: TVariables = {};
|
||||
for (const variable of this.all(cube)) {
|
||||
if (!variable.redacted) values[variable.name] = variable.value;
|
||||
}
|
||||
return values;
|
||||
}
|
||||
|
||||
private create(cube: string, name: string, first: Assignment): Variable {
|
||||
const variable = new Variable(cube, name, first);
|
||||
variable.redacted = this.isSecret(cube, name);
|
||||
return variable;
|
||||
}
|
||||
|
||||
/**
|
||||
* A cube's bucket, seeded on creation with the config `env`.
|
||||
*
|
||||
* `env` applies to every cube, so it becomes a real assignment on each of them
|
||||
* rather than a parallel bag merged in at read time. That is what lets it
|
||||
* carry an origin, show up in the trace, and lose to a prompt by the same rule
|
||||
* as everything else.
|
||||
*/
|
||||
private bucket(cube: string): Record<string, Variable> {
|
||||
const existing = this.store[cube];
|
||||
if (existing) return existing;
|
||||
|
||||
const bucket: Record<string, Variable> = {};
|
||||
this.store[cube] = bucket;
|
||||
for (const [name, value] of Object.entries(this.env)) {
|
||||
bucket[name] = this.create(cube, name, { value, origin: 'env' });
|
||||
}
|
||||
return bucket;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,9 +3,10 @@
|
||||
* @module nopy.executor
|
||||
*/
|
||||
|
||||
import type { DependencySpec } from '@bitsquare/nopy-cube';
|
||||
import { getLogger } from '@logtape/logtape';
|
||||
import { execa } from 'execa';
|
||||
import type { DependencySpec } from './cubes/types.js';
|
||||
import { MASK } from './nopy.common.js';
|
||||
|
||||
const log = getLogger(['nopy', 'executor']);
|
||||
|
||||
@@ -23,10 +24,47 @@ export interface DeployCall {
|
||||
command: string[];
|
||||
/** Environment variables for the cube */
|
||||
env: Record<string, unknown>;
|
||||
/** Schema keys the cube's manifest declared as secrets */
|
||||
secrets?: string[];
|
||||
/** Cube dependencies */
|
||||
dependencies: DependencySpec[];
|
||||
}
|
||||
|
||||
/**
|
||||
* The command as it is safe to show: the SSH password, and every `--data KEY=…`
|
||||
* whose key the manifest declared a secret, have their values replaced.
|
||||
*
|
||||
* pyinfra takes its data on the command line, so the real values have to be in
|
||||
* `call.command` — this is the last point before they would reach a log, a
|
||||
* `--print-only` dump or a dry-run plan.
|
||||
*/
|
||||
export function maskCommand(call: DeployCall): string {
|
||||
const command = call.command.join(' ');
|
||||
const quoteMeta = (key: string) => key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
|
||||
// The builder always quotes a `--data` value, so the closing quote bounds it.
|
||||
const masked = (call.secrets ?? []).reduce(
|
||||
(acc, key) => acc.replace(new RegExp(`(--data "${quoteMeta(key)}=)[^"]*"`, 'g'), `$1${MASK}"`),
|
||||
command
|
||||
);
|
||||
|
||||
return masked.replace(/(--password )\S+/g, `$1${MASK}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* The cube's variables as they are safe to show.
|
||||
*
|
||||
* This used to guess, masking any key whose name contained "password" — which
|
||||
* missed `TOKEN` and `PSK`, and was defeated anyway by the unmasked command
|
||||
* printed on the line above it. The manifest says which keys are secret now.
|
||||
*/
|
||||
export function maskVariables(call: DeployCall): Record<string, string> {
|
||||
const secrets = new Set(call.secrets ?? []);
|
||||
return Object.fromEntries(
|
||||
Object.entries(call.env).map(([key, value]) => [key, secrets.has(key) ? MASK : String(value)])
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Result of executing a deployment command
|
||||
*/
|
||||
@@ -73,7 +111,7 @@ async function executeCall(call: DeployCall): Promise<ExecutionResult> {
|
||||
|
||||
try {
|
||||
log.info(`Executing: ${call.cube} -> ${call.host}`);
|
||||
log.debug(`Command: ${commandStr}`);
|
||||
log.debug(`Command: ${maskCommand(call)}`);
|
||||
|
||||
// Inherit stdio for live output
|
||||
await execa({ shell: true })(commandStr, {
|
||||
@@ -112,8 +150,8 @@ export function outputExecutionPlan(calls: DeployCall[], asJson?: boolean): void
|
||||
const plan = calls.map((call) => ({
|
||||
cube: call.cube,
|
||||
host: call.host,
|
||||
command: call.command.join(' '),
|
||||
variables: call.env,
|
||||
command: maskCommand(call),
|
||||
variables: maskVariables(call),
|
||||
}));
|
||||
console.log(JSON.stringify({ plan }, null, 2));
|
||||
return;
|
||||
@@ -124,15 +162,13 @@ export function outputExecutionPlan(calls: DeployCall[], asJson?: boolean): void
|
||||
for (let i = 0; i < calls.length; i++) {
|
||||
const call = calls[i];
|
||||
console.log(`Step ${i + 1}: ${call.cube} -> ${call.host}`);
|
||||
console.log(` Command: ${call.command.join(' ')}`);
|
||||
console.log(` Command: ${maskCommand(call)}`);
|
||||
|
||||
const envKeys = Object.keys(call.env);
|
||||
if (envKeys.length > 0) {
|
||||
const variables = maskVariables(call);
|
||||
if (Object.keys(variables).length > 0) {
|
||||
console.log(' Variables:');
|
||||
for (const [key, value] of Object.entries(call.env)) {
|
||||
// Mask sensitive values
|
||||
const displayValue = key.toLowerCase().includes('password') ? '********' : String(value);
|
||||
console.log(` ${key}=${displayValue}`);
|
||||
for (const [key, value] of Object.entries(variables)) {
|
||||
console.log(` ${key}=${value}`);
|
||||
}
|
||||
}
|
||||
console.log();
|
||||
|
||||
@@ -8,7 +8,12 @@ import { BuildContext } from './cubes/dependencies.js';
|
||||
import { loadCubes } from './cubes/index.js';
|
||||
import { Variables } from './nopy.common.js';
|
||||
import { getConfigPaths, loadConfig } from './nopy.config.js';
|
||||
import { type ExecutionResult, executeDeployCalls, summarizeResults } from './nopy.executor.js';
|
||||
import {
|
||||
type ExecutionResult,
|
||||
executeDeployCalls,
|
||||
maskCommand,
|
||||
summarizeResults,
|
||||
} from './nopy.executor.js';
|
||||
import { addToHistory, DEFAULT_HISTORY_SIZE } from './nopy.history.js';
|
||||
import { type NopySession, saveSession } from './nopy.session.js';
|
||||
import { runWorkflow } from './nopy.workflow.js';
|
||||
@@ -72,6 +77,9 @@ function printActiveConfig(
|
||||
|
||||
if (config.hosts.length > 0) lines.push(` Hosts: ${config.hosts.join(', ')}`);
|
||||
if (config.cubeDirs.length > 0) lines.push(` Cube dirs: ${config.cubeDirs.join(', ')}`);
|
||||
if (config.cubePackages.length > 0) {
|
||||
lines.push(` Cube pkgs: ${config.cubePackages.map((ref) => ref.spec).join(', ')}`);
|
||||
}
|
||||
if (opts.continueOnError) lines.push(' Execution: continue-on-error');
|
||||
|
||||
const envEntries = Object.entries(config.env);
|
||||
@@ -185,7 +193,7 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
|
||||
const sessionForSaving: NopySession = {
|
||||
...workflow.session,
|
||||
cubes: context.cubeSessions,
|
||||
env: variables.get('global'),
|
||||
env: config.env,
|
||||
};
|
||||
|
||||
if (saveSessionPath && !workflow.isReplay) {
|
||||
@@ -203,7 +211,7 @@ export async function nopy(opts: NopyOptions = {}): Promise<NopyResult | undefin
|
||||
console.log('\n Deploy Commands\n ───────────────\n');
|
||||
for (const call of context.deployCalls) {
|
||||
console.log(` # ${call.cube} -> ${call.host}`);
|
||||
console.log(` ${call.command.join(' ')}\n`);
|
||||
console.log(` ${maskCommand(call)}\n`);
|
||||
}
|
||||
return {
|
||||
success: true,
|
||||
|
||||
@@ -36,11 +36,17 @@ function suggestCubes(input: string | undefined, choices: CubeChoice[]): CubeCho
|
||||
export async function CubeSelection(
|
||||
cubes: Record<string, Cube>
|
||||
): Promise<{ selectedCubes: string[] }> {
|
||||
// The package a cube came from is part of the label rather than a separate
|
||||
// column: `suggest` filters on the label, so typing a package name narrows
|
||||
// the list to that bundle.
|
||||
const cubeChoices: CubeChoice[] = Object.values(cubes)
|
||||
.sort((a, b) => a.id.localeCompare(b.id))
|
||||
.map((cube) => ({
|
||||
name: cube.id,
|
||||
message: `${cube.id} - ${cube.name}`,
|
||||
message:
|
||||
cube.source.type === 'package'
|
||||
? `${cube.id} - ${cube.name} (${cube.source.packageName})`
|
||||
: `${cube.id} - ${cube.name}`,
|
||||
}));
|
||||
|
||||
// Clear terminal and move cursor to top
|
||||
@@ -176,24 +182,35 @@ interface FormChoice {
|
||||
initial: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Asks the user for a cube's variables and records the answers.
|
||||
*
|
||||
* Reads what to offer out of `variables`, so the caller is expected to have
|
||||
* assigned the schema defaults first — which `BuildContext.resolveCube` does.
|
||||
* Deliberately not falling back to `cube.getDefaults()` here: calling it a
|
||||
* second time re-evaluates every lazily declared default, so a cube generating
|
||||
* one would show a different value than the one the run had already recorded.
|
||||
*/
|
||||
export async function VariableAssignment<S extends AnyObjectSchema>(
|
||||
cube: Cube<S>,
|
||||
variables: Variables
|
||||
variables: Variables,
|
||||
opts: { keys?: string[] } = {}
|
||||
) {
|
||||
const schema = cube.manifest.schema.shape;
|
||||
const defaults = cube.getDefaults() as Record<string, unknown>;
|
||||
const params = variables.get(cube.id, 'params');
|
||||
const resolved = variables.get(cube.id);
|
||||
const variablesToConfigure: Record<string, unknown> = {};
|
||||
|
||||
// Every schema key is offered, not just the ones carrying a `.default()` — a
|
||||
// field without one is precisely the field that has to be asked about. Keys a
|
||||
// dependency or hook already supplied are left alone. The value shown is the
|
||||
// Every schema key is offered by default, not just the ones carrying a
|
||||
// `.default()` — a field without one is precisely the field that has to be
|
||||
// asked about. `opts.keys` narrows that to a subset, which is how a replay
|
||||
// asks only about the gaps it cannot fill itself.
|
||||
//
|
||||
// A key a dependency or hook supplied is left alone. The value shown is the
|
||||
// one the run would otherwise use, so `env` from `.nopyrc.json` is visible
|
||||
// (and editable) rather than silently overridden by whatever is typed.
|
||||
for (const key of Object.keys(schema)) {
|
||||
if (params[key] !== undefined) continue;
|
||||
variablesToConfigure[key] = resolved[key] ?? defaults[key];
|
||||
for (const key of opts.keys ?? Object.keys(schema)) {
|
||||
if (variables.of(cube.id, key)?.origin === 'param') continue;
|
||||
variablesToConfigure[key] = resolved[key];
|
||||
}
|
||||
|
||||
if (Object.keys(variablesToConfigure).length === 0) return;
|
||||
@@ -217,7 +234,7 @@ export async function VariableAssignment<S extends AnyObjectSchema>(
|
||||
const zodType = schema[key];
|
||||
coercedResult[key] = zodType ? coerceValue(value, zodType) : value;
|
||||
}
|
||||
variables.assign(cube.id, 'prompts', coercedResult);
|
||||
variables.assign(cube.id, 'prompt', coercedResult);
|
||||
} catch {
|
||||
// User cancelled
|
||||
}
|
||||
|
||||
@@ -195,52 +195,3 @@ export function createSession(params: {
|
||||
env: params.env,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Filters out internal variables from cube variables
|
||||
*
|
||||
* Internal variables are those used by the prompts system
|
||||
* and should not be saved in session files.
|
||||
*
|
||||
* @param variables - Variables object
|
||||
* @returns Filtered variables without internal keys
|
||||
*/
|
||||
export function filterInternalVariables(
|
||||
variables: Record<string, unknown>
|
||||
): Record<string, unknown> {
|
||||
const internalKeys = ['customize'];
|
||||
const filtered: Record<string, unknown> = {};
|
||||
|
||||
for (const [key, value] of Object.entries(variables)) {
|
||||
if (!internalKeys.includes(key)) {
|
||||
filtered[key] = value;
|
||||
}
|
||||
}
|
||||
|
||||
return filtered;
|
||||
}
|
||||
|
||||
/**
|
||||
* Separates environment variables from cube-specific variables
|
||||
*
|
||||
* @param allVariables - All variables including env and cube-specific
|
||||
* @param envVariables - Known environment variables from config
|
||||
* @returns Object with separate env and cube variables
|
||||
*/
|
||||
export function separateEnvAndCubeVariables(
|
||||
allVariables: Record<string, unknown>,
|
||||
envVariables: Record<string, unknown>
|
||||
): { env: Record<string, unknown>; cubeVars: Record<string, unknown> } {
|
||||
const env: Record<string, unknown> = {};
|
||||
const cubeVars: Record<string, unknown> = {};
|
||||
|
||||
for (const [key, value] of Object.entries(allVariables)) {
|
||||
if (key in envVariables) {
|
||||
env[key] = value;
|
||||
} else {
|
||||
cubeVars[key] = value;
|
||||
}
|
||||
}
|
||||
|
||||
return { env, cubeVars };
|
||||
}
|
||||
|
||||
@@ -1,78 +1,203 @@
|
||||
/**
|
||||
* Tests for the Variables scope container.
|
||||
* Tests for Variable and Variables.
|
||||
*
|
||||
* The merge order is what makes a non-interactive run configurable, so it is
|
||||
* pinned down here rather than left to the callers to demonstrate.
|
||||
* Two things are pinned down here rather than left to the callers to
|
||||
* demonstrate: the precedence between origins, which is what makes a
|
||||
* non-interactive run configurable, and the tie-break between two assignments
|
||||
* sharing an origin, which is what keeps a losing dependency visible instead of
|
||||
* overwritten.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { Variables } from '../src/nopy.common.js';
|
||||
import { MASK, Variable, Variables } from '../src/nopy.common.js';
|
||||
|
||||
describe('Variables.assign', () => {
|
||||
it('creates the scope entry on first assignment and merges afterwards', () => {
|
||||
const variables = new Variables();
|
||||
describe('Variable ordering', () => {
|
||||
it('is born with its first assignment', () => {
|
||||
const variable = new Variable('cube-a', 'PORT', { value: 22, origin: 'default' });
|
||||
|
||||
variables.assign('cube-a', 'defaults', { A: 1 });
|
||||
variables.assign('cube-a', 'defaults', { B: 2 });
|
||||
|
||||
expect(variables.get('cube-a', 'defaults')).toEqual({ A: 1, B: 2 });
|
||||
expect(variable.value).toBe(22);
|
||||
expect(variable.origin).toBe('default');
|
||||
expect(variable.assignments).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('defaults to an empty assignment', () => {
|
||||
const variables = new Variables();
|
||||
it('lets a higher origin win however late it arrives', () => {
|
||||
const variable = new Variable('cube-a', 'PORT', { value: 22, origin: 'default' });
|
||||
|
||||
variables.assign('cube-a', 'prompts');
|
||||
variable.assign({ value: 8080, origin: 'param' });
|
||||
variable.assign({ value: 2222, origin: 'env' });
|
||||
|
||||
expect(variables.get('cube-a', 'prompts')).toEqual({});
|
||||
expect(variable.value).toBe(8080);
|
||||
expect(variable.origin).toBe('param');
|
||||
});
|
||||
|
||||
it('keeps scopes and cubes apart', () => {
|
||||
const variables = new Variables();
|
||||
it('keeps the newest of two assignments sharing an origin', () => {
|
||||
const variable = new Variable('cube-a', 'PORT', { value: 1, origin: 'param' });
|
||||
|
||||
variables.assign('cube-a', 'params', { A: 1 });
|
||||
variable.assign({ value: 2, origin: 'param' });
|
||||
|
||||
expect(variables.get('cube-a', 'prompts')).toEqual({});
|
||||
expect(variables.get('cube-b', 'params')).toEqual({});
|
||||
expect(variable.value).toBe(2);
|
||||
// The displaced one is still on record — that is the whole point of the
|
||||
// trace, and a sort that broke ties by rank alone would lose it.
|
||||
expect(variable.ordered.map((a) => a.value)).toEqual([2, 1]);
|
||||
});
|
||||
|
||||
it('keeps the raw trace in assignment order, newest first', () => {
|
||||
const variable = new Variable('cube-a', 'PORT', { value: 22, origin: 'default' });
|
||||
|
||||
variable.assign({ value: 8080, origin: 'param' });
|
||||
variable.assign({ value: 2222, origin: 'env' });
|
||||
|
||||
expect(variable.assignments.map((a) => a.origin)).toEqual(['env', 'param', 'default']);
|
||||
expect(variable.ordered.map((a) => a.origin)).toEqual(['param', 'env', 'default']);
|
||||
});
|
||||
|
||||
it('ranks every origin', () => {
|
||||
const variable = new Variable('cube-a', 'PORT', { value: 'd', origin: 'default' });
|
||||
|
||||
variable.assign({ value: 'e', origin: 'env' });
|
||||
expect(variable.value).toBe('e');
|
||||
variable.assign({ value: 's', origin: 'session' });
|
||||
expect(variable.value).toBe('s');
|
||||
variable.assign({ value: 'p', origin: 'prompt' });
|
||||
expect(variable.value).toBe('p');
|
||||
variable.assign({ value: 'x', origin: 'param' });
|
||||
expect(variable.value).toBe('x');
|
||||
});
|
||||
|
||||
it('never yields the value of a redacted variable when serialised', () => {
|
||||
const variable = new Variable('cube-a', 'PASSWORD', { value: 'hunter2', origin: 'prompt' });
|
||||
variable.redacted = true;
|
||||
|
||||
expect(JSON.parse(JSON.stringify(variable))).toEqual({
|
||||
cube: 'cube-a',
|
||||
name: 'PASSWORD',
|
||||
value: MASK,
|
||||
origin: 'prompt',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('Variables.get precedence', () => {
|
||||
it('lets global env override a schema default', () => {
|
||||
const variables = new Variables({ PORT: 2222 });
|
||||
variables.assign('cube-a', 'defaults', { PORT: 22 });
|
||||
describe('Variables.assign', () => {
|
||||
it('creates a variable on first assignment and appends afterwards', () => {
|
||||
const variables = new Variables();
|
||||
|
||||
expect(variables.get('cube-a').PORT).toBe(2222);
|
||||
variables.assign('cube-a', 'default', { A: 1 });
|
||||
variables.assign('cube-a', 'prompt', { A: 2 });
|
||||
|
||||
expect(variables.of('cube-a', 'A')?.assignments).toHaveLength(2);
|
||||
expect(variables.get('cube-a')).toEqual({ A: 2 });
|
||||
});
|
||||
|
||||
it('lets a prompt override global env', () => {
|
||||
it('tolerates an empty assignment', () => {
|
||||
const variables = new Variables();
|
||||
|
||||
variables.assign('cube-a', 'prompt');
|
||||
|
||||
expect(variables.get('cube-a')).toEqual({});
|
||||
});
|
||||
|
||||
it('keeps cubes apart', () => {
|
||||
const variables = new Variables();
|
||||
|
||||
variables.assign('cube-a', 'param', { A: 1 });
|
||||
|
||||
expect(variables.get('cube-b')).toEqual({});
|
||||
expect(variables.of('cube-b', 'A')).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Variables precedence', () => {
|
||||
it('lets config env override a schema default', () => {
|
||||
const variables = new Variables({ PORT: 2222 });
|
||||
variables.assign('cube-a', 'defaults', { PORT: 22 });
|
||||
variables.assign('cube-a', 'prompts', { PORT: 8080 });
|
||||
variables.assign('cube-a', 'default', { PORT: 22 });
|
||||
|
||||
expect(variables.get('cube-a').PORT).toBe(2222);
|
||||
expect(variables.of('cube-a', 'PORT')?.origin).toBe('env');
|
||||
});
|
||||
|
||||
it('lets a prompt override config env', () => {
|
||||
const variables = new Variables({ PORT: 2222 });
|
||||
variables.assign('cube-a', 'default', { PORT: 22 });
|
||||
variables.assign('cube-a', 'prompt', { PORT: 8080 });
|
||||
|
||||
expect(variables.get('cube-a').PORT).toBe(8080);
|
||||
});
|
||||
|
||||
it('lets a replayed session value override config env', () => {
|
||||
const variables = new Variables({ PORT: 2222 });
|
||||
variables.assign('cube-a', 'default', { PORT: 22 });
|
||||
variables.assign('cube-a', 'session', { PORT: 3000 });
|
||||
|
||||
expect(variables.get('cube-a').PORT).toBe(3000);
|
||||
});
|
||||
|
||||
it('lets a dependency param override everything else', () => {
|
||||
const variables = new Variables({ PORT: 2222 });
|
||||
variables.assign('cube-a', 'defaults', { PORT: 22 });
|
||||
variables.assign('cube-a', 'prompts', { PORT: 8080 });
|
||||
variables.assign('cube-a', 'params', { PORT: 9090 });
|
||||
variables.assign('cube-a', 'default', { PORT: 22 });
|
||||
variables.assign('cube-a', 'prompt', { PORT: 8080 });
|
||||
variables.assign('cube-a', 'param', { PORT: 9090 });
|
||||
|
||||
expect(variables.get('cube-a').PORT).toBe(9090);
|
||||
});
|
||||
|
||||
it('merges keys from every scope', () => {
|
||||
it('merges keys from every origin', () => {
|
||||
const variables = new Variables({ G: 'g' });
|
||||
variables.assign('cube-a', 'defaults', { D: 'd' });
|
||||
variables.assign('cube-a', 'prompts', { P: 'p' });
|
||||
variables.assign('cube-a', 'params', { X: 'x' });
|
||||
variables.assign('cube-a', 'default', { D: 'd' });
|
||||
variables.assign('cube-a', 'prompt', { P: 'p' });
|
||||
variables.assign('cube-a', 'param', { X: 'x' });
|
||||
|
||||
expect(variables.get('cube-a')).toEqual({ G: 'g', D: 'd', P: 'p', X: 'x' });
|
||||
});
|
||||
|
||||
it('applies global env to every cube', () => {
|
||||
it('applies config env to every cube it is asked about', () => {
|
||||
const variables = new Variables({ SHARED: 'yes' });
|
||||
|
||||
expect(variables.get('anything').SHARED).toBe('yes');
|
||||
variables.assign('cube-a', 'default', {});
|
||||
variables.assign('cube-b', 'default', {});
|
||||
|
||||
expect(variables.get('cube-a').SHARED).toBe('yes');
|
||||
expect(variables.get('cube-b').SHARED).toBe('yes');
|
||||
expect(variables.of('cube-a', 'SHARED')?.origin).toBe('env');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Variables secrets', () => {
|
||||
it('excludes a declared secret from what a session records', () => {
|
||||
const variables = new Variables();
|
||||
variables.declareSecrets('cube-a', ['PASSWORD']);
|
||||
variables.assign('cube-a', 'prompt', { USER: 'bob', PASSWORD: 'hunter2' });
|
||||
|
||||
expect(variables.get('cube-a')).toEqual({ USER: 'bob', PASSWORD: 'hunter2' });
|
||||
expect(variables.persistable('cube-a')).toEqual({ USER: 'bob' });
|
||||
});
|
||||
|
||||
it('marks values that arrived before the declaration', () => {
|
||||
const variables = new Variables();
|
||||
variables.assign('cube-a', 'prompt', { PASSWORD: 'hunter2' });
|
||||
|
||||
variables.declareSecrets('cube-a', ['PASSWORD']);
|
||||
|
||||
expect(variables.of('cube-a', 'PASSWORD')?.redacted).toBe(true);
|
||||
expect(variables.persistable('cube-a')).toEqual({});
|
||||
});
|
||||
|
||||
it('redacts a secret supplied through config env', () => {
|
||||
const variables = new Variables({ PASSWORD: 'from-env' });
|
||||
variables.declareSecrets('cube-a', ['PASSWORD']);
|
||||
|
||||
variables.assign('cube-a', 'default', {});
|
||||
|
||||
expect(variables.get('cube-a').PASSWORD).toBe('from-env');
|
||||
expect(variables.persistable('cube-a')).toEqual({});
|
||||
});
|
||||
|
||||
it('keeps secret declarations per cube', () => {
|
||||
const variables = new Variables();
|
||||
variables.declareSecrets('cube-a', ['PASSWORD']);
|
||||
variables.assign('cube-a', 'prompt', { PASSWORD: 'a' });
|
||||
variables.assign('cube-b', 'prompt', { PASSWORD: 'b' });
|
||||
|
||||
expect(variables.persistable('cube-a')).toEqual({});
|
||||
expect(variables.persistable('cube-b')).toEqual({ PASSWORD: 'b' });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -231,6 +231,53 @@ describe('config loading', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('cubePackages', () => {
|
||||
it('tags each package with the directory of the config that named it', () => {
|
||||
const child = path.join(rootDir, 'child');
|
||||
write(rootDir, { cubePackages: ['@acme/cubes-net'] });
|
||||
write(child, { cubePackages: ['@acme/cubes-caddy'] });
|
||||
process.chdir(child);
|
||||
|
||||
// Not a path, so nothing is rewritten — but resolution has to start from
|
||||
// the config that asked, which is the only place `from` can come from.
|
||||
expect(loadConfig().cubePackages).toEqual([
|
||||
{ spec: '@acme/cubes-net', from: rootDir },
|
||||
{ spec: '@acme/cubes-caddy', from: child },
|
||||
]);
|
||||
});
|
||||
|
||||
it('defaults to an empty list', () => {
|
||||
write(rootDir, { hosts: ['web-1'] });
|
||||
expect(loadConfig().cubePackages).toEqual([]);
|
||||
});
|
||||
|
||||
it('lets a child config replace the list with an override strategy', () => {
|
||||
const child = path.join(rootDir, 'child');
|
||||
write(rootDir, { cubePackages: ['@acme/cubes-net'] });
|
||||
write(child, {
|
||||
cubePackages: ['@acme/cubes-caddy'],
|
||||
resolution: { cubePackages: 'override' },
|
||||
});
|
||||
process.chdir(child);
|
||||
|
||||
expect(loadConfig().cubePackages).toEqual([{ spec: '@acme/cubes-caddy', from: child }]);
|
||||
});
|
||||
|
||||
it('keeps both entries when parent and child name the same package', () => {
|
||||
// Refs are objects, so the primitives-only dedupe in mergeValue does not
|
||||
// fire. resolveCubePackages collapses them, last-wins.
|
||||
const child = path.join(rootDir, 'child');
|
||||
write(rootDir, { cubePackages: ['@acme/cubes-net'] });
|
||||
write(child, { cubePackages: ['@acme/cubes-net'] });
|
||||
process.chdir(child);
|
||||
|
||||
expect(loadConfig().cubePackages).toEqual([
|
||||
{ spec: '@acme/cubes-net', from: rootDir },
|
||||
{ spec: '@acme/cubes-net', from: child },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('saveConfig', () => {
|
||||
it('writes a new config file at the given path', () => {
|
||||
const target = path.join(rootDir, 'custom.json');
|
||||
|
||||
@@ -2,10 +2,10 @@
|
||||
* Edge cases for BuildContext: unknown cubes, session replay and auth flags.
|
||||
*/
|
||||
|
||||
import { type AnyObjectSchema, Cube, Manifest } from '@bitsquare/nopy-cube';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { z } from 'zod';
|
||||
import { BuildContext } from '../src/cubes/dependencies.js';
|
||||
import { Cube, Manifest } from '../src/cubes/types.js';
|
||||
import { Variables } from '../src/nopy.common.js';
|
||||
import type { NopyConfig } from '../src/nopy.config.js';
|
||||
import type { NopySession } from '../src/nopy.session.js';
|
||||
@@ -20,6 +20,14 @@ import { VariableAssignment } from '../src/nopy.prompts.js';
|
||||
const testCube = (id: string, schema = z.object({})) =>
|
||||
new Cube(Manifest.create({ id, name: `Test ${id}`, schema }), `/test/${id}`, 'deploy.py');
|
||||
|
||||
/** A cube whose PASSWORD the manifest declares a secret. */
|
||||
const secretCube = (id: string, schema: AnyObjectSchema) =>
|
||||
new Cube(
|
||||
Manifest.create({ id, name: `Test ${id}`, schema, secrets: ['PASSWORD'] }),
|
||||
`/test/${id}`,
|
||||
'deploy.py'
|
||||
);
|
||||
|
||||
const config = { env: {} } as NopyConfig;
|
||||
const session = (cubes: NopySession['cubes'] = []) => ({ cubes }) as NopySession;
|
||||
|
||||
@@ -102,6 +110,134 @@ describe('BuildContext session replay', () => {
|
||||
|
||||
expect(VariableAssignment).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('lets a recorded value beat config env', async () => {
|
||||
const cube = testCube('cube-a', z.object({ PORT: z.string().default('3000') }));
|
||||
const context = new BuildContext(
|
||||
{ 'cube-a': cube },
|
||||
new Variables({ PORT: '2222' }),
|
||||
session([{ key: 'cube-a', variables: { PORT: '9090' } }]),
|
||||
{ env: { PORT: '2222' } } as NopyConfig,
|
||||
{ method: 'ssh' },
|
||||
{ isSessionReplay: true }
|
||||
);
|
||||
|
||||
await context.resolveCube('cube-a', 'host1');
|
||||
|
||||
expect(context.deployCalls[0].env.PORT).toBe('9090');
|
||||
});
|
||||
});
|
||||
|
||||
describe('BuildContext replay gaps', () => {
|
||||
const replay = (cube: Cube, recorded: Record<string, string> = {}, options = {}) =>
|
||||
new BuildContext(
|
||||
{ [cube.id]: cube },
|
||||
new Variables(),
|
||||
session([{ key: cube.id, variables: recorded }]),
|
||||
config,
|
||||
{ method: 'ssh' },
|
||||
{ isSessionReplay: true, ...options }
|
||||
);
|
||||
|
||||
it('asks for a required variable the session never recorded', async () => {
|
||||
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
|
||||
vi.mocked(VariableAssignment).mockImplementation(async (_cube, variables) => {
|
||||
variables.assign('cube-a', 'prompt', { SSID: 'typed' });
|
||||
});
|
||||
|
||||
const context = replay(cube);
|
||||
await context.resolveCube('cube-a', 'host1');
|
||||
|
||||
expect(VariableAssignment).toHaveBeenCalledWith(cube, expect.anything(), { keys: ['SSID'] });
|
||||
expect(context.deployCalls[0].env.SSID).toBe('typed');
|
||||
});
|
||||
|
||||
it('asks for a secret even though a default already filled it in', async () => {
|
||||
const cube = secretCube('cube-a', z.object({ PASSWORD: z.string().default('changeme') }));
|
||||
vi.mocked(VariableAssignment).mockImplementation(async (_cube, variables) => {
|
||||
variables.assign('cube-a', 'prompt', { PASSWORD: 'real' });
|
||||
});
|
||||
|
||||
const context = replay(cube);
|
||||
await context.resolveCube('cube-a', 'host1');
|
||||
|
||||
expect(VariableAssignment).toHaveBeenCalledWith(cube, expect.anything(), {
|
||||
keys: ['PASSWORD'],
|
||||
});
|
||||
expect(context.deployCalls[0].env.PASSWORD).toBe('real');
|
||||
});
|
||||
|
||||
it('asks nothing when the session covers everything', async () => {
|
||||
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
|
||||
|
||||
await replay(cube, { SSID: 'recorded' }).resolveCube('cube-a', 'host1');
|
||||
|
||||
expect(VariableAssignment).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('refuses to deploy when the form was cancelled', async () => {
|
||||
const cube = testCube('cube-a', z.object({ SSID: z.string() }));
|
||||
// The real VariableAssignment swallows a cancelled form, so the gap check
|
||||
// has to run again afterwards or the cube ships without the variable.
|
||||
vi.mocked(VariableAssignment).mockResolvedValue(undefined);
|
||||
|
||||
const context = replay(cube);
|
||||
|
||||
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
||||
'Cube "cube-a" is missing SSID and cannot be deployed.'
|
||||
);
|
||||
expect(context.deployCalls).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('cannot fill a gap when --use-defaults forbids prompting', async () => {
|
||||
const cube = secretCube('cube-a', z.object({ PASSWORD: z.string().default('changeme') }));
|
||||
|
||||
const context = replay(cube, {}, { useDefaults: true });
|
||||
|
||||
await expect(context.resolveCube('cube-a', 'host1')).rejects.toThrow(
|
||||
/cannot be replayed with --use-defaults: PASSWORD/
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('BuildContext session recording', () => {
|
||||
it('records every value the run settled on, not only the prompted ones', async () => {
|
||||
const cube = testCube('cube-a', z.object({ PORT: z.string().default('3000') }));
|
||||
const context = new BuildContext(
|
||||
{ 'cube-a': cube },
|
||||
new Variables({ REGION: 'eu' }),
|
||||
session(),
|
||||
config,
|
||||
{ method: 'ssh' },
|
||||
{ useDefaults: true }
|
||||
);
|
||||
|
||||
await context.resolveCube('cube-a', 'host1');
|
||||
|
||||
expect(context.cubeSessions[0].variables).toEqual({ PORT: '3000', REGION: 'eu' });
|
||||
});
|
||||
|
||||
it('keeps a declared secret out of the session', async () => {
|
||||
const cube = secretCube(
|
||||
'cube-a',
|
||||
z.object({ USER: z.string().default('bob'), PASSWORD: z.string().default('changeme') })
|
||||
);
|
||||
const context = new BuildContext(
|
||||
{ 'cube-a': cube },
|
||||
new Variables(),
|
||||
session(),
|
||||
config,
|
||||
{ method: 'ssh' },
|
||||
{ useDefaults: true }
|
||||
);
|
||||
|
||||
await context.resolveCube('cube-a', 'host1');
|
||||
|
||||
// Still handed to pyinfra — just never written down.
|
||||
expect(context.deployCalls[0].env.PASSWORD).toBe('changeme');
|
||||
expect(context.deployCalls[0].secrets).toEqual(['PASSWORD']);
|
||||
expect(context.cubeSessions[0].variables).toEqual({ USER: 'bob' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('BuildContext --use-defaults', () => {
|
||||
|
||||
@@ -2,10 +2,10 @@
|
||||
* Tests for cubes/dependencies module (BuildContext)
|
||||
*/
|
||||
|
||||
import { Cube, Manifest } from '@bitsquare/nopy-cube';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { z } from 'zod';
|
||||
import { BuildContext } from '../src/cubes/dependencies.js';
|
||||
import { Cube, Manifest } from '../src/cubes/types.js';
|
||||
import { Variables } from '../src/nopy.common.js';
|
||||
|
||||
// Mock VariableAssignment to avoid hanging on prompts
|
||||
@@ -82,7 +82,7 @@ describe('BuildContext.resolveCube', () => {
|
||||
|
||||
// Test with USE_A = false
|
||||
const vars2 = new Variables();
|
||||
vars2.assign('cube-c', 'params', { USE_A: false });
|
||||
vars2.assign('cube-c', 'param', { USE_A: false });
|
||||
const context2 = new BuildContext(cubes, vars2, { cubes: [] } as any, { env: {} } as any, {
|
||||
method: 'ssh',
|
||||
});
|
||||
|
||||
@@ -1,39 +0,0 @@
|
||||
/**
|
||||
* Tests for cubes/factories module
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { z } from 'zod';
|
||||
import { createManifest, manifest } from '../src/cubes/factories.js';
|
||||
|
||||
describe('createManifest', () => {
|
||||
it('creates manifest with basic properties', () => {
|
||||
const m = createManifest({
|
||||
id: 'test-cube',
|
||||
name: 'Test Cube',
|
||||
});
|
||||
|
||||
expect(m.id).toBe('test-cube');
|
||||
expect(m.name).toBe('Test Cube');
|
||||
expect(m.schema).toBeDefined();
|
||||
expect(m.before).toEqual([]);
|
||||
expect(m.after).toEqual([]);
|
||||
});
|
||||
|
||||
it('accepts schema', () => {
|
||||
const schema = z.object({
|
||||
VERSION: z.string().default('1.0'),
|
||||
});
|
||||
|
||||
const m = createManifest({
|
||||
name: 'Test Cube',
|
||||
schema,
|
||||
});
|
||||
|
||||
expect(m.schema).toBe(schema);
|
||||
});
|
||||
|
||||
it('manifest is alias for createManifest', () => {
|
||||
expect(manifest).toBe(createManifest);
|
||||
});
|
||||
});
|
||||
@@ -103,6 +103,36 @@ describe('loader edge cases', () => {
|
||||
expect(cubes['no-schema'].getDefaults()).toEqual({});
|
||||
});
|
||||
|
||||
it('reports a secrets entry that names no schema key', async () => {
|
||||
// A typo here protects nothing and is invisible at runtime — the value
|
||||
// would simply be persisted.
|
||||
cube(
|
||||
'typo',
|
||||
`import { z } from 'zod';
|
||||
export default { id: 'typo', name: 'Typo', secrets: ['PASSWROD'],
|
||||
schema: z.object({ PASSWORD: z.string().default('x') }) };`
|
||||
);
|
||||
|
||||
const { errors } = await loadCubes();
|
||||
|
||||
expect(errors[0]).toMatch(/'secrets' names PASSWROD, which is not in the schema/);
|
||||
});
|
||||
|
||||
it('accepts a secrets entry that matches a schema key', async () => {
|
||||
cube(
|
||||
'ok',
|
||||
`import { z } from 'zod';
|
||||
export default { id: 'ok', name: 'Ok', secrets: ['PASSWORD'],
|
||||
schema: z.object({ PASSWORD: z.string().default('x') }) };`
|
||||
);
|
||||
|
||||
const { cubes, errors } = await loadCubes();
|
||||
|
||||
expect(errors).toEqual([]);
|
||||
expect(cubes.ok.isSecret('PASSWORD')).toBe(true);
|
||||
expect(cubes.ok.isSecret('OTHER')).toBe(false);
|
||||
});
|
||||
|
||||
it('reports a manifest whose default export is not an object', async () => {
|
||||
cube('bad-export', 'export default "just a string"');
|
||||
|
||||
@@ -219,6 +249,91 @@ describe('loader edge cases', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('cubePackages', () => {
|
||||
/** Installs a cube package into the temp project's node_modules. */
|
||||
const installPackage = (name: string, cubeId: string) => {
|
||||
const root = path.join(tmpDir, 'node_modules', name);
|
||||
const cubeDir = path.join(root, 'cubes', cubeId);
|
||||
fs.mkdirSync(cubeDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(root, 'package.json'),
|
||||
JSON.stringify({ name, nopy: { cubes: ['./cubes'] } })
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(cubeDir, 'manifest.mjs'),
|
||||
`export default { id: "${cubeId}", name: "From a package" }`
|
||||
);
|
||||
fs.writeFileSync(path.join(cubeDir, 'deploy.py'), '# deploy');
|
||||
return cubeDir;
|
||||
};
|
||||
|
||||
const config = (extra: Record<string, unknown>) =>
|
||||
fs.writeFileSync(path.join(tmpDir, '.nopyrc.json'), JSON.stringify(extra));
|
||||
|
||||
it('loads cubes from a package and records where they came from', async () => {
|
||||
const cubeDir = installPackage('@acme/cubes-net', 'net:vpn');
|
||||
config({ cubeDirs: [], cubePackages: ['@acme/cubes-net'] });
|
||||
|
||||
const { cubes, errors } = await loadCubes();
|
||||
|
||||
expect(errors).toEqual([]);
|
||||
expect(cubes['net:vpn'].dir).toBe(cubeDir);
|
||||
expect(cubes['net:vpn'].source).toEqual({
|
||||
type: 'package',
|
||||
packageName: '@acme/cubes-net',
|
||||
dir: path.join(tmpDir, 'node_modules', '@acme/cubes-net', 'cubes'),
|
||||
});
|
||||
});
|
||||
|
||||
it('marks a cube found under a plain directory as directory-sourced', async () => {
|
||||
cube('local', 'export default { id: "local", name: "Local" }');
|
||||
|
||||
const { cubes } = await loadCubes();
|
||||
|
||||
expect(cubes.local.source).toEqual({ type: 'dir', dir: tmpDir });
|
||||
});
|
||||
|
||||
it('still skips a node_modules tree nobody asked for', async () => {
|
||||
installPackage('@acme/cubes-net', 'net:vpn');
|
||||
config({ cubeDirs: ['./'] });
|
||||
|
||||
const { cubes, errors } = await loadCubes();
|
||||
|
||||
expect(errors).toEqual([]);
|
||||
expect(cubes['net:vpn']).toBeUndefined();
|
||||
});
|
||||
|
||||
it('names the package and the directory when both claim one id', async () => {
|
||||
installPackage('@acme/cubes-net', 'clash');
|
||||
cube('local', 'export default { id: "clash", name: "Local" }');
|
||||
config({ cubeDirs: ['./'], cubePackages: ['@acme/cubes-net'] });
|
||||
|
||||
const { errors } = await loadCubes();
|
||||
|
||||
expect(errors).toHaveLength(1);
|
||||
expect(errors[0]).toMatch(/Duplicate cube id 'clash' from 2 sources/);
|
||||
// Labels are padded to a common width, so match the pair, not the gap.
|
||||
expect(errors[0]).toMatch(
|
||||
new RegExp(`^\\s+directory\\s+${path.join(tmpDir, 'local')}$`, 'm')
|
||||
);
|
||||
expect(errors[0]).toMatch(
|
||||
new RegExp(
|
||||
`^\\s+package @acme/cubes-net\\s+${path.join(tmpDir, 'node_modules/@acme/cubes-net/cubes/clash')}$`,
|
||||
'm'
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
it('aborts when a named package is not installed', async () => {
|
||||
config({ cubeDirs: [], cubePackages: ['@acme/missing'] });
|
||||
|
||||
const { cubes, errors } = await loadCubes();
|
||||
|
||||
expect(cubes).toEqual({});
|
||||
expect(errors[0]).toMatch(/'@acme\/missing' is not installed/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('getCube', () => {
|
||||
it('returns a single cube by id', async () => {
|
||||
cube('one', 'export default { id: "one", name: "One" }');
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
/**
|
||||
* Resolution of the packages named in `cubePackages`.
|
||||
*
|
||||
* Builds real `node_modules` trees under os.tmpdir() rather than faking the
|
||||
* filesystem: what is under test is Node's own resolution, including the
|
||||
* symlink layout pnpm produces, and neither survives a mock.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||
import { resolveCubePackages } from '../src/cubes/packages.js';
|
||||
|
||||
describe('resolveCubePackages', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
/** Writes a package into `<root>/node_modules/<name>`, cubes and all. */
|
||||
const install = (
|
||||
root: string,
|
||||
name: string,
|
||||
manifest: Record<string, unknown>,
|
||||
cubeDirs: string[] = ['cubes']
|
||||
) => {
|
||||
const dir = path.join(root, 'node_modules', name);
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name, ...manifest }));
|
||||
for (const cubeDir of cubeDirs) fs.mkdirSync(path.join(dir, cubeDir), { recursive: true });
|
||||
return dir;
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
tmpDir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'nopy-packages-')));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('resolves a scoped package to its cube directories', () => {
|
||||
const dir = install(tmpDir, '@acme/cubes-net', { nopy: { cubes: ['./cubes'] } });
|
||||
|
||||
const { packages, errors } = resolveCubePackages([{ spec: '@acme/cubes-net', from: tmpDir }]);
|
||||
|
||||
expect(errors).toEqual([]);
|
||||
expect(packages).toEqual([
|
||||
{ name: '@acme/cubes-net', root: dir, dirs: [path.join(dir, 'cubes')] },
|
||||
]);
|
||||
});
|
||||
|
||||
it('resolves an unscoped package and every directory it declares', () => {
|
||||
const dir = install(tmpDir, 'cubes-net', { nopy: { cubes: ['./cubes', './extra'] } }, [
|
||||
'cubes',
|
||||
'extra',
|
||||
]);
|
||||
|
||||
const { packages, errors } = resolveCubePackages([{ spec: 'cubes-net', from: tmpDir }]);
|
||||
|
||||
expect(errors).toEqual([]);
|
||||
expect(packages[0].dirs).toEqual([path.join(dir, 'cubes'), path.join(dir, 'extra')]);
|
||||
});
|
||||
|
||||
it('resolves through a symlinked package directory, as pnpm installs it', () => {
|
||||
// pnpm puts the real package under .pnpm and symlinks it into place, which
|
||||
// is why the resolver reads package.json instead of scanning node_modules.
|
||||
const store = path.join(tmpDir, 'store', 'cubes-net');
|
||||
fs.mkdirSync(path.join(store, 'cubes'), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(store, 'package.json'),
|
||||
JSON.stringify({ name: 'cubes-net', nopy: { cubes: ['./cubes'] } })
|
||||
);
|
||||
fs.mkdirSync(path.join(tmpDir, 'node_modules'), { recursive: true });
|
||||
fs.symlinkSync(store, path.join(tmpDir, 'node_modules', 'cubes-net'));
|
||||
|
||||
const { packages, errors } = resolveCubePackages([{ spec: 'cubes-net', from: tmpDir }]);
|
||||
|
||||
expect(errors).toEqual([]);
|
||||
expect(packages[0].dirs).toEqual([path.join(tmpDir, 'node_modules', 'cubes-net', 'cubes')]);
|
||||
});
|
||||
|
||||
it('resolves from the declaring config directory, not the working directory', () => {
|
||||
// The package is installed next to the config that names it. Nothing at the
|
||||
// process cwd can see it, which is the case a package named in
|
||||
// ~/.nopyrc.json always hits.
|
||||
const elsewhere = path.join(tmpDir, 'elsewhere');
|
||||
fs.mkdirSync(elsewhere, { recursive: true });
|
||||
install(elsewhere, 'cubes-net', { nopy: { cubes: ['./cubes'] } });
|
||||
|
||||
expect(resolveCubePackages([{ spec: 'cubes-net', from: elsewhere }]).errors).toEqual([]);
|
||||
expect(resolveCubePackages([{ spec: 'cubes-net', from: tmpDir }]).errors).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('reports a package that is not installed', () => {
|
||||
const { packages, errors } = resolveCubePackages([{ spec: '@acme/missing', from: tmpDir }]);
|
||||
|
||||
expect(packages).toEqual([]);
|
||||
expect(errors[0]).toMatch(/'@acme\/missing' is not installed/);
|
||||
expect(errors[0]).toContain(tmpDir);
|
||||
});
|
||||
|
||||
it('reports a package.json that cannot be parsed', () => {
|
||||
const dir = path.join(tmpDir, 'node_modules', 'broken');
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
fs.writeFileSync(path.join(dir, 'package.json'), '{ not json');
|
||||
|
||||
const { errors } = resolveCubePackages([{ spec: 'broken', from: tmpDir }]);
|
||||
|
||||
expect(errors[0]).toMatch(/cannot read/);
|
||||
});
|
||||
|
||||
it('reports a package that declares no cubes', () => {
|
||||
install(tmpDir, 'plain', {});
|
||||
install(tmpDir, 'empty', { nopy: { cubes: [] } });
|
||||
install(tmpDir, 'wrong-type', { nopy: { cubes: 'cubes' } });
|
||||
install(tmpDir, 'not-strings', { nopy: { cubes: [1] } });
|
||||
|
||||
const { packages, errors } = resolveCubePackages(
|
||||
['plain', 'empty', 'wrong-type', 'not-strings'].map((spec) => ({ spec, from: tmpDir }))
|
||||
);
|
||||
|
||||
expect(packages).toEqual([]);
|
||||
expect(errors).toHaveLength(4);
|
||||
for (const error of errors) expect(error).toMatch(/declares no cubes/);
|
||||
});
|
||||
|
||||
it('reports a cube directory that does not exist', () => {
|
||||
install(tmpDir, 'cubes-net', { nopy: { cubes: ['./nope'] } });
|
||||
|
||||
const { packages, errors } = resolveCubePackages([{ spec: 'cubes-net', from: tmpDir }]);
|
||||
|
||||
expect(packages).toEqual([]);
|
||||
expect(errors[0]).toMatch(/'\.\/nope' does not exist/);
|
||||
});
|
||||
|
||||
it('reports a cube directory that points outside the package', () => {
|
||||
install(tmpDir, 'cubes-net', { nopy: { cubes: ['../../..'] } });
|
||||
|
||||
const { errors } = resolveCubePackages([{ spec: 'cubes-net', from: tmpDir }]);
|
||||
|
||||
expect(errors[0]).toMatch(/points outside the package/);
|
||||
});
|
||||
|
||||
it('keeps the directories that are valid when a sibling entry is not', () => {
|
||||
const dir = install(tmpDir, 'cubes-net', { nopy: { cubes: ['./cubes', './nope'] } });
|
||||
|
||||
const { packages, errors } = resolveCubePackages([{ spec: 'cubes-net', from: tmpDir }]);
|
||||
|
||||
expect(errors).toHaveLength(1);
|
||||
expect(packages[0].dirs).toEqual([path.join(dir, 'cubes')]);
|
||||
});
|
||||
|
||||
it('resolves a package named by two configs once, from the more specific one', () => {
|
||||
// Merge order is root-first, so the last ref came from the config nearest
|
||||
// the working directory — and only that one is guaranteed to resolve.
|
||||
const child = path.join(tmpDir, 'child');
|
||||
fs.mkdirSync(child, { recursive: true });
|
||||
const dir = install(child, 'cubes-net', { nopy: { cubes: ['./cubes'] } });
|
||||
|
||||
const { packages, errors } = resolveCubePackages([
|
||||
{ spec: 'cubes-net', from: path.join(tmpDir, 'nowhere') },
|
||||
{ spec: 'cubes-net', from: child },
|
||||
]);
|
||||
|
||||
expect(errors).toEqual([]);
|
||||
expect(packages).toHaveLength(1);
|
||||
expect(packages[0].root).toBe(dir);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,107 @@
|
||||
/**
|
||||
* Tests for the manifest resolve hook.
|
||||
*
|
||||
* Every case runs in a child `node` process. Vitest resolves a dynamic import
|
||||
* through vite, which finds `zod` from the project root whether or not the hook
|
||||
* is installed — so a test run inside the worker passes either way and proves
|
||||
* nothing. Only real Node resolution can tell the two apart.
|
||||
*/
|
||||
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||
|
||||
const CUBES_SRC = fileURLToPath(new URL('../src/cubes', import.meta.url));
|
||||
const HOOK = pathToFileURL(path.join(CUBES_SRC, 'resolve-hook.mjs')).href;
|
||||
// Only ever handed to createRequire, which wants a path inside the package and
|
||||
// never opens it. This is the same URL loader.ts registers with.
|
||||
const FROM = pathToFileURL(path.join(CUBES_SRC, 'loader.ts')).href;
|
||||
|
||||
describe('resolve-hook', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
/** Runs `manifest.mjs` in a fresh Node process and returns its default export. */
|
||||
const importManifest = (source: string, { withHook } = { withHook: true }) => {
|
||||
fs.writeFileSync(path.join(tmpDir, 'manifest.mjs'), source);
|
||||
const manifest = pathToFileURL(path.join(tmpDir, 'manifest.mjs')).href;
|
||||
|
||||
const script = [
|
||||
withHook ? "import module from 'node:module';" : '',
|
||||
withHook
|
||||
? `module.register(${JSON.stringify(HOOK)}, ${JSON.stringify(FROM)}, ` +
|
||||
`{ data: { from: ${JSON.stringify(FROM)} } });`
|
||||
: '',
|
||||
`const loaded = await import(${JSON.stringify(manifest)})`,
|
||||
' .then((m) => m.default, (err) => ({ failed: err.code ?? String(err) }));',
|
||||
'console.log(JSON.stringify(loaded));',
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join('\n');
|
||||
|
||||
return JSON.parse(
|
||||
execFileSync(process.execPath, ['--input-type=module', '-e', script], { encoding: 'utf-8' })
|
||||
);
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
// Under os.tmpdir() precisely because nothing above it links zod: this is a
|
||||
// cube in a directory the user pointed `cubeDirs` at, nothing more.
|
||||
tmpDir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'nopy-hook-')));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('cannot import zod from a bare directory without the hook', () => {
|
||||
const result = importManifest("import 'zod';\nexport default { ok: true };", {
|
||||
withHook: false,
|
||||
});
|
||||
|
||||
expect(result).toEqual({ failed: 'ERR_MODULE_NOT_FOUND' });
|
||||
});
|
||||
|
||||
it('resolves zod from the running CLI', () => {
|
||||
const result = importManifest(
|
||||
"import { z } from 'zod';\nexport default { ok: typeof z.object === 'function' };"
|
||||
);
|
||||
|
||||
expect(result).toEqual({ ok: true });
|
||||
});
|
||||
|
||||
it('resolves a subpath of a covered package', () => {
|
||||
const result = importManifest(
|
||||
"import pkg from '@bitsquare/nopy/package.json' with { type: 'json' };\n" +
|
||||
'export default { name: pkg.name };'
|
||||
);
|
||||
|
||||
expect(result).toEqual({ name: '@bitsquare/nopy' });
|
||||
});
|
||||
|
||||
it('leaves anything else to fail as it would have', () => {
|
||||
const result = importManifest("import 'no-such-package';\nexport default { ok: true };");
|
||||
|
||||
expect(result).toEqual({ failed: 'ERR_MODULE_NOT_FOUND' });
|
||||
});
|
||||
|
||||
it('prefers a copy the cube can already see', () => {
|
||||
// The whole point of trying normal resolution first: a consumer with its
|
||||
// own zod keeps it, so the hook can never introduce version skew.
|
||||
const stub = path.join(tmpDir, 'node_modules', 'zod');
|
||||
fs.mkdirSync(stub, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(stub, 'package.json'),
|
||||
JSON.stringify({ name: 'zod', version: '0.0.0', type: 'module', main: 'index.js' })
|
||||
);
|
||||
fs.writeFileSync(path.join(stub, 'index.js'), "export const z = { from: 'the cube' };");
|
||||
|
||||
const result = importManifest(
|
||||
"import { z } from 'zod';\nexport default { from: z.from ?? 'the CLI' };"
|
||||
);
|
||||
|
||||
expect(result).toEqual({ from: 'the cube' });
|
||||
});
|
||||
});
|
||||
@@ -1,111 +0,0 @@
|
||||
/**
|
||||
* Tests for the Cube runtime wrapper: default extraction and the required-key
|
||||
* check that `--use-defaults` relies on.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { z } from 'zod';
|
||||
import { Cube, Manifest } from '../src/cubes/types.js';
|
||||
import { foreignZodSchema } from './helpers/foreign-zod.js';
|
||||
|
||||
const cube = (schema: z.ZodObject<any>) =>
|
||||
new Cube(Manifest.create({ id: 'c', name: 'C', schema }), '/cubes/c', 'deploy.py');
|
||||
|
||||
describe('Cube.getDefaults', () => {
|
||||
it('resolves every default when the whole schema parses', () => {
|
||||
const c = cube(
|
||||
z.object({
|
||||
PORT: z.number().default(8080),
|
||||
NAME: z.string().default('svc'),
|
||||
})
|
||||
);
|
||||
|
||||
expect(c.getDefaults()).toEqual({ PORT: 8080, NAME: 'svc' });
|
||||
});
|
||||
|
||||
it('keeps the declared defaults when one field has none', () => {
|
||||
const c = cube(
|
||||
z.object({
|
||||
SSID: z.string(),
|
||||
PRIORITY: z.number().default(10),
|
||||
HIDDEN: z.boolean().default(false),
|
||||
})
|
||||
);
|
||||
|
||||
expect(c.getDefaults()).toEqual({ PRIORITY: 10, HIDDEN: false });
|
||||
});
|
||||
|
||||
it('unwraps a default sitting under optional or nullable', () => {
|
||||
const c = cube(
|
||||
z.object({
|
||||
REQUIRED: z.string(),
|
||||
A: z.number().default(1).optional(),
|
||||
B: z.number().default(2).nullable(),
|
||||
C: z.number().optional().default(3),
|
||||
})
|
||||
);
|
||||
|
||||
expect(c.getDefaults()).toEqual({ A: 1, B: 2, C: 3 });
|
||||
});
|
||||
|
||||
it('evaluates a lazily declared default', () => {
|
||||
const c = cube(
|
||||
z.object({ REQUIRED: z.string(), TOKEN: z.string().default(() => 'generated') })
|
||||
);
|
||||
|
||||
expect(c.getDefaults()).toEqual({ TOKEN: 'generated' });
|
||||
});
|
||||
|
||||
it('omits an optional field that declares no default', () => {
|
||||
const c = cube(z.object({ REQUIRED: z.string(), MAYBE: z.string().optional() }));
|
||||
|
||||
expect(c.getDefaults()).toEqual({});
|
||||
});
|
||||
|
||||
it('returns an empty object for an empty schema', () => {
|
||||
expect(cube(z.object({})).getDefaults()).toEqual({});
|
||||
});
|
||||
|
||||
it('reads defaults off a schema built by a different copy of zod', () => {
|
||||
// The per-field fallback reads zod's internals directly. Under `instanceof`
|
||||
// a foreign schema yields no defaults at all, without erroring.
|
||||
const c = cube(
|
||||
foreignZodSchema(
|
||||
z.object({
|
||||
REQUIRED: z.string(),
|
||||
PRIORITY: z.number().default(10),
|
||||
NESTED: z.number().default(2).optional(),
|
||||
})
|
||||
)
|
||||
);
|
||||
|
||||
expect(c.getDefaults()).toEqual({ PRIORITY: 10, NESTED: 2 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('Cube.requiredKeys', () => {
|
||||
it('lists the fields with neither a default nor optionality', () => {
|
||||
const c = cube(
|
||||
z.object({
|
||||
SSID: z.string(),
|
||||
PASSWORD: z.string(),
|
||||
PRIORITY: z.number().default(10),
|
||||
NOTE: z.string().optional(),
|
||||
})
|
||||
);
|
||||
|
||||
expect(c.requiredKeys()).toEqual(['SSID', 'PASSWORD']);
|
||||
});
|
||||
|
||||
it('treats a nullable field without a default as required', () => {
|
||||
const c = cube(z.object({ MAYBE: z.string().nullable() }));
|
||||
|
||||
expect(c.requiredKeys()).toEqual(['MAYBE']);
|
||||
});
|
||||
|
||||
it('is empty when every field can fill itself in', () => {
|
||||
const c = cube(z.object({ A: z.string().default('a'), B: z.string().optional() }));
|
||||
|
||||
expect(c.requiredKeys()).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -1,44 +0,0 @@
|
||||
/**
|
||||
* Tests for cubes/utils module
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { uniqid } from '../src/cubes/utils.js';
|
||||
|
||||
describe('uniqid', () => {
|
||||
it('generates string of default length (5)', () => {
|
||||
const id = uniqid();
|
||||
expect(id).toHaveLength(5);
|
||||
});
|
||||
|
||||
it('generates string of specified length', () => {
|
||||
expect(uniqid(10)).toHaveLength(10);
|
||||
expect(uniqid(3)).toHaveLength(3);
|
||||
expect(uniqid(20)).toHaveLength(20);
|
||||
});
|
||||
|
||||
it('generates alphanumeric characters only', () => {
|
||||
const id = uniqid(100);
|
||||
expect(id).toMatch(/^[A-Za-z0-9]+$/);
|
||||
});
|
||||
|
||||
it('generates different values on subsequent calls', () => {
|
||||
const ids = new Set<string>();
|
||||
for (let i = 0; i < 100; i++) {
|
||||
ids.add(uniqid(10));
|
||||
}
|
||||
// Should have many unique values (some collisions possible but unlikely)
|
||||
expect(ids.size).toBeGreaterThan(90);
|
||||
});
|
||||
|
||||
it('handles edge case of length 1', () => {
|
||||
const id = uniqid(1);
|
||||
expect(id).toHaveLength(1);
|
||||
expect(id).toMatch(/^[A-Za-z0-9]$/);
|
||||
});
|
||||
|
||||
it('handles edge case of length 0', () => {
|
||||
const id = uniqid(0);
|
||||
expect(id).toBe('');
|
||||
});
|
||||
});
|
||||
@@ -6,6 +6,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
type DeployCall,
|
||||
type ExecutionResult,
|
||||
maskCommand,
|
||||
outputExecutionPlan,
|
||||
summarizeResults,
|
||||
} from '../src/nopy.executor.js';
|
||||
@@ -136,17 +137,33 @@ describe('outputExecutionPlan', () => {
|
||||
expect(parsed.plan[0].host).toBe('host1');
|
||||
});
|
||||
|
||||
it('masks password variables in text output', () => {
|
||||
it('masks variables the manifest declared secret', () => {
|
||||
const call: DeployCall = {
|
||||
...createTestCall('cube-a', 'host1'),
|
||||
env: { PASSWORD: 'secret', OTHER: 'visible' },
|
||||
command: ['pyinfra', 'host1', '-y', '--data "PASSWORD=hunter2"', '--data "OTHER=visible"'],
|
||||
env: { PASSWORD: 'hunter2', OTHER: 'visible' },
|
||||
secrets: ['PASSWORD'],
|
||||
};
|
||||
|
||||
outputExecutionPlan([call]);
|
||||
|
||||
const output = consoleLogSpy.mock.calls.map((c) => c[0]).join('\n');
|
||||
expect(output).toContain('********');
|
||||
expect(output).not.toContain('secret');
|
||||
// Both the variable list and the command line above it — the command used
|
||||
// to be printed unmasked, which defeated the masking entirely.
|
||||
expect(output).not.toContain('hunter2');
|
||||
expect(output).toContain('visible');
|
||||
});
|
||||
|
||||
it('leaves a password-looking variable alone when the manifest says nothing', () => {
|
||||
const call: DeployCall = {
|
||||
...createTestCall('cube-a', 'host1'),
|
||||
env: { PASSWORD: 'visible' },
|
||||
};
|
||||
|
||||
outputExecutionPlan([call]);
|
||||
|
||||
const output = consoleLogSpy.mock.calls.map((c) => c[0]).join('\n');
|
||||
expect(output).toContain('visible');
|
||||
});
|
||||
|
||||
@@ -173,3 +190,49 @@ describe('outputExecutionPlan', () => {
|
||||
expect(output).toContain('Total: 3');
|
||||
});
|
||||
});
|
||||
|
||||
describe('maskCommand', () => {
|
||||
const call = (command: string[], secrets?: string[]): DeployCall => ({
|
||||
...createTestCall('cube-a', 'host1'),
|
||||
command,
|
||||
secrets,
|
||||
});
|
||||
|
||||
it('replaces the value of a declared secret', () => {
|
||||
const masked = maskCommand(
|
||||
call(['pyinfra', 'host1', '--data "PASSWORD=hunter2"'], ['PASSWORD'])
|
||||
);
|
||||
|
||||
expect(masked).toBe('pyinfra host1 --data "PASSWORD=********"');
|
||||
});
|
||||
|
||||
it('leaves other data alone', () => {
|
||||
const masked = maskCommand(
|
||||
call(['--data "SSID=home"', '--data "PASSWORD=hunter2"'], ['PASSWORD'])
|
||||
);
|
||||
|
||||
expect(masked).toBe('--data "SSID=home" --data "PASSWORD=********"');
|
||||
});
|
||||
|
||||
it('masks a value containing spaces up to the closing quote', () => {
|
||||
const masked = maskCommand(call(['--data "PASSWORD=two words"', '--chdir /x'], ['PASSWORD']));
|
||||
|
||||
expect(masked).toBe('--data "PASSWORD=********" --chdir /x');
|
||||
});
|
||||
|
||||
it('masks an empty secret value', () => {
|
||||
expect(maskCommand(call(['--data "PASSWORD="'], ['PASSWORD']))).toBe(
|
||||
'--data "PASSWORD=********"'
|
||||
);
|
||||
});
|
||||
|
||||
it('masks the ssh password whether or not the cube declares secrets', () => {
|
||||
const masked = maskCommand(call(['pyinfra', 'host1', '--user bob --password s3cr3t', '-y']));
|
||||
|
||||
expect(masked).toBe('pyinfra host1 --user bob --password ******** -y');
|
||||
});
|
||||
|
||||
it('returns the command untouched when there is nothing to hide', () => {
|
||||
expect(maskCommand(call(['pyinfra', 'host1', '-y']))).toBe('pyinfra host1 -y');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2,10 +2,10 @@
|
||||
* Tests for cube hooks using BuildContext
|
||||
*/
|
||||
|
||||
import { Cube, Manifest } from '@bitsquare/nopy-cube';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { z } from 'zod';
|
||||
import { BuildContext } from '../src/cubes/dependencies.js';
|
||||
import { Cube, Manifest } from '../src/cubes/types.js';
|
||||
import { Variables } from '../src/nopy.common.js';
|
||||
|
||||
describe('Cube Hooks', () => {
|
||||
|
||||
@@ -93,7 +93,7 @@ beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
|
||||
state.config = { hosts: ['web-1'], cubeDirs: [], env: {} };
|
||||
state.config = { hosts: ['web-1'], cubeDirs: [], cubePackages: [], env: {} };
|
||||
state.loadResult = { cubes: { 'cube-a': {} }, errors: [] };
|
||||
state.deployCalls = [call('cube-a')];
|
||||
state.cubeSessions = [{ key: 'cube-a', variables: {} }];
|
||||
@@ -174,6 +174,7 @@ describe('nopy', () => {
|
||||
state.config = {
|
||||
hosts: ['web-1'],
|
||||
cubeDirs: ['/cubes'],
|
||||
cubePackages: [{ spec: '@acme/cubes-net', from: '/project' }],
|
||||
env: { TOKEN: 'secret', EMPTY: '' },
|
||||
};
|
||||
|
||||
@@ -183,6 +184,8 @@ describe('nopy', () => {
|
||||
expect(text).toContain('Configuration');
|
||||
expect(text).toContain('Hosts:');
|
||||
expect(text).toContain('Cube dirs:');
|
||||
// Named by package, not by wherever it resolved to on disk.
|
||||
expect(text).toContain('Cube pkgs: @acme/cubes-net');
|
||||
expect(text).toContain('continue-on-error');
|
||||
// Values are never echoed, only their presence.
|
||||
expect(text).toContain('TOKEN: <VALUE>');
|
||||
@@ -191,7 +194,7 @@ describe('nopy', () => {
|
||||
});
|
||||
|
||||
it('omits empty sections', async () => {
|
||||
state.config = { hosts: [], cubeDirs: [], env: {} };
|
||||
state.config = { hosts: [], cubeDirs: [], cubePackages: [], env: {} };
|
||||
|
||||
await nopy();
|
||||
|
||||
@@ -199,6 +202,7 @@ describe('nopy', () => {
|
||||
expect(text).toContain('Configuration');
|
||||
expect(text).not.toContain('Hosts:');
|
||||
expect(text).not.toContain('Cube dirs:');
|
||||
expect(text).not.toContain('Cube pkgs:');
|
||||
expect(text).not.toContain('Env vars:');
|
||||
});
|
||||
|
||||
|
||||
@@ -37,7 +37,7 @@ vi.mock('enquirer', () => ({
|
||||
},
|
||||
}));
|
||||
|
||||
import { Cube, Manifest } from '../src/cubes/types.js';
|
||||
import { Cube, Manifest } from '@bitsquare/nopy-cube';
|
||||
import { Variables } from '../src/nopy.common.js';
|
||||
import {
|
||||
AuthSelection,
|
||||
@@ -227,7 +227,7 @@ describe('VariableAssignment', () => {
|
||||
|
||||
it('does nothing when every default is already supplied as a param', async () => {
|
||||
const variables = new Variables();
|
||||
variables.assign('svc', 'params', { port: 1, enabled: true, name: 'x' });
|
||||
variables.assign('svc', 'param', { port: 1, enabled: true, name: 'x' });
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||
|
||||
@@ -240,25 +240,30 @@ describe('VariableAssignment', () => {
|
||||
expect(formRun).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('only asks about the variables still missing', async () => {
|
||||
it('leaves out a key a dependency already supplied', async () => {
|
||||
const variables = new Variables();
|
||||
variables.assign('svc', 'params', { port: 9090 });
|
||||
variables.assign('svc', 'param', { port: 9090 });
|
||||
formRun.mockResolvedValue({});
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||
|
||||
expect(formRun).toHaveBeenCalled();
|
||||
expect(variables.get('svc', 'prompts')).toEqual({});
|
||||
expect(formChoices().map((c) => c.name)).toEqual(['enabled', 'name']);
|
||||
});
|
||||
|
||||
it('asks about a field that declares no default, with an empty initial value', async () => {
|
||||
const required = z.object({
|
||||
SSID: z.string().describe('Network name'),
|
||||
PRIORITY: z.number().default(10),
|
||||
});
|
||||
const wifi = cube(
|
||||
'wifi',
|
||||
'WiFi',
|
||||
z.object({
|
||||
SSID: z.string().describe('Network name'),
|
||||
PRIORITY: z.number().default(10),
|
||||
})
|
||||
);
|
||||
const variables = new Variables();
|
||||
variables.assign('wifi', 'default', wifi.getDefaults());
|
||||
formRun.mockResolvedValue({});
|
||||
|
||||
await VariableAssignment(cube('wifi', 'WiFi', required), new Variables());
|
||||
await VariableAssignment(wifi, variables);
|
||||
|
||||
expect(formChoices()).toEqual([
|
||||
{ name: 'SSID', message: 'Network name', initial: '' },
|
||||
@@ -267,21 +272,34 @@ describe('VariableAssignment', () => {
|
||||
});
|
||||
|
||||
it('offers the value the run would use, not the bare schema default', async () => {
|
||||
const svc = cube('svc', 'Service', schema);
|
||||
const variables = new Variables({ port: 2222 });
|
||||
variables.assign('svc', 'default', svc.getDefaults());
|
||||
formRun.mockResolvedValue({});
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||
await VariableAssignment(svc, variables);
|
||||
|
||||
expect(formChoices().find((c) => c.name === 'port')?.initial).toBe('2222');
|
||||
});
|
||||
|
||||
it('asks only about the given keys', async () => {
|
||||
const svc = cube('svc', 'Service', schema);
|
||||
const variables = new Variables();
|
||||
variables.assign('svc', 'default', svc.getDefaults());
|
||||
formRun.mockResolvedValue({});
|
||||
|
||||
await VariableAssignment(svc, variables, { keys: ['name'] });
|
||||
|
||||
expect(formChoices()).toEqual([{ name: 'name', message: 'name', initial: 'svc' }]);
|
||||
});
|
||||
|
||||
it('coerces answers using the schema and stores them under prompts', async () => {
|
||||
const variables = new Variables();
|
||||
formRun.mockResolvedValue({ port: '9090', enabled: 'true', name: 'api' });
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||
|
||||
expect(variables.get('svc', 'prompts')).toEqual({
|
||||
expect(variables.get('svc')).toEqual({
|
||||
port: 9090,
|
||||
enabled: true,
|
||||
name: 'api',
|
||||
@@ -294,8 +312,8 @@ describe('VariableAssignment', () => {
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||
|
||||
expect(variables.get('svc', 'prompts').port).toBe('not-a-number');
|
||||
expect(variables.get('svc', 'prompts').enabled).toBe(false);
|
||||
expect(variables.get('svc').port).toBe('not-a-number');
|
||||
expect(variables.get('svc').enabled).toBe(false);
|
||||
});
|
||||
|
||||
it('accepts yes and 1 as truthy booleans', async () => {
|
||||
@@ -304,7 +322,7 @@ describe('VariableAssignment', () => {
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||
|
||||
expect(variables.get('svc', 'prompts').enabled).toBe(true);
|
||||
expect(variables.get('svc').enabled).toBe(true);
|
||||
});
|
||||
|
||||
it('unwraps optional and nullable schema types', async () => {
|
||||
@@ -318,7 +336,7 @@ describe('VariableAssignment', () => {
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', nullableSchema), variables);
|
||||
|
||||
expect(variables.get('svc', 'prompts')).toEqual({ maybe: null, opt: 7, given: 42 });
|
||||
expect(variables.get('svc')).toEqual({ maybe: null, opt: 7, given: 42 });
|
||||
});
|
||||
|
||||
it('treats an empty string as null for a nullable field', async () => {
|
||||
@@ -328,7 +346,7 @@ describe('VariableAssignment', () => {
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', nullableSchema), variables);
|
||||
|
||||
expect(variables.get('svc', 'prompts').maybe).toBe(null);
|
||||
expect(variables.get('svc').maybe).toBe(null);
|
||||
});
|
||||
|
||||
it('passes non-string answers through untouched', async () => {
|
||||
@@ -337,7 +355,7 @@ describe('VariableAssignment', () => {
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||
|
||||
expect(variables.get('svc', 'prompts').port).toBe(9090);
|
||||
expect(variables.get('svc').port).toBe(9090);
|
||||
});
|
||||
|
||||
it('keeps answers for keys the schema does not describe', async () => {
|
||||
@@ -346,7 +364,7 @@ describe('VariableAssignment', () => {
|
||||
|
||||
await VariableAssignment(cube('svc', 'Service', schema), variables);
|
||||
|
||||
expect(variables.get('svc', 'prompts').extra).toBe('kept');
|
||||
expect(variables.get('svc').extra).toBe('kept');
|
||||
});
|
||||
|
||||
it('assigns nothing when the user cancels the form', async () => {
|
||||
@@ -356,7 +374,7 @@ describe('VariableAssignment', () => {
|
||||
await expect(
|
||||
VariableAssignment(cube('svc', 'Service', schema), variables)
|
||||
).resolves.toBeUndefined();
|
||||
expect(variables.get('svc', 'prompts')).toEqual({});
|
||||
expect(variables.get('svc')).toEqual({});
|
||||
});
|
||||
|
||||
it('coerces against a schema built by a different copy of zod', async () => {
|
||||
@@ -380,6 +398,6 @@ describe('VariableAssignment', () => {
|
||||
variables
|
||||
);
|
||||
|
||||
expect(variables.get('svc', 'prompts')).toEqual({ port: 9090, enabled: true, maybe: null });
|
||||
expect(variables.get('svc')).toEqual({ port: 9090, enabled: true, maybe: null });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -8,12 +8,10 @@ import path from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||
import {
|
||||
createSession,
|
||||
filterInternalVariables,
|
||||
listSessions,
|
||||
loadSession,
|
||||
type NopySession,
|
||||
saveSession,
|
||||
separateEnvAndCubeVariables,
|
||||
} from '../src/nopy.session.js';
|
||||
|
||||
describe('createSession', () => {
|
||||
@@ -157,56 +155,3 @@ describe('listSessions', () => {
|
||||
expect(result[0].endsWith('test.session.mjs')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('filterInternalVariables', () => {
|
||||
it('removes customize key', () => {
|
||||
const input = { customize: true, VAR_A: 'a', VAR_B: 'b' };
|
||||
const result = filterInternalVariables(input);
|
||||
|
||||
expect(result).toEqual({ VAR_A: 'a', VAR_B: 'b' });
|
||||
expect('customize' in result).toBe(false);
|
||||
});
|
||||
|
||||
it('returns empty object for internal-only input', () => {
|
||||
const result = filterInternalVariables({ customize: true });
|
||||
expect(result).toEqual({});
|
||||
});
|
||||
|
||||
it('preserves all non-internal keys', () => {
|
||||
const input = { A: 1, B: 'two', C: true };
|
||||
const result = filterInternalVariables(input);
|
||||
expect(result).toEqual(input);
|
||||
});
|
||||
});
|
||||
|
||||
describe('separateEnvAndCubeVariables', () => {
|
||||
it('separates env variables from cube variables', () => {
|
||||
const allVars = { ENV_VAR: 'env', CUBE_VAR: 'cube' };
|
||||
const envVars = { ENV_VAR: 'original' };
|
||||
|
||||
const result = separateEnvAndCubeVariables(allVars, envVars);
|
||||
|
||||
expect(result.env).toEqual({ ENV_VAR: 'env' });
|
||||
expect(result.cubeVars).toEqual({ CUBE_VAR: 'cube' });
|
||||
});
|
||||
|
||||
it('handles all env variables', () => {
|
||||
const allVars = { A: 1, B: 2 };
|
||||
const envVars = { A: 0, B: 0 };
|
||||
|
||||
const result = separateEnvAndCubeVariables(allVars, envVars);
|
||||
|
||||
expect(result.env).toEqual({ A: 1, B: 2 });
|
||||
expect(result.cubeVars).toEqual({});
|
||||
});
|
||||
|
||||
it('handles all cube variables', () => {
|
||||
const allVars = { A: 1, B: 2 };
|
||||
const envVars = {};
|
||||
|
||||
const result = separateEnvAndCubeVariables(allVars, envVars);
|
||||
|
||||
expect(result.env).toEqual({});
|
||||
expect(result.cubeVars).toEqual({ A: 1, B: 2 });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -11,5 +11,5 @@
|
||||
},
|
||||
"include": ["src/**/*"],
|
||||
"exclude": ["coverage", "node_modules", "dist"],
|
||||
"references": []
|
||||
"references": [{ "path": "../nopy-cube" }]
|
||||
}
|
||||
|
||||
@@ -1,6 +1,15 @@
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { defineConfig } from 'vitest/config';
|
||||
|
||||
export default defineConfig({
|
||||
resolve: {
|
||||
alias: {
|
||||
// The workspace link points at nopy-cube's `dist`, which only exists after
|
||||
// a build. Tests read its source instead, so the gate does not depend on
|
||||
// build ordering and never runs against a stale artifact.
|
||||
'@bitsquare/nopy-cube': fileURLToPath(new URL('../nopy-cube/src/index.ts', import.meta.url)),
|
||||
},
|
||||
},
|
||||
test: {
|
||||
globals: true,
|
||||
environment: 'node',
|
||||
@@ -12,6 +21,8 @@ export default defineConfig({
|
||||
include: ['src/**/*.ts'],
|
||||
exclude: [
|
||||
'src/**/*.test.ts',
|
||||
// Reached through the resolve.alias above; it has its own gate.
|
||||
'**/nopy-cube/**',
|
||||
// Pure re-export barrels: no logic to cover.
|
||||
'src/index.ts',
|
||||
'src/cubes/index.ts',
|
||||
|
||||
Reference in New Issue
Block a user