mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-06 07:29:42 +02:00
fix(cases): tell an unreachable path from an absent one, scope the stall cap
The bounded path probe answered "absent" both when a path did not exist and
when it simply did not answer, so a stalled linked case 404'd and the Run
button scaffolded a stray local case over it, and two stalled paths anywhere
made every unrelated path read as absent (hooks skipped, statusLine
overridden, the clone warning lost).
- probePath()/probePathKind() are tri-state: present (or directory/file),
absent (ENOENT/ENOTDIR only) and unknown (timeout, other errors, refusal).
boundedPathExists() stays as the display-only boolean.
- A stalled path takes only its own mount out of probing (deepest mount
point from /proc/self/mounts, never /; just the path itself when there is
no mount table). Unrelated paths keep probing. The process-wide cap is a
backstop that answers unknown, and a single-path user request can probe
past it ({ pastCap: true }), still bounded and still recorded as stalled.
One console.warn when a path first stalls and one when the cap engages.
- GET /api/cases/:name keeps NOT_FOUND for definite absence only. An
unreachable linked case answers with its registered path and
unreachable: true; a local one answers OPERATION_FAILED. runClaude and
runShell create a case only on errorCode NOT_FOUND. The case list keeps an
unreachable linked case, marked unreachable, instead of dropping it, and
fix-plan reports an unreadable plan as an error, not "no plan".
- applyWorkspaceHooks and the statusLine helpers skip only a workspace that
is absent or on the stalled mount; a capacity refusal no longer stops
hooks being installed elsewhere, and an unreadable settings file never
lets the exporter override a user's own statusLine.
- The clone flow's repo-settings warning is back on its synchronous check,
and stripCaseEnvKeys uses pathExistsForWrite.
- POST /api/sessions (workingDir) and POST /api/quick-start (case folder)
probe with the bounded probe instead of statSync/existsSync. Missing and
non-directory keep INVALID_INPUT; unknown is OPERATION_FAILED, and
quick-start never scaffolds over a folder that did not answer.
- PATH_PROBE_TIMEOUT_MS and MAX_STALLED_PATH_PROBES move to
src/config/path-probe.ts, overridable via CODEMAN_PATH_PROBE_TIMEOUT_MS
(default 1500) and CODEMAN_PATH_PROBE_MAX_STALLED (default 3), and are
documented in the Settings Reference.
- The probe is exported from the utils barrel and imported from there.
This commit is contained in:
Aamer Akhter
+150
-42
@@ -8,59 +8,145 @@
|
||||
* one of libuv's few threadpool workers, which every other `fs`, `dns.lookup`
|
||||
* and `crypto` call in the process shares.
|
||||
*
|
||||
* `boundedPathExists()` therefore:
|
||||
* - probes asynchronously and answers `false` after `PROBE_TIMEOUT_MS`, so a
|
||||
* request never waits on a dead mount for longer than that;
|
||||
* - shares one in-flight probe per path, and keeps answering `false` for a path
|
||||
* whose probe timed out until that probe finally settles (so a dead path is
|
||||
* not re-probed on every request, and is re-probed once the mount recovers);
|
||||
* - stops starting new probes once `MAX_STALLED_PROBES` timed-out probes are
|
||||
* still pending, so stalled stats cannot drain the threadpool. Probes that are
|
||||
* merely in flight do not count, so concurrent healthy probes never get a
|
||||
* false negative.
|
||||
* The probe therefore answers one of THREE things, never two:
|
||||
* - `'present'` / `'absent'`: the filesystem answered (ENOENT and ENOTDIR are
|
||||
* the only errors that mean absent);
|
||||
* - `'unknown'`: it did not answer in `PATH_PROBE_TIMEOUT_MS`, it answered with
|
||||
* some other error (EIO from a soft mount that gave up, EACCES), or the probe
|
||||
* was refused (below). "Unknown" is NOT "absent": a caller that would create,
|
||||
* scaffold or 404 on absence must not do so on unknown.
|
||||
*
|
||||
* Like `existsSync`, it follows symlinks and reports any error as "absent". It
|
||||
* is meant for READ decisions (is it there, show it or not). A writer that must
|
||||
* tell "missing" apart from "unreachable" should not treat its `false` as
|
||||
* permission to create or overwrite anything.
|
||||
* And it keeps a dead mount from draining the threadpool:
|
||||
* - one in-flight probe per path, shared by concurrent callers;
|
||||
* - a path whose probe timed out is "stalled" until that stat finally settles.
|
||||
* Paths NEAR a stalled one are answered "unknown" without a new stat, so one
|
||||
* dead mount costs one worker, not one per case and file on it. "Near" means on
|
||||
* the same mount: under the deepest mount point holding the stalled path, read
|
||||
* from `/proc/self/mounts` (procfs, which never waits on the dead filesystem).
|
||||
* Where that table is unavailable (not Linux), or the deepest mount is `/`, it
|
||||
* narrows to the stalled path and everything under it. Unrelated paths are
|
||||
* probed normally;
|
||||
* - once `MAX_STALLED_PATH_PROBES` stalled stats are pending, new probes are
|
||||
* refused process-wide (answered "unknown"), since each would risk another
|
||||
* worker. Probes merely in flight do not count, so concurrent healthy probes
|
||||
* never get refused. A caller acting on ONE path at a user's explicit request
|
||||
* (opening a case, starting a session in it) may pass `{ pastCap: true }`: its
|
||||
* probe is still bounded and still recorded as stalled if it hangs (so a dead
|
||||
* path costs at most one worker however often it is retried), but it is not
|
||||
* refused just because unrelated mounts are dead. Bulk scans (the case list)
|
||||
* and per-spawn helpers keep the cap.
|
||||
*
|
||||
* Both events are logged once (`console.warn`): a path's first stall, and the
|
||||
* cap engaging, so "my case vanished" and "hooks stopped firing" leave a trace.
|
||||
*
|
||||
* Writers should not use this at all: a writer that must tell "missing" apart
|
||||
* from "unreachable" wants an ENOENT-aware async `lstat` (see
|
||||
* `pathExistsForWrite` in hooks-config.ts).
|
||||
*
|
||||
* @module utils/bounded-path-probe
|
||||
*/
|
||||
|
||||
import { readFileSync } from 'node:fs';
|
||||
import fs from 'node:fs/promises';
|
||||
import { resolve, sep } from 'node:path';
|
||||
import { MAX_STALLED_PATH_PROBES, PATH_PROBE_TIMEOUT_MS } from '../config/path-probe.js';
|
||||
|
||||
/** How long a caller waits for one probe before treating the path as absent. */
|
||||
export const PROBE_TIMEOUT_MS = 1_500;
|
||||
/** Timed-out probes allowed to remain pending before new probes are refused. */
|
||||
export const MAX_STALLED_PROBES = 2;
|
||||
/** What a probe could establish about a path. */
|
||||
export type PathProbeState = 'present' | 'absent' | 'unknown';
|
||||
/** Like {@link PathProbeState}, with "present" split by whether it is a directory. */
|
||||
export type PathProbeKind = 'directory' | 'file' | 'absent' | 'unknown';
|
||||
|
||||
const inFlight = new Map<string, Promise<boolean>>();
|
||||
const stalled = new Set<string>();
|
||||
const inFlight = new Map<string, Promise<PathProbeKind>>();
|
||||
/** Stalled path -> the directory whose subtree is answered "unknown" while it stays stalled. */
|
||||
const stalled = new Map<string, string>();
|
||||
let capWarned = false;
|
||||
|
||||
async function statExists(path: string): Promise<boolean> {
|
||||
async function statKind(path: string): Promise<PathProbeKind> {
|
||||
try {
|
||||
await fs.stat(path);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
return (await fs.stat(path)).isDirectory() ? 'directory' : 'file';
|
||||
} catch (err) {
|
||||
const code = (err as NodeJS.ErrnoException)?.code;
|
||||
return code === 'ENOENT' || code === 'ENOTDIR' ? 'absent' : 'unknown';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve whether `path` exists without letting an unresponsive filesystem
|
||||
* block the caller for longer than `PROBE_TIMEOUT_MS`.
|
||||
*/
|
||||
export async function boundedPathExists(path: string): Promise<boolean> {
|
||||
if (stalled.has(path)) return false;
|
||||
function isWithin(path: string, root: string): boolean {
|
||||
if (path === root) return true;
|
||||
return path.startsWith(root.endsWith(sep) ? root : root + sep);
|
||||
}
|
||||
|
||||
let probe = inFlight.get(path);
|
||||
/** Deepest mount point holding `abs`, from the kernel's mount table; undefined when unreadable. */
|
||||
function mountPointOf(abs: string): string | undefined {
|
||||
let table: string;
|
||||
try {
|
||||
table = readFileSync('/proc/self/mounts', 'utf-8');
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
let best: string | undefined;
|
||||
for (const line of table.split('\n')) {
|
||||
const field = line.split(' ')[1];
|
||||
if (!field) continue;
|
||||
// The table octal-escapes space, tab, newline and backslash in mount points.
|
||||
const mountPoint = field.replace(/\\([0-7]{3})/g, (_m, oct: string) => String.fromCharCode(parseInt(oct, 8)));
|
||||
if (isWithin(abs, mountPoint) && (!best || mountPoint.length > best.length)) best = mountPoint;
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
/** The subtree a stalled path takes down with it: its mount, else just itself (see the module comment). */
|
||||
function stallScope(abs: string): string {
|
||||
const mountPoint = mountPointOf(abs);
|
||||
return mountPoint && mountPoint !== '/' ? mountPoint : abs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether `path` is near a path whose probe is still stalled (see the module
|
||||
* comment), i.e. whether the probe would answer "unknown" for it without a stat.
|
||||
* Lets a caller tell "this workspace sits on the dead mount" apart from "the
|
||||
* probe was refused for capacity".
|
||||
*/
|
||||
export function isNearStalledPath(path: string): boolean {
|
||||
const abs = resolve(path);
|
||||
for (const scope of stalled.values()) {
|
||||
if (isWithin(abs, scope)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Options for {@link probePathKind} / {@link probePath}. */
|
||||
export interface PathProbeOptions {
|
||||
/** Probe even while the stall cap is engaged (see the module comment). */
|
||||
pastCap?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Probe `path` without letting an unresponsive filesystem block the caller for
|
||||
* longer than `PATH_PROBE_TIMEOUT_MS`. Follows symlinks, like `stat()`.
|
||||
*/
|
||||
export async function probePathKind(path: string, options: PathProbeOptions = {}): Promise<PathProbeKind> {
|
||||
const abs = resolve(path);
|
||||
if (isNearStalledPath(abs)) return 'unknown';
|
||||
|
||||
let probe = inFlight.get(abs);
|
||||
if (!probe) {
|
||||
if (stalled.size >= MAX_STALLED_PROBES) return false;
|
||||
probe = statExists(path);
|
||||
inFlight.set(path, probe);
|
||||
void probe.finally(() => {
|
||||
inFlight.delete(path);
|
||||
stalled.delete(path);
|
||||
if (stalled.size >= MAX_STALLED_PATH_PROBES && !options.pastCap) {
|
||||
if (!capWarned) {
|
||||
capWarned = true;
|
||||
console.warn(
|
||||
`[path-probe] ${stalled.size} path probes are stalled on unresponsive filesystems; ` +
|
||||
'not starting new ones until one answers (paths read as unknown meanwhile)'
|
||||
);
|
||||
}
|
||||
return 'unknown';
|
||||
}
|
||||
probe = statKind(abs);
|
||||
const started = probe;
|
||||
inFlight.set(abs, started);
|
||||
void started.finally(() => {
|
||||
inFlight.delete(abs);
|
||||
stalled.delete(abs);
|
||||
if (stalled.size < MAX_STALLED_PATH_PROBES) capWarned = false;
|
||||
});
|
||||
}
|
||||
|
||||
@@ -68,11 +154,17 @@ export async function boundedPathExists(path: string): Promise<boolean> {
|
||||
try {
|
||||
return await Promise.race([
|
||||
probe,
|
||||
new Promise<boolean>((resolve) => {
|
||||
new Promise<PathProbeKind>((resolveTimeout) => {
|
||||
timer = setTimeout(() => {
|
||||
if (inFlight.get(path) === probe) stalled.add(path);
|
||||
resolve(false);
|
||||
}, PROBE_TIMEOUT_MS);
|
||||
if (inFlight.get(abs) === probe && !stalled.has(abs)) {
|
||||
stalled.set(abs, stallScope(abs));
|
||||
console.warn(
|
||||
`[path-probe] ${abs} did not answer within ${PATH_PROBE_TIMEOUT_MS} ms ` +
|
||||
'(unreachable mount?); treating it and its neighbours as unknown until it does'
|
||||
);
|
||||
}
|
||||
resolveTimeout('unknown');
|
||||
}, PATH_PROBE_TIMEOUT_MS);
|
||||
timer.unref?.();
|
||||
}),
|
||||
]);
|
||||
@@ -80,3 +172,19 @@ export async function boundedPathExists(path: string): Promise<boolean> {
|
||||
if (timer) clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
/** Tri-state probe of `path`; see the module comment for what "unknown" means. */
|
||||
export async function probePath(path: string, options: PathProbeOptions = {}): Promise<PathProbeState> {
|
||||
const kind = await probePathKind(path, options);
|
||||
return kind === 'directory' || kind === 'file' ? 'present' : kind;
|
||||
}
|
||||
|
||||
/**
|
||||
* `true` only when `path` is known to exist. For DISPLAY decisions only (does a
|
||||
* case have a CLAUDE.md): it folds "unknown" into `false`, so never use it to
|
||||
* decide that something is absent and may be created, scaffolded or reported
|
||||
* missing; use {@link probePath} for that.
|
||||
*/
|
||||
export async function boundedPathExists(path: string): Promise<boolean> {
|
||||
return (await probePath(path)) === 'present';
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user