mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-09 08:59:40 +02:00
fix: subagent tabs render to correct parent session after restart
- restoreSubagentWindowStates now discovers parent sessions before restoring - closeSubagentWindow discovers parent before minimizing to prevent wrong tab - Fixed async handling in subagent:completed event handler chore: bump version to 0.1389 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
arkon
@@ -25,7 +25,7 @@ Always bump version on every COM, even for small changes.
|
|||||||
|
|
||||||
Claudeman is a Claude Code session manager with a web interface and autonomous Ralph Loop. It spawns Claude CLI processes via PTY, streams output in real-time via SSE, and supports scheduled/timed runs.
|
Claudeman is a Claude Code session manager with a web interface and autonomous Ralph Loop. It spawns Claude CLI processes via PTY, streams output in real-time via SSE, and supports scheduled/timed runs.
|
||||||
|
|
||||||
**Version**: 0.1388 (must match `package.json`)
|
**Version**: 0.1389 (must match `package.json`)
|
||||||
|
|
||||||
**Tech Stack**: TypeScript (ES2022/NodeNext, strict mode), Node.js, Fastify, Server-Sent Events, node-pty
|
**Tech Stack**: TypeScript (ES2022/NodeNext, strict mode), Node.js, Fastify, Server-Sent Events, node-pty
|
||||||
|
|
||||||
@@ -119,10 +119,31 @@ npx vitest run -t "should create session" # By pattern
|
|||||||
| 3152 | browser-e2e.test.ts (hook events tests) |
|
| 3152 | browser-e2e.test.ts (hook events tests) |
|
||||||
| 3153 | browser-e2e.test.ts (Ralph panel tests) |
|
| 3153 | browser-e2e.test.ts (Ralph panel tests) |
|
||||||
| 3154 | file-link-click.test.ts |
|
| 3154 | file-link-click.test.ts |
|
||||||
|
| 3155 | browser-playwright.test.ts |
|
||||||
|
| 3156 | browser-puppeteer.test.ts |
|
||||||
|
| 3157 | browser-agent.test.ts |
|
||||||
|
| 3158-3160 | browser-comparison.test.ts |
|
||||||
|
| 3180-3182 | scripts/browser-comparison.mjs (benchmark) |
|
||||||
|
|
||||||
**Next available port**: 3155
|
**Next available port**: 3183
|
||||||
|
|
||||||
Unit tests (no port needed): respawn-controller, ralph-tracker, pty-interactive, task-queue, task, ralph-loop, session-manager, state-store, types, templates, ralph-config, spawn-detector, spawn-types, spawn-orchestrator, ai-idle-checker, ai-plan-checker
|
**Browser Testing**: Three frameworks available (Playwright, Puppeteer, Agent-Browser). See `docs/browser-testing-guide.md` for full comparison and patterns.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run browser benchmark (standalone - recommended)
|
||||||
|
npx tsx scripts/browser-comparison.mjs
|
||||||
|
|
||||||
|
# Run existing browser E2E tests
|
||||||
|
npm test -- test/browser-e2e.test.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
**Browser Testing Key Points**:
|
||||||
|
- **Vitest hook issue**: Browser tests using `beforeAll`/`afterAll` timeout even when tests pass. Use standalone scripts or run browser code directly in tests.
|
||||||
|
- **Recommended framework**: Playwright for most cases (auto-waiting, debugging). Puppeteer for Chrome-specific/CDP features.
|
||||||
|
- **Required browser args**: `--no-sandbox`, `--disable-setuid-sandbox`, `--disable-dev-shm-usage`
|
||||||
|
- **Install browsers**: `npx playwright install chromium` after npm install
|
||||||
|
|
||||||
|
**Unit tests** (no server needed): `respawn-controller`, `ralph-tracker`, `pty-interactive`, `task-queue`, `task`, `ralph-loop`, `session-manager`, `state-store`, `types`, `templates`, `ralph-config`, `spawn-detector`, `spawn-types`, `spawn-orchestrator`, `ai-idle-checker`, `ai-plan-checker`
|
||||||
|
|
||||||
**Test Utilities**: `test/respawn-test-utils.ts` provides MockSession, MockAiIdleChecker, MockAiPlanChecker, time controller, state tracker, and event recorder for respawn controller testing. See `test/respawn-test-plan.md` for architecture and `test/respawn-scenarios.md` for comprehensive test scenarios.
|
**Test Utilities**: `test/respawn-test-utils.ts` provides MockSession, MockAiIdleChecker, MockAiPlanChecker, time controller, state tracker, and event recorder for respawn controller testing. See `test/respawn-test-plan.md` for architecture and `test/respawn-scenarios.md` for comprehensive test scenarios.
|
||||||
|
|
||||||
@@ -613,6 +634,7 @@ Placeholders replaced:
|
|||||||
- `docs/spawn-protocol.md` - Spawn1337 agent protocol, MCP tools, resource governance
|
- `docs/spawn-protocol.md` - Spawn1337 agent protocol, MCP tools, resource governance
|
||||||
- `docs/ralph-wiggum-guide.md` - Ralph Wiggum loop guide (plugin reference, prompt templates)
|
- `docs/ralph-wiggum-guide.md` - Ralph Wiggum loop guide (plugin reference, prompt templates)
|
||||||
- `docs/claude-code-hooks-reference.md` - Claude Code hooks documentation
|
- `docs/claude-code-hooks-reference.md` - Claude Code hooks documentation
|
||||||
|
- `docs/browser-testing-guide.md` - Browser testing frameworks comparison, patterns, and known issues
|
||||||
|
|
||||||
**Internal planning** (historical context, may be outdated):
|
**Internal planning** (historical context, may be outdated):
|
||||||
- `docs/respawn-improvement-plan.md` - Planned respawn improvements
|
- `docs/respawn-improvement-plan.md` - Planned respawn improvements
|
||||||
|
|||||||
@@ -0,0 +1,338 @@
|
|||||||
|
# Browser Testing Guide for Claudeman
|
||||||
|
|
||||||
|
This guide documents the browser testing infrastructure, framework comparison results, and best practices for testing the Claudeman web UI.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run standalone benchmark (recommended - avoids vitest hook issues)
|
||||||
|
npx tsx scripts/browser-comparison.mjs
|
||||||
|
|
||||||
|
# Run existing browser E2E tests
|
||||||
|
npm test -- test/browser-e2e.test.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
## Framework Comparison Results
|
||||||
|
|
||||||
|
We tested three browser automation frameworks against the Claudeman web UI:
|
||||||
|
|
||||||
|
| Framework | Avg Duration | Best For |
|
||||||
|
|-----------|--------------|----------|
|
||||||
|
| **Puppeteer** | 1223ms | Simple operations, Chrome-specific features |
|
||||||
|
| **Playwright** | 1373ms | Complex interactions, cross-browser, debugging |
|
||||||
|
| **Agent-Browser** | N/A (timeout) | AI agent navigation with semantic locators |
|
||||||
|
|
||||||
|
### Detailed Benchmarks
|
||||||
|
|
||||||
|
| Scenario | Playwright | Puppeteer |
|
||||||
|
|----------|------------|-----------|
|
||||||
|
| Page load | 1445ms | 433ms |
|
||||||
|
| Element selection | 442ms | 373ms |
|
||||||
|
| Modal interaction | 1487ms | 1605ms |
|
||||||
|
| Rapid operations (5 cycles) | 2119ms | 2482ms |
|
||||||
|
|
||||||
|
**Key findings:**
|
||||||
|
- Puppeteer is faster for simple page loads and element selection
|
||||||
|
- Playwright handles rapid/complex interactions better (auto-waiting)
|
||||||
|
- Agent-browser CLI has startup overhead issues in this environment
|
||||||
|
|
||||||
|
## Known Issues
|
||||||
|
|
||||||
|
### Vitest Hook Timeouts
|
||||||
|
|
||||||
|
**Problem:** Browser tests using vitest's `beforeAll`/`afterAll` hooks consistently timeout, even when the tests actually complete successfully.
|
||||||
|
|
||||||
|
**Symptoms:**
|
||||||
|
- Tests show as "skipped"
|
||||||
|
- Error: "Hook timed out in 60000ms"
|
||||||
|
- But cleanup messages appear (indicating tests ran)
|
||||||
|
|
||||||
|
**Root cause:** Unclear - possibly related to:
|
||||||
|
- vitest's module isolation with async browser launches
|
||||||
|
- Interaction between global setup.ts hooks and test-level hooks
|
||||||
|
- Multiple test file imports causing duplicate hook execution
|
||||||
|
|
||||||
|
**Workarounds:**
|
||||||
|
1. **Use standalone scripts** (recommended):
|
||||||
|
```bash
|
||||||
|
npx tsx scripts/browser-comparison.mjs
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Run browser code directly in tests** (not in hooks):
|
||||||
|
```typescript
|
||||||
|
it('should test something', async () => {
|
||||||
|
const browser = await chromium.launch();
|
||||||
|
// ... test code ...
|
||||||
|
await browser.close();
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Use the existing browser-e2e.test.ts pattern** which uses agent-browser CLI commands via `execSync` (avoids async hook issues)
|
||||||
|
|
||||||
|
## Test File Structure
|
||||||
|
|
||||||
|
### Port Allocation
|
||||||
|
|
||||||
|
| Port Range | Test File |
|
||||||
|
|------------|-----------|
|
||||||
|
| 3150-3153 | browser-e2e.test.ts (existing) |
|
||||||
|
| 3154 | file-link-click.test.ts |
|
||||||
|
| 3155 | browser-playwright.test.ts |
|
||||||
|
| 3156 | browser-puppeteer.test.ts |
|
||||||
|
| 3157 | browser-agent.test.ts |
|
||||||
|
| 3158-3160 | browser-comparison.test.ts |
|
||||||
|
| 3180-3182 | scripts/browser-comparison.mjs |
|
||||||
|
|
||||||
|
### File Purposes
|
||||||
|
|
||||||
|
| File | Framework | Status |
|
||||||
|
|------|-----------|--------|
|
||||||
|
| `test/browser-e2e.test.ts` | agent-browser | ✅ Working |
|
||||||
|
| `test/browser-playwright.test.ts` | Playwright | ⚠️ Vitest hook issues |
|
||||||
|
| `test/browser-puppeteer.test.ts` | Puppeteer | ⚠️ Vitest hook issues |
|
||||||
|
| `test/browser-agent.test.ts` | agent-browser | ⚠️ Vitest hook issues |
|
||||||
|
| `scripts/browser-comparison.mjs` | All three | ✅ Working (standalone) |
|
||||||
|
|
||||||
|
## Framework-Specific Patterns
|
||||||
|
|
||||||
|
### Playwright
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { chromium } from 'playwright';
|
||||||
|
|
||||||
|
const browser = await chromium.launch({
|
||||||
|
headless: true,
|
||||||
|
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||||
|
});
|
||||||
|
|
||||||
|
const page = await browser.newPage();
|
||||||
|
await page.goto('http://localhost:3000');
|
||||||
|
|
||||||
|
// Auto-waiting selectors
|
||||||
|
await page.click('.btn-claude');
|
||||||
|
await page.waitForSelector('.session-tab', { state: 'visible' });
|
||||||
|
|
||||||
|
// Assertions with expect
|
||||||
|
await expect(page.locator('.header')).toBeVisible();
|
||||||
|
await expect(page).toHaveTitle('Claudeman');
|
||||||
|
|
||||||
|
await browser.close();
|
||||||
|
```
|
||||||
|
|
||||||
|
**Pros:**
|
||||||
|
- Built-in auto-waiting
|
||||||
|
- Excellent trace viewer for debugging
|
||||||
|
- Cross-browser support (Chromium, Firefox, WebKit)
|
||||||
|
- Native `expect` assertions
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Slightly slower page loads
|
||||||
|
- Larger dependency
|
||||||
|
|
||||||
|
### Puppeteer
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import puppeteer from 'puppeteer';
|
||||||
|
|
||||||
|
const browser = await puppeteer.launch({
|
||||||
|
headless: true,
|
||||||
|
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||||
|
});
|
||||||
|
|
||||||
|
const page = await browser.newPage();
|
||||||
|
await page.goto('http://localhost:3000');
|
||||||
|
|
||||||
|
// Manual waiting often needed
|
||||||
|
await page.click('.btn-claude');
|
||||||
|
await page.waitForSelector('.session-tab', { visible: true });
|
||||||
|
|
||||||
|
// Element queries
|
||||||
|
const title = await page.title();
|
||||||
|
const text = await page.$eval('.logo', el => el.textContent);
|
||||||
|
|
||||||
|
// CDP access for advanced features
|
||||||
|
const client = await page.target().createCDPSession();
|
||||||
|
await client.send('Performance.enable');
|
||||||
|
|
||||||
|
await browser.close();
|
||||||
|
```
|
||||||
|
|
||||||
|
**Pros:**
|
||||||
|
- Faster for simple operations
|
||||||
|
- Direct Chrome DevTools Protocol access
|
||||||
|
- Smaller dependency
|
||||||
|
- Good for Chrome-specific testing
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Chrome/Chromium only
|
||||||
|
- Manual waiting required
|
||||||
|
- Less robust for complex interactions
|
||||||
|
|
||||||
|
### Agent-Browser (CLI)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { execSync } from 'node:child_process';
|
||||||
|
|
||||||
|
function agentBrowser(cmd: string): string {
|
||||||
|
return execSync(`npx agent-browser ${cmd}`, {
|
||||||
|
timeout: 30000,
|
||||||
|
encoding: 'utf-8',
|
||||||
|
}).trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
function agentBrowserJson<T>(cmd: string): T {
|
||||||
|
const result = agentBrowser(`${cmd} --json`);
|
||||||
|
return JSON.parse(result).data;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Usage
|
||||||
|
agentBrowser('open http://localhost:3000');
|
||||||
|
agentBrowser('click ".btn-claude"');
|
||||||
|
const title = agentBrowserJson<{title: string}>('get title');
|
||||||
|
|
||||||
|
// Semantic locators (AI-friendly)
|
||||||
|
agentBrowser('find role button click --name "Submit"');
|
||||||
|
agentBrowser('find text "Settings" click');
|
||||||
|
|
||||||
|
// Accessibility snapshot
|
||||||
|
const snapshot = agentBrowser('snapshot');
|
||||||
|
|
||||||
|
agentBrowser('close');
|
||||||
|
```
|
||||||
|
|
||||||
|
**Pros:**
|
||||||
|
- AI-agent friendly (semantic locators)
|
||||||
|
- Accessibility tree snapshots
|
||||||
|
- Simple CLI interface
|
||||||
|
- Reference-based selection (@e1, @e2)
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- CLI overhead (spawn process per command)
|
||||||
|
- Slower for rapid operations
|
||||||
|
- Less programmatic control
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
### 1. Use Standalone Scripts for Benchmarks
|
||||||
|
|
||||||
|
Vitest has issues with browser hooks. For reliable benchmarking:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create a standalone .mjs script
|
||||||
|
npx tsx scripts/browser-comparison.mjs
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Browser Launch Arguments
|
||||||
|
|
||||||
|
Always include these args for headless environments:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
headless: true,
|
||||||
|
args: [
|
||||||
|
'--no-sandbox', // Required for Docker/CI
|
||||||
|
'--disable-setuid-sandbox',
|
||||||
|
'--disable-dev-shm-usage', // Prevents /dev/shm issues
|
||||||
|
],
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Install Playwright Browsers
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npx playwright install chromium
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Wait for Server Startup
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const server = new WebServer(PORT);
|
||||||
|
await server.start();
|
||||||
|
await new Promise(r => setTimeout(r, 1000)); // Allow server to stabilize
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Clean Up Sessions
|
||||||
|
|
||||||
|
Track created sessions for cleanup:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const createdSessions: string[] = [];
|
||||||
|
|
||||||
|
// In test
|
||||||
|
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
createdSessions.push(data.sessions[0].id);
|
||||||
|
|
||||||
|
// In cleanup
|
||||||
|
for (const id of createdSessions) {
|
||||||
|
await fetch(`${BASE_URL}/api/sessions/${id}`, { method: 'DELETE' });
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6. Handle Modal Timing
|
||||||
|
|
||||||
|
Modals have animation delays:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Playwright (auto-waits)
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal', { state: 'visible' });
|
||||||
|
|
||||||
|
// Puppeteer (manual wait)
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal', { visible: true });
|
||||||
|
|
||||||
|
// Agent-browser (explicit delay)
|
||||||
|
agentBrowser('click ".help-btn"');
|
||||||
|
await new Promise(r => setTimeout(r, 500));
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key DOM Selectors
|
||||||
|
|
||||||
|
For reference when writing browser tests:
|
||||||
|
|
||||||
|
```
|
||||||
|
.btn-claude // Create Claude session button
|
||||||
|
.btn-settings // Settings button
|
||||||
|
.help-btn // Help button
|
||||||
|
.session-tab // Session tabs
|
||||||
|
.session-tab.active // Active session tab
|
||||||
|
.xterm // Terminal container
|
||||||
|
#helpModal // Help modal
|
||||||
|
#appSettingsModal // Settings modal
|
||||||
|
.modal-content // Modal content
|
||||||
|
.modal-close // Modal close button
|
||||||
|
.header-brand .logo // Logo text
|
||||||
|
#versionDisplay // Version display
|
||||||
|
#quickStartCase // Quick start dropdown
|
||||||
|
```
|
||||||
|
|
||||||
|
## Recommendations by Use Case
|
||||||
|
|
||||||
|
| Use Case | Recommended Framework |
|
||||||
|
|----------|----------------------|
|
||||||
|
| CI/CD testing | Playwright |
|
||||||
|
| Chrome-specific features | Puppeteer |
|
||||||
|
| AI agent development | Agent-Browser |
|
||||||
|
| Visual regression | Playwright |
|
||||||
|
| Performance testing | Puppeteer |
|
||||||
|
| Accessibility testing | Agent-Browser |
|
||||||
|
| Cross-browser testing | Playwright |
|
||||||
|
| Quick prototyping | Agent-Browser CLI |
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"devDependencies": {
|
||||||
|
"playwright": "^1.58.0",
|
||||||
|
"puppeteer": "^24.36.0",
|
||||||
|
"agent-browser": "^0.6.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Install browsers after npm install:
|
||||||
|
```bash
|
||||||
|
npx playwright install chromium
|
||||||
|
```
|
||||||
Generated
+1131
-13
File diff suppressed because it is too large
Load Diff
+4
-3
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "claudeman",
|
"name": "claudeman",
|
||||||
"version": "0.1388",
|
"version": "0.1389",
|
||||||
"description": "The missing control plane for Claude Code - run 20 autonomous agents with real-time monitoring and session persistence",
|
"description": "The missing control plane for Claude Code - run 20 autonomous agents with real-time monitoring and session persistence",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "dist/index.js",
|
"main": "dist/index.js",
|
||||||
@@ -54,12 +54,13 @@
|
|||||||
"@types/node": "^20.14.0",
|
"@types/node": "^20.14.0",
|
||||||
"@types/react": "^19.2.9",
|
"@types/react": "^19.2.9",
|
||||||
"@types/uuid": "^10.0.0",
|
"@types/uuid": "^10.0.0",
|
||||||
"@vitest/coverage-v8": "^4.0.17",
|
"@vitest/coverage-v8": "^4.0.18",
|
||||||
"agent-browser": "^0.6.0",
|
"agent-browser": "^0.6.0",
|
||||||
"playwright": "^1.58.0",
|
"playwright": "^1.58.0",
|
||||||
|
"puppeteer": "^24.36.0",
|
||||||
"tsx": "^4.15.0",
|
"tsx": "^4.15.0",
|
||||||
"typescript": "^5.9.3",
|
"typescript": "^5.9.3",
|
||||||
"vitest": "^4.0.17"
|
"vitest": "^4.0.18"
|
||||||
},
|
},
|
||||||
"engines": {
|
"engines": {
|
||||||
"node": ">=18.0.0"
|
"node": ">=18.0.0"
|
||||||
|
|||||||
@@ -0,0 +1,309 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* Browser Framework Comparison Script
|
||||||
|
*
|
||||||
|
* Compares Playwright, Puppeteer, and Agent-Browser for testing Claudeman web UI.
|
||||||
|
* Run: node scripts/browser-comparison.mjs
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { chromium } from 'playwright';
|
||||||
|
import puppeteer from 'puppeteer';
|
||||||
|
import { execSync } from 'node:child_process';
|
||||||
|
import { WebServer } from '../src/web/server.js';
|
||||||
|
|
||||||
|
const PORTS = {
|
||||||
|
playwright: 3180,
|
||||||
|
puppeteer: 3181,
|
||||||
|
agentBrowser: 3182,
|
||||||
|
};
|
||||||
|
|
||||||
|
const results = [];
|
||||||
|
|
||||||
|
function logSection(title) {
|
||||||
|
console.log('\n' + '='.repeat(60));
|
||||||
|
console.log(` ${title}`);
|
||||||
|
console.log('='.repeat(60));
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runBenchmark(name, framework, fn) {
|
||||||
|
const start = Date.now();
|
||||||
|
let success = false;
|
||||||
|
let error = null;
|
||||||
|
try {
|
||||||
|
await fn();
|
||||||
|
success = true;
|
||||||
|
} catch (e) {
|
||||||
|
error = e.message;
|
||||||
|
}
|
||||||
|
const duration = Date.now() - start;
|
||||||
|
results.push({ scenario: name, framework, duration, success, error });
|
||||||
|
console.log(` ${success ? '✅' : '❌'} ${name}: ${duration}ms ${error ? `(${error.slice(0, 50)})` : ''}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Agent-browser helper
|
||||||
|
function agentBrowser(cmd) {
|
||||||
|
return execSync(`npx agent-browser ${cmd}`, {
|
||||||
|
timeout: 30000,
|
||||||
|
encoding: 'utf-8',
|
||||||
|
stdio: ['pipe', 'pipe', 'pipe'],
|
||||||
|
}).trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
function agentBrowserJson(cmd) {
|
||||||
|
const result = agentBrowser(`${cmd} --json`);
|
||||||
|
const parsed = JSON.parse(result);
|
||||||
|
if (!parsed.success) throw new Error(parsed.error || 'unknown error');
|
||||||
|
return parsed.data;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
console.log('🚀 Browser Framework Comparison Benchmark');
|
||||||
|
console.log('Comparing: Playwright, Puppeteer, Agent-Browser\n');
|
||||||
|
|
||||||
|
// Start servers
|
||||||
|
const servers = [];
|
||||||
|
for (const [name, port] of Object.entries(PORTS)) {
|
||||||
|
console.log(`Starting server for ${name} on port ${port}...`);
|
||||||
|
const server = new WebServer(port);
|
||||||
|
await server.start();
|
||||||
|
servers.push(server);
|
||||||
|
}
|
||||||
|
await new Promise(r => setTimeout(r, 2000));
|
||||||
|
|
||||||
|
// ============================================
|
||||||
|
// PLAYWRIGHT TESTS
|
||||||
|
// ============================================
|
||||||
|
logSection('PLAYWRIGHT TESTS');
|
||||||
|
|
||||||
|
let playwrightBrowser;
|
||||||
|
try {
|
||||||
|
playwrightBrowser = await chromium.launch({
|
||||||
|
headless: true,
|
||||||
|
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||||
|
});
|
||||||
|
console.log(' Browser launched');
|
||||||
|
|
||||||
|
// Test 1: Page load
|
||||||
|
await runBenchmark('page-load', 'playwright', async () => {
|
||||||
|
const page = await playwrightBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
const title = await page.title();
|
||||||
|
if (title !== 'Claudeman') throw new Error(`Expected Claudeman, got ${title}`);
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 2: Element selection
|
||||||
|
await runBenchmark('element-selection', 'playwright', async () => {
|
||||||
|
const page = await playwrightBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
const logo = await page.locator('.header-brand .logo').textContent();
|
||||||
|
const version = await page.locator('#versionDisplay').textContent();
|
||||||
|
if (logo !== 'Claudeman') throw new Error('Logo mismatch');
|
||||||
|
if (!version.match(/v\d+/)) throw new Error('Version mismatch');
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 3: Modal interaction
|
||||||
|
await runBenchmark('modal-interaction', 'playwright', async () => {
|
||||||
|
const page = await playwrightBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 4: Rapid operations (5 modal cycles)
|
||||||
|
await runBenchmark('rapid-operations', 'playwright', async () => {
|
||||||
|
const page = await playwrightBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||||
|
}
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
} finally {
|
||||||
|
if (playwrightBrowser) await playwrightBrowser.close();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============================================
|
||||||
|
// PUPPETEER TESTS
|
||||||
|
// ============================================
|
||||||
|
logSection('PUPPETEER TESTS');
|
||||||
|
|
||||||
|
let puppeteerBrowser;
|
||||||
|
try {
|
||||||
|
puppeteerBrowser = await puppeteer.launch({
|
||||||
|
headless: true,
|
||||||
|
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||||
|
});
|
||||||
|
console.log(' Browser launched');
|
||||||
|
|
||||||
|
// Test 1: Page load
|
||||||
|
await runBenchmark('page-load', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
const title = await page.title();
|
||||||
|
if (title !== 'Claudeman') throw new Error(`Expected Claudeman, got ${title}`);
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 2: Element selection
|
||||||
|
await runBenchmark('element-selection', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
const logo = await page.$eval('.header-brand .logo', el => el.textContent);
|
||||||
|
const version = await page.$eval('#versionDisplay', el => el.textContent);
|
||||||
|
if (logo !== 'Claudeman') throw new Error('Logo mismatch');
|
||||||
|
if (!version.match(/v\d+/)) throw new Error('Version mismatch');
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 3: Modal interaction
|
||||||
|
await runBenchmark('modal-interaction', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 4: Rapid operations
|
||||||
|
await runBenchmark('rapid-operations', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await new Promise(r => setTimeout(r, 200));
|
||||||
|
}
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
} finally {
|
||||||
|
if (puppeteerBrowser) await puppeteerBrowser.close();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============================================
|
||||||
|
// AGENT-BROWSER TESTS
|
||||||
|
// ============================================
|
||||||
|
logSection('AGENT-BROWSER TESTS');
|
||||||
|
|
||||||
|
let agentBrowserAvailable = false;
|
||||||
|
try {
|
||||||
|
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||||
|
await new Promise(r => setTimeout(r, 2000));
|
||||||
|
const title = agentBrowserJson('get title');
|
||||||
|
agentBrowserAvailable = title.title === 'Claudeman';
|
||||||
|
console.log(' Browser launched');
|
||||||
|
|
||||||
|
// Test 1: Page load
|
||||||
|
await runBenchmark('page-load', 'agent-browser', async () => {
|
||||||
|
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
const title = agentBrowserJson('get title');
|
||||||
|
if (title.title !== 'Claudeman') throw new Error(`Expected Claudeman, got ${title.title}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 2: Element selection
|
||||||
|
await runBenchmark('element-selection', 'agent-browser', async () => {
|
||||||
|
const logo = agentBrowserJson('get text ".header-brand .logo"');
|
||||||
|
const version = agentBrowserJson('get text "#versionDisplay"');
|
||||||
|
if (logo.text !== 'Claudeman') throw new Error('Logo mismatch');
|
||||||
|
if (!version.text.match(/v\d+/)) throw new Error('Version mismatch');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 3: Modal interaction
|
||||||
|
await runBenchmark('modal-interaction', 'agent-browser', async () => {
|
||||||
|
agentBrowser('click ".help-btn"');
|
||||||
|
await new Promise(r => setTimeout(r, 500));
|
||||||
|
const visible = agentBrowserJson('is visible "#helpModal .modal-content"');
|
||||||
|
if (!visible.visible) throw new Error('Modal not visible');
|
||||||
|
agentBrowser('click "#helpModal .modal-close"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
});
|
||||||
|
|
||||||
|
// Test 4: Rapid operations
|
||||||
|
await runBenchmark('rapid-operations', 'agent-browser', async () => {
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
agentBrowser('click ".help-btn"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
agentBrowser('click "#helpModal .modal-close"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
} catch (e) {
|
||||||
|
console.log(` ⚠️ Agent-browser not available: ${e.message}`);
|
||||||
|
} finally {
|
||||||
|
try { agentBrowser('close'); } catch {}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============================================
|
||||||
|
// RESULTS
|
||||||
|
// ============================================
|
||||||
|
logSection('RESULTS SUMMARY');
|
||||||
|
|
||||||
|
// Group by framework
|
||||||
|
const frameworks = ['playwright', 'puppeteer', 'agent-browser'];
|
||||||
|
for (const fw of frameworks) {
|
||||||
|
const fwResults = results.filter(r => r.framework === fw);
|
||||||
|
if (fwResults.length === 0) continue;
|
||||||
|
|
||||||
|
const passed = fwResults.filter(r => r.success).length;
|
||||||
|
const total = fwResults.length;
|
||||||
|
const avgDuration = Math.round(fwResults.reduce((a, r) => a + r.duration, 0) / total);
|
||||||
|
|
||||||
|
console.log(`\n ${fw.toUpperCase()}`);
|
||||||
|
console.log(` Tests: ${passed}/${total} passed`);
|
||||||
|
console.log(` Avg Duration: ${avgDuration}ms`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Performance ranking
|
||||||
|
console.log('\n🏆 PERFORMANCE RANKING (by avg duration)');
|
||||||
|
const stats = frameworks.map(fw => {
|
||||||
|
const fwResults = results.filter(r => r.framework === fw);
|
||||||
|
if (fwResults.length === 0) return null;
|
||||||
|
return {
|
||||||
|
framework: fw,
|
||||||
|
avg: Math.round(fwResults.reduce((a, r) => a + r.duration, 0) / fwResults.length),
|
||||||
|
passed: fwResults.filter(r => r.success).length,
|
||||||
|
total: fwResults.length,
|
||||||
|
};
|
||||||
|
}).filter(Boolean).sort((a, b) => a.avg - b.avg);
|
||||||
|
|
||||||
|
stats.forEach((s, i) => {
|
||||||
|
const medal = i === 0 ? '🥇' : i === 1 ? '🥈' : '🥉';
|
||||||
|
console.log(` ${medal} ${s.framework}: ${s.avg}ms avg (${s.passed}/${s.total} passed)`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Cleanup
|
||||||
|
console.log('\n Cleaning up...');
|
||||||
|
for (const server of servers) {
|
||||||
|
await server.stop();
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n✅ Benchmark complete!\n');
|
||||||
|
|
||||||
|
// Detailed results table
|
||||||
|
console.log('📋 DETAILED RESULTS');
|
||||||
|
console.table(results.map(r => ({
|
||||||
|
Scenario: r.scenario,
|
||||||
|
Framework: r.framework,
|
||||||
|
Duration: `${r.duration}ms`,
|
||||||
|
Status: r.success ? '✅' : '❌',
|
||||||
|
})));
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch(e => {
|
||||||
|
console.error('Benchmark failed:', e);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
+282
-11
@@ -1764,7 +1764,7 @@ class ClaudemanApp {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
this.eventSource.addEventListener('subagent:completed', (e) => {
|
this.eventSource.addEventListener('subagent:completed', async (e) => {
|
||||||
const data = JSON.parse(e.data);
|
const data = JSON.parse(e.data);
|
||||||
const existing = this.subagents.get(data.agentId);
|
const existing = this.subagents.get(data.agentId);
|
||||||
if (existing) {
|
if (existing) {
|
||||||
@@ -1778,7 +1778,7 @@ class ClaudemanApp {
|
|||||||
if (this.subagentWindows.has(data.agentId)) {
|
if (this.subagentWindows.has(data.agentId)) {
|
||||||
const windowData = this.subagentWindows.get(data.agentId);
|
const windowData = this.subagentWindows.get(data.agentId);
|
||||||
if (windowData && !windowData.minimized) {
|
if (windowData && !windowData.minimized) {
|
||||||
this.closeSubagentWindow(data.agentId); // This minimizes to tab
|
await this.closeSubagentWindow(data.agentId); // This minimizes to tab
|
||||||
this.saveSubagentWindowStates(); // Persist the minimized state
|
this.saveSubagentWindowStates(); // Persist the minimized state
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -2346,6 +2346,245 @@ class ClaudemanApp {
|
|||||||
this.selectSession(ids[nextIndex]);
|
this.selectSession(ids[nextIndex]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ========== Navigation ==========
|
||||||
|
|
||||||
|
goHome() {
|
||||||
|
// Deselect active session and show welcome screen
|
||||||
|
this.activeSessionId = null;
|
||||||
|
this.terminal.clear();
|
||||||
|
this.showWelcome();
|
||||||
|
this.renderSessionTabs();
|
||||||
|
this.renderRalphStatePanel();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ========== Ralph Loop Wizard ==========
|
||||||
|
|
||||||
|
// Wizard state
|
||||||
|
ralphWizardStep = 1;
|
||||||
|
ralphWizardConfig = {
|
||||||
|
taskDescription: '',
|
||||||
|
completionPhrase: 'COMPLETE',
|
||||||
|
maxIterations: 10,
|
||||||
|
caseName: 'testcase',
|
||||||
|
enableRespawn: true,
|
||||||
|
};
|
||||||
|
|
||||||
|
showRalphWizard() {
|
||||||
|
// Reset wizard state
|
||||||
|
this.ralphWizardStep = 1;
|
||||||
|
this.ralphWizardConfig = {
|
||||||
|
taskDescription: '',
|
||||||
|
completionPhrase: 'COMPLETE',
|
||||||
|
maxIterations: 10,
|
||||||
|
caseName: document.getElementById('quickStartCase')?.value || 'testcase',
|
||||||
|
enableRespawn: true,
|
||||||
|
};
|
||||||
|
|
||||||
|
// Reset UI
|
||||||
|
document.getElementById('ralphTaskDescription').value = '';
|
||||||
|
document.getElementById('ralphCompletionPhrase').value = 'COMPLETE';
|
||||||
|
this.selectIterationPreset(10);
|
||||||
|
|
||||||
|
// Populate case selector
|
||||||
|
this.populateRalphCaseSelector();
|
||||||
|
|
||||||
|
// Show wizard modal
|
||||||
|
this.updateRalphWizardUI();
|
||||||
|
document.getElementById('ralphWizardModal').classList.add('active');
|
||||||
|
document.getElementById('ralphTaskDescription').focus();
|
||||||
|
}
|
||||||
|
|
||||||
|
closeRalphWizard() {
|
||||||
|
document.getElementById('ralphWizardModal')?.classList.remove('active');
|
||||||
|
}
|
||||||
|
|
||||||
|
populateRalphCaseSelector() {
|
||||||
|
const select = document.getElementById('ralphCaseSelect');
|
||||||
|
const quickStartSelect = document.getElementById('quickStartCase');
|
||||||
|
|
||||||
|
if (quickStartSelect && select) {
|
||||||
|
select.innerHTML = quickStartSelect.innerHTML;
|
||||||
|
select.value = this.ralphWizardConfig.caseName;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
selectIterationPreset(iterations) {
|
||||||
|
this.ralphWizardConfig.maxIterations = iterations;
|
||||||
|
|
||||||
|
// Update button states
|
||||||
|
document.querySelectorAll('.iteration-preset-btn').forEach(btn => {
|
||||||
|
const btnIterations = parseInt(btn.dataset.iterations);
|
||||||
|
btn.classList.toggle('active', btnIterations === iterations);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
ralphWizardNext() {
|
||||||
|
if (this.ralphWizardStep === 1) {
|
||||||
|
// Validate step 1
|
||||||
|
const taskDescription = document.getElementById('ralphTaskDescription').value.trim();
|
||||||
|
const completionPhrase = document.getElementById('ralphCompletionPhrase').value.trim() || 'COMPLETE';
|
||||||
|
const caseName = document.getElementById('ralphCaseSelect').value;
|
||||||
|
|
||||||
|
if (!taskDescription) {
|
||||||
|
this.showToast('Please enter a task description', 'error');
|
||||||
|
document.getElementById('ralphTaskDescription').focus();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Save config
|
||||||
|
this.ralphWizardConfig.taskDescription = taskDescription;
|
||||||
|
this.ralphWizardConfig.completionPhrase = completionPhrase.toUpperCase();
|
||||||
|
this.ralphWizardConfig.caseName = caseName;
|
||||||
|
|
||||||
|
// Generate preview
|
||||||
|
this.updateRalphPromptPreview();
|
||||||
|
|
||||||
|
// Move to step 2
|
||||||
|
this.ralphWizardStep = 2;
|
||||||
|
this.updateRalphWizardUI();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ralphWizardBack() {
|
||||||
|
if (this.ralphWizardStep === 2) {
|
||||||
|
this.ralphWizardStep = 1;
|
||||||
|
this.updateRalphWizardUI();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
updateRalphWizardUI() {
|
||||||
|
const step = this.ralphWizardStep;
|
||||||
|
|
||||||
|
// Update progress indicators
|
||||||
|
document.querySelectorAll('.wizard-step').forEach(el => {
|
||||||
|
const stepNum = parseInt(el.dataset.step);
|
||||||
|
el.classList.toggle('active', stepNum === step);
|
||||||
|
el.classList.toggle('completed', stepNum < step);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Show/hide pages
|
||||||
|
document.getElementById('ralphWizardStep1').classList.toggle('hidden', step !== 1);
|
||||||
|
document.getElementById('ralphWizardStep2').classList.toggle('hidden', step !== 2);
|
||||||
|
|
||||||
|
// Show/hide buttons
|
||||||
|
document.getElementById('ralphBackBtn').style.display = step === 1 ? 'none' : 'block';
|
||||||
|
document.getElementById('ralphNextBtn').style.display = step === 2 ? 'none' : 'block';
|
||||||
|
document.getElementById('ralphStartBtn').style.display = step === 2 ? 'block' : 'none';
|
||||||
|
}
|
||||||
|
|
||||||
|
updateRalphPromptPreview() {
|
||||||
|
const config = this.ralphWizardConfig;
|
||||||
|
const preview = document.getElementById('ralphPromptPreview');
|
||||||
|
|
||||||
|
// Build the formatted prompt
|
||||||
|
let prompt = config.taskDescription;
|
||||||
|
|
||||||
|
// Add completion instructions
|
||||||
|
prompt += '\n\n---\n\n';
|
||||||
|
prompt += '## Completion Criteria\n\n';
|
||||||
|
prompt += 'When ALL requirements above are complete:\n';
|
||||||
|
prompt += '1. Run all tests to verify\n';
|
||||||
|
prompt += '2. Commit your changes\n';
|
||||||
|
prompt += `3. Output: <promise>${config.completionPhrase}</promise>\n`;
|
||||||
|
|
||||||
|
// Show preview with highlighting
|
||||||
|
const highlightedPrompt = prompt
|
||||||
|
.replace(/<promise>/g, '<span class="preview-highlight"><promise>')
|
||||||
|
.replace(/<\/promise>/g, '</promise></span>');
|
||||||
|
|
||||||
|
preview.innerHTML = highlightedPrompt;
|
||||||
|
|
||||||
|
// Update summary
|
||||||
|
document.getElementById('summaryPhrase').textContent = config.completionPhrase;
|
||||||
|
document.getElementById('summaryIterations').textContent =
|
||||||
|
config.maxIterations === 0 ? 'Unlimited' : config.maxIterations;
|
||||||
|
document.getElementById('summaryCase').textContent = config.caseName;
|
||||||
|
}
|
||||||
|
|
||||||
|
async startRalphLoop() {
|
||||||
|
const config = this.ralphWizardConfig;
|
||||||
|
|
||||||
|
// Read advanced options
|
||||||
|
config.enableRespawn = document.getElementById('ralphEnableRespawn')?.checked ?? true;
|
||||||
|
|
||||||
|
// Close wizard
|
||||||
|
this.closeRalphWizard();
|
||||||
|
|
||||||
|
// Build the full prompt
|
||||||
|
let fullPrompt = config.taskDescription;
|
||||||
|
fullPrompt += '\n\n---\n\n';
|
||||||
|
fullPrompt += '## Completion Criteria\n\n';
|
||||||
|
fullPrompt += 'When ALL requirements above are complete:\n';
|
||||||
|
fullPrompt += '1. Run all tests to verify\n';
|
||||||
|
fullPrompt += '2. Commit your changes\n';
|
||||||
|
fullPrompt += `3. Output: <promise>${config.completionPhrase}</promise>\n`;
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Step 1: Create/verify case exists and start session
|
||||||
|
this.terminal.clear();
|
||||||
|
this.terminal.writeln(`\x1b[1;33m Starting Ralph Loop in ${config.caseName}...\x1b[0m`);
|
||||||
|
this.terminal.writeln('');
|
||||||
|
|
||||||
|
// Create session via quick-start
|
||||||
|
const res = await fetch('/api/quick-start', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ caseName: config.caseName, mode: 'claude' })
|
||||||
|
});
|
||||||
|
|
||||||
|
const data = await res.json();
|
||||||
|
if (!data.success) {
|
||||||
|
this.showToast(data.error || 'Failed to start session', 'error');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const sessionId = data.sessionId;
|
||||||
|
|
||||||
|
// Step 2: Configure Ralph tracker for this session
|
||||||
|
await fetch(`/api/sessions/${sessionId}/ralph-config`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
enabled: true,
|
||||||
|
completionPhrase: config.completionPhrase,
|
||||||
|
maxIterations: config.maxIterations || null,
|
||||||
|
})
|
||||||
|
});
|
||||||
|
|
||||||
|
// Step 3: Enable respawn if requested
|
||||||
|
if (config.enableRespawn) {
|
||||||
|
await fetch(`/api/sessions/${sessionId}/respawn/enable`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
prompt: 'continue',
|
||||||
|
sendClear: true,
|
||||||
|
sendInit: true,
|
||||||
|
})
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 4: Switch to session and wait for it to be ready
|
||||||
|
await this.selectSession(sessionId);
|
||||||
|
|
||||||
|
// Wait a moment for session to be ready, then send the prompt
|
||||||
|
await new Promise(r => setTimeout(r, 2000));
|
||||||
|
|
||||||
|
// Send the prompt to start the loop
|
||||||
|
await fetch(`/api/sessions/${sessionId}/input`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ input: fullPrompt + '\n' })
|
||||||
|
});
|
||||||
|
|
||||||
|
this.showToast(`Ralph Loop started in ${config.caseName}`, 'success');
|
||||||
|
|
||||||
|
} catch (err) {
|
||||||
|
console.error('Failed to start Ralph loop:', err);
|
||||||
|
this.showToast('Failed to start Ralph loop: ' + err.message, 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ========== Quick Start ==========
|
// ========== Quick Start ==========
|
||||||
|
|
||||||
async loadQuickStartCases(selectCaseName = null) {
|
async loadQuickStartCases(selectCaseName = null) {
|
||||||
@@ -4287,13 +4526,35 @@ class ClaudemanApp {
|
|||||||
async restoreSubagentWindowStates() {
|
async restoreSubagentWindowStates() {
|
||||||
const states = await this.loadSubagentWindowStates();
|
const states = await this.loadSubagentWindowStates();
|
||||||
|
|
||||||
// Restore minimized state
|
// First, discover parent sessions for all subagents to establish correct mapping
|
||||||
for (const [sessionId, agentIds] of Object.entries(states.minimized || {})) {
|
// This is important after server restart when parentSessionId isn't set
|
||||||
|
const parentDiscoveryPromises = [];
|
||||||
|
for (const [agentId, agent] of this.subagents) {
|
||||||
|
if (!agent.parentSessionId) {
|
||||||
|
parentDiscoveryPromises.push(this.findParentSessionForSubagent(agentId));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (parentDiscoveryPromises.length > 0) {
|
||||||
|
await Promise.all(parentDiscoveryPromises);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Restore minimized state, but verify session mapping is correct
|
||||||
|
for (const [savedSessionId, agentIds] of Object.entries(states.minimized || {})) {
|
||||||
if (Array.isArray(agentIds) && agentIds.length > 0) {
|
if (Array.isArray(agentIds) && agentIds.length > 0) {
|
||||||
// Verify agents still exist
|
for (const agentId of agentIds) {
|
||||||
const validAgents = agentIds.filter(id => this.subagents.has(id));
|
const agent = this.subagents.get(agentId);
|
||||||
if (validAgents.length > 0) {
|
if (!agent) continue; // Agent no longer exists
|
||||||
this.minimizedSubagents.set(sessionId, new Set(validAgents));
|
|
||||||
|
// Use discovered parentSessionId, or validate saved sessionId exists
|
||||||
|
const correctSessionId = agent.parentSessionId ||
|
||||||
|
(this.sessions.has(savedSessionId) ? savedSessionId : null);
|
||||||
|
|
||||||
|
if (correctSessionId) {
|
||||||
|
if (!this.minimizedSubagents.has(correctSessionId)) {
|
||||||
|
this.minimizedSubagents.set(correctSessionId, new Set());
|
||||||
|
}
|
||||||
|
this.minimizedSubagents.get(correctSessionId).add(agentId);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -4326,6 +4587,7 @@ class ClaudemanApp {
|
|||||||
}
|
}
|
||||||
|
|
||||||
this.renderSessionTabs(); // Update tab badges
|
this.renderSessionTabs(); // Update tab badges
|
||||||
|
this.saveSubagentWindowStates(); // Persist corrected mappings
|
||||||
}
|
}
|
||||||
|
|
||||||
// ========== Help Modal ==========
|
// ========== Help Modal ==========
|
||||||
@@ -5804,14 +6066,23 @@ class ClaudemanApp {
|
|||||||
this.saveSubagentWindowStates();
|
this.saveSubagentWindowStates();
|
||||||
}
|
}
|
||||||
|
|
||||||
closeSubagentWindow(agentId) {
|
async closeSubagentWindow(agentId) {
|
||||||
const windowData = this.subagentWindows.get(agentId);
|
const windowData = this.subagentWindows.get(agentId);
|
||||||
if (!windowData) return;
|
if (!windowData) return;
|
||||||
|
|
||||||
const agent = this.subagents.get(agentId);
|
const agent = this.subagents.get(agentId);
|
||||||
const parentSessionId = agent?.parentSessionId || this.activeSessionId;
|
|
||||||
|
|
||||||
// Always minimize to tab (use active session if no parent)
|
// Try to discover parent session if not already known
|
||||||
|
// This ensures we minimize to the correct tab
|
||||||
|
if (agent && !agent.parentSessionId) {
|
||||||
|
await this.findParentSessionForSubagent(agentId);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Now get the correct parent session (re-read agent after discovery)
|
||||||
|
const updatedAgent = this.subagents.get(agentId);
|
||||||
|
const parentSessionId = updatedAgent?.parentSessionId || this.activeSessionId;
|
||||||
|
|
||||||
|
// Always minimize to tab (use active session if no parent found)
|
||||||
windowData.element.style.display = 'none';
|
windowData.element.style.display = 'none';
|
||||||
windowData.minimized = true;
|
windowData.minimized = true;
|
||||||
|
|
||||||
|
|||||||
+133
-1
@@ -17,7 +17,7 @@
|
|||||||
<!-- Compact Header with Session Tabs -->
|
<!-- Compact Header with Session Tabs -->
|
||||||
<header class="header">
|
<header class="header">
|
||||||
<div class="header-brand">
|
<div class="header-brand">
|
||||||
<span class="logo">Claudeman</span>
|
<span class="logo" onclick="app.goHome()" title="Go to main page">Claudeman</span>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Session Tabs -->
|
<!-- Session Tabs -->
|
||||||
@@ -206,6 +206,9 @@
|
|||||||
<input type="number" id="shellCount" class="tab-count-input" value="1" min="1" max="20" readonly title="Instance count">
|
<input type="number" id="shellCount" class="tab-count-input" value="1" min="1" max="20" readonly title="Instance count">
|
||||||
<button class="tab-count-btn" onclick="app.incrementShellCount()" title="Instance count">+</button>
|
<button class="tab-count-btn" onclick="app.incrementShellCount()" title="Instance count">+</button>
|
||||||
</div>
|
</div>
|
||||||
|
<button class="btn-toolbar btn-ralph" onclick="app.showRalphWizard()" title="Start Ralph Loop">
|
||||||
|
Ralph Loop
|
||||||
|
</button>
|
||||||
<div class="case-select-group">
|
<div class="case-select-group">
|
||||||
<select id="quickStartCase" class="toolbar-select" title="Select case">
|
<select id="quickStartCase" class="toolbar-select" title="Select case">
|
||||||
<option value="testcase">testcase</option>
|
<option value="testcase">testcase</option>
|
||||||
@@ -854,6 +857,135 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<!-- Ralph Loop Wizard Modal -->
|
||||||
|
<div class="modal" id="ralphWizardModal">
|
||||||
|
<div class="modal-backdrop" onclick="app.closeRalphWizard()"></div>
|
||||||
|
<div class="modal-content modal-lg">
|
||||||
|
<div class="modal-header">
|
||||||
|
<h3>Start Ralph Loop</h3>
|
||||||
|
<button class="modal-close" onclick="app.closeRalphWizard()">×</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Wizard Progress Steps -->
|
||||||
|
<div class="wizard-progress">
|
||||||
|
<div class="wizard-step active" data-step="1">
|
||||||
|
<span class="wizard-step-number">1</span>
|
||||||
|
<span class="wizard-step-label">Task Setup</span>
|
||||||
|
</div>
|
||||||
|
<div class="wizard-step" data-step="2">
|
||||||
|
<span class="wizard-step-number">2</span>
|
||||||
|
<span class="wizard-step-label">Launch</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="modal-body">
|
||||||
|
<!-- Step 1: Task & Completion Setup -->
|
||||||
|
<div class="wizard-page" id="ralphWizardStep1">
|
||||||
|
<div class="form-row">
|
||||||
|
<label>Task Description</label>
|
||||||
|
<div class="textarea-with-assist">
|
||||||
|
<textarea
|
||||||
|
id="ralphTaskDescription"
|
||||||
|
rows="6"
|
||||||
|
placeholder="Describe what you want Claude to accomplish...
|
||||||
|
|
||||||
|
Example:
|
||||||
|
Build a REST API for user authentication:
|
||||||
|
- JWT token generation and validation
|
||||||
|
- Login/logout endpoints
|
||||||
|
- Password reset flow
|
||||||
|
- Tests for all endpoints"></textarea>
|
||||||
|
</div>
|
||||||
|
<span class="form-hint">Be specific about requirements and success criteria</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="form-row">
|
||||||
|
<label>Completion Phrase</label>
|
||||||
|
<div class="phrase-input-group">
|
||||||
|
<input type="text" id="ralphCompletionPhrase" value="COMPLETE" placeholder="COMPLETE" list="phrasesSuggestions">
|
||||||
|
<datalist id="phrasesSuggestions">
|
||||||
|
<option value="COMPLETE">
|
||||||
|
<option value="DONE">
|
||||||
|
<option value="FINISHED">
|
||||||
|
<option value="TESTS_PASS">
|
||||||
|
<option value="ALL_TASKS_COMPLETE">
|
||||||
|
</datalist>
|
||||||
|
</div>
|
||||||
|
<span class="form-hint">Claude outputs <code><promise>PHRASE</promise></code> when done</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="form-row">
|
||||||
|
<label>Max Iterations</label>
|
||||||
|
<div class="iteration-presets">
|
||||||
|
<button type="button" class="iteration-preset-btn active" data-iterations="10" onclick="app.selectIterationPreset(10)">10</button>
|
||||||
|
<button type="button" class="iteration-preset-btn" data-iterations="25" onclick="app.selectIterationPreset(25)">25</button>
|
||||||
|
<button type="button" class="iteration-preset-btn" data-iterations="50" onclick="app.selectIterationPreset(50)">50</button>
|
||||||
|
<button type="button" class="iteration-preset-btn" data-iterations="100" onclick="app.selectIterationPreset(100)">100</button>
|
||||||
|
<button type="button" class="iteration-preset-btn" data-iterations="0" onclick="app.selectIterationPreset(0)">∞</button>
|
||||||
|
</div>
|
||||||
|
<span class="form-hint">Safety limit - loop stops after this many iterations</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="form-row">
|
||||||
|
<label>Working Directory</label>
|
||||||
|
<div class="case-select-inline">
|
||||||
|
<select id="ralphCaseSelect" class="form-select"></select>
|
||||||
|
<button class="btn-toolbar btn-sm" onclick="app.showCreateCaseModal()">+ New</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Step 2: Preview & Launch -->
|
||||||
|
<div class="wizard-page hidden" id="ralphWizardStep2">
|
||||||
|
<div class="ralph-preview-section">
|
||||||
|
<label>Prompt Preview</label>
|
||||||
|
<div class="ralph-prompt-preview" id="ralphPromptPreview"></div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="ralph-config-summary">
|
||||||
|
<div class="config-summary-row">
|
||||||
|
<span class="config-label">Completion Phrase:</span>
|
||||||
|
<code id="summaryPhrase">COMPLETE</code>
|
||||||
|
</div>
|
||||||
|
<div class="config-summary-row">
|
||||||
|
<span class="config-label">Max Iterations:</span>
|
||||||
|
<span id="summaryIterations">10</span>
|
||||||
|
</div>
|
||||||
|
<div class="config-summary-row">
|
||||||
|
<span class="config-label">Case:</span>
|
||||||
|
<span id="summaryCase">testcase</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Advanced Options (Collapsible) -->
|
||||||
|
<details class="advanced-options">
|
||||||
|
<summary>Advanced Options</summary>
|
||||||
|
<div class="advanced-options-content">
|
||||||
|
<div class="form-row form-row-switch">
|
||||||
|
<label>Enable Respawn</label>
|
||||||
|
<label class="switch">
|
||||||
|
<input type="checkbox" id="ralphEnableRespawn" checked>
|
||||||
|
<span class="slider"></span>
|
||||||
|
</label>
|
||||||
|
<span class="form-hint">Auto-restart sessions when context fills up</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</details>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Wizard Footer -->
|
||||||
|
<div class="wizard-footer">
|
||||||
|
<button class="btn-toolbar" id="ralphBackBtn" onclick="app.ralphWizardBack()" style="display: none;">Back</button>
|
||||||
|
<div class="wizard-footer-right">
|
||||||
|
<button class="btn-toolbar" onclick="app.closeRalphWizard()">Cancel</button>
|
||||||
|
<button class="btn-toolbar btn-primary" id="ralphNextBtn" onclick="app.ralphWizardNext()">Next</button>
|
||||||
|
<button class="btn-toolbar btn-success" id="ralphStartBtn" onclick="app.startRalphLoop()" style="display: none;">Start Loop</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<!-- Kill All Confirmation Modal -->
|
<!-- Kill All Confirmation Modal -->
|
||||||
<div class="modal" id="killAllModal">
|
<div class="modal" id="killAllModal">
|
||||||
<div class="modal-backdrop" onclick="app.closeKillAllModal()"></div>
|
<div class="modal-backdrop" onclick="app.closeKillAllModal()"></div>
|
||||||
|
|||||||
@@ -67,6 +67,13 @@ body {
|
|||||||
font-weight: 600;
|
font-weight: 600;
|
||||||
color: var(--accent-hover);
|
color: var(--accent-hover);
|
||||||
letter-spacing: -0.02em;
|
letter-spacing: -0.02em;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: color 0.15s, text-shadow 0.15s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.logo:hover {
|
||||||
|
color: #93c5fd;
|
||||||
|
text-shadow: 0 0 8px rgba(96, 165, 250, 0.5);
|
||||||
}
|
}
|
||||||
|
|
||||||
/* Session Tabs */
|
/* Session Tabs */
|
||||||
@@ -1435,6 +1442,21 @@ body {
|
|||||||
color: #a8d8a8;
|
color: #a8d8a8;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Ralph Loop button - golden/yellow theme */
|
||||||
|
.btn-toolbar.btn-ralph {
|
||||||
|
background: linear-gradient(135deg, #3d3415 0%, #4a3f1a 100%);
|
||||||
|
border-color: #6b5a1e;
|
||||||
|
color: #eab308;
|
||||||
|
font-weight: 500;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-toolbar.btn-ralph:hover {
|
||||||
|
background: linear-gradient(135deg, #4a3f1a 0%, #5a4d20 100%);
|
||||||
|
border-color: #8b7a2e;
|
||||||
|
box-shadow: 0 0 8px rgba(234, 179, 8, 0.4);
|
||||||
|
color: #fcd34d;
|
||||||
|
}
|
||||||
|
|
||||||
/* Button icon styling */
|
/* Button icon styling */
|
||||||
.btn-toolbar .btn-icon {
|
.btn-toolbar .btn-icon {
|
||||||
display: inline-flex;
|
display: inline-flex;
|
||||||
@@ -4600,3 +4622,259 @@ kbd {
|
|||||||
content: "📄 Click to stream: ";
|
content: "📄 Click to stream: ";
|
||||||
color: var(--accent);
|
color: var(--accent);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* ========== Ralph Loop Wizard ========== */
|
||||||
|
|
||||||
|
/* Wizard Progress Indicator */
|
||||||
|
.wizard-progress {
|
||||||
|
display: flex;
|
||||||
|
justify-content: center;
|
||||||
|
gap: 2rem;
|
||||||
|
padding: 0.75rem 1rem;
|
||||||
|
background: var(--bg-dark);
|
||||||
|
border-bottom: 1px solid var(--border);
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-step {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
color: var(--text-muted);
|
||||||
|
font-size: 0.8rem;
|
||||||
|
transition: color 0.2s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-step.active {
|
||||||
|
color: var(--accent);
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-step.completed {
|
||||||
|
color: var(--green);
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-step-number {
|
||||||
|
width: 24px;
|
||||||
|
height: 24px;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
background: var(--bg-input);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: 50%;
|
||||||
|
font-size: 0.75rem;
|
||||||
|
font-weight: 600;
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-step.active .wizard-step-number {
|
||||||
|
background: var(--accent);
|
||||||
|
border-color: var(--accent);
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-step.completed .wizard-step-number {
|
||||||
|
background: var(--green);
|
||||||
|
border-color: var(--green);
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-step-label {
|
||||||
|
font-weight: 500;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Wizard Pages */
|
||||||
|
.wizard-page {
|
||||||
|
min-height: 280px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-page.hidden {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Wizard Footer */
|
||||||
|
.wizard-footer {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
align-items: center;
|
||||||
|
padding: 0.75rem 1rem;
|
||||||
|
border-top: 1px solid var(--border);
|
||||||
|
background: var(--bg-card);
|
||||||
|
}
|
||||||
|
|
||||||
|
.wizard-footer-right {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Textarea with AI Assist button */
|
||||||
|
.textarea-with-assist {
|
||||||
|
position: relative;
|
||||||
|
}
|
||||||
|
|
||||||
|
.textarea-with-assist textarea {
|
||||||
|
width: 100%;
|
||||||
|
padding: 0.75rem;
|
||||||
|
background: var(--bg-input);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: 4px;
|
||||||
|
color: var(--text);
|
||||||
|
font-size: 0.85rem;
|
||||||
|
font-family: 'SF Mono', Monaco, 'Courier New', monospace;
|
||||||
|
resize: vertical;
|
||||||
|
min-height: 140px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.textarea-with-assist textarea:focus {
|
||||||
|
outline: none;
|
||||||
|
border-color: var(--accent);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Phrase Input Group */
|
||||||
|
.phrase-input-group {
|
||||||
|
position: relative;
|
||||||
|
}
|
||||||
|
|
||||||
|
.phrase-input-group input {
|
||||||
|
width: 100%;
|
||||||
|
padding: 0.5rem 0.75rem;
|
||||||
|
background: var(--bg-input);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: 4px;
|
||||||
|
color: var(--text);
|
||||||
|
font-size: 0.85rem;
|
||||||
|
font-family: 'SF Mono', Monaco, 'Courier New', monospace;
|
||||||
|
}
|
||||||
|
|
||||||
|
.phrase-input-group input:focus {
|
||||||
|
outline: none;
|
||||||
|
border-color: var(--accent);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Iteration Presets */
|
||||||
|
.iteration-presets {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.35rem;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.iteration-preset-btn {
|
||||||
|
padding: 0.5rem 1rem;
|
||||||
|
font-size: 0.8rem;
|
||||||
|
font-weight: 500;
|
||||||
|
background: var(--bg-input);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: 4px;
|
||||||
|
color: var(--text-dim);
|
||||||
|
cursor: pointer;
|
||||||
|
transition: all 0.15s;
|
||||||
|
min-width: 50px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.iteration-preset-btn:hover {
|
||||||
|
border-color: var(--border-light);
|
||||||
|
color: var(--text);
|
||||||
|
}
|
||||||
|
|
||||||
|
.iteration-preset-btn.active {
|
||||||
|
background: var(--accent);
|
||||||
|
border-color: var(--accent);
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Case Select Inline */
|
||||||
|
.case-select-inline {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.case-select-inline select {
|
||||||
|
flex: 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Ralph Prompt Preview */
|
||||||
|
.ralph-preview-section {
|
||||||
|
margin-bottom: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.ralph-preview-section > label {
|
||||||
|
display: block;
|
||||||
|
font-size: 0.7rem;
|
||||||
|
color: var(--text-dim);
|
||||||
|
margin-bottom: 0.35rem;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.03em;
|
||||||
|
}
|
||||||
|
|
||||||
|
.ralph-prompt-preview {
|
||||||
|
background: var(--bg-dark);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.75rem;
|
||||||
|
font-family: 'SF Mono', Monaco, 'Courier New', monospace;
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: var(--text-dim);
|
||||||
|
max-height: 200px;
|
||||||
|
overflow-y: auto;
|
||||||
|
white-space: pre-wrap;
|
||||||
|
word-break: break-word;
|
||||||
|
}
|
||||||
|
|
||||||
|
.ralph-prompt-preview .preview-highlight {
|
||||||
|
color: var(--accent);
|
||||||
|
font-weight: 500;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Config Summary */
|
||||||
|
.ralph-config-summary {
|
||||||
|
background: var(--bg-input);
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.75rem;
|
||||||
|
margin-bottom: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.config-summary-row {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
padding: 0.35rem 0;
|
||||||
|
font-size: 0.8rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.config-summary-row:not(:last-child) {
|
||||||
|
border-bottom: 1px solid var(--border);
|
||||||
|
}
|
||||||
|
|
||||||
|
.config-label {
|
||||||
|
color: var(--text-dim);
|
||||||
|
}
|
||||||
|
|
||||||
|
.config-summary-row code {
|
||||||
|
background: var(--bg-dark);
|
||||||
|
padding: 0.15rem 0.4rem;
|
||||||
|
border-radius: 3px;
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: var(--green);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Advanced Options */
|
||||||
|
.advanced-options {
|
||||||
|
margin-top: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.advanced-options summary {
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.8rem;
|
||||||
|
color: var(--text-dim);
|
||||||
|
padding: 0.5rem 0;
|
||||||
|
border-top: 1px solid var(--border);
|
||||||
|
}
|
||||||
|
|
||||||
|
.advanced-options summary:hover {
|
||||||
|
color: var(--text);
|
||||||
|
}
|
||||||
|
|
||||||
|
.advanced-options[open] summary {
|
||||||
|
margin-bottom: 0.75rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.advanced-options-content {
|
||||||
|
padding-left: 0.5rem;
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,487 @@
|
|||||||
|
/**
|
||||||
|
* Agent-Browser Enhanced Test Suite
|
||||||
|
*
|
||||||
|
* Tests Claudeman web UI using agent-browser's AI-agent-friendly features:
|
||||||
|
* - Semantic locators (find by role, label, text)
|
||||||
|
* - Accessibility tree snapshots
|
||||||
|
* - Reference-based element selection (@e1, @e2)
|
||||||
|
*
|
||||||
|
* This demonstrates the "agentic" approach to browser testing where an AI
|
||||||
|
* could navigate the UI using natural language descriptions.
|
||||||
|
*
|
||||||
|
* Port: 3157 (see CLAUDE.md test port table)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, it, expect, beforeAll, afterAll, beforeEach, afterEach } from 'vitest';
|
||||||
|
import { execSync, spawnSync } from 'node:child_process';
|
||||||
|
import { WebServer } from '../src/web/server.js';
|
||||||
|
|
||||||
|
const TEST_PORT = 3157;
|
||||||
|
const BASE_URL = `http://localhost:${TEST_PORT}`;
|
||||||
|
const BROWSER_TIMEOUT = 30000;
|
||||||
|
|
||||||
|
// Metrics collection
|
||||||
|
interface TestMetrics {
|
||||||
|
testName: string;
|
||||||
|
duration: number;
|
||||||
|
framework: 'agent-browser';
|
||||||
|
success: boolean;
|
||||||
|
error?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const metrics: TestMetrics[] = [];
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Agent-Browser CLI Helpers
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
function agentBrowser(command: string): string {
|
||||||
|
try {
|
||||||
|
return execSync(`npx agent-browser ${command}`, {
|
||||||
|
timeout: BROWSER_TIMEOUT,
|
||||||
|
encoding: 'utf-8',
|
||||||
|
stdio: ['pipe', 'pipe', 'pipe'],
|
||||||
|
}).trim();
|
||||||
|
} catch (error: any) {
|
||||||
|
if (error.stderr) {
|
||||||
|
throw new Error(`agent-browser failed: ${error.stderr}`);
|
||||||
|
}
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function agentBrowserJson<T = any>(command: string): T {
|
||||||
|
const result = agentBrowser(`${command} --json`);
|
||||||
|
const parsed = JSON.parse(result);
|
||||||
|
if (!parsed.success) {
|
||||||
|
throw new Error(`agent-browser command failed: ${parsed.error || 'unknown error'}`);
|
||||||
|
}
|
||||||
|
return parsed.data;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get accessibility tree snapshot for AI-based navigation
|
||||||
|
function getSnapshot(): string {
|
||||||
|
try {
|
||||||
|
return agentBrowser('snapshot');
|
||||||
|
} catch {
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Find element by semantic criteria (AI-friendly)
|
||||||
|
function findByRole(role: string, name?: string): boolean {
|
||||||
|
try {
|
||||||
|
const cmd = name
|
||||||
|
? `find role ${role} --name "${name}"`
|
||||||
|
: `find role ${role}`;
|
||||||
|
const result = agentBrowser(cmd);
|
||||||
|
return result.includes('Found') || !result.includes('not found');
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function findByText(text: string): boolean {
|
||||||
|
try {
|
||||||
|
const result = agentBrowser(`find text "${text}"`);
|
||||||
|
return result.includes('Found') || !result.includes('not found');
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function findByLabel(label: string): boolean {
|
||||||
|
try {
|
||||||
|
const result = agentBrowser(`find label "${label}"`);
|
||||||
|
return result.includes('Found') || !result.includes('not found');
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Click using semantic finder
|
||||||
|
function clickByText(text: string): boolean {
|
||||||
|
try {
|
||||||
|
agentBrowser(`find text "${text}" click`);
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function clickByRole(role: string, name?: string): boolean {
|
||||||
|
try {
|
||||||
|
const cmd = name
|
||||||
|
? `find role ${role} click --name "${name}"`
|
||||||
|
: `find role ${role} click`;
|
||||||
|
agentBrowser(cmd);
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Standard helpers
|
||||||
|
function getText(selector: string): string {
|
||||||
|
try {
|
||||||
|
return agentBrowserJson<{ text: string }>(`get text "${selector}"`).text || '';
|
||||||
|
} catch {
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function isVisible(selector: string): boolean {
|
||||||
|
try {
|
||||||
|
return agentBrowserJson<{ visible: boolean }>(`is visible "${selector}"`).visible;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function getCount(selector: string): number {
|
||||||
|
try {
|
||||||
|
return agentBrowserJson<{ count: number }>(`get count "${selector}"`).count;
|
||||||
|
} catch {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function waitForElement(selector: string, timeout = 10000): Promise<boolean> {
|
||||||
|
const start = Date.now();
|
||||||
|
while (Date.now() - start < timeout) {
|
||||||
|
if (getCount(selector) > 0) return true;
|
||||||
|
await new Promise(r => setTimeout(r, 500));
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
function closeBrowser() {
|
||||||
|
try {
|
||||||
|
agentBrowser('close');
|
||||||
|
} catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('Agent-Browser Enhanced Tests', () => {
|
||||||
|
let server: WebServer;
|
||||||
|
let createdSessions: string[] = [];
|
||||||
|
let browserAvailable = false;
|
||||||
|
let testStartTime: number;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
closeBrowser();
|
||||||
|
|
||||||
|
// Start web server
|
||||||
|
server = new WebServer(TEST_PORT);
|
||||||
|
await server.start();
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
|
||||||
|
// Test if agent-browser is available
|
||||||
|
try {
|
||||||
|
agentBrowser(`open ${BASE_URL}`);
|
||||||
|
await new Promise(r => setTimeout(r, 2000));
|
||||||
|
const title = agentBrowserJson<{ title: string }>('get title');
|
||||||
|
browserAvailable = title.title === 'Claudeman';
|
||||||
|
} catch (e) {
|
||||||
|
console.warn('Agent-browser not available, skipping tests:', (e as Error).message);
|
||||||
|
browserAvailable = false;
|
||||||
|
}
|
||||||
|
}, 60000);
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
closeBrowser();
|
||||||
|
|
||||||
|
// Cleanup sessions
|
||||||
|
for (const sessionId of createdSessions) {
|
||||||
|
try {
|
||||||
|
await fetch(`${BASE_URL}/api/sessions/${sessionId}`, { method: 'DELETE' });
|
||||||
|
} catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
await server.stop();
|
||||||
|
|
||||||
|
// Output metrics
|
||||||
|
console.log('\n📊 Agent-Browser Test Metrics:');
|
||||||
|
console.table(metrics.map(m => ({
|
||||||
|
test: m.testName.substring(0, 40),
|
||||||
|
duration: `${m.duration}ms`,
|
||||||
|
status: m.success ? '✅' : '❌',
|
||||||
|
})));
|
||||||
|
}, 60000);
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
testStartTime = Date.now();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
const testInfo = expect.getState();
|
||||||
|
metrics.push({
|
||||||
|
testName: testInfo.currentTestName || 'unknown',
|
||||||
|
duration: Date.now() - testStartTime,
|
||||||
|
framework: 'agent-browser',
|
||||||
|
success: !testInfo.assertionCalls?.some((c: any) => !c.pass),
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Core Page Tests (CSS Selectors)
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should load the main page with correct title', async () => {
|
||||||
|
if (!browserAvailable) {
|
||||||
|
console.log('Skipping: browser not available');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const title = agentBrowserJson<{ title: string }>('get title');
|
||||||
|
expect(title.title).toBe('Claudeman');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should display the logo and branding', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
const logoText = getText('.header-brand .logo');
|
||||||
|
expect(logoText).toBe('Claudeman');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should show version in toolbar', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
const version = getText('#versionDisplay');
|
||||||
|
expect(version).toMatch(/v\d+\.\d+/);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Semantic/AI-Friendly Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should find elements using semantic role locators', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
// Find buttons by role
|
||||||
|
const hasButtons = findByRole('button');
|
||||||
|
expect(hasButtons).toBe(true);
|
||||||
|
|
||||||
|
// Find specific buttons (may vary based on page structure)
|
||||||
|
// agent-browser uses ARIA roles for semantic finding
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should capture accessibility tree snapshot', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
const snapshot = getSnapshot();
|
||||||
|
|
||||||
|
// Snapshot should contain accessibility information
|
||||||
|
expect(snapshot.length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// Should include key page elements
|
||||||
|
// The snapshot format varies by agent-browser version
|
||||||
|
console.log('Accessibility snapshot captured, length:', snapshot.length);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should find elements by text content', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
// Find elements containing specific text
|
||||||
|
const foundClaudeman = findByText('Claudeman');
|
||||||
|
expect(foundClaudeman).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Modal Tests (CSS + Semantic)
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should open and close help modal', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
// Open using CSS selector
|
||||||
|
agentBrowser('click ".help-btn"');
|
||||||
|
await new Promise(r => setTimeout(r, 500));
|
||||||
|
|
||||||
|
expect(isVisible('#helpModal .modal-content')).toBe(true);
|
||||||
|
|
||||||
|
// Find heading by text
|
||||||
|
const headingText = getText('#helpModal h3');
|
||||||
|
expect(headingText).toContain('Keyboard Shortcuts');
|
||||||
|
|
||||||
|
// Close modal
|
||||||
|
agentBrowser('click "#helpModal .modal-close"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
|
||||||
|
expect(isVisible('#helpModal .modal-content')).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should open settings modal with correct tabs', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
agentBrowser('click ".btn-settings"');
|
||||||
|
await new Promise(r => setTimeout(r, 500));
|
||||||
|
|
||||||
|
expect(isVisible('#appSettingsModal .modal-content')).toBe(true);
|
||||||
|
|
||||||
|
const tabCount = getCount('#appSettingsModal .modal-tab-btn');
|
||||||
|
expect(tabCount).toBe(4);
|
||||||
|
|
||||||
|
agentBrowser('click "#appSettingsModal .modal-close"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Session Management Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should create a new session via button click', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
// Click create session
|
||||||
|
agentBrowser('click ".btn-claude"');
|
||||||
|
|
||||||
|
// Wait for session tab
|
||||||
|
const tabFound = await waitForElement('.session-tab', 15000);
|
||||||
|
expect(tabFound).toBe(true);
|
||||||
|
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
expect(isVisible('.session-tab.active')).toBe(true);
|
||||||
|
|
||||||
|
// Terminal should be visible
|
||||||
|
const xtermVisible = await waitForElement('.xterm', 10000);
|
||||||
|
expect(xtermVisible).toBe(true);
|
||||||
|
|
||||||
|
// Track for cleanup
|
||||||
|
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
if (data.sessions?.length > 0) {
|
||||||
|
const lastSession = data.sessions[data.sessions.length - 1];
|
||||||
|
if (lastSession && !createdSessions.includes(lastSession.id)) {
|
||||||
|
createdSessions.push(lastSession.id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// AI-Driven Navigation Demo
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should demonstrate AI-friendly navigation workflow', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
// This test demonstrates how an AI agent could navigate the UI
|
||||||
|
// using semantic locators and natural language-like commands
|
||||||
|
|
||||||
|
// Step 1: Check page state via snapshot
|
||||||
|
const snapshot = getSnapshot();
|
||||||
|
expect(snapshot.length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// Step 2: Find and interact with settings
|
||||||
|
// An AI would parse the snapshot and decide what to click
|
||||||
|
const settingsVisible = isVisible('.btn-settings');
|
||||||
|
expect(settingsVisible).toBe(true);
|
||||||
|
|
||||||
|
// Step 3: Open settings
|
||||||
|
agentBrowser('click ".btn-settings"');
|
||||||
|
await new Promise(r => setTimeout(r, 500));
|
||||||
|
|
||||||
|
// Step 4: Take new snapshot to understand modal state
|
||||||
|
const modalSnapshot = getSnapshot();
|
||||||
|
|
||||||
|
// Step 5: Close settings
|
||||||
|
agentBrowser('click "#appSettingsModal .modal-close"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
|
||||||
|
// This workflow shows how an AI could:
|
||||||
|
// 1. Observe the page state (snapshot)
|
||||||
|
// 2. Decide on actions based on semantic understanding
|
||||||
|
// 3. Execute actions
|
||||||
|
// 4. Verify results with new snapshot
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Form Interaction Tests (Semantic)
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should interact with quick start dropdown', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
// Find the quick start dropdown
|
||||||
|
const dropdownVisible = isVisible('#quickStartCase');
|
||||||
|
expect(dropdownVisible).toBe(true);
|
||||||
|
|
||||||
|
// Get current value
|
||||||
|
// Note: agent-browser's get value command for select elements
|
||||||
|
try {
|
||||||
|
const value = agentBrowserJson<{ value: string }>('get value "#quickStartCase"');
|
||||||
|
// Value should be the current selected case
|
||||||
|
expect(typeof value.value).toBe('string');
|
||||||
|
} catch {
|
||||||
|
// May not be a standard select element
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Multiple Element Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should count toolbar elements correctly', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
// Count various toolbar elements
|
||||||
|
const fontControls = getCount('.header-font-controls .font-btn');
|
||||||
|
expect(fontControls).toBeGreaterThanOrEqual(2);
|
||||||
|
|
||||||
|
// Tab count badge should exist
|
||||||
|
expect(isVisible('#tabCount')).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Screenshot Test
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should capture screenshot for AI analysis', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Capture screenshot (useful for visual AI analysis)
|
||||||
|
agentBrowser('screenshot /tmp/claudeman-test.png');
|
||||||
|
|
||||||
|
// Verify file was created (would exist after successful screenshot)
|
||||||
|
// In a real AI scenario, this image would be analyzed
|
||||||
|
} catch {
|
||||||
|
// Screenshot may fail in headless mode without proper setup
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Stress Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should handle rapid interactions', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
// Rapid modal open/close
|
||||||
|
for (let i = 0; i < 3; i++) {
|
||||||
|
agentBrowser('click ".help-btn"');
|
||||||
|
await new Promise(r => setTimeout(r, 200));
|
||||||
|
try {
|
||||||
|
agentBrowser('click "#helpModal .modal-close"');
|
||||||
|
} catch { /* may already be closed */ }
|
||||||
|
await new Promise(r => setTimeout(r, 200));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Page should still be functional
|
||||||
|
expect(isVisible('.header')).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Tab Navigation Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should handle tab count display', async () => {
|
||||||
|
if (!browserAvailable) return;
|
||||||
|
|
||||||
|
const tabCountText = getText('#tabCount');
|
||||||
|
// Should show number of tabs
|
||||||
|
expect(tabCountText).toMatch(/\d+/);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Export metrics for comparison
|
||||||
|
export { metrics as agentBrowserMetrics };
|
||||||
@@ -0,0 +1,624 @@
|
|||||||
|
/**
|
||||||
|
* Browser Framework Comparison Benchmark
|
||||||
|
*
|
||||||
|
* Runs identical test scenarios across Playwright, Puppeteer, and Agent-Browser
|
||||||
|
* to compare performance, reliability, and ease of use for agentic testing.
|
||||||
|
*
|
||||||
|
* This benchmark measures:
|
||||||
|
* - Execution time per operation
|
||||||
|
* - Reliability (success rate)
|
||||||
|
* - Memory usage (where available)
|
||||||
|
* - Ease of implementation (qualitative)
|
||||||
|
*
|
||||||
|
* Ports: 3158 (Playwright), 3159 (Puppeteer), 3160 (Agent-Browser)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||||
|
import { chromium, Browser as PlaywrightBrowser, Page as PlaywrightPage } from 'playwright';
|
||||||
|
import puppeteer, { Browser as PuppeteerBrowser, Page as PuppeteerPage } from 'puppeteer';
|
||||||
|
import { execSync } from 'node:child_process';
|
||||||
|
import { WebServer } from '../src/web/server.js';
|
||||||
|
|
||||||
|
// Port allocation
|
||||||
|
const PORTS = {
|
||||||
|
playwright: 3158,
|
||||||
|
puppeteer: 3159,
|
||||||
|
agentBrowser: 3160,
|
||||||
|
};
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Types
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
interface BenchmarkResult {
|
||||||
|
scenario: string;
|
||||||
|
framework: 'playwright' | 'puppeteer' | 'agent-browser';
|
||||||
|
duration: number;
|
||||||
|
success: boolean;
|
||||||
|
error?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface AggregateStats {
|
||||||
|
framework: string;
|
||||||
|
totalTests: number;
|
||||||
|
passed: number;
|
||||||
|
failed: number;
|
||||||
|
avgDuration: number;
|
||||||
|
minDuration: number;
|
||||||
|
maxDuration: number;
|
||||||
|
successRate: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const results: BenchmarkResult[] = [];
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Agent-Browser Helpers
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
function agentBrowser(command: string): string {
|
||||||
|
try {
|
||||||
|
return execSync(`npx agent-browser ${command}`, {
|
||||||
|
timeout: 30000,
|
||||||
|
encoding: 'utf-8',
|
||||||
|
stdio: ['pipe', 'pipe', 'pipe'],
|
||||||
|
}).trim();
|
||||||
|
} catch (error: any) {
|
||||||
|
throw new Error(`agent-browser failed: ${error.stderr || error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function agentBrowserJson<T>(command: string): T {
|
||||||
|
const result = agentBrowser(`${command} --json`);
|
||||||
|
const parsed = JSON.parse(result);
|
||||||
|
if (!parsed.success) {
|
||||||
|
throw new Error(`agent-browser: ${parsed.error || 'unknown error'}`);
|
||||||
|
}
|
||||||
|
return parsed.data;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Benchmark Scenarios
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Each scenario is a function that takes framework-specific parameters
|
||||||
|
* and returns a promise that resolves when the scenario is complete.
|
||||||
|
*/
|
||||||
|
|
||||||
|
async function runBenchmark(
|
||||||
|
scenario: string,
|
||||||
|
framework: 'playwright' | 'puppeteer' | 'agent-browser',
|
||||||
|
fn: () => Promise<void>
|
||||||
|
): Promise<void> {
|
||||||
|
const start = Date.now();
|
||||||
|
let success = false;
|
||||||
|
let error: string | undefined;
|
||||||
|
|
||||||
|
try {
|
||||||
|
await fn();
|
||||||
|
success = true;
|
||||||
|
} catch (e: any) {
|
||||||
|
error = e.message;
|
||||||
|
}
|
||||||
|
|
||||||
|
results.push({
|
||||||
|
scenario,
|
||||||
|
framework,
|
||||||
|
duration: Date.now() - start,
|
||||||
|
success,
|
||||||
|
error,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Main Comparison Suite
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
describe('Browser Framework Comparison Benchmark', () => {
|
||||||
|
// Framework instances
|
||||||
|
let playwrightBrowser: PlaywrightBrowser;
|
||||||
|
let puppeteerBrowser: PuppeteerBrowser;
|
||||||
|
let agentBrowserAvailable = false;
|
||||||
|
|
||||||
|
// Web servers
|
||||||
|
let servers: WebServer[] = [];
|
||||||
|
|
||||||
|
// Cleanup tracking
|
||||||
|
let createdSessions: { port: number; id: string }[] = [];
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
console.log('\n🚀 Starting Browser Framework Comparison Benchmark\n');
|
||||||
|
console.log('This benchmark compares:');
|
||||||
|
console.log(' - Playwright (direct API)');
|
||||||
|
console.log(' - Puppeteer (Chrome DevTools Protocol)');
|
||||||
|
console.log(' - Agent-Browser (CLI-based, AI-friendly)\n');
|
||||||
|
|
||||||
|
// Start web servers for each framework
|
||||||
|
for (const [name, port] of Object.entries(PORTS)) {
|
||||||
|
const server = new WebServer(port);
|
||||||
|
await server.start();
|
||||||
|
servers.push(server);
|
||||||
|
console.log(`✅ Started server for ${name} on port ${port}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
|
||||||
|
// Initialize Playwright
|
||||||
|
try {
|
||||||
|
playwrightBrowser = await chromium.launch({
|
||||||
|
headless: true,
|
||||||
|
args: ['--no-sandbox', '--disable-setuid-sandbox'],
|
||||||
|
});
|
||||||
|
console.log('✅ Playwright browser launched');
|
||||||
|
} catch (e: any) {
|
||||||
|
console.warn('⚠️ Playwright launch failed:', e.message);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Initialize Puppeteer
|
||||||
|
try {
|
||||||
|
puppeteerBrowser = await puppeteer.launch({
|
||||||
|
headless: true,
|
||||||
|
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||||
|
});
|
||||||
|
console.log('✅ Puppeteer browser launched');
|
||||||
|
} catch (e: any) {
|
||||||
|
console.warn('⚠️ Puppeteer launch failed:', e.message);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Test Agent-Browser availability
|
||||||
|
try {
|
||||||
|
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||||
|
await new Promise(r => setTimeout(r, 2000));
|
||||||
|
const title = agentBrowserJson<{ title: string }>('get title');
|
||||||
|
agentBrowserAvailable = title.title === 'Claudeman';
|
||||||
|
if (agentBrowserAvailable) {
|
||||||
|
console.log('✅ Agent-Browser available');
|
||||||
|
}
|
||||||
|
} catch (e: any) {
|
||||||
|
console.warn('⚠️ Agent-Browser not available:', e.message);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n' + '='.repeat(60) + '\n');
|
||||||
|
}, 120000);
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
// Cleanup sessions
|
||||||
|
for (const { port, id } of createdSessions) {
|
||||||
|
try {
|
||||||
|
await fetch(`http://localhost:${port}/api/sessions/${id}`, { method: 'DELETE' });
|
||||||
|
} catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close browsers
|
||||||
|
if (playwrightBrowser) await playwrightBrowser.close();
|
||||||
|
if (puppeteerBrowser) await puppeteerBrowser.close();
|
||||||
|
try { agentBrowser('close'); } catch { /* ignore */ }
|
||||||
|
|
||||||
|
// Stop servers
|
||||||
|
for (const server of servers) {
|
||||||
|
await server.stop();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Print results summary
|
||||||
|
printResultsSummary();
|
||||||
|
}, 60000);
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Scenario 1: Page Load
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
describe('Scenario: Page Load', () => {
|
||||||
|
it('Playwright - page load', async () => {
|
||||||
|
if (!playwrightBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('page-load', 'playwright', async () => {
|
||||||
|
const context = await playwrightBrowser.newContext();
|
||||||
|
const page = await context.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
await expect(page).toHaveTitle('Claudeman');
|
||||||
|
await context.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Puppeteer - page load', async () => {
|
||||||
|
if (!puppeteerBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('page-load', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
const title = await page.title();
|
||||||
|
expect(title).toBe('Claudeman');
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Agent-Browser - page load', async () => {
|
||||||
|
if (!agentBrowserAvailable) return;
|
||||||
|
|
||||||
|
await runBenchmark('page-load', 'agent-browser', async () => {
|
||||||
|
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
const title = agentBrowserJson<{ title: string }>('get title');
|
||||||
|
expect(title.title).toBe('Claudeman');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Scenario 2: Element Selection
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
describe('Scenario: Element Selection', () => {
|
||||||
|
it('Playwright - element selection', async () => {
|
||||||
|
if (!playwrightBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('element-selection', 'playwright', async () => {
|
||||||
|
const context = await playwrightBrowser.newContext();
|
||||||
|
const page = await context.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
|
||||||
|
// Multiple element selections
|
||||||
|
const logo = await page.locator('.header-brand .logo').textContent();
|
||||||
|
const version = await page.locator('#versionDisplay').textContent();
|
||||||
|
const btnCount = await page.locator('.btn-claude').count();
|
||||||
|
|
||||||
|
expect(logo).toBe('Claudeman');
|
||||||
|
expect(version).toMatch(/v\d+/);
|
||||||
|
expect(btnCount).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
await context.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Puppeteer - element selection', async () => {
|
||||||
|
if (!puppeteerBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('element-selection', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
|
||||||
|
const logo = await page.$eval('.header-brand .logo', el => el.textContent);
|
||||||
|
const version = await page.$eval('#versionDisplay', el => el.textContent);
|
||||||
|
const btns = await page.$$('.btn-claude');
|
||||||
|
|
||||||
|
expect(logo).toBe('Claudeman');
|
||||||
|
expect(version).toMatch(/v\d+/);
|
||||||
|
expect(btns.length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Agent-Browser - element selection', async () => {
|
||||||
|
if (!agentBrowserAvailable) return;
|
||||||
|
|
||||||
|
await runBenchmark('element-selection', 'agent-browser', async () => {
|
||||||
|
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
|
||||||
|
const logo = agentBrowserJson<{ text: string }>('get text ".header-brand .logo"');
|
||||||
|
const version = agentBrowserJson<{ text: string }>('get text "#versionDisplay"');
|
||||||
|
const btnCount = agentBrowserJson<{ count: number }>('get count ".btn-claude"');
|
||||||
|
|
||||||
|
expect(logo.text).toBe('Claudeman');
|
||||||
|
expect(version.text).toMatch(/v\d+/);
|
||||||
|
expect(btnCount.count).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Scenario 3: Modal Interaction
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
describe('Scenario: Modal Open/Close', () => {
|
||||||
|
it('Playwright - modal interaction', async () => {
|
||||||
|
if (!playwrightBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('modal-interaction', 'playwright', async () => {
|
||||||
|
const context = await playwrightBrowser.newContext();
|
||||||
|
const page = await context.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
|
||||||
|
// Open modal
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||||
|
|
||||||
|
// Verify content
|
||||||
|
const heading = await page.locator('#helpModal h3').textContent();
|
||||||
|
expect(heading).toContain('Keyboard Shortcuts');
|
||||||
|
|
||||||
|
// Close modal
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||||
|
|
||||||
|
await context.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Puppeteer - modal interaction', async () => {
|
||||||
|
if (!puppeteerBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('modal-interaction', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
|
||||||
|
// Open modal
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||||
|
|
||||||
|
// Verify content
|
||||||
|
const heading = await page.$eval('#helpModal h3', el => el.textContent);
|
||||||
|
expect(heading).toContain('Keyboard Shortcuts');
|
||||||
|
|
||||||
|
// Close modal
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await page.waitForFunction(() => {
|
||||||
|
const modal = document.querySelector('#helpModal .modal-content');
|
||||||
|
return !modal || window.getComputedStyle(modal).display === 'none';
|
||||||
|
});
|
||||||
|
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Agent-Browser - modal interaction', async () => {
|
||||||
|
if (!agentBrowserAvailable) return;
|
||||||
|
|
||||||
|
await runBenchmark('modal-interaction', 'agent-browser', async () => {
|
||||||
|
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
|
||||||
|
// Open modal
|
||||||
|
agentBrowser('click ".help-btn"');
|
||||||
|
await new Promise(r => setTimeout(r, 500));
|
||||||
|
|
||||||
|
// Verify content
|
||||||
|
const heading = agentBrowserJson<{ text: string }>('get text "#helpModal h3"');
|
||||||
|
expect(heading.text).toContain('Keyboard Shortcuts');
|
||||||
|
|
||||||
|
// Close modal
|
||||||
|
agentBrowser('click "#helpModal .modal-close"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Scenario 4: Session Creation (Complex)
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
describe('Scenario: Session Creation', () => {
|
||||||
|
it('Playwright - session creation', async () => {
|
||||||
|
if (!playwrightBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('session-creation', 'playwright', async () => {
|
||||||
|
const context = await playwrightBrowser.newContext();
|
||||||
|
const page = await context.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
|
||||||
|
// Create session
|
||||||
|
await page.click('.btn-claude');
|
||||||
|
|
||||||
|
// Wait for session tab
|
||||||
|
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||||
|
|
||||||
|
// Verify terminal appears
|
||||||
|
await page.waitForSelector('.xterm', { timeout: 10000 });
|
||||||
|
|
||||||
|
// Track for cleanup
|
||||||
|
const response = await fetch(`http://localhost:${PORTS.playwright}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
const lastSession = data.sessions?.[data.sessions.length - 1];
|
||||||
|
if (lastSession) {
|
||||||
|
createdSessions.push({ port: PORTS.playwright, id: lastSession.id });
|
||||||
|
}
|
||||||
|
|
||||||
|
await context.close();
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
it('Puppeteer - session creation', async () => {
|
||||||
|
if (!puppeteerBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('session-creation', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
|
||||||
|
// Create session
|
||||||
|
await page.click('.btn-claude');
|
||||||
|
|
||||||
|
// Wait for session tab
|
||||||
|
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||||
|
|
||||||
|
// Verify terminal appears
|
||||||
|
await page.waitForSelector('.xterm', { timeout: 10000 });
|
||||||
|
|
||||||
|
// Track for cleanup
|
||||||
|
const response = await fetch(`http://localhost:${PORTS.puppeteer}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
const lastSession = data.sessions?.[data.sessions.length - 1];
|
||||||
|
if (lastSession) {
|
||||||
|
createdSessions.push({ port: PORTS.puppeteer, id: lastSession.id });
|
||||||
|
}
|
||||||
|
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
it('Agent-Browser - session creation', async () => {
|
||||||
|
if (!agentBrowserAvailable) return;
|
||||||
|
|
||||||
|
await runBenchmark('session-creation', 'agent-browser', async () => {
|
||||||
|
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
|
||||||
|
// Create session
|
||||||
|
agentBrowser('click ".btn-claude"');
|
||||||
|
|
||||||
|
// Wait for session tab
|
||||||
|
const start = Date.now();
|
||||||
|
while (Date.now() - start < 15000) {
|
||||||
|
try {
|
||||||
|
const count = agentBrowserJson<{ count: number }>('get count ".session-tab"');
|
||||||
|
if (count.count > 0) break;
|
||||||
|
} catch { /* retry */ }
|
||||||
|
await new Promise(r => setTimeout(r, 500));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify terminal appears
|
||||||
|
await new Promise(r => setTimeout(r, 2000));
|
||||||
|
const xtermCount = agentBrowserJson<{ count: number }>('get count ".xterm"');
|
||||||
|
expect(xtermCount.count).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// Track for cleanup
|
||||||
|
const response = await fetch(`http://localhost:${PORTS.agentBrowser}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
const lastSession = data.sessions?.[data.sessions.length - 1];
|
||||||
|
if (lastSession) {
|
||||||
|
createdSessions.push({ port: PORTS.agentBrowser, id: lastSession.id });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Scenario 5: Rapid Operations
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
describe('Scenario: Rapid Operations (5 modal cycles)', () => {
|
||||||
|
it('Playwright - rapid operations', async () => {
|
||||||
|
if (!playwrightBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('rapid-operations', 'playwright', async () => {
|
||||||
|
const context = await playwrightBrowser.newContext();
|
||||||
|
const page = await context.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||||
|
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||||
|
}
|
||||||
|
|
||||||
|
await context.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Puppeteer - rapid operations', async () => {
|
||||||
|
if (!puppeteerBrowser) return;
|
||||||
|
|
||||||
|
await runBenchmark('rapid-operations', 'puppeteer', async () => {
|
||||||
|
const page = await puppeteerBrowser.newPage();
|
||||||
|
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||||
|
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await new Promise(r => setTimeout(r, 200));
|
||||||
|
}
|
||||||
|
|
||||||
|
await page.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Agent-Browser - rapid operations', async () => {
|
||||||
|
if (!agentBrowserAvailable) return;
|
||||||
|
|
||||||
|
await runBenchmark('rapid-operations', 'agent-browser', async () => {
|
||||||
|
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
agentBrowser('click ".help-btn"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
agentBrowser('click "#helpModal .modal-close"');
|
||||||
|
await new Promise(r => setTimeout(r, 300));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Results Summary
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
function printResultsSummary(): void {
|
||||||
|
console.log('\n' + '='.repeat(70));
|
||||||
|
console.log('📊 BROWSER FRAMEWORK COMPARISON RESULTS');
|
||||||
|
console.log('='.repeat(70) + '\n');
|
||||||
|
|
||||||
|
// Group by framework
|
||||||
|
const frameworks = ['playwright', 'puppeteer', 'agent-browser'] as const;
|
||||||
|
const stats: AggregateStats[] = [];
|
||||||
|
|
||||||
|
for (const framework of frameworks) {
|
||||||
|
const frameworkResults = results.filter(r => r.framework === framework);
|
||||||
|
if (frameworkResults.length === 0) continue;
|
||||||
|
|
||||||
|
const passed = frameworkResults.filter(r => r.success).length;
|
||||||
|
const durations = frameworkResults.map(r => r.duration);
|
||||||
|
|
||||||
|
stats.push({
|
||||||
|
framework,
|
||||||
|
totalTests: frameworkResults.length,
|
||||||
|
passed,
|
||||||
|
failed: frameworkResults.length - passed,
|
||||||
|
avgDuration: Math.round(durations.reduce((a, b) => a + b, 0) / durations.length),
|
||||||
|
minDuration: Math.min(...durations),
|
||||||
|
maxDuration: Math.max(...durations),
|
||||||
|
successRate: ((passed / frameworkResults.length) * 100).toFixed(1) + '%',
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Print aggregate stats
|
||||||
|
console.log('📈 AGGREGATE STATISTICS\n');
|
||||||
|
console.table(stats.map(s => ({
|
||||||
|
Framework: s.framework.toUpperCase(),
|
||||||
|
Tests: s.totalTests,
|
||||||
|
Passed: s.passed,
|
||||||
|
Failed: s.failed,
|
||||||
|
'Success Rate': s.successRate,
|
||||||
|
'Avg (ms)': s.avgDuration,
|
||||||
|
'Min (ms)': s.minDuration,
|
||||||
|
'Max (ms)': s.maxDuration,
|
||||||
|
})));
|
||||||
|
|
||||||
|
// Print detailed results by scenario
|
||||||
|
console.log('\n📋 DETAILED RESULTS BY SCENARIO\n');
|
||||||
|
|
||||||
|
const scenarios = [...new Set(results.map(r => r.scenario))];
|
||||||
|
for (const scenario of scenarios) {
|
||||||
|
console.log(`\n🔹 ${scenario.toUpperCase()}`);
|
||||||
|
const scenarioResults = results.filter(r => r.scenario === scenario);
|
||||||
|
|
||||||
|
console.table(scenarioResults.map(r => ({
|
||||||
|
Framework: r.framework,
|
||||||
|
Duration: `${r.duration}ms`,
|
||||||
|
Status: r.success ? '✅ Pass' : '❌ Fail',
|
||||||
|
Error: r.error ? r.error.substring(0, 40) + '...' : '-',
|
||||||
|
})));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Performance comparison
|
||||||
|
console.log('\n🏆 PERFORMANCE RANKING (by avg duration)\n');
|
||||||
|
const ranked = [...stats].sort((a, b) => a.avgDuration - b.avgDuration);
|
||||||
|
ranked.forEach((s, i) => {
|
||||||
|
const medal = i === 0 ? '🥇' : i === 1 ? '🥈' : '🥉';
|
||||||
|
console.log(`${medal} ${s.framework.toUpperCase()}: ${s.avgDuration}ms avg (${s.successRate} success rate)`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Recommendations
|
||||||
|
console.log('\n💡 RECOMMENDATIONS\n');
|
||||||
|
console.log('• Playwright: Best for comprehensive testing with excellent debugging');
|
||||||
|
console.log('• Puppeteer: Best for Chrome-specific features and CDP access');
|
||||||
|
console.log('• Agent-Browser: Best for AI agents with semantic navigation needs');
|
||||||
|
|
||||||
|
console.log('\n' + '='.repeat(70) + '\n');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Export results for external analysis
|
||||||
|
export { results as comparisonResults };
|
||||||
@@ -0,0 +1,194 @@
|
|||||||
|
/**
|
||||||
|
* Playwright Direct API Test Suite
|
||||||
|
*
|
||||||
|
* Tests Claudeman web UI using Playwright's native API directly.
|
||||||
|
* This provides the most control and best debugging capabilities.
|
||||||
|
*
|
||||||
|
* Port: 3155 (see CLAUDE.md test port table)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||||
|
import { chromium, Browser, Page, BrowserContext } from 'playwright';
|
||||||
|
import { WebServer } from '../src/web/server.js';
|
||||||
|
|
||||||
|
const TEST_PORT = 3155;
|
||||||
|
const BASE_URL = `http://localhost:${TEST_PORT}`;
|
||||||
|
|
||||||
|
// Metrics collection
|
||||||
|
interface TestMetrics {
|
||||||
|
testName: string;
|
||||||
|
duration: number;
|
||||||
|
framework: 'playwright';
|
||||||
|
success: boolean;
|
||||||
|
error?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const metrics: TestMetrics[] = [];
|
||||||
|
let server: WebServer;
|
||||||
|
let browser: Browser;
|
||||||
|
let createdSessions: string[] = [];
|
||||||
|
let setupComplete = false;
|
||||||
|
|
||||||
|
describe('Playwright Direct API Tests', () => {
|
||||||
|
beforeAll(async () => {
|
||||||
|
// Start web server
|
||||||
|
server = new WebServer(TEST_PORT);
|
||||||
|
await server.start();
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
|
||||||
|
// Launch browser
|
||||||
|
browser = await chromium.launch({
|
||||||
|
headless: true,
|
||||||
|
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||||
|
});
|
||||||
|
setupComplete = true;
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
// Cleanup sessions
|
||||||
|
for (const sessionId of createdSessions) {
|
||||||
|
try {
|
||||||
|
await fetch(`${BASE_URL}/api/sessions/${sessionId}`, { method: 'DELETE' });
|
||||||
|
} catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close browser and server
|
||||||
|
if (browser) await browser.close();
|
||||||
|
if (server) await server.stop();
|
||||||
|
|
||||||
|
// Output metrics
|
||||||
|
console.log('\n📊 Playwright Test Metrics:');
|
||||||
|
console.table(metrics.map(m => ({
|
||||||
|
test: m.testName.substring(0, 40),
|
||||||
|
duration: `${m.duration}ms`,
|
||||||
|
status: m.success ? '✅' : '❌',
|
||||||
|
})));
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
// Helper to run test with metrics
|
||||||
|
async function withMetrics(name: string, fn: (page: Page) => Promise<void>) {
|
||||||
|
if (!setupComplete) {
|
||||||
|
console.log('Skipping - setup not complete');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const start = Date.now();
|
||||||
|
let success = false;
|
||||||
|
let error: string | undefined;
|
||||||
|
const context = await browser.newContext({ viewport: { width: 1280, height: 720 } });
|
||||||
|
const page = await context.newPage();
|
||||||
|
try {
|
||||||
|
await fn(page);
|
||||||
|
success = true;
|
||||||
|
} catch (e: any) {
|
||||||
|
error = e.message;
|
||||||
|
throw e;
|
||||||
|
} finally {
|
||||||
|
await context.close();
|
||||||
|
metrics.push({ testName: name, duration: Date.now() - start, framework: 'playwright', success, error });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Core Page Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should load the main page with correct title', async () => {
|
||||||
|
await withMetrics('page-load', async (page) => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
await expect(page).toHaveTitle('Claudeman');
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
it('should display the logo and branding', async () => {
|
||||||
|
await withMetrics('logo-branding', async (page) => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
const logo = page.locator('.header-brand .logo');
|
||||||
|
await expect(logo).toBeVisible();
|
||||||
|
await expect(logo).toHaveText('Claudeman');
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
it('should show version in toolbar', async () => {
|
||||||
|
await withMetrics('version-display', async (page) => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
const version = page.locator('#versionDisplay');
|
||||||
|
await expect(version).toBeVisible();
|
||||||
|
await expect(version).toHaveText(/v\d+\.\d+/);
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Modal Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should open and close help modal', async () => {
|
||||||
|
await withMetrics('help-modal', async (page) => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
await page.click('.help-btn');
|
||||||
|
const modal = page.locator('#helpModal .modal-content');
|
||||||
|
await expect(modal).toBeVisible();
|
||||||
|
const heading = page.locator('#helpModal h3');
|
||||||
|
await expect(heading).toContainText('Keyboard Shortcuts');
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await expect(modal).not.toBeVisible();
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
it('should open settings modal with 4 tabs', async () => {
|
||||||
|
await withMetrics('settings-modal', async (page) => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
await page.click('.btn-settings');
|
||||||
|
const modal = page.locator('#appSettingsModal .modal-content');
|
||||||
|
await expect(modal).toBeVisible();
|
||||||
|
const tabs = page.locator('#appSettingsModal .modal-tab-btn');
|
||||||
|
await expect(tabs).toHaveCount(4);
|
||||||
|
await page.click('#appSettingsModal .modal-close');
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Session Management Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should create a new session via button click', async () => {
|
||||||
|
await withMetrics('session-creation', async (page) => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
await page.click('.btn-claude');
|
||||||
|
const sessionTab = page.locator('.session-tab');
|
||||||
|
await expect(sessionTab.first()).toBeVisible({ timeout: 15000 });
|
||||||
|
await expect(page.locator('.session-tab.active')).toBeVisible();
|
||||||
|
const terminal = page.locator('.xterm');
|
||||||
|
await expect(terminal).toBeVisible({ timeout: 10000 });
|
||||||
|
|
||||||
|
// Track session for cleanup
|
||||||
|
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
if (data.sessions?.length > 0) {
|
||||||
|
const lastSession = data.sessions[data.sessions.length - 1];
|
||||||
|
if (lastSession && !createdSessions.includes(lastSession.id)) {
|
||||||
|
createdSessions.push(lastSession.id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}, 60000);
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Stress Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should handle rapid modal interactions', async () => {
|
||||||
|
await withMetrics('rapid-interactions', async (page) => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||||
|
}
|
||||||
|
await expect(page.locator('.header')).toBeVisible();
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Export metrics for comparison
|
||||||
|
export { metrics as playwrightMetrics };
|
||||||
@@ -0,0 +1,445 @@
|
|||||||
|
/**
|
||||||
|
* Puppeteer/Chrome Test Suite
|
||||||
|
*
|
||||||
|
* Tests Claudeman web UI using Puppeteer for direct Chrome DevTools Protocol access.
|
||||||
|
* Puppeteer provides lower-level Chrome control compared to Playwright.
|
||||||
|
*
|
||||||
|
* Port: 3156 (see CLAUDE.md test port table)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, it, expect, beforeAll, afterAll, beforeEach, afterEach } from 'vitest';
|
||||||
|
import puppeteer, { Browser, Page } from 'puppeteer';
|
||||||
|
import { WebServer } from '../src/web/server.js';
|
||||||
|
|
||||||
|
const TEST_PORT = 3156;
|
||||||
|
const BASE_URL = `http://localhost:${TEST_PORT}`;
|
||||||
|
|
||||||
|
// Metrics collection
|
||||||
|
interface TestMetrics {
|
||||||
|
testName: string;
|
||||||
|
duration: number;
|
||||||
|
framework: 'puppeteer';
|
||||||
|
success: boolean;
|
||||||
|
error?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const metrics: TestMetrics[] = [];
|
||||||
|
|
||||||
|
describe('Puppeteer/Chrome Tests', () => {
|
||||||
|
let server: WebServer;
|
||||||
|
let browser: Browser;
|
||||||
|
let page: Page;
|
||||||
|
let createdSessions: string[] = [];
|
||||||
|
let testStartTime: number;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
// Start web server
|
||||||
|
server = new WebServer(TEST_PORT);
|
||||||
|
await server.start();
|
||||||
|
await new Promise(r => setTimeout(r, 1000));
|
||||||
|
|
||||||
|
// Launch Chrome via Puppeteer
|
||||||
|
browser = await puppeteer.launch({
|
||||||
|
headless: true,
|
||||||
|
args: [
|
||||||
|
'--no-sandbox',
|
||||||
|
'--disable-setuid-sandbox',
|
||||||
|
'--disable-dev-shm-usage',
|
||||||
|
'--disable-gpu',
|
||||||
|
],
|
||||||
|
});
|
||||||
|
}, 60000);
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
// Cleanup sessions
|
||||||
|
for (const sessionId of createdSessions) {
|
||||||
|
try {
|
||||||
|
await fetch(`${BASE_URL}/api/sessions/${sessionId}`, { method: 'DELETE' });
|
||||||
|
} catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close browser and server
|
||||||
|
if (browser) await browser.close();
|
||||||
|
await server.stop();
|
||||||
|
|
||||||
|
// Output metrics
|
||||||
|
console.log('\n📊 Puppeteer Test Metrics:');
|
||||||
|
console.table(metrics.map(m => ({
|
||||||
|
test: m.testName.substring(0, 40),
|
||||||
|
duration: `${m.duration}ms`,
|
||||||
|
status: m.success ? '✅' : '❌',
|
||||||
|
})));
|
||||||
|
}, 60000);
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
page = await browser.newPage();
|
||||||
|
await page.setViewport({ width: 1280, height: 720 });
|
||||||
|
testStartTime = Date.now();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
const testInfo = expect.getState();
|
||||||
|
metrics.push({
|
||||||
|
testName: testInfo.currentTestName || 'unknown',
|
||||||
|
duration: Date.now() - testStartTime,
|
||||||
|
framework: 'puppeteer',
|
||||||
|
success: !testInfo.assertionCalls?.some((c: any) => !c.pass),
|
||||||
|
});
|
||||||
|
|
||||||
|
if (page) await page.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Core Page Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should load the main page with correct title', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
const title = await page.title();
|
||||||
|
expect(title).toBe('Claudeman');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should display the logo and branding', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
await page.waitForSelector('.header-brand .logo');
|
||||||
|
const logoText = await page.$eval('.header-brand .logo', el => el.textContent);
|
||||||
|
expect(logoText).toBe('Claudeman');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should show version in toolbar', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
await page.waitForSelector('#versionDisplay');
|
||||||
|
const version = await page.$eval('#versionDisplay', el => el.textContent);
|
||||||
|
expect(version).toMatch(/v\d+\.\d+/);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Button & Control Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should have all main control buttons visible', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
const btnClaude = await page.$('.btn-claude');
|
||||||
|
const btnSettings = await page.$('.btn-settings');
|
||||||
|
const helpBtn = await page.$('.help-btn');
|
||||||
|
const quickStart = await page.$('#quickStartCase');
|
||||||
|
|
||||||
|
expect(btnClaude).not.toBeNull();
|
||||||
|
expect(btnSettings).not.toBeNull();
|
||||||
|
expect(helpBtn).not.toBeNull();
|
||||||
|
expect(quickStart).not.toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should have font size controls', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
const fontControls = await page.$('.header-font-controls');
|
||||||
|
expect(fontControls).not.toBeNull();
|
||||||
|
|
||||||
|
const fontBtns = await page.$$('.header-font-controls .font-btn');
|
||||||
|
expect(fontBtns.length).toBeGreaterThanOrEqual(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Modal Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should open and close help modal', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
// Open help modal
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||||
|
|
||||||
|
// Check content
|
||||||
|
const heading = await page.$eval('#helpModal h3', el => el.textContent);
|
||||||
|
expect(heading).toContain('Keyboard Shortcuts');
|
||||||
|
|
||||||
|
// Check modal is visible
|
||||||
|
const isVisible = await page.$eval('#helpModal .modal-content', el => {
|
||||||
|
const style = window.getComputedStyle(el);
|
||||||
|
return style.display !== 'none' && style.visibility !== 'hidden';
|
||||||
|
});
|
||||||
|
expect(isVisible).toBe(true);
|
||||||
|
|
||||||
|
// Close modal
|
||||||
|
await page.click('#helpModal .modal-close');
|
||||||
|
await page.waitForFunction(() => {
|
||||||
|
const modal = document.querySelector('#helpModal .modal-content');
|
||||||
|
if (!modal) return true;
|
||||||
|
const style = window.getComputedStyle(modal);
|
||||||
|
return style.display === 'none' || style.visibility === 'hidden';
|
||||||
|
}, { timeout: 5000 }).catch(() => {});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should open settings modal with 4 tabs', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
// Open settings
|
||||||
|
await page.click('.btn-settings');
|
||||||
|
await page.waitForSelector('#appSettingsModal .modal-content', { visible: true });
|
||||||
|
|
||||||
|
// Check tabs
|
||||||
|
const tabs = await page.$$('#appSettingsModal .modal-tab-btn');
|
||||||
|
expect(tabs.length).toBe(4);
|
||||||
|
|
||||||
|
// Verify tab names
|
||||||
|
const tabTexts = await page.$$eval('#appSettingsModal .modal-tab-btn', els =>
|
||||||
|
els.map(el => el.textContent?.trim())
|
||||||
|
);
|
||||||
|
expect(tabTexts).toContain('Display');
|
||||||
|
expect(tabTexts).toContain('Claude CLI');
|
||||||
|
|
||||||
|
// Close
|
||||||
|
await page.click('#appSettingsModal .modal-close');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should switch between settings tabs', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
await page.click('.btn-settings');
|
||||||
|
await page.waitForSelector('#appSettingsModal .modal-content', { visible: true });
|
||||||
|
|
||||||
|
// Click each tab
|
||||||
|
const tabs = await page.$$('#appSettingsModal .modal-tab-btn');
|
||||||
|
|
||||||
|
for (let i = 0; i < tabs.length; i++) {
|
||||||
|
await tabs[i].click();
|
||||||
|
await new Promise(r => setTimeout(r, 100));
|
||||||
|
|
||||||
|
// Check tab is active
|
||||||
|
const isActive = await page.$eval(
|
||||||
|
`#appSettingsModal .modal-tab-btn:nth-child(${i + 1})`,
|
||||||
|
el => el.classList.contains('active')
|
||||||
|
);
|
||||||
|
expect(isActive).toBe(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
await page.click('#appSettingsModal .modal-close');
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Session Management Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should create a new session via button click', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
// Click create session button
|
||||||
|
await page.click('.btn-claude');
|
||||||
|
|
||||||
|
// Wait for session tab to appear
|
||||||
|
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||||
|
|
||||||
|
// Tab should be active
|
||||||
|
const hasActive = await page.$('.session-tab.active');
|
||||||
|
expect(hasActive).not.toBeNull();
|
||||||
|
|
||||||
|
// Terminal should be visible
|
||||||
|
await page.waitForSelector('.xterm', { timeout: 10000 });
|
||||||
|
const terminal = await page.$('.xterm');
|
||||||
|
expect(terminal).not.toBeNull();
|
||||||
|
|
||||||
|
// Track session for cleanup
|
||||||
|
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
if (data.sessions?.length > 0) {
|
||||||
|
const lastSession = data.sessions[data.sessions.length - 1];
|
||||||
|
if (lastSession && !createdSessions.includes(lastSession.id)) {
|
||||||
|
createdSessions.push(lastSession.id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
it('should show tab count correctly', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
// Get initial count
|
||||||
|
const initialText = await page.$eval('#tabCount', el => el.textContent);
|
||||||
|
const initialCount = parseInt(initialText?.match(/\d+/)?.[0] || '0');
|
||||||
|
|
||||||
|
// Create a session
|
||||||
|
await page.click('.btn-claude');
|
||||||
|
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||||
|
|
||||||
|
// Tab count should increase
|
||||||
|
await page.waitForFunction(
|
||||||
|
(expected: number) => {
|
||||||
|
const el = document.querySelector('#tabCount');
|
||||||
|
const text = el?.textContent || '';
|
||||||
|
const match = text.match(/\d+/);
|
||||||
|
return match && parseInt(match[0]) >= expected;
|
||||||
|
},
|
||||||
|
{ timeout: 5000 },
|
||||||
|
initialCount + 1
|
||||||
|
);
|
||||||
|
|
||||||
|
// Track for cleanup
|
||||||
|
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
data.sessions?.forEach((s: any) => {
|
||||||
|
if (!createdSessions.includes(s.id)) createdSessions.push(s.id);
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Terminal Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should initialize xterm terminal with correct dimensions', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
// Create session
|
||||||
|
await page.click('.btn-claude');
|
||||||
|
await page.waitForSelector('.xterm', { timeout: 15000 });
|
||||||
|
|
||||||
|
// Check terminal container exists
|
||||||
|
const terminal = await page.$('.xterm');
|
||||||
|
expect(terminal).not.toBeNull();
|
||||||
|
|
||||||
|
// Check xterm canvas exists (indicates proper initialization)
|
||||||
|
const canvas = await page.$('.xterm-screen');
|
||||||
|
expect(canvas).not.toBeNull();
|
||||||
|
|
||||||
|
// Track for cleanup
|
||||||
|
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||||
|
const data = await response.json();
|
||||||
|
data.sessions?.forEach((s: any) => {
|
||||||
|
if (!createdSessions.includes(s.id)) createdSessions.push(s.id);
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Keyboard Shortcut Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should respond to Ctrl+Enter for quick start', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
const initialSessions = await fetch(`${BASE_URL}/api/sessions`).then(r => r.json());
|
||||||
|
const initialCount = initialSessions.sessions?.length || 0;
|
||||||
|
|
||||||
|
// Press Ctrl+Enter
|
||||||
|
await page.keyboard.down('Control');
|
||||||
|
await page.keyboard.press('Enter');
|
||||||
|
await page.keyboard.up('Control');
|
||||||
|
|
||||||
|
// Wait for session to be created
|
||||||
|
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||||
|
|
||||||
|
// Verify session was created
|
||||||
|
const afterSessions = await fetch(`${BASE_URL}/api/sessions`).then(r => r.json());
|
||||||
|
expect(afterSessions.sessions?.length).toBeGreaterThan(initialCount);
|
||||||
|
|
||||||
|
// Track for cleanup
|
||||||
|
afterSessions.sessions?.forEach((s: any) => {
|
||||||
|
if (!createdSessions.includes(s.id)) createdSessions.push(s.id);
|
||||||
|
});
|
||||||
|
}, 30000);
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Chrome DevTools Protocol Direct Access
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should access CDP for performance metrics', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
// Get performance metrics via CDP
|
||||||
|
const client = await page.target().createCDPSession();
|
||||||
|
await client.send('Performance.enable');
|
||||||
|
|
||||||
|
const { metrics: perfMetrics } = await client.send('Performance.getMetrics');
|
||||||
|
|
||||||
|
// Should have some metrics
|
||||||
|
expect(perfMetrics.length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// Find specific metrics
|
||||||
|
const jsHeapSize = perfMetrics.find(m => m.name === 'JSHeapUsedSize');
|
||||||
|
expect(jsHeapSize).toBeDefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should capture network activity via CDP', async () => {
|
||||||
|
const networkRequests: string[] = [];
|
||||||
|
|
||||||
|
const client = await page.target().createCDPSession();
|
||||||
|
await client.send('Network.enable');
|
||||||
|
|
||||||
|
client.on('Network.requestWillBeSent', (params) => {
|
||||||
|
networkRequests.push(params.request.url);
|
||||||
|
});
|
||||||
|
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
await new Promise(r => setTimeout(r, 2000));
|
||||||
|
|
||||||
|
// Should have captured some requests
|
||||||
|
expect(networkRequests.length).toBeGreaterThan(0);
|
||||||
|
expect(networkRequests.some(url => url.includes('/api/events'))).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Stress Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should handle rapid button clicks without errors', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
// Rapidly click help button multiple times
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
await page.click('.help-btn');
|
||||||
|
await new Promise(r => setTimeout(r, 100));
|
||||||
|
const closeBtn = await page.$('#helpModal .modal-close');
|
||||||
|
if (closeBtn) await closeBtn.click().catch(() => {});
|
||||||
|
await new Promise(r => setTimeout(r, 100));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Page should still be functional
|
||||||
|
const header = await page.$('.header');
|
||||||
|
expect(header).not.toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Screenshot & Visual Tests
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should capture screenshot for visual inspection', async () => {
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
|
||||||
|
// Take a screenshot (could be used for visual regression)
|
||||||
|
const screenshot = await page.screenshot({ encoding: 'base64' });
|
||||||
|
expect(screenshot.length).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// Console & Error Monitoring
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
it('should not have console errors on page load', async () => {
|
||||||
|
const consoleErrors: string[] = [];
|
||||||
|
|
||||||
|
page.on('console', msg => {
|
||||||
|
if (msg.type() === 'error') {
|
||||||
|
consoleErrors.push(msg.text());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
page.on('pageerror', err => {
|
||||||
|
consoleErrors.push(err.message);
|
||||||
|
});
|
||||||
|
|
||||||
|
await page.goto(BASE_URL);
|
||||||
|
await new Promise(r => setTimeout(r, 2000));
|
||||||
|
|
||||||
|
// Filter out expected errors (like SSE reconnection)
|
||||||
|
const unexpectedErrors = consoleErrors.filter(e =>
|
||||||
|
!e.includes('net::ERR') && !e.includes('SSE')
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(unexpectedErrors.length).toBe(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Export metrics for comparison
|
||||||
|
export { metrics as puppeteerMetrics };
|
||||||
Reference in New Issue
Block a user