mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-09-30 12:39:42 +02:00
fix: subagent tabs render to correct parent session after restart
- restoreSubagentWindowStates now discovers parent sessions before restoring - closeSubagentWindow discovers parent before minimizing to prevent wrong tab - Fixed async handling in subagent:completed event handler chore: bump version to 0.1389 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
arkon
@@ -25,7 +25,7 @@ Always bump version on every COM, even for small changes.
|
||||
|
||||
Claudeman is a Claude Code session manager with a web interface and autonomous Ralph Loop. It spawns Claude CLI processes via PTY, streams output in real-time via SSE, and supports scheduled/timed runs.
|
||||
|
||||
**Version**: 0.1388 (must match `package.json`)
|
||||
**Version**: 0.1389 (must match `package.json`)
|
||||
|
||||
**Tech Stack**: TypeScript (ES2022/NodeNext, strict mode), Node.js, Fastify, Server-Sent Events, node-pty
|
||||
|
||||
@@ -119,10 +119,31 @@ npx vitest run -t "should create session" # By pattern
|
||||
| 3152 | browser-e2e.test.ts (hook events tests) |
|
||||
| 3153 | browser-e2e.test.ts (Ralph panel tests) |
|
||||
| 3154 | file-link-click.test.ts |
|
||||
| 3155 | browser-playwright.test.ts |
|
||||
| 3156 | browser-puppeteer.test.ts |
|
||||
| 3157 | browser-agent.test.ts |
|
||||
| 3158-3160 | browser-comparison.test.ts |
|
||||
| 3180-3182 | scripts/browser-comparison.mjs (benchmark) |
|
||||
|
||||
**Next available port**: 3155
|
||||
**Next available port**: 3183
|
||||
|
||||
Unit tests (no port needed): respawn-controller, ralph-tracker, pty-interactive, task-queue, task, ralph-loop, session-manager, state-store, types, templates, ralph-config, spawn-detector, spawn-types, spawn-orchestrator, ai-idle-checker, ai-plan-checker
|
||||
**Browser Testing**: Three frameworks available (Playwright, Puppeteer, Agent-Browser). See `docs/browser-testing-guide.md` for full comparison and patterns.
|
||||
|
||||
```bash
|
||||
# Run browser benchmark (standalone - recommended)
|
||||
npx tsx scripts/browser-comparison.mjs
|
||||
|
||||
# Run existing browser E2E tests
|
||||
npm test -- test/browser-e2e.test.ts
|
||||
```
|
||||
|
||||
**Browser Testing Key Points**:
|
||||
- **Vitest hook issue**: Browser tests using `beforeAll`/`afterAll` timeout even when tests pass. Use standalone scripts or run browser code directly in tests.
|
||||
- **Recommended framework**: Playwright for most cases (auto-waiting, debugging). Puppeteer for Chrome-specific/CDP features.
|
||||
- **Required browser args**: `--no-sandbox`, `--disable-setuid-sandbox`, `--disable-dev-shm-usage`
|
||||
- **Install browsers**: `npx playwright install chromium` after npm install
|
||||
|
||||
**Unit tests** (no server needed): `respawn-controller`, `ralph-tracker`, `pty-interactive`, `task-queue`, `task`, `ralph-loop`, `session-manager`, `state-store`, `types`, `templates`, `ralph-config`, `spawn-detector`, `spawn-types`, `spawn-orchestrator`, `ai-idle-checker`, `ai-plan-checker`
|
||||
|
||||
**Test Utilities**: `test/respawn-test-utils.ts` provides MockSession, MockAiIdleChecker, MockAiPlanChecker, time controller, state tracker, and event recorder for respawn controller testing. See `test/respawn-test-plan.md` for architecture and `test/respawn-scenarios.md` for comprehensive test scenarios.
|
||||
|
||||
@@ -613,6 +634,7 @@ Placeholders replaced:
|
||||
- `docs/spawn-protocol.md` - Spawn1337 agent protocol, MCP tools, resource governance
|
||||
- `docs/ralph-wiggum-guide.md` - Ralph Wiggum loop guide (plugin reference, prompt templates)
|
||||
- `docs/claude-code-hooks-reference.md` - Claude Code hooks documentation
|
||||
- `docs/browser-testing-guide.md` - Browser testing frameworks comparison, patterns, and known issues
|
||||
|
||||
**Internal planning** (historical context, may be outdated):
|
||||
- `docs/respawn-improvement-plan.md` - Planned respawn improvements
|
||||
|
||||
@@ -0,0 +1,338 @@
|
||||
# Browser Testing Guide for Claudeman
|
||||
|
||||
This guide documents the browser testing infrastructure, framework comparison results, and best practices for testing the Claudeman web UI.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Run standalone benchmark (recommended - avoids vitest hook issues)
|
||||
npx tsx scripts/browser-comparison.mjs
|
||||
|
||||
# Run existing browser E2E tests
|
||||
npm test -- test/browser-e2e.test.ts
|
||||
```
|
||||
|
||||
## Framework Comparison Results
|
||||
|
||||
We tested three browser automation frameworks against the Claudeman web UI:
|
||||
|
||||
| Framework | Avg Duration | Best For |
|
||||
|-----------|--------------|----------|
|
||||
| **Puppeteer** | 1223ms | Simple operations, Chrome-specific features |
|
||||
| **Playwright** | 1373ms | Complex interactions, cross-browser, debugging |
|
||||
| **Agent-Browser** | N/A (timeout) | AI agent navigation with semantic locators |
|
||||
|
||||
### Detailed Benchmarks
|
||||
|
||||
| Scenario | Playwright | Puppeteer |
|
||||
|----------|------------|-----------|
|
||||
| Page load | 1445ms | 433ms |
|
||||
| Element selection | 442ms | 373ms |
|
||||
| Modal interaction | 1487ms | 1605ms |
|
||||
| Rapid operations (5 cycles) | 2119ms | 2482ms |
|
||||
|
||||
**Key findings:**
|
||||
- Puppeteer is faster for simple page loads and element selection
|
||||
- Playwright handles rapid/complex interactions better (auto-waiting)
|
||||
- Agent-browser CLI has startup overhead issues in this environment
|
||||
|
||||
## Known Issues
|
||||
|
||||
### Vitest Hook Timeouts
|
||||
|
||||
**Problem:** Browser tests using vitest's `beforeAll`/`afterAll` hooks consistently timeout, even when the tests actually complete successfully.
|
||||
|
||||
**Symptoms:**
|
||||
- Tests show as "skipped"
|
||||
- Error: "Hook timed out in 60000ms"
|
||||
- But cleanup messages appear (indicating tests ran)
|
||||
|
||||
**Root cause:** Unclear - possibly related to:
|
||||
- vitest's module isolation with async browser launches
|
||||
- Interaction between global setup.ts hooks and test-level hooks
|
||||
- Multiple test file imports causing duplicate hook execution
|
||||
|
||||
**Workarounds:**
|
||||
1. **Use standalone scripts** (recommended):
|
||||
```bash
|
||||
npx tsx scripts/browser-comparison.mjs
|
||||
```
|
||||
|
||||
2. **Run browser code directly in tests** (not in hooks):
|
||||
```typescript
|
||||
it('should test something', async () => {
|
||||
const browser = await chromium.launch();
|
||||
// ... test code ...
|
||||
await browser.close();
|
||||
});
|
||||
```
|
||||
|
||||
3. **Use the existing browser-e2e.test.ts pattern** which uses agent-browser CLI commands via `execSync` (avoids async hook issues)
|
||||
|
||||
## Test File Structure
|
||||
|
||||
### Port Allocation
|
||||
|
||||
| Port Range | Test File |
|
||||
|------------|-----------|
|
||||
| 3150-3153 | browser-e2e.test.ts (existing) |
|
||||
| 3154 | file-link-click.test.ts |
|
||||
| 3155 | browser-playwright.test.ts |
|
||||
| 3156 | browser-puppeteer.test.ts |
|
||||
| 3157 | browser-agent.test.ts |
|
||||
| 3158-3160 | browser-comparison.test.ts |
|
||||
| 3180-3182 | scripts/browser-comparison.mjs |
|
||||
|
||||
### File Purposes
|
||||
|
||||
| File | Framework | Status |
|
||||
|------|-----------|--------|
|
||||
| `test/browser-e2e.test.ts` | agent-browser | ✅ Working |
|
||||
| `test/browser-playwright.test.ts` | Playwright | ⚠️ Vitest hook issues |
|
||||
| `test/browser-puppeteer.test.ts` | Puppeteer | ⚠️ Vitest hook issues |
|
||||
| `test/browser-agent.test.ts` | agent-browser | ⚠️ Vitest hook issues |
|
||||
| `scripts/browser-comparison.mjs` | All three | ✅ Working (standalone) |
|
||||
|
||||
## Framework-Specific Patterns
|
||||
|
||||
### Playwright
|
||||
|
||||
```typescript
|
||||
import { chromium } from 'playwright';
|
||||
|
||||
const browser = await chromium.launch({
|
||||
headless: true,
|
||||
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||
});
|
||||
|
||||
const page = await browser.newPage();
|
||||
await page.goto('http://localhost:3000');
|
||||
|
||||
// Auto-waiting selectors
|
||||
await page.click('.btn-claude');
|
||||
await page.waitForSelector('.session-tab', { state: 'visible' });
|
||||
|
||||
// Assertions with expect
|
||||
await expect(page.locator('.header')).toBeVisible();
|
||||
await expect(page).toHaveTitle('Claudeman');
|
||||
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
**Pros:**
|
||||
- Built-in auto-waiting
|
||||
- Excellent trace viewer for debugging
|
||||
- Cross-browser support (Chromium, Firefox, WebKit)
|
||||
- Native `expect` assertions
|
||||
|
||||
**Cons:**
|
||||
- Slightly slower page loads
|
||||
- Larger dependency
|
||||
|
||||
### Puppeteer
|
||||
|
||||
```typescript
|
||||
import puppeteer from 'puppeteer';
|
||||
|
||||
const browser = await puppeteer.launch({
|
||||
headless: true,
|
||||
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||
});
|
||||
|
||||
const page = await browser.newPage();
|
||||
await page.goto('http://localhost:3000');
|
||||
|
||||
// Manual waiting often needed
|
||||
await page.click('.btn-claude');
|
||||
await page.waitForSelector('.session-tab', { visible: true });
|
||||
|
||||
// Element queries
|
||||
const title = await page.title();
|
||||
const text = await page.$eval('.logo', el => el.textContent);
|
||||
|
||||
// CDP access for advanced features
|
||||
const client = await page.target().createCDPSession();
|
||||
await client.send('Performance.enable');
|
||||
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
**Pros:**
|
||||
- Faster for simple operations
|
||||
- Direct Chrome DevTools Protocol access
|
||||
- Smaller dependency
|
||||
- Good for Chrome-specific testing
|
||||
|
||||
**Cons:**
|
||||
- Chrome/Chromium only
|
||||
- Manual waiting required
|
||||
- Less robust for complex interactions
|
||||
|
||||
### Agent-Browser (CLI)
|
||||
|
||||
```typescript
|
||||
import { execSync } from 'node:child_process';
|
||||
|
||||
function agentBrowser(cmd: string): string {
|
||||
return execSync(`npx agent-browser ${cmd}`, {
|
||||
timeout: 30000,
|
||||
encoding: 'utf-8',
|
||||
}).trim();
|
||||
}
|
||||
|
||||
function agentBrowserJson<T>(cmd: string): T {
|
||||
const result = agentBrowser(`${cmd} --json`);
|
||||
return JSON.parse(result).data;
|
||||
}
|
||||
|
||||
// Usage
|
||||
agentBrowser('open http://localhost:3000');
|
||||
agentBrowser('click ".btn-claude"');
|
||||
const title = agentBrowserJson<{title: string}>('get title');
|
||||
|
||||
// Semantic locators (AI-friendly)
|
||||
agentBrowser('find role button click --name "Submit"');
|
||||
agentBrowser('find text "Settings" click');
|
||||
|
||||
// Accessibility snapshot
|
||||
const snapshot = agentBrowser('snapshot');
|
||||
|
||||
agentBrowser('close');
|
||||
```
|
||||
|
||||
**Pros:**
|
||||
- AI-agent friendly (semantic locators)
|
||||
- Accessibility tree snapshots
|
||||
- Simple CLI interface
|
||||
- Reference-based selection (@e1, @e2)
|
||||
|
||||
**Cons:**
|
||||
- CLI overhead (spawn process per command)
|
||||
- Slower for rapid operations
|
||||
- Less programmatic control
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Use Standalone Scripts for Benchmarks
|
||||
|
||||
Vitest has issues with browser hooks. For reliable benchmarking:
|
||||
|
||||
```bash
|
||||
# Create a standalone .mjs script
|
||||
npx tsx scripts/browser-comparison.mjs
|
||||
```
|
||||
|
||||
### 2. Browser Launch Arguments
|
||||
|
||||
Always include these args for headless environments:
|
||||
|
||||
```typescript
|
||||
{
|
||||
headless: true,
|
||||
args: [
|
||||
'--no-sandbox', // Required for Docker/CI
|
||||
'--disable-setuid-sandbox',
|
||||
'--disable-dev-shm-usage', // Prevents /dev/shm issues
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Install Playwright Browsers
|
||||
|
||||
```bash
|
||||
npx playwright install chromium
|
||||
```
|
||||
|
||||
### 4. Wait for Server Startup
|
||||
|
||||
```typescript
|
||||
const server = new WebServer(PORT);
|
||||
await server.start();
|
||||
await new Promise(r => setTimeout(r, 1000)); // Allow server to stabilize
|
||||
```
|
||||
|
||||
### 5. Clean Up Sessions
|
||||
|
||||
Track created sessions for cleanup:
|
||||
|
||||
```typescript
|
||||
const createdSessions: string[] = [];
|
||||
|
||||
// In test
|
||||
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||
const data = await response.json();
|
||||
createdSessions.push(data.sessions[0].id);
|
||||
|
||||
// In cleanup
|
||||
for (const id of createdSessions) {
|
||||
await fetch(`${BASE_URL}/api/sessions/${id}`, { method: 'DELETE' });
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Handle Modal Timing
|
||||
|
||||
Modals have animation delays:
|
||||
|
||||
```typescript
|
||||
// Playwright (auto-waits)
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal', { state: 'visible' });
|
||||
|
||||
// Puppeteer (manual wait)
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal', { visible: true });
|
||||
|
||||
// Agent-browser (explicit delay)
|
||||
agentBrowser('click ".help-btn"');
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
```
|
||||
|
||||
## Key DOM Selectors
|
||||
|
||||
For reference when writing browser tests:
|
||||
|
||||
```
|
||||
.btn-claude // Create Claude session button
|
||||
.btn-settings // Settings button
|
||||
.help-btn // Help button
|
||||
.session-tab // Session tabs
|
||||
.session-tab.active // Active session tab
|
||||
.xterm // Terminal container
|
||||
#helpModal // Help modal
|
||||
#appSettingsModal // Settings modal
|
||||
.modal-content // Modal content
|
||||
.modal-close // Modal close button
|
||||
.header-brand .logo // Logo text
|
||||
#versionDisplay // Version display
|
||||
#quickStartCase // Quick start dropdown
|
||||
```
|
||||
|
||||
## Recommendations by Use Case
|
||||
|
||||
| Use Case | Recommended Framework |
|
||||
|----------|----------------------|
|
||||
| CI/CD testing | Playwright |
|
||||
| Chrome-specific features | Puppeteer |
|
||||
| AI agent development | Agent-Browser |
|
||||
| Visual regression | Playwright |
|
||||
| Performance testing | Puppeteer |
|
||||
| Accessibility testing | Agent-Browser |
|
||||
| Cross-browser testing | Playwright |
|
||||
| Quick prototyping | Agent-Browser CLI |
|
||||
|
||||
## Dependencies
|
||||
|
||||
```json
|
||||
{
|
||||
"devDependencies": {
|
||||
"playwright": "^1.58.0",
|
||||
"puppeteer": "^24.36.0",
|
||||
"agent-browser": "^0.6.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Install browsers after npm install:
|
||||
```bash
|
||||
npx playwright install chromium
|
||||
```
|
||||
Generated
+1131
-13
File diff suppressed because it is too large
Load Diff
+4
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claudeman",
|
||||
"version": "0.1388",
|
||||
"version": "0.1389",
|
||||
"description": "The missing control plane for Claude Code - run 20 autonomous agents with real-time monitoring and session persistence",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
@@ -54,12 +54,13 @@
|
||||
"@types/node": "^20.14.0",
|
||||
"@types/react": "^19.2.9",
|
||||
"@types/uuid": "^10.0.0",
|
||||
"@vitest/coverage-v8": "^4.0.17",
|
||||
"@vitest/coverage-v8": "^4.0.18",
|
||||
"agent-browser": "^0.6.0",
|
||||
"playwright": "^1.58.0",
|
||||
"puppeteer": "^24.36.0",
|
||||
"tsx": "^4.15.0",
|
||||
"typescript": "^5.9.3",
|
||||
"vitest": "^4.0.17"
|
||||
"vitest": "^4.0.18"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18.0.0"
|
||||
|
||||
@@ -0,0 +1,309 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Browser Framework Comparison Script
|
||||
*
|
||||
* Compares Playwright, Puppeteer, and Agent-Browser for testing Claudeman web UI.
|
||||
* Run: node scripts/browser-comparison.mjs
|
||||
*/
|
||||
|
||||
import { chromium } from 'playwright';
|
||||
import puppeteer from 'puppeteer';
|
||||
import { execSync } from 'node:child_process';
|
||||
import { WebServer } from '../src/web/server.js';
|
||||
|
||||
const PORTS = {
|
||||
playwright: 3180,
|
||||
puppeteer: 3181,
|
||||
agentBrowser: 3182,
|
||||
};
|
||||
|
||||
const results = [];
|
||||
|
||||
function logSection(title) {
|
||||
console.log('\n' + '='.repeat(60));
|
||||
console.log(` ${title}`);
|
||||
console.log('='.repeat(60));
|
||||
}
|
||||
|
||||
async function runBenchmark(name, framework, fn) {
|
||||
const start = Date.now();
|
||||
let success = false;
|
||||
let error = null;
|
||||
try {
|
||||
await fn();
|
||||
success = true;
|
||||
} catch (e) {
|
||||
error = e.message;
|
||||
}
|
||||
const duration = Date.now() - start;
|
||||
results.push({ scenario: name, framework, duration, success, error });
|
||||
console.log(` ${success ? '✅' : '❌'} ${name}: ${duration}ms ${error ? `(${error.slice(0, 50)})` : ''}`);
|
||||
}
|
||||
|
||||
// Agent-browser helper
|
||||
function agentBrowser(cmd) {
|
||||
return execSync(`npx agent-browser ${cmd}`, {
|
||||
timeout: 30000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
}).trim();
|
||||
}
|
||||
|
||||
function agentBrowserJson(cmd) {
|
||||
const result = agentBrowser(`${cmd} --json`);
|
||||
const parsed = JSON.parse(result);
|
||||
if (!parsed.success) throw new Error(parsed.error || 'unknown error');
|
||||
return parsed.data;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
console.log('🚀 Browser Framework Comparison Benchmark');
|
||||
console.log('Comparing: Playwright, Puppeteer, Agent-Browser\n');
|
||||
|
||||
// Start servers
|
||||
const servers = [];
|
||||
for (const [name, port] of Object.entries(PORTS)) {
|
||||
console.log(`Starting server for ${name} on port ${port}...`);
|
||||
const server = new WebServer(port);
|
||||
await server.start();
|
||||
servers.push(server);
|
||||
}
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
|
||||
// ============================================
|
||||
// PLAYWRIGHT TESTS
|
||||
// ============================================
|
||||
logSection('PLAYWRIGHT TESTS');
|
||||
|
||||
let playwrightBrowser;
|
||||
try {
|
||||
playwrightBrowser = await chromium.launch({
|
||||
headless: true,
|
||||
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||
});
|
||||
console.log(' Browser launched');
|
||||
|
||||
// Test 1: Page load
|
||||
await runBenchmark('page-load', 'playwright', async () => {
|
||||
const page = await playwrightBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
const title = await page.title();
|
||||
if (title !== 'Claudeman') throw new Error(`Expected Claudeman, got ${title}`);
|
||||
await page.close();
|
||||
});
|
||||
|
||||
// Test 2: Element selection
|
||||
await runBenchmark('element-selection', 'playwright', async () => {
|
||||
const page = await playwrightBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
const logo = await page.locator('.header-brand .logo').textContent();
|
||||
const version = await page.locator('#versionDisplay').textContent();
|
||||
if (logo !== 'Claudeman') throw new Error('Logo mismatch');
|
||||
if (!version.match(/v\d+/)) throw new Error('Version mismatch');
|
||||
await page.close();
|
||||
});
|
||||
|
||||
// Test 3: Modal interaction
|
||||
await runBenchmark('modal-interaction', 'playwright', async () => {
|
||||
const page = await playwrightBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||
await page.click('#helpModal .modal-close');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||
await page.close();
|
||||
});
|
||||
|
||||
// Test 4: Rapid operations (5 modal cycles)
|
||||
await runBenchmark('rapid-operations', 'playwright', async () => {
|
||||
const page = await playwrightBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||
await page.click('#helpModal .modal-close');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||
}
|
||||
await page.close();
|
||||
});
|
||||
|
||||
} finally {
|
||||
if (playwrightBrowser) await playwrightBrowser.close();
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// PUPPETEER TESTS
|
||||
// ============================================
|
||||
logSection('PUPPETEER TESTS');
|
||||
|
||||
let puppeteerBrowser;
|
||||
try {
|
||||
puppeteerBrowser = await puppeteer.launch({
|
||||
headless: true,
|
||||
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||
});
|
||||
console.log(' Browser launched');
|
||||
|
||||
// Test 1: Page load
|
||||
await runBenchmark('page-load', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
const title = await page.title();
|
||||
if (title !== 'Claudeman') throw new Error(`Expected Claudeman, got ${title}`);
|
||||
await page.close();
|
||||
});
|
||||
|
||||
// Test 2: Element selection
|
||||
await runBenchmark('element-selection', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
const logo = await page.$eval('.header-brand .logo', el => el.textContent);
|
||||
const version = await page.$eval('#versionDisplay', el => el.textContent);
|
||||
if (logo !== 'Claudeman') throw new Error('Logo mismatch');
|
||||
if (!version.match(/v\d+/)) throw new Error('Version mismatch');
|
||||
await page.close();
|
||||
});
|
||||
|
||||
// Test 3: Modal interaction
|
||||
await runBenchmark('modal-interaction', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||
await page.click('#helpModal .modal-close');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
await page.close();
|
||||
});
|
||||
|
||||
// Test 4: Rapid operations
|
||||
await runBenchmark('rapid-operations', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||
await page.click('#helpModal .modal-close');
|
||||
await new Promise(r => setTimeout(r, 200));
|
||||
}
|
||||
await page.close();
|
||||
});
|
||||
|
||||
} finally {
|
||||
if (puppeteerBrowser) await puppeteerBrowser.close();
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// AGENT-BROWSER TESTS
|
||||
// ============================================
|
||||
logSection('AGENT-BROWSER TESTS');
|
||||
|
||||
let agentBrowserAvailable = false;
|
||||
try {
|
||||
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
const title = agentBrowserJson('get title');
|
||||
agentBrowserAvailable = title.title === 'Claudeman';
|
||||
console.log(' Browser launched');
|
||||
|
||||
// Test 1: Page load
|
||||
await runBenchmark('page-load', 'agent-browser', async () => {
|
||||
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
const title = agentBrowserJson('get title');
|
||||
if (title.title !== 'Claudeman') throw new Error(`Expected Claudeman, got ${title.title}`);
|
||||
});
|
||||
|
||||
// Test 2: Element selection
|
||||
await runBenchmark('element-selection', 'agent-browser', async () => {
|
||||
const logo = agentBrowserJson('get text ".header-brand .logo"');
|
||||
const version = agentBrowserJson('get text "#versionDisplay"');
|
||||
if (logo.text !== 'Claudeman') throw new Error('Logo mismatch');
|
||||
if (!version.text.match(/v\d+/)) throw new Error('Version mismatch');
|
||||
});
|
||||
|
||||
// Test 3: Modal interaction
|
||||
await runBenchmark('modal-interaction', 'agent-browser', async () => {
|
||||
agentBrowser('click ".help-btn"');
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
const visible = agentBrowserJson('is visible "#helpModal .modal-content"');
|
||||
if (!visible.visible) throw new Error('Modal not visible');
|
||||
agentBrowser('click "#helpModal .modal-close"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
});
|
||||
|
||||
// Test 4: Rapid operations
|
||||
await runBenchmark('rapid-operations', 'agent-browser', async () => {
|
||||
for (let i = 0; i < 5; i++) {
|
||||
agentBrowser('click ".help-btn"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
agentBrowser('click "#helpModal .modal-close"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
}
|
||||
});
|
||||
|
||||
} catch (e) {
|
||||
console.log(` ⚠️ Agent-browser not available: ${e.message}`);
|
||||
} finally {
|
||||
try { agentBrowser('close'); } catch {}
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// RESULTS
|
||||
// ============================================
|
||||
logSection('RESULTS SUMMARY');
|
||||
|
||||
// Group by framework
|
||||
const frameworks = ['playwright', 'puppeteer', 'agent-browser'];
|
||||
for (const fw of frameworks) {
|
||||
const fwResults = results.filter(r => r.framework === fw);
|
||||
if (fwResults.length === 0) continue;
|
||||
|
||||
const passed = fwResults.filter(r => r.success).length;
|
||||
const total = fwResults.length;
|
||||
const avgDuration = Math.round(fwResults.reduce((a, r) => a + r.duration, 0) / total);
|
||||
|
||||
console.log(`\n ${fw.toUpperCase()}`);
|
||||
console.log(` Tests: ${passed}/${total} passed`);
|
||||
console.log(` Avg Duration: ${avgDuration}ms`);
|
||||
}
|
||||
|
||||
// Performance ranking
|
||||
console.log('\n🏆 PERFORMANCE RANKING (by avg duration)');
|
||||
const stats = frameworks.map(fw => {
|
||||
const fwResults = results.filter(r => r.framework === fw);
|
||||
if (fwResults.length === 0) return null;
|
||||
return {
|
||||
framework: fw,
|
||||
avg: Math.round(fwResults.reduce((a, r) => a + r.duration, 0) / fwResults.length),
|
||||
passed: fwResults.filter(r => r.success).length,
|
||||
total: fwResults.length,
|
||||
};
|
||||
}).filter(Boolean).sort((a, b) => a.avg - b.avg);
|
||||
|
||||
stats.forEach((s, i) => {
|
||||
const medal = i === 0 ? '🥇' : i === 1 ? '🥈' : '🥉';
|
||||
console.log(` ${medal} ${s.framework}: ${s.avg}ms avg (${s.passed}/${s.total} passed)`);
|
||||
});
|
||||
|
||||
// Cleanup
|
||||
console.log('\n Cleaning up...');
|
||||
for (const server of servers) {
|
||||
await server.stop();
|
||||
}
|
||||
|
||||
console.log('\n✅ Benchmark complete!\n');
|
||||
|
||||
// Detailed results table
|
||||
console.log('📋 DETAILED RESULTS');
|
||||
console.table(results.map(r => ({
|
||||
Scenario: r.scenario,
|
||||
Framework: r.framework,
|
||||
Duration: `${r.duration}ms`,
|
||||
Status: r.success ? '✅' : '❌',
|
||||
})));
|
||||
}
|
||||
|
||||
main().catch(e => {
|
||||
console.error('Benchmark failed:', e);
|
||||
process.exit(1);
|
||||
});
|
||||
+282
-11
@@ -1764,7 +1764,7 @@ class ClaudemanApp {
|
||||
}
|
||||
});
|
||||
|
||||
this.eventSource.addEventListener('subagent:completed', (e) => {
|
||||
this.eventSource.addEventListener('subagent:completed', async (e) => {
|
||||
const data = JSON.parse(e.data);
|
||||
const existing = this.subagents.get(data.agentId);
|
||||
if (existing) {
|
||||
@@ -1778,7 +1778,7 @@ class ClaudemanApp {
|
||||
if (this.subagentWindows.has(data.agentId)) {
|
||||
const windowData = this.subagentWindows.get(data.agentId);
|
||||
if (windowData && !windowData.minimized) {
|
||||
this.closeSubagentWindow(data.agentId); // This minimizes to tab
|
||||
await this.closeSubagentWindow(data.agentId); // This minimizes to tab
|
||||
this.saveSubagentWindowStates(); // Persist the minimized state
|
||||
}
|
||||
}
|
||||
@@ -2346,6 +2346,245 @@ class ClaudemanApp {
|
||||
this.selectSession(ids[nextIndex]);
|
||||
}
|
||||
|
||||
// ========== Navigation ==========
|
||||
|
||||
goHome() {
|
||||
// Deselect active session and show welcome screen
|
||||
this.activeSessionId = null;
|
||||
this.terminal.clear();
|
||||
this.showWelcome();
|
||||
this.renderSessionTabs();
|
||||
this.renderRalphStatePanel();
|
||||
}
|
||||
|
||||
// ========== Ralph Loop Wizard ==========
|
||||
|
||||
// Wizard state
|
||||
ralphWizardStep = 1;
|
||||
ralphWizardConfig = {
|
||||
taskDescription: '',
|
||||
completionPhrase: 'COMPLETE',
|
||||
maxIterations: 10,
|
||||
caseName: 'testcase',
|
||||
enableRespawn: true,
|
||||
};
|
||||
|
||||
showRalphWizard() {
|
||||
// Reset wizard state
|
||||
this.ralphWizardStep = 1;
|
||||
this.ralphWizardConfig = {
|
||||
taskDescription: '',
|
||||
completionPhrase: 'COMPLETE',
|
||||
maxIterations: 10,
|
||||
caseName: document.getElementById('quickStartCase')?.value || 'testcase',
|
||||
enableRespawn: true,
|
||||
};
|
||||
|
||||
// Reset UI
|
||||
document.getElementById('ralphTaskDescription').value = '';
|
||||
document.getElementById('ralphCompletionPhrase').value = 'COMPLETE';
|
||||
this.selectIterationPreset(10);
|
||||
|
||||
// Populate case selector
|
||||
this.populateRalphCaseSelector();
|
||||
|
||||
// Show wizard modal
|
||||
this.updateRalphWizardUI();
|
||||
document.getElementById('ralphWizardModal').classList.add('active');
|
||||
document.getElementById('ralphTaskDescription').focus();
|
||||
}
|
||||
|
||||
closeRalphWizard() {
|
||||
document.getElementById('ralphWizardModal')?.classList.remove('active');
|
||||
}
|
||||
|
||||
populateRalphCaseSelector() {
|
||||
const select = document.getElementById('ralphCaseSelect');
|
||||
const quickStartSelect = document.getElementById('quickStartCase');
|
||||
|
||||
if (quickStartSelect && select) {
|
||||
select.innerHTML = quickStartSelect.innerHTML;
|
||||
select.value = this.ralphWizardConfig.caseName;
|
||||
}
|
||||
}
|
||||
|
||||
selectIterationPreset(iterations) {
|
||||
this.ralphWizardConfig.maxIterations = iterations;
|
||||
|
||||
// Update button states
|
||||
document.querySelectorAll('.iteration-preset-btn').forEach(btn => {
|
||||
const btnIterations = parseInt(btn.dataset.iterations);
|
||||
btn.classList.toggle('active', btnIterations === iterations);
|
||||
});
|
||||
}
|
||||
|
||||
ralphWizardNext() {
|
||||
if (this.ralphWizardStep === 1) {
|
||||
// Validate step 1
|
||||
const taskDescription = document.getElementById('ralphTaskDescription').value.trim();
|
||||
const completionPhrase = document.getElementById('ralphCompletionPhrase').value.trim() || 'COMPLETE';
|
||||
const caseName = document.getElementById('ralphCaseSelect').value;
|
||||
|
||||
if (!taskDescription) {
|
||||
this.showToast('Please enter a task description', 'error');
|
||||
document.getElementById('ralphTaskDescription').focus();
|
||||
return;
|
||||
}
|
||||
|
||||
// Save config
|
||||
this.ralphWizardConfig.taskDescription = taskDescription;
|
||||
this.ralphWizardConfig.completionPhrase = completionPhrase.toUpperCase();
|
||||
this.ralphWizardConfig.caseName = caseName;
|
||||
|
||||
// Generate preview
|
||||
this.updateRalphPromptPreview();
|
||||
|
||||
// Move to step 2
|
||||
this.ralphWizardStep = 2;
|
||||
this.updateRalphWizardUI();
|
||||
}
|
||||
}
|
||||
|
||||
ralphWizardBack() {
|
||||
if (this.ralphWizardStep === 2) {
|
||||
this.ralphWizardStep = 1;
|
||||
this.updateRalphWizardUI();
|
||||
}
|
||||
}
|
||||
|
||||
updateRalphWizardUI() {
|
||||
const step = this.ralphWizardStep;
|
||||
|
||||
// Update progress indicators
|
||||
document.querySelectorAll('.wizard-step').forEach(el => {
|
||||
const stepNum = parseInt(el.dataset.step);
|
||||
el.classList.toggle('active', stepNum === step);
|
||||
el.classList.toggle('completed', stepNum < step);
|
||||
});
|
||||
|
||||
// Show/hide pages
|
||||
document.getElementById('ralphWizardStep1').classList.toggle('hidden', step !== 1);
|
||||
document.getElementById('ralphWizardStep2').classList.toggle('hidden', step !== 2);
|
||||
|
||||
// Show/hide buttons
|
||||
document.getElementById('ralphBackBtn').style.display = step === 1 ? 'none' : 'block';
|
||||
document.getElementById('ralphNextBtn').style.display = step === 2 ? 'none' : 'block';
|
||||
document.getElementById('ralphStartBtn').style.display = step === 2 ? 'block' : 'none';
|
||||
}
|
||||
|
||||
updateRalphPromptPreview() {
|
||||
const config = this.ralphWizardConfig;
|
||||
const preview = document.getElementById('ralphPromptPreview');
|
||||
|
||||
// Build the formatted prompt
|
||||
let prompt = config.taskDescription;
|
||||
|
||||
// Add completion instructions
|
||||
prompt += '\n\n---\n\n';
|
||||
prompt += '## Completion Criteria\n\n';
|
||||
prompt += 'When ALL requirements above are complete:\n';
|
||||
prompt += '1. Run all tests to verify\n';
|
||||
prompt += '2. Commit your changes\n';
|
||||
prompt += `3. Output: <promise>${config.completionPhrase}</promise>\n`;
|
||||
|
||||
// Show preview with highlighting
|
||||
const highlightedPrompt = prompt
|
||||
.replace(/<promise>/g, '<span class="preview-highlight"><promise>')
|
||||
.replace(/<\/promise>/g, '</promise></span>');
|
||||
|
||||
preview.innerHTML = highlightedPrompt;
|
||||
|
||||
// Update summary
|
||||
document.getElementById('summaryPhrase').textContent = config.completionPhrase;
|
||||
document.getElementById('summaryIterations').textContent =
|
||||
config.maxIterations === 0 ? 'Unlimited' : config.maxIterations;
|
||||
document.getElementById('summaryCase').textContent = config.caseName;
|
||||
}
|
||||
|
||||
async startRalphLoop() {
|
||||
const config = this.ralphWizardConfig;
|
||||
|
||||
// Read advanced options
|
||||
config.enableRespawn = document.getElementById('ralphEnableRespawn')?.checked ?? true;
|
||||
|
||||
// Close wizard
|
||||
this.closeRalphWizard();
|
||||
|
||||
// Build the full prompt
|
||||
let fullPrompt = config.taskDescription;
|
||||
fullPrompt += '\n\n---\n\n';
|
||||
fullPrompt += '## Completion Criteria\n\n';
|
||||
fullPrompt += 'When ALL requirements above are complete:\n';
|
||||
fullPrompt += '1. Run all tests to verify\n';
|
||||
fullPrompt += '2. Commit your changes\n';
|
||||
fullPrompt += `3. Output: <promise>${config.completionPhrase}</promise>\n`;
|
||||
|
||||
try {
|
||||
// Step 1: Create/verify case exists and start session
|
||||
this.terminal.clear();
|
||||
this.terminal.writeln(`\x1b[1;33m Starting Ralph Loop in ${config.caseName}...\x1b[0m`);
|
||||
this.terminal.writeln('');
|
||||
|
||||
// Create session via quick-start
|
||||
const res = await fetch('/api/quick-start', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ caseName: config.caseName, mode: 'claude' })
|
||||
});
|
||||
|
||||
const data = await res.json();
|
||||
if (!data.success) {
|
||||
this.showToast(data.error || 'Failed to start session', 'error');
|
||||
return;
|
||||
}
|
||||
|
||||
const sessionId = data.sessionId;
|
||||
|
||||
// Step 2: Configure Ralph tracker for this session
|
||||
await fetch(`/api/sessions/${sessionId}/ralph-config`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
enabled: true,
|
||||
completionPhrase: config.completionPhrase,
|
||||
maxIterations: config.maxIterations || null,
|
||||
})
|
||||
});
|
||||
|
||||
// Step 3: Enable respawn if requested
|
||||
if (config.enableRespawn) {
|
||||
await fetch(`/api/sessions/${sessionId}/respawn/enable`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
prompt: 'continue',
|
||||
sendClear: true,
|
||||
sendInit: true,
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
// Step 4: Switch to session and wait for it to be ready
|
||||
await this.selectSession(sessionId);
|
||||
|
||||
// Wait a moment for session to be ready, then send the prompt
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
|
||||
// Send the prompt to start the loop
|
||||
await fetch(`/api/sessions/${sessionId}/input`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ input: fullPrompt + '\n' })
|
||||
});
|
||||
|
||||
this.showToast(`Ralph Loop started in ${config.caseName}`, 'success');
|
||||
|
||||
} catch (err) {
|
||||
console.error('Failed to start Ralph loop:', err);
|
||||
this.showToast('Failed to start Ralph loop: ' + err.message, 'error');
|
||||
}
|
||||
}
|
||||
|
||||
// ========== Quick Start ==========
|
||||
|
||||
async loadQuickStartCases(selectCaseName = null) {
|
||||
@@ -4287,13 +4526,35 @@ class ClaudemanApp {
|
||||
async restoreSubagentWindowStates() {
|
||||
const states = await this.loadSubagentWindowStates();
|
||||
|
||||
// Restore minimized state
|
||||
for (const [sessionId, agentIds] of Object.entries(states.minimized || {})) {
|
||||
// First, discover parent sessions for all subagents to establish correct mapping
|
||||
// This is important after server restart when parentSessionId isn't set
|
||||
const parentDiscoveryPromises = [];
|
||||
for (const [agentId, agent] of this.subagents) {
|
||||
if (!agent.parentSessionId) {
|
||||
parentDiscoveryPromises.push(this.findParentSessionForSubagent(agentId));
|
||||
}
|
||||
}
|
||||
if (parentDiscoveryPromises.length > 0) {
|
||||
await Promise.all(parentDiscoveryPromises);
|
||||
}
|
||||
|
||||
// Restore minimized state, but verify session mapping is correct
|
||||
for (const [savedSessionId, agentIds] of Object.entries(states.minimized || {})) {
|
||||
if (Array.isArray(agentIds) && agentIds.length > 0) {
|
||||
// Verify agents still exist
|
||||
const validAgents = agentIds.filter(id => this.subagents.has(id));
|
||||
if (validAgents.length > 0) {
|
||||
this.minimizedSubagents.set(sessionId, new Set(validAgents));
|
||||
for (const agentId of agentIds) {
|
||||
const agent = this.subagents.get(agentId);
|
||||
if (!agent) continue; // Agent no longer exists
|
||||
|
||||
// Use discovered parentSessionId, or validate saved sessionId exists
|
||||
const correctSessionId = agent.parentSessionId ||
|
||||
(this.sessions.has(savedSessionId) ? savedSessionId : null);
|
||||
|
||||
if (correctSessionId) {
|
||||
if (!this.minimizedSubagents.has(correctSessionId)) {
|
||||
this.minimizedSubagents.set(correctSessionId, new Set());
|
||||
}
|
||||
this.minimizedSubagents.get(correctSessionId).add(agentId);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4326,6 +4587,7 @@ class ClaudemanApp {
|
||||
}
|
||||
|
||||
this.renderSessionTabs(); // Update tab badges
|
||||
this.saveSubagentWindowStates(); // Persist corrected mappings
|
||||
}
|
||||
|
||||
// ========== Help Modal ==========
|
||||
@@ -5804,14 +6066,23 @@ class ClaudemanApp {
|
||||
this.saveSubagentWindowStates();
|
||||
}
|
||||
|
||||
closeSubagentWindow(agentId) {
|
||||
async closeSubagentWindow(agentId) {
|
||||
const windowData = this.subagentWindows.get(agentId);
|
||||
if (!windowData) return;
|
||||
|
||||
const agent = this.subagents.get(agentId);
|
||||
const parentSessionId = agent?.parentSessionId || this.activeSessionId;
|
||||
|
||||
// Always minimize to tab (use active session if no parent)
|
||||
// Try to discover parent session if not already known
|
||||
// This ensures we minimize to the correct tab
|
||||
if (agent && !agent.parentSessionId) {
|
||||
await this.findParentSessionForSubagent(agentId);
|
||||
}
|
||||
|
||||
// Now get the correct parent session (re-read agent after discovery)
|
||||
const updatedAgent = this.subagents.get(agentId);
|
||||
const parentSessionId = updatedAgent?.parentSessionId || this.activeSessionId;
|
||||
|
||||
// Always minimize to tab (use active session if no parent found)
|
||||
windowData.element.style.display = 'none';
|
||||
windowData.minimized = true;
|
||||
|
||||
|
||||
+133
-1
@@ -17,7 +17,7 @@
|
||||
<!-- Compact Header with Session Tabs -->
|
||||
<header class="header">
|
||||
<div class="header-brand">
|
||||
<span class="logo">Claudeman</span>
|
||||
<span class="logo" onclick="app.goHome()" title="Go to main page">Claudeman</span>
|
||||
</div>
|
||||
|
||||
<!-- Session Tabs -->
|
||||
@@ -206,6 +206,9 @@
|
||||
<input type="number" id="shellCount" class="tab-count-input" value="1" min="1" max="20" readonly title="Instance count">
|
||||
<button class="tab-count-btn" onclick="app.incrementShellCount()" title="Instance count">+</button>
|
||||
</div>
|
||||
<button class="btn-toolbar btn-ralph" onclick="app.showRalphWizard()" title="Start Ralph Loop">
|
||||
Ralph Loop
|
||||
</button>
|
||||
<div class="case-select-group">
|
||||
<select id="quickStartCase" class="toolbar-select" title="Select case">
|
||||
<option value="testcase">testcase</option>
|
||||
@@ -854,6 +857,135 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Ralph Loop Wizard Modal -->
|
||||
<div class="modal" id="ralphWizardModal">
|
||||
<div class="modal-backdrop" onclick="app.closeRalphWizard()"></div>
|
||||
<div class="modal-content modal-lg">
|
||||
<div class="modal-header">
|
||||
<h3>Start Ralph Loop</h3>
|
||||
<button class="modal-close" onclick="app.closeRalphWizard()">×</button>
|
||||
</div>
|
||||
|
||||
<!-- Wizard Progress Steps -->
|
||||
<div class="wizard-progress">
|
||||
<div class="wizard-step active" data-step="1">
|
||||
<span class="wizard-step-number">1</span>
|
||||
<span class="wizard-step-label">Task Setup</span>
|
||||
</div>
|
||||
<div class="wizard-step" data-step="2">
|
||||
<span class="wizard-step-number">2</span>
|
||||
<span class="wizard-step-label">Launch</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="modal-body">
|
||||
<!-- Step 1: Task & Completion Setup -->
|
||||
<div class="wizard-page" id="ralphWizardStep1">
|
||||
<div class="form-row">
|
||||
<label>Task Description</label>
|
||||
<div class="textarea-with-assist">
|
||||
<textarea
|
||||
id="ralphTaskDescription"
|
||||
rows="6"
|
||||
placeholder="Describe what you want Claude to accomplish...
|
||||
|
||||
Example:
|
||||
Build a REST API for user authentication:
|
||||
- JWT token generation and validation
|
||||
- Login/logout endpoints
|
||||
- Password reset flow
|
||||
- Tests for all endpoints"></textarea>
|
||||
</div>
|
||||
<span class="form-hint">Be specific about requirements and success criteria</span>
|
||||
</div>
|
||||
|
||||
<div class="form-row">
|
||||
<label>Completion Phrase</label>
|
||||
<div class="phrase-input-group">
|
||||
<input type="text" id="ralphCompletionPhrase" value="COMPLETE" placeholder="COMPLETE" list="phrasesSuggestions">
|
||||
<datalist id="phrasesSuggestions">
|
||||
<option value="COMPLETE">
|
||||
<option value="DONE">
|
||||
<option value="FINISHED">
|
||||
<option value="TESTS_PASS">
|
||||
<option value="ALL_TASKS_COMPLETE">
|
||||
</datalist>
|
||||
</div>
|
||||
<span class="form-hint">Claude outputs <code><promise>PHRASE</promise></code> when done</span>
|
||||
</div>
|
||||
|
||||
<div class="form-row">
|
||||
<label>Max Iterations</label>
|
||||
<div class="iteration-presets">
|
||||
<button type="button" class="iteration-preset-btn active" data-iterations="10" onclick="app.selectIterationPreset(10)">10</button>
|
||||
<button type="button" class="iteration-preset-btn" data-iterations="25" onclick="app.selectIterationPreset(25)">25</button>
|
||||
<button type="button" class="iteration-preset-btn" data-iterations="50" onclick="app.selectIterationPreset(50)">50</button>
|
||||
<button type="button" class="iteration-preset-btn" data-iterations="100" onclick="app.selectIterationPreset(100)">100</button>
|
||||
<button type="button" class="iteration-preset-btn" data-iterations="0" onclick="app.selectIterationPreset(0)">∞</button>
|
||||
</div>
|
||||
<span class="form-hint">Safety limit - loop stops after this many iterations</span>
|
||||
</div>
|
||||
|
||||
<div class="form-row">
|
||||
<label>Working Directory</label>
|
||||
<div class="case-select-inline">
|
||||
<select id="ralphCaseSelect" class="form-select"></select>
|
||||
<button class="btn-toolbar btn-sm" onclick="app.showCreateCaseModal()">+ New</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Step 2: Preview & Launch -->
|
||||
<div class="wizard-page hidden" id="ralphWizardStep2">
|
||||
<div class="ralph-preview-section">
|
||||
<label>Prompt Preview</label>
|
||||
<div class="ralph-prompt-preview" id="ralphPromptPreview"></div>
|
||||
</div>
|
||||
|
||||
<div class="ralph-config-summary">
|
||||
<div class="config-summary-row">
|
||||
<span class="config-label">Completion Phrase:</span>
|
||||
<code id="summaryPhrase">COMPLETE</code>
|
||||
</div>
|
||||
<div class="config-summary-row">
|
||||
<span class="config-label">Max Iterations:</span>
|
||||
<span id="summaryIterations">10</span>
|
||||
</div>
|
||||
<div class="config-summary-row">
|
||||
<span class="config-label">Case:</span>
|
||||
<span id="summaryCase">testcase</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Advanced Options (Collapsible) -->
|
||||
<details class="advanced-options">
|
||||
<summary>Advanced Options</summary>
|
||||
<div class="advanced-options-content">
|
||||
<div class="form-row form-row-switch">
|
||||
<label>Enable Respawn</label>
|
||||
<label class="switch">
|
||||
<input type="checkbox" id="ralphEnableRespawn" checked>
|
||||
<span class="slider"></span>
|
||||
</label>
|
||||
<span class="form-hint">Auto-restart sessions when context fills up</span>
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Wizard Footer -->
|
||||
<div class="wizard-footer">
|
||||
<button class="btn-toolbar" id="ralphBackBtn" onclick="app.ralphWizardBack()" style="display: none;">Back</button>
|
||||
<div class="wizard-footer-right">
|
||||
<button class="btn-toolbar" onclick="app.closeRalphWizard()">Cancel</button>
|
||||
<button class="btn-toolbar btn-primary" id="ralphNextBtn" onclick="app.ralphWizardNext()">Next</button>
|
||||
<button class="btn-toolbar btn-success" id="ralphStartBtn" onclick="app.startRalphLoop()" style="display: none;">Start Loop</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Kill All Confirmation Modal -->
|
||||
<div class="modal" id="killAllModal">
|
||||
<div class="modal-backdrop" onclick="app.closeKillAllModal()"></div>
|
||||
|
||||
@@ -67,6 +67,13 @@ body {
|
||||
font-weight: 600;
|
||||
color: var(--accent-hover);
|
||||
letter-spacing: -0.02em;
|
||||
cursor: pointer;
|
||||
transition: color 0.15s, text-shadow 0.15s;
|
||||
}
|
||||
|
||||
.logo:hover {
|
||||
color: #93c5fd;
|
||||
text-shadow: 0 0 8px rgba(96, 165, 250, 0.5);
|
||||
}
|
||||
|
||||
/* Session Tabs */
|
||||
@@ -1435,6 +1442,21 @@ body {
|
||||
color: #a8d8a8;
|
||||
}
|
||||
|
||||
/* Ralph Loop button - golden/yellow theme */
|
||||
.btn-toolbar.btn-ralph {
|
||||
background: linear-gradient(135deg, #3d3415 0%, #4a3f1a 100%);
|
||||
border-color: #6b5a1e;
|
||||
color: #eab308;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.btn-toolbar.btn-ralph:hover {
|
||||
background: linear-gradient(135deg, #4a3f1a 0%, #5a4d20 100%);
|
||||
border-color: #8b7a2e;
|
||||
box-shadow: 0 0 8px rgba(234, 179, 8, 0.4);
|
||||
color: #fcd34d;
|
||||
}
|
||||
|
||||
/* Button icon styling */
|
||||
.btn-toolbar .btn-icon {
|
||||
display: inline-flex;
|
||||
@@ -4600,3 +4622,259 @@ kbd {
|
||||
content: "📄 Click to stream: ";
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
/* ========== Ralph Loop Wizard ========== */
|
||||
|
||||
/* Wizard Progress Indicator */
|
||||
.wizard-progress {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
gap: 2rem;
|
||||
padding: 0.75rem 1rem;
|
||||
background: var(--bg-dark);
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
|
||||
.wizard-step {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
color: var(--text-muted);
|
||||
font-size: 0.8rem;
|
||||
transition: color 0.2s;
|
||||
}
|
||||
|
||||
.wizard-step.active {
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
.wizard-step.completed {
|
||||
color: var(--green);
|
||||
}
|
||||
|
||||
.wizard-step-number {
|
||||
width: 24px;
|
||||
height: 24px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: var(--bg-input);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 50%;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.wizard-step.active .wizard-step-number {
|
||||
background: var(--accent);
|
||||
border-color: var(--accent);
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
.wizard-step.completed .wizard-step-number {
|
||||
background: var(--green);
|
||||
border-color: var(--green);
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
.wizard-step-label {
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
/* Wizard Pages */
|
||||
.wizard-page {
|
||||
min-height: 280px;
|
||||
}
|
||||
|
||||
.wizard-page.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* Wizard Footer */
|
||||
.wizard-footer {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 0.75rem 1rem;
|
||||
border-top: 1px solid var(--border);
|
||||
background: var(--bg-card);
|
||||
}
|
||||
|
||||
.wizard-footer-right {
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
/* Textarea with AI Assist button */
|
||||
.textarea-with-assist {
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.textarea-with-assist textarea {
|
||||
width: 100%;
|
||||
padding: 0.75rem;
|
||||
background: var(--bg-input);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 4px;
|
||||
color: var(--text);
|
||||
font-size: 0.85rem;
|
||||
font-family: 'SF Mono', Monaco, 'Courier New', monospace;
|
||||
resize: vertical;
|
||||
min-height: 140px;
|
||||
}
|
||||
|
||||
.textarea-with-assist textarea:focus {
|
||||
outline: none;
|
||||
border-color: var(--accent);
|
||||
}
|
||||
|
||||
/* Phrase Input Group */
|
||||
.phrase-input-group {
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.phrase-input-group input {
|
||||
width: 100%;
|
||||
padding: 0.5rem 0.75rem;
|
||||
background: var(--bg-input);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 4px;
|
||||
color: var(--text);
|
||||
font-size: 0.85rem;
|
||||
font-family: 'SF Mono', Monaco, 'Courier New', monospace;
|
||||
}
|
||||
|
||||
.phrase-input-group input:focus {
|
||||
outline: none;
|
||||
border-color: var(--accent);
|
||||
}
|
||||
|
||||
/* Iteration Presets */
|
||||
.iteration-presets {
|
||||
display: flex;
|
||||
gap: 0.35rem;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.iteration-preset-btn {
|
||||
padding: 0.5rem 1rem;
|
||||
font-size: 0.8rem;
|
||||
font-weight: 500;
|
||||
background: var(--bg-input);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 4px;
|
||||
color: var(--text-dim);
|
||||
cursor: pointer;
|
||||
transition: all 0.15s;
|
||||
min-width: 50px;
|
||||
}
|
||||
|
||||
.iteration-preset-btn:hover {
|
||||
border-color: var(--border-light);
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
.iteration-preset-btn.active {
|
||||
background: var(--accent);
|
||||
border-color: var(--accent);
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
/* Case Select Inline */
|
||||
.case-select-inline {
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
.case-select-inline select {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
/* Ralph Prompt Preview */
|
||||
.ralph-preview-section {
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.ralph-preview-section > label {
|
||||
display: block;
|
||||
font-size: 0.7rem;
|
||||
color: var(--text-dim);
|
||||
margin-bottom: 0.35rem;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.03em;
|
||||
}
|
||||
|
||||
.ralph-prompt-preview {
|
||||
background: var(--bg-dark);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 4px;
|
||||
padding: 0.75rem;
|
||||
font-family: 'SF Mono', Monaco, 'Courier New', monospace;
|
||||
font-size: 0.75rem;
|
||||
color: var(--text-dim);
|
||||
max-height: 200px;
|
||||
overflow-y: auto;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.ralph-prompt-preview .preview-highlight {
|
||||
color: var(--accent);
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
/* Config Summary */
|
||||
.ralph-config-summary {
|
||||
background: var(--bg-input);
|
||||
border-radius: 4px;
|
||||
padding: 0.75rem;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.config-summary-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
padding: 0.35rem 0;
|
||||
font-size: 0.8rem;
|
||||
}
|
||||
|
||||
.config-summary-row:not(:last-child) {
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
|
||||
.config-label {
|
||||
color: var(--text-dim);
|
||||
}
|
||||
|
||||
.config-summary-row code {
|
||||
background: var(--bg-dark);
|
||||
padding: 0.15rem 0.4rem;
|
||||
border-radius: 3px;
|
||||
font-size: 0.75rem;
|
||||
color: var(--green);
|
||||
}
|
||||
|
||||
/* Advanced Options */
|
||||
.advanced-options {
|
||||
margin-top: 1rem;
|
||||
}
|
||||
|
||||
.advanced-options summary {
|
||||
cursor: pointer;
|
||||
font-size: 0.8rem;
|
||||
color: var(--text-dim);
|
||||
padding: 0.5rem 0;
|
||||
border-top: 1px solid var(--border);
|
||||
}
|
||||
|
||||
.advanced-options summary:hover {
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
.advanced-options[open] summary {
|
||||
margin-bottom: 0.75rem;
|
||||
}
|
||||
|
||||
.advanced-options-content {
|
||||
padding-left: 0.5rem;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,487 @@
|
||||
/**
|
||||
* Agent-Browser Enhanced Test Suite
|
||||
*
|
||||
* Tests Claudeman web UI using agent-browser's AI-agent-friendly features:
|
||||
* - Semantic locators (find by role, label, text)
|
||||
* - Accessibility tree snapshots
|
||||
* - Reference-based element selection (@e1, @e2)
|
||||
*
|
||||
* This demonstrates the "agentic" approach to browser testing where an AI
|
||||
* could navigate the UI using natural language descriptions.
|
||||
*
|
||||
* Port: 3157 (see CLAUDE.md test port table)
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeAll, afterAll, beforeEach, afterEach } from 'vitest';
|
||||
import { execSync, spawnSync } from 'node:child_process';
|
||||
import { WebServer } from '../src/web/server.js';
|
||||
|
||||
const TEST_PORT = 3157;
|
||||
const BASE_URL = `http://localhost:${TEST_PORT}`;
|
||||
const BROWSER_TIMEOUT = 30000;
|
||||
|
||||
// Metrics collection
|
||||
interface TestMetrics {
|
||||
testName: string;
|
||||
duration: number;
|
||||
framework: 'agent-browser';
|
||||
success: boolean;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
const metrics: TestMetrics[] = [];
|
||||
|
||||
// ============================================================================
|
||||
// Agent-Browser CLI Helpers
|
||||
// ============================================================================
|
||||
|
||||
function agentBrowser(command: string): string {
|
||||
try {
|
||||
return execSync(`npx agent-browser ${command}`, {
|
||||
timeout: BROWSER_TIMEOUT,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
}).trim();
|
||||
} catch (error: any) {
|
||||
if (error.stderr) {
|
||||
throw new Error(`agent-browser failed: ${error.stderr}`);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function agentBrowserJson<T = any>(command: string): T {
|
||||
const result = agentBrowser(`${command} --json`);
|
||||
const parsed = JSON.parse(result);
|
||||
if (!parsed.success) {
|
||||
throw new Error(`agent-browser command failed: ${parsed.error || 'unknown error'}`);
|
||||
}
|
||||
return parsed.data;
|
||||
}
|
||||
|
||||
// Get accessibility tree snapshot for AI-based navigation
|
||||
function getSnapshot(): string {
|
||||
try {
|
||||
return agentBrowser('snapshot');
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
// Find element by semantic criteria (AI-friendly)
|
||||
function findByRole(role: string, name?: string): boolean {
|
||||
try {
|
||||
const cmd = name
|
||||
? `find role ${role} --name "${name}"`
|
||||
: `find role ${role}`;
|
||||
const result = agentBrowser(cmd);
|
||||
return result.includes('Found') || !result.includes('not found');
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function findByText(text: string): boolean {
|
||||
try {
|
||||
const result = agentBrowser(`find text "${text}"`);
|
||||
return result.includes('Found') || !result.includes('not found');
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function findByLabel(label: string): boolean {
|
||||
try {
|
||||
const result = agentBrowser(`find label "${label}"`);
|
||||
return result.includes('Found') || !result.includes('not found');
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// Click using semantic finder
|
||||
function clickByText(text: string): boolean {
|
||||
try {
|
||||
agentBrowser(`find text "${text}" click`);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function clickByRole(role: string, name?: string): boolean {
|
||||
try {
|
||||
const cmd = name
|
||||
? `find role ${role} click --name "${name}"`
|
||||
: `find role ${role} click`;
|
||||
agentBrowser(cmd);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// Standard helpers
|
||||
function getText(selector: string): string {
|
||||
try {
|
||||
return agentBrowserJson<{ text: string }>(`get text "${selector}"`).text || '';
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
function isVisible(selector: string): boolean {
|
||||
try {
|
||||
return agentBrowserJson<{ visible: boolean }>(`is visible "${selector}"`).visible;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function getCount(selector: string): number {
|
||||
try {
|
||||
return agentBrowserJson<{ count: number }>(`get count "${selector}"`).count;
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForElement(selector: string, timeout = 10000): Promise<boolean> {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeout) {
|
||||
if (getCount(selector) > 0) return true;
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function closeBrowser() {
|
||||
try {
|
||||
agentBrowser('close');
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
describe('Agent-Browser Enhanced Tests', () => {
|
||||
let server: WebServer;
|
||||
let createdSessions: string[] = [];
|
||||
let browserAvailable = false;
|
||||
let testStartTime: number;
|
||||
|
||||
beforeAll(async () => {
|
||||
closeBrowser();
|
||||
|
||||
// Start web server
|
||||
server = new WebServer(TEST_PORT);
|
||||
await server.start();
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
// Test if agent-browser is available
|
||||
try {
|
||||
agentBrowser(`open ${BASE_URL}`);
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
const title = agentBrowserJson<{ title: string }>('get title');
|
||||
browserAvailable = title.title === 'Claudeman';
|
||||
} catch (e) {
|
||||
console.warn('Agent-browser not available, skipping tests:', (e as Error).message);
|
||||
browserAvailable = false;
|
||||
}
|
||||
}, 60000);
|
||||
|
||||
afterAll(async () => {
|
||||
closeBrowser();
|
||||
|
||||
// Cleanup sessions
|
||||
for (const sessionId of createdSessions) {
|
||||
try {
|
||||
await fetch(`${BASE_URL}/api/sessions/${sessionId}`, { method: 'DELETE' });
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
await server.stop();
|
||||
|
||||
// Output metrics
|
||||
console.log('\n📊 Agent-Browser Test Metrics:');
|
||||
console.table(metrics.map(m => ({
|
||||
test: m.testName.substring(0, 40),
|
||||
duration: `${m.duration}ms`,
|
||||
status: m.success ? '✅' : '❌',
|
||||
})));
|
||||
}, 60000);
|
||||
|
||||
beforeEach(() => {
|
||||
testStartTime = Date.now();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
const testInfo = expect.getState();
|
||||
metrics.push({
|
||||
testName: testInfo.currentTestName || 'unknown',
|
||||
duration: Date.now() - testStartTime,
|
||||
framework: 'agent-browser',
|
||||
success: !testInfo.assertionCalls?.some((c: any) => !c.pass),
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Core Page Tests (CSS Selectors)
|
||||
// ============================================================================
|
||||
|
||||
it('should load the main page with correct title', async () => {
|
||||
if (!browserAvailable) {
|
||||
console.log('Skipping: browser not available');
|
||||
return;
|
||||
}
|
||||
|
||||
const title = agentBrowserJson<{ title: string }>('get title');
|
||||
expect(title.title).toBe('Claudeman');
|
||||
});
|
||||
|
||||
it('should display the logo and branding', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
const logoText = getText('.header-brand .logo');
|
||||
expect(logoText).toBe('Claudeman');
|
||||
});
|
||||
|
||||
it('should show version in toolbar', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
const version = getText('#versionDisplay');
|
||||
expect(version).toMatch(/v\d+\.\d+/);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Semantic/AI-Friendly Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should find elements using semantic role locators', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
// Find buttons by role
|
||||
const hasButtons = findByRole('button');
|
||||
expect(hasButtons).toBe(true);
|
||||
|
||||
// Find specific buttons (may vary based on page structure)
|
||||
// agent-browser uses ARIA roles for semantic finding
|
||||
});
|
||||
|
||||
it('should capture accessibility tree snapshot', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
const snapshot = getSnapshot();
|
||||
|
||||
// Snapshot should contain accessibility information
|
||||
expect(snapshot.length).toBeGreaterThan(0);
|
||||
|
||||
// Should include key page elements
|
||||
// The snapshot format varies by agent-browser version
|
||||
console.log('Accessibility snapshot captured, length:', snapshot.length);
|
||||
});
|
||||
|
||||
it('should find elements by text content', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
// Find elements containing specific text
|
||||
const foundClaudeman = findByText('Claudeman');
|
||||
expect(foundClaudeman).toBe(true);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Modal Tests (CSS + Semantic)
|
||||
// ============================================================================
|
||||
|
||||
it('should open and close help modal', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
// Open using CSS selector
|
||||
agentBrowser('click ".help-btn"');
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
|
||||
expect(isVisible('#helpModal .modal-content')).toBe(true);
|
||||
|
||||
// Find heading by text
|
||||
const headingText = getText('#helpModal h3');
|
||||
expect(headingText).toContain('Keyboard Shortcuts');
|
||||
|
||||
// Close modal
|
||||
agentBrowser('click "#helpModal .modal-close"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
|
||||
expect(isVisible('#helpModal .modal-content')).toBe(false);
|
||||
});
|
||||
|
||||
it('should open settings modal with correct tabs', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
agentBrowser('click ".btn-settings"');
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
|
||||
expect(isVisible('#appSettingsModal .modal-content')).toBe(true);
|
||||
|
||||
const tabCount = getCount('#appSettingsModal .modal-tab-btn');
|
||||
expect(tabCount).toBe(4);
|
||||
|
||||
agentBrowser('click "#appSettingsModal .modal-close"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Session Management Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should create a new session via button click', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
// Click create session
|
||||
agentBrowser('click ".btn-claude"');
|
||||
|
||||
// Wait for session tab
|
||||
const tabFound = await waitForElement('.session-tab', 15000);
|
||||
expect(tabFound).toBe(true);
|
||||
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
expect(isVisible('.session-tab.active')).toBe(true);
|
||||
|
||||
// Terminal should be visible
|
||||
const xtermVisible = await waitForElement('.xterm', 10000);
|
||||
expect(xtermVisible).toBe(true);
|
||||
|
||||
// Track for cleanup
|
||||
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||
const data = await response.json();
|
||||
if (data.sessions?.length > 0) {
|
||||
const lastSession = data.sessions[data.sessions.length - 1];
|
||||
if (lastSession && !createdSessions.includes(lastSession.id)) {
|
||||
createdSessions.push(lastSession.id);
|
||||
}
|
||||
}
|
||||
}, 30000);
|
||||
|
||||
// ============================================================================
|
||||
// AI-Driven Navigation Demo
|
||||
// ============================================================================
|
||||
|
||||
it('should demonstrate AI-friendly navigation workflow', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
// This test demonstrates how an AI agent could navigate the UI
|
||||
// using semantic locators and natural language-like commands
|
||||
|
||||
// Step 1: Check page state via snapshot
|
||||
const snapshot = getSnapshot();
|
||||
expect(snapshot.length).toBeGreaterThan(0);
|
||||
|
||||
// Step 2: Find and interact with settings
|
||||
// An AI would parse the snapshot and decide what to click
|
||||
const settingsVisible = isVisible('.btn-settings');
|
||||
expect(settingsVisible).toBe(true);
|
||||
|
||||
// Step 3: Open settings
|
||||
agentBrowser('click ".btn-settings"');
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
|
||||
// Step 4: Take new snapshot to understand modal state
|
||||
const modalSnapshot = getSnapshot();
|
||||
|
||||
// Step 5: Close settings
|
||||
agentBrowser('click "#appSettingsModal .modal-close"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
|
||||
// This workflow shows how an AI could:
|
||||
// 1. Observe the page state (snapshot)
|
||||
// 2. Decide on actions based on semantic understanding
|
||||
// 3. Execute actions
|
||||
// 4. Verify results with new snapshot
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Form Interaction Tests (Semantic)
|
||||
// ============================================================================
|
||||
|
||||
it('should interact with quick start dropdown', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
// Find the quick start dropdown
|
||||
const dropdownVisible = isVisible('#quickStartCase');
|
||||
expect(dropdownVisible).toBe(true);
|
||||
|
||||
// Get current value
|
||||
// Note: agent-browser's get value command for select elements
|
||||
try {
|
||||
const value = agentBrowserJson<{ value: string }>('get value "#quickStartCase"');
|
||||
// Value should be the current selected case
|
||||
expect(typeof value.value).toBe('string');
|
||||
} catch {
|
||||
// May not be a standard select element
|
||||
}
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Multiple Element Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should count toolbar elements correctly', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
// Count various toolbar elements
|
||||
const fontControls = getCount('.header-font-controls .font-btn');
|
||||
expect(fontControls).toBeGreaterThanOrEqual(2);
|
||||
|
||||
// Tab count badge should exist
|
||||
expect(isVisible('#tabCount')).toBe(true);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Screenshot Test
|
||||
// ============================================================================
|
||||
|
||||
it('should capture screenshot for AI analysis', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
try {
|
||||
// Capture screenshot (useful for visual AI analysis)
|
||||
agentBrowser('screenshot /tmp/claudeman-test.png');
|
||||
|
||||
// Verify file was created (would exist after successful screenshot)
|
||||
// In a real AI scenario, this image would be analyzed
|
||||
} catch {
|
||||
// Screenshot may fail in headless mode without proper setup
|
||||
}
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Stress Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should handle rapid interactions', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
// Rapid modal open/close
|
||||
for (let i = 0; i < 3; i++) {
|
||||
agentBrowser('click ".help-btn"');
|
||||
await new Promise(r => setTimeout(r, 200));
|
||||
try {
|
||||
agentBrowser('click "#helpModal .modal-close"');
|
||||
} catch { /* may already be closed */ }
|
||||
await new Promise(r => setTimeout(r, 200));
|
||||
}
|
||||
|
||||
// Page should still be functional
|
||||
expect(isVisible('.header')).toBe(true);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Tab Navigation Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should handle tab count display', async () => {
|
||||
if (!browserAvailable) return;
|
||||
|
||||
const tabCountText = getText('#tabCount');
|
||||
// Should show number of tabs
|
||||
expect(tabCountText).toMatch(/\d+/);
|
||||
});
|
||||
});
|
||||
|
||||
// Export metrics for comparison
|
||||
export { metrics as agentBrowserMetrics };
|
||||
@@ -0,0 +1,624 @@
|
||||
/**
|
||||
* Browser Framework Comparison Benchmark
|
||||
*
|
||||
* Runs identical test scenarios across Playwright, Puppeteer, and Agent-Browser
|
||||
* to compare performance, reliability, and ease of use for agentic testing.
|
||||
*
|
||||
* This benchmark measures:
|
||||
* - Execution time per operation
|
||||
* - Reliability (success rate)
|
||||
* - Memory usage (where available)
|
||||
* - Ease of implementation (qualitative)
|
||||
*
|
||||
* Ports: 3158 (Playwright), 3159 (Puppeteer), 3160 (Agent-Browser)
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import { chromium, Browser as PlaywrightBrowser, Page as PlaywrightPage } from 'playwright';
|
||||
import puppeteer, { Browser as PuppeteerBrowser, Page as PuppeteerPage } from 'puppeteer';
|
||||
import { execSync } from 'node:child_process';
|
||||
import { WebServer } from '../src/web/server.js';
|
||||
|
||||
// Port allocation
|
||||
const PORTS = {
|
||||
playwright: 3158,
|
||||
puppeteer: 3159,
|
||||
agentBrowser: 3160,
|
||||
};
|
||||
|
||||
// ============================================================================
|
||||
// Types
|
||||
// ============================================================================
|
||||
|
||||
interface BenchmarkResult {
|
||||
scenario: string;
|
||||
framework: 'playwright' | 'puppeteer' | 'agent-browser';
|
||||
duration: number;
|
||||
success: boolean;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
interface AggregateStats {
|
||||
framework: string;
|
||||
totalTests: number;
|
||||
passed: number;
|
||||
failed: number;
|
||||
avgDuration: number;
|
||||
minDuration: number;
|
||||
maxDuration: number;
|
||||
successRate: string;
|
||||
}
|
||||
|
||||
const results: BenchmarkResult[] = [];
|
||||
|
||||
// ============================================================================
|
||||
// Agent-Browser Helpers
|
||||
// ============================================================================
|
||||
|
||||
function agentBrowser(command: string): string {
|
||||
try {
|
||||
return execSync(`npx agent-browser ${command}`, {
|
||||
timeout: 30000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
}).trim();
|
||||
} catch (error: any) {
|
||||
throw new Error(`agent-browser failed: ${error.stderr || error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
function agentBrowserJson<T>(command: string): T {
|
||||
const result = agentBrowser(`${command} --json`);
|
||||
const parsed = JSON.parse(result);
|
||||
if (!parsed.success) {
|
||||
throw new Error(`agent-browser: ${parsed.error || 'unknown error'}`);
|
||||
}
|
||||
return parsed.data;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Benchmark Scenarios
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Each scenario is a function that takes framework-specific parameters
|
||||
* and returns a promise that resolves when the scenario is complete.
|
||||
*/
|
||||
|
||||
async function runBenchmark(
|
||||
scenario: string,
|
||||
framework: 'playwright' | 'puppeteer' | 'agent-browser',
|
||||
fn: () => Promise<void>
|
||||
): Promise<void> {
|
||||
const start = Date.now();
|
||||
let success = false;
|
||||
let error: string | undefined;
|
||||
|
||||
try {
|
||||
await fn();
|
||||
success = true;
|
||||
} catch (e: any) {
|
||||
error = e.message;
|
||||
}
|
||||
|
||||
results.push({
|
||||
scenario,
|
||||
framework,
|
||||
duration: Date.now() - start,
|
||||
success,
|
||||
error,
|
||||
});
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Main Comparison Suite
|
||||
// ============================================================================
|
||||
|
||||
describe('Browser Framework Comparison Benchmark', () => {
|
||||
// Framework instances
|
||||
let playwrightBrowser: PlaywrightBrowser;
|
||||
let puppeteerBrowser: PuppeteerBrowser;
|
||||
let agentBrowserAvailable = false;
|
||||
|
||||
// Web servers
|
||||
let servers: WebServer[] = [];
|
||||
|
||||
// Cleanup tracking
|
||||
let createdSessions: { port: number; id: string }[] = [];
|
||||
|
||||
beforeAll(async () => {
|
||||
console.log('\n🚀 Starting Browser Framework Comparison Benchmark\n');
|
||||
console.log('This benchmark compares:');
|
||||
console.log(' - Playwright (direct API)');
|
||||
console.log(' - Puppeteer (Chrome DevTools Protocol)');
|
||||
console.log(' - Agent-Browser (CLI-based, AI-friendly)\n');
|
||||
|
||||
// Start web servers for each framework
|
||||
for (const [name, port] of Object.entries(PORTS)) {
|
||||
const server = new WebServer(port);
|
||||
await server.start();
|
||||
servers.push(server);
|
||||
console.log(`✅ Started server for ${name} on port ${port}`);
|
||||
}
|
||||
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
// Initialize Playwright
|
||||
try {
|
||||
playwrightBrowser = await chromium.launch({
|
||||
headless: true,
|
||||
args: ['--no-sandbox', '--disable-setuid-sandbox'],
|
||||
});
|
||||
console.log('✅ Playwright browser launched');
|
||||
} catch (e: any) {
|
||||
console.warn('⚠️ Playwright launch failed:', e.message);
|
||||
}
|
||||
|
||||
// Initialize Puppeteer
|
||||
try {
|
||||
puppeteerBrowser = await puppeteer.launch({
|
||||
headless: true,
|
||||
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||
});
|
||||
console.log('✅ Puppeteer browser launched');
|
||||
} catch (e: any) {
|
||||
console.warn('⚠️ Puppeteer launch failed:', e.message);
|
||||
}
|
||||
|
||||
// Test Agent-Browser availability
|
||||
try {
|
||||
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
const title = agentBrowserJson<{ title: string }>('get title');
|
||||
agentBrowserAvailable = title.title === 'Claudeman';
|
||||
if (agentBrowserAvailable) {
|
||||
console.log('✅ Agent-Browser available');
|
||||
}
|
||||
} catch (e: any) {
|
||||
console.warn('⚠️ Agent-Browser not available:', e.message);
|
||||
}
|
||||
|
||||
console.log('\n' + '='.repeat(60) + '\n');
|
||||
}, 120000);
|
||||
|
||||
afterAll(async () => {
|
||||
// Cleanup sessions
|
||||
for (const { port, id } of createdSessions) {
|
||||
try {
|
||||
await fetch(`http://localhost:${port}/api/sessions/${id}`, { method: 'DELETE' });
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
// Close browsers
|
||||
if (playwrightBrowser) await playwrightBrowser.close();
|
||||
if (puppeteerBrowser) await puppeteerBrowser.close();
|
||||
try { agentBrowser('close'); } catch { /* ignore */ }
|
||||
|
||||
// Stop servers
|
||||
for (const server of servers) {
|
||||
await server.stop();
|
||||
}
|
||||
|
||||
// Print results summary
|
||||
printResultsSummary();
|
||||
}, 60000);
|
||||
|
||||
// ============================================================================
|
||||
// Scenario 1: Page Load
|
||||
// ============================================================================
|
||||
|
||||
describe('Scenario: Page Load', () => {
|
||||
it('Playwright - page load', async () => {
|
||||
if (!playwrightBrowser) return;
|
||||
|
||||
await runBenchmark('page-load', 'playwright', async () => {
|
||||
const context = await playwrightBrowser.newContext();
|
||||
const page = await context.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
await expect(page).toHaveTitle('Claudeman');
|
||||
await context.close();
|
||||
});
|
||||
});
|
||||
|
||||
it('Puppeteer - page load', async () => {
|
||||
if (!puppeteerBrowser) return;
|
||||
|
||||
await runBenchmark('page-load', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
const title = await page.title();
|
||||
expect(title).toBe('Claudeman');
|
||||
await page.close();
|
||||
});
|
||||
});
|
||||
|
||||
it('Agent-Browser - page load', async () => {
|
||||
if (!agentBrowserAvailable) return;
|
||||
|
||||
await runBenchmark('page-load', 'agent-browser', async () => {
|
||||
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
const title = agentBrowserJson<{ title: string }>('get title');
|
||||
expect(title.title).toBe('Claudeman');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Scenario 2: Element Selection
|
||||
// ============================================================================
|
||||
|
||||
describe('Scenario: Element Selection', () => {
|
||||
it('Playwright - element selection', async () => {
|
||||
if (!playwrightBrowser) return;
|
||||
|
||||
await runBenchmark('element-selection', 'playwright', async () => {
|
||||
const context = await playwrightBrowser.newContext();
|
||||
const page = await context.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
|
||||
// Multiple element selections
|
||||
const logo = await page.locator('.header-brand .logo').textContent();
|
||||
const version = await page.locator('#versionDisplay').textContent();
|
||||
const btnCount = await page.locator('.btn-claude').count();
|
||||
|
||||
expect(logo).toBe('Claudeman');
|
||||
expect(version).toMatch(/v\d+/);
|
||||
expect(btnCount).toBeGreaterThan(0);
|
||||
|
||||
await context.close();
|
||||
});
|
||||
});
|
||||
|
||||
it('Puppeteer - element selection', async () => {
|
||||
if (!puppeteerBrowser) return;
|
||||
|
||||
await runBenchmark('element-selection', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
|
||||
const logo = await page.$eval('.header-brand .logo', el => el.textContent);
|
||||
const version = await page.$eval('#versionDisplay', el => el.textContent);
|
||||
const btns = await page.$$('.btn-claude');
|
||||
|
||||
expect(logo).toBe('Claudeman');
|
||||
expect(version).toMatch(/v\d+/);
|
||||
expect(btns.length).toBeGreaterThan(0);
|
||||
|
||||
await page.close();
|
||||
});
|
||||
});
|
||||
|
||||
it('Agent-Browser - element selection', async () => {
|
||||
if (!agentBrowserAvailable) return;
|
||||
|
||||
await runBenchmark('element-selection', 'agent-browser', async () => {
|
||||
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
const logo = agentBrowserJson<{ text: string }>('get text ".header-brand .logo"');
|
||||
const version = agentBrowserJson<{ text: string }>('get text "#versionDisplay"');
|
||||
const btnCount = agentBrowserJson<{ count: number }>('get count ".btn-claude"');
|
||||
|
||||
expect(logo.text).toBe('Claudeman');
|
||||
expect(version.text).toMatch(/v\d+/);
|
||||
expect(btnCount.count).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Scenario 3: Modal Interaction
|
||||
// ============================================================================
|
||||
|
||||
describe('Scenario: Modal Open/Close', () => {
|
||||
it('Playwright - modal interaction', async () => {
|
||||
if (!playwrightBrowser) return;
|
||||
|
||||
await runBenchmark('modal-interaction', 'playwright', async () => {
|
||||
const context = await playwrightBrowser.newContext();
|
||||
const page = await context.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
|
||||
// Open modal
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||
|
||||
// Verify content
|
||||
const heading = await page.locator('#helpModal h3').textContent();
|
||||
expect(heading).toContain('Keyboard Shortcuts');
|
||||
|
||||
// Close modal
|
||||
await page.click('#helpModal .modal-close');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||
|
||||
await context.close();
|
||||
});
|
||||
});
|
||||
|
||||
it('Puppeteer - modal interaction', async () => {
|
||||
if (!puppeteerBrowser) return;
|
||||
|
||||
await runBenchmark('modal-interaction', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
|
||||
// Open modal
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||
|
||||
// Verify content
|
||||
const heading = await page.$eval('#helpModal h3', el => el.textContent);
|
||||
expect(heading).toContain('Keyboard Shortcuts');
|
||||
|
||||
// Close modal
|
||||
await page.click('#helpModal .modal-close');
|
||||
await page.waitForFunction(() => {
|
||||
const modal = document.querySelector('#helpModal .modal-content');
|
||||
return !modal || window.getComputedStyle(modal).display === 'none';
|
||||
});
|
||||
|
||||
await page.close();
|
||||
});
|
||||
});
|
||||
|
||||
it('Agent-Browser - modal interaction', async () => {
|
||||
if (!agentBrowserAvailable) return;
|
||||
|
||||
await runBenchmark('modal-interaction', 'agent-browser', async () => {
|
||||
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
// Open modal
|
||||
agentBrowser('click ".help-btn"');
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
|
||||
// Verify content
|
||||
const heading = agentBrowserJson<{ text: string }>('get text "#helpModal h3"');
|
||||
expect(heading.text).toContain('Keyboard Shortcuts');
|
||||
|
||||
// Close modal
|
||||
agentBrowser('click "#helpModal .modal-close"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Scenario 4: Session Creation (Complex)
|
||||
// ============================================================================
|
||||
|
||||
describe('Scenario: Session Creation', () => {
|
||||
it('Playwright - session creation', async () => {
|
||||
if (!playwrightBrowser) return;
|
||||
|
||||
await runBenchmark('session-creation', 'playwright', async () => {
|
||||
const context = await playwrightBrowser.newContext();
|
||||
const page = await context.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
|
||||
// Create session
|
||||
await page.click('.btn-claude');
|
||||
|
||||
// Wait for session tab
|
||||
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||
|
||||
// Verify terminal appears
|
||||
await page.waitForSelector('.xterm', { timeout: 10000 });
|
||||
|
||||
// Track for cleanup
|
||||
const response = await fetch(`http://localhost:${PORTS.playwright}/api/sessions`);
|
||||
const data = await response.json();
|
||||
const lastSession = data.sessions?.[data.sessions.length - 1];
|
||||
if (lastSession) {
|
||||
createdSessions.push({ port: PORTS.playwright, id: lastSession.id });
|
||||
}
|
||||
|
||||
await context.close();
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
it('Puppeteer - session creation', async () => {
|
||||
if (!puppeteerBrowser) return;
|
||||
|
||||
await runBenchmark('session-creation', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
|
||||
// Create session
|
||||
await page.click('.btn-claude');
|
||||
|
||||
// Wait for session tab
|
||||
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||
|
||||
// Verify terminal appears
|
||||
await page.waitForSelector('.xterm', { timeout: 10000 });
|
||||
|
||||
// Track for cleanup
|
||||
const response = await fetch(`http://localhost:${PORTS.puppeteer}/api/sessions`);
|
||||
const data = await response.json();
|
||||
const lastSession = data.sessions?.[data.sessions.length - 1];
|
||||
if (lastSession) {
|
||||
createdSessions.push({ port: PORTS.puppeteer, id: lastSession.id });
|
||||
}
|
||||
|
||||
await page.close();
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
it('Agent-Browser - session creation', async () => {
|
||||
if (!agentBrowserAvailable) return;
|
||||
|
||||
await runBenchmark('session-creation', 'agent-browser', async () => {
|
||||
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
// Create session
|
||||
agentBrowser('click ".btn-claude"');
|
||||
|
||||
// Wait for session tab
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < 15000) {
|
||||
try {
|
||||
const count = agentBrowserJson<{ count: number }>('get count ".session-tab"');
|
||||
if (count.count > 0) break;
|
||||
} catch { /* retry */ }
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
}
|
||||
|
||||
// Verify terminal appears
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
const xtermCount = agentBrowserJson<{ count: number }>('get count ".xterm"');
|
||||
expect(xtermCount.count).toBeGreaterThan(0);
|
||||
|
||||
// Track for cleanup
|
||||
const response = await fetch(`http://localhost:${PORTS.agentBrowser}/api/sessions`);
|
||||
const data = await response.json();
|
||||
const lastSession = data.sessions?.[data.sessions.length - 1];
|
||||
if (lastSession) {
|
||||
createdSessions.push({ port: PORTS.agentBrowser, id: lastSession.id });
|
||||
}
|
||||
});
|
||||
}, 30000);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Scenario 5: Rapid Operations
|
||||
// ============================================================================
|
||||
|
||||
describe('Scenario: Rapid Operations (5 modal cycles)', () => {
|
||||
it('Playwright - rapid operations', async () => {
|
||||
if (!playwrightBrowser) return;
|
||||
|
||||
await runBenchmark('rapid-operations', 'playwright', async () => {
|
||||
const context = await playwrightBrowser.newContext();
|
||||
const page = await context.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.playwright}`);
|
||||
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||
await page.click('#helpModal .modal-close');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||
}
|
||||
|
||||
await context.close();
|
||||
});
|
||||
});
|
||||
|
||||
it('Puppeteer - rapid operations', async () => {
|
||||
if (!puppeteerBrowser) return;
|
||||
|
||||
await runBenchmark('rapid-operations', 'puppeteer', async () => {
|
||||
const page = await puppeteerBrowser.newPage();
|
||||
await page.goto(`http://localhost:${PORTS.puppeteer}`);
|
||||
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||
await page.click('#helpModal .modal-close');
|
||||
await new Promise(r => setTimeout(r, 200));
|
||||
}
|
||||
|
||||
await page.close();
|
||||
});
|
||||
});
|
||||
|
||||
it('Agent-Browser - rapid operations', async () => {
|
||||
if (!agentBrowserAvailable) return;
|
||||
|
||||
await runBenchmark('rapid-operations', 'agent-browser', async () => {
|
||||
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
for (let i = 0; i < 5; i++) {
|
||||
agentBrowser('click ".help-btn"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
agentBrowser('click "#helpModal .modal-close"');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Results Summary
|
||||
// ============================================================================
|
||||
|
||||
function printResultsSummary(): void {
|
||||
console.log('\n' + '='.repeat(70));
|
||||
console.log('📊 BROWSER FRAMEWORK COMPARISON RESULTS');
|
||||
console.log('='.repeat(70) + '\n');
|
||||
|
||||
// Group by framework
|
||||
const frameworks = ['playwright', 'puppeteer', 'agent-browser'] as const;
|
||||
const stats: AggregateStats[] = [];
|
||||
|
||||
for (const framework of frameworks) {
|
||||
const frameworkResults = results.filter(r => r.framework === framework);
|
||||
if (frameworkResults.length === 0) continue;
|
||||
|
||||
const passed = frameworkResults.filter(r => r.success).length;
|
||||
const durations = frameworkResults.map(r => r.duration);
|
||||
|
||||
stats.push({
|
||||
framework,
|
||||
totalTests: frameworkResults.length,
|
||||
passed,
|
||||
failed: frameworkResults.length - passed,
|
||||
avgDuration: Math.round(durations.reduce((a, b) => a + b, 0) / durations.length),
|
||||
minDuration: Math.min(...durations),
|
||||
maxDuration: Math.max(...durations),
|
||||
successRate: ((passed / frameworkResults.length) * 100).toFixed(1) + '%',
|
||||
});
|
||||
}
|
||||
|
||||
// Print aggregate stats
|
||||
console.log('📈 AGGREGATE STATISTICS\n');
|
||||
console.table(stats.map(s => ({
|
||||
Framework: s.framework.toUpperCase(),
|
||||
Tests: s.totalTests,
|
||||
Passed: s.passed,
|
||||
Failed: s.failed,
|
||||
'Success Rate': s.successRate,
|
||||
'Avg (ms)': s.avgDuration,
|
||||
'Min (ms)': s.minDuration,
|
||||
'Max (ms)': s.maxDuration,
|
||||
})));
|
||||
|
||||
// Print detailed results by scenario
|
||||
console.log('\n📋 DETAILED RESULTS BY SCENARIO\n');
|
||||
|
||||
const scenarios = [...new Set(results.map(r => r.scenario))];
|
||||
for (const scenario of scenarios) {
|
||||
console.log(`\n🔹 ${scenario.toUpperCase()}`);
|
||||
const scenarioResults = results.filter(r => r.scenario === scenario);
|
||||
|
||||
console.table(scenarioResults.map(r => ({
|
||||
Framework: r.framework,
|
||||
Duration: `${r.duration}ms`,
|
||||
Status: r.success ? '✅ Pass' : '❌ Fail',
|
||||
Error: r.error ? r.error.substring(0, 40) + '...' : '-',
|
||||
})));
|
||||
}
|
||||
|
||||
// Performance comparison
|
||||
console.log('\n🏆 PERFORMANCE RANKING (by avg duration)\n');
|
||||
const ranked = [...stats].sort((a, b) => a.avgDuration - b.avgDuration);
|
||||
ranked.forEach((s, i) => {
|
||||
const medal = i === 0 ? '🥇' : i === 1 ? '🥈' : '🥉';
|
||||
console.log(`${medal} ${s.framework.toUpperCase()}: ${s.avgDuration}ms avg (${s.successRate} success rate)`);
|
||||
});
|
||||
|
||||
// Recommendations
|
||||
console.log('\n💡 RECOMMENDATIONS\n');
|
||||
console.log('• Playwright: Best for comprehensive testing with excellent debugging');
|
||||
console.log('• Puppeteer: Best for Chrome-specific features and CDP access');
|
||||
console.log('• Agent-Browser: Best for AI agents with semantic navigation needs');
|
||||
|
||||
console.log('\n' + '='.repeat(70) + '\n');
|
||||
}
|
||||
|
||||
// Export results for external analysis
|
||||
export { results as comparisonResults };
|
||||
@@ -0,0 +1,194 @@
|
||||
/**
|
||||
* Playwright Direct API Test Suite
|
||||
*
|
||||
* Tests Claudeman web UI using Playwright's native API directly.
|
||||
* This provides the most control and best debugging capabilities.
|
||||
*
|
||||
* Port: 3155 (see CLAUDE.md test port table)
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import { chromium, Browser, Page, BrowserContext } from 'playwright';
|
||||
import { WebServer } from '../src/web/server.js';
|
||||
|
||||
const TEST_PORT = 3155;
|
||||
const BASE_URL = `http://localhost:${TEST_PORT}`;
|
||||
|
||||
// Metrics collection
|
||||
interface TestMetrics {
|
||||
testName: string;
|
||||
duration: number;
|
||||
framework: 'playwright';
|
||||
success: boolean;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
const metrics: TestMetrics[] = [];
|
||||
let server: WebServer;
|
||||
let browser: Browser;
|
||||
let createdSessions: string[] = [];
|
||||
let setupComplete = false;
|
||||
|
||||
describe('Playwright Direct API Tests', () => {
|
||||
beforeAll(async () => {
|
||||
// Start web server
|
||||
server = new WebServer(TEST_PORT);
|
||||
await server.start();
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
// Launch browser
|
||||
browser = await chromium.launch({
|
||||
headless: true,
|
||||
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'],
|
||||
});
|
||||
setupComplete = true;
|
||||
}, 30000);
|
||||
|
||||
afterAll(async () => {
|
||||
// Cleanup sessions
|
||||
for (const sessionId of createdSessions) {
|
||||
try {
|
||||
await fetch(`${BASE_URL}/api/sessions/${sessionId}`, { method: 'DELETE' });
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
// Close browser and server
|
||||
if (browser) await browser.close();
|
||||
if (server) await server.stop();
|
||||
|
||||
// Output metrics
|
||||
console.log('\n📊 Playwright Test Metrics:');
|
||||
console.table(metrics.map(m => ({
|
||||
test: m.testName.substring(0, 40),
|
||||
duration: `${m.duration}ms`,
|
||||
status: m.success ? '✅' : '❌',
|
||||
})));
|
||||
}, 30000);
|
||||
|
||||
// Helper to run test with metrics
|
||||
async function withMetrics(name: string, fn: (page: Page) => Promise<void>) {
|
||||
if (!setupComplete) {
|
||||
console.log('Skipping - setup not complete');
|
||||
return;
|
||||
}
|
||||
const start = Date.now();
|
||||
let success = false;
|
||||
let error: string | undefined;
|
||||
const context = await browser.newContext({ viewport: { width: 1280, height: 720 } });
|
||||
const page = await context.newPage();
|
||||
try {
|
||||
await fn(page);
|
||||
success = true;
|
||||
} catch (e: any) {
|
||||
error = e.message;
|
||||
throw e;
|
||||
} finally {
|
||||
await context.close();
|
||||
metrics.push({ testName: name, duration: Date.now() - start, framework: 'playwright', success, error });
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Core Page Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should load the main page with correct title', async () => {
|
||||
await withMetrics('page-load', async (page) => {
|
||||
await page.goto(BASE_URL);
|
||||
await expect(page).toHaveTitle('Claudeman');
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
it('should display the logo and branding', async () => {
|
||||
await withMetrics('logo-branding', async (page) => {
|
||||
await page.goto(BASE_URL);
|
||||
const logo = page.locator('.header-brand .logo');
|
||||
await expect(logo).toBeVisible();
|
||||
await expect(logo).toHaveText('Claudeman');
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
it('should show version in toolbar', async () => {
|
||||
await withMetrics('version-display', async (page) => {
|
||||
await page.goto(BASE_URL);
|
||||
const version = page.locator('#versionDisplay');
|
||||
await expect(version).toBeVisible();
|
||||
await expect(version).toHaveText(/v\d+\.\d+/);
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
// ============================================================================
|
||||
// Modal Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should open and close help modal', async () => {
|
||||
await withMetrics('help-modal', async (page) => {
|
||||
await page.goto(BASE_URL);
|
||||
await page.click('.help-btn');
|
||||
const modal = page.locator('#helpModal .modal-content');
|
||||
await expect(modal).toBeVisible();
|
||||
const heading = page.locator('#helpModal h3');
|
||||
await expect(heading).toContainText('Keyboard Shortcuts');
|
||||
await page.click('#helpModal .modal-close');
|
||||
await expect(modal).not.toBeVisible();
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
it('should open settings modal with 4 tabs', async () => {
|
||||
await withMetrics('settings-modal', async (page) => {
|
||||
await page.goto(BASE_URL);
|
||||
await page.click('.btn-settings');
|
||||
const modal = page.locator('#appSettingsModal .modal-content');
|
||||
await expect(modal).toBeVisible();
|
||||
const tabs = page.locator('#appSettingsModal .modal-tab-btn');
|
||||
await expect(tabs).toHaveCount(4);
|
||||
await page.click('#appSettingsModal .modal-close');
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
// ============================================================================
|
||||
// Session Management Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should create a new session via button click', async () => {
|
||||
await withMetrics('session-creation', async (page) => {
|
||||
await page.goto(BASE_URL);
|
||||
await page.click('.btn-claude');
|
||||
const sessionTab = page.locator('.session-tab');
|
||||
await expect(sessionTab.first()).toBeVisible({ timeout: 15000 });
|
||||
await expect(page.locator('.session-tab.active')).toBeVisible();
|
||||
const terminal = page.locator('.xterm');
|
||||
await expect(terminal).toBeVisible({ timeout: 10000 });
|
||||
|
||||
// Track session for cleanup
|
||||
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||
const data = await response.json();
|
||||
if (data.sessions?.length > 0) {
|
||||
const lastSession = data.sessions[data.sessions.length - 1];
|
||||
if (lastSession && !createdSessions.includes(lastSession.id)) {
|
||||
createdSessions.push(lastSession.id);
|
||||
}
|
||||
}
|
||||
});
|
||||
}, 60000);
|
||||
|
||||
// ============================================================================
|
||||
// Stress Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should handle rapid modal interactions', async () => {
|
||||
await withMetrics('rapid-interactions', async (page) => {
|
||||
await page.goto(BASE_URL);
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'visible' });
|
||||
await page.click('#helpModal .modal-close');
|
||||
await page.waitForSelector('#helpModal .modal-content', { state: 'hidden' });
|
||||
}
|
||||
await expect(page.locator('.header')).toBeVisible();
|
||||
});
|
||||
}, 30000);
|
||||
});
|
||||
|
||||
// Export metrics for comparison
|
||||
export { metrics as playwrightMetrics };
|
||||
@@ -0,0 +1,445 @@
|
||||
/**
|
||||
* Puppeteer/Chrome Test Suite
|
||||
*
|
||||
* Tests Claudeman web UI using Puppeteer for direct Chrome DevTools Protocol access.
|
||||
* Puppeteer provides lower-level Chrome control compared to Playwright.
|
||||
*
|
||||
* Port: 3156 (see CLAUDE.md test port table)
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeAll, afterAll, beforeEach, afterEach } from 'vitest';
|
||||
import puppeteer, { Browser, Page } from 'puppeteer';
|
||||
import { WebServer } from '../src/web/server.js';
|
||||
|
||||
const TEST_PORT = 3156;
|
||||
const BASE_URL = `http://localhost:${TEST_PORT}`;
|
||||
|
||||
// Metrics collection
|
||||
interface TestMetrics {
|
||||
testName: string;
|
||||
duration: number;
|
||||
framework: 'puppeteer';
|
||||
success: boolean;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
const metrics: TestMetrics[] = [];
|
||||
|
||||
describe('Puppeteer/Chrome Tests', () => {
|
||||
let server: WebServer;
|
||||
let browser: Browser;
|
||||
let page: Page;
|
||||
let createdSessions: string[] = [];
|
||||
let testStartTime: number;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Start web server
|
||||
server = new WebServer(TEST_PORT);
|
||||
await server.start();
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
// Launch Chrome via Puppeteer
|
||||
browser = await puppeteer.launch({
|
||||
headless: true,
|
||||
args: [
|
||||
'--no-sandbox',
|
||||
'--disable-setuid-sandbox',
|
||||
'--disable-dev-shm-usage',
|
||||
'--disable-gpu',
|
||||
],
|
||||
});
|
||||
}, 60000);
|
||||
|
||||
afterAll(async () => {
|
||||
// Cleanup sessions
|
||||
for (const sessionId of createdSessions) {
|
||||
try {
|
||||
await fetch(`${BASE_URL}/api/sessions/${sessionId}`, { method: 'DELETE' });
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
// Close browser and server
|
||||
if (browser) await browser.close();
|
||||
await server.stop();
|
||||
|
||||
// Output metrics
|
||||
console.log('\n📊 Puppeteer Test Metrics:');
|
||||
console.table(metrics.map(m => ({
|
||||
test: m.testName.substring(0, 40),
|
||||
duration: `${m.duration}ms`,
|
||||
status: m.success ? '✅' : '❌',
|
||||
})));
|
||||
}, 60000);
|
||||
|
||||
beforeEach(async () => {
|
||||
page = await browser.newPage();
|
||||
await page.setViewport({ width: 1280, height: 720 });
|
||||
testStartTime = Date.now();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
const testInfo = expect.getState();
|
||||
metrics.push({
|
||||
testName: testInfo.currentTestName || 'unknown',
|
||||
duration: Date.now() - testStartTime,
|
||||
framework: 'puppeteer',
|
||||
success: !testInfo.assertionCalls?.some((c: any) => !c.pass),
|
||||
});
|
||||
|
||||
if (page) await page.close();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Core Page Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should load the main page with correct title', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
const title = await page.title();
|
||||
expect(title).toBe('Claudeman');
|
||||
});
|
||||
|
||||
it('should display the logo and branding', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
await page.waitForSelector('.header-brand .logo');
|
||||
const logoText = await page.$eval('.header-brand .logo', el => el.textContent);
|
||||
expect(logoText).toBe('Claudeman');
|
||||
});
|
||||
|
||||
it('should show version in toolbar', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
await page.waitForSelector('#versionDisplay');
|
||||
const version = await page.$eval('#versionDisplay', el => el.textContent);
|
||||
expect(version).toMatch(/v\d+\.\d+/);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Button & Control Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should have all main control buttons visible', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
const btnClaude = await page.$('.btn-claude');
|
||||
const btnSettings = await page.$('.btn-settings');
|
||||
const helpBtn = await page.$('.help-btn');
|
||||
const quickStart = await page.$('#quickStartCase');
|
||||
|
||||
expect(btnClaude).not.toBeNull();
|
||||
expect(btnSettings).not.toBeNull();
|
||||
expect(helpBtn).not.toBeNull();
|
||||
expect(quickStart).not.toBeNull();
|
||||
});
|
||||
|
||||
it('should have font size controls', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
const fontControls = await page.$('.header-font-controls');
|
||||
expect(fontControls).not.toBeNull();
|
||||
|
||||
const fontBtns = await page.$$('.header-font-controls .font-btn');
|
||||
expect(fontBtns.length).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Modal Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should open and close help modal', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
// Open help modal
|
||||
await page.click('.help-btn');
|
||||
await page.waitForSelector('#helpModal .modal-content', { visible: true });
|
||||
|
||||
// Check content
|
||||
const heading = await page.$eval('#helpModal h3', el => el.textContent);
|
||||
expect(heading).toContain('Keyboard Shortcuts');
|
||||
|
||||
// Check modal is visible
|
||||
const isVisible = await page.$eval('#helpModal .modal-content', el => {
|
||||
const style = window.getComputedStyle(el);
|
||||
return style.display !== 'none' && style.visibility !== 'hidden';
|
||||
});
|
||||
expect(isVisible).toBe(true);
|
||||
|
||||
// Close modal
|
||||
await page.click('#helpModal .modal-close');
|
||||
await page.waitForFunction(() => {
|
||||
const modal = document.querySelector('#helpModal .modal-content');
|
||||
if (!modal) return true;
|
||||
const style = window.getComputedStyle(modal);
|
||||
return style.display === 'none' || style.visibility === 'hidden';
|
||||
}, { timeout: 5000 }).catch(() => {});
|
||||
});
|
||||
|
||||
it('should open settings modal with 4 tabs', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
// Open settings
|
||||
await page.click('.btn-settings');
|
||||
await page.waitForSelector('#appSettingsModal .modal-content', { visible: true });
|
||||
|
||||
// Check tabs
|
||||
const tabs = await page.$$('#appSettingsModal .modal-tab-btn');
|
||||
expect(tabs.length).toBe(4);
|
||||
|
||||
// Verify tab names
|
||||
const tabTexts = await page.$$eval('#appSettingsModal .modal-tab-btn', els =>
|
||||
els.map(el => el.textContent?.trim())
|
||||
);
|
||||
expect(tabTexts).toContain('Display');
|
||||
expect(tabTexts).toContain('Claude CLI');
|
||||
|
||||
// Close
|
||||
await page.click('#appSettingsModal .modal-close');
|
||||
});
|
||||
|
||||
it('should switch between settings tabs', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
await page.click('.btn-settings');
|
||||
await page.waitForSelector('#appSettingsModal .modal-content', { visible: true });
|
||||
|
||||
// Click each tab
|
||||
const tabs = await page.$$('#appSettingsModal .modal-tab-btn');
|
||||
|
||||
for (let i = 0; i < tabs.length; i++) {
|
||||
await tabs[i].click();
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
|
||||
// Check tab is active
|
||||
const isActive = await page.$eval(
|
||||
`#appSettingsModal .modal-tab-btn:nth-child(${i + 1})`,
|
||||
el => el.classList.contains('active')
|
||||
);
|
||||
expect(isActive).toBe(true);
|
||||
}
|
||||
|
||||
await page.click('#appSettingsModal .modal-close');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Session Management Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should create a new session via button click', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
// Click create session button
|
||||
await page.click('.btn-claude');
|
||||
|
||||
// Wait for session tab to appear
|
||||
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||
|
||||
// Tab should be active
|
||||
const hasActive = await page.$('.session-tab.active');
|
||||
expect(hasActive).not.toBeNull();
|
||||
|
||||
// Terminal should be visible
|
||||
await page.waitForSelector('.xterm', { timeout: 10000 });
|
||||
const terminal = await page.$('.xterm');
|
||||
expect(terminal).not.toBeNull();
|
||||
|
||||
// Track session for cleanup
|
||||
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||
const data = await response.json();
|
||||
if (data.sessions?.length > 0) {
|
||||
const lastSession = data.sessions[data.sessions.length - 1];
|
||||
if (lastSession && !createdSessions.includes(lastSession.id)) {
|
||||
createdSessions.push(lastSession.id);
|
||||
}
|
||||
}
|
||||
}, 30000);
|
||||
|
||||
it('should show tab count correctly', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
// Get initial count
|
||||
const initialText = await page.$eval('#tabCount', el => el.textContent);
|
||||
const initialCount = parseInt(initialText?.match(/\d+/)?.[0] || '0');
|
||||
|
||||
// Create a session
|
||||
await page.click('.btn-claude');
|
||||
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||
|
||||
// Tab count should increase
|
||||
await page.waitForFunction(
|
||||
(expected: number) => {
|
||||
const el = document.querySelector('#tabCount');
|
||||
const text = el?.textContent || '';
|
||||
const match = text.match(/\d+/);
|
||||
return match && parseInt(match[0]) >= expected;
|
||||
},
|
||||
{ timeout: 5000 },
|
||||
initialCount + 1
|
||||
);
|
||||
|
||||
// Track for cleanup
|
||||
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||
const data = await response.json();
|
||||
data.sessions?.forEach((s: any) => {
|
||||
if (!createdSessions.includes(s.id)) createdSessions.push(s.id);
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
// ============================================================================
|
||||
// Terminal Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should initialize xterm terminal with correct dimensions', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
// Create session
|
||||
await page.click('.btn-claude');
|
||||
await page.waitForSelector('.xterm', { timeout: 15000 });
|
||||
|
||||
// Check terminal container exists
|
||||
const terminal = await page.$('.xterm');
|
||||
expect(terminal).not.toBeNull();
|
||||
|
||||
// Check xterm canvas exists (indicates proper initialization)
|
||||
const canvas = await page.$('.xterm-screen');
|
||||
expect(canvas).not.toBeNull();
|
||||
|
||||
// Track for cleanup
|
||||
const response = await fetch(`${BASE_URL}/api/sessions`);
|
||||
const data = await response.json();
|
||||
data.sessions?.forEach((s: any) => {
|
||||
if (!createdSessions.includes(s.id)) createdSessions.push(s.id);
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
// ============================================================================
|
||||
// Keyboard Shortcut Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should respond to Ctrl+Enter for quick start', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
const initialSessions = await fetch(`${BASE_URL}/api/sessions`).then(r => r.json());
|
||||
const initialCount = initialSessions.sessions?.length || 0;
|
||||
|
||||
// Press Ctrl+Enter
|
||||
await page.keyboard.down('Control');
|
||||
await page.keyboard.press('Enter');
|
||||
await page.keyboard.up('Control');
|
||||
|
||||
// Wait for session to be created
|
||||
await page.waitForSelector('.session-tab', { timeout: 15000 });
|
||||
|
||||
// Verify session was created
|
||||
const afterSessions = await fetch(`${BASE_URL}/api/sessions`).then(r => r.json());
|
||||
expect(afterSessions.sessions?.length).toBeGreaterThan(initialCount);
|
||||
|
||||
// Track for cleanup
|
||||
afterSessions.sessions?.forEach((s: any) => {
|
||||
if (!createdSessions.includes(s.id)) createdSessions.push(s.id);
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
// ============================================================================
|
||||
// Chrome DevTools Protocol Direct Access
|
||||
// ============================================================================
|
||||
|
||||
it('should access CDP for performance metrics', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
// Get performance metrics via CDP
|
||||
const client = await page.target().createCDPSession();
|
||||
await client.send('Performance.enable');
|
||||
|
||||
const { metrics: perfMetrics } = await client.send('Performance.getMetrics');
|
||||
|
||||
// Should have some metrics
|
||||
expect(perfMetrics.length).toBeGreaterThan(0);
|
||||
|
||||
// Find specific metrics
|
||||
const jsHeapSize = perfMetrics.find(m => m.name === 'JSHeapUsedSize');
|
||||
expect(jsHeapSize).toBeDefined();
|
||||
});
|
||||
|
||||
it('should capture network activity via CDP', async () => {
|
||||
const networkRequests: string[] = [];
|
||||
|
||||
const client = await page.target().createCDPSession();
|
||||
await client.send('Network.enable');
|
||||
|
||||
client.on('Network.requestWillBeSent', (params) => {
|
||||
networkRequests.push(params.request.url);
|
||||
});
|
||||
|
||||
await page.goto(BASE_URL);
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
|
||||
// Should have captured some requests
|
||||
expect(networkRequests.length).toBeGreaterThan(0);
|
||||
expect(networkRequests.some(url => url.includes('/api/events'))).toBe(true);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Stress Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should handle rapid button clicks without errors', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
// Rapidly click help button multiple times
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await page.click('.help-btn');
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
const closeBtn = await page.$('#helpModal .modal-close');
|
||||
if (closeBtn) await closeBtn.click().catch(() => {});
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
}
|
||||
|
||||
// Page should still be functional
|
||||
const header = await page.$('.header');
|
||||
expect(header).not.toBeNull();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Screenshot & Visual Tests
|
||||
// ============================================================================
|
||||
|
||||
it('should capture screenshot for visual inspection', async () => {
|
||||
await page.goto(BASE_URL);
|
||||
|
||||
// Take a screenshot (could be used for visual regression)
|
||||
const screenshot = await page.screenshot({ encoding: 'base64' });
|
||||
expect(screenshot.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Console & Error Monitoring
|
||||
// ============================================================================
|
||||
|
||||
it('should not have console errors on page load', async () => {
|
||||
const consoleErrors: string[] = [];
|
||||
|
||||
page.on('console', msg => {
|
||||
if (msg.type() === 'error') {
|
||||
consoleErrors.push(msg.text());
|
||||
}
|
||||
});
|
||||
|
||||
page.on('pageerror', err => {
|
||||
consoleErrors.push(err.message);
|
||||
});
|
||||
|
||||
await page.goto(BASE_URL);
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
|
||||
// Filter out expected errors (like SSE reconnection)
|
||||
const unexpectedErrors = consoleErrors.filter(e =>
|
||||
!e.includes('net::ERR') && !e.includes('SSE')
|
||||
);
|
||||
|
||||
expect(unexpectedErrors.length).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
// Export metrics for comparison
|
||||
export { metrics as puppeteerMetrics };
|
||||
Reference in New Issue
Block a user