Commit Graph

303 Commits

Author SHA1 Message Date
Peter Pistorius 3cf520033f Cleanup. 2026-03-07 07:46:16 +02:00
Peter Pistorius 07eee7403f fix: exclude ui from typecheck, fix formatting
- Switch typecheck from 'pnpm -r exec tsgo' to 'pnpm -r run typecheck'
  to skip ui package (electrobun ships .ts source with type errors)
- Add typecheck script to cli/package.json
- Fix formatting in cli/src/cli.ts
2026-03-07 00:43:50 +02:00
Peter Pistorius baabf8e0df Rename supervisor to cli. 2026-03-06 22:57:53 +02:00
Peter Pistorius 971619275b Allow CLI to exit. 2026-03-06 22:43:33 +02:00
Peter Pistorius 838ace96f0 Remove unused env-file. 2026-03-06 22:26:38 +02:00
Peter Pistorius 62b88160bd Remove bridge. 2026-03-06 22:02:13 +02:00
Peter Pistorius 922b01df41 Rewrite the past. Closes #14. 2026-03-06 21:54:22 +02:00
Peter Pistorius fae08d809e Merge pull request #24 from redwoodjs/pp-headless
feat: headless repo-scoped mode (.machinen/)
2026-03-06 21:10:05 +02:00
Peter Pistorius c4ef2baef3 chore: simplify test workflow, rename to machinen in cache workflow
- Remove get-runner job indirection, run tests directly on ubuntu-latest
- Update cache.yml comment: opposite-actions → machinen
2026-03-06 21:07:12 +02:00
Peter Pistorius 74af540f60 chore: update build config and test setup
- electrobun.config: remove stale config
- package.json: dependency version update
- test/e2e/setup: align label with machinen rename
2026-03-06 21:03:06 +02:00
Peter Pistorius f818e68ddd feat(ui): migrate views from HTTP API to Electrobun RPC
- commits: use RPC for branches, commits, working tree status
- runs: use RPC for run detail, logs, timeline, errors, Finder
- workflows: pass RPC to getAppStateAsync
- state: remove server-side ui-state, use RPC for cross-view state
- runs.html: add 'Open in Finder' button
- commits.html: minor layout tweaks
2026-03-06 21:02:57 +02:00
Peter Pistorius bcfdb0deb0 feat(ui): add RPC schema and bun process handlers
Add typed RPC methods for cross-view state management and direct
filesystem access from the bun process:
- getInitialState, setActiveRunId, getActiveRunId
- getBranches, getCommits, getWorkingTreeDirty
- getRunDetail, getRunLogs, getRunTimeline, getRunErrors
- openRunInFinder

This replaces HTTP API calls with Electrobun RPC, solving the
views:// protocol limitation where query params and hash fragments
are treated as part of the file path.
2026-03-06 21:02:44 +02:00
Peter Pistorius 4ac8c14a17 feat(ui): remove repos view, start on commits page
The repos selection view is no longer needed — the app now boots
directly into the commits page for the active repository.
2026-03-06 21:02:34 +02:00
Peter Pistorius 6e187339d1 fix(supervisor): rename opposite-actions → machinen, fix git shim, update cache paths
- Rename runner label from 'opposite-actions' to 'machinen' everywhere
- Fix git shim: create seed commit when HEAD is invalid so branches work
- Move warm-modules under cache/ subdirectory
- Add /runs/path endpoint for Finder integration
- Remove /ui-state endpoints (replaced by Electrobun RPC)
- Log runner CLI args to stdout and log file for debugging
2026-03-06 21:02:26 +02:00
Peter Pistorius 3976611c34 chore: remove config-dev.jsonc
No longer needed — working directory is determined automatically.
2026-03-06 21:02:15 +02:00
Peter Pistorius 8db72ed72e Add run script. 2026-03-06 08:51:27 +02:00
Peter Pistorius b349c3557a chore(core): centralize default working directory in working-directory.ts 2026-03-06 08:50:29 +02:00
justinvdm f9f984af38 Add e2e tests for derive test generation command. 2026-03-06 01:14:02 +02:00
justinvdm ced763d148 Add test generation command. 2026-03-06 01:05:32 +02:00
justinvdm 9e4daef127 Implement gen-tests command for derive. 2026-03-06 00:54:54 +02:00
justinvdm f9b4a3e52a Implement env var overrides and e2e test infrastructure for derive. 2026-03-06 00:24:47 +02:00
justinvdm 7a16e8dbdd Implement derive test infrastructure. 2026-03-06 00:15:56 +02:00
justinvdm f4eb46d1c3 Implement fake claude binary for deterministic gherkin output. 2026-03-05 23:58:37 +02:00
justinvdm 8d3537f020 Implement derive test generation pipeline. 2026-03-05 21:36:03 +02:00
Peter Pistorius 335b14df0b Try to fix UI. 2026-03-05 19:16:04 +02:00
Peter Pistorius f0a37d2249 docs: document headless mode + update README for OrbStack and machinen naming
README.md:
- Recommend OrbStack as the preferred Docker provider on macOS
- Rename dtu/github-actions → dtu-github-actions throughout
- Add 'Headless Mode (This Repository Only)' section with oa run examples
- Add 'Full Services Stack' heading before existing pnpm dev content

.env.example: fix GITHUB_REPO example value to use machinen repo name
2026-03-05 18:28:18 +02:00
Peter Pistorius f2f7079cdc chore: gitignore .machinen dirs + fix commits list click handler
.gitignore: add .machinen so repo-scoped run state is never accidentally
committed to version control.

ui/src/mainview/commits.ts: fix run group header — change cursor from
'default' to 'pointer' and wire up a click handler that navigates to
the runs view for the selected run.
2026-03-05 18:28:03 +02:00
Peter Pistorius f5d90dee2e feat(supervisor): scope run state to per-repo .machinen/runs/ directory
Each repository now maintains its own isolated run history under
<repo>/.machinen/runs/ instead of the shared global working directory.

run-store.ts:
- getRunsForCommit(): scans <repo>/.machinen/runs/ for machinen-* dirs;
  reads metadata from runs/<name>/logs/metadata.json
- getRecentRuns(): iterates each registered repo's own .machinen/runs/
  instead of the shared logsDir
- getRunDetail() / getRunStats() / getStatsHistory(): resolve runDir via
  resolveRunDir() which checks each repo's .machinen/runs/
- resolveRunDir(): new helper to locate a runId across all known repos
- getRunsDirForRepo(): returns <repo>/.machinen/runs/

runner.ts:
- Uses runDir (co-located runs/<name>/) for work/, shims/, diag/
- Docker network renamed from oa-net-* to machinen-net-*
- Runner container named machinen-<n> instead of oa-runner-<n>

local-job.ts:
- Cleanup now removes the entire runDir on success (single rm)
- On failure, retains runDir so work/, shims/, diag/, logs/ are all
  available for inspection

service-containers.ts / .test.ts: minor reference updates
2026-03-05 18:27:53 +02:00
Peter Pistorius 06b5f5705c feat(dtu): add ephemeral in-process DTU + setCacheDir
startEphemeralDtu() spins up an independent DTU server bound to a random
OS-assigned port (port 0). Each call creates isolated state, making it
suitable for per-repo headless runs.

- ephemeral.ts: new module exporting startEphemeralDtu(cacheDir)
  - Binds to 127.0.0.1:0 and returns the assigned port + close()
  - Injects repo-scoped cacheDir before starting
- store.ts: adds setCacheDir() so the cache path can be overridden at
  runtime for ephemeral instances (default still reads from env)
2026-03-05 18:27:31 +02:00
Peter Pistorius fdaad7c341 refactor(shutdown): rename oa-* Docker resources to machinen-*
Update all Docker container/network filters to use the 'machinen-' prefix.
Also updates pruneStaleWorkspaces to scan runs/ for 'machinen-' entries
instead of work/ for 'oa-runner-' entries, matching the new directory layout.

- killAllRunnerContainers: filter 'machinen-' containers, 'machinen-net-' networks
- killRunnerContainers: remove 'machinen-net-<name>' network
- pruneOrphanedDockerResources: filter 'machinen-net-' networks
- pruneStaleWorkspaces: scan runs/ for 'machinen-*' run directories
- Tests updated to reflect new naming and directory structure
2026-03-05 18:27:19 +02:00
Peter Pistorius 4fd26851ab refactor(logger): move run dirs from logs/<name> to runs/<name>/logs
Each run now gets its own top-level run directory under <workDir>/runs/:
  runs/<name>/          — runDir  (work, shims, diag all live here)
  runs/<name>/logs/     — logDir  (output.log, debug.log, metadata.json)

This co-locates all per-run artefacts so cleanup and debugging are easier.

- getRunsDir(): new export returning <workDir>/runs/
- createLogContext(): adds runDir field; logDir is now a sub-dir of runDir
- getNextLogNum(): updated to parse new machinen-<n>-j<n>-m<n> suffix convention
- ensureLogDirs(): creates runs/ instead of logs/
- Tests rewritten to use real temp dirs (no fs mocks) for reliability
- vitest.config.ts: bumps testTimeout to 30s for fs integration tests
2026-03-05 18:27:07 +02:00
Peter Pistorius 064c5cfefe refactor: replace JSONC config file with OA_WORKING_DIR env var
The global ~/.config/oa/config.jsonc is no longer read at startup.
All components now source the working directory from the OA_WORKING_DIR
environment variable, which is already injected by the supervisor when
spawning child runner processes.

- supervisor/config.ts: removes loadOaConfig, parseJsonc, DEFAULT_CONFIG_PATH
- supervisor/cli.ts: removes --config flag and loadOaConfig calls; reads
  OA_WORKING_DIR directly; drops config-path forwarding to server
- supervisor/index.ts: removes loadOaConfig (legacy entry-point)
- supervisor/server/index.ts: removes JSONC persistence from PUT /concurrency
- dtu/config.ts: removes loadOaConfig, parseJsonc, DEFAULT_CONFIG_PATH
- dtu/server/index.ts: extracts process start-up into server/start.ts
- dtu/server/start.ts: new dedicated entry-point that reads OA_WORKING_DIR
- ui/src/bun/config.ts: removes JSONC loading; reads OA_WORKING_DIR directly
- ui/src/bun/index.ts: removes uiConfigPath/parsedConfig references
2026-03-05 18:26:34 +02:00
Peter Pistorius dd6e2862dd Merge pull request #23 from redwoodjs/pp-clean-up-runner
Fix concurrent container runner race + remove stats collection
2026-03-05 16:44:46 +02:00
Peter Pistorius e02628f220 test(service-containers): update emit message assertions for new timing strings 2026-03-05 16:30:18 +02:00
Peter Pistorius a1ba2ca310 ci: add cache workflow 2026-03-05 16:23:35 +02:00
Peter Pistorius 9055e28c83 feat(ui): warm/cold node_modules badge + live elapsed timer
- Show warm 🔥 / cold ❄️ cache badge per workflow run in commits view
- Live-ticking elapsed time for running jobs (updates every second)
- Remove CPU/memory/network stats graphs from runs page
2026-03-05 16:23:24 +02:00
Peter Pistorius 5098cd45b8 feat(server): warm-status API endpoint + persist concurrency setting
- Add GET /workflows/warm-status to expose whether node_modules cache
  is warm for a given repo (used by UI warm/cold badge)
- Persist maxConcurrentJobs to config file on PUT /concurrency so the
  setting survives supervisor restarts
- Forward OA_WORKING_DIR env var in CLI so spawned runners inherit the
  server's working directory
2026-03-05 16:23:12 +02:00
Peter Pistorius 75e8429c8e test(runner): add wave scheduling and warm-cache serialization tests 2026-03-05 16:22:52 +02:00
Peter Pistorius e980c89dc4 fix(run-store): derive run status from activeRuns only, drop Docker inspect
deriveRunStatus no longer calls docker inspect to check exit codes.
activeRuns (populated by setupJob, cleared by proc.on('close')) is the
authoritative source for live job state. metadata.status covers
everything after the process exits.
2026-03-05 16:22:38 +02:00
Peter Pistorius ac752d0021 test(cleanup): add computeLockfileHash unit tests 2026-03-05 16:22:25 +02:00
Peter Pistorius f76cd291b0 feat(service-containers): add per-second health check timing logs
Show elapsed/timeout progress every second while waiting for a service
container (e.g. MySQL) to become healthy. Also log total time taken
once healthy so users can see how long it actually took.
2026-03-05 16:18:07 +02:00
Peter Pistorius 5b3a976f9c fix(runner): remove docker stats polling
No longer collecting CPU / memory / network metrics per user request.
Removes the background `docker stats` loop that polled every 5s and
wrote peakCpu, peakMemMB, peakNetRxMB, peakNetTxMB, statsHistory
into metadata.json. Also cleans up the now-unused execFile/promisify/execAsync.
2026-03-05 16:17:55 +02:00
Peter Pistorius 0e85fa6cb4 fix(local-job): per-container runner binary copy + proxy readiness poll
Root cause: all concurrent custom-container jobs shared a single
`_/work/runner/` bind-mount at /home/runner. Concurrent config.sh calls
raced on .runner/.credentials, causing run.sh to fail with 'Not configured'.

Fix:
- Extract runner binary once into shared seed dir (`_/work/runner/`)
- Give each container its own writable copy (`_/work/runner-<name>/`)
- Strip stale .runner/.credentials from seed + per-container copy on startup
- Clean per-container copy after job completes
- Replace fixed `sleep 0.3` with nc poll loop so config.sh only runs
  once the Node.js DTU proxy has actually bound port 80
- Log proxy startup timing ([OA] DTU proxy ready in Xms)
2026-03-05 16:17:39 +02:00
Peter Pistorius 4329339440 Merge pull request #21 from redwoodjs/pp-clean-up-runner
refactor: split orchestrator, remove warm-pool, co-locate tests
2026-03-05 16:15:28 +02:00
Peter Pistorius e6e9a81639 Fix PR #18 review comments: shell injection + signal handler leak
- cleanup.ts: rewrite copyWorkspace() to avoid shell injection
  - Get file list from git, then copy each file via execFileSync/spawnSync
    with argument arrays so dest and file paths are never shell-interpolated
  - macOS: per-file cp -c via spawnSync (no xargs/sh -c)
  - Linux: rsync via spawnSync with file list piped to stdin (--files-from=-)
  - Fallback: Node.js fs.cpSync (extracted to copyViaNodeFs helper)

- local-job.ts: fix signal handler listener leak in executeLocalJob()
  - Use process.once instead of process.on to prevent listener accumulation
    across multiple job invocations in the same process
  - Wrap entire job body in try/finally so SIGINT/SIGTERM handlers are always
    removed even when the function throws before normal completion
2026-03-05 11:13:42 +02:00
Peter Pistorius 7a062b837d test(supervisor): co-locate tests with their modules
server.test.ts covered two unrelated concerns — split into:

- runner.test.ts — multi-job fan-out, single-job, matrix, retry tests
                   (imports from ./runner.js)
- repos.test.ts  — HTTP route integration tests: /status, /repos,
                   /repos/watched, /workflows (imports from ./index.js)

dtu.test.ts: update imports from ./orchestrator.js
  → getEventLog/clearEventLog from ./events.js
  → setDtuReadinessCheck/setDtuSpawner/resetDtuStateForTest from ./dtu.js
2026-03-05 11:13:42 +02:00
Peter Pistorius 45ceb19146 refactor(supervisor): split orchestrator.ts into focused modules
orchestrator.ts was 1851 lines handling 5 unrelated concerns.
Extracted into:

- events.ts      — SSE client registry + event log ring buffer (~40 lines)
- repos.ts       — recent repos, watched repos/FS watcher, workflow
                   overrides + enabled state (~280 lines)
- run-store.ts   — run queries: status, logs, timeline, stats, errors
                   (~390 lines)
- runner.ts      — spawn, setup, wave scheduling, matrix expansion,
                   retry logic (~560 lines)
- dtu.ts         — DTU process lifecycle + test seams (~175 lines)

orchestrator.ts is now a 6-line barrel (export * from ...) so all
existing import paths in server/index.ts remain unchanged.
2026-03-05 11:13:42 +02:00
Peter Pistorius b3ee0e67cc refactor(supervisor): remove warm-pool
The warm-pool concept (pre-warming runner containers) is no longer used.
All job execution goes through the wave-based orchestrator in server/.

- Delete warm-pool.ts (WarmPool class, startWarmPool, stopWarmPool)
- Remove startWarmPool/stopWarmPool calls from src/index.ts
2026-03-05 11:13:42 +02:00
Peter Pistorius 34d3b6d244 fix(dtu): handle job request renewal vs. finish correctly
The PATCH jobrequests handler was always injecting lockedUntil, even on
finish requests (which carry result + finishTime). This caused the runner
to see conflicting state on job completion.

- Only set lockedUntil on renewal requests (no result/finishTime present)
- Register both bare path and parameterised /:requestId for safety
- Add tests covering renewal, parameterised renewal, and finish
2026-03-05 11:13:42 +02:00
Peter Pistorius ba2a479a70 Merge pull request #18 from redwoodjs/pp-copy-on-write
Workspace cleanup & disk I/O reduction
2026-03-05 11:13:00 +02:00