Files
mksglu__context-mode/scripts/install-openclaw-plugin.sh
Ben Younes c805410878 fix(pi): bridge MCP tools into pi.registerTool() so the LLM can call them (#426) (#472)
* fix(pi): bridge MCP tools into pi.registerTool() so the LLM can call them (#426)

Pi 0.73.x has no native MCP support — its README is explicit:

> No MCP. Build CLI tools with READMEs (see Skills), or build an
> extension that adds MCP support.

Without a bridge inside the context-mode Pi extension, the routing
block tells the LLM to call `ctx_execute` / `ctx_search` / etc. but
those tools never enter Pi's tool list and the LLM cannot reach them.
The reporter measured 18 sessions over 2 days: ~2,500 tokens of
system-prompt overhead per window, 0 actual ctx_* calls, 447 events
recorded but never retrieved. Net ROI on Pi was negative.

This adds a stdio JSON-RPC client (`MCPStdioClient`) plus a thin
bootstrap (`bootstrapMCPTools`) that:

  - spawns `server.bundle.mjs` as a long-lived MCP child,
  - performs the standard MCP handshake (initialize →
    notifications/initialized),
  - lists tools once via `tools/list`, and
  - registers each tool through `pi.registerTool({ name, label,
    description, parameters, execute })` so the LLM sees the canonical
    bare names (matching what hooks/core/tool-naming.mjs emits for
    Pi).

Each Pi `execute()` callback forwards into the MCP child via
`tools/call`. Errors are translated to `throw` (Pi's contract for
"tool failed") so the LLM sees the MCP server's diagnostic text.

Lifecycle:

  - Bridge bootstrap is fire-and-forget at extension load — the rest
    of the extension (session capture, hooks, slash commands) is not
    blocked by spawn / handshake latency.
  - `session_shutdown` terminates the child via SIGTERM.
  - A missing `server.bundle.mjs` or any spawn / handshake error is
    surfaced once on stderr, then the extension keeps running with
    only the existing hooks + commands. Defense-in-depth so the bridge
    can never break Pi sessions for users with broken installs.

## Why a JSON Schema parameters object instead of TypeBox

MCP `tools/list` returns JSON Schema. Pi's parameter validator accepts
JSON Schema directly (TypeBox just produces JSON Schema with extra
Symbol metadata for type inference). Passing the schema through
unchanged avoids a runtime translation pass and keeps the bridge a
true thin layer over the MCP protocol — what works in Claude Code,
Gemini CLI, and the other adapters now also works in Pi.

## No new runtime dependencies

Pure `node:child_process` + `node:path`. The `@earendil-works/pi-*`
packages are NOT pulled in as build deps — `pi` is typed structurally
as `any` (matching the existing src/pi-extension.ts style) and the
bridge only touches the documented `pi.registerTool()` shape.

## Tests

Added two new `describe` blocks in `tests/pi-extension.test.ts`:

  1. `MCPStdioClient` (5 tests) — wire-protocol contract pinned with
     fake stdio servers: id-matched responses, concurrent in-flight
     requests with out-of-order delivery, child-exit cancellation,
     timeout, non-JSON noise tolerance.
  2. `bootstrapMCPTools` (2 integration tests) — spawn the real
     `start.mjs` MCP server, assert that the canonical ctx_* set
     (`ctx_execute`, `ctx_execute_file`, `ctx_search`, `ctx_index`,
     `ctx_batch_execute`, `ctx_fetch_and_index`, `ctx_doctor`,
     `ctx_stats`, `ctx_purge`) is registered, and round-trip
     `ctx_index` through `tools/call` to confirm execute() forwards
     args and returns text.

## Test plan

  - [x] `npm run build`
  - [x] `npm run typecheck` clean
  - [x] `npm test` — 73 files, 2406 pass / 25 skipped / 0 fail
  - [x] `npx vitest run tests/pi-extension.test.ts` — 44 pass
        (37 pre-existing + 7 new for the bridge)
  - [x] see-real-bug repro: `pi.registerTool` count = 0 in installed
        binary (`/home/$USER/.nvm/.../context-mode/build/pi-extension.js`)
        on `next` @ 1f70bee, plus Pi README's "No MCP" stance, plus
        the issue reporter's 18-session measurements — all three
        agree.
  - [-] Live LLM tool-call probe in Pi: blocked — free-tier Gemini
        quota was exhausted on every available key during the fix
        session. The integration test exercises the same code path
        (real MCP server + the Pi-facing registerTool surface), so
        the regression contract is enforced from CI.

## Out of scope

  - Removing the MCP server stanza from the Pi install README. Once
    this lands, the `~/.pi/agent/mcp.json` step is still harmless but
    no longer load-bearing. Cleanup left to a docs-only follow-up.
  - In-process refactor of server.ts handlers. The subprocess bridge
    is the same model used by every other adapter; a refactor that
    inlines the handlers is its own scope.

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* fix(pi): address self-review findings on the MCP bridge (#426)

Three follow-up changes from the empirical self-review on PR #472:

1. **C1 HIGH — wiring not test-covered.**
   Phase A of the empirical review only failed because removing
   `src/pi-mcp-bridge.ts` produced an import error, not because the
   bug reproduced behaviorally. If a future refactor dropped the
   `bootstrapMCPTools(pi, …)` call from `src/pi-extension.ts` while
   keeping the bridge module intact, every existing bridge test
   stayed green and the bug silently re-entered.

   Fix: export `_mcpBridgeReady: Promise<void>` from
   `src/pi-extension.ts`. Bootstrap is still fire-and-forget (so
   spawn / handshake latency does not block session_start), but the
   promise gives tests a deterministic await point. Reset to a fresh
   promise on every `piExtension(pi)` call so multiple registrations
   in one process do not see a stale resolution.

   New test in `tests/pi-extension.test.ts` ("pi-extension.ts wiring
   (#426 regression guard)"): calls `registerPiExtension(api)`,
   awaits `_mcpBridgeReady`, asserts `api.registerTool.mock.calls`
   includes at least the canonical `ctx_execute` / `ctx_search` /
   `ctx_index` / `ctx_batch_execute` / `ctx_fetch_and_index` set.
   Verified red-on-revert: with the bridge module intact but the
   `bootstrapMCPTools(...)` call reverted to next, this test fails
   with `registeredNames: []`. Pre-fix it would have stayed green.

2. **C2 LOW — duplicated path resolution in the integration tests.**
   `tests/pi-extension.test.ts` had `path.dirname(...) +
   path.resolve(here, "..", "start.mjs")` recomputed in each `it()`.
   Lifted to a single `mcpEntry` const at the top of the
   `bootstrapMCPTools — registers every ctx_* tool with Pi` describe
   block, plus a shared `mcpEnv` for the `CONTEXT_MODE_DISABLE_VERSION_CHECK`
   override. One place to update if `start.mjs` ever moves.

3. **C3 LOW — dead `running` getter on MCPStdioClient.**
   Exported in the original commit but had zero callers anywhere in
   `src/` or `tests/`. Dropped — five lines, no behavioral impact.

## Test plan

- [x] `npm run build`
- [x] `npm run typecheck` — clean
- [x] `npm test` — 73/73 files, 2407 pass / 25 skipped / 0 fail
- [x] `npx vitest run tests/pi-extension.test.ts -t "MCP bridge|wiring"` — 8 pass
- [x] Phase A re-validation: revert ONLY the wiring in
      `src/pi-extension.ts` (keep `src/pi-mcp-bridge.ts` intact),
      run the wiring test → fails with `registeredNames: []`. Restore
      and the test goes green again.

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* refactor(openclaw): consolidate src/openclaw/* into src/adapters/openclaw/

Pre-fix layout split OpenClaw across two locations:
  - src/adapters/openclaw/ — config, hooks, index, session-db (standard
    adapter pattern matching every other platform)
  - src/openclaw/         — mcp-tools, workspace-router (rogue location)

The split predates the adapter pattern: workspace-router.ts was added
first by Pedro Almeida (#aa8d93c), then mcp-tools.ts by the maintainer
(#ff0a9a2 v1.0.107), while the adapter dir was bootstrapped later by
the copilot-swe-agent (#5fd6a9e). Source-of-truth for every platform
should live under src/adapters/<platform>/, so we move the two
stragglers in.

## Changes

  - git mv src/openclaw/mcp-tools.ts        → src/adapters/openclaw/mcp-tools.ts
  - git mv src/openclaw/workspace-router.ts → src/adapters/openclaw/workspace-router.ts
  - rmdir src/openclaw

  - src/openclaw-plugin.ts: 3 import-path updates
  - tests/plugins/openclaw.test.ts: 1 import-path update
  - tests/core/cli.test.ts: 1 readFileSync source-grep path update
    (the existing PR #183 path-traversal regression test reads the
    workspace-router source file directly to grep for safe-regex
    patterns; pin updated to the new location)

## Test plan

  - [x] npm run typecheck — clean
  - [x] npx vitest run tests/plugins/openclaw.test.ts tests/core/cli.test.ts
        → 225/225 pass
  - [x] npm test — 73 files, 2405+ pass / 25 skipped / 0 fail

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* refactor: flatten src/concurrency/runPool.ts → src/runPool.ts

The src/concurrency/ directory held a single file. A whole directory
for one module is structural noise — flatten it to src/runPool.ts.

## Changes

  - git mv src/concurrency/runPool.ts → src/runPool.ts
  - rmdir src/concurrency
  - src/server.ts: import path updated
  - tests/core/server.test.ts: import path updated

## Test plan

  - [x] npm run typecheck — clean
  - [x] npx vitest run tests/core/server.test.ts -t "runPool" — pass

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* refactor: relocate plugin entry files into src/adapters/<platform>/

Pre-fix layout had three platform plugin entry files at the src/ root:

  src/pi-extension.ts        — Pi Coding Agent extension
  src/pi-mcp-bridge.ts       — Pi MCP bridge (added in #426)
  src/openclaw-plugin.ts     — OpenClaw gateway plugin
  src/opencode-plugin.ts     — OpenCode plugin

Every other platform follows the src/adapters/<name>/ pattern (config,
hooks, index, …). The four root-level files were the last hold-outs:
inconsistent layout, plus they made adapter discovery harder for new
contributors.

## Changes (file moves)

  - git mv src/pi-extension.ts        → src/adapters/pi/extension.ts
  - git mv src/pi-mcp-bridge.ts       → src/adapters/pi/mcp-bridge.ts
  - git mv src/openclaw-plugin.ts     → src/adapters/openclaw/plugin.ts
  - git mv src/opencode-plugin.ts     → src/adapters/opencode/plugin.ts

## Internal import-path updates inside the moved files

  - ./session/db.js    → ../../session/db.js   (depth +2)
  - ./types.js         → ../../types.js
  - ./adapters/X/Y.js  → ./Y.js                (now sibling)
  - ./adapters/types.js → ../types.js          (now parent)
  - ./pi-mcp-bridge.js → ./mcp-bridge.js       (renamed + sibling)

## Runtime path-resolution updates

The plugins read sibling resources (hooks/, package.json, etc.) via
`resolve(buildDir, "..")`. After the move buildDir lives 2 dirs
deeper, so every `..` is now `../../..`:

  - resolve(buildDir, "..")                                 → resolve(buildDir, "..", "..", "..")
  - resolve(buildDir, "..", "hooks", "core", "routing.mjs")    → resolve(buildDir, "..", "..", "..", "hooks", "core", "routing.mjs")
  - (and similar for routing-block / tool-naming / auto-injection)

For opencode/plugin.ts the version-from-package.json walker prepends
`../../../package.json` to its search list (keeps the legacy
`../package.json` and `./package.json` entries as fall-backs so
unbundled or old-layout dev environments still resolve).

## Build-output paths in package.json

tsc preserves src/ structure under build/, so:

  ./build/pi-extension.js     → ./build/adapters/pi/extension.js
  ./build/openclaw-plugin.js  → ./build/adapters/openclaw/plugin.js
  ./build/opencode-plugin.js  → ./build/adapters/opencode/plugin.js

Updated:

  - package.json: pi.extensions[0], openclaw.extensions[0], main,
    exports["."], exports["./plugin"], exports["./openclaw"]
  - .pi/extensions/context-mode/index.ts: re-export delegate path
  - .openclaw-plugin/index.ts: re-export delegate path + JSDoc

## Test-side updates

  - tests/pi-extension.test.ts: dynamic-import paths updated
  - tests/opencode-plugin.test.ts: dynamic-import paths updated
  - tests/plugins/openclaw.test.ts: dynamic-import paths updated
  - tests/core/cli.test.ts: 4 dynamic-import paths + 1
    `readFileSync(src/openclaw-plugin.ts)` source-grep updated to the
    new location
  - src/adapters/detect.ts: comment-line ref updated
  - tests/adapters/detect.test.ts: comment-line ref updated

## Test plan

  - [x] npm run build                                        clean
  - [x] npm run typecheck                                    clean
  - [x] npm test                                             73 files,
        2407 pass / 25 skipped / 0 fail
  - [x] npx vitest run tests/opencode-plugin.test.ts         33/33 pass
        (regression: marker test that needed package.json walker fix)
  - [x] npx vitest run tests/plugins/openclaw.test.ts        225/225 pass
  - [x] npx vitest run tests/pi-extension.test.ts            45/45 pass
        (incl. the wiring guard added in the previous commit)
  - [x] npx vitest run tests/core/cli.test.ts -t "openclaw-plugin.ts doctor/upgrade"
        passes against the new src/adapters/openclaw/plugin.ts location
  - [x] Manual sanity: every old root-level path (build/pi-extension.js,
        src/opencode-plugin.ts, etc.) is gone from the repo — grep
        confirms zero stale refs in src/ + tests/ + package.json + the
        .pi/.openclaw-plugin/ thin wrappers.

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* fix(ci): update E2E + install scripts for relocated openclaw plugin path

The structural refactor in 4911c07 (src/openclaw-plugin.ts → src/adapters/
openclaw/plugin.ts) moved the build output from build/openclaw-plugin.js
to build/adapters/openclaw/plugin.js. Three scripts still pointed at the
legacy path and broke on next-CI.

## OpenClaw E2E (failing on ubuntu-latest + macos-latest)

  scripts/test-openclaw-e2e.sh:34
    join(process.cwd(), "build", "openclaw-plugin.js")

The Phase 1 plugin-load check failed at "❌ build/openclaw-plugin.js
exists" → exit 1. Updated to look for the new path first, fall back to
the legacy one for transition safety:

  build/adapters/openclaw/plugin.js → fall back → build/openclaw-plugin.js

Loaded-tag tracks which path actually resolved.

## OpenClaw global install (would have broken at user-install time)

  scripts/install-openclaw-plugin.sh:49 (auto-generated index.ts stub)

Updated the absolute re-export path written into the generated stub
plus the jiti cache-clear glob (now matches both
`build-adapters-openclaw-plugin.*.cjs` and the legacy
`build-openclaw-plugin.*.cjs` filenames).

## Bonus: security.js path was wrong post-refactor

The opencode + openclaw plugins called `routing.initSecurity(buildDir)`
where buildDir = build/adapters/<platform>/. That made initSecurity look
for build/adapters/<platform>/security.js — which never exists. The
security module lives at build/security.js (top-level). The fix-open
fallback meant tests still passed but every plugin load emitted a
spurious WARNING about deny-policy enforcement being off.

  - opencode/plugin.ts: pass `resolve(buildDir, "..", "..")` (= build/)
  - openclaw/plugin.ts: same

Verified locally: `bash scripts/test-openclaw-e2e.sh` → 39/39 pass, no
security warning.

## Test plan

  - [x] npm run build              clean
  - [x] npm run typecheck          clean
  - [x] npm test                   73 files, 2405+ pass / 25 skipped /
        0 fail (2 pre-existing flake worker-pool timeouts on
        kiro-hooks + insight-cors; both pass when run in isolation)
  - [x] bash scripts/test-openclaw-e2e.sh
        → "Results: 39 passed  0 warned  0 failed"  +  "✅ E2E test PASSED"

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* docs(openclaw): update Key Files paths after src/adapters/<platform>/ refactor

Independent PR review on #472 caught 3 stale path strings in
`docs/adapters/openclaw.md` that the structural refactor (4911c07)
missed:

  - src/openclaw-plugin.ts          → src/adapters/openclaw/plugin.ts
  - src/openclaw/workspace-router.ts → src/adapters/openclaw/workspace-router.ts (×2)

Doc-only — no code paths reference these strings.

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

---------

Co-authored-by: Ora Studio <noreply@oratelecom.net>
2026-05-08 02:30:46 +03:00

93 lines
3.9 KiB
Bash
Executable File

#!/usr/bin/env bash
# One-shot installer: register context-mode plugin into a local OpenClaw instance.
# Usage: ./scripts/install-openclaw-plugin.sh [OPENCLAW_STATE_DIR]
# Default OPENCLAW_STATE_DIR: /openclaw
set -euo pipefail
PLUGIN_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
OPENCLAW_STATE_DIR="${1:-${OPENCLAW_STATE_DIR:-/openclaw}}"
# — preflight checks —
if ! command -v node &>/dev/null; then
echo "✗ node is required but not found in PATH" >&2
exit 1
fi
if [ ! -d "$OPENCLAW_STATE_DIR" ]; then
echo "✗ OPENCLAW_STATE_DIR ($OPENCLAW_STATE_DIR) does not exist. Is OpenClaw installed?" >&2
exit 1
fi
# OpenClaw uses openclaw.json as the single config file.
# https://docs.openclaw.ai/tools/plugin — plugins.entries, plugins.allow
OPENCLAW_JSON="$OPENCLAW_STATE_DIR/openclaw.json"
if [ ! -f "$OPENCLAW_JSON" ]; then
echo "✗ $OPENCLAW_JSON not found." >&2
echo " Start OpenClaw once first, then re-run this script." >&2
exit 1
fi
EXT_DIR="$OPENCLAW_STATE_DIR/extensions/context-mode"
echo "→ context-mode plugin installer"
echo " plugin root : $PLUGIN_ROOT"
echo " openclaw : $OPENCLAW_STATE_DIR"
# 1. Build TypeScript + rebuild native bindings for system Node
echo "→ building..."
npm --prefix "$PLUGIN_ROOT" install --silent
npm --prefix "$PLUGIN_ROOT" run build
npm --prefix "$PLUGIN_ROOT" rebuild better-sqlite3
# 2. Create global extension dir (real directory — OpenClaw does NOT follow dir symlinks)
echo "→ installing extension into $EXT_DIR..."
mkdir -p "$EXT_DIR"
cp "$PLUGIN_ROOT/.openclaw-plugin/openclaw.plugin.json" "$EXT_DIR/openclaw.plugin.json"
cat > "$EXT_DIR/index.ts" << TSEOF
// context-mode OpenClaw global extension — auto-generated by install script.
// Uses absolute path so this stub can live in any extensions directory.
export { default } from "$PLUGIN_ROOT/build/adapters/openclaw/plugin.js";
TSEOF
# 3. Clear jiti cache so stale compiled TS doesn't survive extension dir update
# Match both the new (build-adapters-openclaw-plugin.*.cjs) and legacy
# (build-openclaw-plugin.*.cjs) cache filenames so a host that ran an
# older install still gets cleared on upgrade.
echo "→ clearing jiti cache for context-mode..."
rm -f /tmp/jiti/context-mode-index.*.cjs /tmp/jiti/build-adapters-openclaw-plugin.*.cjs /tmp/jiti/build-openclaw-plugin.*.cjs 2>/dev/null || true
echo " ✓ jiti cache cleared"
# 4. Verify plugin is discovered by OpenClaw
echo "→ verifying discovery..."
OPENCLAW_BIN="$(command -v openclaw 2>/dev/null || echo "")"
if [ -z "$OPENCLAW_BIN" ]; then
# Try common npm-global locations
for candidate in "$HOME/.npm-global/bin/openclaw" "$HOME/.local/bin/openclaw" "$(npm root -g 2>/dev/null)/../bin/openclaw"; do
[ -x "$candidate" ] && OPENCLAW_BIN="$candidate" && break
done
fi
if [ -n "$OPENCLAW_BIN" ] && OPENCLAW_STATE_DIR="$OPENCLAW_STATE_DIR" "$OPENCLAW_BIN" plugins list 2>&1 | grep -q 'context-mode'; then
echo " ✓ context-mode discovered"
else
echo " ⚠ could not verify discovery (openclaw not in PATH or plugin not found) — continuing anyway"
fi
# 5. Register in runtime config (idempotent) — delegates to
# scripts/lib/register-openclaw-config.mjs so the logic can be unit-tested
# and so we can extend it (now also registers mcp.servers.context-mode).
echo "→ registering in $OPENCLAW_JSON..."
node "$PLUGIN_ROOT/scripts/lib/register-openclaw-config.mjs" "$OPENCLAW_JSON" "$PLUGIN_ROOT"
# 6. Restart gateway
echo "→ restarting openclaw gateway..."
GATEWAY_PID=$(pgrep -f "node.*openclaw/dist/index.js" 2>/dev/null | head -1 || true)
if [ -n "$GATEWAY_PID" ]; then
# SIGUSR1 triggers a clean config reload/restart in OpenClaw
kill -USR1 "$GATEWAY_PID" 2>/dev/null && echo " ✓ SIGUSR1 sent to gateway (PID $GATEWAY_PID)" || echo " ⚠ could not signal gateway"
else
echo " ⚠ gateway not running — start it manually: OPENCLAW_STATE_DIR=$OPENCLAW_STATE_DIR openclaw gateway start"
fi
echo ""
echo "✓ done — context-mode plugin installed and active"