2pac
406e3e282e
* fix(core): stop full index scans on every write (event-loop stalls, daemon restarts) Every write, including binary PUT /raw uploads, ran two synchronous full-table scans to find one row: a DELETE on the FTS5 table by path (FTS5 only uses MATCH and rowid, so it scanned the whole index) and a lookup on content_chunks, which had no (drive_id, file_path) index. Both scale with the size of the whole search index, not the file being written. On a 1.5 GB synthetic index that is about 17 s of blocked event loop per write on a laptop, minutes on a shared-cpu Fly machine, which matches the 70 s stalls and /health-driven restarts in prod. - files_fts becomes an FTS5 external-content table over a new files_fts_docs table with a unique (drive_id, path) index; triggers keep the index in sync. Writers only touch files_fts_docs. - Add idx_content_chunks_drive_path. - grep reads files_fts_docs directly (index range instead of FTS scan). - Existing databases migrate in place: the daemon renames the old table at startup (instant), then copies rows into the new layout in small committed batches in the background, resumable via a cursor in the new meta table. crisismerge is raised during the copy so FTS5 does not block on whole-level merges, and restored afterwards. Search and grep read both layouts until the copy finishes. createDatabase() leaves a legacy table alone so a newer CLI cannot rename it under an older daemon. Measured after: indexFile 1-2 ms, removeFromIndex < 1 ms, chunk lookup < 1 ms at 20k documents. * chore: migrate to Bun 1.4 Pin bun 1.4.0 in packageManager, CI setup-bun, and the Docker base images; bun-types ^1.4.0; engines.bun >= 1.4.0. Typecheck, unit tests, build, and the local E2E suite pass on 1.4.0. * chore: release v0.13.1
7.5 KiB
A persistent, searchable filesystem for AI agents.
Built by desplega.sh — by builders, for builders.
Agent FS gives AI agents a structured filesystem with built-in semantic search, versioning, and identity management. It runs as a CLI and an HTTP server with integrated MCP support — so any AI coding assistant can use it as a long-term memory and file store.
agent-fs was built to power the shared filesystem in agent-swarm — a multi-agent coordination framework. While it was designed for swarm agents to share files, search content, and collaborate, it works as a standalone filesystem for any AI agent.
Key Features
- Semantic search — Index and search files using vector embeddings (OpenAI, Google GenAI, or local llama.cpp)
- SQL queries — Run sandboxed DuckDB SQL over stored documents (CSV, TSV, Parquet, Excel, JSON, NDJSON, SQLite, DuckDB) — see docs/sql.md
- Structured storage — SQLite-backed file operations with metadata and versioning
- S3-compatible sync — Sync agent workspaces to any S3-compatible object store
- Identity management — Persistent agent identity files that evolve over time
- MCP integration — Expose filesystem operations as MCP tools for Claude Code, Codex, and other assistants
- HTTP API — RESTful server powered by Hono for programmatic access
- CLI — Single binary (
agent-fs) for local use and scripting
Quick Start
Install
Requires Bun >= 1.4.0.
bun add -g @desplega.ai/agent-fs
Or build from source:
git clone https://github.com/desplega-ai/agent-fs.git
cd agent-fs
bun install
bun run build
Usage
# Show available commands
agent-fs --help
Architecture
Agent FS is a Bun monorepo with four packages:
| Package | Description |
|---|---|
@desplega.ai/agent-fs-core |
Core library — storage engine, semantic search, identity, S3 sync |
@desplega.ai/agent-fs |
CLI binary (agent-fs) |
@desplega.ai/agent-fs-mcp |
MCP stdio proxy + tool registration for the HTTP server |
@desplega.ai/agent-fs-server |
HTTP server — RESTful API powered by Hono |
FUSE mount (Linux)
agent-fs can expose your drives as a Linux FUSE filesystem so agents can use plain shell verbs (cat, grep, mv, rm) against agent-fs content.
# Linux only — the FUSE helper sub-package auto-installs via optionalDependencies.
# Use Bun's installer; stock Ubuntu/Debian Node (18.x) chokes on a dep's
# postinstall and aborts the global install.
bun install -g @desplega.ai/agent-fs
agent-fs daemon start
agent-fs mount /mnt/agent-fs
Requirements
- Linux x86_64 or aarch64. macOS and Windows are not supported (FUSE is a Linux kernel feature here).
/dev/fusemust be accessible. In Docker, run with--cap-add SYS_ADMIN --device /dev/fuse.- Sandboxes that use gVisor (GKE Sandbox, Cloud Run gen1), GitHub Codespaces, Modal sandboxes, or Fly.io Machines cannot mount FUSE — fall back to the CLI/MCP path.
Local-dev escape hatch (AGENT_FS_FUSE_BIN)
If you're building the helper from source (e.g. macOS dev host or a custom Linux build), point the CLI at your local binary:
cd packages/fuse-helper && cargo build --release
export AGENT_FS_FUSE_BIN="$PWD/target/release/agent-fs-fuse"
agent-fs mount /tmp/m
AGENT_FS_FUSE_BIN takes precedence over the auto-resolved sub-package binary.
Remote mount (--remote)
When your agents run inside a sandbox that can't host a local daemon (Sprite, E2B, ephemeral CI runners), point the mount at a remote agent-fs HTTP API instead of a local Unix socket:
agent-fs mount /mnt/agent-fs --remote \
--api-url https://my-agent-fs.example.com \
--api-key "$AGENT_FS_API_KEY"
The helper talks to the remote API directly — no local daemon, no S3 credentials in the sandbox. End-to-end coverage for this topology lives at scripts/e2e-remote-mount.ts:
# Spins up MinIO, a host-side daemon, and a Docker container with fuse3.
# Mounts via --remote against the daemon's HTTP API and exercises ~8 ops.
# Requires Docker Desktop / OrbStack on Mac.
bun run scripts/e2e-remote-mount.ts
macOS testing harness
macOS can build the helper but not mount it. Use the Docker harness to test mount behaviour from a Mac host:
bash packages/fuse-helper/docker/run-mount-test.sh
Per-environment guides
See docs/mounting/ for the general overview plus step-by-step guides for Sprite, E2B, and Hetzner Cloud. Existing FUSE references — fuse-mount.md, fuse-compat.md, fuse-troubleshooting.md — cover mount semantics, sandbox compatibility, and the full error catalogue.
Documentation
- Mounting Guides — General overview + per-env guides (Sprite, E2B, Hetzner)
- MCP Setup Guide — Connect agent-fs to Claude Code, Cursor, or any MCP client
- Deployment Guide — Local, remote S3, team, and multi-agent deployments
- API Reference — HTTP API and OpenAPI spec
Development
bun install # Install dependencies
bun run typecheck # Type checking
bun run test # Run tests
bun run build # Bundle CLI for npm
See CONTRIBUTING.md for the full development guide.
Releasing
Releases are automatic — a version change landing on main tags and publishes.
./scripts/release.sh 0.13.0
See RELEASING.md for the full process, version-sync rules, and recovery steps.
Deploy to Fly.io
Deploy a persistent agent-fs instance to Fly.io with Tigris S3 storage:
git clone https://github.com/desplega-ai/agent-fs && cd agent-fs
bun run scripts/fly-deploy.ts
See DEPLOYMENT.md for Docker, BYOK storage, and manual setup options.
Live Viewer
live.agent-fs.dev — A stateless browser UI (local storage only) for inspecting any agent-fs deployment. Point it at your server URL to browse files and search content — nothing is stored server-side.
Contributing
We welcome contributions! Whether it's bug reports, feature requests, docs improvements, or code — all are welcome.
- Fork the repo
- Create a branch (
git checkout -b my-feature) - Make your changes
- Open a PR
Join our Discord if you have questions or want to discuss ideas.
License
MIT — 2025-2026 desplega.ai