mirror of
https://github.com/gastownhall/beads.git
synced 2026-09-14 20:17:24 +08:00
Julian Knutsen
9a1f086a41
The install instructions told users to run the upstream install.sh from
releases/latest, which today resolves to Dolt 2.3.1. Dolt 2.3.0 regressed
CALL DOLT_RESET('--hard'): a few percent of freshly created databases come
up with that procedure answering "Error 1105 (HY000): context canceled" for
the life of the server process. bd flatten and the Dolt-history compaction
in bd admin compact both finish by hard-resetting main onto a temporary
branch, and the merge-settle path behind bd dolt pull falls back to a hard
reset when it abandons a merge — so the documented happy path installed a
Dolt that breaks those operations on a database that otherwise looks fine.
Point the docs at a pinned version instead, with the measurements and the
raise criterion, and tell anyone already on 2.3.x how to check a database
and what to do about it. The pin and its rationale live in
scripts/ci/install-dolt.sh; the docs name the version once, on the Dolt
page, and everything else links there.
Also note that releases/latest resolves to the most recently created
release rather than the highest version, so it can move backwards — v1.88.2
was created after v2.2.4 and v2.3.0 — which makes "install latest" unsafe
independently of this defect.
test/docsync/doltpin_test.go keeps the documented pin, the copy-paste
install snippet and testutil.DoltDockerImage from drifting apart, which is
how the docs ended up on a broken release in the first place.
This documents the pin that gastownhall/beads#5894 applies to CI; land
after it so the reference to scripts/ci/install-dolt.sh resolves.
300 lines
8.4 KiB
Markdown
300 lines
8.4 KiB
Markdown
---
|
|
title: Sync Setup Guide
|
|
description: "Set up Dolt sync so issue data follows you across machines: remotes, bootstrapping a clone, and day-to-day push and pull"
|
|
---
|
|
|
|
Set up beads with Dolt sync so your issues follow you across computers.
|
|
|
|
## Prerequisites
|
|
|
|
You need two tools installed on every machine:
|
|
|
|
| Tool | Version | Install |
|
|
|------|---------|---------|
|
|
| **bd** (beads CLI) | 0.59.0+ | See [Installation](/getting-started/installation) |
|
|
| **Dolt** | A specific pinned version, not "latest" | See [Which Dolt version to install](/architecture/dolt#which-dolt-version-to-install) |
|
|
|
|
Verify both are installed:
|
|
|
|
```bash
|
|
bd version # must be 0.59.0+
|
|
dolt version # must match the pinned version
|
|
```
|
|
|
|
Install a specific Dolt version rather than `releases/latest`. On Dolt 2.3.x a
|
|
few percent of freshly created databases come up with `CALL
|
|
DOLT_RESET('--hard')` broken for the life of the server process, which breaks
|
|
`bd flatten`, `bd admin compact` and the rollback behind `bd dolt pull`;
|
|
separately, the upstream `latest` URL can resolve to a *lower* version than
|
|
one released days earlier.
|
|
[Which Dolt version to install](/architecture/dolt#which-dolt-version-to-install)
|
|
has the measurements, the install command, and how to check a database you
|
|
already have.
|
|
|
|
## Initial Setup (First Computer)
|
|
|
|
### 1. Initialize beads
|
|
|
|
```bash
|
|
cd your-project
|
|
bd init
|
|
```
|
|
|
|
This creates the `.beads/` directory with a Dolt database. If the git repo has
|
|
an `origin` remote, `bd init` also configures a Dolt remote named `origin`
|
|
pointing at that same git URL. Dolt stores issue data under `refs/dolt/data`,
|
|
separate from normal source branches.
|
|
|
|
### 2. Create some issues
|
|
|
|
```bash
|
|
bd create "Set up CI pipeline" -p 1 -t task
|
|
bd create "Add authentication" -p 2 -t feature
|
|
bd list
|
|
```
|
|
|
|
### 3. Verify or add a Dolt remote
|
|
|
|
In a normal git repo with `origin`, this should already be configured:
|
|
|
|
```bash
|
|
bd dolt remote list
|
|
# Expected: origin <your git origin URL>
|
|
```
|
|
|
|
If the repo had no `origin` during init, point beads at your Git remote for
|
|
sync:
|
|
|
|
```bash
|
|
# GitHub (SSH — recommended)
|
|
bd dolt remote add origin git+ssh://git@github.com/org/repo.git
|
|
|
|
# GitHub (HTTPS)
|
|
bd dolt remote add origin git+https://github.com/org/repo.git
|
|
|
|
# Other options: DoltHub, S3, GCS, local path
|
|
# See DOLT.md for all remote types
|
|
```
|
|
|
|
### 4. Push your issues
|
|
|
|
```bash
|
|
bd dolt push
|
|
```
|
|
|
|
Verify the push worked:
|
|
|
|
```bash
|
|
git ls-remote origin | grep dolt
|
|
# Expected: <hash> refs/dolt/data
|
|
```
|
|
|
|
## Existing Projects Without a Dolt Remote
|
|
|
|
Projects initialized by older versions of `bd init` may have a local embedded
|
|
Dolt database and a committed `.beads/issues.jsonl`, but no Dolt remote. Fix
|
|
that from the machine whose local database is authoritative:
|
|
|
|
```bash
|
|
bd dolt remote list
|
|
bd export -o .beads/issues.pre-remote.jsonl # optional issue audit export
|
|
bd dolt remote add origin git+ssh://git@github.com/org/repo.git
|
|
bd dolt push
|
|
```
|
|
|
|
`bd dolt remote add origin ...` writes `sync.remote` to `.beads/config.yaml`.
|
|
Commit and push that config file with your normal git workflow. Other clones can
|
|
then run `bd bootstrap` if their database is missing/stale, or `bd dolt pull`
|
|
when they already have the right database.
|
|
|
|
## Cloning to a New Computer
|
|
|
|
When you clone a repo that already has beads data on the remote, a standard `git clone` does **not** fetch `refs/dolt/data`. You need to bootstrap the Dolt database.
|
|
|
|
### Quick path: bd bootstrap
|
|
|
|
On recent versions of bd, `bd bootstrap` handles everything automatically:
|
|
|
|
```bash
|
|
git clone git@github.com:org/repo.git
|
|
cd repo
|
|
|
|
bd bootstrap
|
|
```
|
|
|
|
`bd bootstrap` auto-detects `refs/dolt/data` on origin, clones the Dolt database, and configures the remote. Verify with:
|
|
|
|
```bash
|
|
bd list # should show your issues
|
|
bd history # should show recent issue history
|
|
```
|
|
|
|
If `bd bootstrap` succeeds, you're done — skip to [Day-to-day Sync](#day-to-day-sync).
|
|
|
|
### Manual path (if bootstrap fails)
|
|
|
|
If `bd bootstrap` doesn't work (older bd versions, unusual remote configs), follow these steps:
|
|
|
|
**Step 1: Confirm the remote has beads data**
|
|
|
|
```bash
|
|
git ls-remote origin | grep dolt
|
|
# Expected: <hash> refs/dolt/data
|
|
# If missing, the remote has no beads data — use bd init normally.
|
|
```
|
|
|
|
**Step 2: Initialize beads**
|
|
|
|
```bash
|
|
bd init
|
|
```
|
|
|
|
This creates `.beads/` with an empty database. Ignore any warnings about `bd bootstrap` — we'll replace the empty database manually.
|
|
|
|
**Step 3: Stop the Dolt server**
|
|
|
|
```bash
|
|
bd dolt stop
|
|
```
|
|
|
|
**Step 4: Find your database name and remove the empty database**
|
|
|
|
```bash
|
|
# Check your database name
|
|
cat .beads/metadata.json # look for "dolt_database"
|
|
```
|
|
|
|
The `dolt_database` field is your `<dbname>` (typically the repo name).
|
|
|
|
```bash
|
|
# Remove the empty database
|
|
rm -rf .beads/dolt/<dbname>/
|
|
```
|
|
|
|
**Step 5: Clone the Dolt data from the remote**
|
|
|
|
```bash
|
|
cd .beads/dolt
|
|
dolt clone git@github.com:org/repo.git <dbname>
|
|
cd ../..
|
|
```
|
|
|
|
**Step 6: Start the server and migrate**
|
|
|
|
```bash
|
|
bd dolt start
|
|
bd migrate --yes
|
|
```
|
|
|
|
**Step 7: Ensure the remote is registered**
|
|
|
|
```bash
|
|
bd dolt remote add origin git+ssh://git@github.com/org/repo.git
|
|
```
|
|
|
|
If you see "remote already exists", that's fine — `dolt clone` already set it up.
|
|
|
|
**Step 8: Verify**
|
|
|
|
```bash
|
|
bd dolt remote list # should show origin
|
|
bd list # should show your issues
|
|
```
|
|
|
|
## Day-to-day Sync
|
|
|
|
Once set up on both machines, sync is two commands:
|
|
|
|
```bash
|
|
# Push your changes to the remote
|
|
bd dolt push
|
|
|
|
# Pull changes from the remote
|
|
bd dolt pull
|
|
```
|
|
|
|
### Typical workflow
|
|
|
|
```
|
|
Machine A Machine B
|
|
───────── ─────────
|
|
bd create "New task" -p 1
|
|
bd dolt push
|
|
bd dolt pull
|
|
bd update bd-a1b2 --claim
|
|
bd close bd-a1b2 --reason "Done"
|
|
bd dolt push
|
|
bd dolt pull
|
|
bd list # sees the closed task
|
|
```
|
|
|
|
### Important rules
|
|
|
|
- **Always use `bd dolt ...` commands** — never run raw `dolt` CLI commands while the Dolt server is running. It causes journal corruption.
|
|
- **Commit before pulling** — if you have uncommitted working set changes, `bd dolt pull` will fail with "cannot merge with uncommitted changes". Run `bd dolt commit` first.
|
|
- **Push before switching machines** — unpushed changes only exist locally.
|
|
- **Do not use JSONL as sync** — `.beads/issues.jsonl` is an export for viewers and interchange. It is not the source of truth, not a full database backup, and cannot safely reconcile deletes or pruning.
|
|
|
|
## Troubleshooting
|
|
|
|
### "no common ancestor" on push
|
|
|
|
A stale `refs/dolt/data` from a previous database is conflicting. Clear it and retry:
|
|
|
|
```bash
|
|
git update-ref -d refs/dolt/data
|
|
bd dolt push
|
|
```
|
|
|
|
### "cannot merge with uncommitted changes" on pull
|
|
|
|
Commit your working set first:
|
|
|
|
```bash
|
|
bd dolt commit
|
|
bd dolt pull
|
|
```
|
|
|
|
### "no store available" on push or commit
|
|
|
|
This was a bug in bd < 0.59.0. Upgrade bd:
|
|
|
|
```bash
|
|
brew upgrade beads
|
|
# or re-run the install script
|
|
```
|
|
|
|
### bd list shows nothing after clone
|
|
|
|
The Dolt database wasn't bootstrapped. Either run `bd bootstrap` or follow the [manual path](#manual-path-if-bootstrap-fails) above.
|
|
|
|
### Stale lock files after crash
|
|
|
|
```bash
|
|
bd doctor --fix --yes
|
|
```
|
|
|
|
**WARNING**: Do NOT manually remove files inside `.dolt/` directories (including
|
|
`noms/LOCK`). These are Dolt-internal files and removing them **will cause
|
|
unrecoverable data corruption**. Dolt manages these files itself.
|
|
|
|
### "fatal: Unable to read current working directory"
|
|
|
|
The Dolt server's working directory no longer exists (common after branch switches). Restart it:
|
|
|
|
```bash
|
|
bd dolt stop
|
|
bd dolt start
|
|
```
|
|
|
|
## See Also
|
|
|
|
- [Sync Concepts](/core-concepts/sync-concepts) — The conceptual model behind this setup (why Dolt is the source of truth, what JSONL is for)
|
|
- [Quick Start](/getting-started/quickstart) — Getting started with beads
|
|
- [Dolt Backend for Beads](/architecture/dolt) — Dolt backend details, server modes, federation, remote types, and sync modes
|
|
- [Installation](/getting-started/installation) — Installation for all platforms
|
|
|
|
## Attribution
|
|
|
|
This guide was inspired by [@leonletto](https://github.com/leonletto)'s community setup guide at [leonletto.github.io/thrum](https://leonletto.github.io/thrum/docs.html#guides/beads-setup.html), which documented the end-to-end setup and sync process including the manual bootstrap workflow. Thanks for contributing to the beads community!
|