mirror of
https://github.com/browser-use/browser-use.git
synced 2026-09-14 19:59:47 +08:00
Saurav Panda
9a8c3b4710
Adds a purpose-built eval image and a task harness that reports latency alongside pass rate, so speed changes can be A/B'd instead of guessed at. docker/Dockerfile.eval builds in ~24s and skips --all-extras, which pulled langchain, agentmail, boto3 and the OCI SDK into the existing image for no benefit. pyyaml is installed explicitly since it only reached the runtime as a transitive dep of those extras, and it lands after uv sync, which prunes anything the lockfile does not list. The entrypoint dispatches on BU_EVAL_BROWSER: local - prewarms chromium at boot and hands the agent its CDP url cloud - Browser Use cloud browser cdp - attaches to an external browser (sidecar, grid) Attaching to a prewarmed browser takes 0.09s against 0.46s to launch one. A single prewarmed browser cannot be shared by concurrent tasks, so parallel runs fall back to a browser per task rather than letting sessions collide. The harness runs each task in its own subprocess and records wall time split into LLM, DOM observation and remainder, plus step count and token usage. Tasks carrying expected_substrings are scored deterministically, keeping the judge LLM out of the measurement path; the rest fall back to an LLM judge. Two suites: a hermetic static site with no network variance for speed numbers, and the existing real-web tasks as an accuracy guard. The hermetic suite is refused with guidance under a cloud browser, which cannot reach a site served on the container's localhost. Also re-includes browser_use/**/*.json in .dockerignore, whose blanket *.json rule was dropping browser_use/mcp/manifest.json from every image built.