mirror of
https://github.com/browser-use/browser-use.git
synced 2026-09-14 19:59:47 +08:00
Magnus Müller
6c53f20d11
- Add new `extract_data` action that uses CSS selectors (querySelectorAll) to extract structured data directly from the page without LLM cost. Supports extracting text, href, src, or any HTML attribute from matching elements. Returns structured JSON and optionally saves to CSV/JSONL/JSON files. - Add `save_to_file` parameter to existing `extract` action so LLM-extracted content can be written directly to a file instead of consuming context window tokens. Enables accumulating data across multiple pages. - Update both system prompts with a new `<data_extraction>` section that guides the agent on when to use extract_data vs extract vs evaluate, and documents the multi-page extraction pattern (extract → click Next → extract → repeat with save_to_file appending to same file). These changes address the core data extraction inefficiency where the agent was forced to output all extracted tokens through the LLM, making bulk extraction (100+ items, multi-page scraping) slow and expensive. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Codebase Structure
The code structure inspired by https://github.com/Netflix/dispatch.
Very good structure on how to make a scalable codebase is also in this repo.
Just a brief document about how we should structure our backend codebase.
Code Structure
src/
/<service name>/
models.py
services.py
prompts.py
views.py
utils.py
routers.py
/_<subservice name>/
Service.py
Always a single file, except if it becomes too long - more than ~500 lines, split it into _subservices
Views.py
Always split the views into two parts
# All
...
# Requests
...
# Responses
...
If too long → split into multiple files
Prompts.py
Single file; if too long → split into multiple files (one prompt per file or so)
Routers.py
Never split into more than one file