6bbc6e01c5
feat(mcp/cli): Include open Shadow DOM content in the response HTML.
2026-09-13 23:37:56 +03:00
5edd203076
feat(browsers): Include open Shadow DOM content in the response HTML.
...
Solves #440
2026-09-13 23:30:08 +03:00
28c3296714
docs: adding a new sponsor
2026-09-04 15:33:23 +03:00
1f40d9b11b
docs: moving up a sponsor
2026-09-02 15:09:56 +03:00
196c81ad67
docs: update a sponsor url
2026-09-01 15:38:52 +03:00
458e2a2ac9
docs: removing a sponsor
2026-08-25 21:38:48 +03:00
3e03f78bc0
chore: pump deps up and update stars history image
2026-08-23 22:14:09 +03:00
76395b7273
docs: update the intro
2026-08-23 20:31:17 +03:00
f164713efa
feat(fetchers): keep browser tabs open and reuse them across requests
...
Browser sessions no longer close a tab after its request is completed. The tab returns to the pool as ready, and the next request reuses it, reapplying its own timeouts, extra headers, and resource/domain routes (after `unroute_all`) so nothing leaks between requests. Tabs that hit an error or got closed by the browser are closed and evicted, and the internal response listener is detached after every request so reused tabs don't stack handlers.
Adds `close_pages()` to all browser sessions to close every open tab; the next request opens a fresh one. `PagePool` gains `get_ready_page`/`remove_page`/`clear` and `PageInfo.mark_ready`, new pages start busy, and the unused `cleanup_error_pages` is removed.
Proxy-rotation contexts keep closing per request. Docs and the agent skill describe the new lifecycle.
2026-08-23 20:28:30 +03:00
6c5b691db7
docs: add new ai sub-section with the new changes + add changelog file
2026-08-23 18:00:15 +03:00
6ca35d0448
feat(spiders): Adding easy interface for building RAG systems
2026-08-23 16:26:15 +03:00
6815ca4f4f
docs: add a page for the agent skill and restructure the website sections to clearer
2026-08-22 19:32:11 +03:00
6e6373c997
feat(mcp): add two tools to open requests sessions and use them
2026-08-22 19:17:32 +03:00
0cb3e971c9
Merge branch 'main' into dev
2026-08-21 20:28:49 +03:00
4ce0fb1c1a
docs: adding a new sponsor
2026-08-21 20:24:35 +03:00
0154400933
docs: updating a sponsor
2026-08-21 19:38:45 +03:00
1181fc6a1f
feat(mcp)!: rename get to make_request and add more methods.
2026-08-21 19:26:47 +03:00
93ae0ac9c4
fix(mcp): Set the session proxy to be browser-level argument
...
Thanks for @Yigtwxx for pointing that out in #418
2026-08-21 19:25:29 +03:00
73ad18be58
refactor(mcp)!: More reliable structure + new tools
...
A better fix than #418 is to eliminate this bug class and keep the AI agent knowing the default parameter values.
I have also shortened the docstrings where possible to save tokens.
2026-08-21 03:16:17 +03:00
39ecbf0f9a
docs: update stars history
2026-08-19 22:04:12 +03:00
9376a3d8c4
Merge branch 'main' into dev
2026-08-19 20:23:42 +03:00
cb00f1bba8
docs: adding a new sponsor
2026-08-19 20:06:03 +03:00
e90875f31f
docs: fix missing parts from the updates
...
Missing from #414
2026-08-19 02:03:32 +03:00
029d098c06
fix: require authentication by default for the HTTP transport ( #414 )
...
Co-authored-by: MJ <kaalu@MJs-MacBook-Air.local >
2026-08-19 01:56:23 +03:00
1cd938e040
build: pump version and deps
2026-08-19 01:13:30 +03:00
2b9d0e2933
style: image size adjustment
2026-08-11 01:07:32 +03:00
2c371e0f2e
docs: add stars history manually because of Github API changes
...
Ref.: https://www.star-history.com/blog/github-stargazer-api-restriction
2026-08-11 01:03:00 +03:00
3f32fa9050
docs: re-adding a sponsor
...
Revert: https://github.com/D4Vinci/Scrapling/commit/253d316a89c4207b6959d08484fcbd4243bb879f#diff-d0a2065f1a22f388fa5e96f04ebba0efa783c45613b500d78b3b2f1a2341d9cb
2026-08-11 00:37:19 +03:00
9af7e2796a
docs: adding a new sponsor
2026-08-11 00:30:14 +03:00
b498fe7ee0
docs: update benchmarks results with the latest numbers
2026-08-09 21:14:53 +03:00
fad6893d66
docs: add the new templates to the docs
2026-08-09 20:56:29 +03:00
9bbd7c8b25
feat(spiders): Add XML/CSV feed templates + docs
2026-08-09 20:44:12 +03:00
0f2397c769
feat(mcp): add a scrapling-mcp console shortcut for easier MCP registration
2026-08-09 19:19:52 +03:00
39847bd2da
docs: removing a sponsor
2026-08-06 16:00:32 +03:00
bc0cf7dede
docs: typo fix
2026-07-30 17:31:34 +03:00
efc0de2936
docs: adding new sponsor
2026-07-30 17:30:17 +03:00
80d78cc362
docs: typo fix
2026-07-27 14:01:05 +03:00
87e010a2e8
docs: fix
2026-07-27 12:38:38 +03:00
0a7fec8f86
feat(spiders): Add exporters to CSV/XML
2026-07-26 23:13:35 +03:00
347db4c76b
docs: add the auto throttle option
2026-07-26 23:04:28 +03:00
563b134f9e
feat(mcp): Add auth to the mcp server and the option to set allowed host
2026-07-26 19:02:16 +03:00
0ca4aeabae
docs: Update main pages with the recently added features
2026-07-26 16:40:49 +03:00
9ecffdb42f
fix(cdp): Make browsers accept CDP urls with http protocol
2026-07-26 16:23:00 +03:00
d08aa90d9e
Merge branch 'main' into dev
2026-07-26 16:01:45 +03:00
08bed1589d
docs: add a badge for the number of docker pulls
2026-07-24 16:39:32 +03:00
2f400555c3
fix: remove dead fetchall() after INSERT and correct retrieve() error message ( #377 )
...
* docs: updating a sponsor URL
* Remove unnecessary fetchall() after INSERT OR REPLACE
cursor.fetchall() after an INSERT OR REPLACE statement is a no-op
since INSERT does not return any rows. This call is dead code.
* fix: correct error message in retrieve() abstract method
---------
Co-authored-by: Karim shoair <D4Vinci@users.noreply.github.com >
2026-07-24 01:48:24 +03:00
07a548362f
docs: moving up a sponsor
2026-07-20 16:04:27 +03:00
5320319155
docs: updating a sponsor URL
2026-07-15 18:21:18 +03:00
027a7cd95d
docs: update docs and skill with the new cli option
2026-07-12 22:32:25 +03:00
b1c250fb29
docs: add a new page for the Shopify template
2026-07-12 18:45:18 +03:00
Karim shoair