309 lines
7.1 KiB
Markdown
309 lines
7.1 KiB
Markdown
|
|
---
|
|||
|
|
name: topic-monitor
|
|||
|
|
version: 1.5.0
|
|||
|
|
description: "监控感兴趣的主题并在重要发展发生时主动提醒。"
|
|||
|
|
metadata: {"openclaw":{"requires":{"bins":["python3"],"env":{"TOPIC_MONITOR_TELEGRAM_ID":"optional - Telegram chat ID for alerts","TOPIC_MONITOR_DATA_DIR":"optional - defaults to .data/ in skill dir","WEB_SEARCH_PLUS_PATH":"optional - defaults to relative path"},"note":"All env vars optional. Defaults work out of the box."}}}
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Topic Monitor
|
|||
|
|
|
|||
|
|
Monitor topics continuously and alert only when something looks relevant.
|
|||
|
|
|
|||
|
|
## What’s new in v1.5.0
|
|||
|
|
|
|||
|
|
- **RSS/Atom feed monitoring** as a first-class source via `feeds`
|
|||
|
|
- **GitHub release monitoring** via `github_repos` → `https://github.com/{owner}/{repo}/releases.atom`
|
|||
|
|
- **Feed auto-discovery** from normal URLs
|
|||
|
|
- **OPML import** for feed lists
|
|||
|
|
- **Advanced filters** with `required_keywords` and `exclude_keywords`
|
|||
|
|
- **Sentiment analysis** on findings: `positive`, `negative`, `neutral`, `mixed`
|
|||
|
|
- **Sentiment shift alerts** with `alert_on_sentiment_shift`
|
|||
|
|
|
|||
|
|
## Quick start
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/quick.py "AI Model Releases"
|
|||
|
|
python3 scripts/quick.py "OpenClaw Releases" --github-repos "openclaw/openclaw"
|
|||
|
|
python3 scripts/quick.py "Security Advisories" --feeds "https://example.com/security.xml"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Then test it:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/monitor.py --dry-run --verbose
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Core model
|
|||
|
|
|
|||
|
|
Each topic can mix multiple sources:
|
|||
|
|
|
|||
|
|
- **Web search** via `query`
|
|||
|
|
- **RSS/Atom feeds** via `feeds`
|
|||
|
|
- **GitHub releases** via `github_repos`
|
|||
|
|
|
|||
|
|
All collected results flow into the same pipeline:
|
|||
|
|
|
|||
|
|
1. gather results
|
|||
|
|
2. deduplicate
|
|||
|
|
3. apply advanced filters
|
|||
|
|
4. score importance
|
|||
|
|
5. classify sentiment
|
|||
|
|
6. alert immediately or save for digest
|
|||
|
|
7. track state and sentiment history
|
|||
|
|
|
|||
|
|
## Topic config
|
|||
|
|
|
|||
|
|
Each topic supports these keys:
|
|||
|
|
|
|||
|
|
- `id`
|
|||
|
|
- `name`
|
|||
|
|
- `query`
|
|||
|
|
- `keywords`
|
|||
|
|
- `feeds`
|
|||
|
|
- `github_repos`
|
|||
|
|
- `required_keywords`
|
|||
|
|
- `exclude_keywords`
|
|||
|
|
- `frequency` → `hourly|daily|weekly`
|
|||
|
|
- `importance_threshold` → `high|medium|low`
|
|||
|
|
- `channels`
|
|||
|
|
- `context`
|
|||
|
|
- `alert_on`
|
|||
|
|
- `alert_on_sentiment_shift`
|
|||
|
|
- `ignore_sources`
|
|||
|
|
- `boost_sources`
|
|||
|
|
|
|||
|
|
### Example config
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
{
|
|||
|
|
"topics": [
|
|||
|
|
{
|
|||
|
|
"id": "openclaw-news",
|
|||
|
|
"name": "OpenClaw Updates",
|
|||
|
|
"query": "OpenClaw AI assistant update release",
|
|||
|
|
"keywords": ["OpenClaw", "release", "update"],
|
|||
|
|
"feeds": ["https://example.com/blog/rss.xml"],
|
|||
|
|
"github_repos": ["openclaw/openclaw"],
|
|||
|
|
"required_keywords": ["release"],
|
|||
|
|
"exclude_keywords": ["rumor", "affiliate"],
|
|||
|
|
"frequency": "daily",
|
|||
|
|
"importance_threshold": "medium",
|
|||
|
|
"channels": ["telegram"],
|
|||
|
|
"context": "Track product updates and releases",
|
|||
|
|
"alert_on": ["keyword_exact_match", "github_release"],
|
|||
|
|
"alert_on_sentiment_shift": true,
|
|||
|
|
"ignore_sources": [],
|
|||
|
|
"boost_sources": ["github.com"]
|
|||
|
|
}
|
|||
|
|
]
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Advanced filters
|
|||
|
|
|
|||
|
|
### `required_keywords`
|
|||
|
|
|
|||
|
|
All listed terms must appear in the title/snippet before scoring.
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
"required_keywords": ["release", "stable"]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### `exclude_keywords`
|
|||
|
|
|
|||
|
|
Any matching term filters the result out before scoring.
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
"exclude_keywords": ["beta", "rumor", "affiliate"]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
This is intentionally simple boolean logic:
|
|||
|
|
|
|||
|
|
- any exclude match → reject
|
|||
|
|
- any missing required term → reject
|
|||
|
|
|
|||
|
|
## RSS/Atom feeds
|
|||
|
|
|
|||
|
|
### Direct feeds
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/manage_topics.py add "Security Feeds" \
|
|||
|
|
--feeds "https://example.com/rss.xml,https://example.com/atom.xml" \
|
|||
|
|
--keywords "security,CVE,patch"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Feed discovery
|
|||
|
|
|
|||
|
|
Discover feeds from a normal website URL:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/manage_topics.py discover-feed https://example.com/blog
|
|||
|
|
python3 scripts/monitor.py --discover-feed https://example.com/blog
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Add a topic and auto-discover feeds in one step:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/manage_topics.py add "Vendor Blog" \
|
|||
|
|
--discover-feeds "https://example.com/blog" \
|
|||
|
|
--keywords "release,announcement"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### OPML import
|
|||
|
|
|
|||
|
|
Import feed subscriptions from an OPML file:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/manage_topics.py import-opml feeds.opml
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Imported topics default to daily / medium unless you override:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/manage_topics.py import-opml feeds.opml --frequency hourly --importance high
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Feed caching
|
|||
|
|
|
|||
|
|
Feed polling uses `feedparser` and stores per-feed cache data in monitor state:
|
|||
|
|
|
|||
|
|
- `etag`
|
|||
|
|
- `last-modified`
|
|||
|
|
- last check metadata
|
|||
|
|
|
|||
|
|
That allows efficient conditional requests and avoids reprocessing unchanged feeds.
|
|||
|
|
|
|||
|
|
## GitHub release monitoring
|
|||
|
|
|
|||
|
|
Track repo releases with:
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
"github_repos": ["openclaw/openclaw", "anthropics/claude-code"]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
These map to GitHub Atom feeds automatically:
|
|||
|
|
|
|||
|
|
- `https://github.com/openclaw/openclaw/releases.atom`
|
|||
|
|
- `https://github.com/anthropics/claude-code/releases.atom`
|
|||
|
|
|
|||
|
|
CLI example:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/manage_topics.py add "CLI Releases" \
|
|||
|
|
--github-repos "openclaw/openclaw,anthropics/claude-code" \
|
|||
|
|
--keywords "release,version"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
GitHub release items are labeled clearly in alerts.
|
|||
|
|
|
|||
|
|
## Sentiment analysis
|
|||
|
|
|
|||
|
|
Each scored finding also gets a sentiment label:
|
|||
|
|
|
|||
|
|
- `positive`
|
|||
|
|
- `negative`
|
|||
|
|
- `neutral`
|
|||
|
|
- `mixed`
|
|||
|
|
|
|||
|
|
Alerts and digest entries include that sentiment.
|
|||
|
|
|
|||
|
|
### Sentiment shift alerts
|
|||
|
|
|
|||
|
|
Enable:
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
"alert_on_sentiment_shift": true
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
When enabled, a result that changes sentiment versus the topic’s previous sentiment history gets promoted for alerting.
|
|||
|
|
|
|||
|
|
State tracks:
|
|||
|
|
|
|||
|
|
- `last_sentiment`
|
|||
|
|
- `sentiment_history`
|
|||
|
|
|
|||
|
|
## Scripts
|
|||
|
|
|
|||
|
|
### `scripts/manage_topics.py`
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# Add topic
|
|||
|
|
python3 scripts/manage_topics.py add "Topic Name" \
|
|||
|
|
--query "search query" \
|
|||
|
|
--keywords "word1,word2" \
|
|||
|
|
--feeds "https://example.com/rss.xml" \
|
|||
|
|
--github-repos "openclaw/openclaw" \
|
|||
|
|
--required-keywords "release" \
|
|||
|
|
--exclude-keywords "beta,rumor"
|
|||
|
|
|
|||
|
|
# List topics
|
|||
|
|
python3 scripts/manage_topics.py list
|
|||
|
|
|
|||
|
|
# Edit topic
|
|||
|
|
python3 scripts/manage_topics.py edit topic-id --feeds "https://example.com/rss.xml"
|
|||
|
|
|
|||
|
|
# Discover feeds
|
|||
|
|
python3 scripts/manage_topics.py discover-feed https://example.com/blog
|
|||
|
|
|
|||
|
|
# Import OPML
|
|||
|
|
python3 scripts/manage_topics.py import-opml feeds.opml
|
|||
|
|
|
|||
|
|
# Test topic
|
|||
|
|
python3 scripts/manage_topics.py test topic-id
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### `scripts/monitor.py`
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 scripts/monitor.py
|
|||
|
|
python3 scripts/monitor.py --dry-run
|
|||
|
|
python3 scripts/monitor.py --topic openclaw-news --verbose
|
|||
|
|
python3 scripts/monitor.py --discover-feed https://example.com/blog
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Alert output
|
|||
|
|
|
|||
|
|
Alerts can now include:
|
|||
|
|
|
|||
|
|
- source label (`Web`, `Feed`, `GitHub Release`)
|
|||
|
|
- score and reason
|
|||
|
|
- sentiment
|
|||
|
|
- sentiment shift marker when applicable
|
|||
|
|
|
|||
|
|
## Installation note
|
|||
|
|
|
|||
|
|
Feed support uses Python `feedparser`.
|
|||
|
|
|
|||
|
|
Install if needed:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pip3 install feedparser
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Troubleshooting
|
|||
|
|
|
|||
|
|
### Feeds not showing results
|
|||
|
|
|
|||
|
|
- verify the feed URL manually
|
|||
|
|
- try `discover-feed` against the site URL
|
|||
|
|
- install `feedparser`
|
|||
|
|
- run `python3 scripts/monitor.py --dry-run --verbose`
|
|||
|
|
|
|||
|
|
### Too much noise
|
|||
|
|
|
|||
|
|
- tighten `required_keywords`
|
|||
|
|
- add `exclude_keywords`
|
|||
|
|
- increase `importance_threshold`
|
|||
|
|
|
|||
|
|
### Missing GitHub release alerts
|
|||
|
|
|
|||
|
|
- confirm repo is `owner/repo`
|
|||
|
|
- verify releases exist on GitHub
|
|||
|
|
- test with `--verbose`
|
|||
|
|
|
|||
|
|
## Notes
|
|||
|
|
|
|||
|
|
- Search and feed results are merged into the same scoring pipeline.
|
|||
|
|
- Existing web-search behavior remains supported.
|
|||
|
|
- Digest entries store sentiment so weekly output reflects tone changes too.
|