robbyczgw-cla_web-search-plus/TROUBLESHOOTING.md

Troubleshooting Guide

Caching Issues (v2.7.0+)

Cache not working / always fetching fresh

Symptoms:

• Every request hits the API
• "cached": false even for repeated queries

Solutions:

1. Check cache directory exists and is writable:

   ls -la .cache/  # Should exist in skill directory
   

2. Verify --no-cache isn't being passed
3. Check disk space isn't full
4. Ensure query is EXACTLY the same (including provider and max_results)

Stale results from cache

Symptoms:

• Getting outdated information
• Cache TTL seems too long

Solutions:

1. Use --no-cache to force fresh results
2. Reduce TTL: --cache-ttl 1800 (30 minutes)
3. Clear cache: python3 scripts/search.py --clear-cache

Cache growing too large

Symptoms:

• Disk space filling up
• Many .json files in .cache/

Solutions:

1. Clear cache periodically:

   python3 scripts/search.py --clear-cache
   

2. Set up a cron job to clear weekly
3. Use a smaller TTL so entries expire faster

"Permission denied" when caching

Symptoms:

• Cache write errors in stderr
• Searches work but don't cache

Solutions:

1. Check directory permissions: chmod 755 .cache/
2. Use custom cache dir: export WSP_CACHE_DIR="$TMP_DIR/wsp-cache"

---

Common Issues

"No API key found" error

Symptoms:

Error: No API key found for serper

Solutions:

1. Check .env exists in skill folder with export VAR=value format
2. Keys auto-load from skill's .env since v2.2.0
3. Or set in system environment: export SERPER_API_KEY="..."
4. Verify key format in config.json:

   { "serper": { "api_key": "your-key" } }
   

Priority order: config.json > .env > environment variable

---

Getting empty results

Symptoms:

• Search returns no results
• "results": [] in JSON output

Solutions:

1. Check API key is valid (try the provider's web dashboard)
2. Try a different provider with -p
3. Some queries have no results (very niche topics)
4. Check if provider is rate-limited
5. Verify internet connectivity

Debug:

python3 scripts/search.py -q "test query" --verbose

---

Rate limited

Symptoms:

Error: 429 Too Many Requests
Error: Rate limit exceeded

Good news: Since v2.2.5, automatic fallback kicks in! If one provider hits rate limits, the script automatically tries the next provider.

Solutions:

1. Wait for rate limit to reset (usually 1 hour or end of day)
2. Use a different provider: -p tavily instead of -p serper
3. Check free tier limits:
   • Serper: 2,500 free total
   • Tavily: 1,000/month free
   • Exa: 1,000/month free
4. Upgrade to paid tier for higher limits
5. Use SearXNG (self-hosted, unlimited)

Fallback info: Response will include routing.fallback_used: true when fallback was used.

---

SearXNG: "403 Forbidden"

Symptoms:

Error: 403 Forbidden
Error: JSON format not allowed

Cause: Most public SearXNG instances disable JSON API to prevent bot abuse.

Solution: Self-host your own instance:

docker run -d -p 8080:8080 searxng/searxng

Then enable JSON in settings.yml:

search:
  formats:
    - html
    - json  # Add this!

Restart the container and update your config:

{
  "searxng": {
    "instance_url": "http://localhost:8080"
  }
}

---

SearXNG: Slow responses

Symptoms:

• SearXNG takes 2-5 seconds
• Other providers are faster

Explanation: This is expected behavior. SearXNG queries 70+ upstream engines in parallel, which takes longer than direct API calls.

Trade-off: Slower but privacy-preserving + multi-source + $0 cost.

Solutions:

1. Accept the trade-off for privacy benefits
2. Limit engines for faster results:

   python3 scripts/search.py -p searxng -q "query" --engines "google,bing"
   

3. Use SearXNG as fallback (put last in priority list)

---

Auto-routing picks wrong provider

Symptoms:

• Query about research goes to Serper
• Query about shopping goes to Tavily

Debug:

python3 scripts/search.py --explain-routing -q "your query"

This shows the full analysis:

{
  "query": "how much does iPhone 16 Pro cost",
  "routing_decision": {
    "provider": "serper",
    "confidence": 0.68,
    "reason": "moderate_confidence_match"
  },
  "scores": {"serper": 7.0, "tavily": 0.0, "exa": 0.0},
  "top_signals": [
    {"matched": "how much", "weight": 4.0},
    {"matched": "brand + product detected", "weight": 3.0}
  ]
}

Solutions:

1. Override with explicit provider: -p tavily
2. Rephrase query to be more explicit about intent
3. Adjust confidence_threshold in config.json (default: 0.3)

---

Config not loading

Symptoms:

• Changes to config.json not applied
• Using default values instead

Solutions:

1. Check JSON syntax (use a validator)
2. Ensure file is in skill directory: /path/to/skills/web-search-plus/config.json
3. Check file permissions
4. Run setup wizard to regenerate:

   python3 scripts/setup.py --reset
   

Validate JSON:

python3 -m json.tool config.json

---

Python dependencies missing

Symptoms:

ModuleNotFoundError: No module named 'requests'

Solution:

pip3 install requests

Or install all dependencies:

pip3 install -r requirements.txt

---

Timeout errors

Symptoms:

Error: Request timeout after 30s

Causes:

• Slow network connection
• Provider API issues
• SearXNG instance overloaded

Solutions:

1. Try again (temporary issue)
2. Switch provider: -p serper
3. Check your internet connection
4. If using SearXNG, check instance health

---

Duplicate results

Symptoms:

• Same result appears multiple times
• Results overlap between providers

Solution: This is expected when using auto-fallback or multiple providers. The skill doesn't deduplicate across providers.

For single-provider results:

python3 scripts/search.py -p serper -q "query"

---

Debug Mode

For detailed debugging:

# Verbose output
python3 scripts/search.py -q "query" --verbose

# Show routing decision
python3 scripts/search.py -q "query" --explain-routing

# Dry run (no actual search)
python3 scripts/search.py -q "query" --dry-run

# Test specific provider
python3 scripts/search.py -p tavily -q "query" --verbose

---

Getting Help

Still stuck?

1. Check the full documentation in README.md
2. Run the setup wizard: python3 scripts/setup.py
3. Review FAQ.md for common questions
4. Open an issue: https://github.com/robbyczgw-cla/web-search-plus/issues