28 Commits

Author SHA1 Message Date
Matt Van Horn b15e2eb46a feat(x): X search over the official X API, plus the Grok Bot X connector (#1124)
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 23:31:23 -07:00
Trevin Chow ef6c279ceb fix(security): block SessionStart .env key RCE and align project trust (#914) 2026-07-30 10:14:55 -07:00
Matt Van Horn 06b68a883a fix(hooks): avoid bash 5.3 heredoc deadlock in check-config session hook (#809)
Homebrew bash 5.3.15 can block forever in heredoc_write while feeding a
heredoc to a child inside command substitution: the write to the heredoc
pipe never completes and python3 is never exec'd, so the session-start
hook hangs indefinitely. Observed reproducibly on macOS after the
2026-07-11 Homebrew bash upgrade; /bin/bash 3.2 is unaffected.

Replace the last-run heredoc with python3 -c (no pipe at all) and the
new-user welcome cat-heredocs with printf, removing every heredoc from
the hook. Verified: 15/15 clean sub-second runs under bash 5.3.15 where
the previous script hung, and the four hook-driven test files pass
without pinning system bash onto PATH.

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-07-12 07:13:28 -07:00
Trevin Chow 3e36224927 Merge remote-tracking branch 'origin/main' into pr-626-keychain-setup-safety 2026-06-24 18:46:49 -07:00
Trevin Chow 50d062b88b Merge pull request #614 from 23241a6749/fix/573-env-file-permissions
fix: auto-tighten .env permissions to 0o600 instead of warning only
2026-06-24 18:34:20 -07:00
Sk 859a63146f feat: perplexity: add direct API modes and async Deep Research (#629)
* perplexity: add direct API modes and async Deep Research

* perplexity: avoid conflicting search date filters

* perplexity: preserve malformed async metadata

* Address greptile comments
2026-06-20 21:00:25 -07:00
Jeffrey Sperling 96ab2e276b Respect Keychain config in setup checks
Make the SessionStart setup check recognize macOS Keychain credential presence without reading secret values. This keeps the hook and first-run docs aligned with runtime credential resolution, and removes the dangerous permission-skip flag from the compare helper.
2026-06-18 17:37:48 -07:00
23241a6749 0381df5025 fix: only print auto-fixed message on successful chmod; remove duplicate import
Greptile P1: silent chmod failure (e.g. read-only mount, wrong owner)
would still print 'Auto-fixed'. Now the message is conditional on chmod
exit code — success prints 'auto-fixed', failure prints the original
'Fix: chmod 600' manual instruction.

Greptile P2: duplicate import shutil removed from test file.
2026-06-18 06:29:25 +00:00
23241a6749 d39d25f8d6 fix: auto-tighten .env permissions to 0o600 instead of warning only
check_perms() in check-config.sh warned about loose .env permissions
but never fixed them. Now it runs chmod 600  before emitting
the warning, so the next SessionStart doesn't re-warn.

The .env creation path (write_setup_config) already used _open_secret_append
with O_CREAT mode 0o600 + explicit chmod — that half was done. This closes
the gap for pre-existing files and any path that bypasses the wizard.

Closes #573.
2026-06-18 06:29:24 +00:00
j-sperling faecb03b7f fix(hooks/check-config): exit 0 explicitly (#430)
The configured-user branches end with

    [[ -n "$LAST_RUN_LINE" ]] && echo "$LAST_RUN_LINE"

When LAST_RUN_LINE is empty (no $LAST30DAYS_CONFIG_DIR/last-run.json yet),
the [[ ]] test returns 1 and \&\& short-circuits, leaving that statement as
the script's last command. Bash's exit status is the last command's status,
so the script exits 1 despite printing the normal 'Ready — N sources active'
message correctly.

SessionStart hook drivers surface this as a non-blocking error in their
session logs. Add a trailing `exit 0` so the configured branches match
the welcome branch's exit-code semantics.
2026-06-17 15:39:43 -07:00
SAI MOHANEESH 402b1eedf4 fix(hooks): mention yt-dlp in new-user welcome when detected (fixes #394) (#480)
* fix(hooks): mention yt-dlp in new-user welcome when detected (fixes #394)

The SessionStart hook's new-user early-exit at the top of the script
runs before the capability-detection block, so users who install the
skill fresh and happen to have yt-dlp on PATH are never told YouTube
is available. They run the pipeline for the first time with YouTube
silently dark.

Move the yt-dlp detection (the only capability check that doesn't
require a config file or env var) ABOVE the new-user early-exit, and
emit a 'Detected: yt-dlp is installed' line in the welcome message
when present. The setup-done branch reuses the same HAS_YTDLP variable,
so the source-count logic is unchanged.

Tests: tests/test_check_config_ytdlp_detection.py covers:
  - new user + yt-dlp on PATH -> 'Detected: yt-dlp' line appears
  - new user + no yt-dlp -> welcome is unchanged
  - existing user + yt-dlp -> setup-done branch still runs (regression)

Full suite: 1611 passed, 4 skipped.

* docs(contributors): add @23241a6749 entry for #480

* fix(hooks): address Greptile review on #480

Greptile flagged two issues on PR #480:

1. Contradictory welcome text: with yt-dlp detected, the welcome
   simultaneously said 'The setup wizard can unlock X/Twitter, YouTube,
   and more' AND 'YouTube transcripts ready, no setup needed.' Split
   the welcome into two variants so the two lines are never both
   printed: when yt-dlp is detected, the wizard line drops 'YouTube'
   and the out-of-the-box line adds 'and YouTube (yt-dlp detected).'

2. Regression test for the setup-done path only checked that 'Ready'
   appeared, which would pass even if HAS_YTDLP silently zeroed out
   before the counting block. Strengthen it to run the hook twice
   (with and without yt-dlp on PATH) and assert the parsed numeric
   source count is exactly 1 higher with yt-dlp. Also write a
   minimal last-run.json under tmp_path so the test isn't coupled
   to the pre-existing #440 exit-1 bug.

Full suite: 1611 passed, 4 skipped.

* fix: resolve CONTRIBUTORS merge markers

---------

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
2026-06-17 12:43:07 -07:00
SAI MOHANEESH 1cc1b2efa1 fix(hooks): auto-create LAST30DAYS_MEMORY_DIR on SessionStart (fixes #395) (#476)
* fix(hooks): auto-create LAST30DAYS_MEMORY_DIR on SessionStart (fixes #395)

The README and SKILL.md advertise that HTML briefs and raw research files
save to ${LAST30DAYS_MEMORY_DIR} (default ~/Documents/Last30Days), but
nothing in the skill ever created that directory. Fresh installs hit
silent save failures on the first --emit=html run.

The SessionStart hook now mkdir -p's the configured memory dir on every
session start, with stderr suppressed so permission errors don't crash
Claude Code startup. Stderr is also swallowed because the welcome-message
output path is best-effort.

Tests: tests/test_check_config_memory_dir.py covers the four cases
(non-existent path, existing path, default fallback, unwritable path).
Full suite: 1612 passed, 4 skipped.

* docs(contributors): add @23241a6749 entry for #476

* fix(hooks): drop redundant null-guard on mkdir (Greptile review)

Per Greptile review on #476: `if [[ -n "$MEM_DIR" ]]` is dead code.
${VAR:-default} substitutes for both unset and empty values, and
`set -u` at the top of the script already aborts on an unset $HOME.

Collapse the four-line block into a single line.

---------

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
2026-06-17 12:41:35 -07:00
SAI MOHANEESH 7d90275d57 fix: setup wizard missing CT0 and check-config.sh miscounts X with only AUTH_TOKEN (#502)
* fix: setup wizard missing CT0 and check-config.sh miscounts X with only AUTH_TOKEN

- CONFIGURATION.md: .env skeleton now lists AUTH_TOKEN + CT0 as
  primary X auth option (was missing CT0 entirely)
- hooks/scripts/check-config.sh: HAS_X requires both AUTH_TOKEN and
  CT0 before counting X as an active source
- skills/last30days/SKILL.md: JIT unlock wizard offers AUTH_TOKEN/CT0
  cookie-entry alongside browser-scan and xAI key options
- test_last_run_state.py: three new tests verifying source-count delta
  (neither, only AUTH_TOKEN, only CT0, both)

Fixes #396

* fix(test): isolate _run_hook from inherited CI credentials

Strip AUTH_TOKEN, CT0, XAI_API_KEY, BSKY_HANDLE, EXA_API_KEY, and
SCRAPECREATORS_API_KEY from the subprocess env so baseline comparisons
aren't corrupted by credentials present in the test-runner environment.

* chore: add explanatory comments to env strip loop per Greptile suggestion

---------

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
2026-06-17 12:12:04 -07:00
SAI MOHANEESH 5f499d4eec fix: check-config.sh exits 0 when no prior run (empty LAST_RUN_LINE) (#539)
* fix: check-config.sh exits 0 when no prior run (empty LAST_RUN_LINE)

The final statement in the ScrapeCreators-configured branch was
. When no
last-run.json exists, LAST_RUN_LINE is empty, the test exits 1,
and && short-circuits — making the script itself exit 1.

Replaced both  guards with if-blocks so the script always
exits cleanly regardless of whether LAST_RUN_LINE is populated.

Fixes #463

* fix: convert remaining && guard at new-user welcome path to if-block for consistency

---------

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
2026-06-17 12:03:02 -07:00
SAI MOHANEESH 2cc88ecaf1 fix: replace xargs with sed in check-config.sh to avoid unterminated quote crash (#545)
xargs interprets quotes in its input, so a .env value containing an
unbalanced quote (e.g. XAI_API_KEY=xai-Tom's-key) causes
'xargs: unterminated quote'. With set -euo pipefail, pipefail
propagates that non-zero status and the entire SessionStart hook
exits 1 on every session start.

Replaced with sed for whitespace trimming, which doesn't interpret
quotes. The quote-stripping step (sed 's/^["\x27\]//') is unchanged.

Fixes #506
2026-06-17 00:26:20 -07:00
SAI MOHANEESH fd6e64dfae fix: SKILL.md references Python setup wizard instead of missing nux-wizard.md (#578)
* fix(windows): utf-8 reads + skip noacl permission check

- last30days.py: open(plan_str) -> encoding='utf-8' (2 sites). The
  plan file holds entity/topic names; UnicodeDecodeError (a subclass
  of ValueError) was not caught by the surrounding except OSError,
  crashing on non-ASCII input.
- skill_meta.py: read_text() -> encoding='utf-8'. SKILL.md has emoji;
  cp1252 made version detection silently return None on Windows.
- check-config.sh: skip POSIX 600 check on MSYS/MinGW/Cygwin, where
  stat runs in noacl mode and always reports 644 regardless of NTFS
  ACL, firing a false 'should be 600' warning every session.

Closes #549

* fix: SKILL.md references Python setup wizard instead of missing nux-wizard.md

SKILL.md line 285 told the model to Read nux-wizard.md, but that
file was never added to the repo — first-run setup silently failed.

Replace with a  command that invokes
the existing Python setup_wizard module, which actually exists and
handles platform detection, cookie extraction, ScrapeCreators opt-in,
and topic picker.

Closes #574.

* test: add symmetric guard that setup subcommand exists in CLI

Greptile P2: negative-only test wasn't enough — if a future refactor
removes the setup subcommand, SKILL.md would silently point at a no-op.
Companion test confirms last30days.py setup --help prints usage.
2026-06-16 23:42:43 -07:00
Trevin Chow b296a65515 fix(last-run): guard python3 absence + hoist datetime + use context manager 2026-05-17 00:44:30 -07:00
Amit Patnaik dd7e6a1562 Preserve clean mode for last run state 2026-05-17 00:39:44 -07:00
Trevin Chow 46cf2328aa fix(hooks): use printf -v for bash 3.2 compat (declare -g is 4.2+)
macOS ships /bin/bash 3.2 and the script uses #!/bin/bash with
set -euo pipefail, so declare -g would abort the SessionStart hook
with "invalid option" on every Mac. printf -v writes via assignment
semantics (global from inside a function on 3.2+) — same scope
outcome, broader compatibility.
2026-05-16 23:49:44 -07:00
Trevin Chow 7506cbd542 fix(hooks): scope ENV_* to global (declare -g) so caller sees values 2026-05-16 23:49:05 -07:00
Dave Morin a6bd481e61 fix(hooks): replace unsafe eval with declare in check-config.sh
The load_env_vars function used eval to assign .env values, which
executes command substitutions in backtick-containing comments.
Replace eval with declare and strip inline comments before assignment.

Fixes #361
2026-05-16 23:49:05 -07:00
voidborne-d b04212680d docs: correct ScrapeCreators free tier to 100 credits (closes #367)
The skill advertises ScrapeCreators as offering "10,000 free API calls" in
six places. The actual free tier on the ScrapeCreators pricing page is
"100 credits free · No credit card required · Credits never expire" — a
100x overstatement that surprises users on signup.

Reporter (#367) burned through their full free allocation on a single
/last30days run after taking the 10,000-call claim at face value. They
verified the actual tier directly against scrapecreators.com plus an
independent review at fahimai.com.

Sweep:
- hooks/scripts/check-config.sh:110  (SessionStart hook tip line)
- README.md:228                      (Sources × Cost table row)
- HERMES_SETUP.md:62                 (Optional: ScrapeCreators bullet)
- skills/last30days/scripts/lib/ui.py:199  (PROMO_SINGLE_KEY["reddit"])
- skills/last30days/SKILL.md:1648    ("PAYG after 10,000 free API calls")
- skills/last30days/SKILL.md:1661    ("10,000 free API calls, then PAYG")

Wording defaults to the provider's own framing — "100 free credits" — and
keeps PAYG language where it was already explicit, since the paid step is
the part users were actually getting blindsided by.

CI gates: tests/test_plugin_contract.py (4) + tests/test_version_consistency.py (4)
all pass.  shellcheck clean.  No tests pin the "10,000" string.
2026-05-16 22:33:41 -07:00
Tobi 095bcae915 fix(check-config): normalize EXCLUDE_SOURCES (lowercase + whitespace) before matching
The bash banner accounting used raw substring matching while
pipeline.py normalises EXCLUDE_SOURCES via .strip().lower(). With
EXCLUDE_SOURCES=TikTok,Instagram (or with surrounding spaces),
pipeline correctly excludes the sources but the banner did not
deduct them — count showed 1-2 higher than what the pipeline
actually runs. Normalisation now mirrors the Python side
(lowercase, collapse whitespace around commas, strip outer whitespace).

Reproducer (clean HOME with config EXCLUDE_SOURCES=TikTok,Instagram):
  before: /last30days: Ready — 7 sources active.
  after:  /last30days: Ready — 5 sources active.

Addresses Greptile review comment P1 on #399.
2026-05-16 08:05:34 +02:00
Tobi 4f6b86c456 feat: honor EXCLUDE_SOURCES env var in source count + pipeline filter
Adds a per-run denylist via the existing-but-unused EXCLUDE_SOURCES
config key. Two coupled changes:

1. pipeline.available_sources() filters out any source listed in
   config["EXCLUDE_SOURCES"] (comma-separated, case-insensitive,
   whitespace-tolerant) before returning.
2. hooks/scripts/check-config.sh "Ready — N sources active" banner
   subtracts excluded sources from the ScrapeCreators +3 (Reddit
   comments + TikTok + Instagram) so the count matches what the
   pipeline actually runs.

Use case: skip TikTok/Instagram on runs where you only want
text-substantive sources, without unsetting SCRAPECREATORS_API_KEY
(which would also kill Reddit comments). The existing INCLUDE_SOURCES
allowlist covers Perplexity opt-in but doesn't cover this denylist case
— tiktok and instagram are added unconditionally when
SCRAPECREATORS_API_KEY is set, with no opt-out short of removing the key.

Tests (tests/test_pipeline_v3.py::TestExcludeSources):
- excludes tiktok+instagram when listed
- no exclusion when env unset or empty string
- case-insensitive + whitespace-tolerant parsing
- works for any source (e.g. EXCLUDE_SOURCES=hackernews), not just SC-backed
2026-05-16 00:54:18 +02:00
george231224 3d57db9644 fix: use GNU stat first in check_perms so Linux doesn't false-warn
`stat -f '%Lp'` is BSD/macOS syntax. On Linux, `stat -f` prints
filesystem info (Block size / Inodes / ...) and still exits 0, so the
`||` fallback to `stat -c '%a'` never fires. That left `$perms` as
multi-line garbage, the `!= "600"` check was always true, and every
Linux SessionStart hook invocation printed a bogus warning plus the
whole `stat -f` filesystem dump.

Reorder to try GNU stat first, fall back to BSD for macOS. Verified on
Linux (cpython-3.12 / bash 5.x) — hook now emits the expected compact
Ready banner with no false warning.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 18:41:08 +08:00
Matt Van Horn 0a9ff16dfc feat: v3.0.0 - intelligent search, GitHub person/project mode, ELI5, 13+ sources
v3 rewrites the search engine from the ground up:

- Intelligent pre-research: resolves X handles, GitHub repos, subreddits,
  TikTok hashtags, and YouTube channels before searching
- GitHub person-mode: PR velocity, top repos by stars, release notes
- GitHub project-mode: live star counts, README, releases, top issues
- ELI5 mode: plain language synthesis, no jargon
- 13+ sources: Reddit, X, YouTube, TikTok, Instagram, HN, Polymarket,
  GitHub, Threads, Pinterest, Perplexity, Bluesky, Web
- Free Reddit comments via public JSON (no API key needed)
- Fun judge v2: humor scoring baked into narrative
- Cookie consent before browser scanning
- 10,000 free ScrapeCreators calls
- 1,012 tests

Thank you to the community contributors whose issues and PRs shaped v3:
@uppinote20 (#143), @zerone0x (#134, #136), @thinkun (#116),
@thomasmktong (#124), @fanispoulinakisai-boop (#100), @pejmanjohn (#78),
@zl190 (#115), @hnshah (#84, #85, #86)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:52:23 -07:00
Matt Van Horn 775596ce21 feat: v2.9.6 — free-first NUX, cookie extraction, quality scoring
Setup wizard with consent-first cookie extraction (Chrome/Firefox/Safari),
yt-dlp auto-install, ScrapeCreators push, quality scoring (5 core sources),
status banner redesign, honest Reddit labeling, inline YouTube transcripts,
Exa free web search, Reddit public fallback, and post-research quality nudge.

Co-authored-by: Matt Van Horn <mvanhorn@MacBook-Pro.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 14:33:17 -07:00
P 9a58fe6481 feat: add SessionStart hook for config check
Add a lightweight hook that runs on session start to check if any
API keys are configured. Warns users if no config is found and
checks file permissions on existing config files.

Checks (in order): .claude/last30days.env, ~/.config/last30days/.env,
OPENAI_API_KEY env var, SCRAPECREATORS_API_KEY env var.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 16:47:36 -04:00