Files
fission-ai__openspec/test/cli-e2e/completion-tip.test.ts
Clay Good 7276c6c268 fix(packaging): print the completions tip from the CLI, not a postinstall script (#1704)
* fix(packaging): print the completions tip from the CLI, not a postinstall script

The package's only install script existed to print one line suggesting
`openspec completion install`. Shipping it made every `npm install -g`
emit an npm allow-scripts warning, and `npm approve-scripts` then failed
with ENOMATCH because it looks in the local project, not a global install
— so the warning looked like a packaging fault with no way to clear it.

The tip now prints once on the CLI's first run, recorded via a
`completionTipSeen` flag in the existing global config alongside the
telemetry notice's `noticeSeen`. It writes to stderr so it can never
contaminate piped stdout, and is suppressed under CI,
OPENSPEC_NO_COMPLETIONS=1, `--json` runs, and `openspec completion`
itself. The published package now ships no lifecycle scripts at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(completions): stop the first-run tip from corrupting global config

Adversarial review of the previous commit found it wrote a defaults-merged
config: `saveGlobalConfig({ ...getGlobalConfig(), completionTipSeen: true })`
stamped `profile: "core"` into every user's config.json on first run.
`migrateIfNeeded` treats a raw `profile` as "already migrated", so the
one-time profile migration would never run again — and `openspec update`
then deleted the user's installed workflow skills. Reproduced: 2 skill
directories removed where main reports "Migrated: custom profile with 8
workflows". The same write also overwrote an unparsable config with
defaults and made `openspec config list` report defaults as explicit.

The tip now reads and writes the raw config file and touches only its own
key, leaving an unreadable config strictly alone.

Other hardening from the same review:

- Suppress the tip for the hidden `__complete` resolver. Generated
  completion scripts call it on every Tab press with stderr discarded, so
  the one-shot tip was consumed where nobody could see it.
- Defer, never consume, when stderr is not a terminal. Agents and pipes
  drive this CLI far more often than humans do and would otherwise spend
  the tip into a log nobody opens.
- Skip the tip when completions are already installed. Previously the CLI
  advertised `completion install` to users who had run it — including on
  the very next command after installing. Adds `isInstalled()` to the
  bash/fish/powershell installers, mirroring the zsh one.
- Use the repo's `isCiEnvironment()` instead of a `CI === 'true'` string
  check, so `CI=yes`/`True`/`on` are as quiet as telemetry is.
- Move the call to `postAction` so the tip trails the command's output
  instead of pushing errors and `init`'s setup summary down the screen.
- Record before printing, so an unwritable config dir means silence rather
  than nagging on every run.

Tests: assert the message literal (mutation testing showed the message text
was the one unguarded behavior), the raw-write shape, corrupt-config
safety, the already-installed path, the defer policy, and an e2e case
pinning the non-TTY contract.

Docs: SECURITY.md no longer claims zero lifecycle scripts — `prepare` is
still declared and runs for git/directory installs; the registry-install
claim is the accurate one. `OPENSPEC_NO_COMPLETIONS` is now documented.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(completions): make the unwritable-config case portable to Windows

fs.chmodSync(dir, 0o555) does not stop a write on Windows, so this test's
unwritable condition never existed there: markTipSeen succeeded, the tip
printed, and windows-pwsh was the only failing job.

Occupy the config directory's path with a file instead. mkdirSync with
recursive: true tolerates an existing directory but throws on an existing
file on every platform, so the persist fails where a real permission error
would - before anything is printed. Also asserts the path is still a file,
so a partial write through the failure would be caught.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(completions): retire the first-run tip instead of advising a dead end

Second adversarial pass over the tip, covering the hardening commit itself.

- An undetected or unsupported shell now retires the tip quietly. It used
  to print, but `openspec completion install` exits 1 for exactly those
  users ("Shell 'tcsh' is not supported yet" / "Could not auto-detect
  shell"), so the one message they would ever get about completions sent
  them to a command that fails.
- `markTipSeen` re-reads the config immediately before writing and swaps
  the file in by rename. Deciding whether to show the tip costs a `ps`
  spawn plus a stat, and a sibling process writing config in that window
  got clobbered — on a first run that is exactly when telemetry mints
  `anonymousId`. Concurrent-process loss drops from 15/40 to ~2/40, and
  what now usually loses is the tip's own flag (it simply shows once
  more) rather than telemetry identity. The residual is the non-atomic
  read-modify-write shape shared with telemetry's own writer.
- `isInstalled()` uses stat().isFile(), so a directory at the install
  path no longer counts as an installed completion script.
- Documented what `isInstalled()` actually promises: the script file, not
  the profile sourcing line that bash and PowerShell also need. Callers
  deciding whether to *advertise* completions want the loose reading — a
  user whose profile config failed has already met the installer.
- Corrected a comment claiming the probe costs "one stat": detectShell()
  forks `ps` to read the parent process on every non-Windows run.

Tests: mutation testing found four surviving mutants — dropping
isCompletionRun from the defer policy, reverting isCiEnvironment to a
CI==='true' string check, failing closed on an undetected shell, and
neutering the non-object config guard (which lets a JSON array config be
rewritten as {"0":...}). All four now fail a test. Adds direct coverage
for the three new isInstalled() implementations, which had none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(validate): stop `change validate` exiting past commander's postAction

`change validate` on a failing change called process.exit(exitCode). That
tears down before commander's postAction hook, which is the same trap the
`update` command documents 165 lines earlier: "exiting here would skip
commander's postAction hook, killing the telemetry flush mid-request".

A change that fails validation is a routine outcome, not an error, so this
silently dropped the telemetry flush and — since the completions tip moved
to postAction — the first-run tip for anyone whose first command was a
failing validate. Verified under a pty: before, the tip never printed and
completionTipSeen was never recorded; after, both happen and the exit code
is still 1 (validate() already sets process.exitCode, which Node honours at
natural exit — top-level `validate --all` has always relied on exactly
that). The existing e2e in validate-scenario-loss.test.ts pins the exit
code.

Also wraps the postAction tip in try/finally so the telemetry flush runs
even if the hint throws: program.parse() is synchronous, so a rejection
there has no catch above it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 20:19:51 +00:00

51 lines
1.9 KiB
TypeScript

import { describe, it, expect } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import { runCLI } from '../helpers/run-cli.js';
/**
* The completions tip is a one-shot hint aimed at a human at a terminal.
* Spawned runs — agents driving the CLI, shell pipelines, CI — have no TTY on
* stderr, so they must leave the tip unconsumed for the next interactive run.
* A regression here is invisible in normal use: the user simply never sees the
* tip, because a background `openspec status` already spent it.
*/
describe('completions tip in non-interactive runs', () => {
async function freshConfigHome(): Promise<string> {
return fs.mkdtemp(path.join(os.tmpdir(), 'openspec-tip-e2e-'));
}
async function tipSeenFlag(configHome: string): Promise<unknown> {
try {
const raw = await fs.readFile(path.join(configHome, 'openspec', 'config.json'), 'utf-8');
return JSON.parse(raw).completionTipSeen;
} catch {
return undefined;
}
}
it('never prints or consumes the tip when stderr is not a terminal', async () => {
const configHome = await freshConfigHome();
// CI is explicitly off, so only the non-TTY guard can suppress the tip.
const result = await runCLI(['list'], { env: { XDG_CONFIG_HOME: configHome, CI: '' } });
expect(result.stdout).not.toContain('completion install');
expect(result.stderr).not.toContain('completion install');
expect(await tipSeenFlag(configHome)).toBeUndefined();
});
it('leaves stdout parseable on a --json run', async () => {
const configHome = await freshConfigHome();
const result = await runCLI(['list', '--json'], {
env: { XDG_CONFIG_HOME: configHome, CI: '' },
});
expect(() => JSON.parse(result.stdout)).not.toThrow();
expect(await tipSeenFlag(configHome)).toBeUndefined();
});
});