Files
backnotprop__plannotator/packages/editor/annotateSubmission.test.ts
Raúl 9a450a69e7 feat(annotate): preserve notes on structured approval (#1092)
* feat(annotate): add strict atomic result output

* feat(annotate): exit 2 for strict-gate usage and publication errors

Adopt the grep convention for the strict annotate gate's exit codes:
0 = approved, 1 = negative human outcome (annotated/dismissed under
--require-approval), 2 = the gate itself was misconfigured or could not
start/deliver a decision. Previously all usage/startup/validation
failures shared exit 1 with "reviewer did not approve", so callers could
not tell a denied review from a broken gate.

- parseStrictAnnotateOptions failures (bad flag combos, strict flags
  outside annotate --gate --json) now exit 2
- --result-file preflight failures (missing parent, pre-existing or
  dangling-symlink destination) now exit 2
- post-decision publication failures (destination raced into existence,
  hard links unavailable, stdout write failure) now exit 2: they deliver
  no decision record at all, so the code's own fail-closed handling
  presents them as environment errors, never as a reviewer outcome --
  and never approval, since only 0 means approved
- decision outcomes keep 0/1 exactly as before; signal deaths keep 128+n
- document the contract in AGENTS.md and the annotate-gates guide

Claude-Session: https://claude.ai/code/session_01YXkgsNucxDwAL4GdR4XYRk

* feat(annotate): preserve notes on structured approval

* test(pi): use exact annotate outcome import

* fix(annotate): exit 2 for strict-gate startup failures

The six startup-failure sites in the annotate path (missing path, unreachable
URL, empty folder, ambiguous name, missing/unsupported file, oversized file)
run after flag parsing and exited 1. Under --require-approval / --result-file,
1 is the "reviewer requested changes" signal, so a typo'd path made automation
misclassify a configuration error as a legitimate rejection.

Route those sites through exitAnnotateStartupFailure(), which picks its code
from the already-parsed strict options via the new pure helper
annotateStartupFailureExitCode(). Non-strict invocations still exit 1 with
byte-identical stderr; strict invocations exit STRICT_GATE_ERROR_EXIT_CODE (2).

Claude-Session: https://claude.ai/code/session_01H5KQWqXqjrPxyxUNso1QHS

* fix(annotate): emit the strict decision on stdout before publishing it

writeResultFile ran before the decision JSON reached stdout. On a filesystem
without hard links (exFAT, FAT32, most SMB/NFS, some container bind mounts)
publication fails deterministically, the catch exited 2 with nothing written
anywhere — and the reviewer's autosaved draft had already been deleted by the
feedback flow, so their completed decision was lost.

Emit the stdout record first, then publish the result file. Exit semantics are
unchanged: a publication failure still exits 2, but the decision has reached
stdout by then. Only a stdout write failure now leaves no record at all.

Correct the docs and comments that claimed exit 2 delivers no decision record:
it means the result *file* was not published. Also document the two publication
caveats: the 0600 mode is a no-op on Windows, and the atomic link/rename is not
followed by a parent-directory fsync, so publication is atomic but not
crash-durable.

Claude-Session: https://claude.ai/code/session_01H5KQWqXqjrPxyxUNso1QHS

* fix(annotate): parse linked docs with the render-side frontmatter rule on export

buildCompleteAnnotateFeedback re-parsed each linked document with
parseMarkdownToBlocks(entry.markdown) — no options, so frontmatter
stripping defaulted on. The render side parses with
{ frontmatter: shouldStripFrontmatter(path) }.

For plain-text linked docs (.yaml/.json/.toml/…) a leading `---` is real
content, not frontmatter: a multi-document YAML opens with it. Stripping
it on the export side shifted every block id, so ordinary Send Feedback
and deny emitted wrong `(line N)` labels — or dropped them entirely when
the annotation's block no longer existed.

Pass the same shouldStripFrontmatter(filepath) option at the export call
site so both sides agree.

Claude-Session: https://claude.ai/code/session_01H5KQWqXqjrPxyxUNso1QHS

* fix(annotate): carry the message scope through approve-with-notes

/api/feedback forwards selectedMessageId and feedbackScope; /api/approve
dropped them. Pi resolves the anchor message from those fields, so notes
delivered on the approve path anchored to the last message instead of the
one the reviewer picked in a multi-message annotate-last session — while
Send Feedback in the same session anchored correctly.

Forward both fields on the approve path in the Bun and Pi servers, and
have the client build the approval body with the same scope resolution
Send Feedback uses (extracted as getFeedbackMessageScope so the two can
no longer drift).

Claude-Session: https://claude.ai/code/session_01H5KQWqXqjrPxyxUNso1QHS

* docs(annotate): tell agents an approval may carry notes

The skill and slash-command files still described `"decision": "approved"`
as "acknowledge and stop", with no mention of the feedback field the gate
can now attach — so an agent reading them would silently drop the
reviewer's approval notes.

Update the Claude core/claude skills, the Copilot commands, the Gemini
annotate command, and the annotate command reference so the approved
branch names the optional feedback field and says what to do with it:
carry it into subsequent work, do not treat it as a change request.

Claude-Session: https://claude.ai/code/session_01H5KQWqXqjrPxyxUNso1QHS

* docs(annotate): document the real approvedWithNotes default

The default annotate.approvedWithNotes template is
`{{contextBlock}}{{feedback}}`, not `{{context}}` on its own line, and
{{contextBlock}} was missing from the variable table entirely.

Show the actual default, add {{contextBlock}} to the variable table, and
explain why the default prefers it: it collapses to nothing for message
annotations instead of leaving a stray blank line.

Claude-Session: https://claude.ai/code/session_01H5KQWqXqjrPxyxUNso1QHS

---------

Co-authored-by: Michael Ramos <mdramos8@gmail.com>
2026-07-26 21:09:28 -07:00

271 lines
9.0 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { describe, expect, test } from "bun:test";
import { AnnotationType, type Annotation, type CodeAnnotation, type EditorAnnotation } from "@plannotator/ui/types";
import { parseMarkdownToBlocks, type LinkedDocAnnotationEntry } from "@plannotator/ui/utils/parser";
import {
buildAnnotateApprovalBody,
buildCompleteAnnotateFeedback,
getAnnotateApprovalPolicy,
} from "./annotateSubmission";
describe("annotate approval submission", () => {
test("includes notes only when the transport supports approval notes", () => {
const input = {
draftGeneration: 4,
feedback: "Keep the retry bounded.",
annotations: [{ id: "a1" }],
codeAnnotations: [{ id: "c1" }],
};
expect(buildAnnotateApprovalBody({ supported: true, ...input })).toEqual(input);
expect(buildAnnotateApprovalBody({ supported: false, ...input })).toEqual({
draftGeneration: 4,
});
});
// The notes have to anchor to the same message Send Feedback would target.
test("carries the message scope so approval notes anchor like feedback", () => {
const input = {
draftGeneration: 4,
feedback: "Scope this to the picked message.",
annotations: [],
codeAnnotations: [],
};
expect(buildAnnotateApprovalBody({
supported: true,
...input,
selectedMessageId: "message-2",
feedbackScope: "messages" as const,
})).toEqual({
...input,
selectedMessageId: "message-2",
feedbackScope: "messages",
});
// Omitted entirely when there is no message scope (ordinary file annotate).
expect(buildAnnotateApprovalBody({ supported: true, ...input })).toEqual(input);
// Incapable transports still send nothing but the draft generation.
expect(buildAnnotateApprovalBody({
supported: false,
...input,
selectedMessageId: "message-2",
})).toEqual({ draftGeneration: 4 });
});
test("labels capable feedback approvals and requires a non-blocking confirmation", () => {
expect(getAnnotateApprovalPolicy({
gate: true,
approvalNotesSupported: true,
hasFeedback: true,
})).toEqual({
label: "Approve with Notes",
title: "Approve with Notes — send notes as non-blocking guidance",
confirmation: {
title: "Approve with Notes?",
message: "This approves the artifact, sends your notes as non-blocking guidance, and closes the gate. Unlike Send Feedback, it does not request changes.",
confirmText: "Approve with Notes",
},
});
});
test("keeps ordinary approval presentation when notes are absent or unsupported", () => {
expect(getAnnotateApprovalPolicy({
gate: true,
approvalNotesSupported: true,
hasFeedback: false,
})).toEqual({
label: "Approve",
title: "Approve — no changes requested",
confirmation: null,
});
expect(getAnnotateApprovalPolicy({
gate: true,
approvalNotesSupported: false,
hasFeedback: true,
})).toEqual({
label: "Approve",
title: "Approve — no changes requested",
confirmation: null,
});
});
test("composes every annotate feedback source into approval notes", () => {
const markdown = "# Retry\n\nRetry forever.";
const blocks = parseMarkdownToBlocks(markdown);
const paragraph = blocks.find((block) => block.type === "paragraph");
if (!paragraph) throw new Error("expected paragraph block");
const annotation: Annotation = {
id: "a1",
blockId: paragraph.id,
startOffset: 0,
endOffset: 5,
type: AnnotationType.COMMENT,
text: "Keep the retry bounded.",
originalText: "Retry",
createdA: 1,
images: [{ path: "/tmp/retry.png", name: "retry-diagram" }],
};
const linkedAnnotation: Annotation = {
...annotation,
id: "linked-1",
text: "Update the linked runbook.",
originalText: "Runbook",
images: undefined,
};
const linkedDocuments = new Map<string, LinkedDocAnnotationEntry>([
["/docs/runbook.md", {
annotations: [linkedAnnotation],
globalAttachments: [],
markdown: "# Runbook\n\nRunbook",
}],
]);
const codeAnnotation: CodeAnnotation = {
id: "c1",
type: "comment",
filePath: "src/retry.ts",
lineStart: 8,
lineEnd: 8,
side: "new",
text: "Cap this loop.",
originalCode: "while (true)",
createdAt: 1,
};
const editorAnnotation: EditorAnnotation = {
id: "e1",
filePath: "src/config.ts",
selectedText: "MAX_RETRIES",
lineStart: 3,
lineEnd: 3,
comment: "Make the limit configurable.",
createdAt: 1,
};
const feedback = buildCompleteAnnotateFeedback({
blocks,
annotations: [annotation],
globalAttachments: [{ path: "/tmp/global.png", name: "global-reference" }],
linkedDocuments,
editorAnnotations: [editorAnnotation],
codeAnnotations: [codeAnnotation],
title: "File Feedback",
subject: "file",
sourceConverted: false,
directEditsSection: "# Direct Edits\n\nBound the retry loop.",
savedFileChangesSection: "# Saved File Changes\n\n## /docs/retry.md",
});
expect(feedback).toContain("retry-diagram");
expect(feedback).toContain("global-reference");
expect(feedback).toContain("# Code File Feedback");
expect(feedback).toContain("# Direct Edits");
expect(feedback).toContain("# Linked Document Feedback");
expect(feedback).toContain("# Editor File Annotations");
expect(feedback).toContain("# Saved File Changes");
expect(buildAnnotateApprovalBody({
supported: true,
draftGeneration: 4,
feedback,
annotations: [annotation],
codeAnnotations: [codeAnnotation],
})).toMatchObject({
draftGeneration: 4,
feedback,
annotations: [annotation],
codeAnnotations: [codeAnnotation],
});
});
// Regression: the export must parse each linked doc the same way the viewer
// rendered it. A plain-text source (.yaml/.json/.toml/…) whose first line is
// `---` is real content — a multi-document YAML, not frontmatter. Stripping
// it here shifted every block id, so the line labels came out wrong (or the
// annotation's block vanished and the label was dropped entirely).
test("keeps linked-doc line labels correct for plain-text sources that open with ---", () => {
const yaml = "---\napiVersion: v1\nkind: Service\n---\napiVersion: v1\nkind: ConfigMap\n";
const yamlBlocks = parseMarkdownToBlocks(yaml, { frontmatter: false });
const firstDocument = yamlBlocks.find(
(block) => block.type === "paragraph" && block.content.startsWith("apiVersion: v1"),
);
if (!firstDocument) throw new Error("expected the first YAML document to parse as a block");
expect(firstDocument.startLine).toBe(2);
const linkedAnnotation: Annotation = {
id: "yaml-1",
blockId: firstDocument.id,
startOffset: 0,
endOffset: 14,
type: AnnotationType.COMMENT,
text: "Pin the API version.",
originalText: "apiVersion: v1",
createdA: 1,
};
const feedback = buildCompleteAnnotateFeedback({
blocks: [],
annotations: [],
globalAttachments: [],
linkedDocuments: new Map<string, LinkedDocAnnotationEntry>([
["/infra/deploy.yaml", {
annotations: [linkedAnnotation],
globalAttachments: [],
markdown: yaml,
}],
]),
editorAnnotations: [],
codeAnnotations: [],
title: "File Feedback",
subject: "file",
sourceConverted: false,
directEditsSection: "",
savedFileChangesSection: "",
});
expect(feedback).toContain("(lines 2–3) ");
expect(feedback).toContain('Feedback on: "apiVersion: v1"');
// The frontmatter-stripping parse would have relabeled this block to line 5.
expect(feedback).not.toContain("lines 5–6");
});
test("still strips frontmatter for markdown linked docs", () => {
const markdown = "---\ntitle: Runbook\n---\n\nRestart the worker.\n";
const markdownBlocks = parseMarkdownToBlocks(markdown);
const body = markdownBlocks.find((block) => block.type === "paragraph");
if (!body) throw new Error("expected a body block");
expect(body.startLine).toBe(5);
const feedback = buildCompleteAnnotateFeedback({
blocks: [],
annotations: [],
globalAttachments: [],
linkedDocuments: new Map<string, LinkedDocAnnotationEntry>([
["/docs/runbook.md", {
annotations: [{
id: "md-1",
blockId: body.id,
startOffset: 0,
endOffset: 7,
type: AnnotationType.COMMENT,
text: "Say which worker.",
originalText: "Restart",
createdA: 1,
}],
globalAttachments: [],
markdown,
}],
]),
editorAnnotations: [],
codeAnnotations: [],
title: "File Feedback",
subject: "file",
sourceConverted: false,
directEditsSection: "",
savedFileChangesSection: "",
});
expect(feedback).toContain("(line 5) ");
});
});