mirror of https://github.com/nexu-io/open-design.git synced 2026-09-20 06:15:06 +08:00

Files

T

lefarcen 3ddff71205 feat(ci): publish the What's New card from the repository (#7892 )

* feat(ci): publish the What's New card from the repository

Move the post-update highlight document into the repo and publish it from
GitHub Actions, so changing the card is a pull request instead of a task
that needs personal Cloudflare credentials.

- docs/whats-new.json is the source of truth, next to its spec doc.
- pnpm guard validates it through the daemon's own parser on every PR, so a
  document that would silently show no card cannot merge.
- whats-new-publish.yml uploads on merge to main (and on manual dispatch,
  with a read-only dry-run mode) and verifies the object by reading it back
  from the origin the daemon fetches, past the edge cache.

* docs: name the CI job that actually runs the guard check

* fix(ci): gate the What's New publish behind a main-only environment

A workflow_dispatch runs the workflow file from the ref it is dispatched
against, so the in-workflow main-only assertion was editable by whoever
triggered it: anyone with write access could publish unreviewed content to
every installed client from their own branch.

Split the workflow so credentials live in one place and that place is gated by
GitHub rather than by YAML. `validate` holds no secrets and runs on any ref;
`publish` declares the `whats-new-publish` environment, whose deployment-branch
policy allows main only and which owns the R2 secrets. Dropping the environment
declaration to escape the branch policy also drops the secrets.

Also points the card at the new Arena URLs supplied by product.

* fix(ci): pin the What's New publish trust boundary in a check that runs

The missing-credentials error still told operators the R2 keys are
repository secrets. Following that string is the exact setup the previous
revision closed: repository secrets are readable from any job on any
branch, so a modified workflow dispatched from an unreviewed ref could
publish to every installed client — or just take the keys. Point the
error at the `whats-new-publish` environment and refuse the repository
path in the same breath.

The boundary itself had no coverage: nothing read the workflow, so moving
`environment:` or a `secrets.*` expression back onto the arbitrary-ref
job would stay green while still publishing correctly from main.

That coverage cannot be lane-routed. Running the planner over
`.github/workflows/whats-new-publish.yml` selects neither
`web_tests_required` nor `ui_p0_validation_required`, so the `e2e_vitest`
workload does not run for a workflow-only edit — an assertion living only
there would be skipped for exactly the change class it guards. The check
therefore lives in `pnpm guard`, which CI invokes from the unconditional
preflight job, same reasoning as the document guard next to it.

It asserts that the credential-bearing job declares the environment by
its exact name, needs `validate` and is gated on its output with no
`always()`/`cancelled()` escape, reads all four bucket bindings, and that
no other job attaches an environment or reads any secret at all.

The e2e suite drives it over the real workflow mutated into each of those
regressions in turn, so every assertion is proven able to fail; gutting
the checker turns all eleven red.

* chore(whats-new): seed the document from the live card

Bringing the workflow in with new copy attached ties the infrastructure to
a content decision. The Arena CTA this document pointed at is not live —
`/llm-arena-for-design/` and `/zh/llm-arena/llm-arena-for-design/` both
404 — and the card renders regardless, so merging would have put a dead
button on every client's Home.

Seed the repository copy from the object that is already serving instead:
byte-identical to `https://whatsnew.open-design.ai/whats-new.json`, id
`0.21.0`. That puts the current state under version control and makes the
first automatic publish idempotent — it uploads exactly what is already
there, so the merge exercises the whole chain (validate, upload, read-back
against the origin the daemon fetches) without changing anything a user
sees.

The Arena copy is not lost; it moves to the PR body as a follow-up for the
release owner, who can now change the card through this workflow without
holding personal Cloudflare credentials.

* fix(ci): detect secrets by context, not by access shape

The guard added last round matched `${{ secrets.NAME }}` with a regex over
the raw workflow text. A guard whose whole job is to stop a secret reaching
the arbitrary-ref job had a blind spot for most of the ways to read one.

Verified against the previous implementation, each returning zero
violations while a secret reached `validate`: `secrets['NAME']`,
`secrets["NAME"]`, `secrets [ 'NAME' ]`, `secrets . NAME`,
`toJSON(secrets)`, a bare `secrets`, and `fromJSON(toJSON(secrets)).NAME`.

Extending the regex would have repeated the mistake that caused it. One
bypass proves why: a double-quoted YAML scalar may carry a backslash line
continuation, so

    STOLEN: "${{ sec\
      rets.NAME }}"

parses to a live secret read while the source text contains no such
substring. No text scanner can see that, however many shapes it lists.

So the workflow is read through a YAML parser, and detection is a single
property instead of an enumeration: GitHub's expression language has no
string-to-context evaluation, so a secret cannot be reached without naming
the `secrets` context as an identifier. Finding that identifier inside any
`${{ … }}` expression covers every access form, including ones nobody has
thought of.

Parsing also retires the hand-rolled job splitter and its own failure
modes, and lets the check distinguish "no violations" from "nothing was
examined" — an unparseable document, a flow-style jobs mapping, or a
renamed publish job now fail closed rather than passing quietly.

Two holes the shape list never covered are closed as a side effect: a
job-level `secrets:` key passes credentials to a called workflow with no
expression at all, and workflow-level `env:` is inherited by every job.

`yaml@2.9.0` becomes a root devDependency. It was already in the lockfile
via `tools/release` and pinned repository-wide by the existing pnpm
override, so the lockfile gains only the root importer entry.

Test coverage moves with it: 26 of 46 cases now go red when the checker is
gutted, up from 11.

---------

Co-authored-by: lefarcen <ontf116@gmail.com>

2026-09-08 11:14:16 +00:00

20 KiB

Raw Permalink Blame History

tools/pack

Follow the root AGENTS.md and tools/AGENTS.md first. This tool owns the repo-external packaged build/start/stop/logs command surface.

Read tools/pack/CACHE.md before changing any build-cache node key, adding a cache node, or changing what a cached node reads or writes. It is the source of truth for the build-graph cache under --cache-dir: determinant rules, materialization-time parameters, the signing boundary, and confidence grading.

Owns

Local packaging orchestration for packaged Open Design artifacts.
mac build/install/start/stop/logs/uninstall/cleanup smoke commands.
Windows NSIS build/install/start/stop/logs/uninstall/cleanup/list/reset smoke commands.
Windows registry observation/cleanup must go through reg.exe and stay scoped to entries matching the namespace install/uninstaller paths.
Windows lifecycle logs must expose NSIS automation logs/markers/timings in addition to app runtime logs.
Linux AppImage build/install/start/stop/logs/uninstall/cleanup smoke commands.
Linux headless (no-Electron) install/start/stop via --headless flag on install, start, and stop.
Linux containerized builds via electronuserland/builder Docker image for distro-agnostic glibc compat.
Consuming sidecar/process/path primitives from @open-design/sidecar-proto, @open-design/sidecar, and @open-design/platform.

Does not own

Product business logic.
Sidecar protocol definitions.
A second process identity model.
Product/business update runtime integration.

Rules

Keep cross-platform source responsibilities in named directories such as cache/, config/, launcher/, resources/, updates/, and versioning/; keep platform-owned behavior below mac/ or win/. Mirror those responsibilities in tests/ when a test set has the same ownership. A new root-level src/*.ts file is intentionally treated as unclassified by CI and pays the conservative Windows payload fallback until it is placed in an owned source unit.
Tests import source modules through the test-only @/* alias. Tests that intentionally inspect source text use the same alias with Vitest's ?raw suffix; do not reintroduce directory-depth-dependent ../src/ imports or file URLs.
Do not hand-build stamp argv, IPC paths, or process matching; use @open-design/sidecar launch/discovery/invoke/stop atomics.
Do not use port numbers in data/log/runtime/cache path decisions. Namespace decides paths; ports are only transient transports.
Public release artifacts must use channel-specific app identity: stable uses Open Design, beta uses Open Design Beta, prerelease uses Open Design Prerelease, and preview uses Open Design Preview. Local tools-pack installs may still use namespace-scoped install paths only as a developer multi-instance validation convention.
Do not let namespace-named .app installs change data/log/runtime/cache path conventions.
--dir controls tools-pack output/runtime/install validation roots only. It must not be treated as the cache root. The default workspace tools-pack cache is the hot path. --cache-dir is a special-case escape hatch for cache isolation or cold-cache validation, not a routine QA/build parameter.
Use --portable for public/release artifacts so packaged config does not bake local tools-pack runtime roots from the build machine.
Pack resource files used by electron-builder belong under tools/pack/resources/; do not point pack logic at Downloads, web public assets, docs assets, or other app-owned resource paths.
For ordinary Windows NSIS smoke tests, use short namespaces such as rg, smoke, or nsis-a. NSIS extracts deeply nested Next.js standalone files under the namespace-scoped install directory; long namespaces can push installed paths past the traditional Windows 260-character limit even when builder win-unpacked output is correct. During merge regression, namespace regression-merge-nsis produced an installed path length of 264 characters and missed next/dist/server/route-matcher-providers/helpers/cached-route-matcher-provider.js in the installed directory, while the same NSIS smoke passed with namespace rg. Use long namespaces only when intentionally testing installer path-length behavior.

Packaged auto-update architecture and harness

Read this section before changing packaged auto-update behavior. The updater crosses package, desktop, web UI, release-feed, and installer surfaces, so bugs often hide between otherwise-green package tests.

Architecture map

apps/desktop/src/main/updater.ts owns updater state, release metadata parsing, artifact selection, checksum verification, download-store ownership, progress events, and opening the downloaded installer. It is pure main-process logic and is tested under apps/desktop/tests/main/updater.test.ts.
apps/desktop/src/main/runtime.ts exposes updater IPC to the renderer through od:update:status|check|download|install|quit and emits od:update:status-changed. Keep installer launch separate from process shutdown; quit is an explicit post-installer action.
apps/desktop/src/main/index.ts wires the scheduler and the packaged macOS app-menu update item. The native item mirrors updater state and opens the renderer-owned update dialog; it must not create a second updater or a native result dialog. Windows and Linux menus do not expose update actions.
apps/web/src/lib/updater.ts normalizes host updater snapshots into UI-ready state.
apps/web/src/components/UpdaterPopup.tsx remains the ready-update surface in the left rail. apps/web/src/components/UpdateDialog.tsx owns the explicit macOS app-menu check flow. All visible copy and native menu labels must go through apps/web/src/i18n.
packages/launcher-proto owns launcher pointer, attempt, and desktop-handoff journal shapes plus payload selection. runtime.json together with attempt.json is the only payload-version state machine.
apps/packaged/src/index.ts delegates to the selected payload desktop before initializing the outer Electron runtime, then passes packaged appVersion and namespace-scoped updateRoot into desktop main only when the outer itself must run.
apps/daemon/src/sidecar/payload-desktop-handoff.ts owns the historical-outer bridge. It uses the daemon's SidecarFactory client to confirm and shut down the exact outer desktop, then spawnSidecar to launch the payload generation while preserving the true previous launcher pointer across the prepared → armed → confirmed journal lifecycle.
install.json continues to identify the physically installed outer executable for recovery. Payload activation or handoff must not rewrite it to a versioned payload executable.
tools/serve owns deterministic local updater fixtures only. It must not contain product updater runtime logic.
tools/pack owns packaged build/install/start/inspect/logs/uninstall/cleanup and the platform installer harness, including Windows NSIS registry observation and cleanup.

Release metadata shape

The runtime updater reads https://releases.open-design.ai/<channel>/latest/metadata.json unless OD_UPDATE_METADATA_URL overrides it. For package-launcher updates:

A valid packaged-launcher context prefers platforms.<platform>.artifacts.payload; the platform installer (dmg on macOS or installer on Windows) remains the recovery/fallback path.
The artifact must have a checksum, preferably sha256Url; the updater verifies bytes before exposing an install action.
OD_UPDATE_CURRENT_VERSION may override the packaged version for tests, but user-flow package validation should prefer building the package with the intended --app-version.
Release metadata may include releaseNote.content, with a defaultLocale and locale descriptors containing url, mediaType, sha256, and size. The updater does not currently consume this block; tools-release owns its publication and verification independently from updater UI behavior.
Release metadata may include control.launcher.version.{min, url} — the installer-reinstall floor. The updater compares min against the physically installed outer package version (read at check time from the outer bundle's open-design-config.json via the launcher launch path; OD_UPDATE_INSTALLED_VERSION overrides it for tests), NOT the running payload version — payload updates never touch the outer bundle, so a broken outer generation must reach the installer path even when its payload is current. When the floor trips (or min is set but the outer version is unreadable — conservative), the updater selects the installer artifact and, when no newer release exists, offers a same-version installer reinstall; snapshot field reinstall carries {reason, installedVersion, minVersion, url} and the web UI presents the optional operator url as a jump link with default i18n copy as fallback. Publication: channel policy is managed as one repo-vars pair per channel — RELEASE_LAUNCHER_VERSION_MIN_<CHANNEL> + RELEASE_LAUNCHER_VERSION_MIN_URL_<CHANNEL> — passed through workflows verbatim (no YAML fallback expressions) and resolved by the shared resolver in tools/release/src/storage/launcher-version-floor.ts: a non-stable channel whose own pair is unset falls back to the STABLE pair as a unit, and format/https/floor validation is applied at that single point. publish-metadata hard-fails when min exceeds the release version — a floor this release cannot satisfy would make the same-version reinstall offer nag forever. verify-metadata re-resolves the same channel policy and checks the published block against it; summary-metadata surfaces the floor in the step summary.
The updater exposes a manual disaster-recovery clear-cache action (od:update:clear-cache IPC, sidecar action clear-cache, Settings → About "Clear update cache" row with a two-stage inline confirm). It resets one-shot update state (downloaded release, install freeze) back to idle, purges releases/, staging/, downloads/, and .back/, removes a stale launcher attempt.json plus any non-confirmed desktop-handoff journal, and deletes non-retained launcher payload versions. Runtime active/lastSuccessful versions, explicit retained cleanup entries, install.json, and a confirmed handoff journal are never touched; locked files defer through the existing cleanup.json retry machinery. An installer helper already spawned by a prior install action is not cancelled.
Release-note source lives at docs/CHANGELOG/v<full-releaseVersion>/<locale>.md. All channels use the same publication pipeline, while stable additionally requires en and zh-CN before platform builds proceed.
Post-update "what's new" highlights are NOT carried in release metadata.json. The daemon's /api/whats-new fetches a single hand-curated document on a dedicated R2 bucket (https://whatsnew.open-design.ai/whats-new.json, overridable with OD_WHATS_NEW_URL); the web home surface shows a one-time card driven by that document's id, not the running version. The document is maintained in this repository at docs/whats-new.json and published by .github/workflows/whats-new-publish.yml; operators edit that one file after a release — there is no per-version publish tooling. See docs/whats-new.md.

Channel identity rules

Channel identity must be stable across install, update install, shortcuts, registry entries, and app data:

Stable: Open Design, namespace default or stable release namespace.
Beta Windows: Open Design Beta, namespace release-beta-win, uninstall key Open Design-release-beta-win.
Prerelease Windows: Open Design Prerelease, namespace release-prerelease-win, uninstall key Open Design-release-prerelease-win.
Preview Windows: Open Design Preview, namespace release-preview-win, uninstall key Open Design-release-preview-win.
Beta-like ad hoc namespaces such as beta-local-flow are test namespaces, not the beta channel. They must not be used for user-flow beta validation because they create a different registry key while sharing a confusing display name/path.

If a local release-channel package is meant to be updated by a real feed, build it with the matching release namespace and an older matching --app-version such as --namespace release-beta-win --app-version 0.10.0-beta.1 or --namespace release-prerelease-win --app-version 0.10.0-prerelease.1. Otherwise the installed package and the downloaded package can appear as separate registry entries even though they target the same display name.

Deterministic fixture harness

Use tools-serve start updater for fast, deterministic tests and e2e automation where network release state is not the thing under test. Fixture flow:

pnpm tools-serve start updater --json --channel beta --version 99.0.0-beta.1 --platform win

Then launch packaged desktop with:

OD_UPDATE_ENABLED=1
OD_UPDATE_METADATA_URL=<fixture metadataUrl>
OD_UPDATE_CURRENT_VERSION=99.0.0-beta.0
OD_UPDATE_OPEN_DRY_RUN=1
OD_UPDATE_AUTO_CHECK=1

This harness is appropriate for asserting IPC, popup rendering, progress, checksum/download-store behavior, and dry-run installer opening. It is not a full user-view validation because it replaces the public release feed and uses synthetic artifact bytes.

High-confidence local user-flow acceptance

Use this when validating release-channel behavior before handing a Windows beta build to a human tester. This path intentionally avoids mock services and exercises the selected real beta feed. For the self-hosted release-beta-s lane, the real feed is the Nexu S3 origin configured by release_public_origin, currently https://s3.nexu.space/od-releases.

Confirm the latest beta metadata first:

curl.exe --ssl-no-revoke -fsSL https://releases.open-design.ai/beta/latest/metadata.json

For release-beta-s, check the internal feed instead:

curl.exe --ssl-no-revoke -fsSL https://s3.nexu.space/od-releases/betas/latest/metadata.json

Build a non-portable Windows beta package with the real beta namespace and a version lower than latest:

pnpm tools-pack win build --dir C:\odtp-beta-release-fixed --namespace release-beta-win --to nsis --app-version 0.8.0-beta.5 --json

Give the tester the generated installer:

C:\odtp-beta-release-fixed\out\win\namespaces\release-beta-win\builder\Open Design-release-beta-win-setup.exe

Expected user flow:

User installs 0.8.0-beta.5 through the NSIS UI.
User launches Open Design Beta.
App auto-checks the real beta feed and selects the latest Windows launcher payload when the package-launcher context is valid. The installer is the fallback path when the payload artifact or launcher context is unavailable.
For the payload path, the app downloads platforms.win.artifacts.payload, verifies sha256, prepares the payload under %APPDATA%\Open Design\launcher\channels\beta\namespaces\release-beta-win\versions\<version>\payload, and shows the web updater popup.
The native Windows File menu must not expose update actions. On macOS, the app menu exposes the state-aware update item and opens the renderer update dialog without making background checks intrusive.
The updater popup uses i18n strings and download progress must not flash to 100% before real bytes arrive.
Applying the payload update should quit and relaunch the exact executable under the prepared version's payload directory, then mark launcher active and lastSuccessful to that version and clear attempt.json.
For a historical outer, the handoff must retain the true previous launcher pointer for fail-closed recovery and terminate with desktop-handoff.json absent or confirmed; prepared and armed are not successful terminal states.
After a full stop, launching the installed shortcut/outer again must still converge on the same active payload desktop and preserve daemon/API behavior, including a real PPTX export.
If the updater falls back to the installer path, clicking Open installer opens the real downloaded beta installer. Installing it should overwrite the same Open Design-release-beta-win registry key, not create a second beta key.

Registry and launcher sanity check after beta.6 update:

Get-ItemProperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*' -ErrorAction SilentlyContinue |
  Where-Object { $_.DisplayName -like 'Open Design*' } |
  Select-Object PSChildName,DisplayName,DisplayVersion,InstallLocation

Get-Content "$env:APPDATA\Open Design\launcher\channels\beta\namespaces\release-beta-win\runtime.json"

For a clean beta channel result, expect one beta entry with PSChildName Open Design-release-beta-win and the latest DisplayVersion. For the payload path, also expect launcher active.version and lastSuccessful.version to match the latest beta version, attempt.json to be absent, and the running desktop executable to resolve below that version's payload directory. desktop-handoff.json may be absent for a current outer or confirmed for a historical outer; prepared and armed are not successful terminal states. Windows Settings > Apps may cache uninstall metadata within the current view. If Settings still shows the previous beta version after the registry query is correct, switch away from the Apps view and back, or reopen Settings, before treating it as an installer failure. The registry query above is the source of truth for this harness.

Avoid leaving validation residue. Stop running app processes first, then use tools-pack uninstall/cleanup for tool-managed namespaces. Only delete explicit temp roots after verifying the resolved path is exactly the intended directory. --dir is an output/runtime root, not the default cache root. Do not add --cache-dir to routine validation; it is an escape hatch for cache isolation or cold-cache validation only.

pnpm tools-pack win stop --dir C:\odtp-beta-release-fixed --namespace release-beta-win --json
pnpm tools-pack win uninstall --dir C:\odtp-beta-release-fixed --namespace release-beta-win --remove-product-user-data --remove-data --remove-logs --remove-sidecars --json
pnpm tools-pack win cleanup --dir C:\odtp-beta-release-fixed --namespace release-beta-win --remove-product-user-data --remove-data --remove-logs --remove-sidecars --json

Validation matrix for updater changes

docs/testing/updater-lifecycle.md is the full lifecycle-to-test coverage map (including deliberate manual-only nodes); consult it to find the owning tests for the node you touched, then run the narrow tests plus the repo checks:

pnpm --filter @open-design/desktop test -- tests/main/updater.test.ts tests/main/updater-host-boundary.test.ts tests/main/preload-host-boundary.test.ts
pnpm --filter @open-design/web test -- tests/components/UpdaterPopup.test.tsx tests/lib/updater.test.ts
pnpm --filter @open-design/tools-serve test
pnpm --filter @open-design/tools-pack test -- tests/win-identity.test.ts tests/win-app.test.ts tests/win-builder.test.ts
pnpm --filter @open-design/desktop typecheck
pnpm --filter @open-design/web typecheck
pnpm --filter @open-design/tools-pack typecheck
pnpm --filter @open-design/tools-serve typecheck
git diff --check
pnpm guard
pnpm typecheck

Run the high-confidence local user-flow acceptance whenever a change touches real release feed selection, channel identity, Windows registry/install behavior, installer opening, or visible updater UI behavior. For launcher payload or handoff changes, also run the platform full spec. The full profile must validate exact desktop executable identity, a real PPTX response, a complete stop followed by an installed-outer cold start, and the same checks again after restart. Windows beta full validation must use release-beta-win; a beta-like local namespace is not equivalent delivery evidence.

20 KiB Raw Permalink Blame History