mirror of
https://github.com/nexu-io/open-design.git
synced 2026-09-20 06:15:06 +08:00
open-design-crew[bot]
2828a086c8
* feat(web): add an Open Design Labs settings page with the Design Harness switch OD Next has been opt-in through `odNextStrategyMode` since #7362, but the only way to reach it was `od config set` or an environment variable — and `PACKAGED_CHILD_ENV_ALLOWLIST` strips that variable before the packaged daemon ever sees it. A packaged user had no way to choose either direction, so the strategy was effectively invisible to everyone outside a terminal. Add the missing surface: a Labs page in Settings, between General and Instructions, carrying one row for Design Harness. The section reads and writes on its own rather than threading through `cfg` / `setCfg`. `apps/web/src/types.ts`'s `AppConfig` is a web-local projection that does not carry `odNextStrategyMode`, and `syncConfigToDaemon` serialises an explicit allow-list, so the dialog route costs three edits and fails silently if the allow-list one is missed. One `GET /api/strategies/od-next/rollout` answers the saved mode, who set it, and whether the local safety latch has tripped; one single-field `PUT /api/app-config` writes it back, which the daemon merges per key. Three states lock the switch, each with its own sentence: an environment variable owns the mode, the local latch has tripped (it overrides the saved mode, so the switch would otherwise read as on while every run took the ordinary route), or the status could not be read at all. The lock uses `aria-disabled` rather than `disabled` so the explanation stays reachable by keyboard. Turning the switch off writes `'off'` rather than clearing the key: an absent value means "never touched", which is worth telling apart from a deliberate opt-out. `observe` renders as off and is never rewritten — it is a developer diagnostic mode, and clearing it silently would break a debugging session. "Applies to your next generation" is a permanent line under the item name, not a confirmation that appears after the fact. The switch has no visible effect until the next run, so the user should read that before flipping it. Save outcome reports on the dialog's existing autosave pill, the same surface every other Settings edit uses. `settingsSectionToTracking` also gains a `labs` case. Its default returns `configure_execution_mode`, so without this a Labs page view would have been counted as an execution-mode view and inflated that funnel. * fix(web): start one harness write per click burst Found by clicking the switch three times in a row against a live daemon: all three writes went out. `busy` is state, so the second and third clicks in the same tick still read the pre-render closure, where `busy` is false and `on` still holds the old value — each one starts its own write from a stale baseline. Gate re-entry on a ref that flips synchronously instead. `busy` keeps driving the dimmed styling, which is all it was ever able to do reliably. * feat(analytics): report which harness produced a run, and Labs opt-in The Labs switch exists so a new generation strategy can be compared against the old one. That comparison needs two things nothing was reporting: which side of it a given run landed on, and whether anyone is turning the switch on at all. `run_created` / `run_finished` gain `harness` plus `harness_fallback_reason`. The reason is the more useful half: the switch can be on and the run still take the ordinary route — the project's task type, the selected agent, an explicitly chosen plugin, or an unverified runtime capability each route back, silently. Without the reason code, "I enabled it and nothing changed" is unanswerable from the data; with it, the size of each silent-fallback bucket is a query. Stamped once, on the run-created base props. `run_finished` spreads those, and the crash-recovery replay stores them verbatim, so the dimension reaches all three emitters from a single assignment rather than three that can drift. `labs_item_toggled` is deliberately generic: the experiment is a property (`item_id`), not part of the event name, so a second experiment reuses the event and every dashboard built on it keeps working. Same for `harness` — a third harness is one more enum member, not a new field. The event fires after the preference is persisted, not on click. A failed write rolls the switch back, and an event asserting a preference the machine does not hold is worse than a missing one. Nothing new is built for delivery: `$insert_id` already dedupes (analytics.ts), identity already rides as a PostHog super-property (identity.ts), and both sinks already gate on consent. Best-effort throughout — no local queue, no retry, no compensation path. * feat(web): collect why users turn a Labs experiment off The opt-out event carried no reason, so PostHog could show how many people turned Design Harness off and nothing about why — and the `reason` fields the contract already reserved stayed permanently empty. Ask inline, right after the switch moves. Not a dialog: the user has just declined something, and a modal asking them to justify it reads as friction. The panel resolves exactly once, and every way out of it reports — a chosen reason, an explicit skip, the 120-second timeout, or leaving the page with the question still open. That last one matters: without it an abandoned question is a silent gap, and the share of people who declined to answer reads lower than it is. `skipped` is not a reason, it is the recorded absence of one, which is why a timeout and a deliberate skip store the same value. Picking "other" stops the clock. It is an explicit intent to write something, and taking the panel away mid-sentence would discard it. The opt-out still emits immediately without a reason, and the answer arrives as a second event. Opt-out counts stay whole even when nobody answers; reason distribution reads off the follow-up. Also aligns the switch itself with the toggle the rest of Settings already uses (`.toggle-slider` — 36x20, `--brand`); it had been drawn larger and in a different green. Hover and focus-visible states are new: the shared toggles only lift on hover in dark mode, so in the default theme this control gave no pointer affordance at all and keyboard focus was invisible. * fix(web): match the Labs reason panel to the approved mock The chips shipped as full pills copied from the older assistant-feedback panel; the mock asks for rounded rectangles. Take the geometry from the mock directly — 8px radius, 6x11 padding, 7px gaps, 9px on the input and submit — with tokens standing in for its literal greys. Also stop the switch drifting. The row was centre-aligned, so opening the reason panel pushed the switch down to the middle of a block it does not belong to. Top-align the row and offset the switch to sit level with the name and hint instead, which is where the mock puts it and where it stays whether or not the panel is open. * fix(daemon): keep the harness decision across a clarification continuation Answering a clarification creates a second physical Run, and the rollout is only evaluated on the branch that resolves a project — which that path does not take. So a continuation carried no decision at all, and its `run_created`, `run_finished` and recovery replay reported no harness. The runs that hit this are exactly the OD Next runs that stopped to ask a question. Dropping them leaves the comparison the dimension exists for biased toward tasks that never needed clarifying, which is the opposite of a neutral sample. Inherit from the source Run rather than re-reading the setting: the user may have flipped the switch while the question was on screen, and this Run belongs to the decision its task started under. `resolveClarificationContinuation` already validates that source against the locked task, so it is the trustworthy copy — the same reasoning, and the same seam, as the task input snapshot the continuation already inherits beside it. An idempotent strategy retry skips the evaluation branch for the same reason and gets the same treatment, reading the decision off the run it is retrying. Regression coverage asserts inheritance through both `/api/runs` and `/api/chat`, and that a source Run with no decision still yields none — absent and "took the ordinary route" are different facts, and a task that predates the strategy must not be back-filled into either arm. Reported in QA of #7445. * feat(web): say what the Design Harness switch does, not when it applies The line under the item explained timing to someone who had already decided. It now says what turning it on gets you — the latest OpenDesign generation strategy, better-looking output — and marks it as beta, which is the part that sets expectations for something shipped through Labs. * fix(web): three defects in the Labs switch found in QA **Double-counted opt-outs.** Turning the switch off opens the reason question; turning it back on left that question open, still asking about a switch that was now on. The stale question settled later as a skip, so a fumbled off/on reported two opt-outs against a single reason row. Re-enabling now retracts the question, restoring the invariant every count depends on: one reported opt-out, one reason row. **The saved confirmation never went away.** The dialog's autosave pill is a shared surface with no timer of its own — whoever sets it owns clearing it, and this switch only ever set it. The confirmation sat there for the rest of the session and followed the user into every other settings section. It now returns to idle after three seconds, and immediately if the user leaves the section first, since by then it describes an edit they can no longer see. **The disabled submit button read as blank.** The mock draws white-on-grey there, which against this palette lands at roughly 1.9:1 — the label was effectively invisible rather than merely dimmed. Mute the surface instead of the text so the word stays legible while the button is unavailable. All three verified against the running client, and each has a regression test. * style(web): sit autosave status on the chrome row and soften the settings card edge Two visual corrections to the Settings page. The autosave pill moves from the bottom of the viewport to the app's own top chrome row, centred on it. OPEND-2148 parked it at the bottom to get it off the Local CLI pickers; the chrome row clears them just as well and is the better home — on the full page that band is empty, and in the dialog it sits above the panel rather than over any of its content. What that ticket guarantees is unchanged: a viewport-level layer, out of the controls' band, and under the body-portaled model picker. Its spec is updated to pin that guarantee rather than the edge it hangs from. The content card's hairline steps down from `--border` to `--border-soft`. At that size the card is a large, quiet plane and the standard #dbdbdb rule read as a hard box drawn around it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Cheems <94773058+itscheems@users.noreply.github.com> Co-authored-by: wangchenglong <honam884844@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
32 lines
1.1 KiB
TypeScript
32 lines
1.1 KiB
TypeScript
import { describe, expect, it } from 'vitest';
|
|
|
|
import { settingsSectionToTracking } from '../src/analytics/events.js';
|
|
|
|
describe('settingsSectionToTracking', () => {
|
|
it('gives the Labs settings page an area of its own', () => {
|
|
// The fallback is `configure_execution_mode`, so a section that reaches it
|
|
// is not merely unlabelled — it is counted as an execution-mode view and
|
|
// inflates that funnel. Labs is user-reachable from the sidebar, so it
|
|
// needs its own area.
|
|
expect(settingsSectionToTracking('labs')).toBe('labs');
|
|
});
|
|
|
|
it('keeps each mapped sidebar section distinct from the execution fallback', () => {
|
|
const sections = [
|
|
'instructions',
|
|
'memory',
|
|
'media',
|
|
'privacy',
|
|
'about',
|
|
'labs',
|
|
] as const;
|
|
const areas = sections.map((section) => settingsSectionToTracking(section));
|
|
expect(areas).not.toContain('configure_execution_mode');
|
|
expect(new Set(areas).size).toBe(sections.length);
|
|
});
|
|
|
|
it('still resolves an unknown section rather than throwing', () => {
|
|
expect(settingsSectionToTracking('not-a-section')).toBe('configure_execution_mode');
|
|
});
|
|
});
|