Files
nexu-io__open-design/apps/web/tests/analytics/export-error-code.test.ts
lefarcen 50843ff149 give the unattributable client failures a real error code (#5960)
* give the unattributable client failures a real error code

Three buckets in the daily stability report name nothing, and in every case the
evidence was already in scope at the emission site:

- BYOK connection test: 801 failures across 201 devices in 24h — 10% of ALL
  connection tests — reported `result.kind` verbatim, so every response the
  daemon's status→kind map had no case for collapsed into `unknown`. The user
  is told "connection failed" and we cannot say whether it was an out-of-credits
  402, a self-signed cert, or a base URL pointing at an HTML login portal.
  `ConnectionTestResponse` already carried `status` and a redacted `detail`;
  only the emission site threw them away.

- Frontend exceptions: the Top-5 table literally renders a row as
  `Error  — 235 次 / ~74 台`. Two independent sources — `buildExceptionList`
  copied `error.name`/`error.message` straight through and both can be '', and
  `parseStack` returned a zero-frame stacktrace for a stackless error, which
  PostHog drops at ingestion so neither `$exception_type` nor
  `$exception_value` survives.

- Export failures (8.1%, 210/24h): everything that was not a sidecar-skew
  message fell through to `err.name`, i.e. the literal "Error".

The daemon side of the same path flattened every sidecar contract violation to
`UPSTREAM_UNAVAILABLE`, which forced the web layer to regex-match English prose
to tell a version-skewed mesh from a dead renderer; it now forwards the
sidecar's real code, and the web classifier treats the remaining generic
envelope codes as non-classifications so a specific message still wins.

Two existing export-code expectations changed on purpose and say why inline: a
generic envelope `.code` no longer outranks message classification, and a
message carrying an HTTP status now reports that status instead of "Error".

* keep the forwarded sidecar code inside ApiErrorCode

Follow-up on the same PR. Forwarding the sidecar contract violation's `code`
compiled, but it was leaving the declared union: `ctx.http.sendApiError` is
typed `(...args: any[]) => any` in server-context.ts, so nothing caught that
`SIDECAR_UNKNOWN_MESSAGE` is not an `ApiErrorCode`. Every web-side handler
typed against that union would have failed to match the code the daemon
actually put on the wire.

Declare the two SIDECAR_* codes in API_ERROR_CODES and narrow the helper's
return type to ApiErrorCode, allowlisting exactly those two so a future sidecar
code cannot silently leave the union again.

No consumer of the export 502 switches on the old code — the UPSTREAM_UNAVAILABLE
call sites are finalize (useFinalizeProject), AMR run guidance, and ChatPane run
errors, none of which are on this route — so this is additive, not a behavior
change for anything that exists today.

* make the attribution actually fire on the production paths

Three review findings, all correct, all verified against the real code paths
rather than the synthetic ones the first cut tested.

1. The sidecar code passthrough was dead. `requestJsonIpc` rejects with
   `new Error(response.error?.message)` — its response type does not even
   declare a `code` — so SIDECAR_UNKNOWN_MESSAGE never reaches the route, and
   the helper always fell back to UPSTREAM_UNAVAILABLE. Removed the passthrough
   (and the two API_ERROR_CODES entries added for it: nothing emits them). The
   message-based classifier already achieves the attribution and is tested;
   dead code that reads as working is worse than no code. Threading the code
   through IPC → API → web export result is a three-layer change and belongs in
   its own PR.

2. The zero-frame guard checked the input instead of the output. A header-only
   stack ("Error: Script error.") is truthy, so it skipped the fallback, and the
   parser then dropped that single line as a message — yielding `frames: []`,
   the exact ingestion failure this change exists to prevent. The guard now runs
   on the PARSED result, covering missing / empty / header-only / unparseable
   stacks, with a fixture matrix over all four.

3. The certificate bucket could not fire in production. undici reports a TLS
   failure as a TypeError whose message is the generic `fetch failed`, with the
   reason only on `cause.code`, and `networkErrorToKind` maps just an allowlist
   — so a self-signed cert became `kind: unknown` + `detail: "fetch failed"`.
   The unit test passed only because it was fed a detail string production never
   emits. The daemon now appends the cause code to `detail`, and the client
   tests assert against that real shape.

* stop a 2xx status from claiming a failure it cannot explain

Review catch, and it made this PR's headline case unreachable. The status
fallback ran before every detail classifier, so the daemon's
`{ kind: 'unknown', status: 200, detail: 'Unexpected token < …' }` — which is
exactly the 'base URL points at an HTML login portal' shape, returned when a 2xx
body fails JSON.parse (connectionTest.ts) — was reported as HTTP_200, and
INVALID_JSON_RESPONSE could never fire.

Reproduced before fixing:
  byokErrorCode({kind:'unknown',status:200,detail:'Unexpected token < …'})
    → 'HTTP_200'

The cut is now at 300 rather than 0. A 2xx that still failed did so for a reason
the status cannot express, so it falls through to the detail classifiers; 3xx
stays on the status side because the daemon fetches with `redirect: 'error'`,
making a redirect a real failure the status does name.

* read the daemon-appended cause code instead of enumerating codes

Review catch on a gap my own previous commit opened: forwarding cause.code from
the daemon put OpenSSL's TLS codes on the wire, and this file could not read
them. UNABLE_TO_GET_ISSUER_CERT_LOCALLY, CERT_NOT_YET_VALID and CERT_HAS_EXPIRED
have no Node-errno shape, and the `\bcertificate\b` fallback never matches a
name that spells it CERT — so the signal was preserved on the wire and dropped
here, landing back in UNKNOWN_NO_SIGNAL.

Read the daemon's known trailing `(CODE)` position directly rather than
extending an allowlist: an enumeration would silently drop every code added
upstream after this file was written, which is the same failure mode one layer
down. Inline codes keep a widened pattern, and the TLS fallback now also matches
CERT-shaped names.

Fixtures cover all seven codes the daemon can now forward.
2026-07-22 06:57:48 +00:00

100 lines
4.9 KiB
TypeScript

import { describe, expect, it } from 'vitest';
import { exportErrorCode } from '../../src/analytics/export-error-code';
describe('exportErrorCode', () => {
it('classifies the daemon↔desktop sidecar version skew from its wrapped message', () => {
// The exact string the daemon surfaces when a freshly-updated daemon sends
// `render-slides` to an older desktop that predates the message. Before this
// helper it was reported as the generic "Error", hiding the skew in analytics.
const err = new Error(
'desktop renderer unavailable: unknown desktop sidecar message: render-slides',
);
expect(exportErrorCode(err)).toBe('DESKTOP_SIDECAR_UNKNOWN_MESSAGE');
});
it('classifies a plain renderer-unavailable failure separately from the skew', () => {
expect(exportErrorCode(new Error('desktop renderer unavailable: connection refused'))).toBe(
'DESKTOP_RENDERER_UNAVAILABLE',
);
});
it('prefers a SPECIFIC structured .code over message classification', () => {
const err = Object.assign(new Error('desktop renderer unavailable: something else'), {
code: 'SIDECAR_UNKNOWN_MESSAGE',
});
expect(exportErrorCode(err)).toBe('SIDECAR_UNKNOWN_MESSAGE');
});
it('does not let a generic envelope .code outrank message classification', () => {
// This case used to assert the opposite. `UPSTREAM_UNAVAILABLE` is the
// daemon's catch-all envelope for the whole 502 branch, not a
// classification — letting it win meant the version-skew fingerprint in the
// message was thrown away in favour of a code that says nothing. The daemon
// now forwards the sidecar's real code (see
// apps/daemon/src/import-export-routes.ts), so a specific code still wins;
// only the generic wrappers defer to the message.
const err = Object.assign(
new Error('desktop renderer unavailable: unknown desktop sidecar message: render-slides'),
{ code: 'UPSTREAM_UNAVAILABLE' },
);
expect(exportErrorCode(err)).toBe('DESKTOP_SIDECAR_UNKNOWN_MESSAGE');
});
it('falls back to the error name for unclassified failures', () => {
expect(exportErrorCode(new TypeError('boom'))).toBe('TypeError');
// Was asserted as 'Error' — a message carrying an HTTP status is now
// reported as that status, which is strictly more attributable than the
// name of the base Error class.
expect(exportErrorCode(new Error('export request failed (500)'))).toBe('HTTP_500');
// A genuinely signal-free message still falls back to the name.
expect(exportErrorCode(new Error('export request failed'))).toBe('Error');
});
it('returns UNKNOWN for non-Error throwables', () => {
expect(exportErrorCode('nope')).toBe('UNKNOWN');
expect(exportErrorCode(undefined)).toBe('UNKNOWN');
});
});
/**
* Export failures run at 8.1% (210 in 24h across 48 devices). Before these
* branches every capture-stage and transport failure fell through to
* `err.name`, which is the literal string "Error" for anything thrown as a
* plain Error — the single largest export bucket named nothing at all.
*/
describe('exportErrorCode — non-sidecar failures', () => {
it('does not let a generic daemon envelope code outrank message classification', () => {
// The daemon wraps unrelated failures in BAD_REQUEST/INTERNAL. Those are
// envelopes, not classifications, so the message must still win.
const err = Object.assign(new Error('export timed out after 60s'), { code: 'BAD_REQUEST' });
expect(exportErrorCode(err)).toBe('TIMEOUT');
});
it('separates the capture-stage failures from each other', () => {
expect(exportErrorCode(new Error('nothing was captured'))).toBe('EMPTY_CAPTURE');
expect(exportErrorCode(new Error('canvas is not available'))).toBe('CANVAS_UNAVAILABLE');
expect(exportErrorCode(new Error('unreadable response'))).toBe('UNREADABLE_RESPONSE');
expect(exportErrorCode(new Error('invalid data url'))).toBe('INVALID_DATA_URL');
expect(exportErrorCode(new Error('download failed'))).toBe('DOWNLOAD_FAILED');
});
it('names transport failures instead of leaving them as "Error"', () => {
expect(exportErrorCode(new Error('fetch failed'))).toBe('NETWORK');
expect(exportErrorCode(new Error('getaddrinfo ENOTFOUND host'))).toBe('NETWORK');
expect(exportErrorCode(new Error('ETIMEDOUT'))).toBe('TIMEOUT');
expect(exportErrorCode(new Error('429 Too Many Requests'))).toBe('RATE_LIMITED');
expect(exportErrorCode(new Error('unauthorized'))).toBe('FORBIDDEN');
});
it('falls back to the HTTP status when the message only carries one', () => {
expect(exportErrorCode(new Error('render service replied 503'))).toBe('HTTP_503');
});
it('still yields a usable code for a bare Error', () => {
// Cannot do better than the name here, but it must not be empty.
expect(exportErrorCode(new Error(''))).toBe('Error');
expect(exportErrorCode('a string throw')).toBe('UNKNOWN');
});
});