One catalog.
Two ways to find it.

The decision we can make now

Two practical release trains

Train 01 ships the useful local product. Train 02 adds optional Smart ranking without another research program or staged rollout ladder.

flowchart LR
  T1["TRAIN 01<br/>Curated local catalog"] --> G1{"Quality + offline + install<br/>gates pass?"}
  G1 -->|yes| T2["TRAIN 02<br/>Optional Smart beta"]
  G1 -->|no| T1
  T2 -. disable Smart .-> T1
  classDef first fill:#d7ff4f,stroke:#171713,color:#171713,stroke-width:2px
  classDef gated fill:#3859ff,stroke:#171713,color:#fff,stroke-width:2px
  class T1 first
  class T2 gated

Responsibility and handoff matrix

sequenceDiagram
  participant HF as Hyperframes OSS
  participant MU as media-use
  participant CQ as Catalog quality
  participant API as HeyGen API / EF
  HF->>HF: Ship artifact + local scorer + CLI/Studio/SDK
  HF->>MU: Publish stable JSON + install contract
  MU->>HF: Resolve canonical ID through runAdd
  HF->>CQ: Review public entries against the quality bar
  CQ-->>HF: Publish approved allowlist + exclusion reasons
  HF->>API: Publish the exact curated catalog digest
  API->>HF: Serve optional Smart results for that digest
  API-->>HF: On error, client keeps local results

Named individuals and repo-specific approvers are intentionally deferred; the ownership boundaries above are the source-of-truth contract until those names are assigned.

The catalog is generated, not hand-counted

The evaluated branch scans committed primitive source artifacts and the registry manifests at Git HEAD. It validates each source, merges reviewed duplicate names, sorts the result, and emits a shelf plus a machine-readable index.

All five methods, in words a teammate can use

“Recall at top 20” means: among 405 test briefs, after a method returned only 20 catalog entries, how often did those 20 include at least one entry the gold set considered acceptable? It does not mean users liked the first result, and it is not a production conversion rate.

Source: evals/catalog-retrieval/round4/summary4.json, 405 scored briefs, k=20. The dithered bars make the candidate winner visible; the exact values remain printed so texture is never the only signal.

The default we take from the existing evidence

Use the 70% word-rank hybrid as the initial Smart recipe because it is the strongest existing top-20 result. Do not start another candidate bake-off. Train 02 gets ordinary release QA: representative query smoke tests, contract checks, load, privacy and operational verification. Local remains the default regardless of Smart relevance.

How people use it

Mode simulator

Six paths the product must make obvious

flowchart TD
  Start["User opens CLI, Studio or media-use"] --> Local["Load bundled catalog artifact"]
  Local --> Search["Search / browse locally"]
  Search --> Choice{"Choose Smart?"}
  Choice -->|no| Select["Select canonical ID"]
  Choice -->|yes| Auth{"Signed in?"}
  Auth -->|no| Login["OAuth login"]
  Auth -->|yes| Remote["QUERY + exact catalog digest"]
  Login --> Remote
  Remote --> Healthy{"Valid compatible response?"}
  Healthy -->|yes| Select
  Healthy -->|no / timeout / 429| Fallback["Keep local results visible"]
  Fallback --> Search
  Select --> Verify["Verify manifest, files, hashes, compatibility"]
  Verify --> Install["Existing runAdd installer"]
  Install --> Done["Record provenance + written files"]
  classDef local fill:#d7ff4f,stroke:#171713,color:#171713
  classDef remote fill:#3859ff,stroke:#171713,color:#fff
  class Local,Search,Fallback local
  class Remote remote

One result contract, two retrieval paths

Ownership

End-to-end request flow

flowchart LR
  U[Hyperframes user] --> C{Search mode}
  C -->|local default| A[Bundled catalog artifact]
  A --> L[Local lexical scorer]
  C -->|explicit smart| H[Shared HeyGen credentials]
  H --> API[QUERY /v3/hyperframes/catalog]
  API --> N[EF immutable index blob + memory cache]
  N --> K[Keyword arm]
  N --> S[Semantic arm]
  K --> R[Fixed 70 / 30 RRF]
  S --> R
  L --> G[Shared result contract]
  R --> G
  G --> V[Compatibility validation]
  V --> I[Existing runAdd installer]

Where the data lives

The API can never return something the client cannot install

{
  "schemaVersion": 1,
  "catalogVersion": "0.8.0",
  "digest": "sha256:…",
  "entries": [{
    "id": "block:data-chart",
    "name": "data-chart",
    "type": "block",
    "title": "Data chart",
    "description": "…",
    "tags": ["data", "chart"],
    "aliases": [],
    "searchText": "…",
    "minCliVersion": "0.8.0",
    "deprecated": null,
    "replacementId": null
  }]
}

QUERY /v3/hyperframes/catalog
Authorization: Bearer …
X-HeyGen-Client-Source: hyperframes
Content-Type: application/json

{
  "query": "animated pricing comparison",
  "catalogDigest": "sha256:…"
}

Required response fields

Code-complete is not production-ready

Capacity equations

The plan deliberately does not invent a demand forecast. Product and API owners must record expected steady QPS, burst QPS, daily active searchers and queries per searcher before capacity approval.

Activation gates

How the system behaves when reality is messy

stateDiagram-v2
  [*] --> LocalReady: client starts
  LocalReady --> SmartPending: user explicitly enables Smart
  SmartPending --> SmartResults: compatible success
  SmartPending --> LocalReady: any HTTP error or timeout
  SmartPending --> LocalReady: catalog digest mismatch
  SmartResults --> LocalReady: Smart toggled off
  SmartResults --> LocalReady: Smart disabled
  LocalReady --> PriorArtifact: client/artifact rollback
  PriorArtifact --> LocalReady: prior release starts

Eight units across two trains

flowchart LR
  U0[U0 HTML source of truth] --> U1[U1 artifact + integrity]
  U1 --> U2[U2 local scorer]
  U2 --> U3[U3 local CLI / Studio / SDK]
  U3 --> U4[U4 media-use adapter]
  U3 --> T1{"TRAIN 01 DONE"}
  U4 --> T1
  T1 --> U5[U5 immutable Smart index]
  U5 --> U6[U6 QUERY API + clients]
  U6 --> U7[U7 opt-in beta operations]
  U7 --> T2{"TRAIN 02 READY"}

Definition of done

• Every public item passes the quality bar, resolves against the exact client digest and passes compatibility.
• Local search works without sign-in or network and remains visible during every remote failure.
• Smart search charges zero credits, requires authentication and observes approved limits.
• Adding/removing/renaming or quality-cutting an item produces a deterministic new digest and cannot corrupt old clients.
• CLI, Studio, SDK and media-use share identity, result schema and installer ownership.
• Contract, relevance smoke, load, privacy, abuse and observability checks pass before the opt-in beta.

Terms used in this plan

Release train

An independently shippable stage with its own user value, exit checks and rollback. Train 01 remains complete without Train 02.

Catalog

The list of Hyperframes things a user can discover and use: blocks, components and any primitives that have a real install/open contract.

Artifact

A generated JSON file shipped with a release. It is the search-ready view of the source manifests, not a second hand-edited catalog.

Canonical ID

A stable identity such as block:data-chart. Display names may change; the identity must not silently point to different code.

Digest

A SHA-256 fingerprint of canonical artifact bytes. Matching digests means client and server are talking about the exact same catalog.

Local / lexical

Ranking by exact names, tags and words on the user’s machine. It needs no embedding service and cannot have stale remote vectors.

Semantic

Ranking by meaning rather than shared spelling. “Make it feel faster” may find “speed ramp” even when those words differ.

Embedding / vector

A model turns text into a fixed-length list of numbers. Nearby lists are treated as similar meanings.

Pinecone

The hosted database that stores vectors and metadata and returns nearby entries. It is infrastructure, not the thing that makes an embedding intelligent.

BM25

A standard keyword-ranking method that rewards useful shared terms and discounts very common terms.

Hybrid / RRF

Hybrid combines word and meaning rankings. Reciprocal Rank Fusion (RRF) combines their positions instead of incompatible raw scores.

Top-k

The number of results returned or shown. “Top 20” means k=20.

Namespace

An isolated partition inside the vector database. Here it is pinned to environment and catalog digest so releases cannot mix.

OAuth

The browser sign-in flow that gives the CLI a revocable HeyGen access token without asking the user to paste an API key.

Rate limit

A bound on requests in a time window. The endpoint is free of credits, but still protected from accidental or abusive demand.

p95 / p99

The latency below which 95% / 99% of requests finish. They expose the slow tail that an average hides.

SLO

A measurable service objective for latency, availability or correctness that operators agree to meet.

Backpressure

Refusing or queuing bounded work when downstream capacity is full instead of creating unlimited concurrent calls.

Circuit breaker

A temporary stop on remote calls after repeated failures, giving the dependency time to recover while users get local results.

Rollback

Return to the prior client/index/strategy. Immutable digests make that a pointer/config change rather than a data reconstruction.

Deferred / Open Questions

From 2026-08-04 review