feat(connectors): ETag-based bypass for incremental S3 ingestion (#14628) (#14677)

### What problem does this PR solve?

S3-family connector syncs currently re-download every in-window object
just so we can compute `xxhash128(blob)` and compare against
`Document.content_hash`. Anything that bumps `LastModified` without
changing bytes (`aws s3 cp` touches, bucket re-encryption, etc.) pays
full bandwidth and re-parses files that didn't actually change. #14628
covers the broader incremental-ingestion redesign; this PR is the first
slice.

The fix is a pre-listing short-circuit. `BlobStorageConnector` (S3 / R2
/ GCS / OCI / S3-compat) now implements a new `FingerprintConnector`
interface: `list_keys()` paginates `list_objects_v2` and yields
`KeyRecord(key, fingerprint)` where `fingerprint = xxhash128(ETag)`. The
orchestrator joins those against the connector's existing `{doc_id:
content_hash}` map and only calls `get_value(key)` when the fingerprint
differs. Unchanged keys are skipped entirely — no `GetObject`, no
re-parse.

No DDL. xxhash128(ETag) is 32 hex chars and reuses the existing
`Document.content_hash` column per @yingfeng's suggestion; the connector
decides at listing time whether to populate it. Local uploads and
connectors that don't opt in fall through to the existing post-download
`xxhash128(blob)` path with no behavior change.

This is PR-1 of a 4-PR series — full design lives on #14628. Subsequent
PRs extend tier 1 to local FS / WebDAV / Dropbox / Seafile / RDBMS
(PR-2), wire up tier 2 cursor connectors with `SyncLogs.next_checkpoint`
(PR-3), and unify deletion via `KeyRecord(deleted=True)` reconciliation
(PR-4). Holding those back keeps this PR additive and reviewable on its
own.

#### Files touched

- `common/data_source/models.py` — new `KeyRecord`; optional
`fingerprint` on `Document`
- `common/data_source/interfaces.py` — `IncrementalCapability` enum,
`FingerprintConnector` ABC
- `common/data_source/blob_connector.py` — `BlobStorageConnector`
implements `FingerprintConnector`; per-object download factored into
`_build_document_from_obj()` so `_yield_blob_objects`, `list_keys`,
`get_value` all share it
- `rag/svr/sync_data_source.py` —
`_BlobLikeBase._fingerprint_filtered_generator` does the bypass loop;
`_run_task_logic` plumbs `doc.fingerprint` into the upload dict
- `api/db/services/document_service.py` —
`list_id_content_hash_map_by_kb_and_source_type()` helper
- `api/db/services/connector_service.py` + `file_service.py` —
fingerprint flows through `duplicate_and_parse → upload_document` and
lands in `content_hash`
- `test/unit_test/common/test_blob_connector_fingerprint.py` — 14 tests
covering ETag normalization (single-part, multipart, quoted, empty),
`list_keys()` not calling `GetObject`, `get_value()` materializing with
fingerprint, deterministic/stable fingerprints, and the bypass loop
asserting `GetObject` is *not* called on a match

#### Worth flagging for review

Old `_BlobLikeBase._generate` called `poll_source(start, now)` with a
`LastModified` window when `poll_range_start` was set. New code uses
`_fingerprint_filtered_generator` (full bucket listing + fingerprint
compare) outside of explicit `reindex=1`. Strictly better for
unchanged-bucket cases since it skips `GetObject`, but it does mean
every sync now does a full `list_objects_v2` paginate. Should still be
cheap for most buckets — flagging in case anyone has a very large bucket
where the time-window filter was meaningful.

On migration: existing rows have `content_hash = xxhash128(blob)` from
the old code. The first sync after this lands sees ETag-derived
fingerprints that don't match, re-fetches every object once, and writes
the new fingerprint. From the second sync onward the bypass works as
expected. "Slow day one, fast every day after." A `fingerprint_backfill:
trust` opt-out is sketched in the design doc but not in this PR.

#### Test plan

- [x] `uv run ruff check` — clean on all 8 touched files
- [x] `uv run pytest
test/unit_test/common/test_blob_connector_fingerprint.py -v` — 14 passed
- [x] Broader unit-test suite — no regressions in anything I touched
- [ ] Manual smoke against a real S3 bucket — configure a connector, run
sync twice, expect the second sync to log `bypassed=N, fetched=0` and no
`GetObject` calls in CloudTrail / bucket access logs
- [ ] Manual smoke with `reindex=1` — confirm the full re-download path
still works

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>

common/data_source/interfaces.py

@@ -2,7 +2,7 @@
 import abc
 import uuid
 from abc import ABC, abstractmethod
-from enum import IntFlag, auto
+from enum import IntEnum, IntFlag, auto
 from types import TracebackType
 from typing import Any, Dict, Generator, TypeVar, Generic, Callable, TypeAlias
 from collections.abc import Iterator
@@ -10,12 +10,26 @@ from anthropic import BaseModel
 
 from common.data_source.models import (
     Document,
+    KeyRecord,
     SlimDocument,
     ConnectorCheckpoint,
     ConnectorFailure,
     SecondsSinceUnixEpoch, GenerateSlimDocumentOutput
 )
 
+
+class IncrementalCapability(IntEnum):
+    """How a connector handles incremental sync.
+
+    FULL_RESYNC  -- every sync re-pulls; no per-key state.
+    CURSOR       -- "give me everything since cursor X"; opaque cursor persisted across syncs.
+    FINGERPRINT  -- list_keys() returns (key, fingerprint) cheaply; bodies fetched lazily.
+    """
+    FULL_RESYNC = 0
+    CURSOR = 1
+    FINGERPRINT = 2
+
+
 GenerateDocumentsOutput = Iterator[list[Document]]
 
 class LoadConnector(ABC):
@@ -415,3 +429,39 @@ class IndexingHeartbeatInterface(ABC):
         just to act as a keep-alive.
         """
 
+
+class FingerprintConnector(ABC):
+    """Tier 1 connector: cheap full listing with per-key fingerprint.
+
+    Sources that can enumerate their entire keyspace via a metadata-only call
+    (e.g. S3 list_objects_v2 returning ETag + LastModified) implement this to
+    let the orchestrator skip GetObject for keys whose fingerprint hasn't
+    changed since the last sync.
+
+    The fingerprint is an opaque equality token: two equal fingerprints mean
+    the content is unchanged from the orchestrator's point of view. Format is
+    a 32-char hex string so it fits the existing Document.content_hash column;
+    connectors are responsible for normalizing whatever the source exposes
+    (typically by hashing it with xxhash128).
+    """
+
+    INCREMENTAL_CAPABILITY: IncrementalCapability = IncrementalCapability.FINGERPRINT
+
+    @abstractmethod
+    def list_keys(self) -> Iterator[KeyRecord]:
+        """Yield one KeyRecord per object currently in the source.
+
+        Must enumerate the full current keyspace -- the orchestrator diffs the
+        result against persisted state to detect adds, updates, and deletes.
+        """
+        raise NotImplementedError
+
+    @abstractmethod
+    def get_value(self, key: str) -> Document:
+        """Fetch the body for a single key, returning a fully populated Document.
+
+        Called only when list_keys()'s fingerprint differs from the persisted
+        content_hash for that key (or when no persisted fingerprint exists).
+        """
+        raise NotImplementedError
+