Add tags_all / tags_any / tags_none tag filters to the assets list API (#15332)

* Implement tags_all/tags_any/tags_none on the assets list API (BE-6600)

Adds the three canonically-named tag filter params to GET /api/assets and
GET /api/assets/tags/refine:

- tags_all: asset carries every tag (replaces include_tags)
- tags_any: asset carries at least one tag (new)
- tags_none: asset carries no tag (replaces exclude_tags)

Clauses intersect; tags_none always wins. include_tags/exclude_tags remain
as permanent deprecated aliases and behave exactly as before when used on
their own.

Invalid combinations return 400 INVALID_TAG_FILTER, but only when the
request uses at least one new-name parameter (non-empty after
normalisation):

- mixed spellings of one slot (include_tags with tags_all, exclude_tags
  with tags_none)
- the same tag in the effective all-list and none-list (query can never
  match)

Old-names-only requests gain no new error paths: include_tags=a&exclude_tags=a
still returns an empty 200. tags_any/tags_none overlap stays valid (dead
term, not a dead query).

* Address review findings: positional-compat, deprecation metadata, test matrix

- Move any_tags to the end of the four touched signatures: inserting it
  mid-signature silently misbound pre-existing positional callers (e.g.
  a caller passing name_contains positionally would have it consumed as
  any_tags).
- Mark include_tags/exclude_tags Field(deprecated=True) on both list
  schemas so generated schema metadata matches the contract, not just a
  comment (schemas_out.py already uses this form for Asset.name).
- Add tests: legal cross-slot old/new combinations, repeated query-key
  concatenation (pins Core behavior; outside the cross-platform
  contract), tags_any two-page cursor consistency (total/has_more/
  no-overlap), refine-route mixed-spelling rejection + legacy-conflict
  preservation, and schema deprecation metadata.

* Pin tag-value opacity: case-sensitive matching, byte-exact conflict check

The prod tag survey (~/comfy/prod-model-tag-shape.md) found live
case-distinct tag pairs (SEEDVR2/seedvr2) that resolve differently, so
the contract now states tag values are opaque byte-strings. Pin that:
case-distinct tags filter separately, and a case-distinct all/none pair
is not an INVALID_TAG_FILTER conflict.

* Document tags_all/tags_any/tags_none in openapi.yaml, deprecate aliases

Add the three tag-filter parameters to both listAssets and
getAssetTagHistogram parameter blocks and mark include_tags/exclude_tags
deprecated: true, keeping the spec in step with the runtime schemas so
generated clients can discover the new filters while the aliases stay
present for existing consumers.

* Move schemas_in import to module scope in test_list_filter

Review feedback: no import cycle requires the local import.

* Silence per-request DeprecationWarning in the tag-filter remap shim

Reading the deprecated include_tags/exclude_tags fields by attribute
fires pydantic's DeprecationWarning on every list/refine request even
for callers using only the new names. The warning is aimed at API
clients, not the server's own remap; read via model_dump instead.

* Cap tag-filter lists at 100 entries, all spellings

Review finding: unbounded tag lists fan out into one correlated EXISTS
per tag on both page and count statements. Cap each list at 100
normalized entries with 400 INVALID_TAG_FILTER naming the parameter.

Applies to the legacy spellings as well — a deliberate, decided
exception to the old-names-behave-identically rule, since a cap only on
new names would leave the same fan-out reachable through the aliases.

* Strip process narration from comments

Comments carried decision dates, contract cross-references, and review
context. Keep only the constraints the code cannot show, one line each.
This commit is contained in:
Simon Pinfold
2026-08-10 14:05:21 -07:00
committed by GitHub
parent 7d11ec31cb
commit 34744cd29e
9 changed files with 624 additions and 19 deletions

View File

@@ -18,7 +18,7 @@ from app.assets.api.schemas_in import (
AssetValidationError,
UploadError,
)
from app.assets.helpers import validate_blake3_hash
from app.assets.helpers import normalize_tags, validate_blake3_hash
from app.assets.api.upload import (
delete_temp_file_if_exists,
parse_multipart_upload,
@@ -117,6 +117,87 @@ def _build_validation_error_response(code: str, ve: ValidationError) -> web.Resp
return _build_error_response(400, code, "Validation failed.", {"errors": errors})
class InvalidTagFilterError(Exception):
"""Invalid combination of tag-filter query parameters."""
def __init__(self, message: str, details: dict):
super().__init__(message)
self.details = details
# Caps the per-tag EXISTS fan-out; deliberately covers the legacy spellings too.
MAX_TAG_FILTER_TAGS = 100
def _resolve_tag_filters(
q: schemas_in.ListAssetsQuery | schemas_in.TagsRefineQuery,
) -> tuple[list[str], list[str], list[str]]:
"""Resolve legacy (include/exclude) and new (all/any/none) tag-filter
spellings into effective (all, any, none) lists.
Combination validation applies only when the request uses at least one
new-name parameter (non-empty after normalisation); requests using only
the legacy names keep their historical behaviour, including degenerate
combinations like include_tags=a&exclude_tags=a.
"""
# model_dump, not attribute access: deprecated fields warn on every attribute read.
legacy = q.model_dump(include={"include_tags", "exclude_tags"})
include_tags = normalize_tags(legacy["include_tags"])
exclude_tags = normalize_tags(legacy["exclude_tags"])
tags_all = normalize_tags(q.tags_all)
tags_any = normalize_tags(q.tags_any)
tags_none = normalize_tags(q.tags_none)
for param_name, values in (
("include_tags", include_tags),
("exclude_tags", exclude_tags),
("tags_all", tags_all),
("tags_any", tags_any),
("tags_none", tags_none),
):
if len(values) > MAX_TAG_FILTER_TAGS:
raise InvalidTagFilterError(
f"'{param_name}' lists {len(values)} tags; the maximum is "
f"{MAX_TAG_FILTER_TAGS}.",
{
"parameter": param_name,
"count": len(values),
"max": MAX_TAG_FILTER_TAGS,
},
)
if not (tags_all or tags_any or tags_none):
return include_tags, [], exclude_tags
if include_tags and tags_all:
raise InvalidTagFilterError(
"Cannot combine 'include_tags' with 'tags_all'; use 'tags_all'.",
{"parameters": ["include_tags", "tags_all"]},
)
if exclude_tags and tags_none:
raise InvalidTagFilterError(
"Cannot combine 'exclude_tags' with 'tags_none'; use 'tags_none'.",
{"parameters": ["exclude_tags", "tags_none"]},
)
all_param, all_list = (
("tags_all", tags_all) if tags_all else ("include_tags", include_tags)
)
none_param, none_list = (
("tags_none", tags_none) if tags_none else ("exclude_tags", exclude_tags)
)
conflicting = sorted(set(all_list) & set(none_list))
if conflicting:
raise InvalidTagFilterError(
f"Query can never match: {', '.join(repr(t) for t in conflicting)} "
f"required by '{all_param}' but rejected by '{none_param}'.",
{"conflicting_tags": conflicting, "parameters": [all_param, none_param]},
)
return all_list, tags_any, none_list
def _validate_sort_field(requested: str | None) -> str:
if not requested:
return "created_at"
@@ -217,6 +298,11 @@ async def list_assets_route(request: web.Request) -> web.Response:
except ValidationError as ve:
return _build_validation_error_response("INVALID_QUERY", ve)
try:
tags_all, tags_any, tags_none = _resolve_tag_filters(q)
except InvalidTagFilterError as e:
return _build_error_response(400, "INVALID_TAG_FILTER", str(e), e.details)
sort = _validate_sort_field(q.sort)
order_candidate = (q.order or "desc").lower()
order = order_candidate if order_candidate in {"asc", "desc"} else "desc"
@@ -224,8 +310,9 @@ async def list_assets_route(request: web.Request) -> web.Response:
try:
result = list_assets_page(
owner_id=USER_MANAGER.get_request_user_id(request),
include_tags=q.include_tags,
exclude_tags=q.exclude_tags,
include_tags=tags_all,
exclude_tags=tags_none,
any_tags=tags_any,
name_contains=q.name_contains,
metadata_filter=q.metadata_filter,
limit=q.limit,
@@ -715,10 +802,16 @@ async def get_tags_refine(request: web.Request) -> web.Response:
except ValidationError as ve:
return _build_validation_error_response("INVALID_QUERY", ve)
try:
tags_all, tags_any, tags_none = _resolve_tag_filters(q)
except InvalidTagFilterError as e:
return _build_error_response(400, "INVALID_TAG_FILTER", str(e), e.details)
tag_counts = list_tag_histogram(
owner_id=USER_MANAGER.get_request_user_id(request),
include_tags=q.include_tags,
exclude_tags=q.exclude_tags,
include_tags=tags_all,
exclude_tags=tags_none,
any_tags=tags_any,
name_contains=q.name_contains,
metadata_filter=q.metadata_filter,
limit=q.limit,

View File

@@ -50,8 +50,12 @@ class ParsedUpload:
class ListAssetsQuery(BaseModel):
include_tags: list[str] = Field(default_factory=list)
exclude_tags: list[str] = Field(default_factory=list)
# Deprecated spellings: include_tags ≡ tags_all, exclude_tags ≡ tags_none.
include_tags: list[str] = Field(default_factory=list, deprecated=True)
exclude_tags: list[str] = Field(default_factory=list, deprecated=True)
tags_all: list[str] = Field(default_factory=list)
tags_any: list[str] = Field(default_factory=list)
tags_none: list[str] = Field(default_factory=list)
name_contains: str | None = None
# Accept either a JSON string (query param) or a dict
@@ -70,7 +74,10 @@ class ListAssetsQuery(BaseModel):
)
order: Literal["asc", "desc"] = "desc"
@field_validator("include_tags", "exclude_tags", mode="before")
@field_validator(
"include_tags", "exclude_tags", "tags_all", "tags_any", "tags_none",
mode="before",
)
@classmethod
def _split_csv_tags(cls, v):
# Accept "a,b,c" or ["a","b"] (we are liberal in what we accept)
@@ -154,13 +161,20 @@ class CreateFromHashBody(BaseModel):
class TagsRefineQuery(BaseModel):
include_tags: list[str] = Field(default_factory=list)
exclude_tags: list[str] = Field(default_factory=list)
# Deprecated spellings: include_tags ≡ tags_all, exclude_tags ≡ tags_none.
include_tags: list[str] = Field(default_factory=list, deprecated=True)
exclude_tags: list[str] = Field(default_factory=list, deprecated=True)
tags_all: list[str] = Field(default_factory=list)
tags_any: list[str] = Field(default_factory=list)
tags_none: list[str] = Field(default_factory=list)
name_contains: str | None = None
metadata_filter: dict[str, Any] | None = None
limit: conint(ge=1, le=1000) = 100
@field_validator("include_tags", "exclude_tags", mode="before")
@field_validator(
"include_tags", "exclude_tags", "tags_all", "tags_any", "tags_none",
mode="before",
)
@classmethod
def _split_csv_tags(cls, v):
if v is None: