Fix: Chat completion generation parameter overrides (#15389)

### What problem does this PR solve?

Closes #15388.

Chat completion routes did not reliably honor per-request generation
settings:

- `/api/v1/chat/completions` copied generation settings with a
truthiness check, so valid zero values such as `temperature: 0`, `top_p:
0`, `frequency_penalty: 0`, `presence_penalty: 0`, and `max_tokens: 0`
were dropped.
- `/api/v1/openai/{chat_id}/chat/completions` did not forward standard
generation settings into the request-specific dialog LLM settings before
calling `async_chat`.

This PR preserves explicitly supplied generation parameters, including
zero values, and merges request-level overrides into existing dialog
settings where appropriate.

The supported generation parameter keys and merge behavior live in a
shared REST API helper to keep both completion routes aligned.

Validation:

- `git diff --check`
- `python3 -m py_compile api/apps/restful_apis/_generation_params.py
api/apps/restful_apis/chat_api.py api/apps/restful_apis/openai_api.py
test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py`
- `uv run ruff check api/apps/restful_apis/_generation_params.py
api/apps/restful_apis/chat_api.py api/apps/restful_apis/openai_api.py
test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py`
- `ZHIPU_AI_API_KEY=dummy uv run pytest
test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py
-q -k generation_params`

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
This commit is contained in:
bitloi
2026-06-03 00:46:10 -03:00
committed by GitHub
parent 76968af0ba
commit a75ea7ba7c
4 changed files with 226 additions and 7 deletions

View File

@@ -20,6 +20,7 @@ import time
from quart import Response, jsonify
from api.apps import current_user, login_required
from api.apps.restful_apis._generation_params import extract_generation_config, merge_generation_config
from api.db.services.dialog_service import DialogService, async_chat
from api.db.services.doc_metadata_service import DocMetadataService
from api.db.joint_services.tenant_model_service import get_model_config_from_provider_instance, get_api_key
@@ -29,6 +30,7 @@ from common.metadata_utils import convert_conditions, meta_filter
from common.token_utils import num_tokens_from_string
from rag.prompts.generator import chunks_format
def _validate_llm_id(llm_id, tenant_id, llm_setting=None):
if not llm_id:
return None
@@ -278,6 +280,7 @@ async def openai_chat_completions(chat_id):
dia.llm_id = requested_model
if not get_api_key(tenant_id=dia.tenant_id, model_name=requested_model):
return get_error_data_result(message=f"Cannot use specified model {requested_model}.")
merge_generation_config(dia, extract_generation_config(req))
metadata_condition = extra_body.get("metadata_condition") or {}
if metadata_condition and not isinstance(metadata_condition, dict):