Commit Graph

3 Commits

Author SHA1 Message Date
jay77721
fe02c9b95f feat(go-models): unify OpenAI-compatible token usage extraction (#17634)
## Summary
Relate to #17284 .

- Add `usage_parser.go`: `extractOpenAIUsage` /
`extractOpenAIStreamUsage`
  with multi-field fallback (`prompt_tokens`/`input_tokens`,
  `completion_tokens`/`output_tokens`).
- Add `parser_config.go`: `ParserConfig` struct and
`OpenAIParserConfig`.
- Add `response_handler.go`: `HandleNonStreamingResponse` /
  `HandleStreamingResponse` as single entry points that extract usage,
  populate `chatConfig.UsageResult`, and emit a `StreamUsage` log.
- Extend `base_model.go` with `doRequest` / `doStreamRequest`.
- Migrate `deepseek` driver to shared handlers, cutting ~100 lines.
- Add `usage_parser_test.go`.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Haruko386 <tryeverypossible@163.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-07-31 18:32:07 +08:00
oktofeesh
6fc3955cab fix(go-models): normalize Qwen reasoning families (#15735)
## Summary

Normalizes Qwen model-family names before reasoning extraction so
provider-prefixed Qwen models use the existing `<think>...</think>`
fallback.
2026-06-08 19:32:19 +08:00
bitloi
220ee9dbfb fix: normalize reasoning model families (#15612)
### What problem does this PR solve?

Closes #15611.

RAGFlow's fallback reasoning parser only recognized the exact model
family `qwen3`. For provider-prefixed Qwen model names such as
SiliconFlow's `qwen/qwen3-8b`, the derived model class can be
`qwen/qwen3`, so inline `<think>...</think>` content was not split from
the visible answer when `reasoning_content` was absent.

This PR normalizes model-family detection before fallback reasoning
extraction, keeps the parser nil-safe, and adds focused tests for Qwen3
variants plus Gitee and SiliconFlow chat responses.

It also makes SiliconFlow propagate `ChatConfig.Thinking` into the chat
request body, matching the existing Gitee behavior, so Qwen thinking
mode is actually enabled when requested.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

### Validation

- `/root/go/bin/gofmt -l internal/entity/models/common.go
internal/entity/models/common_test.go
internal/entity/models/reasoning_family_provider_test.go
internal/entity/models/siliconflow.go`
- `git diff --check`
- `/root/go/bin/go test ./internal/entity/models -run
'Test(NormalizeModelFamily|GetThinkingAndAnswer|GiteeChatExtractsQwenThinkingFromInlineContent|SiliconflowChatExtractsProviderPrefixedQwenThinkingFromInlineContent)'
-vet=off -count=1`

Note: the full package command `/root/go/bin/go test
./internal/entity/models -vet=off -count=1` now runs locally, but it
currently fails on an unrelated existing
`TestAstraflowEmbedReturnsNoSuchMethod` panic in
`internal/entity/models/astraflow.go:482`.
2026-06-08 13:32:52 +08:00