mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-06-29 23:41:12 +08:00
93f3b90121
### What problem does this PR solve? Implement the vLLM model provider for RAGFlow to fully support local and self-hosted open-source models (e.g., Qwen, GLM, Llama) via the vLLM framework, and fix several critical bugs related to model instance management and API requests. **Key changes and fixes:** 1. **Added Standard vLLM Provider (`vllm.go`, `vllm.json`):** - Implemented `VllmModel` driver strictly adhering to the OpenAI API specification. - Removed hardcoded and dangerous routing logic (e.g., forcing `AsyncChat` for Qwen/GLM prefixes), ensuring standard `/v1/chat/completions` compatibility. - Refactored `ListModels` to use safe JSON parsing (resolving nil pointer panics) and standard `GET` requests without bodies. - Added `APIConfig.Region` fallback logic to prevent empty `base_url` fetching when checking models. 2. **Fixed `ChatToModelStreamWithSender` Bug (`model_service.go`):** - Resolved the `model is disabled` error when streaming chat with local database-saved models. - Added the missing `if modelInfo.Status == "active"` block to correctly invoke `NewInstance` and inject the dynamic `base_url` into the provider driver before starting the SSE stream. 3. **Fixed `ListSupportedModels` Bug (`model_service.go`):** - Added dynamic `NewInstance` injection for `base_url`. Previously, the list models function used the static JSON config without injecting user-configured dynamic URLs from the database, resulting in an `unsupported protocol scheme ""` error. ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) - [x] New Feature (non-breaking change which adds functionality)