2024-11-13 15:56:40 +08:00
ragflow :
host : 0.0 .0 .0
http_port : 9380
Feat: add admin CLI and admin service (#10186)
### What problem does this PR solve?
Introduce new feature: RAGFlow system admin service and CLI
### Introduction
Admin Service is a dedicated management component designed to monitor,
maintain, and administrate the RAGFlow system. It provides comprehensive
tools for ensuring system stability, performing operational tasks, and
managing users and permissions efficiently.
The service offers monitoring of critical components, including the
RAGFlow server, Task Executor processes, and dependent services such as
MySQL, Infinity / Elasticsearch, Redis, and MinIO. It automatically
checks their health status, resource usage, and uptime, and performs
restarts in case of failures to minimize downtime.
For user and system management, it supports listing, creating,
modifying, and deleting users and their associated resources like
knowledge bases and Agents.
Built with scalability and reliability in mind, the Admin Service
ensures smooth system operation and simplifies maintenance workflows.
It consists of a server-side Service and a command-line client (CLI),
both implemented in Python. User commands are parsed using the Lark
parsing toolkit.
- **Admin Service**: A backend service that interfaces with the RAGFlow
system to execute administrative operations and monitor its status.
- **Admin CLI**: A command-line interface that allows users to connect
to the Admin Service and issue commands for system management.
### Starting the Admin Service
1. Before start Admin Service, please make sure RAGFlow system is
already started.
2. Run the service script:
```bash
python admin/admin_server.py
```
The service will start and listen for incoming connections from the CLI
on the configured port.
### Using the Admin CLI
1. Ensure the Admin Service is running.
2. Launch the CLI client:
```bash
python admin/admin_client.py -h 0.0.0.0 -p 9381
## Supported Commands
Commands are case-insensitive and must be terminated with a semicolon
(`;`).
### Service Management Commands
- [x] `LIST SERVICES;`
- Lists all available services within the RAGFlow system.
- [ ] `SHOW SERVICE <id>;`
- Shows detailed status information for the service identified by
`<id>`.
- [ ] `STARTUP SERVICE <id>;`
- Attempts to start the service identified by `<id>`.
- [ ] `SHUTDOWN SERVICE <id>;`
- Attempts to gracefully shut down the service identified by `<id>`.
- [ ] `RESTART SERVICE <id>;`
- Attempts to restart the service identified by `<id>`.
### User Management Commands
- [x] `LIST USERS;`
- Lists all users known to the system.
- [ ] `SHOW USER '<username>';`
- Shows details and permissions for the specified user. The username
must be enclosed in single or double quotes.
- [ ] `DROP USER '<username>';`
- Removes the specified user from the system. Use with caution.
- [ ] `ALTER USER PASSWORD '<username>' '<new_password>';`
- Changes the password for the specified user.
### Data and Agent Commands
- [ ] `LIST DATASETS OF '<username>';`
- Lists the datasets associated with the specified user.
- [ ] `LIST AGENTS OF '<username>';`
- Lists the agents associated with the specified user.
### Meta-Commands
Meta-commands are prefixed with a backslash (`\`).
- `\?` or `\help`
- Shows help information for the available commands.
- `\q` or `\quit`
- Exits the CLI application.
## Examples
```commandline
admin> list users;
+-------------------------------+------------------------+-----------+-------------+
| create_date | email | is_active | nickname |
+-------------------------------+------------------------+-----------+-------------+
| Fri, 22 Nov 2024 16:03:41 GMT | jeffery@infiniflow.org | 1 | Jeffery |
| Fri, 22 Nov 2024 16:10:55 GMT | aya@infiniflow.org | 1 | Waterdancer |
+-------------------------------+------------------------+-----------+-------------+
admin> list services;
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
| extra | host | id | name | port | service_type |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
| {} | 0.0.0.0 | 0 | ragflow_0 | 9380 | ragflow_server |
| {'meta_type': 'mysql', 'password': 'infini_rag_flow', 'username': 'root'} | localhost | 1 | mysql | 5455 | meta_data |
| {'password': 'infini_rag_flow', 'store_type': 'minio', 'user': 'rag_flow'} | localhost | 2 | minio | 9000 | file_store |
| {'password': 'infini_rag_flow', 'retrieval_type': 'elasticsearch', 'username': 'elastic'} | localhost | 3 | elasticsearch | 1200 | retrieval |
| {'db_name': 'default_db', 'retrieval_type': 'infinity'} | localhost | 4 | infinity | 23817 | retrieval |
| {'database': 1, 'mq_type': 'redis', 'password': 'infini_rag_flow'} | localhost | 5 | redis | 6379 | message_queue |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Signed-off-by: jinhai <haijin.chn@gmail.com>
2025-09-22 10:37:49 +08:00
admin :
host : 0.0 .0 .0
http_port : 9381
2024-11-13 15:56:40 +08:00
mysql :
name : 'rag_flow'
user : 'root'
password : 'infini_rag_flow'
2025-02-28 14:28:00 +08:00
host : 'localhost'
2026-03-24 20:08:36 +08:00
port : 3306
2025-05-09 17:52:03 +08:00
max_connections : 900
stale_timeout : 300
2025-06-12 11:37:42 +08:00
max_allowed_packet : 1073741824
2024-11-13 15:56:40 +08:00
minio :
user : 'rag_flow'
password : 'infini_rag_flow'
2025-02-28 14:28:00 +08:00
host : 'localhost:9000'
2025-12-11 12:22:47 +01:00
bucket : ''
prefix_path : ''
2024-11-13 15:56:40 +08:00
es :
2025-02-28 14:28:00 +08:00
hosts : 'http://localhost:1200'
2024-11-13 15:56:40 +08:00
username : 'elastic'
password : 'infini_rag_flow'
2025-04-24 16:03:31 +08:00
os :
hosts : 'http://localhost:1201'
username : 'admin'
password : 'infini_rag_flow_OS_01'
fix(opensearch): keep the BM25 leg in hybrid search (#15760)
### What problem does this PR solve?
Fixes the OpenSearch side of #10747: hybrid search drops the keyword
(BM25) leg and
ends up doing plain vector search.
When a search has both a text and a vector leg, `OSConnection.search()`
throws the text
query away:
del q["query"]
q["query"] = {"knn": knn_query}
The text clause only stays on as a filter inside the knn query, so it
narrows the
candidate set but doesn't count towards scoring. So hybrid search on
OpenSearch behaves
like plain vector search, unlike the Elasticsearch backend.
What I changed:
- when both legs are present, send a real hybrid query
`{"hybrid": {"queries": [bm25, {"knn": ...}]}}` and let a
normalization-processor
search pipeline score and combine the two legs
- only the actual filters (kb_id, available_int, ...) go in the knn
filter, not the
text must clause
- create the pipeline on startup if it's missing, so there's no separate
provisioning
step. name and weights can be set under `os:` in service_conf.yaml, or
via
`OS_HYBRID_PIPELINE`; defaults are `ragflow_hybrid_pipeline` and `[0.5,
0.5]`
- normalization-processor needs OpenSearch 2.10+. on older clusters, or
when the
pipeline can't be created, log a warning and fall back to vector-only
instead of
pointing at a pipeline that doesn't exist
This is only the hybrid-search fix; `create_doc_meta_idx` is already on
main.
Testing (there's no OpenSearch path in CI): added a unit test
(`test/unit_test/rag/utils/test_opensearch_hybrid_search.py`, no
services needed) that
checks the query built in each case — hybrid + pipeline param for
text+vector, plain knn
for vector-only, plain bool for text-only, the knn filter never carrying
the text
query_string, and the vector-only fallback when the pipeline isn't
available. Also ran
it against a real OpenSearch 2.19.1 container with a doc that matches
the keyword but
sits outside the knn top-k: pure knn returns `['D1','D2','D5']` (keyword
doc missing),
the hybrid query returns `['A','D1','D2','D5']` (keyword doc present).
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Signed-off-by: Danut Matei <matei.danut.dm@gmail.com>
2026-06-08 11:17:47 +03:00
# Optional hybrid (BM25 + KNN) search tuning. The connector self-provisions the
# normalization search pipeline on start-up (requires OpenSearch >= 2.10).
# hybrid_search_pipeline: 'ragflow_hybrid_pipeline'
# hybrid_search_weights: [0.5, 0.5] # [text/BM25 leg, vector/KNN leg]
2024-11-25 11:53:58 +08:00
infinity :
2025-02-28 14:28:00 +08:00
uri : 'localhost:23817'
2026-01-19 19:35:14 +08:00
postgres_port : 5432
2024-11-25 11:53:58 +08:00
db_name : 'default_db'
2025-11-20 10:00:14 +08:00
oceanbase :
scheme : 'oceanbase' # set 'mysql' to create connection using mysql config
config :
db_name : 'test'
user : 'root@ragflow'
password : 'infini_rag_flow'
host : 'localhost'
port : 2881
2024-11-13 15:56:40 +08:00
redis :
db : 1
2025-12-01 11:26:20 +08:00
username : ''
2025-01-06 11:06:24 +07:00
password : 'infini_rag_flow'
2026-01-28 09:27:47 +08:00
host : 'localhost:6379'
2026-06-12 14:56:44 +08:00
nats :
host : "0.0.0.0"
port : 4222
2025-11-10 12:51:39 +08:00
task_executor :
2026-06-12 14:56:44 +08:00
message_queue_type : 'nats'
2025-10-23 23:02:27 +08:00
user_default_llm :
default_models :
embedding_model :
2026-03-18 11:51:03 +08:00
name : 'bge-m3'
factory : 'xxxx'
2025-10-23 23:02:27 +08:00
api_key : 'xxx'
base_url : 'http://localhost:6380'
2024-11-13 15:56:40 +08:00
# postgres:
# name: 'rag_flow'
# user: 'rag_flow'
# password: 'infini_rag_flow'
# host: 'postgres'
# port: 5432
# max_connections: 100
# stale_timeout: 30
# s3:
# access_key: 'access_key'
# secret_key: 'secret_key'
# region: 'region'
2025-12-04 09:44:05 +07:00
#gcs:
# bucket: 'bridgtl-edm-d-bucket-ragflow'
2025-02-27 17:02:42 +08:00
# oss:
# access_key: 'access_key'
# secret_key: 'secret_key'
2026-01-22 11:43:55 +08:00
# endpoint_url: 'https://s3.oss-cn-hangzhou.aliyuncs.com'
2025-02-27 17:02:42 +08:00
# region: 'cn-hangzhou'
# bucket: 'bucket_name'
2026-01-22 11:43:55 +08:00
# signature_version: 's3'
# addressing_style: 'virtual'
2024-11-13 15:56:40 +08:00
# azure:
# auth_type: 'sas'
# container_url: 'container_url'
# sas_token: 'sas_token'
# azure:
# auth_type: 'spn'
# account_url: 'account_url'
# client_id: 'client_id'
# secret: 'secret'
# tenant_id: 'tenant_id'
# container_name: 'container_name'
2025-06-12 11:37:42 +08:00
# The OSS object storage uses the MySQL configuration above by default. If you need to switch to another object storage service, please uncomment and configure the following parameters.
# opendal:
2025-06-13 14:56:51 +08:00
# scheme: 'mysql' # Storage type, such as s3, oss, azure, etc.
2025-06-12 11:37:42 +08:00
# config:
2025-07-02 16:45:01 +08:00
# oss_table: 'opendal_storage'
2024-11-13 15:56:40 +08:00
# user_default_llm:
2025-08-13 09:46:05 +08:00
# factory: 'BAAI'
# api_key: 'backup'
# base_url: 'backup_base_url'
# default_models:
# chat_model:
# name: 'qwen2.5-7b-instruct'
# factory: 'xxxx'
# api_key: 'xxxx'
# base_url: 'https://api.xx.com'
# embedding_model:
2025-10-23 23:02:27 +08:00
# api_key: 'xxx'
# base_url: 'http://localhost:6380'
2025-08-13 09:46:05 +08:00
# rerank_model: 'bge-reranker-v2'
# asr_model:
# model: 'whisper-large-v3' # alias of name
# image2text_model: ''
2024-11-13 15:56:40 +08:00
# oauth:
2025-05-16 14:17:39 +08:00
# oauth2:
# display_name: "OAuth2"
# client_id: "your_client_id"
# client_secret: "your_client_secret"
# authorization_url: "https://your-oauth-provider.com/oauth/authorize"
# token_url: "https://your-oauth-provider.com/oauth/token"
# userinfo_url: "https://your-oauth-provider.com/oauth/userinfo"
# redirect_uri: "https://your-app.com/v1/user/oauth/callback/oauth2"
# oidc:
# display_name: "OIDC"
# client_id: "your_client_id"
# client_secret: "your_client_secret"
# issuer: "https://your-oauth-provider.com/oidc"
# scope: "openid email profile"
# redirect_uri: "https://your-app.com/v1/user/oauth/callback/oidc"
2024-11-13 15:56:40 +08:00
# github:
2025-05-16 14:17:39 +08:00
# type: "github"
# icon: "github"
2025-05-15 14:39:37 +08:00
# display_name: "Github"
2025-05-16 14:17:39 +08:00
# client_id: "your_client_id"
# client_secret: "your_client_secret"
# redirect_uri: "https://your-app.com/v1/user/oauth/callback/github"
2024-11-13 15:56:40 +08:00
# authentication:
# client:
# switch: false
# http_app_key:
# http_secret_key:
# site:
# switch: false
# permission:
# switch: false
# component: false
# dataset: false
2025-08-15 18:12:20 +08:00
# smtp:
# mail_server: ""
# mail_port: 465
# mail_use_ssl: true
# mail_use_tls: false
# mail_username: ""
# mail_password: ""
# mail_default_sender:
# - "RAGFlow" # display name
# - "" # sender email address
# mail_frontend_url: "https://your-frontend.example.com"
2025-10-27 15:14:58 +08:00
# tcadp_config:
# secret_id: 'tencent_secret_id'
# secret_key: 'tencent_secret_key'
# region: 'tencent_region'