This document explains the JSON field conventions used in `conf/models/*.json` and `conf/all_models.json`, and the decimal vs. binary conventions used by different model vendors.
- [How to Add a New Model](#how-to-add-a-new-model)
- [How to Update an Existing Model](#how-to-update-an-existing-model)
- [Quick Reference](#quick-reference)
- [Troubleshooting](#troubleshooting)
---
## JSON Fields
Each model entry in a provider JSON file (`conf/models/<provider>.json`) or in the global catalog (`conf/all_models.json`) supports the following fields:
| Field | Type | Required | Description |
|---|---|---|---|
| `name` | string | Yes | Canonical model identifier (e.g. `gpt-4o`, `claude-opus-4-8`). Must be unique within a provider file. |
| `content_length` | integer | No | Maximum **context window** in tokens — the total number of tokens (input + output) the model can process in a single request. Previously named `max_tokens` (until PR #17807). |
| `max_output` | integer | No | Maximum **output generation** in tokens — the upper bound for tokens the model will generate. It may be a fixed vendor limit, or dynamic (computed as `content_length - input_tokens`). See [Vendor Breakdown](#vendor-breakdown). |
| `model_types` | string[] | Yes | Capabilities of the model. Common values: `chat`, `vision`, `embedding`, `rerank`, `asr`, `tts`, `ocr`, `doc_parse`. |
| `thinking` | object | No | Extended-thinking configuration (see [Thinking Object](#thinking-object)). |
| `tools` | object | No | Tool-use capability (see [Tools Object](#tools-object)). |
| `class` | string | No | Provider-specific model class used to select the correct driver (e.g. `glm`, `kimi`). |
| `max_dimension` | integer | No | Maximum supported embedding dimension. Used by `embedding`-type models (e.g. `1536`). |
| `dimensions` | integer[] | No | Supported embedding dimensions (e.g. `[256, 512, 1024, 1536]`). When non-empty, a requested dimension must match one of these values. When empty `[]` (or omitted), any dimension up to `max_dimension` is accepted. |
| `batch_size` | integer | No | Maximum number of text inputs that can be submitted to the embedding API in a single request. Used by `embedding`-type models. Values come from each provider's official documentation; models with no documented provider limit use a conservative high cap. When omitted, no explicit cap is declared. |
| `alias` | string[] | No | Alternative names for the same model. Used for model lookup when a tenant refers to the model by an alias. **Must be unique across all models.** |
| `rank` | integer | No | Sort priority (lower = higher rank). Used when ordering model lists in the UI. |
### Example Entry
```json
{
"name": "claude-opus-4-8",
"content_length": 1000000,
"max_output": 128000,
"model_types": ["chat", "vision"],
"thinking": {
"default_value": true,
"clear_thinking": true
},
"tools": {
"support": true
}
}
```
### Thinking Object
```jsonc
{
"thinking": {
"default_value": true, // Whether thinking mode is enabled by default
"clear_thinking": true // Whether the API can disable thinking per-request
}
}
```
### Tools Object
```jsonc
{
"tools": {
"support": true // Whether the model supports function/tool calling
-`content_length` is the **total** budget (input + output).
-`max_output` is the **generation** budget alone.
- For most models, `max_output <= content_length`. Some vendors set them equal (output can fill the entire window).
- **Dynamic max_output**: Some models (e.g. Kimi K2.6) define max_output as `content_length - input_tokens`. In these cases, the configured `max_output` represents the upper bound; the actual available output decreases as the prompt grows.
---
## Migration Note
Before PR #17807, a single `max_tokens` field served double duty — it was documented as the context window but often used as the output cap at runtime. The split into `content_length` + `max_output` removes this ambiguity:
- **Old `max_tokens`** → used only as migration context; do not copy it blindly.
- **New `content_length`** → set the vendor-documented context window.
- **New `max_output`** → set the vendor-documented generation cap.
Every migrated model **must define both `content_length` and `max_output`**, each taken from the official vendor model specification.
---
## Decimal vs. Binary Conventions
Different vendors express context windows using different numerical conventions. **This configuration preserves the exact numbers from each vendor's official documentation**, even when vendors disagree on whether "128K" means 128,000 or 131,072.
### How to Identify
| Convention | Pattern | Example |
|---|---|---|
| **Decimal (base-10)** | Round numbers in powers of 10 | 128,000 · 200,000 · 400,000 · 1,000,000 |
| **DeepSeek** | Varies by model — binary (128K, 1M) | Varies by model — binary (8K, 32K, 64K, 384K) | [DeepSeek API Docs](https://api-docs.deepseek.com/) |
1.**Never round or convert** a value to match a different convention. If Anthropic says 200K, write `200000` — not `2097152` or `262144`.
2.**OpenAI, Google, Meta, NVIDIA, DeepSeek, Qwen, Kimi, Mistral** all use binary (powers of 2).
3.**Anthropic, xAI, GLM/Zhipu, MiniMax, Cohere, Baichuan, Amazon** use decimal (powers of 10, or vendor-specific round numbers).
4.**Some vendors mix conventions** within their own catalog (e.g. Anthropic uses decimal for context but binary for output).
5.**When in doubt**, check the official API documentation linked above. The number in this config should match the vendor's stated limit exactly.
---
## Aggregators & Platforms
The following providers are **aggregators** — they host models from multiple upstream creators. Their `content_length` / `max_output` values inherit from the underlying model, not from a native convention of their own. When updating an aggregator's model entry, refer to the upstream creator's documentation (see table above).
| Aggregator | Notes |
|---|---|
| **302ai** | Hosts OpenAI, Anthropic, Google, etc. |
3. For `embedding`-type models, also determine `batch_size` — the provider's documented maximum number of inputs per request — and add it to the entry.
4. Add the entry to the appropriate `conf/models/<provider>.json` file.
5. If the model is also listed in `conf/all_models.json`, update that entry too (or add it).
6. Run `go test ./internal/entity/models/...` to verify the config loads correctly.
Example: A 10,000-character English document ≈ 3,000 tokens.
### Validation Command
```bash
go test ./internal/entity/models/...
```
This loads all provider configs and `conf/all_models.json`, checking for:
- Valid JSON syntax
- Unique aliases across all models
- Correct field types
---
## Troubleshooting
### Duplicate Alias Error
```
InitProviderManager: duplicate alias "X" for models "A" and "B"
```
**Cause**: Two models share the same alias. Aliases must be globally unique.
**Fix**: In `conf/all_models.json`, find the conflicting entries and remove or rename the duplicate alias. Also check `conf/models/*.json` files for the same alias.
### Model Not Found
**Cause**: Model name or alias mismatch between tenant configuration and provider catalog.
**Fix**: Check both `conf/all_models.json` (aliases) and the specific `conf/models/<provider>.json` for the model name.
### Context Length Mismatch
**Symptom**: API returns errors about exceeding context limits.
**Cause**: `content_length` in config does not match the vendor's actual limit.
**Fix**: Verify against official vendor documentation and update accordingly.
