mirror of
https://github.com/personamanagmentlayer/pcl.git
synced 2026-09-14 15:40:22 +08:00
5860a45fe5
* feat: Add comprehensive skill system, MCP integration, and enhanced provider support
Core Features Added:
- Skill system with compiler, resolver, merger, and runtime integration
- MCP (Model Context Protocol) client/server implementation with stdio and HTTP-SSE transports
- Enhanced provider system with Azure, Bedrock, DeepSeek, Gemini, and Ollama support
- Cost tracking and rate limiting capabilities
- Health monitoring for providers
- Fallback provider chain support
- Shell completion support for CLI
New Files:
- src/skills/: Complete skill system implementation
- skill-compiler.ts: Compiles PCL skills into executable format
- skill-resolver.ts: Resolves skill dependencies and references
- skill-merger.ts: Merges skills with conflict resolution
- skill-context.ts: Manages skill execution context
- prompt-integration.ts: Integrates skills into prompts
- runtime-integration.ts: Runtime execution support
- src/mcp/: Full MCP protocol implementation
- client/: MCP client with connection management
- server/: PCL MCP server implementation
- transports/: stdio and HTTP-SSE transport layers
- types/: JSON-RPC and MCP type definitions
- src/runtime/providers/: Enhanced provider implementations
- azure.ts: Azure OpenAI provider
- bedrock.ts: AWS Bedrock provider
- deepseek.ts: DeepSeek provider
- gemini.ts: Google Gemini provider
- ollama.ts: Ollama local models provider
- cost-tracker.ts: Track API costs across providers
- rate-limiter.ts: Rate limiting implementation
- health.ts: Provider health monitoring
- fallback.ts: Automatic fallback chain
- src/formatter/: Code formatting utilities
- src/cli/commands/completion.ts: Shell completion generation
- docs/SKILL_RUNTIME.md: Skill system documentation
- docs/mcp/: MCP integration guides
- docs/providers/: Provider setup and troubleshooting guides
Modified Files:
- README.md: Updated with new features and capabilities
- package.json: Added dependencies for MCP, new providers
- package-lock.json: Updated lockfile with new dependencies
- src/index.ts: Export new modules (skills, MCP, formatter)
- src/cli/index.ts: Add completion command and enhance CLI
- src/runtime/providers/index.ts: Refactor provider registry
- docs/README.md: Add links to new documentation
Removed Files:
- docs/api/PARSER-ENHANCEMENT-SUMMARY.md: Consolidated into other docs
- docs/api/PARSER-ENHANCEMENTS.md: Consolidated into other docs
Tests Added:
- tests/skills/: Comprehensive skill system tests
- skill-compiler.test.ts
- skill-integration.test.ts
- skill-merger.test.ts
- tests/providers/registry-enhancements.test.ts
Configuration Updates:
- .claude/CLAUDE.md: Updated configuration
- .github/dependabot.yml: Enhanced dependency management
This is a major feature release that significantly expands PCL capabilities with:
1. Enterprise-grade skill management system
2. MCP protocol support for AI tool integration
3. Multi-provider support with intelligent fallback
4. Production-ready monitoring and cost tracking
5. Enhanced developer experience with shell completions
* feat: Add CLI skill management commands and registry integration
New CLI Commands (src/cli/commands/skills/):
- bundle.ts: Bundle skills for distribution
- compile.ts: Compile individual skills
- create.ts: Create new skill from template
- info.ts: Display detailed skill information
- install.ts: Install skills from registry
- list.ts: List installed and available skills
- optimize.ts: Optimize skill performance
- publish.ts: Publish skills to registry
- search.ts: Search skill marketplace
- test.ts: Test skill functionality
- trending.ts: Show trending skills
- validate.ts: Validate skill definitions
Registry Extensions (src/registry/):
- skill-metadata.ts: Skill metadata management and validation
- skill-registry.ts: Enhanced registry with skill-specific operations
These commands provide a complete skill lifecycle management experience:
- Discovery: search, trending, list
- Development: create, compile, validate, test, optimize
- Distribution: bundle, publish, install
- Information: info
Integration with existing registry system for seamless skill management.
* feat: Add comprehensive provider system, LSP enhancements, registry extensions, and tooling
Major Features:
1. Provider System Enhancements (src/providers/):
- Base provider abstraction with common functionality
- 10 AI provider integrations:
* Anthropic (Claude)
* OpenAI (GPT)
* Google (Gemini)
* DeepSeek
* Ollama (local models)
* Groq
* Cohere
* Mistral
* Azure OpenAI (existing)
* AWS Bedrock (existing)
- Provider registry with auto-discovery
- Unified provider interface
- Provider adapter for runtime integration
- Integrated providers configuration
2. LSP (Language Server Protocol) Enhancements (src/lsp/):
- Skill-aware code completions
- Skill-specific diagnostics
- Hover information for skills
- Skill navigation (go to definition)
- Skill integration with existing LSP features
- Code actions support
- Rename refactoring support
3. Registry System Extensions (src/registry/):
- Cache Layer:
* Memory cache implementation
* Redis cache adapter
* Multi-layer cache with fallback
- Search Integration:
* Elasticsearch integration for skill search
- Version Management:
* Semantic versioning manager
* Version constraint resolution
4. CLI Enhancements (src/cli/commands/skills/):
- lint: Lint skill definitions
- wizard: Interactive skill creation wizard
5. Documentation:
- AI_PLATFORM_INTEGRATION.md: Platform integration guide
- MCP_INTEGRATION_GUIDE.md: Complete MCP integration documentation
- PROVIDERS.md: Provider system documentation
- SKILL-AUTHORING-GUIDE.md: Comprehensive skill authoring guide
- SKILL-COMPOSITION-PATTERNS.md: Patterns for skill composition
- SKILL-MARKETPLACE-GUIDELINES.md: Guidelines for skill marketplace
6. Examples:
- openapi-spec.yaml: OpenAPI specification example
- rest-api-wrapper.ts: REST API wrapper pattern
- runtime-providers.ts: Provider configuration examples
- workflows/advanced-operators.pcl: Advanced workflow operators demo
7. Claude Code Integration:
- 4 specialized AI agents (.claude/agents/):
* documentation-specialist.md
* qa-testing-expert.md
* security-analyst.md
* software-architect.md
8. Configuration:
- .vscodeignore: VS Code extension packaging configuration
9. Core Module Updates:
- src/ast/index.ts: Enhanced AST exports
- src/index.ts: Export new modules (providers, LSP)
- src/types/index.ts: Additional type definitions
- src/registry/index.ts: Export cache and search modules
- src/registry/interfaces.ts: Enhanced registry interfaces
- src/mcp/*: MCP server refinements
- src/parser/index.ts: Parser improvements
- src/runtime/providers/index.ts: Runtime provider integration
10. Comprehensive Testing:
- tests/lsp/: LSP feature tests (6 test files)
- tests/mcp/: MCP server tests
- tests/parser/: Advanced workflow operator tests
- tests/providers/: Provider system tests
- tests/registry/: Cache and version management tests
This release establishes PCL as a comprehensive AI persona management platform with:
- Multi-provider AI model support (10+ providers)
- Production-ready LSP with skill intelligence
- Scalable registry with caching and search
- Rich developer tooling and documentation
- Extensible architecture for future growth
913 lines
29 KiB
JavaScript
913 lines
29 KiB
JavaScript
/**
|
|
* Provider System Tests
|
|
*
|
|
* Tests for provider registry and Anthropic provider
|
|
*/
|
|
|
|
import { describe, it, before, after } from 'node:test';
|
|
import assert from 'node:assert';
|
|
import {
|
|
getProviderRegistry,
|
|
registerProvider,
|
|
getProvider,
|
|
listProviders,
|
|
AnthropicProvider,
|
|
OpenAIProvider,
|
|
GoogleProvider,
|
|
DeepSeekProvider,
|
|
OllamaProvider,
|
|
MistralProvider,
|
|
GroqProvider,
|
|
CohereProvider,
|
|
} from '../../dist/providers/index.js';
|
|
|
|
describe('Provider Registry', () => {
|
|
let registry;
|
|
|
|
before(() => {
|
|
registry = getProviderRegistry();
|
|
});
|
|
|
|
after(async () => {
|
|
await registry.shutdownAll();
|
|
});
|
|
|
|
it('should have Anthropic provider registered', () => {
|
|
const providers = listProviders();
|
|
const anthropic = providers.find((p) => p.name === 'anthropic');
|
|
|
|
assert.ok(anthropic, 'Anthropic provider should be registered');
|
|
assert.strictEqual(anthropic.displayName, 'Anthropic');
|
|
});
|
|
|
|
it('should list all registered providers', () => {
|
|
const providers = listProviders();
|
|
assert.ok(Array.isArray(providers), 'Should return array of providers');
|
|
assert.ok(providers.length > 0, 'Should have at least one provider');
|
|
});
|
|
|
|
it('should check if provider exists', () => {
|
|
assert.ok(registry.has('anthropic'), 'Should have anthropic provider');
|
|
assert.ok(!registry.has('nonexistent'), 'Should not have nonexistent provider');
|
|
});
|
|
|
|
it('should get provider metadata', () => {
|
|
const metadata = registry.getMetadata('anthropic');
|
|
assert.ok(metadata, 'Should return metadata');
|
|
assert.strictEqual(metadata.name, 'anthropic');
|
|
assert.strictEqual(metadata.displayName, 'Anthropic');
|
|
});
|
|
});
|
|
|
|
describe('Anthropic Provider', () => {
|
|
let provider;
|
|
|
|
it('should create provider instance', () => {
|
|
provider = new AnthropicProvider();
|
|
assert.ok(provider, 'Provider should be created');
|
|
assert.strictEqual(provider.name, 'anthropic');
|
|
assert.strictEqual(provider.displayName, 'Anthropic');
|
|
});
|
|
|
|
it('should require API key for initialization', async () => {
|
|
const provider = new AnthropicProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.initialize({});
|
|
},
|
|
{
|
|
message: 'Anthropic API key is required',
|
|
},
|
|
'Should reject without API key'
|
|
);
|
|
});
|
|
|
|
it('should initialize with API key', async () => {
|
|
const provider = new AnthropicProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
}, 'Should initialize with API key');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should list available models', async () => {
|
|
const provider = new AnthropicProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const models = await provider.getModels();
|
|
assert.ok(Array.isArray(models), 'Should return array of models');
|
|
assert.ok(models.length > 0, 'Should have at least one model');
|
|
|
|
// Check for Claude 3.5 Sonnet
|
|
const sonnet = models.find((m) => m.id === 'claude-3-5-sonnet-20241022');
|
|
assert.ok(sonnet, 'Should have Claude 3.5 Sonnet');
|
|
assert.strictEqual(sonnet.name, 'Claude 3.5 Sonnet');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should get specific model info', async () => {
|
|
const provider = new AnthropicProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const model = await provider.getModel('claude-3-5-sonnet-20241022');
|
|
assert.ok(model, 'Should return model');
|
|
assert.strictEqual(model.id, 'claude-3-5-sonnet-20241022');
|
|
assert.strictEqual(model.name, 'Claude 3.5 Sonnet');
|
|
assert.ok(model.capabilities, 'Should have capabilities');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should check if model exists', async () => {
|
|
const provider = new AnthropicProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const hasSonnet = await provider.hasModel('claude-3-5-sonnet-20241022');
|
|
assert.ok(hasSonnet, 'Should have Claude 3.5 Sonnet');
|
|
|
|
const hasNonexistent = await provider.hasModel('nonexistent-model');
|
|
assert.ok(!hasNonexistent, 'Should not have nonexistent model');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should report capabilities', async () => {
|
|
const provider = new AnthropicProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const capabilities = provider.getCapabilities();
|
|
assert.ok(capabilities, 'Should have capabilities');
|
|
assert.strictEqual(capabilities.streaming, true);
|
|
assert.strictEqual(capabilities.functionCalling, true);
|
|
assert.strictEqual(capabilities.vision, true);
|
|
assert.strictEqual(capabilities.systemMessages, true);
|
|
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should estimate token count', async () => {
|
|
const provider = new AnthropicProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const text = 'Hello, world!';
|
|
const tokens = await provider.countTokens(text);
|
|
assert.ok(tokens > 0, 'Should return positive token count');
|
|
assert.ok(tokens <= text.length, 'Token count should be reasonable');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should require initialization before completion', async () => {
|
|
const provider = new AnthropicProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.complete({
|
|
model: 'claude-3-5-sonnet-20241022',
|
|
messages: [{ role: 'user', content: 'Hello' }],
|
|
});
|
|
},
|
|
{
|
|
message: /not initialized/,
|
|
},
|
|
'Should reject if not initialized'
|
|
);
|
|
});
|
|
|
|
it('should validate completion request', async () => {
|
|
const provider = new AnthropicProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.complete({
|
|
model: '',
|
|
messages: [],
|
|
});
|
|
},
|
|
{
|
|
message: /Model is required|At least one message/,
|
|
},
|
|
'Should reject invalid request'
|
|
);
|
|
|
|
await provider.shutdown();
|
|
});
|
|
});
|
|
|
|
describe('OpenAI Provider', () => {
|
|
let provider;
|
|
|
|
it('should create provider instance', () => {
|
|
provider = new OpenAIProvider();
|
|
assert.ok(provider, 'Provider should be created');
|
|
assert.strictEqual(provider.name, 'openai');
|
|
assert.strictEqual(provider.displayName, 'OpenAI');
|
|
});
|
|
|
|
it('should require API key for initialization', async () => {
|
|
const provider = new OpenAIProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.initialize({});
|
|
},
|
|
{
|
|
message: 'OpenAI API key is required',
|
|
},
|
|
'Should reject without API key'
|
|
);
|
|
});
|
|
|
|
it('should initialize with API key', async () => {
|
|
const provider = new OpenAIProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
}, 'Should initialize with API key');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should list available models', async () => {
|
|
const provider = new OpenAIProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const models = await provider.getModels();
|
|
assert.ok(Array.isArray(models), 'Should return array of models');
|
|
assert.ok(models.length > 0, 'Should have at least one model');
|
|
|
|
// Check for GPT-4 Turbo
|
|
const gpt4turbo = models.find((m) => m.id === 'gpt-4-turbo-2024-04-09');
|
|
assert.ok(gpt4turbo, 'Should have GPT-4 Turbo');
|
|
assert.strictEqual(gpt4turbo.name, 'GPT-4 Turbo');
|
|
|
|
// Check for GPT-4o
|
|
const gpt4o = models.find((m) => m.id === 'gpt-4o');
|
|
assert.ok(gpt4o, 'Should have GPT-4o');
|
|
|
|
// Check for GPT-3.5 Turbo
|
|
const gpt35 = models.find((m) => m.id === 'gpt-3.5-turbo-0125');
|
|
assert.ok(gpt35, 'Should have GPT-3.5 Turbo');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should get specific model info', async () => {
|
|
const provider = new OpenAIProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const model = await provider.getModel('gpt-4-turbo-2024-04-09');
|
|
assert.ok(model, 'Should return model');
|
|
assert.strictEqual(model.id, 'gpt-4-turbo-2024-04-09');
|
|
assert.strictEqual(model.name, 'GPT-4 Turbo');
|
|
assert.ok(model.capabilities, 'Should have capabilities');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should check if model exists', async () => {
|
|
const provider = new OpenAIProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const hasGPT4 = await provider.hasModel('gpt-4-turbo-2024-04-09');
|
|
assert.ok(hasGPT4, 'Should have GPT-4 Turbo');
|
|
|
|
const hasNonexistent = await provider.hasModel('nonexistent-model');
|
|
assert.ok(!hasNonexistent, 'Should not have nonexistent model');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should report capabilities', async () => {
|
|
const provider = new OpenAIProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const capabilities = provider.getCapabilities();
|
|
assert.ok(capabilities, 'Should have capabilities');
|
|
assert.strictEqual(capabilities.streaming, true);
|
|
assert.strictEqual(capabilities.functionCalling, true);
|
|
assert.strictEqual(capabilities.vision, true);
|
|
assert.strictEqual(capabilities.jsonMode, true);
|
|
assert.strictEqual(capabilities.systemMessages, true);
|
|
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should estimate token count', async () => {
|
|
const provider = new OpenAIProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const text = 'Hello, world!';
|
|
const tokens = await provider.countTokens(text);
|
|
assert.ok(tokens > 0, 'Should return positive token count');
|
|
assert.ok(tokens <= text.length, 'Token count should be reasonable');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should have OpenAI provider registered', () => {
|
|
const providers = listProviders();
|
|
const openai = providers.find((p) => p.name === 'openai');
|
|
|
|
assert.ok(openai, 'OpenAI provider should be registered');
|
|
assert.strictEqual(openai.displayName, 'OpenAI');
|
|
});
|
|
});
|
|
|
|
describe('Google Provider', () => {
|
|
let provider;
|
|
|
|
it('should create provider instance', () => {
|
|
provider = new GoogleProvider();
|
|
assert.ok(provider, 'Provider should be created');
|
|
assert.strictEqual(provider.name, 'google');
|
|
assert.strictEqual(provider.displayName, 'Google');
|
|
});
|
|
|
|
it('should require API key for initialization', async () => {
|
|
const provider = new GoogleProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.initialize({});
|
|
},
|
|
{
|
|
message: 'Google API key is required',
|
|
},
|
|
'Should reject without API key'
|
|
);
|
|
});
|
|
|
|
it('should initialize with API key', async () => {
|
|
const provider = new GoogleProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
}, 'Should initialize with API key');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should list available models', async () => {
|
|
const provider = new GoogleProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const models = await provider.getModels();
|
|
assert.ok(Array.isArray(models), 'Should return array of models');
|
|
assert.ok(models.length > 0, 'Should have at least one model');
|
|
|
|
// Check for Gemini 2.0 Flash
|
|
const gemini2 = models.find((m) => m.id === 'gemini-2.0-flash-exp');
|
|
assert.ok(gemini2, 'Should have Gemini 2.0 Flash');
|
|
assert.strictEqual(gemini2.name, 'Gemini 2.0 Flash (Experimental)');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should get specific model info', async () => {
|
|
const provider = new GoogleProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const model = await provider.getModel('gemini-1.5-pro');
|
|
assert.ok(model, 'Should return model');
|
|
assert.strictEqual(model.id, 'gemini-1.5-pro');
|
|
assert.strictEqual(model.name, 'Gemini 1.5 Pro');
|
|
assert.ok(model.capabilities, 'Should have capabilities');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should check if model exists', async () => {
|
|
const provider = new GoogleProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const hasGemini = await provider.hasModel('gemini-1.5-pro');
|
|
assert.ok(hasGemini, 'Should have Gemini 1.5 Pro');
|
|
|
|
const hasNonexistent = await provider.hasModel('nonexistent-model');
|
|
assert.ok(!hasNonexistent, 'Should not have nonexistent model');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should report capabilities', async () => {
|
|
const provider = new GoogleProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const capabilities = provider.getCapabilities();
|
|
assert.ok(capabilities, 'Should have capabilities');
|
|
assert.strictEqual(capabilities.streaming, true);
|
|
assert.strictEqual(capabilities.functionCalling, true);
|
|
assert.strictEqual(capabilities.vision, true);
|
|
assert.strictEqual(capabilities.systemMessages, true);
|
|
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should estimate token count', async () => {
|
|
const provider = new GoogleProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const text = 'Hello, world!';
|
|
const tokens = await provider.countTokens(text);
|
|
assert.ok(tokens > 0, 'Should return positive token count');
|
|
assert.ok(tokens <= text.length, 'Token count should be reasonable');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should have Google provider registered', () => {
|
|
const providers = listProviders();
|
|
const google = providers.find((p) => p.name === 'google');
|
|
|
|
assert.ok(google, 'Google provider should be registered');
|
|
assert.strictEqual(google.displayName, 'Google');
|
|
});
|
|
});
|
|
|
|
describe('DeepSeek Provider', () => {
|
|
let provider;
|
|
|
|
it('should create provider instance', () => {
|
|
provider = new DeepSeekProvider();
|
|
assert.ok(provider, 'Provider should be created');
|
|
assert.strictEqual(provider.name, 'deepseek');
|
|
assert.strictEqual(provider.displayName, 'DeepSeek');
|
|
});
|
|
|
|
it('should require API key for initialization', async () => {
|
|
const provider = new DeepSeekProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.initialize({});
|
|
},
|
|
{
|
|
message: 'DeepSeek API key is required',
|
|
},
|
|
'Should reject without API key'
|
|
);
|
|
});
|
|
|
|
it('should initialize with API key', async () => {
|
|
const provider = new DeepSeekProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
}, 'Should initialize with API key');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should list available models', async () => {
|
|
const provider = new DeepSeekProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const models = await provider.getModels();
|
|
assert.ok(Array.isArray(models), 'Should return array of models');
|
|
assert.ok(models.length > 0, 'Should have at least one model');
|
|
|
|
// Check for DeepSeek Chat
|
|
const chat = models.find((m) => m.id === 'deepseek-chat');
|
|
assert.ok(chat, 'Should have DeepSeek Chat');
|
|
assert.strictEqual(chat.name, 'DeepSeek Chat');
|
|
|
|
// Check for DeepSeek Coder
|
|
const coder = models.find((m) => m.id === 'deepseek-coder');
|
|
assert.ok(coder, 'Should have DeepSeek Coder');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should get specific model info', async () => {
|
|
const provider = new DeepSeekProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const model = await provider.getModel('deepseek-chat');
|
|
assert.ok(model, 'Should return model');
|
|
assert.strictEqual(model.id, 'deepseek-chat');
|
|
assert.strictEqual(model.name, 'DeepSeek Chat');
|
|
assert.ok(model.capabilities, 'Should have capabilities');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should check if model exists', async () => {
|
|
const provider = new DeepSeekProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const hasChat = await provider.hasModel('deepseek-chat');
|
|
assert.ok(hasChat, 'Should have DeepSeek Chat');
|
|
|
|
const hasNonexistent = await provider.hasModel('nonexistent-model');
|
|
assert.ok(!hasNonexistent, 'Should not have nonexistent model');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should report capabilities', async () => {
|
|
const provider = new DeepSeekProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const capabilities = provider.getCapabilities();
|
|
assert.ok(capabilities, 'Should have capabilities');
|
|
assert.strictEqual(capabilities.streaming, true);
|
|
assert.strictEqual(capabilities.functionCalling, true);
|
|
assert.strictEqual(capabilities.systemMessages, true);
|
|
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should estimate token count', async () => {
|
|
const provider = new DeepSeekProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const text = 'Hello, world!';
|
|
const tokens = await provider.countTokens(text);
|
|
assert.ok(tokens > 0, 'Should return positive token count');
|
|
assert.ok(tokens <= text.length, 'Token count should be reasonable');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should have DeepSeek provider registered', () => {
|
|
const providers = listProviders();
|
|
const deepseek = providers.find((p) => p.name === 'deepseek');
|
|
|
|
assert.ok(deepseek, 'DeepSeek provider should be registered');
|
|
assert.strictEqual(deepseek.displayName, 'DeepSeek');
|
|
});
|
|
});
|
|
|
|
describe('Ollama Provider', () => {
|
|
let provider;
|
|
|
|
it('should create provider instance', () => {
|
|
provider = new OllamaProvider();
|
|
assert.ok(provider, 'Provider should be created');
|
|
assert.strictEqual(provider.name, 'ollama');
|
|
assert.strictEqual(provider.displayName, 'Ollama');
|
|
});
|
|
|
|
it('should allow initialization without API key', async () => {
|
|
const provider = new OllamaProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({});
|
|
}, 'Should initialize without API key');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should initialize with custom baseUrl', async () => {
|
|
const provider = new OllamaProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({ baseUrl: 'http://custom:11434' });
|
|
}, 'Should initialize with custom baseUrl');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should reject invalid baseUrl', async () => {
|
|
const provider = new OllamaProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.initialize({ baseUrl: 'invalid-url' });
|
|
},
|
|
{
|
|
message: /must start with http/,
|
|
},
|
|
'Should reject invalid baseUrl'
|
|
);
|
|
});
|
|
|
|
it('should list available models', async () => {
|
|
const provider = new OllamaProvider();
|
|
await provider.initialize({});
|
|
|
|
const models = await provider.getModels();
|
|
assert.ok(Array.isArray(models), 'Should return array of models');
|
|
// Ollama may have 0 models if none installed, so just check array type
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should report capabilities', async () => {
|
|
const provider = new OllamaProvider();
|
|
await provider.initialize({});
|
|
|
|
const capabilities = provider.getCapabilities();
|
|
assert.ok(capabilities, 'Should have capabilities');
|
|
assert.strictEqual(capabilities.streaming, true);
|
|
assert.strictEqual(capabilities.jsonMode, true);
|
|
assert.strictEqual(capabilities.systemMessages, true);
|
|
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should estimate token count', async () => {
|
|
const provider = new OllamaProvider();
|
|
await provider.initialize({});
|
|
|
|
const text = 'Hello, world!';
|
|
const tokens = await provider.countTokens(text);
|
|
assert.ok(tokens > 0, 'Should return positive token count');
|
|
assert.ok(tokens <= text.length, 'Token count should be reasonable');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should have Ollama provider registered', () => {
|
|
const providers = listProviders();
|
|
const ollama = providers.find((p) => p.name === 'ollama');
|
|
|
|
assert.ok(ollama, 'Ollama provider should be registered');
|
|
assert.strictEqual(ollama.displayName, 'Ollama');
|
|
});
|
|
|
|
it('should have free pricing for local models', async () => {
|
|
const provider = new OllamaProvider();
|
|
await provider.initialize({});
|
|
|
|
const models = await provider.getModels();
|
|
// All Ollama models should be free (0 cost)
|
|
for (const model of models) {
|
|
assert.strictEqual(model.inputTokenCost, 0, 'Input tokens should be free');
|
|
assert.strictEqual(model.outputTokenCost, 0, 'Output tokens should be free');
|
|
}
|
|
|
|
await provider.shutdown();
|
|
});
|
|
});
|
|
|
|
describe('Mistral AI Provider', () => {
|
|
let provider;
|
|
|
|
it('should create provider instance', () => {
|
|
provider = new MistralProvider();
|
|
assert.ok(provider, 'Provider should be created');
|
|
assert.strictEqual(provider.name, 'mistral');
|
|
assert.strictEqual(provider.displayName, 'Mistral AI');
|
|
});
|
|
|
|
it('should require API key for initialization', async () => {
|
|
const provider = new MistralProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.initialize({});
|
|
},
|
|
{
|
|
message: 'Mistral API key is required',
|
|
},
|
|
'Should reject without API key'
|
|
);
|
|
});
|
|
|
|
it('should initialize with API key', async () => {
|
|
const provider = new MistralProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
}, 'Should initialize with API key');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should list available models', async () => {
|
|
const provider = new MistralProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const models = await provider.getModels();
|
|
assert.ok(Array.isArray(models), 'Should return array of models');
|
|
assert.ok(models.length > 0, 'Should have at least one model');
|
|
|
|
// Check for Mistral Large
|
|
const large = models.find((m) => m.id === 'mistral-large-latest');
|
|
assert.ok(large, 'Should have Mistral Large');
|
|
assert.strictEqual(large.name, 'Mistral Large');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should get specific model info', async () => {
|
|
const provider = new MistralProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const model = await provider.getModel('mistral-small-latest');
|
|
assert.ok(model, 'Should return model');
|
|
assert.strictEqual(model.id, 'mistral-small-latest');
|
|
assert.strictEqual(model.name, 'Mistral Small');
|
|
assert.ok(model.capabilities, 'Should have capabilities');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should report capabilities', async () => {
|
|
const provider = new MistralProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const capabilities = provider.getCapabilities();
|
|
assert.ok(capabilities, 'Should have capabilities');
|
|
assert.strictEqual(capabilities.streaming, true);
|
|
assert.strictEqual(capabilities.functionCalling, true);
|
|
assert.strictEqual(capabilities.jsonMode, true);
|
|
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should have Mistral provider registered', () => {
|
|
const providers = listProviders();
|
|
const mistral = providers.find((p) => p.name === 'mistral');
|
|
|
|
assert.ok(mistral, 'Mistral provider should be registered');
|
|
assert.strictEqual(mistral.displayName, 'Mistral AI');
|
|
});
|
|
});
|
|
|
|
describe('Groq Provider', () => {
|
|
let provider;
|
|
|
|
it('should create provider instance', () => {
|
|
provider = new GroqProvider();
|
|
assert.ok(provider, 'Provider should be created');
|
|
assert.strictEqual(provider.name, 'groq');
|
|
assert.strictEqual(provider.displayName, 'Groq');
|
|
});
|
|
|
|
it('should require API key for initialization', async () => {
|
|
const provider = new GroqProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.initialize({});
|
|
},
|
|
{
|
|
message: 'Groq API key is required',
|
|
},
|
|
'Should reject without API key'
|
|
);
|
|
});
|
|
|
|
it('should initialize with API key', async () => {
|
|
const provider = new GroqProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
}, 'Should initialize with API key');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should list available models', async () => {
|
|
const provider = new GroqProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const models = await provider.getModels();
|
|
assert.ok(Array.isArray(models), 'Should return array of models');
|
|
assert.ok(models.length > 0, 'Should have at least one model');
|
|
|
|
// Check for Llama 3.3 70B
|
|
const llama = models.find((m) => m.id === 'llama-3.3-70b-versatile');
|
|
assert.ok(llama, 'Should have Llama 3.3 70B');
|
|
assert.strictEqual(llama.name, 'Llama 3.3 70B');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should get specific model info', async () => {
|
|
const provider = new GroqProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const model = await provider.getModel('llama-3.1-8b-instant');
|
|
assert.ok(model, 'Should return model');
|
|
assert.strictEqual(model.id, 'llama-3.1-8b-instant');
|
|
assert.strictEqual(model.name, 'Llama 3.1 8B Instant');
|
|
assert.ok(model.capabilities, 'Should have capabilities');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should report capabilities', async () => {
|
|
const provider = new GroqProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const capabilities = provider.getCapabilities();
|
|
assert.ok(capabilities, 'Should have capabilities');
|
|
assert.strictEqual(capabilities.streaming, true);
|
|
assert.strictEqual(capabilities.functionCalling, true);
|
|
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should have Groq provider registered', () => {
|
|
const providers = listProviders();
|
|
const groq = providers.find((p) => p.name === 'groq');
|
|
|
|
assert.ok(groq, 'Groq provider should be registered');
|
|
assert.strictEqual(groq.displayName, 'Groq');
|
|
});
|
|
});
|
|
|
|
describe('Cohere Provider', () => {
|
|
let provider;
|
|
|
|
it('should create provider instance', () => {
|
|
provider = new CohereProvider();
|
|
assert.ok(provider, 'Provider should be created');
|
|
assert.strictEqual(provider.name, 'cohere');
|
|
assert.strictEqual(provider.displayName, 'Cohere');
|
|
});
|
|
|
|
it('should require API key for initialization', async () => {
|
|
const provider = new CohereProvider();
|
|
|
|
await assert.rejects(
|
|
async () => {
|
|
await provider.initialize({});
|
|
},
|
|
{
|
|
message: 'Cohere API key is required',
|
|
},
|
|
'Should reject without API key'
|
|
);
|
|
});
|
|
|
|
it('should initialize with API key', async () => {
|
|
const provider = new CohereProvider();
|
|
|
|
await assert.doesNotReject(async () => {
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
}, 'Should initialize with API key');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should list available models', async () => {
|
|
const provider = new CohereProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const models = await provider.getModels();
|
|
assert.ok(Array.isArray(models), 'Should return array of models');
|
|
assert.ok(models.length > 0, 'Should have at least one model');
|
|
|
|
// Check for Command R+
|
|
const commandRPlus = models.find((m) => m.id === 'command-r-plus');
|
|
assert.ok(commandRPlus, 'Should have Command R+');
|
|
assert.strictEqual(commandRPlus.name, 'Command R+');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should get specific model info', async () => {
|
|
const provider = new CohereProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const model = await provider.getModel('command-r');
|
|
assert.ok(model, 'Should return model');
|
|
assert.strictEqual(model.id, 'command-r');
|
|
assert.strictEqual(model.name, 'Command R');
|
|
assert.ok(model.capabilities, 'Should have capabilities');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should report capabilities', async () => {
|
|
const provider = new CohereProvider();
|
|
await provider.initialize({ apiKey: 'test-key' });
|
|
|
|
const capabilities = provider.getCapabilities();
|
|
assert.ok(capabilities, 'Should have capabilities');
|
|
assert.strictEqual(capabilities.streaming, true);
|
|
assert.strictEqual(capabilities.functionCalling, true);
|
|
assert.strictEqual(capabilities.topK, true);
|
|
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
|
|
|
|
await provider.shutdown();
|
|
});
|
|
|
|
it('should have Cohere provider registered', () => {
|
|
const providers = listProviders();
|
|
const cohere = providers.find((p) => p.name === 'cohere');
|
|
|
|
assert.ok(cohere, 'Cohere provider should be registered');
|
|
assert.strictEqual(cohere.displayName, 'Cohere');
|
|
});
|
|
});
|
|
|
|
console.log('✅ All provider tests passed');
|