Files
personamanagmentlayer__pcl/tests/providers/provider-system.test.mjs
ANGX 5860a45fe5 feat: Comprehensive PCL enhancements - Providers, LSP, Skills, and MCP (#20)
* feat: Add comprehensive skill system, MCP integration, and enhanced provider support

Core Features Added:
- Skill system with compiler, resolver, merger, and runtime integration
- MCP (Model Context Protocol) client/server implementation with stdio and HTTP-SSE transports
- Enhanced provider system with Azure, Bedrock, DeepSeek, Gemini, and Ollama support
- Cost tracking and rate limiting capabilities
- Health monitoring for providers
- Fallback provider chain support
- Shell completion support for CLI

New Files:
- src/skills/: Complete skill system implementation
  - skill-compiler.ts: Compiles PCL skills into executable format
  - skill-resolver.ts: Resolves skill dependencies and references
  - skill-merger.ts: Merges skills with conflict resolution
  - skill-context.ts: Manages skill execution context
  - prompt-integration.ts: Integrates skills into prompts
  - runtime-integration.ts: Runtime execution support

- src/mcp/: Full MCP protocol implementation
  - client/: MCP client with connection management
  - server/: PCL MCP server implementation
  - transports/: stdio and HTTP-SSE transport layers
  - types/: JSON-RPC and MCP type definitions

- src/runtime/providers/: Enhanced provider implementations
  - azure.ts: Azure OpenAI provider
  - bedrock.ts: AWS Bedrock provider
  - deepseek.ts: DeepSeek provider
  - gemini.ts: Google Gemini provider
  - ollama.ts: Ollama local models provider
  - cost-tracker.ts: Track API costs across providers
  - rate-limiter.ts: Rate limiting implementation
  - health.ts: Provider health monitoring
  - fallback.ts: Automatic fallback chain

- src/formatter/: Code formatting utilities
- src/cli/commands/completion.ts: Shell completion generation
- docs/SKILL_RUNTIME.md: Skill system documentation
- docs/mcp/: MCP integration guides
- docs/providers/: Provider setup and troubleshooting guides

Modified Files:
- README.md: Updated with new features and capabilities
- package.json: Added dependencies for MCP, new providers
- package-lock.json: Updated lockfile with new dependencies
- src/index.ts: Export new modules (skills, MCP, formatter)
- src/cli/index.ts: Add completion command and enhance CLI
- src/runtime/providers/index.ts: Refactor provider registry
- docs/README.md: Add links to new documentation

Removed Files:
- docs/api/PARSER-ENHANCEMENT-SUMMARY.md: Consolidated into other docs
- docs/api/PARSER-ENHANCEMENTS.md: Consolidated into other docs

Tests Added:
- tests/skills/: Comprehensive skill system tests
  - skill-compiler.test.ts
  - skill-integration.test.ts
  - skill-merger.test.ts
- tests/providers/registry-enhancements.test.ts

Configuration Updates:
- .claude/CLAUDE.md: Updated configuration
- .github/dependabot.yml: Enhanced dependency management

This is a major feature release that significantly expands PCL capabilities with:
1. Enterprise-grade skill management system
2. MCP protocol support for AI tool integration
3. Multi-provider support with intelligent fallback
4. Production-ready monitoring and cost tracking
5. Enhanced developer experience with shell completions

* feat: Add CLI skill management commands and registry integration

New CLI Commands (src/cli/commands/skills/):
- bundle.ts: Bundle skills for distribution
- compile.ts: Compile individual skills
- create.ts: Create new skill from template
- info.ts: Display detailed skill information
- install.ts: Install skills from registry
- list.ts: List installed and available skills
- optimize.ts: Optimize skill performance
- publish.ts: Publish skills to registry
- search.ts: Search skill marketplace
- test.ts: Test skill functionality
- trending.ts: Show trending skills
- validate.ts: Validate skill definitions

Registry Extensions (src/registry/):
- skill-metadata.ts: Skill metadata management and validation
- skill-registry.ts: Enhanced registry with skill-specific operations

These commands provide a complete skill lifecycle management experience:
- Discovery: search, trending, list
- Development: create, compile, validate, test, optimize
- Distribution: bundle, publish, install
- Information: info

Integration with existing registry system for seamless skill management.

* feat: Add comprehensive provider system, LSP enhancements, registry extensions, and tooling

Major Features:

1. Provider System Enhancements (src/providers/):
   - Base provider abstraction with common functionality
   - 10 AI provider integrations:
     * Anthropic (Claude)
     * OpenAI (GPT)
     * Google (Gemini)
     * DeepSeek
     * Ollama (local models)
     * Groq
     * Cohere
     * Mistral
     * Azure OpenAI (existing)
     * AWS Bedrock (existing)
   - Provider registry with auto-discovery
   - Unified provider interface
   - Provider adapter for runtime integration
   - Integrated providers configuration

2. LSP (Language Server Protocol) Enhancements (src/lsp/):
   - Skill-aware code completions
   - Skill-specific diagnostics
   - Hover information for skills
   - Skill navigation (go to definition)
   - Skill integration with existing LSP features
   - Code actions support
   - Rename refactoring support

3. Registry System Extensions (src/registry/):
   - Cache Layer:
     * Memory cache implementation
     * Redis cache adapter
     * Multi-layer cache with fallback
   - Search Integration:
     * Elasticsearch integration for skill search
   - Version Management:
     * Semantic versioning manager
     * Version constraint resolution

4. CLI Enhancements (src/cli/commands/skills/):
   - lint: Lint skill definitions
   - wizard: Interactive skill creation wizard

5. Documentation:
   - AI_PLATFORM_INTEGRATION.md: Platform integration guide
   - MCP_INTEGRATION_GUIDE.md: Complete MCP integration documentation
   - PROVIDERS.md: Provider system documentation
   - SKILL-AUTHORING-GUIDE.md: Comprehensive skill authoring guide
   - SKILL-COMPOSITION-PATTERNS.md: Patterns for skill composition
   - SKILL-MARKETPLACE-GUIDELINES.md: Guidelines for skill marketplace

6. Examples:
   - openapi-spec.yaml: OpenAPI specification example
   - rest-api-wrapper.ts: REST API wrapper pattern
   - runtime-providers.ts: Provider configuration examples
   - workflows/advanced-operators.pcl: Advanced workflow operators demo

7. Claude Code Integration:
   - 4 specialized AI agents (.claude/agents/):
     * documentation-specialist.md
     * qa-testing-expert.md
     * security-analyst.md
     * software-architect.md

8. Configuration:
   - .vscodeignore: VS Code extension packaging configuration

9. Core Module Updates:
   - src/ast/index.ts: Enhanced AST exports
   - src/index.ts: Export new modules (providers, LSP)
   - src/types/index.ts: Additional type definitions
   - src/registry/index.ts: Export cache and search modules
   - src/registry/interfaces.ts: Enhanced registry interfaces
   - src/mcp/*: MCP server refinements
   - src/parser/index.ts: Parser improvements
   - src/runtime/providers/index.ts: Runtime provider integration

10. Comprehensive Testing:
    - tests/lsp/: LSP feature tests (6 test files)
    - tests/mcp/: MCP server tests
    - tests/parser/: Advanced workflow operator tests
    - tests/providers/: Provider system tests
    - tests/registry/: Cache and version management tests

This release establishes PCL as a comprehensive AI persona management platform with:
- Multi-provider AI model support (10+ providers)
- Production-ready LSP with skill intelligence
- Scalable registry with caching and search
- Rich developer tooling and documentation
- Extensible architecture for future growth
2026-01-23 04:29:29 +01:00

913 lines
29 KiB
JavaScript

/**
* Provider System Tests
*
* Tests for provider registry and Anthropic provider
*/
import { describe, it, before, after } from 'node:test';
import assert from 'node:assert';
import {
getProviderRegistry,
registerProvider,
getProvider,
listProviders,
AnthropicProvider,
OpenAIProvider,
GoogleProvider,
DeepSeekProvider,
OllamaProvider,
MistralProvider,
GroqProvider,
CohereProvider,
} from '../../dist/providers/index.js';
describe('Provider Registry', () => {
let registry;
before(() => {
registry = getProviderRegistry();
});
after(async () => {
await registry.shutdownAll();
});
it('should have Anthropic provider registered', () => {
const providers = listProviders();
const anthropic = providers.find((p) => p.name === 'anthropic');
assert.ok(anthropic, 'Anthropic provider should be registered');
assert.strictEqual(anthropic.displayName, 'Anthropic');
});
it('should list all registered providers', () => {
const providers = listProviders();
assert.ok(Array.isArray(providers), 'Should return array of providers');
assert.ok(providers.length > 0, 'Should have at least one provider');
});
it('should check if provider exists', () => {
assert.ok(registry.has('anthropic'), 'Should have anthropic provider');
assert.ok(!registry.has('nonexistent'), 'Should not have nonexistent provider');
});
it('should get provider metadata', () => {
const metadata = registry.getMetadata('anthropic');
assert.ok(metadata, 'Should return metadata');
assert.strictEqual(metadata.name, 'anthropic');
assert.strictEqual(metadata.displayName, 'Anthropic');
});
});
describe('Anthropic Provider', () => {
let provider;
it('should create provider instance', () => {
provider = new AnthropicProvider();
assert.ok(provider, 'Provider should be created');
assert.strictEqual(provider.name, 'anthropic');
assert.strictEqual(provider.displayName, 'Anthropic');
});
it('should require API key for initialization', async () => {
const provider = new AnthropicProvider();
await assert.rejects(
async () => {
await provider.initialize({});
},
{
message: 'Anthropic API key is required',
},
'Should reject without API key'
);
});
it('should initialize with API key', async () => {
const provider = new AnthropicProvider();
await assert.doesNotReject(async () => {
await provider.initialize({ apiKey: 'test-key' });
}, 'Should initialize with API key');
await provider.shutdown();
});
it('should list available models', async () => {
const provider = new AnthropicProvider();
await provider.initialize({ apiKey: 'test-key' });
const models = await provider.getModels();
assert.ok(Array.isArray(models), 'Should return array of models');
assert.ok(models.length > 0, 'Should have at least one model');
// Check for Claude 3.5 Sonnet
const sonnet = models.find((m) => m.id === 'claude-3-5-sonnet-20241022');
assert.ok(sonnet, 'Should have Claude 3.5 Sonnet');
assert.strictEqual(sonnet.name, 'Claude 3.5 Sonnet');
await provider.shutdown();
});
it('should get specific model info', async () => {
const provider = new AnthropicProvider();
await provider.initialize({ apiKey: 'test-key' });
const model = await provider.getModel('claude-3-5-sonnet-20241022');
assert.ok(model, 'Should return model');
assert.strictEqual(model.id, 'claude-3-5-sonnet-20241022');
assert.strictEqual(model.name, 'Claude 3.5 Sonnet');
assert.ok(model.capabilities, 'Should have capabilities');
await provider.shutdown();
});
it('should check if model exists', async () => {
const provider = new AnthropicProvider();
await provider.initialize({ apiKey: 'test-key' });
const hasSonnet = await provider.hasModel('claude-3-5-sonnet-20241022');
assert.ok(hasSonnet, 'Should have Claude 3.5 Sonnet');
const hasNonexistent = await provider.hasModel('nonexistent-model');
assert.ok(!hasNonexistent, 'Should not have nonexistent model');
await provider.shutdown();
});
it('should report capabilities', async () => {
const provider = new AnthropicProvider();
await provider.initialize({ apiKey: 'test-key' });
const capabilities = provider.getCapabilities();
assert.ok(capabilities, 'Should have capabilities');
assert.strictEqual(capabilities.streaming, true);
assert.strictEqual(capabilities.functionCalling, true);
assert.strictEqual(capabilities.vision, true);
assert.strictEqual(capabilities.systemMessages, true);
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
await provider.shutdown();
});
it('should estimate token count', async () => {
const provider = new AnthropicProvider();
await provider.initialize({ apiKey: 'test-key' });
const text = 'Hello, world!';
const tokens = await provider.countTokens(text);
assert.ok(tokens > 0, 'Should return positive token count');
assert.ok(tokens <= text.length, 'Token count should be reasonable');
await provider.shutdown();
});
it('should require initialization before completion', async () => {
const provider = new AnthropicProvider();
await assert.rejects(
async () => {
await provider.complete({
model: 'claude-3-5-sonnet-20241022',
messages: [{ role: 'user', content: 'Hello' }],
});
},
{
message: /not initialized/,
},
'Should reject if not initialized'
);
});
it('should validate completion request', async () => {
const provider = new AnthropicProvider();
await provider.initialize({ apiKey: 'test-key' });
await assert.rejects(
async () => {
await provider.complete({
model: '',
messages: [],
});
},
{
message: /Model is required|At least one message/,
},
'Should reject invalid request'
);
await provider.shutdown();
});
});
describe('OpenAI Provider', () => {
let provider;
it('should create provider instance', () => {
provider = new OpenAIProvider();
assert.ok(provider, 'Provider should be created');
assert.strictEqual(provider.name, 'openai');
assert.strictEqual(provider.displayName, 'OpenAI');
});
it('should require API key for initialization', async () => {
const provider = new OpenAIProvider();
await assert.rejects(
async () => {
await provider.initialize({});
},
{
message: 'OpenAI API key is required',
},
'Should reject without API key'
);
});
it('should initialize with API key', async () => {
const provider = new OpenAIProvider();
await assert.doesNotReject(async () => {
await provider.initialize({ apiKey: 'test-key' });
}, 'Should initialize with API key');
await provider.shutdown();
});
it('should list available models', async () => {
const provider = new OpenAIProvider();
await provider.initialize({ apiKey: 'test-key' });
const models = await provider.getModels();
assert.ok(Array.isArray(models), 'Should return array of models');
assert.ok(models.length > 0, 'Should have at least one model');
// Check for GPT-4 Turbo
const gpt4turbo = models.find((m) => m.id === 'gpt-4-turbo-2024-04-09');
assert.ok(gpt4turbo, 'Should have GPT-4 Turbo');
assert.strictEqual(gpt4turbo.name, 'GPT-4 Turbo');
// Check for GPT-4o
const gpt4o = models.find((m) => m.id === 'gpt-4o');
assert.ok(gpt4o, 'Should have GPT-4o');
// Check for GPT-3.5 Turbo
const gpt35 = models.find((m) => m.id === 'gpt-3.5-turbo-0125');
assert.ok(gpt35, 'Should have GPT-3.5 Turbo');
await provider.shutdown();
});
it('should get specific model info', async () => {
const provider = new OpenAIProvider();
await provider.initialize({ apiKey: 'test-key' });
const model = await provider.getModel('gpt-4-turbo-2024-04-09');
assert.ok(model, 'Should return model');
assert.strictEqual(model.id, 'gpt-4-turbo-2024-04-09');
assert.strictEqual(model.name, 'GPT-4 Turbo');
assert.ok(model.capabilities, 'Should have capabilities');
await provider.shutdown();
});
it('should check if model exists', async () => {
const provider = new OpenAIProvider();
await provider.initialize({ apiKey: 'test-key' });
const hasGPT4 = await provider.hasModel('gpt-4-turbo-2024-04-09');
assert.ok(hasGPT4, 'Should have GPT-4 Turbo');
const hasNonexistent = await provider.hasModel('nonexistent-model');
assert.ok(!hasNonexistent, 'Should not have nonexistent model');
await provider.shutdown();
});
it('should report capabilities', async () => {
const provider = new OpenAIProvider();
await provider.initialize({ apiKey: 'test-key' });
const capabilities = provider.getCapabilities();
assert.ok(capabilities, 'Should have capabilities');
assert.strictEqual(capabilities.streaming, true);
assert.strictEqual(capabilities.functionCalling, true);
assert.strictEqual(capabilities.vision, true);
assert.strictEqual(capabilities.jsonMode, true);
assert.strictEqual(capabilities.systemMessages, true);
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
await provider.shutdown();
});
it('should estimate token count', async () => {
const provider = new OpenAIProvider();
await provider.initialize({ apiKey: 'test-key' });
const text = 'Hello, world!';
const tokens = await provider.countTokens(text);
assert.ok(tokens > 0, 'Should return positive token count');
assert.ok(tokens <= text.length, 'Token count should be reasonable');
await provider.shutdown();
});
it('should have OpenAI provider registered', () => {
const providers = listProviders();
const openai = providers.find((p) => p.name === 'openai');
assert.ok(openai, 'OpenAI provider should be registered');
assert.strictEqual(openai.displayName, 'OpenAI');
});
});
describe('Google Provider', () => {
let provider;
it('should create provider instance', () => {
provider = new GoogleProvider();
assert.ok(provider, 'Provider should be created');
assert.strictEqual(provider.name, 'google');
assert.strictEqual(provider.displayName, 'Google');
});
it('should require API key for initialization', async () => {
const provider = new GoogleProvider();
await assert.rejects(
async () => {
await provider.initialize({});
},
{
message: 'Google API key is required',
},
'Should reject without API key'
);
});
it('should initialize with API key', async () => {
const provider = new GoogleProvider();
await assert.doesNotReject(async () => {
await provider.initialize({ apiKey: 'test-key' });
}, 'Should initialize with API key');
await provider.shutdown();
});
it('should list available models', async () => {
const provider = new GoogleProvider();
await provider.initialize({ apiKey: 'test-key' });
const models = await provider.getModels();
assert.ok(Array.isArray(models), 'Should return array of models');
assert.ok(models.length > 0, 'Should have at least one model');
// Check for Gemini 2.0 Flash
const gemini2 = models.find((m) => m.id === 'gemini-2.0-flash-exp');
assert.ok(gemini2, 'Should have Gemini 2.0 Flash');
assert.strictEqual(gemini2.name, 'Gemini 2.0 Flash (Experimental)');
await provider.shutdown();
});
it('should get specific model info', async () => {
const provider = new GoogleProvider();
await provider.initialize({ apiKey: 'test-key' });
const model = await provider.getModel('gemini-1.5-pro');
assert.ok(model, 'Should return model');
assert.strictEqual(model.id, 'gemini-1.5-pro');
assert.strictEqual(model.name, 'Gemini 1.5 Pro');
assert.ok(model.capabilities, 'Should have capabilities');
await provider.shutdown();
});
it('should check if model exists', async () => {
const provider = new GoogleProvider();
await provider.initialize({ apiKey: 'test-key' });
const hasGemini = await provider.hasModel('gemini-1.5-pro');
assert.ok(hasGemini, 'Should have Gemini 1.5 Pro');
const hasNonexistent = await provider.hasModel('nonexistent-model');
assert.ok(!hasNonexistent, 'Should not have nonexistent model');
await provider.shutdown();
});
it('should report capabilities', async () => {
const provider = new GoogleProvider();
await provider.initialize({ apiKey: 'test-key' });
const capabilities = provider.getCapabilities();
assert.ok(capabilities, 'Should have capabilities');
assert.strictEqual(capabilities.streaming, true);
assert.strictEqual(capabilities.functionCalling, true);
assert.strictEqual(capabilities.vision, true);
assert.strictEqual(capabilities.systemMessages, true);
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
await provider.shutdown();
});
it('should estimate token count', async () => {
const provider = new GoogleProvider();
await provider.initialize({ apiKey: 'test-key' });
const text = 'Hello, world!';
const tokens = await provider.countTokens(text);
assert.ok(tokens > 0, 'Should return positive token count');
assert.ok(tokens <= text.length, 'Token count should be reasonable');
await provider.shutdown();
});
it('should have Google provider registered', () => {
const providers = listProviders();
const google = providers.find((p) => p.name === 'google');
assert.ok(google, 'Google provider should be registered');
assert.strictEqual(google.displayName, 'Google');
});
});
describe('DeepSeek Provider', () => {
let provider;
it('should create provider instance', () => {
provider = new DeepSeekProvider();
assert.ok(provider, 'Provider should be created');
assert.strictEqual(provider.name, 'deepseek');
assert.strictEqual(provider.displayName, 'DeepSeek');
});
it('should require API key for initialization', async () => {
const provider = new DeepSeekProvider();
await assert.rejects(
async () => {
await provider.initialize({});
},
{
message: 'DeepSeek API key is required',
},
'Should reject without API key'
);
});
it('should initialize with API key', async () => {
const provider = new DeepSeekProvider();
await assert.doesNotReject(async () => {
await provider.initialize({ apiKey: 'test-key' });
}, 'Should initialize with API key');
await provider.shutdown();
});
it('should list available models', async () => {
const provider = new DeepSeekProvider();
await provider.initialize({ apiKey: 'test-key' });
const models = await provider.getModels();
assert.ok(Array.isArray(models), 'Should return array of models');
assert.ok(models.length > 0, 'Should have at least one model');
// Check for DeepSeek Chat
const chat = models.find((m) => m.id === 'deepseek-chat');
assert.ok(chat, 'Should have DeepSeek Chat');
assert.strictEqual(chat.name, 'DeepSeek Chat');
// Check for DeepSeek Coder
const coder = models.find((m) => m.id === 'deepseek-coder');
assert.ok(coder, 'Should have DeepSeek Coder');
await provider.shutdown();
});
it('should get specific model info', async () => {
const provider = new DeepSeekProvider();
await provider.initialize({ apiKey: 'test-key' });
const model = await provider.getModel('deepseek-chat');
assert.ok(model, 'Should return model');
assert.strictEqual(model.id, 'deepseek-chat');
assert.strictEqual(model.name, 'DeepSeek Chat');
assert.ok(model.capabilities, 'Should have capabilities');
await provider.shutdown();
});
it('should check if model exists', async () => {
const provider = new DeepSeekProvider();
await provider.initialize({ apiKey: 'test-key' });
const hasChat = await provider.hasModel('deepseek-chat');
assert.ok(hasChat, 'Should have DeepSeek Chat');
const hasNonexistent = await provider.hasModel('nonexistent-model');
assert.ok(!hasNonexistent, 'Should not have nonexistent model');
await provider.shutdown();
});
it('should report capabilities', async () => {
const provider = new DeepSeekProvider();
await provider.initialize({ apiKey: 'test-key' });
const capabilities = provider.getCapabilities();
assert.ok(capabilities, 'Should have capabilities');
assert.strictEqual(capabilities.streaming, true);
assert.strictEqual(capabilities.functionCalling, true);
assert.strictEqual(capabilities.systemMessages, true);
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
await provider.shutdown();
});
it('should estimate token count', async () => {
const provider = new DeepSeekProvider();
await provider.initialize({ apiKey: 'test-key' });
const text = 'Hello, world!';
const tokens = await provider.countTokens(text);
assert.ok(tokens > 0, 'Should return positive token count');
assert.ok(tokens <= text.length, 'Token count should be reasonable');
await provider.shutdown();
});
it('should have DeepSeek provider registered', () => {
const providers = listProviders();
const deepseek = providers.find((p) => p.name === 'deepseek');
assert.ok(deepseek, 'DeepSeek provider should be registered');
assert.strictEqual(deepseek.displayName, 'DeepSeek');
});
});
describe('Ollama Provider', () => {
let provider;
it('should create provider instance', () => {
provider = new OllamaProvider();
assert.ok(provider, 'Provider should be created');
assert.strictEqual(provider.name, 'ollama');
assert.strictEqual(provider.displayName, 'Ollama');
});
it('should allow initialization without API key', async () => {
const provider = new OllamaProvider();
await assert.doesNotReject(async () => {
await provider.initialize({});
}, 'Should initialize without API key');
await provider.shutdown();
});
it('should initialize with custom baseUrl', async () => {
const provider = new OllamaProvider();
await assert.doesNotReject(async () => {
await provider.initialize({ baseUrl: 'http://custom:11434' });
}, 'Should initialize with custom baseUrl');
await provider.shutdown();
});
it('should reject invalid baseUrl', async () => {
const provider = new OllamaProvider();
await assert.rejects(
async () => {
await provider.initialize({ baseUrl: 'invalid-url' });
},
{
message: /must start with http/,
},
'Should reject invalid baseUrl'
);
});
it('should list available models', async () => {
const provider = new OllamaProvider();
await provider.initialize({});
const models = await provider.getModels();
assert.ok(Array.isArray(models), 'Should return array of models');
// Ollama may have 0 models if none installed, so just check array type
await provider.shutdown();
});
it('should report capabilities', async () => {
const provider = new OllamaProvider();
await provider.initialize({});
const capabilities = provider.getCapabilities();
assert.ok(capabilities, 'Should have capabilities');
assert.strictEqual(capabilities.streaming, true);
assert.strictEqual(capabilities.jsonMode, true);
assert.strictEqual(capabilities.systemMessages, true);
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
await provider.shutdown();
});
it('should estimate token count', async () => {
const provider = new OllamaProvider();
await provider.initialize({});
const text = 'Hello, world!';
const tokens = await provider.countTokens(text);
assert.ok(tokens > 0, 'Should return positive token count');
assert.ok(tokens <= text.length, 'Token count should be reasonable');
await provider.shutdown();
});
it('should have Ollama provider registered', () => {
const providers = listProviders();
const ollama = providers.find((p) => p.name === 'ollama');
assert.ok(ollama, 'Ollama provider should be registered');
assert.strictEqual(ollama.displayName, 'Ollama');
});
it('should have free pricing for local models', async () => {
const provider = new OllamaProvider();
await provider.initialize({});
const models = await provider.getModels();
// All Ollama models should be free (0 cost)
for (const model of models) {
assert.strictEqual(model.inputTokenCost, 0, 'Input tokens should be free');
assert.strictEqual(model.outputTokenCost, 0, 'Output tokens should be free');
}
await provider.shutdown();
});
});
describe('Mistral AI Provider', () => {
let provider;
it('should create provider instance', () => {
provider = new MistralProvider();
assert.ok(provider, 'Provider should be created');
assert.strictEqual(provider.name, 'mistral');
assert.strictEqual(provider.displayName, 'Mistral AI');
});
it('should require API key for initialization', async () => {
const provider = new MistralProvider();
await assert.rejects(
async () => {
await provider.initialize({});
},
{
message: 'Mistral API key is required',
},
'Should reject without API key'
);
});
it('should initialize with API key', async () => {
const provider = new MistralProvider();
await assert.doesNotReject(async () => {
await provider.initialize({ apiKey: 'test-key' });
}, 'Should initialize with API key');
await provider.shutdown();
});
it('should list available models', async () => {
const provider = new MistralProvider();
await provider.initialize({ apiKey: 'test-key' });
const models = await provider.getModels();
assert.ok(Array.isArray(models), 'Should return array of models');
assert.ok(models.length > 0, 'Should have at least one model');
// Check for Mistral Large
const large = models.find((m) => m.id === 'mistral-large-latest');
assert.ok(large, 'Should have Mistral Large');
assert.strictEqual(large.name, 'Mistral Large');
await provider.shutdown();
});
it('should get specific model info', async () => {
const provider = new MistralProvider();
await provider.initialize({ apiKey: 'test-key' });
const model = await provider.getModel('mistral-small-latest');
assert.ok(model, 'Should return model');
assert.strictEqual(model.id, 'mistral-small-latest');
assert.strictEqual(model.name, 'Mistral Small');
assert.ok(model.capabilities, 'Should have capabilities');
await provider.shutdown();
});
it('should report capabilities', async () => {
const provider = new MistralProvider();
await provider.initialize({ apiKey: 'test-key' });
const capabilities = provider.getCapabilities();
assert.ok(capabilities, 'Should have capabilities');
assert.strictEqual(capabilities.streaming, true);
assert.strictEqual(capabilities.functionCalling, true);
assert.strictEqual(capabilities.jsonMode, true);
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
await provider.shutdown();
});
it('should have Mistral provider registered', () => {
const providers = listProviders();
const mistral = providers.find((p) => p.name === 'mistral');
assert.ok(mistral, 'Mistral provider should be registered');
assert.strictEqual(mistral.displayName, 'Mistral AI');
});
});
describe('Groq Provider', () => {
let provider;
it('should create provider instance', () => {
provider = new GroqProvider();
assert.ok(provider, 'Provider should be created');
assert.strictEqual(provider.name, 'groq');
assert.strictEqual(provider.displayName, 'Groq');
});
it('should require API key for initialization', async () => {
const provider = new GroqProvider();
await assert.rejects(
async () => {
await provider.initialize({});
},
{
message: 'Groq API key is required',
},
'Should reject without API key'
);
});
it('should initialize with API key', async () => {
const provider = new GroqProvider();
await assert.doesNotReject(async () => {
await provider.initialize({ apiKey: 'test-key' });
}, 'Should initialize with API key');
await provider.shutdown();
});
it('should list available models', async () => {
const provider = new GroqProvider();
await provider.initialize({ apiKey: 'test-key' });
const models = await provider.getModels();
assert.ok(Array.isArray(models), 'Should return array of models');
assert.ok(models.length > 0, 'Should have at least one model');
// Check for Llama 3.3 70B
const llama = models.find((m) => m.id === 'llama-3.3-70b-versatile');
assert.ok(llama, 'Should have Llama 3.3 70B');
assert.strictEqual(llama.name, 'Llama 3.3 70B');
await provider.shutdown();
});
it('should get specific model info', async () => {
const provider = new GroqProvider();
await provider.initialize({ apiKey: 'test-key' });
const model = await provider.getModel('llama-3.1-8b-instant');
assert.ok(model, 'Should return model');
assert.strictEqual(model.id, 'llama-3.1-8b-instant');
assert.strictEqual(model.name, 'Llama 3.1 8B Instant');
assert.ok(model.capabilities, 'Should have capabilities');
await provider.shutdown();
});
it('should report capabilities', async () => {
const provider = new GroqProvider();
await provider.initialize({ apiKey: 'test-key' });
const capabilities = provider.getCapabilities();
assert.ok(capabilities, 'Should have capabilities');
assert.strictEqual(capabilities.streaming, true);
assert.strictEqual(capabilities.functionCalling, true);
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
await provider.shutdown();
});
it('should have Groq provider registered', () => {
const providers = listProviders();
const groq = providers.find((p) => p.name === 'groq');
assert.ok(groq, 'Groq provider should be registered');
assert.strictEqual(groq.displayName, 'Groq');
});
});
describe('Cohere Provider', () => {
let provider;
it('should create provider instance', () => {
provider = new CohereProvider();
assert.ok(provider, 'Provider should be created');
assert.strictEqual(provider.name, 'cohere');
assert.strictEqual(provider.displayName, 'Cohere');
});
it('should require API key for initialization', async () => {
const provider = new CohereProvider();
await assert.rejects(
async () => {
await provider.initialize({});
},
{
message: 'Cohere API key is required',
},
'Should reject without API key'
);
});
it('should initialize with API key', async () => {
const provider = new CohereProvider();
await assert.doesNotReject(async () => {
await provider.initialize({ apiKey: 'test-key' });
}, 'Should initialize with API key');
await provider.shutdown();
});
it('should list available models', async () => {
const provider = new CohereProvider();
await provider.initialize({ apiKey: 'test-key' });
const models = await provider.getModels();
assert.ok(Array.isArray(models), 'Should return array of models');
assert.ok(models.length > 0, 'Should have at least one model');
// Check for Command R+
const commandRPlus = models.find((m) => m.id === 'command-r-plus');
assert.ok(commandRPlus, 'Should have Command R+');
assert.strictEqual(commandRPlus.name, 'Command R+');
await provider.shutdown();
});
it('should get specific model info', async () => {
const provider = new CohereProvider();
await provider.initialize({ apiKey: 'test-key' });
const model = await provider.getModel('command-r');
assert.ok(model, 'Should return model');
assert.strictEqual(model.id, 'command-r');
assert.strictEqual(model.name, 'Command R');
assert.ok(model.capabilities, 'Should have capabilities');
await provider.shutdown();
});
it('should report capabilities', async () => {
const provider = new CohereProvider();
await provider.initialize({ apiKey: 'test-key' });
const capabilities = provider.getCapabilities();
assert.ok(capabilities, 'Should have capabilities');
assert.strictEqual(capabilities.streaming, true);
assert.strictEqual(capabilities.functionCalling, true);
assert.strictEqual(capabilities.topK, true);
assert.ok(capabilities.maxContextTokens > 0, 'Should have max context tokens');
await provider.shutdown();
});
it('should have Cohere provider registered', () => {
const providers = listProviders();
const cohere = providers.find((p) => p.name === 'cohere');
assert.ok(cohere, 'Cohere provider should be registered');
assert.strictEqual(cohere.displayName, 'Cohere');
});
});
console.log('✅ All provider tests passed');