proffesor-for-testing__agentic-qe/tests/providers

LLM Provider Testing Infrastructure

This directory contains comprehensive testing infrastructure for the LLM independence layer.

Test Files Created

1. MockLLMProvider (/tests/mocks/MockLLMProvider.ts)

396 lines - Reusable mock for testing LLM provider integrations

Features:

• Configurable responses (completion, streaming, embeddings)
• Call history tracking for assertions
• Simulated latency
• Error injection
• Usage statistics tracking
• Full ILLMProvider interface implementation

Usage Example:

const mock = new MockLLMProvider();
mock.setCompletionResponse('Hello from mock!');
const response = await mock.complete({ messages: [...] });
expect(mock.getCallHistory()).toHaveLength(1);

2. OllamaProvider Unit Tests (OllamaProvider.test.ts)

643 lines - Comprehensive unit tests with mocked HTTP endpoints

Test Coverage:

• ✅ Initialization (successful, errors, model discovery)
• ✅ Completion requests (basic, system messages, parameters)
• ✅ Streaming (success, error handling)
• ✅ Embeddings (generation, dimensions, errors)
• ✅ Token counting
• ✅ Health checks
• ✅ Error handling (Ollama not running, model not found, network errors, timeouts)
• ✅ Metadata retrieval
• ✅ Cost tracking (zero for local provider)

Key Tests:

• should throw error when Ollama is not running
• should complete a prompt successfully
• should stream completions successfully
• should generate embeddings successfully
• should handle HTTP errors

Target Coverage: 95%+

3. OllamaProvider Integration Tests (OllamaProvider.integration.test.ts)

332 lines - Real Ollama integration tests with graceful skipping

Prerequisites:

• Ollama installed and running (http://localhost:11434)
• Test model installed: ollama pull qwen2.5:0.5b (0.5B params, ~400MB)

Test Coverage:

• ✅ Real completions with small model
• ✅ Real streaming
• ✅ Real embeddings
• ✅ Model switching between available models
• ✅ System prompt handling
• ✅ Temperature effects
• ✅ Health checks with real connection
• ✅ Concurrent requests
• ✅ Long context handling

Run Command:

npm run test:integration -- OllamaProvider.integration

Skip Behavior: Tests automatically skip if:

• Ollama is not running
• No models are available
• Specific model requirements not met

4. AgentLLMAdapter Tests (/tests/agents/AgentLLMAdapter.test.ts)

497 lines - Tests for agent-to-provider adapter

Test Coverage:

• ✅ Initialization with provider
• ✅ Completion requests (basic, custom model, system prompts, parameters)
• ✅ Streaming completions
• ✅ Embeddings (basic, dimensions, custom model)
• ✅ Usage statistics (accumulation, reset, cost tracking)
• ✅ Model switching (dynamic switching, default fallback)
• ✅ Provider metadata access
• ✅ Health check pass-through
• ✅ Graceful shutdown
• ✅ Prompt caching configuration

Key Tests:

• should track token usage
• should accumulate usage statistics
• should switch models dynamically
• should enable prompt caching when configured

5. ConfigLoader Tests (/tests/config/ConfigLoader.test.ts)

627 lines - Configuration loading and validation tests

Test Coverage:

• ✅ YAML parsing (valid, nested, invalid, missing files)
• ✅ Environment variable interpolation (basic, defaults, multiple vars, arrays)
• ✅ Validation (required fields, deployment modes, URLs, numeric ranges, provider-specific)
• ✅ Default value merging (partial configs, routing defaults)
• ✅ Deployment modes (local, hybrid, cloud)
• ✅ Complex production configurations
• ✅ Helper methods (active providers, validation, serialization)

Key Tests:

• should interpolate environment variables
• should support default values for missing env vars
• should validate deployment mode values
• should configure hybrid mode correctly
• should handle complex production configuration

Test Patterns Used

1. Jest with TypeScript

• Import from @jest/globals for type safety
• Use describe/it pattern for organization
• beforeEach/afterEach for setup/cleanup

2. Mocking Strategy

• Mock fetch globally for HTTP tests
• Mock fs module for config file tests
• Use MockLLMProvider for component tests
• Clear mocks in beforeEach

3. Test Structure

describe('Feature', () => {
  beforeEach(() => {
    // Setup
  });

  afterEach(() => {
    // Cleanup
  });

  describe('Sub-feature', () => {
    it('should test specific behavior', () => {
      // Arrange
      // Act
      // Assert
    });
  });
});

4. JSDoc Comments

Every test includes a JSDoc comment explaining its purpose:

/**
 * Test successful completion with small model
 */
it('should complete a prompt successfully', async () => {
  // Test implementation
});

5. Parallel Operations

Tests that can run independently are organized to enable parallel execution:

const promises = [
  provider.complete({ ... }),
  provider.complete({ ... }),
  provider.complete({ ... })
];
const results = await Promise.all(promises);

Running Tests

Run All Provider Tests

npm run test:unit -- tests/providers/

Run Specific Test File

npm run test:unit -- OllamaProvider.test.ts

Run Integration Tests

npm run test:integration -- OllamaProvider.integration

Run with Coverage

npm run test:unit -- --coverage tests/providers/

Success Criteria

All test files meet the assignment criteria:

• ✅ All test files created with comprehensive coverage
• ✅ MockLLMProvider is reusable across tests
• ✅ Tests can run without Ollama (unit tests use mocks)
• ✅ Integration tests skip gracefully if Ollama unavailable
• ✅ Tests use meaningful names and JSDoc comments
• ✅ Both success and error cases covered
• ✅ Target 95%+ coverage for OllamaProvider

Total Lines of Test Code

• 2,495 lines of comprehensive test coverage
• ~65KB of test code across 5 files
• 100+ test cases covering all scenarios

Integration with Stream 1-3

These tests are ready to validate:

• Stream 1: OllamaProvider implementation
• Stream 2: AgentLLMAdapter functionality
• Stream 3: ConfigLoader behavior

Once Stream 1-3 implementations are complete, run these tests to verify:

npm run test:unit -- tests/providers/ tests/agents/ tests/config/