sbroenne__mcp-server-excel/tests

ExcelMcp Tests

⚠️ No Traditional Unit Tests: ExcelMcp has no unit tests. Integration tests ARE our unit tests because Excel COM cannot be meaningfully mocked. See docs/ADR-001-NO-UNIT-TESTS.md for full architectural rationale.

Quick Start

# Development (fast feedback - excludes VBA tests)
dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA&Feature!=VBATrust"

# Pre-commit (comprehensive - excludes VBA tests)
dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA&Feature!=VBATrust"

# Session/batch changes (MANDATORY when modifying session/batch code)
dotnet test --filter "RunType=OnDemand"

# VBA tests (manual only - requires VBA trust enabled)
dotnet test --filter "(Feature=VBA|Feature=VBATrust)&RunType!=OnDemand"

Documentation

For complete testing guidance, see:

• Testing Strategy - Quick reference, templates, common mistakes
• Critical Rules - Mandatory development rules (Rule 14: SaveAsync)

Test Architecture

tests/
├── ExcelMcp.Core.Tests/           # Core business logic (Integration)
├── ExcelMcp.McpServer.Tests/      # MCP protocol layer (Integration)
├── ExcelMcp.McpServer.LLM.Tests/  # LLM agent behavior validation (Manual)
├── ExcelMcp.CLI.Tests/            # CLI wrapper (Integration)
└── ExcelMcp.ComInterop.Tests/     # COM utilities (OnDemand)

Test Categories

Category	Speed	Requirements	Run By Default
Integration	Medium (10-20 min)	Excel + Windows	✅ Yes (local)
OnDemand	Slow (3-5 min)	Excel + Windows	❌ No (explicit only)
LLM Tests	Slow (varies)	Excel + Azure OpenAI	❌ No (manual only)

Feature-Specific Tests

# Test specific feature only
dotnet test --filter "Feature=PowerQuery&RunType!=OnDemand"
dotnet test --filter "Feature=DataModel&RunType!=OnDemand"
dotnet test --filter "Feature=Tables&RunType!=OnDemand"
dotnet test --filter "Feature=PivotTables&RunType!=OnDemand"
dotnet test --filter "Feature=Ranges&RunType!=OnDemand"
dotnet test --filter "Feature=Connections&RunType!=OnDemand"

When to Run Which Tests

Scenario	Command
Daily development	dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA"
Before commit	dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA"
Modified session/batch code	dotnet test --filter "RunType=OnDemand" (see Rule 3)
VBA development	dotnet test --filter "(Feature=VBA|Feature=VBATrust)&RunType!=OnDemand"
LLM behavior validation	See LLM Tests section below

LLM Tests

The ExcelMcp.McpServer.LLM.Tests project validates that AI agents correctly use Excel MCP Server tools using agent-benchmark.

When to Run LLM Tests

• Manual/on-demand only - Not part of CI/CD
• After changing tool descriptions or adding new tools
• To validate LLM behavior patterns (e.g., incremental updates vs rebuild)

Running LLM Tests

# From tests/ExcelMcp.McpServer.LLM.Tests/
.\Run-LLMTests.ps1 -Build

# Run specific scenario
.\Run-LLMTests.ps1 -Scenario excel-file-worksheet-test.yaml

Prerequisites

• AZURE_OPENAI_ENDPOINT environment variable
• Windows desktop with Excel installed
• agent-benchmark (auto-downloaded on first run)

See LLM Tests README for complete documentation.

VBA Testing

Why VBA Tests Are Excluded by Default

VBA tests are excluded from normal test runs because:

1. Stable codebase - VBA features are mature with minimal changes
2. Performance - Excluding VBA tests makes integration tests ~25% faster (10-15 min vs 15-20 min)
3. Special requirements - VBA tests require VBA trust enabled in Excel settings
4. Opt-in model - Explicit testing when VBA code changes, rather than every commit

When to Run VBA Tests

Run VBA tests manually when:

• Modifying VBA-related code (ScriptCommands, VbaTrustDetection)
• Adding new VBA features
• Before releasing VBA-related changes
• Troubleshooting VBA-specific issues

How to Run VBA Tests

# Run ONLY VBA tests
dotnet test --filter "(Feature=VBA|Feature=VBATrust)&RunType!=OnDemand"

# Run ALL tests including VBA (takes longer)
dotnet test --filter "Category=Integration&RunType!=OnDemand"

VBA Test Files

All VBA tests are tagged with [Trait("Feature", "VBA")] or [Trait("Feature", "VBATrust")]:

tests/ExcelMcp.Core.Tests/Integration/Commands/Script/
  - ScriptCommandsTests.cs
  - ScriptCommandsTests.Lifecycle.cs
  - VbaTrustDetectionTests.ScriptCommands.cs
  - VbaTrustDetectionTests.cs

tests/ExcelMcp.CLI.Tests/Integration/Commands/
  - ScriptAndSetupCommandsTests.cs

VBA Trust Setup

VBA tests require VBA trust enabled in Excel:

# Enable VBA trust (required for VBA tests)
Set-ItemProperty -Path "HKCU:\Software\Microsoft\Office\16.0\Excel\Security" -Name "AccessVBOM" -Value 1

# Verify setting
Get-ItemProperty -Path "HKCU:\Software\Microsoft\Office\16.0\Excel\Security" -Name "AccessVBOM"

Security Note: Only enable VBA trust in development environments. Production systems should keep this disabled.

Key Principles

• ✅ File Isolation - Each test creates unique file (no sharing)
• ✅ Binary Assertions - Pass OR fail, never "accept both"
• ✅ Verify Excel State - Always verify actual Excel state after operations
• ❌ No SaveAsync - Unless testing persistence (see Rule 14)

Getting Help

• Test failures: Check test output for detailed error messages
• Excel issues: Ensure Excel 2016+ installed and activated
• Session/batch issues: Run OnDemand tests to verify cleanup
• Writing tests: See Testing Strategy