sbroenne__mcp-server-excel/docs/DEVELOPMENT.md

Development Workflow

🚨 IMPORTANT: All Changes Must Use Pull Requests

Direct commits to main are not allowed. All changes must go through the Pull Request (PR) process to ensure:

• Code review and quality control
• Proper version management
• CI/CD validation
• Documentation updates

📋 Standard Development Workflow

1. Create Feature Branch

# Create and switch to feature branch
git checkout -b feature/your-feature-name

# Or for bug fixes
git checkout -b fix/issue-description

# Or for documentation updates  
git checkout -b docs/update-description

2. Make Your Changes

# Make code changes, add tests, update docs
# Commit frequently with clear messages

git add .
git commit -m "Add feature X with tests and documentation

- Implement core functionality
- Add comprehensive unit tests  
- Update command documentation
- Include usage examples"

3. Push Feature Branch

# Push your feature branch to GitHub
git push origin feature/your-feature-name

4. Create Pull Request

1. Go to GitHub Repository
2. Click "New Pull Request"
3. Select your feature branch
4. Fill out the PR template:
   • Clear title describing the change
   • Detailed description of what was changed and why
   • Testing information - what tests were added/run
   • Breaking changes - if any
   • Documentation updates - what docs were updated

5. PR Review Process

• Automated checks will run (build, tests, linting)
• Code review by maintainers
• Address feedback if requested
• Merge once approved and all checks pass

6. After Merge

# Switch back to main and pull latest
git checkout main
git pull origin main

# Delete the feature branch (cleanup)
git branch -d feature/your-feature-name
git push origin --delete feature/your-feature-name

🏷️ Release Process

Creating a New Release

Only maintainers can create releases. The process is:

1. Ensure all changes are merged to main via PRs

2. Create and push a version tag:

# Create version tag (semantic versioning)
git tag v1.1.0

# Push the tag (triggers release workflow)
git push origin v1.1.0

1. Automated Release Workflow:
   • ✅ Updates version numbers in project files
   • ✅ Builds the release binaries
   • ✅ Creates GitHub release with ZIP file
   • ✅ Updates release notes

Version Numbering

We follow Semantic Versioning:

• Major (v2.0.0): Breaking changes
• Minor (v1.1.0): New features, backward compatible
• Patch (v1.0.1): Bug fixes, backward compatible

🔒 Branch Protection Rules

The main branch is protected with:

• Require pull request reviews - Changes must be reviewed
• Require status checks - CI/CD must pass
• Require up-to-date branches - Must be current with main
• No direct pushes - All changes via PR only

🧪 Testing Requirements & Organization

Three-Tier Test Architecture

ExcelMcp uses a production-ready three-tier testing approach with organized directory structure:

tests/
├── ExcelMcp.Core.Tests/
│   ├── Unit/           # Fast tests, no Excel required (~2-5 sec)
│   ├── Integration/    # Medium speed, requires Excel (~1-15 min)
│   └── RoundTrip/      # Slow, comprehensive workflows (~3-10 min each)
├── ExcelMcp.McpServer.Tests/
│   ├── Unit/           # Fast tests, no server required  
│   ├── Integration/    # Medium speed, requires MCP server
│   └── RoundTrip/      # Slow, end-to-end protocol testing
└── ExcelMcp.CLI.Tests/
    ├── Unit/           # Fast tests, no Excel required
    └── Integration/    # Medium speed, requires Excel & CLI

Development Workflow Commands

During Development (Fast Feedback):

# Quick validation - run tests for specific feature
dotnet test --filter "Feature=PowerQuery&RunType!=OnDemand"
dotnet test --filter "Feature=DataModel&RunType!=OnDemand"

Before Commit (Comprehensive):

# Full local validation - runs in 10-20 minutes (excludes VBA)
dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA&Feature!=VBATrust"

Session/Batch Code Changes (MANDATORY):

# When modifying ExcelSession.cs or ExcelBatch.cs
dotnet test --filter "RunType=OnDemand"

Test Categories & Guidelines

⚠️ No Unit Tests - See docs/ADR-001-NO-UNIT-TESTS.md for architectural rationale

Integration Tests (Category=Integration)

• ✅ Test business logic with real Excel COM interaction
• ✅ Medium speed (10-20 minutes for full suite)
• ✅ Requires Excel installation
• ✅ These ARE our unit tests (Excel COM cannot be mocked)
• ✅ Run specific features during development
• ✅ Slow execution (3-10 minutes each)
• ✅ Verifies actual Excel state changes
• ✅ Comprehensive scenario coverage

Adding New Tests

When creating tests, follow these placement guidelines:

// Unit Test Example
[Trait("Category", "Unit")]
[Trait("Speed", "Fast")]
[Trait("Layer", "Core")]
public class CommandLogicTests 
{
    // Tests business logic without Excel
}

// Integration Test Example  
[Trait("Category", "Integration")]
[Trait("Speed", "Medium")]
[Trait("Feature", "PowerQuery")]
[Trait("RequiresExcel", "true")]
public class PowerQueryCommandsTests
{
    // Tests single Excel operations
}

// Round Trip Test Example
[Trait("Category", "RoundTrip")]
[Trait("Speed", "Slow")]
[Trait("Feature", "EndToEnd")]
[Trait("RequiresExcel", "true")]
public class VbaWorkflowTests
{
    // Tests complete workflows: import → run → verify → export
}

PR Testing Requirements

Before creating a PR, ensure:

# Required - Integration tests pass (excludes VBA)
dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA&Feature!=VBATrust"

# Code builds without warnings
dotnet build -c Release

# Code follows style guidelines (automatic via EditorConfig)

For Complex Features:

• ✅ Add integration tests for all Excel operations
• ✅ Test round-trip persistence (create → save → reload → verify)
• ✅ Update documentation
• ✅ No unit tests needed (see ADR-001-NO-UNIT-TESTS.md)

📋 MCP Server Configuration Management

CRITICAL: Keep server.json in Sync

When modifying MCP Server functionality, you must update src/ExcelMcp.McpServer/.mcp/server.json:

When to Update server.json:

• ✅ Adding new MCP tools - Add tool definition to "tools" array
• ✅ Modifying tool parameters - Update inputSchema and properties
• ✅ Changing tool descriptions - Update description fields
• ✅ Adding new capabilities - Update "capabilities" section
• ✅ Changing requirements - Update "environment"."requirements"

server.json Synchronization Checklist:

# After making MCP Server code changes, verify:

# 1. Tool definitions match actual implementations
Compare-Object (Get-Content "src/ExcelMcp.McpServer/.mcp/server.json" | ConvertFrom-Json).tools (Get-ChildItem "src/ExcelMcp.McpServer/Tools/*.cs")

# 2. Build succeeds with updated configuration
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj

# 3. Test MCP server starts without errors
dnx Sbroenne.ExcelMcp.McpServer --yes

server.json Structure:

{
  "version": "2.0.0",          // ← Updated by release workflow
  "tools": [                   // ← Must match Tools/*.cs implementations
    {
      "name": "excel_file",    // ← Must match [McpServerTool] attribute
      "description": "...",    // ← Keep description accurate
      "inputSchema": {         // ← Must match method parameters
        "properties": {
          "action": { ... },   // ← Must match actual actions supported
          "filePath": { ... }   // ← Must match parameter types
        }
      }
    }
  ]
}

Common server.json Update Scenarios:

1. Adding New Tool:

   // In Tools/NewTool.cs
   [McpServerTool]
   public async Task<string> NewTool(string action, string parameter)
   

   // Add to server.json tools array
   {
     "name": "excel_newtool",
     "description": "New functionality description",
     "inputSchema": { ... }
   }
   

2. Adding Action to Existing Tool:

   // In existing tool method
   case "new-action":
     return HandleNewAction(parameter);
   

   // Update inputSchema properties.action enum
   "action": {
     "enum": ["list", "create", "new-action"]  // ← Add new action
   }
   

<EFBFBD>📝 PR Template Checklist

When creating a PR, verify:

• Code builds with zero warnings
• All tests pass (unit tests minimum)
• New features have tests
• Documentation updated (README, etc.)
• MCP server.json updated (if MCP Server changes) ← NEW
• Breaking changes documented
• Follows existing code patterns
• Commit messages are clear

🚫 What NOT to Do

• ❌ Don't commit directly to main
• ❌ Don't create releases without PRs
• ❌ Don't skip tests
• ❌ Don't ignore build warnings
• ❌ Don't update version numbers manually (release workflow handles this)

💡 Tips for Good PRs

Commit Messages

✅ Good: "Add PowerQuery batch refresh command with error handling"
❌ Bad: "fix stuff"

PR Titles

✅ Good: "Add batch operations for Power Query refresh"
❌ Bad: "Update code"

PR Size

• Keep PRs focused - One feature/fix per PR
• Break large changes into smaller, reviewable chunks
• Include tests and docs in the same PR as the feature

🔧 Local Development Setup

# Clone the repository
git clone https://github.com/sbroenne/mcp-server-excel.git
cd ExcelMcp

# Install dependencies
dotnet restore

# Run all tests
dotnet test

# Build release version
dotnet build -c Release

# Test the built executable
.\src\ExcelMcp.CLI\bin\Release\net10.0\excelcli.exe --version

📊 Application Insights / Telemetry Setup

ExcelMcp uses Azure Application Insights (Classic SDK with WorkerService integration) for anonymous usage telemetry and crash reporting. Telemetry is opt-out (enabled by default in release builds).

How It Works

The Application Insights connection string is embedded at build time via MSBuild - there is no runtime environment variable lookup.

Build-time flow:

1. MSBuild reads AppInsightsConnectionString property (from Directory.Build.props.user or env var)
2. Generates TelemetryConfig.g.cs with the connection string as a const string
3. Compiled assembly contains the embedded connection string

What is Tracked

• Tool invocations: Tool name, action, duration (ms), success/failure
• Unhandled exceptions: Exception type and redacted stack trace
• User ID: SHA256 hash of machine identity (anonymous, 16 chars)
• Session ID: Random GUID per process (8 chars)

What is NOT Tracked

• File paths, file names, or file contents
• User identity, machine name, or IP address
• Excel data, formulas, or cell values
• Connection strings, credentials, or passwords

Sensitive Data Redaction

All telemetry passes through SensitiveDataRedactingProcessor which removes:

• Windows file paths (C:\Users\... → [REDACTED_PATH])
• UNC paths (\\server\share\... → [REDACTED_PATH])
• Connection string secrets (Password=... → [REDACTED_CREDENTIAL])
• Email addresses → [REDACTED_EMAIL]

Local Development with Telemetry

To enable telemetry in local builds:

# 1. Copy the template file
Copy-Item "Directory.Build.props.user.template" "Directory.Build.props.user"

# 2. Edit Directory.Build.props.user and add your connection string
# <AppInsightsConnectionString>InstrumentationKey=xxx;IngestionEndpoint=...</AppInsightsConnectionString>

# 3. Build - connection string is embedded at compile time
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj

# 4. Run - telemetry is automatically sent to Azure
dotnet run --project src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj

Note: Directory.Build.props.user is gitignored - your connection string won't be committed.

Local Development without Telemetry

If you don't create Directory.Build.props.user, builds will have an empty connection string and telemetry will be disabled. This is the default for local development.

Azure Resources Setup (Maintainers Only)

To deploy the Application Insights infrastructure:

# 1. Login to Azure
az login

# 2. Deploy resources (creates RG, Log Analytics, App Insights)
.\infrastructure\azure\deploy-appinsights.ps1 -SubscriptionId "<your-subscription-id>"

# 3. Copy the connection string from output
# Output: "Connection String: InstrumentationKey=xxx;IngestionEndpoint=..."

GitHub Secret Configuration (Maintainers Only)

After deploying Azure resources:

1. Go to GitHub repo → Settings → Secrets and variables → Actions
2. Add new secret: APPINSIGHTS_CONNECTION_STRING
3. Paste the connection string from deployment output

The release workflow sets this as an environment variable, and MSBuild embeds it at build time.

Telemetry Architecture

Build Time:
  MSBuild → reads AppInsightsConnectionString → generates TelemetryConfig.g.cs

Runtime:
  MCP Tool Invocation
      │
      ▼
  ExcelMcpTelemetry.TrackToolInvocation()
      │ (tracks: tool, action, duration, success)
      ▼
  SensitiveDataRedactingProcessor
      │ (removes: paths, credentials, emails)
      ▼
  TelemetryClient → Application Insights

Files Overview

File	Purpose
Telemetry/ExcelMcpTelemetry.cs	Static helper for tracking events
Telemetry/ExcelMcpTelemetryInitializer.cs	Sets User.Id and Session.Id on telemetry
Telemetry/SensitiveDataRedactingProcessor.cs	Redacts PII before transmission
Program.cs	Application Insights WorkerService configuration
ExcelMcp.McpServer.csproj	MSBuild target that generates TelemetryConfig.g.cs
Directory.Build.props.user.template	Template for local dev connection string
infrastructure/azure/appinsights.bicep	Azure resource definitions
infrastructure/azure/deploy-appinsights.ps1	Deployment script

📞 Need Help?

• Read the docs: Contributing Guide
• Ask questions: Create a GitHub Issue with the question label
• Report bugs: Use the bug report template

---

Remember: Every change, no matter how small, must go through a Pull Request!

This ensures code quality, proper testing, and maintains the project's reliability for all users.