mirror of
https://github.com/sbroenne/mcp-server-excel.git
synced 2026-09-19 07:53:08 +08:00
Stefan Broenner
d297387bd8
* feat: JSON token optimization, LLM tests, and .NET 10 upgrade (#341) (#342) * spec(006): Add .NET 10 upgrade feature specification Complete speckit workflow for upgrading from .NET 8 to .NET 10: - Update constitution.md to v1.1.0 with .NET 10 SDK requirement - Create spec.md with 4 user stories and 15 functional requirements - Create plan.md with implementation strategy and file inventory - Create research.md with SDK version, package compatibility findings - Create tasks.md with 38 tasks organized by user story - Update copilot-instructions.md with .NET 10 context Covers: SDK (global.json), 8 csproj files, 5 workflows, Dockerfile, and 7+ documentation files. No code changes - configuration only. * Add LLM-based tests for MCP Server using agent-benchmark Closes #340 - Add LLM test infrastructure with PowerShell runner (Run-LLMTests.ps1) - Create test scenario for excel_file and excel_worksheet tools - Configure Azure OpenAI with gpt-4.1 and gpt-5.2-chat agents - Update testing-strategy.instructions.md with LLM test documentation - All 20 tests passing with 100% success rate Test results: 464,502 tokens consumed across 20 tests Average time: ~7.4s per test * feat: optimize JSON token usage for LLM efficiency (#341) - Set WriteIndented=false for compact JSON output - Add DefaultIgnoreCondition.WhenWritingNull to omit null properties - Add [JsonPropertyName] attributes with short names to result types: - ResultBase: ok, err, fp - OperationResult: act - WorksheetInfo: n, i, v - RangeValueResult: sn, ra, d, r, c - TableInfo: n, sn, ra, hh, st, r, c, cols, tot - And more... - Update SerializeToolError to use short property names - Fix UUID templating syntax in LLM test file Token usage reduced from 464,502 to 443,601 (~4.5% reduction) Closes #341 * feat(mcp): JSON token optimization - tool splitting and parameter abbreviation Issue #341: Reduce JSON token consumption for LLM efficiency ## Tool Splitting (12 → 21 tools) - excel_range → excel_range, excel_range_edit, excel_range_format, excel_range_link - excel_table → excel_table, excel_table_column - excel_pivottable → excel_pivottable, excel_pivottable_field, excel_pivottable_calc - excel_datamodel → excel_datamodel, excel_datamodel_rel - excel_chart → excel_chart, excel_chart_config - excel_worksheet → excel_worksheet, excel_worksheet_style ## Parameter Name Abbreviation - sessionId → sid, sheetName → sn, tableName → tn - rangeAddress → addr, values → vals, excelPath → path - pivotTableName → ptn, chartName → cn, and more ## XML Documentation Optimization - Removed redundant action lists (schema provides) - Standardized 'Related:' cross-tool reference pattern - Trimmed verbose descriptions (~500+ tokens saved) ## Test Updates - McpServerSmokeTests: Updated all parameter names - McpServerIntegrationTests: Updated expected tool count (12 → 21) - Other integration tests: Updated parameter names Note: Pre-commit coverage audit requires update to account for tool splitting Build: 0 errors, 0 warnings Smoke test: 15 steps passed * docs: Update documentation for 21 tools, backfill changelog, add LLM testing callout - Updated all docs to reflect 21 tools with 182 operations (was 12 tools) - Backfilled CHANGELOG.md with missing versions 1.4.35-1.4.49 - Added JSON Token Optimization and LLM Integration Testing to changelog - Added LLM-Tested Quality callout to README, gh-pages, and VS Code extension - Added LLM-Tested Quality feature card to gh-pages index - Updated testing-strategy.instructions.md with LLM test guidance - Updated architecture-patterns.instructions.md with 21 tools reference - Fixed audit-core-coverage.ps1 to handle sub-tool enum grouping Files updated: - README.md, FEATURES.md, gh-pages/*.md - vscode-extension/CHANGELOG.md, vscode-extension/README.md - src/ExcelMcp.McpServer/README.md, src/ExcelMcp.CLI/README.md - tests/README.md, tests/ExcelMcp.McpServer.LLM.Tests/README.md - .github/instructions/*.md - scripts/audit-core-coverage.ps1 * test(llm): consolidate and improve LLM modification pattern tests - Rename excel-incremental-update-test.yaml to excel-modification-patterns-test.yaml - Consolidate tests from 9 sessions down to 4 focused sessions: - Range Updates: Formula canary (D1=1004) proves targeted cell updates - Table Operations: Table count canary verifies no delete/recreate - Chart Modifications: Chart count canary confirms title change only - Sheet Structural Changes: ROW() formulas prove proper row shifting - Add canary verification to detect 'delete and rebuild' anti-patterns - Simplify other test files: - excel-file-worksheet-test.yaml: 11 tests -> 2 consolidated tests - excel-range-test.yaml: 10 tests -> 2 consolidated tests - excel-table-test.yaml: 12 tests -> 2 consolidated tests - Remove unsupported tool_param_contains assertions (agent-benchmark limitation) - Remove custom system_prompt and rate_limits (use defaults) - Increase server_delay to 30s for reliability - Clean up CHANGELOG.md (remove maintenance releases) Addresses user feedback about 'small step-by-step changes instead of frequently deleting and rebuild' behavior. All 10 LLM tests passing (4 modification patterns + 6 other scenarios). * feat: upgrade to .NET 10 - Update SDK to .NET 10.0.100 - Update all source and test projects to target net10.0 - Update package versions to latest stable for .NET 10: - Microsoft.Extensions packages remain at 10.0.1 (latest available) - Spectre.Console at 0.54.0 - ModelContextProtocol at 0.5.0-preview.1 - Fix IDE0060 unused parameter warning in test class - Build succeeds with 0 warnings, 0 errors All projects now target net10.0 and build successfully. * ci: update to .NET 10 and document changelog - Update GitHub Actions workflows to use .NET 10 SDK: - build-mcp-server.yml: 8.0.x → 10.0.x - build-cli.yml: 8.0.x → 10.0.x - Update build artifact paths from net8.0 to net10.0 - Update CHANGELOG.md with .NET 10 upgrade details GitHub Actions now build and test against .NET 10.0. * refactor: Upgrade to C# 14 and finalize .NET 10 migration - Explicitly set LangVersion=14 in Directory.Build.props (was: latest) - Refactor telemetry code to use modern Dictionary initializer syntax - Apply C# 14 collection expressions for empty list patterns - Update workflow artifacts from net8.0 to net10.0 paths - Update all remaining .NET 8.0 references to .NET 10 across workflow and docs Files modified: 16 - Directory.Build.props - Core refactoring: ExcelMcpTelemetry.cs, OlapPivotTableFieldStrategy.cs, TableCommands.Data.cs - Workflow/documentation: .github/workflows/*.yml, vscode-extension/*, examples/*, docs/* * refactor: remove dead code, use LINQ, and add proper exception suppression comments - DataModelCommands.RenameTable.cs: Remove dead try-catch for COM rename (always fails), add CA1031 pragmas with justification - PivotTableCommands.Fields.cs: Simplify ListCubeFieldsAsync and ListRegularFieldsAsync by removing redundant try-catch wrappers - RenameNameRules.cs: Convert foreach loop to LINQ expression per code review feedback --------- Co-authored-by: Stefan Broenner <stefan.broenner@microsoft.comm> * Update LLM tests to use TEMP_DIR variable Use new agent-benchmark TEMP_DIR variable instead of hardcoded C:/temp paths for cross-platform compatibility. * Enable clarification_detection for both GPT-4.1 and GPT-5.2-chat agents Both agents now have clarification_detection enabled after fixing false positive patterns in agent-benchmark (mykhaliev/agent-benchmark#21). * Add LLM test infrastructure and improve tool prompts LLM Test Infrastructure: - Add Run-LLMTests.ps1 script for running agent-benchmark tests - Add test scenarios for Power Query, Data Model, Tables, Ranges - Add amazon.csv fixture data for Power Query import tests - Update testing-strategy.instructions.md with LLM test documentation Tool Prompt Improvements: - Add excel_datamodel.md prompt with star schema guidance - Add excel_table.md prompt for table operations - Update excel_powerquery.md with Data Model loading guidance - Improve tool descriptions for better LLM comprehension Tool Enhancements: - ExcelDataModelTool: Improved action guidance - ExcelPowerQueryTool: Better load destination documentation - ExcelTableTool: Enhanced add-to-datamodel guidance --------- Co-authored-by: Stefan Broenner <stefan.broenner@microsoft.comm>