Skip to content

Commit 073f254

Browse files
sbroennesbroenne
authored
refactor: simplify telemetry - always enabled, build-time connection string (#246)
* refactor: simplify telemetry - always enabled, build-time connection string - Remove opt-out functionality (EXCELMCP_TELEMETRY_OPTOUT) - Embed AppInsights connection string at build time via MSBuild - Add Directory.Build.props.user for local dev connection string - Remove runtime environment variable lookup for connection string - Update release workflows to pass connection string to build - Remove obsolete test.runsettings file - Simplify IsEnabled to always return true * refactor: consolidate prompts and improve tool descriptions - Reduce prompts from 14 to 2 (keep only high-value guides) - Remove 13 orphaned .md content files - Fix ExcelWorksheetTool incorrect 'save' action reference - Improve ExcelRangeTool and ExcelNamedRangeTool descriptions - Slim ServerInstructions to essential info only - Remove redundant telemetry flush (SDK handles automatically) * refactor(telemetry): disable dependency tracking for HTTP calls in telemetry configuration * test hook * fix: correct Application Insights telemetry context - Set Component.Version explicitly to override SDK auto-detection (was picking up Excel COM interop 15.0.0.0 instead of app 1.0.0) - Anonymize cloud_RoleInstance (was exposing machine name) - Now all telemetry context values are privacy-safe: - application_Version: from assembly metadata - cloud_RoleName: fixed 'ExcelMcp.McpServer' - cloud_RoleInstance: anonymous hash 'instance-xxxx' - user_Id: anonymous hash of machine identity - session_Id: random per session * fix: telemetry improvements for Users/Sessions analytics - Remove debug mode from telemetry (simplify code) - Fix PageView tracking to include duration (required for User Flows) - Always override application_Version (SDK picks up Excel 15.0.0.0) - Add explicit Flush() call in smoke tests for reliable telemetry delivery - Fix smoke test shutdown to use graceful shutdown before cancellation Telemetry now correctly populates: - requests table (with duration, success, resultCode) - pageViews table (with duration for User Flows) - user_Id and session_Id (for Users/Sessions blades) - application_Version (MCP Server version, not Excel)
1 parent 403ecfa commit 073f254

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+1340
-1898
lines changed

‎.github/workflows/release-mcp-server.yml‎

Lines changed: 4 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -96,26 +96,14 @@ jobs:
9696
dotnet restore src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
9797
dotnet restore src/ExcelMcp.CLI/ExcelMcp.CLI.csproj
9898
99-
- name: Inject Application Insights Connection String
100-
run: |
101-
$telemetryFile = "src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs"
102-
$connectionString = "${{ secrets.APPINSIGHTS_CONNECTION_STRING }}"
103-
104-
if ([string]::IsNullOrWhiteSpace($connectionString)) {
105-
Write-Output "⚠️ APPINSIGHTS_CONNECTION_STRING secret not configured - telemetry will be disabled"
106-
} else {
107-
Write-Output "Injecting Application Insights connection string..."
108-
$content = Get-Content $telemetryFile -Raw
109-
$content = $content -replace '__APPINSIGHTS_CONNECTION_STRING__', $connectionString
110-
Set-Content $telemetryFile $content
111-
Write-Output "✅ Telemetry connection string injected"
112-
}
113-
shell: pwsh
114-
11599
- name: Build MCP Server & CLI
116100
run: |
117101
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj --configuration Release --no-restore
118102
dotnet build src/ExcelMcp.CLI/ExcelMcp.CLI.csproj --configuration Release --no-restore
103+
env:
104+
# Application Insights connection string is embedded at build time by MSBuild
105+
# See ExcelMcp.McpServer.csproj for the GenerateTelemetryConfig target
106+
APPINSIGHTS_CONNECTION_STRING: ${{ secrets.APPINSIGHTS_CONNECTION_STRING }}
119107

120108
- name: Skip Tests (require Excel)
121109
run: |

‎.github/workflows/release-vscode-extension.yml‎

Lines changed: 7 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -75,28 +75,19 @@ jobs:
7575
cd vscode-extension
7676
npm install
7777
78-
- name: Inject Application Insights Connection String
79-
run: |
80-
$telemetryFile = "src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs"
81-
$connectionString = "${{ secrets.APPINSIGHTS_CONNECTION_STRING }}"
82-
83-
if ([string]::IsNullOrWhiteSpace($connectionString)) {
84-
Write-Output "⚠️ APPINSIGHTS_CONNECTION_STRING secret not configured - telemetry will be disabled"
85-
} else {
86-
Write-Output "Injecting Application Insights connection string..."
87-
$content = Get-Content $telemetryFile -Raw
88-
$content = $content -replace '__APPINSIGHTS_CONNECTION_STRING__', $connectionString
89-
Set-Content $telemetryFile $content
90-
Write-Output "✅ Telemetry connection string injected"
91-
}
92-
shell: pwsh
93-
9478
- name: Build and Package Extension
9579
run: |
9680
cd vscode-extension
9781
# Run the package script which does: build:mcp-server + vsce package
9882
npm run package
83+
env:
84+
# Application Insights connection string is embedded at build time by MSBuild
85+
# See ExcelMcp.McpServer.csproj for the GenerateTelemetryConfig target
86+
APPINSIGHTS_CONNECTION_STRING: ${{ secrets.APPINSIGHTS_CONNECTION_STRING }}
9987

88+
- name: Prepare VSIX
89+
run: |
90+
cd vscode-extension
10091
$version = "${{ env.PACKAGE_VERSION }}"
10192
10293
# vsce creates filename based on package.json name (excel-mcp)

‎.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,3 +226,11 @@ gh-pages/_site/*
226226

227227
codeql-db/*
228228
infrastructure/azure/appinsights.secrets.local
229+
230+
# Local environment files (secrets, connection strings)
231+
.env
232+
.env.local
233+
.env.*.local
234+
235+
# Local MSBuild properties (secrets, connection strings)
236+
Directory.Build.props.user

‎Directory.Build.props‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,4 +49,7 @@
4949
</PropertyGroup>
5050

5151

52+
<!-- Import user-specific properties (gitignored, for local secrets) -->
53+
<Import Project="Directory.Build.props.user" Condition="Exists('Directory.Build.props.user')" />
54+
5255
</Project>

‎Directory.Build.props.user.template‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
<!--
2+
Local developer settings (COPY to Directory.Build.props.user)
3+
4+
This file is a template. To use:
5+
1. Copy this file to Directory.Build.props.user
6+
2. Fill in your actual values
7+
3. Directory.Build.props.user is gitignored and won't be committed
8+
9+
The values here are picked up by MSBuild at build time and embedded into the compiled assembly.
10+
-->
11+
<Project>
12+
<PropertyGroup>
13+
<!-- Application Insights connection string for telemetry -->
14+
<!-- Get from Azure Portal: Application Insights > Overview > Connection String -->
15+
<AppInsightsConnectionString>YOUR_CONNECTION_STRING_HERE</AppInsightsConnectionString>
16+
</PropertyGroup>
17+
</Project>

‎Directory.Packages.props‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,10 @@
1313
<PackageVersion Include="Microsoft.Extensions.Hosting" Version="9.0.10" />
1414
<PackageVersion Include="Microsoft.Extensions.Logging" Version="9.0.10" />
1515
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.0" />
16-
<!-- Application Insights SDK for telemetry (Users, Sessions, Funnels, User Flows) -->
16+
<!-- Application Insights Worker Service SDK for telemetry (Users, Sessions, Funnels, User Flows) -->
17+
<!-- Worker Service SDK provides proper DI integration, auto-collection modules, and host lifetime awareness -->
18+
<PackageVersion Include="Microsoft.ApplicationInsights.WorkerService" Version="2.23.0" />
19+
<!-- Base SDK needed for tests that directly use TelemetryClient -->
1720
<PackageVersion Include="Microsoft.ApplicationInsights" Version="2.23.0" />
1821
<PackageVersion Include="Microsoft.Extensions.Resilience" Version="10.0.0" />
1922
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="9.0.10" />
@@ -37,4 +40,4 @@
3740
<PackageVersion Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.5" />
3841
<PackageVersion Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="10.0.100" />
3942
</ItemGroup>
40-
</Project>
43+
</Project>

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,3 +140,4 @@ This means you get:
140140
### SEO & Discovery
141141

142142
`Excel Automation` • `Automate Excel with AI` • `MCP Server` • `Model Context Protocol` • `GitHub Copilot Excel` • `AI Excel Assistant` • `Power Query Automation` • `Power Query M Code` • `Power Pivot Automation` • `DAX Measures` • `DAX Automation` • `Data Model Automation` • `PivotTable Automation` • `VBA Automation` • `Excel Tables Automation` • `Excel AI Integration` • `COM Interop` • `Windows Excel Automation` • `Excel Development Tools` • `Excel Productivity` • `Excel Scripting` • `Conversational Excel` • `Natural Language Excel`
143+
# test

‎docs/DEVELOPMENT.md‎

Lines changed: 57 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -373,13 +373,23 @@ dotnet build -c Release
373373

374374
## 📊 **Application Insights / Telemetry Setup**
375375

376-
ExcelMcp uses Azure Application Insights for anonymous usage telemetry and crash reporting. Telemetry is **opt-out** (enabled by default in release builds).
376+
ExcelMcp uses Azure Application Insights (Classic SDK with WorkerService integration) for anonymous usage telemetry and crash reporting. Telemetry is **opt-out** (enabled by default in release builds).
377+
378+
### **How It Works**
379+
380+
The Application Insights connection string is **embedded at build time** via MSBuild - there is no runtime environment variable lookup.
381+
382+
**Build-time flow:**
383+
1. MSBuild reads `AppInsightsConnectionString` property (from `Directory.Build.props.user` or env var)
384+
2. Generates `TelemetryConfig.g.cs` with the connection string as a `const string`
385+
3. Compiled assembly contains the embedded connection string
377386

378387
### **What is Tracked**
379388

380389
- **Tool invocations**: Tool name, action, duration (ms), success/failure
381390
- **Unhandled exceptions**: Exception type and redacted stack trace
382-
- **Session ID**: Random GUID per process (no user identification)
391+
- **User ID**: SHA256 hash of machine identity (anonymous, 16 chars)
392+
- **Session ID**: Random GUID per process (8 chars)
383393

384394
### **What is NOT Tracked**
385395

@@ -396,6 +406,30 @@ All telemetry passes through `SensitiveDataRedactingProcessor` which removes:
396406
- Connection string secrets (`Password=...` → `[REDACTED_CREDENTIAL]`)
397407
- Email addresses → `[REDACTED_EMAIL]`
398408

409+
### **Local Development with Telemetry**
410+
411+
To enable telemetry in local builds:
412+
413+
```powershell
414+
# 1. Copy the template file
415+
Copy-Item "Directory.Build.props.user.template" "Directory.Build.props.user"
416+
417+
# 2. Edit Directory.Build.props.user and add your connection string
418+
# <AppInsightsConnectionString>InstrumentationKey=xxx;IngestionEndpoint=...</AppInsightsConnectionString>
419+
420+
# 3. Build - connection string is embedded at compile time
421+
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
422+
423+
# 4. Run - telemetry is automatically sent to Azure
424+
dotnet run --project src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
425+
```
426+
427+
**Note:** `Directory.Build.props.user` is gitignored - your connection string won't be committed.
428+
429+
### **Local Development without Telemetry**
430+
431+
If you don't create `Directory.Build.props.user`, builds will have an empty connection string and telemetry will be disabled. This is the default for local development.
432+
399433
### **Azure Resources Setup (Maintainers Only)**
400434

401435
To deploy the Application Insights infrastructure:
@@ -419,106 +453,37 @@ After deploying Azure resources:
419453
2. Add new secret: `APPINSIGHTS_CONNECTION_STRING`
420454
3. Paste the connection string from deployment output
421455

422-
The release workflow automatically injects this at build time.
423-
424-
### **Local Development**
425-
426-
During local development, telemetry is **disabled by default** because the placeholder connection string is not replaced. This is intentional - no telemetry data is sent from dev builds.
427-
428-
#### **Debug Mode: Console Output**
429-
430-
To test telemetry locally without Azure, enable debug mode which logs to stderr:
431-
432-
```powershell
433-
# Enable debug telemetry (logs to console instead of Azure)
434-
$env:EXCELMCP_DEBUG_TELEMETRY = "true"
435-
436-
# Build and run the MCP server
437-
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
438-
dotnet run --project src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
439-
440-
# You'll see telemetry output like:
441-
# [Telemetry] Debug mode enabled - logging to stderr
442-
# Activity.TraceId: abc123...
443-
# Activity.DisplayName: ToolInvocation
444-
# Activity.Tags:
445-
# tool.name: excel_file
446-
# tool.action: list
447-
# tool.duration_ms: 42
448-
# tool.success: true
449-
```
450-
451-
#### **Testing with Real Azure Resources**
452-
453-
To test with actual Application Insights:
454-
455-
```powershell
456-
# 1. Deploy Azure resources
457-
.\infrastructure\azure\deploy-appinsights.ps1 -SubscriptionId "<your-sub-id>"
458-
459-
# 2. Temporarily inject connection string (DON'T COMMIT!)
460-
$connStr = "InstrumentationKey=xxx;IngestionEndpoint=https://..."
461-
(Get-Content "src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs") -replace `
462-
'__APPINSIGHTS_CONNECTION_STRING__', $connStr | `
463-
Set-Content "src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs"
464-
465-
# 3. Build and run
466-
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
467-
dotnet run --project src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
468-
469-
# 4. Check Azure Portal → Application Insights → Transaction search
470-
471-
# 5. IMPORTANT: Revert the file (don't commit connection string!)
472-
git checkout src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs
473-
```
474-
475-
To verify telemetry state:
476-
```csharp
477-
// ExcelMcpTelemetry.IsEnabled returns false when:
478-
// - Connection string is placeholder "__APPINSIGHTS_CONNECTION_STRING__"
479-
// - User has opted out via EXCELMCP_TELEMETRY_OPTOUT=true
480-
481-
// ExcelMcpTelemetry.IsEnabled returns true when:
482-
// - EXCELMCP_DEBUG_TELEMETRY=true (console output mode)
483-
// - Connection string is real (injected at build time)
484-
```
485-
486-
### **User Opt-Out**
487-
488-
Users can disable telemetry by setting an environment variable:
489-
490-
```powershell
491-
# Windows
492-
$env:EXCELMCP_TELEMETRY_OPTOUT = "true"
493-
494-
# Or permanently via System Properties → Environment Variables
495-
```
456+
The release workflow sets this as an environment variable, and MSBuild embeds it at build time.
496457

497458
### **Telemetry Architecture**
498459

499-
```
500-
MCP Tool Invocation
501-
│
502-
▼
503-
ExcelToolsBase.ExecuteToolAction()
504-
│ (tracks: tool, action, duration, success)
505-
▼
506-
ExcelMcpTelemetry.TrackToolInvocation()
507-
│
508-
▼
509-
SensitiveDataRedactingProcessor
510-
│ (removes: paths, credentials, emails)
511-
▼
512-
Azure Monitor Exporter → Application Insights
460+
```text
461+
Build Time:
462+
MSBuild → reads AppInsightsConnectionString → generates TelemetryConfig.g.cs
463+
464+
Runtime:
465+
MCP Tool Invocation
466+
│
467+
▼
468+
ExcelMcpTelemetry.TrackToolInvocation()
469+
│ (tracks: tool, action, duration, success)
470+
▼
471+
SensitiveDataRedactingProcessor
472+
│ (removes: paths, credentials, emails)
473+
▼
474+
TelemetryClient → Application Insights
513475
```
514476

515477
### **Files Overview**
516478

517479
| File | Purpose |
518480
|------|---------|
519-
| `Telemetry/ExcelMcpTelemetry.cs` | Static helper for tracking |
481+
| `Telemetry/ExcelMcpTelemetry.cs` | Static helper for tracking events |
482+
| `Telemetry/ExcelMcpTelemetryInitializer.cs` | Sets User.Id and Session.Id on telemetry |
520483
| `Telemetry/SensitiveDataRedactingProcessor.cs` | Redacts PII before transmission |
521-
| `Program.cs` | OpenTelemetry configuration |
484+
| `Program.cs` | Application Insights WorkerService configuration |
485+
| `ExcelMcp.McpServer.csproj` | MSBuild target that generates TelemetryConfig.g.cs |
486+
| `Directory.Build.props.user.template` | Template for local dev connection string |
522487
| `infrastructure/azure/appinsights.bicep` | Azure resource definitions |
523488
| `infrastructure/azure/deploy-appinsights.ps1` | Deployment script |
524489

0 commit comments

Comments
 (0)