Skip to content

Commit d7a2d3d

Browse files
sbroennesbroenne
authored
feat: Add Application Insights telemetry via OpenTelemetry (#240)
- Add OpenTelemetry + Azure Monitor for usage tracking and unhandled exceptions - Implement SensitiveDataRedactingProcessor to protect file paths/connection strings - Add telemetry to all 12 MCP tools via ExecuteToolAction pattern - Support opt-out via EXCELMCP_TELEMETRY_OPTOUT environment variable - Add debug mode via EXCELMCP_DEBUG_TELEMETRY for local testing (console output) - Build-time connection string injection in release workflow - Add Bicep templates for Azure Application Insights deployment - Add 29 tests (28 unit + 1 integration) for telemetry components - Update developer docs with telemetry setup instructions Closes #239
1 parent d2fcc5f commit d7a2d3d

29 files changed

+1217
-83
lines changed

‎.github/workflows/release-mcp-server.yml‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,9 @@ name: Release MCP Server & CLI
77
# Unified Versioning: MCP Server and CLI always release together with the same version
88
# Tag pattern: v1.2.3 (removes mcp-v prefix, CLI and MCP Server share version)
99
#
10-
# Required GitHub Secret:
10+
# Required GitHub Secrets:
1111
# - NUGET_USER: Your NuGet.org username (profile name, NOT email)
12+
# - APPINSIGHTS_CONNECTION_STRING: Application Insights connection string for telemetry
1213
#
1314
# Required NuGet.org Configuration:
1415
# - Package: Sbroenne.ExcelMcp.McpServer
@@ -95,6 +96,22 @@ jobs:
9596
dotnet restore src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
9697
dotnet restore src/ExcelMcp.CLI/ExcelMcp.CLI.csproj
9798
99+
- name: Inject Application Insights Connection String
100+
run: |
101+
$telemetryFile = "src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs"
102+
$connectionString = "${{ secrets.APPINSIGHTS_CONNECTION_STRING }}"
103+
104+
if ([string]::IsNullOrWhiteSpace($connectionString)) {
105+
Write-Output "⚠️ APPINSIGHTS_CONNECTION_STRING secret not configured - telemetry will be disabled"
106+
} else {
107+
Write-Output "Injecting Application Insights connection string..."
108+
$content = Get-Content $telemetryFile -Raw
109+
$content = $content -replace '__APPINSIGHTS_CONNECTION_STRING__', $connectionString
110+
Set-Content $telemetryFile $content
111+
Write-Output "✅ Telemetry connection string injected"
112+
}
113+
shell: pwsh
114+
98115
- name: Build MCP Server & CLI
99116
run: |
100117
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj --configuration Release --no-restore

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -225,3 +225,4 @@ gh-pages/vendor
225225
gh-pages/_site/*
226226

227227
codeql-db/*
228+
infrastructure/azure/appinsights.secrets.local

‎Directory.Packages.props‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@
1313
<PackageVersion Include="Microsoft.Extensions.Hosting" Version="9.0.10" />
1414
<PackageVersion Include="Microsoft.Extensions.Logging" Version="9.0.10" />
1515
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.0" />
16+
<!-- OpenTelemetry for Application Insights -->
17+
<PackageVersion Include="Azure.Monitor.OpenTelemetry.Exporter" Version="1.3.0" /> <PackageVersion Include="OpenTelemetry.Exporter.Console" Version="1.11.2" /> <PackageVersion Include="OpenTelemetry.Extensions.Hosting" Version="1.11.2" />
1618
<PackageVersion Include="Microsoft.Extensions.Resilience" Version="10.0.0" />
1719
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="9.0.10" />
1820
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="9.0.10" />
@@ -35,4 +37,4 @@
3537
<PackageVersion Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.5" />
3638
<PackageVersion Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="10.0.100" />
3739
</ItemGroup>
38-
</Project>
40+
</Project>

‎docs/DEVELOPMENT.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -371,6 +371,157 @@ dotnet build -c Release
371371
.\src\ExcelMcp.CLI\bin\Release\net10.0\excelcli.exe --version
372372
```
373373

374+
## 📊 **Application Insights / Telemetry Setup**
375+
376+
ExcelMcp uses Azure Application Insights for anonymous usage telemetry and crash reporting. Telemetry is **opt-out** (enabled by default in release builds).
377+
378+
### **What is Tracked**
379+
380+
- **Tool invocations**: Tool name, action, duration (ms), success/failure
381+
- **Unhandled exceptions**: Exception type and redacted stack trace
382+
- **Session ID**: Random GUID per process (no user identification)
383+
384+
### **What is NOT Tracked**
385+
386+
- File paths, file names, or file contents
387+
- User identity, machine name, or IP address
388+
- Excel data, formulas, or cell values
389+
- Connection strings, credentials, or passwords
390+
391+
### **Sensitive Data Redaction**
392+
393+
All telemetry passes through `SensitiveDataRedactingProcessor` which removes:
394+
- Windows file paths (`C:\Users\...` → `[REDACTED_PATH]`)
395+
- UNC paths (`\\server\share\...` → `[REDACTED_PATH]`)
396+
- Connection string secrets (`Password=...` → `[REDACTED_CREDENTIAL]`)
397+
- Email addresses → `[REDACTED_EMAIL]`
398+
399+
### **Azure Resources Setup (Maintainers Only)**
400+
401+
To deploy the Application Insights infrastructure:
402+
403+
```powershell
404+
# 1. Login to Azure
405+
az login
406+
407+
# 2. Deploy resources (creates RG, Log Analytics, App Insights)
408+
.\infrastructure\azure\deploy-appinsights.ps1 -SubscriptionId "<your-subscription-id>"
409+
410+
# 3. Copy the connection string from output
411+
# Output: "Connection String: InstrumentationKey=xxx;IngestionEndpoint=..."
412+
```
413+
414+
### **GitHub Secret Configuration (Maintainers Only)**
415+
416+
After deploying Azure resources:
417+
418+
1. Go to GitHub repo → **Settings** → **Secrets and variables** → **Actions**
419+
2. Add new secret: `APPINSIGHTS_CONNECTION_STRING`
420+
3. Paste the connection string from deployment output
421+
422+
The release workflow automatically injects this at build time.
423+
424+
### **Local Development**
425+
426+
During local development, telemetry is **disabled by default** because the placeholder connection string is not replaced. This is intentional - no telemetry data is sent from dev builds.
427+
428+
#### **Debug Mode: Console Output**
429+
430+
To test telemetry locally without Azure, enable debug mode which logs to stderr:
431+
432+
```powershell
433+
# Enable debug telemetry (logs to console instead of Azure)
434+
$env:EXCELMCP_DEBUG_TELEMETRY = "true"
435+
436+
# Build and run the MCP server
437+
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
438+
dotnet run --project src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
439+
440+
# You'll see telemetry output like:
441+
# [Telemetry] Debug mode enabled - logging to stderr
442+
# Activity.TraceId: abc123...
443+
# Activity.DisplayName: ToolInvocation
444+
# Activity.Tags:
445+
# tool.name: excel_file
446+
# tool.action: list
447+
# tool.duration_ms: 42
448+
# tool.success: true
449+
```
450+
451+
#### **Testing with Real Azure Resources**
452+
453+
To test with actual Application Insights:
454+
455+
```powershell
456+
# 1. Deploy Azure resources
457+
.\infrastructure\azure\deploy-appinsights.ps1 -SubscriptionId "<your-sub-id>"
458+
459+
# 2. Temporarily inject connection string (DON'T COMMIT!)
460+
$connStr = "InstrumentationKey=xxx;IngestionEndpoint=https://..."
461+
(Get-Content "src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs") -replace `
462+
'__APPINSIGHTS_CONNECTION_STRING__', $connStr | `
463+
Set-Content "src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs"
464+
465+
# 3. Build and run
466+
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
467+
dotnet run --project src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
468+
469+
# 4. Check Azure Portal → Application Insights → Transaction search
470+
471+
# 5. IMPORTANT: Revert the file (don't commit connection string!)
472+
git checkout src/ExcelMcp.McpServer/Telemetry/ExcelMcpTelemetry.cs
473+
```
474+
475+
To verify telemetry state:
476+
```csharp
477+
// ExcelMcpTelemetry.IsEnabled returns false when:
478+
// - Connection string is placeholder "__APPINSIGHTS_CONNECTION_STRING__"
479+
// - User has opted out via EXCELMCP_TELEMETRY_OPTOUT=true
480+
481+
// ExcelMcpTelemetry.IsEnabled returns true when:
482+
// - EXCELMCP_DEBUG_TELEMETRY=true (console output mode)
483+
// - Connection string is real (injected at build time)
484+
```
485+
486+
### **User Opt-Out**
487+
488+
Users can disable telemetry by setting an environment variable:
489+
490+
```powershell
491+
# Windows
492+
$env:EXCELMCP_TELEMETRY_OPTOUT = "true"
493+
494+
# Or permanently via System Properties → Environment Variables
495+
```
496+
497+
### **Telemetry Architecture**
498+
499+
```
500+
MCP Tool Invocation
501+
│
502+
▼
503+
ExcelToolsBase.ExecuteToolAction()
504+
│ (tracks: tool, action, duration, success)
505+
▼
506+
ExcelMcpTelemetry.TrackToolInvocation()
507+
│
508+
▼
509+
SensitiveDataRedactingProcessor
510+
│ (removes: paths, credentials, emails)
511+
▼
512+
Azure Monitor Exporter → Application Insights
513+
```
514+
515+
### **Files Overview**
516+
517+
| File | Purpose |
518+
|------|---------|
519+
| `Telemetry/ExcelMcpTelemetry.cs` | Static helper for tracking |
520+
| `Telemetry/SensitiveDataRedactingProcessor.cs` | Redacts PII before transmission |
521+
| `Program.cs` | OpenTelemetry configuration |
522+
| `infrastructure/azure/appinsights.bicep` | Azure resource definitions |
523+
| `infrastructure/azure/deploy-appinsights.ps1` | Deployment script |
524+
374525
## 📞 **Need Help?**
375526

376527
- **Read the docs**: [Contributing Guide](CONTRIBUTING.md)

‎infrastructure/azure/appinsights-resources.bicep‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
// Application Insights resources module
2+
// Called by appinsights.bicep - deploys into an existing resource group
3+
4+
param location string
5+
param logAnalyticsName string
6+
param appInsightsName string
7+
param retentionInDays int
8+
param tags object
9+
10+
// Log Analytics Workspace (required backend for Application Insights)
11+
resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2023-09-01' = {
12+
name: logAnalyticsName
13+
location: location
14+
tags: tags
15+
properties: {
16+
sku: {
17+
name: 'PerGB2018'
18+
}
19+
retentionInDays: retentionInDays
20+
features: {
21+
enableLogAccessUsingOnlyResourcePermissions: true
22+
}
23+
workspaceCapping: {
24+
dailyQuotaGb: -1 // No cap
25+
}
26+
publicNetworkAccessForIngestion: 'Enabled'
27+
publicNetworkAccessForQuery: 'Enabled'
28+
}
29+
}
30+
31+
// Application Insights (workspace-based)
32+
resource appInsights 'Microsoft.Insights/components@2020-02-02' = {
33+
name: appInsightsName
34+
location: location
35+
tags: tags
36+
kind: 'other' // 'other' for non-web applications like console apps
37+
properties: {
38+
Application_Type: 'other'
39+
WorkspaceResourceId: logAnalytics.id
40+
IngestionMode: 'LogAnalytics'
41+
publicNetworkAccessForIngestion: 'Enabled'
42+
publicNetworkAccessForQuery: 'Enabled'
43+
RetentionInDays: retentionInDays
44+
}
45+
}
46+
47+
// Outputs
48+
output logAnalyticsWorkspaceId string = logAnalytics.id
49+
output appInsightsName string = appInsights.name
50+
output appInsightsConnectionString string = appInsights.properties.ConnectionString
51+
output appInsightsInstrumentationKey string = appInsights.properties.InstrumentationKey

‎infrastructure/azure/appinsights.bicep‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
// Application Insights infrastructure for ExcelMcp telemetry
2+
// Deploys: Resource Group, Log Analytics Workspace, Application Insights
3+
//
4+
// Deploy with: az deployment sub create --location <location> --template-file appinsights.bicep --parameters appinsights.parameters.json
5+
6+
targetScope = 'subscription'
7+
8+
@description('Name of the resource group to create')
9+
param resourceGroupName string = 'excelmcp-observability'
10+
11+
@description('Azure region for all resources')
12+
param location string = 'westeurope'
13+
14+
@description('Name of the Log Analytics workspace')
15+
param logAnalyticsName string = 'excelmcp-logs'
16+
17+
@description('Name of the Application Insights resource')
18+
param appInsightsName string = 'excelmcp-appinsights'
19+
20+
@description('Data retention in days (30-730)')
21+
@minValue(30)
22+
@maxValue(730)
23+
param retentionInDays int = 90
24+
25+
@description('Tags to apply to all resources')
26+
param tags object = {
27+
project: 'ExcelMcp'
28+
purpose: 'Telemetry'
29+
managedBy: 'Bicep'
30+
}
31+
32+
// Resource Group
33+
resource rg 'Microsoft.Resources/resourceGroups@2024-03-01' = {
34+
name: resourceGroupName
35+
location: location
36+
tags: tags
37+
}
38+
39+
// Deploy resources into the resource group
40+
module observability 'appinsights-resources.bicep' = {
41+
name: 'observability-deployment'
42+
scope: rg
43+
params: {
44+
location: location
45+
logAnalyticsName: logAnalyticsName
46+
appInsightsName: appInsightsName
47+
retentionInDays: retentionInDays
48+
tags: tags
49+
}
50+
}
51+
52+
// Outputs
53+
output resourceGroupName string = rg.name
54+
output logAnalyticsWorkspaceId string = observability.outputs.logAnalyticsWorkspaceId
55+
output appInsightsName string = observability.outputs.appInsightsName
56+
output appInsightsConnectionString string = observability.outputs.appInsightsConnectionString
57+
output appInsightsInstrumentationKey string = observability.outputs.appInsightsInstrumentationKey

‎infrastructure/azure/appinsights.parameters.json‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{
2+
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
3+
"contentVersion": "1.0.0.0",
4+
"parameters": {
5+
"resourceGroupName": {
6+
"value": "excelmcp-observability"
7+
},
8+
"location": {
9+
"value": "swedencentral"
10+
},
11+
"logAnalyticsName": {
12+
"value": "excelmcp-logs"
13+
},
14+
"appInsightsName": {
15+
"value": "excelmcp-appinsights"
16+
},
17+
"retentionInDays": {
18+
"value": 90
19+
},
20+
"tags": {
21+
"value": {
22+
"project": "ExcelMcp",
23+
"purpose": "Telemetry",
24+
"managedBy": "Bicep"
25+
}
26+
}
27+
}
28+
}

0 commit comments

Comments
 (0)