Skip to content

Commit cc64f69

Browse files
[Identity] Enable identity binding mode for WorkloadIdentityCredential in AKS (#53436)
* Add new environment variables * Add new property to use the proxy * Add config class for the proxy * Add custom pipeline * Add tests * Export API * Remove comments * Rename AzureKubernetesTokenProxy to IsAzureKubernetesTokenProxyEnabled * Add section in readme and changelog entry * Improve validation for CA data in TryCreate method * Add a semaphore to KubernetesProxyHttpHandler instead of waiting a fixed amount of time * Use PemReader.LoadCertificate * Remove TestEnvVar class * fb * fb * Apply suggestions from code review * fixed up handler disposal * fix tests * load cert without PK * mark live tests as Ignore * cert validation tests * fix test * fb * Add conditional compilation for X509Certificate2.CreateFromPem * Update code snippet to remove deprecated ctor * Update KubernetesProxy tests to use a fixed token file path * Replace deprecated X509Certificate2.CreateFromPem with PemReader.LoadCertificate in tests * Enhance SSL certificate validation in KubernetesProxyHttpHandler to reject non-chain errors and ensure proper handler reloading on CA file changes * Use snippets in Readme * Add logging for Kubernetes proxy CA certificate handling events * Refactor CA certificate comparison to use Span for improved performance and remove redundant method * Replace Thread.Sleep with Task.Delay in KubernetesProxyHttpHandlerTests * Add method to create handler and make requests; refactor tests for CA file handling * Refactor SendAsync test to use KubernetesProxyHttpHandler and validate cached handler behavior on CA file read failure * Refactor SendAsync tests to use KubernetesProxyHttpHandler directly and verify handler behavior on CA file changes --------- Co-authored-by: Christopher Scott <chriscott@hotmail.com>
1 parent 72eb981 commit cc64f69

15 files changed

+1534
-4
lines changed

sdk/identity/Azure.Identity/CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
### Features Added
66

7+
- Added Kubernetes token proxy support (identity binding mode) to `WorkloadIdentityCredential`. When enabled via the `IsAzureKubernetesTokenProxyEnabled ` option, the credential redirects token requests to an AKS-provided proxy to work around Entra ID's limit on federated identity credentials per managed identity. This feature is opt-in and only available when using `WorkloadIdentityCredential` directly (not supported by `DefaultAzureCredential` or `ManagedIdentityCredential`).
8+
79
### Breaking Changes
810

911
### Bugs Fixed

sdk/identity/Azure.Identity/README.md

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,45 @@ While `DefaultAzureCredential` is generally the quickest way to authenticate app
107107

108108
As of version 1.8.0, `ManagedIdentityCredential` supports [token caching](#token-caching).
109109

110+
## Identity binding mode (WorkloadIdentityCredential)
111+
112+
`WorkloadIdentityCredential` supports an opt-in identity binding mode to work around [Entra ID's limit on federated identity credentials (FICs)](https://learn.microsoft.com/entra/workload-id/workload-identity-federation-considerations#federated-identity-credential-considerations) per managed identity. When enabled via the `IsAzureKubernetesTokenProxyEnabled ` option, the credential redirects token requests to an AKS-provided proxy that handles the FIC exchange centrally, allowing multiple pods to share the same identity without hitting FIC limits.
113+
114+
**Note:** This feature is only available when using `WorkloadIdentityCredential` directly. It is not supported by `DefaultAzureCredential` or `ManagedIdentityCredential`.
115+
116+
### Usage
117+
118+
```C# Snippet:WorkloadIdentityCredentialWithIdentityBinding
119+
var credential = new WorkloadIdentityCredential(new WorkloadIdentityCredentialOptions
120+
{
121+
IsAzureKubernetesTokenProxyEnabled = true // Enable identity binding mode
122+
});
123+
```
124+
125+
When enabled, the credential reads these environment variables (typically configured by AKS):
126+
127+
* `AZURE_KUBERNETES_TOKEN_PROXY` - Base HTTPS URL for the proxy endpoint
128+
* `AZURE_KUBERNETES_CA_FILE` - Path to PEM bundle with proxy CA certificates
129+
* `AZURE_KUBERNETES_CA_DATA` - PEM-encoded CA bundle (mutually exclusive with `AZURE_KUBERNETES_CA_FILE `)
130+
* `AZURE_KUBERNETES_SNI_NAME` - TLS Server Name Indication (optional)
131+
132+
The credential validates the configuration at construction time and throws `InvalidOperationException` if the configuration is invalid or incomplete.
133+
134+
### Migration from ManagedIdentityCredential
135+
136+
If you're currently using `ManagedIdentityCredential` for workload identity in AKS and need to use identity binding mode, migrate to `WorkloadIdentityCredential`:
137+
138+
```C# Snippet:MigrationToWorkloadIdentityCredential
139+
// Before (no identity binding support):
140+
// var credential = new ManagedIdentityCredential(ManagedIdentityId.SystemAssigned);
141+
142+
// After (with identity binding support):
143+
var credential = new WorkloadIdentityCredential(new WorkloadIdentityCredentialOptions
144+
{
145+
IsAzureKubernetesTokenProxyEnabled = true
146+
});
147+
```
148+
110149
## Sovereign cloud configuration
111150

112151
By default, credentials authenticate to the Microsoft Entra endpoint for the Azure Public Cloud. To access resources in other clouds, such as Azure US Government or a private cloud, use one of the following solutions:
@@ -142,7 +181,7 @@ Not all credentials require this configuration. Credentials that authenticate th
142181
|-|-|-|
143182
|[`EnvironmentCredential`][ref_EnvironmentCredential]|Authenticates a service principal or user via credential information specified in [environment variables](#environment-variables).||
144183
|[`ManagedIdentityCredential`][ref_ManagedIdentityCredential]|Authenticates the managed identity of an Azure resource.|[user-assigned managed identity][uami_doc]<br>[system-assigned managed identity][sami_doc]|
145-
|[`WorkloadIdentityCredential`][ref_WorkloadIdentityCredential]|Supports [Microsoft Entra Workload ID](https://learn.microsoft.com/azure/aks/workload-identity-overview) on Kubernetes.||
184+
|[`WorkloadIdentityCredential`][ref_WorkloadIdentityCredential]|Supports [Microsoft Entra Workload ID](https://learn.microsoft.com/azure/aks/workload-identity-overview) on Kubernetes. Supports [identity binding mode](#identity-binding-mode-workloadidentitycredential) to work around FIC limits in AKS.||
146185

147186
### Authenticate service principals
148187

sdk/identity/Azure.Identity/api/Azure.Identity.net8.0.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -502,6 +502,7 @@ public WorkloadIdentityCredentialOptions() { }
502502
public System.Collections.Generic.IList<string> AdditionallyAllowedTenants { get { throw null; } }
503503
public string ClientId { get { throw null; } set { } }
504504
public bool DisableInstanceDiscovery { get { throw null; } set { } }
505+
public bool IsAzureKubernetesTokenProxyEnabled { get { throw null; } set { } }
505506
public string TenantId { get { throw null; } set { } }
506507
public string TokenFilePath { get { throw null; } set { } }
507508
}

sdk/identity/Azure.Identity/api/Azure.Identity.netstandard2.0.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -499,6 +499,7 @@ public WorkloadIdentityCredentialOptions() { }
499499
public System.Collections.Generic.IList<string> AdditionallyAllowedTenants { get { throw null; } }
500500
public string ClientId { get { throw null; } set { } }
501501
public bool DisableInstanceDiscovery { get { throw null; } set { } }
502+
public bool IsAzureKubernetesTokenProxyEnabled { get { throw null; } set { } }
502503
public string TenantId { get { throw null; } set { } }
503504
public string TokenFilePath { get { throw null; } set { } }
504505
}

sdk/identity/Azure.Identity/src/AzureIdentityEventSource.cs

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,9 @@ internal sealed class AzureIdentityEventSource : AzureEventSource, IIdentityLogg
4343
private const int ServiceFabricManagedIdentityRuntimeConfigurationNotSupportedEvent = 22;
4444
private const int ManagedIdentitySourceAttemptedEvent = 25;
4545
private const int ManagedIdentityCredentialSelectedEvent = 26;
46+
private const int KubernetesProxyCaCertificateReloadSkippedEvent = 27;
47+
private const int KubernetesProxyCaCertificateReloadFailedEvent = 28;
48+
private const int KubernetesProxyCaCertificateReloadedEvent = 29;
4649

4750
internal const string TenantIdDiscoveredAndNotUsedEventMessage = "A token was request for a different tenant than was configured on the credential, but the configured value was used since multi tenant authentication has been disabled. Configured TenantId: {0}, Requested TenantId {1}";
4851
internal const string TenantIdDiscoveredAndUsedEventMessage = "A token was requested for a different tenant than was configured on the credential, and the requested tenant id was used to authenticate. Configured TenantId: {0}, Requested TenantId {1}";
@@ -53,6 +56,9 @@ internal sealed class AzureIdentityEventSource : AzureEventSource, IIdentityLogg
5356
internal const string ServiceFabricManagedIdentityRuntimeConfigurationNotSupportedMessage = "Service Fabric user assigned managed identity ClientId or ResourceId is not configurable at runtime.";
5457
internal const string ManagedIdentitySourceAttemptedMessage = "ManagedIdentitySource {0} was attempted. IsSelected={1}.";
5558
internal const string ManagedIdentityCredentialSelectedMessage = "Managed Identity source selected: {0} with ID: {1}";
59+
internal const string KubernetesProxyCaCertificateReloadSkippedMessage = "Kubernetes proxy CA certificate reload skipped. Reason: {0}";
60+
internal const string KubernetesProxyCaCertificateReloadFailedMessage = "Kubernetes proxy CA certificate read failed. Error: {0}";
61+
internal const string KubernetesProxyCaCertificateReloadedMessage = "Kubernetes proxy CA certificate changed, handler will be reloaded.";
5662

5763
private AzureIdentityEventSource() : base(EventSourceName) { }
5864

@@ -425,5 +431,32 @@ public void ManagedIdentityCredentialSelected(string credentialType, string id)
425431
WriteEvent(ManagedIdentityCredentialSelectedEvent, credentialType, id);
426432
}
427433
}
434+
435+
[Event(KubernetesProxyCaCertificateReloadSkippedEvent, Level = EventLevel.Informational, Message = KubernetesProxyCaCertificateReloadSkippedMessage)]
436+
public void KubernetesProxyCaCertificateReloadSkipped(string reason)
437+
{
438+
if (IsEnabled(EventLevel.Informational, EventKeywords.All))
439+
{
440+
WriteEvent(KubernetesProxyCaCertificateReloadSkippedEvent, reason);
441+
}
442+
}
443+
444+
[Event(KubernetesProxyCaCertificateReloadFailedEvent, Level = EventLevel.Warning, Message = KubernetesProxyCaCertificateReloadFailedMessage)]
445+
public void KubernetesProxyCaCertificateReloadFailed(string error)
446+
{
447+
if (IsEnabled(EventLevel.Warning, EventKeywords.All))
448+
{
449+
WriteEvent(KubernetesProxyCaCertificateReloadFailedEvent, error);
450+
}
451+
}
452+
453+
[Event(KubernetesProxyCaCertificateReloadedEvent, Level = EventLevel.Informational, Message = KubernetesProxyCaCertificateReloadedMessage)]
454+
public void KubernetesProxyCaCertificateReloaded()
455+
{
456+
if (IsEnabled(EventLevel.Informational, EventKeywords.All))
457+
{
458+
WriteEvent(KubernetesProxyCaCertificateReloadedEvent);
459+
}
460+
}
428461
}
429462
}

sdk/identity/Azure.Identity/src/Credentials/WorkloadIdentityCredential.cs

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,10 @@
22
// Licensed under the MIT License.
33

44
using System;
5-
using System.Collections.Generic;
6-
using System.Text;
75
using System.Threading;
86
using System.Threading.Tasks;
97
using Azure.Core;
108
using Azure.Core.Pipeline;
11-
using Microsoft.Identity.Client;
129

1310
namespace Azure.Identity
1411
{
@@ -47,6 +44,18 @@ public WorkloadIdentityCredential(WorkloadIdentityCredentialOptions options)
4744
clientAssertionCredentialOptions.Pipeline = options.Pipeline;
4845
clientAssertionCredentialOptions.MsalClient = options.MsalClient;
4946

47+
// Configure Kubernetes token proxy if user opted in
48+
if (options.IsAzureKubernetesTokenProxyEnabled)
49+
{
50+
var proxyConfig = KubernetesProxyConfig.TryCreate();
51+
if (proxyConfig != null)
52+
{
53+
var proxyHandler = new KubernetesProxyHttpHandler(proxyConfig);
54+
var httpClient = new System.Net.Http.HttpClient(proxyHandler);
55+
clientAssertionCredentialOptions.Transport = new HttpClientTransport(httpClient);
56+
}
57+
}
58+
5059
_clientAssertionCredential = new ClientAssertionCredential(options.TenantId, options.ClientId, _tokenFileCache.GetTokenFileContentsAsync, clientAssertionCredentialOptions);
5160
}
5261
_pipeline = _clientAssertionCredential?.Pipeline ?? CredentialPipeline.GetInstance(default);

sdk/identity/Azure.Identity/src/Credentials/WorkloadIdentityCredentialOptions.cs

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,13 @@ public class WorkloadIdentityCredentialOptions : TokenCredentialOptions, ISuppor
3939
/// <inheritdoc />
4040
public bool DisableInstanceDiscovery { get; set; }
4141

42+
/// <summary>
43+
/// Enables Azure Kubernetes token proxy mode to work around Entra's limit on federated identity credentials.
44+
/// When enabled and proxy configuration environment variables are set, requests are sent to the AKS proxy instead of directly to Entra ID.
45+
/// This feature is not supported when using DefaultAzureCredential.
46+
/// </summary>
47+
public bool IsAzureKubernetesTokenProxyEnabled { get; set; }
48+
4249
/// <summary>
4350
/// Specifies tenants in addition to the specified <see cref="TenantId"/> for which the credential may acquire tokens.
4451
/// Add the wildcard value "*" to allow the credential to acquire tokens for any tenant the logged in account can access.
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
// Copyright (c) Microsoft Corporation. All rights reserved.
2+
// Licensed under the MIT License.
3+
4+
using System;
5+
using System.Net.Http;
6+
using System.Threading;
7+
using System.Threading.Tasks;
8+
using Azure.Core.Pipeline;
9+
10+
namespace Azure.Identity
11+
{
12+
/// <summary>
13+
/// HTTP message handler that handles disposing itself only after in-flight requests complete.
14+
/// </summary>
15+
internal class DisposingHttpClientHandler : HttpClientHandler, IDisposable
16+
{
17+
private int _count;
18+
private TaskCompletionSource<Task> _zeroTcs =
19+
new(TaskCreationOptions.RunContinuationsAsynchronously);
20+
21+
public IDisposable StartSend()
22+
{
23+
var newCount = Interlocked.Increment(ref _count);
24+
if (newCount == 1)
25+
{
26+
// leaving zero -> ensure future waiters actually wait
27+
var tcs = Volatile.Read(ref _zeroTcs);
28+
if (tcs.Task.IsCompleted)
29+
{
30+
Interlocked.CompareExchange(
31+
ref _zeroTcs,
32+
new TaskCompletionSource<Task>(TaskCreationOptions.RunContinuationsAsynchronously),
33+
tcs);
34+
}
35+
}
36+
return new Releaser(this);
37+
}
38+
39+
public void CompleteSend()
40+
{
41+
if (Interlocked.Decrement(ref _count) == 0)
42+
{
43+
Volatile.Read(ref _zeroTcs).TrySetResult(Task.CompletedTask);
44+
}
45+
}
46+
47+
public Task WaitForOutstandingRequests() => Volatile.Read(ref _zeroTcs).Task;
48+
49+
private sealed class Releaser : IDisposable
50+
{
51+
private DisposingHttpClientHandler _owner;
52+
public Releaser(DisposingHttpClientHandler owner) => _owner = owner;
53+
public void Dispose() { _owner?.CompleteSend(); _owner = null; }
54+
}
55+
}
56+
}

sdk/identity/Azure.Identity/src/EnvironmentVariables.cs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,11 @@ internal class EnvironmentVariables
3333

3434
public static string AzureFederatedTokenFile => GetNonEmptyStringOrNull(Environment.GetEnvironmentVariable("AZURE_FEDERATED_TOKEN_FILE"));
3535

36+
public static string AzureKubernetesTokenProxy => GetNonEmptyStringOrNull(Environment.GetEnvironmentVariable("AZURE_KUBERNETES_TOKEN_PROXY"));
37+
public static string AzureKubernetesCaFile => GetNonEmptyStringOrNull(Environment.GetEnvironmentVariable("AZURE_KUBERNETES_CA_FILE"));
38+
public static string AzureKubernetesCaData => GetNonEmptyStringOrNull(Environment.GetEnvironmentVariable("AZURE_KUBERNETES_CA_DATA"));
39+
public static string AzureKubernetesSniName => GetNonEmptyStringOrNull(Environment.GetEnvironmentVariable("AZURE_KUBERNETES_SNI_NAME"));
40+
3641
public static string CredentialSelection => GetNonEmptyStringOrNull(Environment.GetEnvironmentVariable("AZURE_TOKEN_CREDENTIALS"));
3742

3843
private static string GetNonEmptyStringOrNull(string str)

0 commit comments

Comments
 (0)