DRIVERS-3207: Custom AWS credential providers execute earlier (#1838)

Author: durran

source/auth/auth.md

@@ -959,6 +959,12 @@ Examples are provided below.
 
         Drivers MUST allow the user to specify an AWS session token for authentication with temporary credentials.
 
+    - AWS_CREDENTIAL_PROVIDER
+
+        An AWS [Custom Credential Provider](#custom-credential-providers) that returns AWS credentials. Drivers MAY allow
+        the user to specify an object or function, depending on what is idiomatic for the driver. This property MUST
+        follow the same API as the driver language's AWS SDK credential provider.
+
 #### Obtaining Credentials
 
 Drivers will need AWS IAM credentials (an access key, a secret access key and optionally a session token) to complete
@@ -1006,8 +1012,8 @@ Drivers MAY expose API for default providers for the following scenarios when ap
 The order in which Drivers MUST search for credentials is:
 
 1. The URI
-2. Environment variables
-3. A custom AWS credential provider if the driver supports it.
+2. A custom AWS credential provider if the driver supports it.
+3. Environment variables
 4. Using `AssumeRoleWithWebIdentity` if `AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_ARN` are set.
 5. The ECS endpoint if `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` is set. Otherwise, the EC2 endpoint.
 
@@ -2159,6 +2165,8 @@ practice to avoid this. (See
 
 ## Changelog
 
+- 2025-09-10: Update precedence of MONGODB-AWS credential fetching behaviour.
+
 - 2025-01-29: Add support for custom AWS credential providers.
 
 - 2024-10-02: Add Kubernetes built-in OIDC provider integration.

source/auth/tests/mongodb-aws.md

@@ -2,7 +2,7 @@
 
 Drivers MUST test the following scenarios:
 
-1. `Regular Credentials`: Auth via an `ACCESS_KEY_ID` and `SECRET_ACCESS_KEY` pair
+1. `Regular Credentials`: Auth via an `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` pair
 2. `EC2 Credentials`: Auth from an EC2 instance via temporary credentials assigned to the machine
 3. `ECS Credentials`: Auth from an ECS instance via temporary credentials assigned to the task
 4. `Assume Role`: Auth via temporary credentials obtained from an STS AssumeRole request
@@ -21,15 +21,41 @@ SecretAccessKey=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
 Token=AQoDYXdzEJr...<remainder of security token>
 ```
 
-If the driver supports user provided custom AWS credential providers, then the driver MUST also test the above scenarios
-2-6 with a user provided `AWS_CREDENTIAL_PROVIDER` auth mechanism property. This value MUST be the default credential
-provider from the AWS SDK. If the default provider does not cover all scenarios above, those not covered MAY be skipped.
-In these tests the driver MUST also assert that the user provided credential provider was called at least once in each
-test.
+## Testing custom credential providers
 
-If the driver supports a custom AWS credential provider, it MUST verify the custom provider was used when testing. This
-may be via a custom function or object that wraps the calls to the custom provider and asserts that it was called at
-least once.
+If the driver supports custom AWS credential providers, the driver MUST test the following:
+
+### 1. Custom Credential Provider Authenticates
+
+Scenarios 1-6 from the previous section with a user provided `AWS_CREDENTIAL_PROVIDER` auth mechanism property. This
+credentials MAY be obtained from the default credential provider from the AWS SDK. If the default provider does not
+cover all scenarios above, those not covered MAY be skipped. In these tests the driver MUST also assert that the user
+provided credential provider was called in each test. This may be via a custom function or object that wraps the calls
+to the custom provider and asserts that it was called at least once. For test scenarios where the drivers tools scripts
+put the credentials in the MONGODB_URI, drivers MAY extract the credentials from the URI and return the AWS credentials
+directly from the custom provider instead of using the AWS SDK default provider.
+
+### 2. Custom Credential Provider Authentication Precedence
+
+#### Case 1: Credentials in URI Take Precedence
+
+Create a `MongoClient` configured with AWS auth and credentials in the URI. Example:
+`mongodb://<AccessKeyId>:<SecretAccessKey>@localhost:27017/?authMechanism=MONGODB-AWS`
+
+Configure a custom credential provider to pass valid AWS credentials. The provider must track if it was called.
+
+Expect authentication to succeed and the custom credential provider was *not* called.
+
+#### Case 2: Custom Provider Takes Precedence Over Environment Variables
+
+Run this test in an environment with AWS credentials configured as environment variables (e.g. `AWS_ACCESS_KEY_ID`,
+`AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN`)
+
+Create a `MongoClient` configured to use AWS auth. Example: `mongodb://localhost:27017/?authMechanism=MONGODB-AWS`.
+
+Configure a custom credential provider to pass valid AWS credentials. The provider must track if it was called.
+
+Expect authentication to succeed and the custom credential provider was called.
 
 ## Regular credentials
 

source/client-side-encryption/tests/README.md

@@ -3788,6 +3788,57 @@ class AutoEncryptionOpts {
 
 Assert that an error is thrown.
 
+#### Case 4: ClientEncryption with `credentialProviders` and valid environment variables.
+
+Ensure a valid `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are present in the environment.
+
+Create a MongoClient named `setupClient`.
+
+Create a [ClientEncryption](../client-side-encryption.md#clientencryption) object with the following options:
+
+```typescript
+class ClientEncryptionOpts {
+  keyVaultClient: <setupClient>,
+  keyVaultNamespace: "keyvault.datakeys",
+  kmsProviders: { "aws": {} },
+  credentialProviders: { "aws": <object/function that returns valid credentials from the secrets manager> }
+}
+```
+
+Use the client encryption to create a datakey using the "aws" KMS provider. This should successfully load and use the
+AWS credentials that were provided by the secrets manager for the remote provider. Assert the datakey was created and
+that the custom credential provider was called at least once.
+
+An example of this in Node.js:
+
+```typescript
+import { ClientEncryption, MongoClient } from 'mongodb';
+
+let calledCount = 0;
+const masterKey = {
+  region: '<aws region>',
+  key: '<key for arn>'
+};
+const keyVaultClient = new MongoClient(process.env.MONGODB_URI);
+const options = {
+  keyVaultNamespace: 'keyvault.datakeys',
+  kmsProviders: { aws: {} },
+  credentialProviders: {
+    aws: async () => {
+      calledCount++;
+      return {
+        accessKeyId: process.env.FLE_AWS_KEY,
+        secretAccessKey: process.env.FLE_AWS_SECRET
+      };
+    }
+  }
+};
+const clientEncryption = new ClientEncryption(keyVaultClient, options);
+const dk = await clientEncryption.createDataKey('aws', { masterKey });
+expect(dk).to.be.a(Binary);
+expect(calledCount).to.be.greaterThan(0);
+```
+
 ### 27. Text Explicit Encryption
 
 The Text Explicit Encryption tests utilize Queryable Encryption (QE) range protocol V2 and require MongoDB server 8.2.0+