MQE-1918: MFTF AWS Secrets Manager - Local Use

Author: jilu1

dev/tests/unit/Magento/FunctionalTestFramework/DataGenerator/Handlers/SecretStorage/AwsSecretManagerStorageTest.php renamed to dev/tests/unit/Magento/FunctionalTestFramework/DataGenerator/Handlers/SecretStorage/AwsSecretsManagerStorageTest.php

@@ -7,15 +7,15 @@
 namespace tests\unit\Magento\FunctionalTestFramework\DataGenerator\Handlers\SecretStorage;
 
 use Aws\SecretsManager\SecretsManagerClient;
-use Magento\FunctionalTestingFramework\DataGenerator\Handlers\SecretStorage\AwsSecretManagerStorage;
+use Magento\FunctionalTestingFramework\DataGenerator\Handlers\SecretStorage\AwsSecretsManagerStorage;
 use Aws\Result;
 use Magento\FunctionalTestingFramework\Util\MagentoTestCase;
 use ReflectionClass;
 
-class AwsSecretManagerStorageTest extends MagentoTestCase
+class AwsSecretsManagerStorageTest extends MagentoTestCase
 {
     /**
-     * Test encryption/decryption functionality in AwsSecretManagerStorage class.
+     * Test encryption/decryption functionality in AwsSecretsManagerStorage class.
      */
     public function testEncryptAndDecrypt()
     {
@@ -44,7 +44,7 @@ public function testEncryptAndDecrypt()
             });
 
         /** @var SecretsManagerClient */
-        $credentialStorage = new AwsSecretManagerStorage($testRegion, $testProfile);
+        $credentialStorage = new AwsSecretsManagerStorage($testRegion, $testProfile);
         $reflection = new ReflectionClass($credentialStorage);
         $reflection_property = $reflection->getProperty('client');
         $reflection_property->setAccessible(true);

docs/configuration.md

@@ -277,26 +277,26 @@ Example:
 CREDENTIAL_VAULT_SECRET_BASE_PATH=secret
 ```
 
-### CREDENTIAL_AWS_SECRET_MANAGER_REGION
+### CREDENTIAL_AWS_SECRETS_MANAGER_REGION
 
-The region that Aws Secret Manager is located.
+The region that AWS Secrets Manager is located.
 
 Example:
 
 ```conf
-# Region of Aws Secret Manager
-CREDENTIAL_AWS_SECRET_MANAGER_REGION=us-east-1
+# Region of AWS Secrets Manager
+CREDENTIAL_AWS_SECRETS_MANAGER_REGION=us-east-1
 ```
 
-### CREDENTIAL_AWS_SECRET_MANAGER_PROFILE
+### CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE
 
-The profile used to connect to Aws Secret Manager.
+The profile used to connect to AWS Secrets Manager.
 
 Example:
 
 ```conf
-# Profile used to connect to Aws Secret Manager.
-CREDENTIAL_AWS_SECRET_MANAGER_PROFILE=default
+# Profile used to connect to AWS Secrets Manager.
+CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE=default
 ```
 
 ### ENABLE_BROWSER_LOG

docs/credentials.md

@@ -7,7 +7,7 @@ Currently the MFTF supports three types of credential storage:
 
 -  **.credentials file**
 -  **HashiCorp Vault**
--  **Aws Secret Manager**
+-  **AWS Secrets Manager**
 
 ## Configure File Storage
 
@@ -136,22 +136,22 @@ CREDENTIAL_VAULT_ADDRESS=http://127.0.0.1:8200
 CREDENTIAL_VAULT_SECRET_BASE_PATH=secret
 ```
 
-## Configure Aws Secret Manager
+## Configure AWS Secrets Manager
 
-Aws Secrets Manager offers secret management that supports:
+AWS Secrets Manager offers secret management that supports:
 - Secret rotation with built-in integration for Amazon RDS, Amazon Redshift, and Amazon DocumentDB
 - Fine-grained policies and permissions
 - Audit secret rotation centrally for resources in the AWS Cloud, third-party services, and on-premises
 
 ### Prerequisites
 - AWS account
-- AWS Secret Manger is created and configured
-- IAM User or Role is created
+- AWS Secrets Manger is created and configured
+- IAM User or Role is created with appropriate AWS Secrets Manger access permission
 
-### Store secrets in Aws Secret Manager
+### Store secrets in AWS Secrets Manager
 
 #### Secrets format
-`Secret Name`, `Secret Key`, `Secret Value` are three key pieces of information to construct an Aws Secret. 
+`Secret Name`, `Secret Key`, `Secret Value` are three key pieces of information to construct an AWS Secret. 
 `Secret Key` and `Secret Value` can be any content you want to secure, `Secret Name` must follow the format:
 
 ```conf
@@ -172,18 +172,18 @@ mftf/magento/carriers_usps_password
 carriers_usps_password
 ```
 
-### Setup MFTF to use Aws Secret Manager
+### Setup MFTF to use AWS Secrets Manager
 
-To use Aws Secret Manager, the Aws region to connect to is required. You can set it through environment variable [`CREDENTIAL_AWS_SECRET_MANAGER_REGION`][] in `.env`.
+To use AWS Secrets Manager, the AWS region to connect to is required. You can set it through environment variable [`CREDENTIAL_AWS_SECRETS_MANAGER_REGION`][] in `.env`.
 
-MFTF uses the recommended [Default Credential Provider Chain][credential chain] to establish connection to Aws Secret Manager service. 
+MFTF uses the recommended [Default Credential Provider Chain][credential chain] to establish connection to AWS Secrets Manager service. 
 You can setup credentials according to [Default Credential Provider Chain][credential chain] and there is no MFTF specific setup required. 
-Optionally, however, you can explicitly set Aws profile through environment variable [`CREDENTIAL_AWS_SECRET_MANAGER_PROFILE`][] in `.env`.
+Optionally, however, you can explicitly set AWS profile through environment variable [`CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE`][] in `.env`.
 
 ```conf
-# Sample Aws Secret Manager configuration
-CREDENTIAL_AWS_SECRET_MANAGER_REGION=us-east-1
-CREDENTIAL_AWS_SECRET_MANAGER_PROFILE=default
+# Sample AWS Secrets Manager configuration
+CREDENTIAL_AWS_SECRETS_MANAGER_REGION=us-east-1
+CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE=default
 ```
 
 ## Configure multiple credential storage
@@ -192,7 +192,7 @@ It is possible and sometimes useful to setup and use multiple credential storage
 In this case, the MFTF tests are able to read secret data at runtime from all storage options, in this case MFTF use the following precedence:
 
 ```
-.credentials File > HashiCorp Vault > Aws Secret Manager
+.credentials File > HashiCorp Vault > AWS Secrets Manager
 ```
 <!-- {% raw %} -->
 
@@ -238,5 +238,5 @@ The MFTF tests delivered with Magento application do not use credentials and do
 [`CREDENTIAL_VAULT_ADDRESS`]: configuration.md#credential_vault_address
 [`CREDENTIAL_VAULT_SECRET_BASE_PATH`]: configuration.md#credential_vault_secret_base_path
 [credential chain]: https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials.html
-[`CREDENTIAL_AWS_SECRET_MANAGER_PROFILE`]: configuration.md#credential_aws_secret_manager_profile
-[`CREDENTIAL_AWS_SECRET_MANAGER_REGION`]: configuration.md#credential_aws_secret_manager_region
+[`CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE`]: configuration.md#credential_aws_secrets_manager_profile
+[`CREDENTIAL_AWS_SECRETS_MANAGER_REGION`]: configuration.md#credential_aws_secrets_manager_region

etc/config/.env.example

@@ -34,9 +34,9 @@ BROWSER=chrome
 #CREDENTIAL_VAULT_ADDRESS=http://127.0.0.1:8200
 #CREDENTIAL_VAULT_SECRET_BASE_PATH=secret
 
-#*** To use AWS Secret Manager to manage _CREDS secrets, uncomment and set region, profile is optional, when omitted, AWS default credential provider chain will be used ***#
-#CREDENTIAL_AWS_SECRET_MANAGER_PROFILE=default
-#CREDENTIAL_AWS_SECRET_MANAGER_REGION=us-east-1
+#*** To use AWS Secrets Manager to manage _CREDS secrets, uncomment and set region, profile is optional, when omitted, AWS default credential provider chain will be used ***#
+#CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE=default
+#CREDENTIAL_AWS_SECRETS_MANAGER_REGION=us-east-1
 
 #*** Uncomment these properties to set up a dev environment with symlinked projects ***#
 #TESTS_BP=

src/Magento/FunctionalTestingFramework/DataGenerator/Handlers/CredentialStore.php

@@ -9,17 +9,17 @@
 use Magento\FunctionalTestingFramework\Exceptions\TestFrameworkException;
 use Magento\FunctionalTestingFramework\DataGenerator\Handlers\SecretStorage\FileStorage;
 use Magento\FunctionalTestingFramework\DataGenerator\Handlers\SecretStorage\VaultStorage;
-use Magento\FunctionalTestingFramework\DataGenerator\Handlers\SecretStorage\AwsSecretManagerStorage;
+use Magento\FunctionalTestingFramework\DataGenerator\Handlers\SecretStorage\AwsSecretsManagerStorage;
 use Magento\FunctionalTestingFramework\Util\Path\UrlFormatter;
 
 class CredentialStore
 {
     const ARRAY_KEY_FOR_VAULT = 'vault';
     const ARRAY_KEY_FOR_FILE = 'file';
-    const ARRAY_KEY_FOR_AWS_SECRET_MANAGER = 'aws';
+    const ARRAY_KEY_FOR_AWS_SECRETS_MANAGER = 'aws';
 
     const CREDENTIAL_STORAGE_INFO = 'MFTF uses Credential Storage in the following precedence: '
-        . '.credentials file, HashiCorp Vault and AWS Secret Manager. '
+        . '.credentials file, HashiCorp Vault and AWS Secrets Manager. '
         . 'You need to configure at least one to use _CREDS in tests.';
 
     /**
@@ -77,15 +77,15 @@ private function __construct()
             }
         }
 
-        // Initialize AWS secret manager storage
-        $awsRegion = getenv('CREDENTIAL_AWS_SECRET_MANAGER_REGION');
-        $awsProfile = getenv('CREDENTIAL_AWS_SECRET_MANAGER_PROFILE');
+        // Initialize AWS Secrets Manager storage
+        $awsRegion = getenv('CREDENTIAL_AWS_SECRETS_MANAGER_REGION');
+        $awsProfile = getenv('CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE');
         if ($awsRegion !== false) {
             if ($awsProfile === false) {
                 $awsProfile = null;
             }
             try {
-                $this->credStorage[self::ARRAY_KEY_FOR_AWS_SECRET_MANAGER] = new AwsSecretManagerStorage(
+                $this->credStorage[self::ARRAY_KEY_FOR_AWS_SECRETS_MANAGER] = new AwsSecretsManagerStorage(
                     $awsRegion,
                     $awsProfile
                 );
@@ -109,8 +109,8 @@ private function __construct()
      */
     public function getSecret($key)
     {
-        // Get secret data from storage according to the order they are stored
-        // File storage is preferred over vault storage to allow local secret value overriding remote secret value
+        // Get secret data from storage according to the order they are stored which follows this precedence:
+        // FileStorage > VaultStorage > AwsSecretsManagerStorage
         foreach ($this->credStorage as $storage) {
             $value = $storage->getEncryptedValue($key);
             if (null !== $value) {

src/Magento/FunctionalTestingFramework/DataGenerator/Handlers/SecretStorage/AwsSecretManagerStorage.php renamed to src/Magento/FunctionalTestingFramework/DataGenerator/Handlers/SecretStorage/AwsSecretsManagerStorage.php

@@ -15,15 +15,15 @@
 use InvalidArgumentException;
 use Exception;
 
-class AwsSecretManagerStorage extends BaseStorage
+class AwsSecretsManagerStorage extends BaseStorage
 {
     /**
      * Mftf project path
      */
     const MFTF_PATH = 'mftf';
 
     /**
-     * AWS Secret Manager version
+     * AWS Secrets Manager version
      *
      * Last tested version '2017-10-17'
      */
@@ -37,7 +37,7 @@ class AwsSecretManagerStorage extends BaseStorage
     private $client = null;
 
     /**
-     * AwsSecretManagerStorage constructor
+     * AwsSecretsManagerStorage constructor
      *
      * @param string $region
      * @param string $profile
@@ -47,7 +47,7 @@ class AwsSecretManagerStorage extends BaseStorage
     public function __construct($region, $profile = null)
     {
         parent::__construct();
-        $this->createAwsSecretManagerClient($region, $profile);
+        $this->createAwsSecretsManagerClient($region, $profile);
     }
 
     /**
@@ -65,8 +65,8 @@ public function getEncryptedValue($key)
         }
 
         if (MftfApplicationConfig::getConfig()->verboseEnabled()) {
-            LoggingUtil::getInstance()->getLogger(VaultStorage::class)->debug(
-                "Retrieving secret for key name {$key} from AWS Secret Manager"
+            LoggingUtil::getInstance()->getLogger(AwsSecretsManagerStorage::class)->debug(
+                "Retrieving value for key name {$key} from AWS Secrets Manager"
             );
         }
 
@@ -79,7 +79,7 @@ public function getEncryptedValue($key)
                 . $vendor
                 . '/'
                 . $key;
-            // Read value by id from AWS Secret Manager, and parse the result
+            // Read value by id from AWS Secrets Manager, and parse the result
             $value = $this->parseAwsSecretResult(
                 $this->client->getSecretValue(['SecretId' => $secretId]),
                 $key
@@ -90,14 +90,14 @@ public function getEncryptedValue($key)
         } catch (AwsException $e) {
             $error = $e->getAwsErrorCode();
             if (MftfApplicationConfig::getConfig()->verboseEnabled()) {
-                LoggingUtil::getInstance()->getLogger(VaultStorage::class)->debug(
-                    "AWS error code: {$error}. Unable to read secret for key {$key} from AWS Secret Manager"
+                LoggingUtil::getInstance()->getLogger(AwsSecretsManagerStorage::class)->debug(
+                    "AWS error code: {$error}. Unable to read value for key {$key} from AWS Secrets Manager"
                 );
             }
         } catch (\Exception $e) {
             if (MftfApplicationConfig::getConfig()->verboseEnabled()) {
-                LoggingUtil::getInstance()->getLogger(VaultStorage::class)->debug(
-                    "Unable to read secret for key {$key} from AWS Secret Manager"
+                LoggingUtil::getInstance()->getLogger(AwsSecretsManagerStorage::class)->debug(
+                    "Unable to read value for key {$key} from AWS Secrets Manager"
                 );
             }
         }
@@ -118,39 +118,39 @@ private function parseAwsSecretResult($awsResult, $key)
         if (isset($awsResult['SecretString'])) {
             $rawSecret = $awsResult['SecretString'];
         } else {
-            throw new TestFrameworkException("Error parsing AWS secret result");
+            throw new TestFrameworkException("Error parsing result from AWS Secrets Manager");
         }
         $secret = json_decode($rawSecret, true);
         if (isset($secret[$key])) {
             return $secret[$key];
         }
-        throw new TestFrameworkException("Error parsing AWS secret result");
+        throw new TestFrameworkException("Error parsing result from AWS Secrets Manager");
     }
 
     /**
-     * Create Aws Secret Manager client
+     * Create Aws Secrets Manager client
      *
      * @param string $region
      * @param string $profile
      * @return void
      * @throws TestFrameworkException
      * @throws InvalidArgumentException
      */
-    private function createAwsSecretManagerClient($region, $profile)
+    private function createAwsSecretsManagerClient($region, $profile)
     {
         if (null !== $this->client) {
             return;
         }
 
-        // Create AWS Secret Manager client
+        // Create AWS Secrets Manager client
         $this->client = new SecretsManagerClient([
             'profile' => $profile,
             'region' => $region,
             'version' => self::LATEST_VERSION
         ]);
 
         if ($this->client === null) {
-            throw new TestFrameworkException("Unable to create AWS Secret Manager client");
+            throw new TestFrameworkException("Unable to create AWS Secrets Manager client");
         }
     }
 }