Release test for AWS IAM Postgres passwordless #192
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add database_passwordless_aws_use_iam and database_passwordless_aws_region variables - Add DATABASE_AUTH_USE_AWS_IAM and DATABASE_AUTH_AWS_DB_REGION environment variables - Update both runtime_container_engine_config and tfe_init modules - Follow Redis passwordless authentication pattern for PostgreSQL
- Add Database.Passwordless.AWSUseInstanceProfile structure to tfe_init template variables - Add database_aws_iam_auth_enabled and database_aws_iam_region variables to settings module - This provides the correct nested structure that TFE templates expect for IAM authentication - Templates look for .Database.Passwordless.AWSUseInstanceProfile to set DATABASE_AUTH_USE_AWS_IAM Fixes: PostgreSQL passwordless authentication was failing because DATABASE_AUTH_USE_AWS_IAM was not being set correctly
- Added database_iam_username variable to tfe_init module - Updated template to use proper database connection variables - Template now automatically creates PostgreSQL IAM user with proper permissions - Fixed variable reference from database_user to database_iam_username - Added comprehensive error handling and database readiness checks This completes the automation requirements for PostgreSQL passwordless authentication release tests.
Set AWS_DEFAULT_REGION environment variable before AWS CLI commands:
- Export AWS_DEFAULT_REGION=${database_aws_iam_region}
- Ensures AWS CLI commands in PostgreSQL IAM user creation have proper region
- Resolves 'You must specify a region' AWS CLI errors
This should fix the user_data script failures and allow proper PostgreSQL IAM user creation.
TFE's auth.go code requires DATABASE_URL environment variable for IAM authentication to work: - Added DATABASE_URL construction using database connection parameters - Format: postgresql://user:password@host:5432/dbname?parameters - Critical for TFE to properly use AWS IAM database authentication This should resolve the 'database password must be set' configuration error.
Handle null password properly in DATABASE_URL construction: - Only include password part if database_password is not null - Format: postgresql://user@host:5432/dbname (no password part for IAM auth) - Ensures valid PostgreSQL connection string for IAM authentication This should fix DATABASE_URL format for pgmultiauth IAM authentication.
TFE requires a password in the DATABASE_URL even when IAM authentication is enabled. Added 'aws-iam-auth' placeholder password to ensure proper DATABASE_URL format: postgresql://user:aws-iam-auth@host:5432/db?params
- Set DATABASE_URL to null for IAM auth (generated at runtime) - Add RDS IAM token generation function in tfe.sh.tpl - Generate authentication tokens using 'aws rds generate-db-auth-token' - Inject DATABASE_URL with actual IAM token into Docker Compose at runtime - Use IAM username and token-based authentication - Ensure proper URL encoding for special characters in tokens
Based on Atlas app implementation and TFE auth.go analysis: - Set DATABASE_URL to base connection string WITHOUT password for IAM auth - Remove runtime token generation (pgmultiauth library handles this internally) - Let TFE's pgmultiauth library generate IAM tokens automatically - DATABASE_URL format: postgresql://user@host:5432/db?sslmode=require - DATABASE_AUTH_USE_AWS_IAM=true triggers IAM authentication in pgmultiauth
database_host already includes ':5432' port from RDS endpoint, so don't add ':5432' again. This was causing malformed DATABASE_URL which resulted in 'DATABASE_URL: null' in compose.yaml. Correct format: postgresql://user@host:5432/db?params
- Set TFE_DATABASE_PASSWORD to null when IAM auth is enabled - Add TFE_DATABASE_USE_INSTANCE_PROFILE for AWS instance profile - Filter out null values so TFE_DATABASE_PASSWORD doesn't appear in compose at all - This prevents TFE config validation from checking for password when using IAM
…nt variables - Add TFE_DATABASE_PASSWORDLESS_AWS_USE_INSTANCE_PROFILE mapping to fix TFE config validation - Add TFE_DATABASE_PASSWORDLESS_AWS_REGION for completeness - These environment variables map to Database.Passwordless struct fields in TFE config - Resolves 502 error by ensuring TFE validation recognizes IAM auth is enabled
- Set TFE_DATABASE_PASSWORD to empty string for IAM auth (required by TFE) - Add DATABASE_AUTH_AWS_SERVICE_NAME=rds-db for Atlas IAM provider - Fix DATABASE_URL format with empty password placeholder (:@) - Remove duplicate port from DATABASE_URL (host already includes port) These fixes address the 502 errors by ensuring Atlas can properly: 1. Initialize AWS IAM token provider with correct service name 2. Parse database connection parameters from properly formatted URL 3. Generate IAM tokens for PostgreSQL RDS connections
Remove DATABASE_AUTH_* and DATABASE_URL variables as they should not be required for PostgreSQL IAM authentication. The TFE core system should handle IAM authentication using only the TFE_DATABASE_* variables.
- Add SSM document execution in user_data template - Execute SSM command during TFE FDO startup to create IAM user automatically - Add postgres_iam_setup_ssm_document variable to tfe_init module - Wait for command completion and log results - Prevents 502 errors caused by missing IAM user in PostgreSQL database
- Compress PostgreSQL IAM setup script significantly - Reduce verbose logging to minimize user_data size - Keep essential functionality: SSM execution and status monitoring - Use shorter variable names and compact loops
- Add IMDSv2 token support with IMDSv1 fallback for metadata access - Fix bash syntax errors in conditional statements and loops - Add proper error handling and validation for SSM commands - Use while loops instead of bash ranges for compatibility - Add metadata debugging output for troubleshooting
- Remove complex SSM command execution that was causing failures - Create IAM user directly in user_data script using psql - Install PostgreSQL client and handle database connection inline - Use template variables for database connection parameters - Eliminate dependency on SSM document and metadata service - Simpler, more reliable approach for IAM user creation
- Compress entire IAM setup into 4 concise lines - Remove all verbose logging and error handling to save space - Use compact for loop with seq instead of while loops - Combine all SQL grants into single psql command - Essential functionality preserved in minimal footprint
- Add detailed logging for each step of the PostgreSQL setup - Add retry logic for PostgreSQL client installation - Add proper connectivity testing with version check - Add user verification after creation - Add explicit error handling with exit 1 on failure - Add more descriptive log messages to debug cloud-init issues
PostgreSQL IAM user creation is now handled by null_resource in terraform-aws-terraform-enterprise database module instead of user_data script. This provides better error handling and ensures the user is created before TFE container starts.
- Add robust PostgreSQL IAM user creation back to EC2 user_data - Include proper error handling and logging - EC2 instances have network access to RDS unlike CI machines - Compact but complete implementation for AWS size limits
- Reduced from ~50 lines to ~15 lines of PostgreSQL setup - Install specific postgresql-client-16 package - Compact single-line SQL command for IAM user creation - Reduced logging but kept essential status messages - File now 10KB vs previous ~12KB to ensure AWS compliance - Maintains all required functionality for IAM user creation
- Remove unused variable from tfe_init module - Update user_data template condition to use database_iam_username - Clean up SSM document approach in favor of direct automation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
This release test tests the passwordless authentication to AWS postgres using IAM based token.
Relates OR Closes #(https://hashicorp.atlassian.net/browse/IND-4009)
How has this been tested?
The release test ran on Github:https://github.com/hashicorp/terraform-enterprise/actions/runs/19146818577/job/54726521821
Checked the TFE logs, database connection was success.
Did you add a new setting?
No
CI/CD: https://github.com/hashicorp/terraform-enterprise/actions/runs/19464088212/job/55694964359
Screenshot:
