|
| 1 | +--- |
| 2 | +title: "Initializing an RDS Database with AWS CDK and LocalStack" |
| 3 | +description: Learn how to provision and initialize Amazon RDS databases locally using AWS CDK and LocalStack. This tutorial demonstrates database schema creation, data seeding, and testing with Cloud Pods for reproducible development environments. |
| 4 | +services: |
| 5 | +- rds |
| 6 | +- lambda |
| 7 | +- sec |
| 8 | +platform: |
| 9 | +- JavaScript |
| 10 | +- Node.js |
| 11 | +deployment: |
| 12 | +- aws-cdk |
| 13 | +- CloudFormation |
| 14 | +pro: true |
| 15 | +leadimage: "rds-database-initialization-featured-image.png" |
| 16 | +--- |
| 17 | + |
| 18 | +## Introduction |
| 19 | + |
| 20 | +Database initialization is a critical aspect of application development and testing. Setting up databases with proper schemas and seed data consistently across development, testing, and CI environments can be challenging. Amazon RDS provides managed database services, but testing database initialization scripts and configurations requires a reliable local development environment. |
| 21 | + |
| 22 | +[LocalStack Pro](https://app.localstack.cloud/) enables you to emulate Amazon RDS, Lambda, and Secrets Manager locally, allowing you to develop and test database initialization workflows without connecting to AWS. This approach accelerates development cycles, reduces costs, and ensures consistent environments across different stages of your development pipeline. |
| 23 | + |
| 24 | +In this tutorial, we will demonstrate how to provision and initialize an Amazon RDS database using AWS CDK and LocalStack. We'll create a Lambda function that executes custom SQL scripts to set up database schemas and seed data. Additionally, we'll explore how Cloud Pods can streamline CI workflows by providing pre-seeded database environments. |
| 25 | + |
| 26 | +## Prerequisites |
| 27 | + |
| 28 | +For this tutorial, you will need: |
| 29 | + |
| 30 | +- [LocalStack Pro](https://localstack.cloud/pricing/) with a valid auth token |
| 31 | +- [AWS CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) with the [`awslocal` wrapper](https://docs.localstack.cloud/user-guide/integrations/aws-cli/#localstack-aws-cli-awslocal) |
| 32 | +- [AWS CDK](https://docs.localstack.cloud/user-guide/integrations/aws-cdk/) with the [`cdklocal`](https://www.npmjs.com/package/aws-cdk-local) wrapper |
| 33 | +- [Node.js](https://nodejs.org/en/download/) (version 16 or later) |
| 34 | +- [Docker](https://docker.io/) |
| 35 | +- MySQL or PostgreSQL client (for testing database connections) |
| 36 | +- [`make`](https://www.gnu.org/software/make/) (optional, but recommended) |
| 37 | + |
| 38 | +## Architecture |
| 39 | + |
| 40 | +The following diagram shows the architecture that this sample application builds and deploys: |
| 41 | + |
| 42 | + |
| 43 | + |
| 44 | +The architecture consists of: |
| 45 | + |
| 46 | +- **Amazon RDS**: The central database instance that will be initialized and pre-filled with data |
| 47 | +- **AWS Lambda**: A Node.js function that executes SQL scripts to initialize the database schema and seed data |
| 48 | +- **AWS Secrets Manager**: Stores database credentials and connection details securely |
| 49 | +- **CloudFormation Custom Resource**: Triggers the Lambda function during deployment to perform database initialization |
| 50 | + |
| 51 | +The initialization process works as follows: |
| 52 | +1. CDK deploys the RDS instance and related resources |
| 53 | +2. A CloudFormation Custom Resource triggers the Lambda function |
| 54 | +3. The Lambda function retrieves database credentials from Secrets Manager |
| 55 | +4. The function connects to the RDS instance and executes initialization SQL scripts |
| 56 | +5. Database tables are created and populated with seed data |
| 57 | + |
| 58 | +## Getting Started |
| 59 | + |
| 60 | +### Project Setup |
| 61 | + |
| 62 | +First, clone the sample repository and install dependencies: |
| 63 | + |
| 64 | +```bash |
| 65 | +git clone https://github.com/localstack-samples/sample-cdk-rds-database-initialization.git |
| 66 | +cd sample-cdk-rds-database-initialization |
| 67 | +``` |
| 68 | + |
| 69 | +Install the project dependencies: |
| 70 | + |
| 71 | +```bash |
| 72 | +npm install |
| 73 | +# or if you prefer using make |
| 74 | +make install |
| 75 | +``` |
| 76 | + |
| 77 | +### Configure LocalStack |
| 78 | + |
| 79 | +Start LocalStack with your auth token: |
| 80 | + |
| 81 | +```bash |
| 82 | +localstack auth set-token <your-auth-token> |
| 83 | +localstack start |
| 84 | +``` |
| 85 | + |
| 86 | +> **Note**: By default, LocalStack uses the MariaDB engine for RDS (see [RDS documentation](https://docs.localstack.cloud/user-guide/aws/rds/#mysql-engine)). To use the real MySQL engine in a separate Docker container, set the environment variable `RDS_MYSQL_DOCKER=1`. |
| 87 | +
|
| 88 | +## Deployment |
| 89 | + |
| 90 | +Deploy the sample application using CDK: |
| 91 | + |
| 92 | +```bash |
| 93 | +make deploy |
| 94 | +# or manually: |
| 95 | +cdklocal deploy |
| 96 | +``` |
| 97 | + |
| 98 | +The deployment process will: |
| 99 | +1. Create an RDS MySQL instance |
| 100 | +2. Set up a Secrets Manager secret with database credentials |
| 101 | +3. Deploy a Lambda function with database initialization code |
| 102 | +4. Execute the initialization script via CloudFormation Custom Resource |
| 103 | + |
| 104 | +After successful deployment, you'll see output similar to: |
| 105 | + |
| 106 | +```bash |
| 107 | +Outputs: |
| 108 | +RdsInitExample.RdsInitFnResponse = {"status":"OK","results":[/*...SQL operations...*/]} |
| 109 | +RdsInitExample.functionName = my-lambda-rds-query-helper |
| 110 | +RdsInitExample.secretName = /rdsinitexample/rds/creds/mysql-01 |
| 111 | +Stack ARN: |
| 112 | +arn:aws:cloudformation:us-east-1:000000000000:stack/RdsInitExample/3f53b7bd |
| 113 | + |
| 114 | +✨ Total time: 80.21s |
| 115 | + |
| 116 | +CDK deployed successfully. |
| 117 | +``` |
| 118 | + |
| 119 | +The outputs include: |
| 120 | +- `RdsInitFnResponse`: Results from executing the database initialization script |
| 121 | +- `functionName`: Lambda function name for running test queries |
| 122 | +- `secretName`: Secrets Manager secret containing database connection details |
| 123 | + |
| 124 | +## Testing the Application |
| 125 | + |
| 126 | +The sample application creates a database with tables and sample data. Let's verify the initialization was successful by running queries against the database. |
| 127 | + |
| 128 | +### Querying the Database via Lambda |
| 129 | + |
| 130 | +The deployed Lambda function `my-lambda-rds-query-helper` can execute SQL queries against the initialized database. The function requires two parameters: |
| 131 | +- `sqlQuery`: The SQL command to execute |
| 132 | +- `secretName`: The Secrets Manager secret containing database credentials |
| 133 | + |
| 134 | +**For AWS CLI v1:** |
| 135 | +```bash |
| 136 | +awslocal lambda invoke \ |
| 137 | + --function-name my-lambda-rds-query-helper \ |
| 138 | + --payload '{"sqlQuery": "select Author from books", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ |
| 139 | + output |
| 140 | +``` |
| 141 | + |
| 142 | +**For AWS CLI v2:** |
| 143 | +```bash |
| 144 | +awslocal lambda invoke \ |
| 145 | + --cli-binary-format raw-in-base64-out \ |
| 146 | + --function-name my-lambda-rds-query-helper \ |
| 147 | + --payload '{"sqlQuery": "select Author from books", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ |
| 148 | + output |
| 149 | +``` |
| 150 | + |
| 151 | +View the results: |
| 152 | +```bash |
| 153 | +cat output |
| 154 | +``` |
| 155 | + |
| 156 | +Expected output: |
| 157 | +```json |
| 158 | +{ |
| 159 | + "status": "SUCCESS", |
| 160 | + "results": [ |
| 161 | + {"Author": "Jane Doe"}, |
| 162 | + {"Author": "Jane Doe"}, |
| 163 | + {"Author": "LocalStack"} |
| 164 | + ] |
| 165 | +} |
| 166 | +``` |
| 167 | + |
| 168 | +You can also run more detailed queries to explore the data: |
| 169 | + |
| 170 | +**Query all book details:** |
| 171 | +```bash |
| 172 | +awslocal lambda invoke \ |
| 173 | + --cli-binary-format raw-in-base64-out \ |
| 174 | + --function-name my-lambda-rds-query-helper \ |
| 175 | + --payload '{"sqlQuery": "SELECT * FROM books LIMIT 5", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ |
| 176 | + output && cat output |
| 177 | +``` |
| 178 | + |
| 179 | +### Testing Different SQL Operations |
| 180 | + |
| 181 | +Test various database operations to verify the initialization: |
| 182 | + |
| 183 | +**Check table structure:** |
| 184 | +```bash |
| 185 | +awslocal lambda invoke \ |
| 186 | + --cli-binary-format raw-in-base64-out \ |
| 187 | + --function-name my-lambda-rds-query-helper \ |
| 188 | + --payload '{"sqlQuery": "DESCRIBE books", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ |
| 189 | + output && cat output |
| 190 | +``` |
| 191 | + |
| 192 | +**Count records:** |
| 193 | +```bash |
| 194 | +awslocal lambda invoke \ |
| 195 | + --cli-binary-format raw-in-base64-out \ |
| 196 | + --function-name my-lambda-rds-query-helper \ |
| 197 | + --payload '{"sqlQuery": "SELECT COUNT(*) as total_books FROM books", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ |
| 198 | + output && cat output |
| 199 | +``` |
| 200 | + |
| 201 | +**Filter by author:** |
| 202 | +```bash |
| 203 | +awslocal lambda invoke \ |
| 204 | + --cli-binary-format raw-in-base64-out \ |
| 205 | + --function-name my-lambda-rds-query-helper \ |
| 206 | + --payload '{"sqlQuery": "SELECT title, published_year FROM books WHERE author = \"George Orwell\"", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ |
| 207 | + output && cat output |
| 208 | +``` |
| 209 | + |
| 210 | +### Connecting Directly to the Database |
| 211 | + |
| 212 | +For more comprehensive testing, you can connect directly to the RDS instance using a MySQL client. First, retrieve the database connection details: |
| 213 | + |
| 214 | +```bash |
| 215 | +# Get the database endpoint |
| 216 | +awslocal rds describe-db-instances --query 'DBInstances[0].Endpoint.Address' --output text |
| 217 | + |
| 218 | +# Get credentials from Secrets Manager |
| 219 | +awslocal secretsmanager get-secret-value --secret-id /rdsinitexample/rds/creds/mysql-01 --query SecretString --output text |
| 220 | +``` |
| 221 | + |
| 222 | +Connect using the MySQL command-line client: |
| 223 | +```bash |
| 224 | +mysql -h <endpoint> -P 4510 -u <username> -p<password> <database_name> |
| 225 | +``` |
| 226 | + |
| 227 | +Once connected, you can run SQL queries directly: |
| 228 | +```sql |
| 229 | +USE your_database_name; |
| 230 | +SHOW TABLES; |
| 231 | +SELECT * FROM books; |
| 232 | +``` |
| 233 | + |
| 234 | +### Running Integration Tests |
| 235 | + |
| 236 | +Execute the complete test suite to validate all functionality: |
| 237 | + |
| 238 | +```bash |
| 239 | +make test |
| 240 | +``` |
| 241 | + |
| 242 | +This will run end-to-end tests that verify: |
| 243 | +- Database connectivity |
| 244 | +- Schema creation |
| 245 | +- Data seeding |
| 246 | +- Query operations |
| 247 | +- Error handling |
| 248 | + |
| 249 | +## Conclusion |
| 250 | + |
| 251 | +This tutorial demonstrated how to provision and initialize an Amazon RDS database locally using AWS CDK and LocalStack. You learned how to: |
| 252 | + |
| 253 | +- **Set up LocalStack Pro** for local AWS service emulation |
| 254 | +- **Deploy RDS infrastructure** using AWS CDK and CloudFormation |
| 255 | +- **Initialize database schemas and data** via Lambda functions during deployment |
| 256 | +- **Test the initialized database** using both Lambda queries and direct MySQL connections |
| 257 | +- **Create repeatable database setups** for development and testing environments |
| 258 | + |
| 259 | +This approach provides several key benefits for database development: |
| 260 | + |
| 261 | +- **Consistent Environments**: Reproducible database setup across development, testing, and CI environments |
| 262 | +- **Faster Development Cycles**: Test database initialization scripts locally without AWS dependencies |
| 263 | +- **Cost-Effective Testing**: No AWS charges during development and testing phases |
| 264 | +- **Reliable CI/CD**: Automated database setup ensures consistent test environments |
| 265 | + |
| 266 | +The patterns demonstrated in this tutorial provide a solid foundation for managing database initialization in your LocalStack-based development workflow, enabling you to develop and test database-driven applications more efficiently and reliably. |
0 commit comments