Skip to content

Commit f04225b

Browse files
cjobermaiercjobermaier
committed
build preview
1 parent c1fbe29 commit f04225b

File tree

3 files changed

+323
-0
lines changed

3 files changed

+323
-0
lines changed

‎content/well-architected-framework/data/docs-nav-data.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -425,6 +425,19 @@
425425
{
426426
"title": "Decommission resources",
427427
"path": "optimize-systems/lifecycle-management/decommission-infrastructure"
428+
},
429+
{
430+
"title": "Tag cloud resources",
431+
"path": "optimize-systems/lifecycle-management/tag-cloud-resources"
432+
}
433+
]
434+
},
435+
{
436+
"title": "Manage cost",
437+
"routes": [
438+
{
439+
"title": "Create cloud budgets",
440+
"path": "optimize-systems/manage-cost/create-cloud-budgets"
428441
}
429442
]
430443
},

‎content/well-architected-framework/docs/docs/optimize-systems/lifecycle-management/tag-cloud-resources.mdx‎

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
---
2+
page_title: Tag cloud resources
3+
description: Implement cloud resource tagging best practices with Terraform for AWS, Azure, and GCP. Learn to automate tags, enforce policies, and optimize cost allocation using infrastructure as code.
4+
---
5+
6+
# Tag cloud resources
7+
8+
Managing thousands of cloud resources across regions, environments, and teams is complex without a systematic approach. Tags are key-value pairs that help you manage, identify, organize, locate, and filter resources. It is important to have a clear, well-defined cloud resource tagging strategy. You can also use tags to track cost allocation and usage, and automate resource management tasks.
9+
[AWS](https://docs.aws.amazon.com/whitepapers/latest/tagging-best-practices/tagging-best-practices.html), [Azure](https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/resource-tagging), and [IBM](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-tagging-resources) maintain best practices and strategies for tagging your cloud resources. Follow these best practices when creating your tagging strategy. You can apply these concepts to other infrastructure providers.
10+
11+
You can implement your tagging strategy using infrastructure as code (IaC) and enforce compliance with policy as code, preventing the deployment of resources that don't meet your tagging requirements.
12+
13+
When you implement a tagging strategy, you gain the following benefits:
14+
- **Easier resource tracking:** Find resources by environment, owner, or other custom tag. For example, quickly find all servers in the 'dev' environment.
15+
- **Granular cost allocation:** Track costs by project, team, or application.
16+
- **Tag-based resource automation:** Automate resource management tasks based on tags, such as starting or stopping instances.
17+
- **Default resource compliance:** Enforce tagging policies to ensure all resources are tagged correctly.
18+
19+
## Deploy tags using infrastructure as code
20+
21+
Consistent implementation of your tagging strategy helps you track infrastructure costs, manage resources, and ensure compliance. When you use an inconsistent tagging strategy, such as manual tagging, you may end up with resources with incorrect or missing tags.
22+
23+
When you manage your infrastructure with Terraform, you can define tags within your modules. Terraform will automatically apply these tags to all resources it creates.
24+
25+
The following is an example of tags defined in a Terraform module:
26+
27+
```hcl
28+
resource "aws_instance" "web_server" {
29+
ami = "ami-0c55b159cbfafe1f0"
30+
instance_type = "t3.micro"
31+
32+
tags = {
33+
Name = "web-server-prod"
34+
Environment = "production"
35+
Owner = "platform-team"
36+
CostCenter = "engineering"
37+
Application = "website"
38+
}
39+
}
40+
41+
```
42+
43+
The AWS and GCP Terraform providers let you add default tags to all resources they create, making it easier to implement a consistent tagging strategy across all the resources you manage with Terraform. You can then override these default tags on a per-resource basis. Default tags ensure that all resources have the minimum required tags.
44+
45+
The following is an example of default tags:
46+
47+
```hcl
48+
49+
provider "aws" {
50+
profile = "default"
51+
region = "us-east-2"
52+
53+
default_tags {
54+
tags = {
55+
Environment = "Test"
56+
Service = "Payment API"
57+
}
58+
}
59+
}
60+
```
61+
62+
HashiCorp resources:
63+
64+
- AWS provider [resource tagging](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/resource-tagging)
65+
- [AWS default tags](https://developer.hashicorp.com/terraform/tutorials/aws/aws-default-tags)
66+
- [GCP default tags](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#default_labels-1)
67+
- Learn how to [configure default tags for AWS resources](/terraform/tutorials/aws/aws-default-tags)
68+
69+
## Enforce tagging strategy
70+
71+
Once you define and implement your tagging strategy using infrastructure as code, you can enforce it to prevent the deployment of resources that do not comply.
72+
73+
### Use the Terraform validation block
74+
75+
You can use the [Terraform validation block](/terraform/language/values/variables#validation) to enforce tagging policies. The validation block allows you to define custom validation rules for input variables. You can use this feature to ensure that the resources you tag follow your tagging strategy.
76+
77+
The following is an example of a Terraform validation block that ensures the `environment` tag is set to either `dev`, `staging`, or `prod`:
78+
79+
```hcl
80+
variable "environment" {
81+
type = string
82+
description = "Environment name for resource tagging"
83+
84+
validation {
85+
condition = contains(["dev", "staging", "prod"], var.environment)
86+
error_message = "Environment must be one of: dev, staging, prod."
87+
}
88+
}
89+
```
90+
91+
If you create a resource with an invalid `environment` tag, Terraform returns an error and prevents the deployment.
92+
93+
The following tag passes the validation:
94+
95+
```hcl
96+
environment = "prod"
97+
```
98+
99+
The following tag fails the validation due to not meeting the condition of being `dev`, `staging`, or `prod`:
100+
101+
```hcl
102+
environment = "testing"
103+
```
104+
105+
### Use policy as code
106+
107+
For more advanced enforcement of your tagging strategy, you can use policy as code tools such as HashiCorp Sentinel or the Open Policy Agent (OPA) to create policies that enforce tagging rules. You can integrate these policies into your CI/CD pipelines to ensure that all resources comply with your tagging strategy before deployment.
108+
109+
The following is an example of a [`Pass` or `Fail` Sentinel policy](/terraform/tutorials/policy/sentinel-policy#review-your-policy) that ensures that all AWS EC2 instances have a `Name` tag:
110+
111+
```script
112+
import "tfplan/v2" as tfplan
113+
114+
# Get all AWS instances from all modules
115+
ec2_instances = filter tfplan.resource_changes as _, rc {
116+
rc.type is "aws_instance" and
117+
(rc.change.actions contains "create" or rc.change.actions is ["update"])
118+
}
119+
120+
# Mandatory Instance Tags
121+
mandatory_tags = [
122+
"Name",
123+
]
124+
125+
# Rule to enforce "Name" tag on all instances
126+
mandatory_instance_tags = rule {
127+
all ec2_instances as _, instance {
128+
all mandatory_tags as mt {
129+
instance.change.after.tags contains mt
130+
}
131+
}
132+
}
133+
134+
main = rule {
135+
mandatory_instance_tags else true
136+
}
137+
```
138+
139+
You can write similar policies with OPA and HCP Terraform. Refer to the following external resources for more information.
140+
141+
HashiCorp resources:
142+
143+
- Read about the [Terraform validation block](/terraform/language/values/variables#validation)
144+
- Write a [Sentinel policy for a Terraform deployment](/terraform/tutorials/policy/sentinel-policy) to ensure that the EC2 instance has a `Name` tag.
145+
- [Get started with Sentinel](/sentinel/tutorials/get-started)
146+
- Learn how to [define Open Policy Agent policies for HCP Terraform](/terraform/enterprise/policy-enforcement/define-policies/opa)
147+
- [HCP Terraform policy enforcement overview](/terraform/enterprise/policy-enforcement)
148+
149+
External resources:
150+
- Use [OPA to write policies](https://www.openpolicyagent.org/docs/terraform) ensuring all resources have tags before you create them.
151+
152+
## Next steps
153+
154+
In this section of Manage cost, you learned how to tag resources using infrastructure as code and enforce tagging policies. Tag resources is part of the Optimize systems pillar.
155+
156+
To learn more about how to manage our resources, visit the following resources:
157+
- [Implement data management policies](/well-architected-framework/optimize-systems/lifecycle-management/data-management)
158+
- [Decommission resources](/well-architected-framework/optimize-systems/lifecycle-management/decommission-infrastructure)

‎content/well-architected-framework/docs/docs/optimize-systems/manage-cost/create-cloud-budgets.mdx‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
---
2+
page_title: Create cloud budgets
3+
description:
4+
---
5+
6+
# Create cloud budgets
7+
8+
Cloud spending can quickly get out of control without proper oversight and controls. According to the [2023 HashiCorp State of Cloud Strategy Survey](https://www.hashicorp.com/en/blog/hashicorp-state-of-cloud-strategy-survey-2023-maturity-drives-operational-efficiency), 94% of respondents experienced avoidable cloud costs. Proactive budget creation, automated alerts, and anomaly detection gives you the visibility and control you need to maintain predictable spending and prevent cost overruns before they occur.
9+
10+
Implementing a budget provides you with the following benefits:
11+
12+
- **Visibility into cloud spending:** Understand where your money is going.
13+
- **Proactive cost management:** Take action before costs exceed budgets.
14+
- **Notification of spending anomalies:** Get alerts when spending patterns change.
15+
- **Ability to improve financial planning and forecasting:** Use historical data to make informed budget decisions.
16+
17+
<Note>
18+
19+
The Terraform example in this document has a `tags` block. Refer to the [Tag cloud resources](/well-architected-framework/docs/docs/optimize-systems/lifecycle-management/tag-cloud-resources) document to learn about implementing a tagging strategy.
20+
21+
</Note>
22+
23+
## Create spending limits and notifications
24+
25+
Most major cloud providers offer native tools to create budgets. These tools allow you to set budget thresholds, monitor spending, and receive alerts when spending approaches or exceeds defined limits.
26+
27+
You can use Terraform to define and manage cloud budgets across your organization. You can create Terraform modules to create budgets for different teams, projects, or environments. These modules can automatically apply appropriate budget thresholds, alerting mechanisms, and spending limits to new or existing cloud resources.
28+
29+
If you're tracking resources by tags, it is important to have a well-defined tagging strategy to ensure budgets are applied correctly. Terraform can help you enforce tagging policies and ensure that all resources are tagged consistently. Creating infrastructure manually can lead to incorrect or missed tags on resources, resulting in inaccurate budget tracking.
30+
31+
The following is an example of a Terraform configuration that creates an AWS EC2 budget. This budget tracks EC2 instance costs and sends an alert to test@example.com when the forecasted cost exceeds 100% of the budget. You can set similar budgets and alerts for other cloud providers, such as Azure and GCP.
32+
33+
```hcl
34+
resource "aws_budgets_budget" "ec2" {
35+
name = "budget-ec2-monthly"
36+
budget_type = "COST"
37+
limit_amount = "1200"
38+
limit_unit = "USD"
39+
time_period_end = "2087-06-15_00:00"
40+
time_period_start = "2017-07-01_00:00"
41+
time_unit = "MONTHLY"
42+
43+
cost_filter {
44+
name = "Service"
45+
values = [
46+
"Amazon Elastic Compute Cloud - Compute",
47+
]
48+
}
49+
50+
notification {
51+
comparison_operator = "GREATER_THAN"
52+
threshold = 100
53+
threshold_type = "PERCENTAGE"
54+
notification_type = "FORECASTED"
55+
subscriber_email_addresses = ["test@example.com"]
56+
}
57+
58+
tags = {
59+
Environment = "production"
60+
Team = "engineering"
61+
ManagedBy = "terraform"
62+
}
63+
}
64+
```
65+
66+
In the previous example
67+
- **limit_amount:** Defines the monthly spend limit.
68+
- **notification:** Defines the notification criteria and sends an email to test@example.com.
69+
- **tags:** The tag is applied to the budget resource, not the EC2 instance. The tags allow you to filter and organize budgets in the billing console.
70+
71+
For AWS environments, you can use the `aws_budgets_budget` resource to create budgets that track spending by service, linked account, tag, or other dimensions. You can specify the budget amount, time period, and notification thresholds.
72+
73+
For Azure environments, the `azurerm_consumption_budget_subscription` resource lets you create subscription-level budgets with similar notification capabilities. You can define multiple notification rules that trigger at different spending thresholds.
74+
75+
For Google Cloud Platform, the `google_billing_budget` resource operates at the billing account level, and you can filter by project, service, or label. GCP budgets support both actual and forecasted spending alerts.
76+
77+
HashiCorp resources:
78+
79+
- Learn how to [Tag cloud resources](/well-architected-framework/docs/docs/optimize-systems/lifecycle-management/tag-cloud-resources)
80+
- Terraform resource: [aws_budgets_budget](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/budgets_budget)
81+
- Terraform resource: [azurerm_consumption_budget_subscription](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/consumption_budget_subscription)
82+
- Terraform resource: [google_billing_budget](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/billing_budget)
83+
84+
External resources:
85+
86+
- AWS Budgets: [Getting started with AWS Budgets](https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-managing-costs.html)
87+
- Azure Cost Management and Billing: [Create and manage budgets](https://learn.microsoft.com/en-us/azure/cost-management-billing/costs/tutorial-acm-create-budgets)
88+
- Google Cloud Budgets and alerts: [Creating budgets](https://cloud.google.com/billing/docs/how-to/budgets)
89+
90+
## Detect spending anomalies
91+
92+
Anomaly detection identifies unusual patterns rather than absolute thresholds. For example, if your monthly EC2 spending suddenly doubles from $2,000 to $4,000 but remains under your $5,000 budget, a budget alert would not trigger. However, anomaly detection would flag this unusual increase for investigation. Anomaly detection helps you catch issues like misconfigured autoscaling, forgotten resources, or unauthorized usage before they significantly impact costs.
93+
94+
Most cloud providers offer machine learning-based anomaly detection that learns your normal usage patterns and alerts you when spending deviates from the baseline. You can configure anomaly detection using Terraform for AWS Cost Anomaly Detection and Azure Cost Management.
95+
96+
The following is an example Terraform code that sets up a cost anomaly detection with email alerts in AWS. This cost anomaly detection will detect the previous EC2 scenario.
97+
98+
```hcl
99+
resource "aws_ce_anomaly_monitor" "test" {
100+
name = "AWSServiceMonitor"
101+
monitor_type = "DIMENSIONAL"
102+
monitor_dimension = "SERVICE"
103+
}
104+
105+
resource "aws_ce_anomaly_subscription" "test" {
106+
name = "DAILYSUBSCRIPTION"
107+
frequency = "DAILY"
108+
109+
monitor_arn_list = [
110+
aws_ce_anomaly_monitor.test.arn
111+
]
112+
113+
subscriber {
114+
type = "EMAIL"
115+
address = "abc@example.com"
116+
}
117+
118+
threshold_expression {
119+
dimension {
120+
key = "ANOMALY_TOTAL_IMPACT_ABSOLUTE"
121+
match_options = ["GREATER_THAN_OR_EQUAL"]
122+
values = ["100"]
123+
}
124+
}
125+
}
126+
```
127+
128+
In the previous example:
129+
- **aws_ce_anomaly_monitor:** Tracks spending patterns across all AWS services such as EC2, S3, and Lambda.
130+
- **frequency = "DAILY":** Sends a daily summary of detected anomalies.
131+
- **threshold_expression:** Only alerts when the anomaly's financial impact exceeds $100.
132+
133+
HashiCorp resources:
134+
135+
- Terraform resource: [aws_ce_anomaly_subscription](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ce_anomaly_subscription)
136+
- Terraform resource: [azurerm_cost_anomaly_alert](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/cost_anomaly_alert)
137+
138+
External resources:
139+
140+
- AWS Cost Anomaly Detection: [What is AWS Cost Anomaly Detection?](https://docs.aws.amazon.com/cost-anomaly/latest/userguide/what-is-cost-anomaly.html)
141+
- Azure Cost Management anomalies: [Detect anomalies in your cost data](https://learn.microsoft.com/en-us/azure/cost-management-billing/costs/analyze-cost-data#detect-anomalies-in-your-cost-data)
142+
- Google Cloud's cost anomaly detection: [Using anomaly detection](https://cloud.google.com/billing/docs/how-to/using-anomaly-detection)
143+
144+
## Next steps
145+
146+
In this section of Manage cost, you learned about creating budgets and alerts to manage and control cloud spending, including creating spending limits with cloud provider budgets and detecting spending anomalies automatically. Create cloud budgets is part of the [Optimize systems](/well-architected-framework/optimize-systems).
147+
148+
To learn more about managing resources with Terraform, view the following resources:
149+
- [Create reusable infrastructure modules](/well-architected-framework/define-and-automate-processes/define/modules)
150+
- [Implement CI/CD](/well-architected-framework/define-and-automate-processes/automate/cicd)
151+
- [Reduce costs with Terraform Cloud ephemeral workspaces](https://www.youtube.com/watch?v=-woCmG8yGdA)
152+
- [Tag cloud resources](/well-architected-framework/docs/docs/optimize-systems/lifecycle-management/tag-cloud-resources)

0 commit comments

Comments
 (0)