Skip to content

Commit 0644bc1

Browse files
erezzarumerezzarum
authored
docs: deploying on Amazon EKS (#66)
Provides guidance on how to deploy Strands Agents using EKS with an example weather application.
1 parent 10ab59a commit 0644bc1

File tree

19 files changed

+1014
-1
lines changed

19 files changed

+1014
-1
lines changed

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@
55
site/
66
*.bak
77
__pycache__
8-
.venv
8+
.venv
9+
.idea

‎docs/examples/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@ Available CDK examples:
5757
- [Deploy to Fargate](cdk/deploy_to_fargate/README.md) - Guide for deploying agents to AWS Fargate
5858
- [Deploy to Lambda](cdk/deploy_to_lambda/README.md) - Guide for deploying agents to AWS Lambda
5959

60+
### Amazon EKS Example
61+
62+
The `/examples/deploy_to_eks` directory contains examples for using Amazon EKS with agents.
63+
The [Deploy to Amazon EKS](deploy_to_eks/README.md) includes its own documentation with instruction for setup and deployment.
64+
6065
## Example Structure
6166

6267
Each example typically follows this structure:

‎docs/examples/deploy_to_eks/README.md‎

Lines changed: 262 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,262 @@
1+
# Amazon EKS Deployment Example
2+
3+
## Introduction
4+
5+
This is an example that demonstrates how to deploy a Python application to Amazon EKS.
6+
The example deploys a weather forecaster application that runs as a containerized service in Amazon EKS with an Application Load Balancer. The application is built with FastAPI and provides two weather endpoints:
7+
8+
1. `/weather` - A standard endpoint that returns weather information based on the provided prompt
9+
2. `/weather-streaming` - A streaming endpoint that delivers weather information in real-time as it's being generated
10+
11+
## Prerequisites
12+
13+
- [AWS CLI](https://aws.amazon.com/cli/) installed and configured
14+
- [eksctl](https://eksctl.io/installation/) (v0.208.x or later) installed
15+
- [Helm](https://helm.sh/) (v3 or later) installed
16+
- [kubectl](https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html) installed
17+
- Either:
18+
- [Podman](https://podman.io/) installed and running
19+
- (or) [Docker](https://www.docker.com/) installed and running
20+
- Amazon Bedrock Anthropic Claude 3.7 model enabled in your AWS environment
21+
You'll need to enable model access in the Amazon Bedrock console following the [AWS documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access-modify.html)
22+
23+
## Project Structure
24+
25+
- `chart/` - Contains the Helm chart
26+
- `values.yaml` - Helm chart default values
27+
- `docker/` - Contains the Dockerfile and application code for the container:
28+
- `Dockerfile` - Docker image definition
29+
- `app/` - Application code
30+
- `requirements.txt` - Python dependencies for the container & local development
31+
32+
## Create EKS Auto Mode cluster
33+
34+
Set environment variables
35+
```bash
36+
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)
37+
export AWS_REGION=us-east-1
38+
export CLUSTER_NAME=eks-strands-agents-demo
39+
```
40+
41+
Create EKS Auto Mode cluster
42+
```bash
43+
eksctl create cluster --name $CLUSTER_NAME --enable-auto-mode
44+
```
45+
Configure kubeconfig context
46+
```bash
47+
aws eks update-kubeconfig --name $CLUSTER_NAME
48+
```
49+
50+
## Building and Pushing Docker Image to ECR
51+
52+
Follow these steps to build the Docker image and push it to Amazon ECR:
53+
54+
1. Authenticate to Amazon ECR:
55+
```bash
56+
aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com
57+
```
58+
59+
2. Create the ECR repository if it doesn't exist:
60+
```bash
61+
aws ecr create-repository --repository-name strands-agents-weather --region ${AWS_REGION}
62+
```
63+
64+
3. Build the Docker image:
65+
```bash
66+
docker build --platform linux/amd64 -t strands-agents-weather:latest docker/
67+
```
68+
69+
4. Tag the image for ECR:
70+
```bash
71+
docker tag strands-agents-weather:latest ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather:latest
72+
```
73+
74+
5. Push the image to ECR:
75+
```bash
76+
docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather:latest
77+
```
78+
79+
## Configure EKS Pod Identity to access Amazon Bedrock
80+
81+
Create an IAM policy to allow InvokeModel & InvokeModelWithResponseStream to all Amazon Bedrock models
82+
```bash
83+
cat > bedrock-policy.json << EOF
84+
{
85+
"Version": "2012-10-17",
86+
"Statement": [
87+
{
88+
"Effect": "Allow",
89+
"Action": [
90+
"bedrock:InvokeModel",
91+
"bedrock:InvokeModelWithResponseStream"
92+
],
93+
"Resource": "*"
94+
}
95+
]
96+
}
97+
EOF
98+
99+
aws iam create-policy \
100+
--policy-name strands-agents-weather-bedrock-policy \
101+
--policy-document file://bedrock-policy.json
102+
rm -f bedrock-policy.json
103+
```
104+
105+
Create an EKS Pod Identity association
106+
```bash
107+
eksctl create podidentityassociation --cluster $CLUSTER_NAME \
108+
--namespace default \
109+
--service-account-name strands-agents-weather \
110+
--permission-policy-arns arn:aws:iam::$AWS_ACCOUNT_ID:policy/strands-agents-weather-bedrock-policy \
111+
--role-name eks-strands-agents-weather
112+
```
113+
114+
## Deploy strands-agents-weather application
115+
116+
Deploy the helm chart with the image from ECR
117+
```bash
118+
helm install strands-agents-weather ./chart \
119+
--set image.repository=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather --set image.tag=latest
120+
```
121+
122+
Wait for Deployment to be available (Pods Running)
123+
```bash
124+
kubectl wait --for=condition=available deployments strands-agents-weather --all
125+
```
126+
127+
## Test the Agent
128+
129+
Using kubernetes port-forward
130+
```
131+
kubectl --namespace default port-forward service/strands-agents-weather 8080:80 &
132+
```
133+
134+
Call the weather service
135+
```
136+
curl -X POST \
137+
http://localhost:8080/weather \
138+
-H 'Content-Type: application/json' \
139+
-d '{"prompt": "What is the weather in Seattle?"}'
140+
```
141+
142+
Call the weather streaming endpoint
143+
```
144+
curl -X POST \
145+
http://localhost:8080/weather-streaming \
146+
-H 'Content-Type: application/json' \
147+
-d '{"prompt": "What is the weather in New York in Celsius?"}'
148+
```
149+
150+
## Expose Agent through Application Load Balancer
151+
152+
[Create an IngressClass to configure an Application Load Balancer](https://docs.aws.amazon.com/eks/latest/userguide/auto-configure-alb.html)
153+
```bash
154+
cat <<EOF | kubectl apply -f -
155+
apiVersion: eks.amazonaws.com/v1
156+
kind: IngressClassParams
157+
metadata:
158+
name: alb
159+
spec:
160+
scheme: internet-facing
161+
EOF
162+
```
163+
164+
```bash
165+
cat <<EOF | kubectl apply -f -
166+
apiVersion: networking.k8s.io/v1
167+
kind: IngressClass
168+
metadata:
169+
name: alb
170+
annotations:
171+
ingressclass.kubernetes.io/is-default-class: "true"
172+
spec:
173+
controller: eks.amazonaws.com/alb
174+
parameters:
175+
apiGroup: eks.amazonaws.com
176+
kind: IngressClassParams
177+
name: alb
178+
EOF
179+
```
180+
181+
Update helm deployment to create Ingress using the IngressClass created
182+
```bash
183+
helm upgrade strands-agents-weather ./chart \
184+
--set image.repository=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather --set image.tag=latest \
185+
--set ingress.enabled=true \
186+
--set ingress.className=alb
187+
```
188+
189+
Get the ALB URL
190+
```bash
191+
export ALB_URL=$(kubectl get ingress strands-agents-weather -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
192+
echo "The shared ALB is available at: http://$ALB_URL"
193+
```
194+
195+
Wait for ALB to be active
196+
```bash
197+
aws elbv2 wait load-balancer-available --load-balancer-arns $(aws elbv2 describe-load-balancers --query 'LoadBalancers[?DNSName==`'"$ALB_URL"'`].LoadBalancerArn' --output text)
198+
```
199+
200+
Call the weather service Application Load Balancer endpoint
201+
```bash
202+
curl -X POST \
203+
http://$ALB_URL/weather \
204+
-H 'Content-Type: application/json' \
205+
-d '{"prompt": "What is the weather in Portland?"}'
206+
```
207+
208+
## Configure High Availability and Resiliency
209+
210+
- Increase replicas to 3
211+
- [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/): Spread workload across multi-az
212+
- [Pod Disruption Budgets](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/#pod-disruption-budgets): Tolerate minAvailable of 1
213+
214+
```bash
215+
helm upgrade strands-agents-weather ./chart -f - <<EOF
216+
image:
217+
repository: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather
218+
tag: latest
219+
220+
ingress:
221+
enabled: true
222+
className: alb
223+
224+
replicaCount: 3
225+
226+
topologySpreadConstraints:
227+
- maxSkew: 1
228+
minDomains: 3
229+
topologyKey: topology.kubernetes.io/zone
230+
whenUnsatisfiable: DoNotSchedule
231+
labelSelector:
232+
matchLabels:
233+
app.kubernetes.io/name: strands-agents-weather
234+
- maxSkew: 1
235+
topologyKey: kubernetes.io/hostname
236+
whenUnsatisfiable: ScheduleAnyway
237+
labelSelector:
238+
matchLabels:
239+
app.kubernetes.io/instance: strands-agents-weather
240+
241+
podDisruptionBudget:
242+
enabled: true
243+
minAvailable: 1
244+
EOF
245+
```
246+
247+
## Cleanup
248+
249+
Uninstall helm chart
250+
```bash
251+
helm uninstall strands-agents-weather
252+
```
253+
254+
Delete EKS Auto Mode cluster
255+
```bash
256+
eksctl delete cluster --name $CLUSTER_NAME --wait
257+
```
258+
259+
Delete IAM policy
260+
```bash
261+
aws iam delete-policy --policy-arn arn:aws:iam::$AWS_ACCOUNT_ID:policy/strands-agents-weather-bedrock-policy
262+
```

‎docs/examples/deploy_to_eks/chart/.helmignore‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*.orig
18+
*~
19+
# Various IDEs
20+
.project
21+
.idea/
22+
*.tmproj
23+
.vscode/

‎docs/examples/deploy_to_eks/chart/Chart.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
apiVersion: v2
2+
name: strands-agents-weather
3+
description: A Helm chart for deploying strands-agents-weather application
4+
type: application
5+
version: 0.1.0
6+
appVersion: "1.0.0"

‎docs/examples/deploy_to_eks/chart/templates/NOTES.txt‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
1. Get the application URL by running these commands:
2+
{{- if .Values.ingress.enabled }}
3+
{{- if .Values.ingress.hosts }}
4+
{{- range $host := .Values.ingress.hosts }}
5+
http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host }}{{ $.Values.ingress.path }}
6+
{{- end }}
7+
{{- else }}
8+
export ALB_URL=$(kubectl get ingress strands-agents-weather -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
9+
echo "The shared ALB is available at: http://$ALB_URL"
10+
11+
NOTE: It may take a few minutes for the LoadBalancer to be active, you can wait for it by running
12+
aws elbv2 wait load-balancer-available --load-balancer-arns $(aws elbv2 describe-load-balancers --query 'LoadBalancers[?DNSName==`'"$ALB_URL"'`].LoadBalancerArn' --output text)
13+
{{- end }}
14+
{{- else if contains "NodePort" .Values.service.type }}
15+
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "strands-agents-weather.fullname" . }})
16+
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
17+
echo http://$NODE_IP:$NODE_PORT
18+
{{- else if contains "LoadBalancer" .Values.service.type }}
19+
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
20+
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "strands-agents-weather.fullname" . }}'
21+
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "strands-agents-weather.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
22+
echo http://$SERVICE_IP:{{ .Values.service.port }}
23+
{{- else if contains "ClusterIP" .Values.service.type }}
24+
export SVC_NAME=$(kubectl get svc --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "strands-agents-weather.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
25+
export SVC_PORT=$(kubectl get svc --namespace {{ .Release.Namespace }} $SVC_NAME -o jsonpath="{.spec.ports[0].port}")
26+
echo "Visit http://127.0.0.1:8080 to use your application"
27+
kubectl --namespace {{ .Release.Namespace }} port-forward service/$SVC_NAME 8080:$SVC_PORT
28+
{{- end }}

0 commit comments

Comments
 (0)