Skip to content

Commit a20913f

Browse files
julianwoodjulianwood
authored
Merge pull request #2508 from gbiagini97/gbiagini-appsync-events-bedrock
New serverless pattern: appsync-events-bedrock-cdk
2 parents dca61cd + c7f4768 commit a20913f

35 files changed

+1391
-0
lines changed

‎appsync-events-bedrock-cdk/.gitignore‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
*.js
2+
!jest.config.js
3+
*.d.ts
4+
node_modules
5+
6+
# CDK asset staging directory
7+
.cdk.staging
8+
cdk-outputs.json
9+
cdk.out

‎appsync-events-bedrock-cdk/.npmignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
*.ts
2+
!*.d.ts
3+
4+
# CDK asset staging directory
5+
.cdk.staging
6+
cdk.out

‎appsync-events-bedrock-cdk/README.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Stream Amazon Bedrock completions via AWS AppSync Events API
2+
3+
This pattern shows how to stream Amazon Bedrock completions via AWS AppSync Events API.
4+
5+
Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/appsync-events-bedrock
6+
7+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
8+
9+
10+
## Requirements
11+
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
12+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
13+
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
14+
* [Node and NPM](https://nodejs.org/en/download/) installed
15+
* [AWS Cloud Development Kit](https://docs.aws.amazon.com/cdk/v2/guide/cli.html) (AWS CDK) installed
16+
* Make sure to enable the **Anthropic - Claude 3 Haiku** model on the [Bedrock console](https://console.aws.amazon.com/bedrock/home#/modelaccess).
17+
18+
19+
## How it works
20+
![diagram](diagram.png)
21+
1. The client connects to the WebSocket using the AWS Amplify client or a custom made library following the [instructions described in the documentation](https://docs.aws.amazon.com/appsync/latest/eventapi/event-api-websocket-protocol.html);
22+
2. The client chooses a namespace or a channel (or a segment) and subscribes to it. [Additional info the documentation](https://docs.aws.amazon.com/appsync/latest/eventapi/event-api-websocket-protocol.html);
23+
3. The client performs a request towards the Amazon API Gateway that exposes a `/chat` route passing the `userId` and `prompt` parameters. The `userId` will be used as a segment for the channel. The prompt will be used for GenAI completions on Bedrock;
24+
4. The triggered Lambda function receives payload;
25+
5. The Lambda function proceeds to invoke a Bedrock model using the streaming API;
26+
6. The Lambda function will receive the completion `chunks` as soon as they are available;
27+
7. For each completion chunk, the Lambda function will publish them through the [Publish HTTP API on AppSync Events](https://docs.aws.amazon.com/appsync/latest/eventapi/publish-http.html);
28+
8. The client is now able to retrieve the completion chunks from AppSync Events;
29+
9. The client can finally unsubscribe from the channel and close the Websocket connection.
30+
31+
### Permissions and Security
32+
In this demo the client is only able to subscribe to the AppSync Event WebSocket via the API_KEY. The same API_KEY will be used by the Lambda function to perform publish requests.
33+
34+
35+
## Deployment Instructions
36+
37+
1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
38+
```sh
39+
git clone https://github.com/aws-samples/serverless-patterns
40+
```
41+
2. Change directory to the pattern directory:
42+
```sh
43+
cd appsync-events-bedrock-cdk
44+
```
45+
3. Install the required dependencies:
46+
```sh
47+
npm install
48+
```
49+
4. Deploy the stack to your default AWS account and region:
50+
```sh
51+
npm run deploy
52+
```
53+
5. Export the environment variables for the demo web app:
54+
```sh
55+
npm run export-outputs
56+
```
57+
6. Install dependencies for the demo web app:
58+
```sh
59+
cd demo-app && npm install
60+
```
61+
7. Launch the demo web app
62+
```sh
63+
npm run dev
64+
```
65+
66+
## Testing
67+
Integration tests can be ran with jest from the pattern's root directory:
68+
```sh
69+
npm run test
70+
```
71+
72+
From the demo app it's possible to see all connection events and data events in a structured way.
73+
First of all connect to the WebSocket
74+
![step1](step1.png)
75+
76+
Then subscribe to the namespace. Please note: The default namespace is `BedrockChat`. By inserting `BedrockChat/*` you will be able to receive all the messages in the sub-segments.
77+
![step2](step2.png)
78+
79+
Finally, insert your username (the final segment will become `BedrockChat/username`) and your prompt.
80+
![step3](step3.png)
81+
82+
By clicking on the `realtime` entry on the networking tab and selecting `messages` you will see all the messages exchanged between the client demo app and AppSync Event API. For convienence I've setup two scrollable boxes that display the same data.
83+
84+
85+
## Cleanup
86+
Close the demo web app by hitting `^C` on MacOs.
87+
88+
Return to the pattern's root folder and delete the stack:
89+
```sh
90+
cdk destroy --all
91+
```
92+
----
93+
Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
94+
SPDX-License-Identifier: MIT-0

‎appsync-events-bedrock-cdk/appsync-events-bedrock-cdk.json‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
{
2+
"title": "Stream Amazon Bedrock completions via AWS AppSync Events API",
3+
"description": "Leverage AppSync Events API to stream completions from Amazon Bedrock.",
4+
"language": "TypeScript",
5+
"level": "200",
6+
"framework": "CDK",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"In this pattern the end user is able to provide a prompt that will be used to invoke an Amazon Bedrock model from via a Lambda middleware.",
11+
"The client will receive the model's completions through WebSocket provided by AppSync Events."
12+
]
13+
},
14+
"gitHub": {
15+
"template": {
16+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/appsync-events-bedrock-cdk",
17+
"templateURL": "serverless-patterns/appsync-events-bedrock-cdk",
18+
"projectFolder": "appsync-events-bedrock-cdk",
19+
"templateFile": "lib/appsync-events-bedrock-cdk-stack.ts"
20+
}
21+
},
22+
"resources": {
23+
"bullets": [
24+
{
25+
"text": "AppSync Events API",
26+
"link": "https://docs.aws.amazon.com/appsync/latest/eventapi/event-api-welcome.html"
27+
},
28+
{
29+
"text": "Amazon Bedrock - Generative AI",
30+
"link": "https://aws.amazon.com/bedrock/"
31+
},
32+
{
33+
"text": "Bedrock's Response Stream API",
34+
"link": "https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html"
35+
}
36+
]
37+
},
38+
"deploy": {
39+
"text": [
40+
"npm run deploy"
41+
]
42+
},
43+
"testing": {
44+
"text": [
45+
"npm run test"
46+
]
47+
},
48+
"cleanup": {
49+
"text": [
50+
"cdk destroy"
51+
]
52+
},
53+
"authors": [
54+
{
55+
"name": "Gabriele Biagini",
56+
"bio": "Solutions Architect @ AWS",
57+
"linkedin": "gbiagini97"
58+
}
59+
],
60+
"patternArch": {
61+
"icon1": {
62+
"x": 20,
63+
"y": 50,
64+
"service": "apigw",
65+
"label": "Amazon API Gateway"
66+
},
67+
"icon2": {
68+
"x": 50,
69+
"y": 50,
70+
"service": "lambda",
71+
"label": "AWS Lambda"
72+
},
73+
"icon3": {
74+
"x": 80,
75+
"y": 50,
76+
"service": "bedrock",
77+
"label": "Amazon Bedrock"
78+
},
79+
"line1": {
80+
"from": "icon1",
81+
"to": "icon2",
82+
"label": "WebSocket"
83+
},
84+
"line2": {
85+
"from": "icon2",
86+
"to": "icon3",
87+
"label": ""
88+
}
89+
}
90+
}

‎appsync-events-bedrock-cdk/bin/appsync-events-bedrock-cdk.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
#!/usr/bin/env node
2+
import 'source-map-support/register';
3+
import * as cdk from 'aws-cdk-lib';
4+
import { AppsyncEventsBedrockCdkStack } from '../lib/appsync-events-bedrock-cdk-stack';
5+
6+
const app = new cdk.App();
7+
new AppsyncEventsBedrockCdkStack(app, 'AppsyncEventsBedrockCdkStack', {
8+
/* If you don't specify 'env', this stack will be environment-agnostic.
9+
* Account/Region-dependent features and context lookups will not work,
10+
* but a single synthesized template can be deployed anywhere. */
11+
12+
/* Uncomment the next line to specialize this stack for the AWS Account
13+
* and Region that are implied by the current CLI configuration. */
14+
// env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
15+
16+
/* Uncomment the next line if you know exactly what Account and Region you
17+
* want to deploy the stack to. */
18+
// env: { account: '123456789012', region: 'us-east-1' },
19+
20+
/* For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html */
21+
});

‎appsync-events-bedrock-cdk/cdk.json‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
{
2+
"app": "npx tsx bin/appsync-events-bedrock-cdk.ts",
3+
"watch": {
4+
"include": [
5+
"**"
6+
],
7+
"exclude": [
8+
"README.md",
9+
"cdk*.json",
10+
"**/*.d.ts",
11+
"**/*.js",
12+
"tsconfig.json",
13+
"package*.json",
14+
"yarn.lock",
15+
"node_modules",
16+
"test"
17+
]
18+
},
19+
"context": {
20+
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
21+
"@aws-cdk/core:checkSecretUsage": true,
22+
"@aws-cdk/core:target-partitions": [
23+
"aws",
24+
"aws-cn"
25+
],
26+
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
27+
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
28+
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
29+
"@aws-cdk/aws-iam:minimizePolicies": true,
30+
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
31+
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
32+
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
33+
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
34+
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
35+
"@aws-cdk/core:enablePartitionLiterals": true,
36+
"@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
37+
"@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
38+
"@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
39+
"@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
40+
"@aws-cdk/aws-route53-patters:useCertificate": true,
41+
"@aws-cdk/customresources:installLatestAwsSdkDefault": false,
42+
"@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
43+
"@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
44+
"@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
45+
"@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
46+
"@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
47+
"@aws-cdk/aws-redshift:columnId": true,
48+
"@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
49+
"@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
50+
"@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
51+
"@aws-cdk/aws-kms:aliasNameRef": true,
52+
"@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
53+
"@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
54+
"@aws-cdk/aws-efs:denyAnonymousAccess": true,
55+
"@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
56+
"@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
57+
"@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
58+
"@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
59+
"@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
60+
"@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true,
61+
"@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true,
62+
"@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true,
63+
"@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true,
64+
"@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true,
65+
"@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true,
66+
"@aws-cdk/aws-eks:nodegroupNameAttribute": true,
67+
"@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
68+
"@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
69+
"@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false,
70+
"@aws-cdk/aws-s3:keepNotificationInImportedBucket": false,
71+
"@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true,
72+
"@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true,
73+
"@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true,
74+
"@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true,
75+
"@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true,
76+
"@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true,
77+
"@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true,
78+
"@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true
79+
}
80+
}

‎appsync-events-bedrock-cdk/demo-app/.gitignore‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Logs
2+
logs
3+
*.log
4+
npm-debug.log*
5+
yarn-debug.log*
6+
yarn-error.log*
7+
pnpm-debug.log*
8+
lerna-debug.log*
9+
10+
node_modules
11+
dist
12+
dist-ssr
13+
*.local
14+
15+
# Editor directories and files
16+
.vscode/*
17+
!.vscode/extensions.json
18+
.idea
19+
.DS_Store
20+
*.suo
21+
*.ntvs*
22+
*.njsproj
23+
*.sln
24+
*.sw?

‎appsync-events-bedrock-cdk/demo-app/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Follow instructions in the pattern's root folder [README](../README.md).

‎appsync-events-bedrock-cdk/demo-app/index.html‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8" />
5+
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
6+
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
7+
<title>Vite + React + TS</title>
8+
</head>
9+
<body>
10+
<div id="root"></div>
11+
<script type="module" src="/src/main.tsx"></script>
12+
</body>
13+
</html>

‎appsync-events-bedrock-cdk/demo-app/package.json‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"name": "demo-app",
3+
"private": true,
4+
"version": "0.0.0",
5+
"type": "module",
6+
"scripts": {
7+
"dev": "vite",
8+
"build": "tsc -b && vite build",
9+
"lint": "eslint .",
10+
"preview": "vite preview"
11+
},
12+
"dependencies": {
13+
"@types/uuid": "^10.0.0",
14+
"react": "^18.3.1",
15+
"react-dom": "^18.3.1",
16+
"uuid": "^11.0.2"
17+
},
18+
"devDependencies": {
19+
"@eslint/js": "^9.13.0",
20+
"@types/react": "^18.3.12",
21+
"@types/react-dom": "^18.3.1",
22+
"@vitejs/plugin-react-swc": "^3.5.0",
23+
"esbuild": "^0.24.0",
24+
"eslint": "^9.13.0",
25+
"eslint-plugin-react-hooks": "^5.0.0",
26+
"eslint-plugin-react-refresh": "^0.4.14",
27+
"globals": "^15.11.0",
28+
"ts-node": "^10.9.2",
29+
"typescript": "~5.6.2",
30+
"typescript-eslint": "^8.11.0",
31+
"vite": "^5.4.10",
32+
"ws": "^8.18.0"
33+
}
34+
}

0 commit comments

Comments
 (0)