Skip to content

Commit 12c8aad

Browse files
cabutlermitcabutlermit
authored
Merge pull request #69 from MITLibraries/dev
Dev-to-Stage: Update ECR Outputs for marimo-launcher
2 parents 94143cd + cba0e96 commit 12c8aad

13 files changed

+312
-25
lines changed

‎.pre-commit-config.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ repos:
1212
- id: terraform-docs-go
1313
args: ["markdown", "table", "--config", "./.terraform-docs.yaml", "--recursive", "--output-file", "README.md", "./"]
1414
- repo: https://github.com/bridgecrewio/checkov.git
15-
rev: '3.2.461'
15+
rev: '3.2.471'
1616
hooks:
1717
- id: checkov
1818
verbose: false

‎README.md‎

Lines changed: 16 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ For each new ECR repository (or linked collection of ECR repositories), create a
6060
* a stage_build caller workflow
6161
* a prod_promote caller workflow
6262

63-
The [ppod_ecr.tf](./ppod_ecr.tf) is a good example of a single ECR repository for a Lambda function build around a containerized app. The [timdex_ecrs.tf](./timdex_ecrs.tf) is a good example of a collection of ECR repositories all linked to one project (and there are both Fargate-linked ECRs and Lambda-linked ECRs in that file).
63+
The [ppod_ecr.tf](./ppod_ecr.tf) is a good example of a single ECR repository for a Lambda function built around a containerized app. The [timdex_ecrs.tf](./timdex_ecrs.tf) is a good example of a collection of ECR repositories all linked to one project (and there are both Fargate-linked ECRs and Lambda-linked ECRs in that file).
6464

6565
**Note**: For Lambda function ECRs, it is imperative that the Infra engineer coordinates with the software engineer to determine the name of the Lambda function as part of the creation of the ECR by this repository.
6666

@@ -96,38 +96,40 @@ This is a core infrastructure repository that defines infrastructure related to
9696
* [Alma Hook](https://github.com/MITLibraries/mitlib-tf-workloads-almahook)
9797
* [Alma Webhook Lambdas](https://github.com/MITLibraries/alma-webhook-lambdas)
9898
* [Alma Patron Load](https://github.com/MITLibraries/mitlib-tf-workloads-patronload)
99-
* [Alma Patron Load Application Container](https://github.com/MITLibraries/alma-patronload)
99+
* [Alma Patron Load Application](https://github.com/MITLibraries/alma-patronload)
100100
* [Archival Packaging Tool](https://github.com/MITLibraries/mitlib-tf-workloads-apt)
101-
* [Archival Packaging Tool Application Container](https://github.com/MITLibraries/archival-packaging-tool)
101+
* [Archival Packaging Tool Application](https://github.com/MITLibraries/archival-packaging-tool)
102102
* [ASATI](https://github.com/MITLibraries/mitlib-tf-workloads-asati)
103-
* [ASATI Application Container](https://github.com/MITLibraries/asati)
103+
* [ASATI Application](https://github.com/MITLibraries/asati)
104104
* [Carbon](https://github.com/MITLibraries/mitlib-tf-workloads-carbon)
105105
* [CDPS](https://github.com/MITLibraries/mitlib-tf-workloads-cdps-storage)
106106
* [S3 BagIt Validator](https://github.com/MITLibraries/s3-bagit-validator)
107107
* [CDPS CURT](https://github.com/MITLibraries/cdps-curt)
108-
* [DSC](https://github.com/MITLibraries/mitlib-tf-workloads-dsc)
109-
* [DSC Application Container](https://github.com/MITLibraries/dspace-submission-composer)
110-
* [DSS](https://github.com/MITLibraries/mitlib-tf-workloads-dss)
111-
* [DSpace Submission Service Application Container](https://github.com/MITLibraries/dspace-submission-service)
112-
* [ETD](https://github.com/MITLibraries/mitlib-tf-workloads-etd)
108+
* [DSO Infrastructure](https://github.com/MITLibraries/mitlib-tf-workloads-dso)
109+
* [DSpace Submission Composer Application](https://github.com/MITLibraries/dspace-submission-composer)
110+
* [DSpace Submission Service Application](https://github.com/MITLibraries/dspace-submission-service)
111+
* **DEPRECATED**: [DSC](https://github.com/MITLibraries/mitlib-tf-workloads-dsc)
112+
* **DEPRECATED**: [DSS](https://github.com/MITLibraries/mitlib-tf-workloads-dss)
113+
* [ETD Infrastructure](https://github.com/MITLibraries/mitlib-tf-workloads-etd)
113114
* [HRQB](https://github.com/MITLibraries/mitlib-tf-workloads-hrqb-loader)
114115
* [HRQB Client](https://github.com/MITLibraries/hrqb-client)
115116
* [marimo notebooks](https://github.com/MITLibraries/mitlib-tf-workloads-notebooks)
116117
* [marimo-launcher](https://github.com/MITLibraries/marimo-launcher)
117118
* [marimo-helloworld](https://github.com/MITLibraries/marimo-helloworld)
118119
* [Matomo](https://github.com/MITLibraries/mitlib-tf-workloads-matomo)
119-
* [Matomo Application Container](https://github.com/MITLibraries/docker-matomo)
120+
* [Matomo Application](https://github.com/MITLibraries/docker-matomo)
120121
* [PPOD](https://github.com/MITLibraries/mitlib-tf-workloads-ppod)
121-
* [PPOD Application Container](https://github.com/MITLibraries/ppod)
122+
* [PPOD Application](https://github.com/MITLibraries/ppod)
122123
* [TACOS](https://github.com/MITLibraries/mitlib-tf-workloads-tacos)
123124
* [tacos-detectors-lambdas](https://github.com/MITLibraries/tacos-detectors-lambdas)
124125
* [TIMDEX](https://github.com/MITLibraries/mitlib-tf-workloads-timdex-infrastructure)
125-
* [TIMDEX Application Container](https://github.com/MITLibraries/timdex)
126+
* [TIMDEX Application](https://github.com/MITLibraries/timdex)
126127
* [TIMDEX Dataset API](https://github.com/MITLibraries/timdex-dataset-api)
127128
* [TIMDEX Index Manager](https://github.com/MITLibraries/timdex-index-manager)
128129
* [TIMDEX Pipeline Lambdas](https://github.com/MITLibraries/timdex-pipeline-lambdas)
129130
* [TIMDEX UI](https://github.com/MITLibraries/timdex-ui)
130131
* [TIMDEX Simulator](https://github.com/MITLibraries/timdex-simulator)
132+
* [TIMDEX Transmogrifier](https://github.com/MITLibraries/transmogrifier)
131133
* [WCD2Reshare](https://github.com/MITLibraries/mitlib-tf-workloads-wcd2reshare)
132134
* [WCD2Reshare Application Container](https://github.com/MITLibraries/wcd2reshare)
133135
* **DEPRECATED**: [Wiley](https://github.com/MITLibraries/mitlib-tf-workloads-wiley)
@@ -137,7 +139,7 @@ This is a core infrastructure repository that defines infrastructure related to
137139

138140
* Owner: See [CODEOWNERS](./.github/CODEOWNERS)
139141
* Team: See [CODEOWNERS](./.github/CODEOWNERS)
140-
* Last Maintenance: 2025-08
142+
* Last Maintenance: 2025-09
141143

142144
## TF markdown is automatically inserted at the bottom of this file, nothing should be written beyond this point
143145

@@ -146,7 +148,7 @@ This is a core infrastructure repository that defines infrastructure related to
146148

147149
| Name | Version |
148150
|------|---------|
149-
| terraform | ~> 1.11 |
151+
| terraform | ~> 1.13 |
150152
| aws | ~> 5.0 |
151153

152154
## Providers

‎ect_workflow_text_ecr.tf‎ renamed to ‎ecr_workflow_test_ecr.tf‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ module "ecr_workflowtest" {
2222
## For workflowtest application repo and ECR repository
2323
# Outputs in dev
2424
output "workflowtest_dev_build_workflow" {
25-
value = var.environment == "prod" || var.environment == "stage" ? null : templatefile("${path.module}/files/dev-build.tpl", {
25+
value = var.environment == "prod" || var.environment == "stage" ? null : templatefile("${path.module}/files/dev-build-cpu-arch.tpl", {
2626
region = var.aws_region
2727
role = module.ecr_workflowtest.gha_role
2828
ecr = module.ecr_workflowtest.repository_name
@@ -32,7 +32,7 @@ output "workflowtest_dev_build_workflow" {
3232
description = "Full contents of the dev-build.yml for the ecr-workflow-test repo"
3333
}
3434
output "workflowtest_makefile" {
35-
value = var.environment == "prod" || var.environment == "stage" ? null : templatefile("${path.module}/files/makefile.tpl", {
35+
value = var.environment == "prod" || var.environment == "stage" ? null : templatefile("${path.module}/files/makefile-cpu-arch.tpl", {
3636
ecr_name = module.ecr_workflowtest.repository_name
3737
ecr_url = module.ecr_workflowtest.repository_url
3838
function = ""
@@ -43,7 +43,7 @@ output "workflowtest_makefile" {
4343

4444
# Outputs in stage
4545
output "workflowtest_stage_build_workflow" {
46-
value = var.environment == "prod" || var.environment == "dev" ? null : templatefile("${path.module}/files/stage-build.tpl", {
46+
value = var.environment == "prod" || var.environment == "dev" ? null : templatefile("${path.module}/files/stage-build-cpu-arch.tpl", {
4747
region = var.aws_region
4848
role = module.ecr_workflowtest.gha_role
4949
ecr = module.ecr_workflowtest.repository_name
@@ -55,7 +55,7 @@ output "workflowtest_stage_build_workflow" {
5555

5656
# Outputs after promotion to prod
5757
output "workflowtest_prod_promote_workflow" {
58-
value = var.environment == "stage" || var.environment == "dev" ? null : templatefile("${path.module}/files/prod-promote.tpl", {
58+
value = var.environment == "stage" || var.environment == "dev" ? null : templatefile("${path.module}/files/prod-promote-cpu-arch.tpl", {
5959
region = var.aws_region
6060
role_stage = "${module.ecr_workflowtest.repo_name}-gha-stage"
6161
role_prod = "${module.ecr_workflowtest.repo_name}-gha-prod"

‎files/dev-build-cpu-arch-extra-region.tpl‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
### This is the Terraform-generated extra workflow job for the ###
2+
### ${ecr} app repository. ###
3+
### This should be added to jobs section of the dev-build.yml. If this is ###
4+
### a Lambda function, uncomment the FUNCTION: line ###
5+
6+
deploy-${region}:
7+
needs: prep
8+
name: Dev Deploy ${region}
9+
uses: mitlibraries/.github/.github/workflows/ecr-multi-arch-deploy-dev.yml@main
10+
secrets: inherit
11+
with:
12+
AWS_REGION: "${region}"
13+
GHA_ROLE: "${role}"
14+
ECR: "${ecr}"
15+
CPU_ARCH: $${{ needs.prep.outputs.cpuarch }}
16+
# FUNCTION: "${function}"
17+
# PREBUILD:

‎files/dev-build-cpu-arch.tpl‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
### This is the Terraform-generated dev-build.yml workflow for the ###
2+
### ${ecr} app repository. ###
3+
### If this is a Lambda repo, uncomment the FUNCTION line at the end of ###
4+
### the document. If the container requires any additional pre-build ###
5+
### commands, uncomment and edit the PREBUILD line at the end of the ###
6+
### document. ###
7+
8+
name: Dev Container Build and Deploy
9+
on:
10+
workflow_dispatch:
11+
pull_request:
12+
branches:
13+
- main
14+
paths-ignore:
15+
- '.github/**'
16+
17+
permissions:
18+
id-token: write
19+
contents: read
20+
21+
jobs:
22+
prep:
23+
name: Prep for Build
24+
runs-on: ubuntu-latest
25+
outputs:
26+
cpuarch: $${{ steps.setarch.outputs.cpuarch }}
27+
steps:
28+
- name: Checkout
29+
uses: actions/checkout@v5
30+
31+
- name: Set CPU Architecture
32+
id: setarch
33+
run: |
34+
echo "### :abacus: Architecture Selection" >> $GITHUB_STEP_SUMMARY
35+
if [[ -f .aws-architecture ]]; then
36+
ARCH=$(cat .aws-architecture)
37+
echo "\`$ARCH\` was read from \`.aws-architecture\` and passed to the deploy job." >> $GITHUB_STEP_SUMMARY
38+
else
39+
ARCH="linux/amd64"
40+
echo "No \`.aws-architecture\` file, so default \`$ARCH\` was passed to the deploy job." >> $GITHUB_STEP_SUMMARY
41+
fi
42+
if [[ "$ARCH" != "linux/arm64" && "$ARCH" != "linux/amd64" ]]; then
43+
echo "$ARCH is INVALID architecture!"
44+
echo "$ARCH is INVALID architecture!" >> $GITHUB_STEP_SUMMARY
45+
exit 1
46+
fi
47+
echo "cpuarch=$ARCH" >> $GITHUB_OUTPUT
48+
49+
deploy:
50+
needs: prep
51+
name: Dev Deploy
52+
uses: mitlibraries/.github/.github/workflows/ecr-multi-arch-deploy-dev.yml@main
53+
secrets: inherit
54+
with:
55+
AWS_REGION: "${region}"
56+
GHA_ROLE: "${role}"
57+
ECR: "${ecr}"
58+
CPU_ARCH: $${{ needs.prep.outputs.cpuarch }}
59+
# FUNCTION: "${function}"
60+
# PREBUILD:

‎files/makefile-cpu-arch.tpl‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
### This is the Terraform-generated header for ${ecr_name}. If ###
2+
### this is a Lambda repo, uncomment the FUNCTION line below ###
3+
### and review the other commented lines in the document. ###
4+
ECR_NAME_DEV := ${ecr_name}
5+
ECR_URL_DEV := ${ecr_url}
6+
CPU_ARCH ?= $(shell cat .aws-architecture 2>/dev/null || echo "linux/amd64")
7+
# FUNCTION_DEV := ${function}
8+
### End of Terraform-generated header ###
9+
10+
11+
### Terraform-generated Developer Deploy Commands for Dev environment ###
12+
check-arch:
13+
@ARCH_FILE=".aws-architecture"; \
14+
if [[ "$(CPU_ARCH)" != "linux/amd64" && "$(CPU_ARCH)" != "linux/arm64" ]]; then \
15+
echo "Invalid CPU_ARCH: $(CPU_ARCH)"; exit 1; \
16+
fi; \
17+
if [[ -f $$ARCH_FILE ]]; then \
18+
echo "latest-$(shell echo $(CPU_ARCH) | cut -d'/' -f2)" > .arch_tag; \
19+
else \
20+
echo "latest" > .arch_tag; \
21+
fi
22+
23+
dist-dev: check-arch ## Build docker container (intended for developer-based manual build)
24+
@ARCH_TAG=$$(cat .arch_tag); \
25+
docker buildx inspect $(ECR_NAME_DEV) >/dev/null 2>&1 || docker buildx create --name $(ECR_NAME_DEV) --use; \
26+
docker buildx use $(ECR_NAME_DEV); \
27+
docker buildx build --platform $(CPU_ARCH) \
28+
--load \
29+
--tag $(ECR_URL_DEV):make-$$ARCH_TAG \
30+
--tag $(ECR_URL_DEV):make-$(shell git describe --always) \
31+
--tag $(ECR_NAME_DEV):$$ARCH_TAG \
32+
.
33+
34+
publish-dev: dist-dev ## Build, tag and push (intended for developer-based manual publish)
35+
@ARCH_TAG=$$(cat .arch_tag); \
36+
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin $(ECR_URL_DEV); \
37+
docker push $(ECR_URL_DEV):make-$$ARCH_TAG; \
38+
docker push $(ECR_URL_DEV):make-$(shell git describe --always); \
39+
docker push $(ECR_URL_DEV):make-$(shell echo $(CPU_ARCH) | cut -d'/' -f2)
40+
41+
### If this is a Lambda repo, uncomment the two lines below ###
42+
# update-lambda-dev: ## Updates the lambda with whatever is the most recent image in the ecr (intended for developer-based manual update)
43+
# @ARCH_TAG=$$(cat .arch_tag); \
44+
# aws lambda update-function-code \
45+
# --region us-east-1 \
46+
# --function-name $(FUNCTION_DEV) \
47+
# --image-uri $(ECR_URL_DEV):make-$$ARCH_TAG
48+
49+
docker-clean: ## Clean up Docker detritus
50+
@ARCH_TAG=$$(cat .arch_tag); \
51+
echo "Cleaning up Docker leftovers (containers, images, builders)"; \
52+
docker rmi -f $(ECR_URL_DEV):make-$$ARCH_TAG; \
53+
docker rmi -f $(ECR_URL_DEV):make-$(shell git describe --always) || true; \
54+
docker rmi -f $(ECR_URL_DEV):make-$(shell echo $(CPU_ARCH) | cut -d'/' -f2) || true; \
55+
docker rmi -f $(ECR_NAME_DEV):$$ARCH_TAG || true; \
56+
docker buildx rm $(ECR_NAME_DEV) || true
57+
@rm -rf .arch_tag

‎files/prod-promote-cpu-arch-extra-region.tpl‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
### This should be added to jobs section of the prod-promote.yml.
2+
### If this is a Lambda function, uncomment the FUNCTION: line
3+
4+
deploy-${region}:
5+
needs: prep
6+
name: Deploy ${region}
7+
uses: mitlibraries/.github/.github/workflows/ecr-multi-arch-promote-prod.yml@main
8+
secrets: inherit
9+
with:
10+
AWS_REGION: "${region}"
11+
GHA_ROLE_STAGE: ${role_stage}
12+
GHA_ROLE_PROD: ${role_prod}
13+
ECR_STAGE: "${ecr_stage}"
14+
ECR_PROD: "${ecr_prod}"
15+
CPU_ARCH: $${{ needs.prep.outputs.cpuarch }}
16+
# FUNCTION: "${function}"
17+

‎files/prod-promote-cpu-arch.tpl‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
### This is the Terraform-generated prod-promote.yml workflow for the ###
2+
### ${ecr_prod} repository. ###
3+
### If this is a Lambda repo, uncomment the FUNCTION line at the end of ###
4+
### the document. ###
5+
6+
name: Prod Container Promote
7+
on:
8+
workflow_dispatch:
9+
release:
10+
types: [published]
11+
12+
permissions:
13+
id-token: write
14+
contents: read
15+
16+
jobs:
17+
prep:
18+
name: Prep for Promote
19+
runs-on: ubuntu-latest
20+
outputs:
21+
cpuarch: $${{ steps.setarch.outputs.cpuarch }}
22+
steps:
23+
- name: Checkout
24+
uses: actions/checkout@v5
25+
26+
- name: Set CPU Architecture
27+
id: setarch
28+
run: |
29+
echo "### :abacus: Architecture Selection" >> $GITHUB_STEP_SUMMARY
30+
if [[ -f .aws-architecture ]]; then
31+
ARCH=$(cat .aws-architecture)
32+
echo "\`$ARCH\` was read from \`.aws-architecture\` and passed to the deploy job." >> $GITHUB_STEP_SUMMARY
33+
else
34+
ARCH="linux/amd64"
35+
echo "No \`.aws-architecture\` file, so default \`$ARCH\` was passed to the deploy job." >> $GITHUB_STEP_SUMMARY
36+
fi
37+
if [[ "$ARCH" != "linux/arm64" && "$ARCH" != "linux/amd64" ]]; then
38+
echo "$ARCH is INVALID architecture!"
39+
echo "$ARCH is INVALID architecture!" >> $GITHUB_STEP_SUMMARY
40+
exit 1
41+
fi
42+
echo "cpuarch=$ARCH" >> $GITHUB_OUTPUT
43+
44+
deploy:
45+
needs: prep
46+
name: Deploy
47+
uses: mitlibraries/.github/.github/workflows/ecr-multi-arch-promote-prod.yml@main
48+
secrets: inherit
49+
with:
50+
AWS_REGION: "${region}"
51+
GHA_ROLE_STAGE: ${role_stage}
52+
GHA_ROLE_PROD: ${role_prod}
53+
ECR_STAGE: "${ecr_stage}"
54+
ECR_PROD: "${ecr_prod}"
55+
CPU_ARCH: $${{ needs.prep.outputs.cpuarch }}
56+
# FUNCTION: "${function}"
57+

‎files/stage-build-cpu-arch-extra-region.tpl‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
### This is the Terraform-generated extra workflow job for the ###
2+
### ${ecr} app repository. ###
3+
### This should be added to jobs section of the stage-build.yml. If this ###
4+
### is a Lambda function, uncomment the FUNCTION: line ###
5+
6+
deploy-${region}:
7+
needs: prep
8+
name: Stage Deploy ${region}
9+
uses: mitlibraries/.github/.github/workflows/ecr-multi-arch-deploy-stage.yml@main
10+
secrets: inherit
11+
with:
12+
AWS_REGION: "${region}"
13+
GHA_ROLE: "${role}"
14+
ECR: "${ecr}"
15+
CPU_ARCH: $${{ needs.prep.outputs.cpuarch }}
16+
# FUNCTION: "${function}"
17+
# PREBUILD:

0 commit comments

Comments
 (0)