Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/deploy_mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ on:
- 'CONTRIBUTING.md'
- 'docs/**'
- '.github/workflows/deploy_mkdocs.yml'
workflow_dispatch: # Allow manual triggering
schedule:
# Run daily at 6 AM UTC to check for updates in eoapi-k8s docs
- cron: '0 6 * * *'

jobs:
build:
Expand All @@ -20,6 +24,51 @@ jobs:
- name: Checkout main
uses: actions/checkout@v3

- name: Checkout eoapi-k8s docs
uses: actions/checkout@v3
with:
repository: developmentseed/eoapi-k8s
path: eoapi-k8s-temp
ref: main

- name: Setup Kubernetes docs directory
run: |
mkdir -p docs/src/deployment/kubernetes

if [ -f "eoapi-k8s-temp/docs/docs-config.json" ]; then
echo "Using docs-config.json for structured documentation sync"

find eoapi-k8s-temp/docs -name "*.md" -not -name "README.md" \
-exec cp {} docs/src/deployment/kubernetes/ \;

ASSETS_DIR=$(grep -o '"assets_dir": *"[^"]*"' eoapi-k8s-temp/docs/docs-config.json | cut -d'"' -f4)
if [ -n "$ASSETS_DIR" ] && [ -d "eoapi-k8s-temp/docs/$ASSETS_DIR" ]; then
cp -r "eoapi-k8s-temp/docs/$ASSETS_DIR" docs/src/deployment/kubernetes/
fi

find eoapi-k8s-temp/docs -name "*.svg" \
-exec cp {} docs/src/deployment/kubernetes/ \;
else
echo "Fallback: copying all documentation files"

find eoapi-k8s-temp/docs -name "*.md" \
-exec cp {} docs/src/deployment/kubernetes/ \;
if [ -d "eoapi-k8s-temp/docs/images" ]; then
cp -r eoapi-k8s-temp/docs/images docs/src/deployment/kubernetes/
fi
find eoapi-k8s-temp/docs -name "*.svg" \
-exec cp {} docs/src/deployment/kubernetes/ \;
fi

cat >> docs/src/deployment/kubernetes/index.md << EOF

---

*Documentation last synchronized: $(date -u +"%Y-%m-%d %H:%M:%S UTC")*
EOF

rm -rf eoapi-k8s-temp

- name: Set up Python 3.8
uses: actions/setup-python@v4
with:
Expand Down
7 changes: 5 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,10 @@ node_modules/
# Demo files to ignore
demo/cmip6/CMIP6_daily_*stac_items.ndjson


# browser compiled code and default config
infrastructure/aws/stac-browser
infrastructure/aws/browser_config.js
infrastructure/aws/browser_config.js

# imported docs
docs/src/deployment/kubernetes/
eoapi-k8s/
44 changes: 44 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
.PHONY: docs-sync docs-build docs-serve docs-clean help

# Default target
help:
@echo "Available commands:"
@echo " docs-sync - Sync eoapi-k8s documentation"
@echo " docs-build - Build documentation site"
@echo " docs-serve - Serve documentation locally"
@echo " docs-clean - Clean documentation build files"

# Sync eoapi-k8s documentation
docs-sync:
@echo "Syncing eoapi-k8s documentation..."
@mkdir -p docs/src/deployment/kubernetes
@if [ ! -d "eoapi-k8s" ]; then \
git clone https://github.com/developmentseed/eoapi-k8s.git; \
fi
@cd eoapi-k8s && git fetch && git checkout main
@if [ -f "eoapi-k8s/docs/docs-config.json" ]; then \
find eoapi-k8s/docs -name "*.md" -not -name "README.md" -exec cp {} docs/src/deployment/kubernetes/ \;; \
ASSETS_DIR=$$(grep -o '"assets_dir": *"[^"]*"' eoapi-k8s/docs/docs-config.json | cut -d'"' -f4); \
if [ -n "$$ASSETS_DIR" ] && [ -d "eoapi-k8s/docs/$$ASSETS_DIR" ]; then \
cp -r "eoapi-k8s/docs/$$ASSETS_DIR" docs/src/deployment/kubernetes/; \
fi; \
find eoapi-k8s/docs -name "*.svg" -exec cp {} docs/src/deployment/kubernetes/ \;; \
fi
@echo "" >> docs/src/deployment/kubernetes/index.md
@echo "---" >> docs/src/deployment/kubernetes/index.md
@echo "" >> docs/src/deployment/kubernetes/index.md
@echo "*Documentation last synchronized: $$(date -u +"%Y-%m-%d %H:%M:%S UTC")*" >> docs/src/deployment/kubernetes/index.md
@echo "Documentation sync complete!"

# Build documentation
docs-build: docs-sync
@cd docs && mkdocs build

# Serve documentation locally
docs-serve: docs-sync
@cd docs && mkdocs serve

# Clean documentation build files
docs-clean:
@rm -rf docs/build docs/src/deployment/kubernetes eoapi-k8s
@echo "Documentation cleaned!"
18 changes: 17 additions & 1 deletion docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,23 @@ nav:
- Contributing: contributing.md
- Advanced User Guide:
- Customization: customization.md
- Deployment: deployment.md
- Deployment:
- deployment.md
- Kubernetes:
- Overview: deployment/kubernetes/index.md
- Installation:
- Quick Start: deployment/kubernetes/quick-start.md
- Helm Install: deployment/kubernetes/helm-install.md
- Configuration: deployment/kubernetes/configuration.md
- Operations:
- Manage Data: deployment/kubernetes/manage-data.md
- Autoscaling: deployment/kubernetes/autoscaling.md
- Unified Ingress: deployment/kubernetes/unified-ingress.md
- STAC Auth Proxy: deployment/kubernetes/stac-auth-proxy.md
- Release: deployment/kubernetes/release.md
- AWS CDK:
- Overview: deployment/aws/index.md


plugins:
- search
Expand Down
162 changes: 14 additions & 148 deletions docs/src/deployment.md
Original file line number Diff line number Diff line change
@@ -1,157 +1,23 @@
---
hide:
- navigation
---
# Deployment

eoAPI can be deployed using Kubernetes or AWS CDK.

## Via [eoapi-cdk](https://github.com/developmentseed/eoapi-cdk)
## Kubernetes (Recommended)

Production-ready deployment with high availability, auto-scaling, and monitoring.

[eoapi-cdk](https://github.com/developmentseed/eoapi-cdk) is a set of AWS CDK constructs that can be used to easily deploy eoAPI services on AWS with the CDK.
**[Complete Kubernetes Documentation](./deployment/kubernetes/index.md)**

[eoapi-template](https://github.com/developmentseed/eoapi-template) is an AWS CDK app that shows how to configure the [eoapi-cdk](https://github.com/developmentseed/eoapi-cdk) constructs.
- Multi-cloud support (AWS, GCP, Azure)
- Helm-based installation
- Built-in monitoring and autoscaling

An example of Cloud Stack is available for AWS (RDS for the database and Lambda for the APIs).
## AWS CDK

The stack is deployed by the [AWS CDK](https://aws.amazon.com/cdk/) utility. Under the hood, CDK will create the deployment packages required for AWS Lambda, upload them to AWS, and handle the creation of the Lambda and API Gateway resources.
Serverless deployment using AWS Lambda, RDS, and API Gateway.

The example commands here will deploy a CloudFormation stack called `eoAPI-staging`.
**[Complete AWS CDK Documentation](./deployment/aws/index.md)**

1. Clone the `eoapi` repo and install dependencies
```bash
# Download eoapi repo
git clone https://github.com/developmentseed/eoapi-template.git
cd eoapi-template

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate

# install cdk dependencies
python -m pip install -r requirements.txt
```

2. Install node dependency - requires node version 14+
```bash
npm install
```

3. Update settings

Set environment variable or complex code in the `.env` or `config.yaml` file (e.g., https://github.com/developmentseed/eoapi-template/blob/main/config.yaml.example).

See https://github.com/developmentseed/eoapi-template/blob/main/infrastructure/config.py for more info on the configuration options.


4. Install CDK and connect to your AWS account. This step is only necessary once per AWS account. The environment variables `PROJECT_ID` and `STAGE` determines the name of the stack
(e.g., eoAPI-staging or eoAPI-production)
```bash
# Deploy the CDK toolkit stack into an AWS environment.
PROJECT_ID=eoAPI \
STAGE=staging \
npx cdk bootstrap
# or to a specific region
AWS_DEFAULT_REGION=us-west-2 \
AWS_REGION=us-west-2 \
PROJECT_ID=eoAPI \
STAGE=staging \
npx cdk bootstrap
```

5. Pre-Generate CFN template

```bash
PROJECT_ID=eoAPI \
STAGE=staging \
npx cdk synth --all # Synthesizes and prints the CloudFormation template for this stack
```

6. Deploy

```bash
# Note: a VPC stack is needed for the database
PROJECT_ID=eoAPI \
STAGE=staging \
npx cdk deploy vpceoAPI-staging eoAPI-staging
# Deploy in a specific region
AWS_DEFAULT_REGION=eu-central-1 \
AWS_REGION=eu-central-1 \
PROJECT_ID=eoAPI \
STAGE=staging \
npx cdk deploy vpceoAPI-staging eoAPI-stagingg --profile {my-aws-profile}
```

If you get an error saying that the max VPCs have been reached, you have hit the limit for the number of VPCs per unique AWS account and region combination. You can change the AWS region to a region with fewer VPCs and deploy again to fix this.

## Via [eoapi-k8s](https://github.com/developmentseed/eoapi-k8s)

[eoapi-k8s](https://github.com/developmentseed/eoapi-k8s) has the IaC and Helm charts for deploying eoAPI services on AWS and GCP.

**Getting started**

If you still need to set up a k8s cluster on AWS or GCP, then follow an IaC guide below that is relevant to you.

> &#9432; The helm chart in this repo assumes your cluster has a few third-party add-ons and controllers installed. So
> It's in your best interest to read through the IaC guides to understand what those defaults are
* :octicons-link-external-16: [AWS EKS Cluster Setup](https://github.com/developmentseed/eoapi-k8s/blob/main/docs/aws-eks.md)
* :octicons-link-external-16: [TBD: GCP GKE Cluster Setup](https://github.com/developmentseed/eoapi-k8s/blob/main/docs/gcp-gke.md)
**Helm Installation**
Once you have a k8s cluster set up, you can `helm install` eoAPI as follows
1. `helm install` from this repo's `helm-chart/` folder:

```python
######################################################
# create os environment variables for required secrets
######################################################
$ export GITSHA=$(git rev-parse HEAD | cut -c1-10)
$ export PGUSER=s00pers3cr3t
$ export POSTGRES_USER=s00pers3cr3t
$ export POSTGRES_PASSWORD=superuserfoobar
$ export PGPASSWORD=foobar
$ cd ./helm-chart
$ helm install \
--namespace eoapi \
--create-namespace \
--set gitSha=$GITSHA \
--set db.settings.secrets.PGUSER=$PGUSER \
--set db.settings.secrets.POSTGRES_USER=$POSTGRES_USER \
--set db.settings.secrets.PGPASSWORD=$PGPASSWORD \
--set db.settings.secrets.POSTGRES_PASSWORD=$POSTGRES_PASSWORD \
eoapi \
./eoapi
```

2. or `helm install` from https://devseed.com/eoapi-k8s/:

```python
# add the eoapi helm repo locally
$ helm repo add eoapi https://devseed.com/eoapi-k8s/
# List out the eoapi chart versions
$ helm search repo eoapi
NAME CHART VERSION APP VERSION DESCRIPTION
eoapi/eoapi 0.1.1 0.1.0 Create a full Earth Observation API with Metada...
eoapi/eoapi 0.1.2 0.1.0 Create a full Earth Observation API with Metada...
# add the required secret overrides to an arbitrarily named `.yaml` file (`config.yaml` below)
$ cat config.yaml
db:
settings:
secrets:
PGUSER: "username"
POSTGRES_USER: "username"
PGPASSWORD: "password"
POSTGRES_PASSWORD: "password"
# then run `helm install` with those overrides
helm install eoapi eoapi/eoapi --version 0.1.1 -f config.yaml
```
- Pay-per-request pricing
- Managed infrastructure
- AWS-native integration
Loading