Skip to content

Commit 2821907

Browse files
Allow upload of CA certs for self signed certificate use (#1383)
* testing changes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Add JS for file validation Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * JS cleanup Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * linting fixes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Working till adding gateway Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Use ca cert for tool calls Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Fix health checks Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Use ca_cert in update_gateway Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Flake8 fixes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Linting fixes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Update doctest Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Add Ed25519 signing code Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Add validator for public key Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Add cert validation Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Allow multiple uploads Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * linting fixes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * bandit fix Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Fix some tests Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Fix test Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Fix tests Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Linting fixes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Fix fstring Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * eslint fixes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * lint-web fixes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Add alembic migration Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Make signing certs optional Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Update README Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Minor change to README Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Update sso_provider field validator Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Update charts Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * flake8 fix Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * flake8 fixes Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Use Containerfile.lite in docker compose Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * checking compose upgrade for pg 18 Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Include pg_hba.conf step in upgrade Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * lint fix Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Mention about Postgres upgrade in Changelog Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Minor fix to commented alembic upgrade Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * Add documentation on self signed certs Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> * fix: resolve rebase conflicts and update plugin API calls - Fix imports: mcpgateway.models -> mcpgateway.common.models - Add missing ToolHookType import - Update plugin manager API: tool_pre_invoke -> invoke_hook with ToolHookType - Update plugin manager API: tool_post_invoke -> invoke_hook with ToolHookType - Update HttpHeaderPayload: headers -> root parameter - Create alembic merge migration for CA cert and observability heads - Apply pre-commit formatting fixes (trailing whitespace, tabs, encoding pragma) Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * Rebase Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * Coverage Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> --------- Signed-off-by: Madhav Kandukuri <madhav165@gmail.com> Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> Co-authored-by: Mihai Criveti <crivetimihai@gmail.com>
1 parent c01e91f commit 2821907

26 files changed

+1990
-336
lines changed

.env.example

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1179,3 +1179,11 @@ PAGINATION_INCLUDE_LINKS=true
11791179

11801180
# Enable event logging within spans
11811181
# OBSERVABILITY_EVENTS_ENABLED=true
1182+
1183+
#####################################
1184+
# Ed25519 Key Support
1185+
#####################################
1186+
# Ed25519 private key for signing
1187+
ENABLE_ED25519_SIGNING=false
1188+
PREV_ED2519_PRIVATE_KEY=""
1189+
ED25519_PRIVATE_KEY=private_key

.github/workflows/pytest.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
# ===============================================================
44
#
55
# - runs the full test-suite across three Python versions
6-
# - measures branch + line coverage (fails < 40 %)
6+
# - measures branch + line coverage (fails < 60% main, < 40% doctest)
77
# - uploads the XML/HTML coverage reports as build artifacts
88
# - (optionally) generates / commits an SVG badge - kept disabled
99
# - posts a concise per-file coverage table to the job summary
@@ -67,7 +67,7 @@ jobs:
6767
python-version: ${{ matrix.python }}
6868

6969
# -----------------------------------------------------------
70-
# 3️⃣ Run the tests with coverage (fail under 65% coverage)
70+
# 3️⃣ Run the tests with coverage (fail under 60% coverage)
7171
# -----------------------------------------------------------
7272
- name: 🧪 Run pytest
7373
run: |
@@ -78,7 +78,7 @@ jobs:
7878
--cov-report=html \
7979
--cov-report=term \
8080
--cov-branch \
81-
--cov-fail-under=65
81+
--cov-fail-under=60
8282
8383
# -----------------------------------------------------------
8484
# 4️⃣ Run doctests (fail under 40% coverage)

CHANGELOG.md

Lines changed: 114 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,11 @@
44
55
---
66

7-
## [0.9.0] - 2025-11-04 [WIP] - REST Passthrough, Multi-Tenancy Fixes & Platform Enhancements
7+
## [0.9.0] - 2025-11-05 - REST Passthrough, Ed25519 Certificate Signing, Multi-Tenancy Fixes & Platform Enhancements
88

99
### Overview
1010

11-
This release delivers **REST API Passthrough Capabilities**, **API & UI Pagination**, **Multi-Tenancy Bug Fixes**, and **Platform Enhancements** with **60+ issues resolved** and **50+ PRs merged**, bringing significant improvements across security, observability, and developer experience:
11+
This release delivers **Ed25519 Certificate Signing**, **REST API Passthrough Capabilities**, **API & UI Pagination**, **Multi-Tenancy Bug Fixes**, and **Platform Enhancements** with **60+ issues resolved** and **50+ PRs merged**, bringing significant improvements across security, observability, and developer experience:
1212

1313
- **📄 REST API & UI Pagination** - Comprehensive pagination support for all admin endpoints with HTMX-based UI and performance testing up to 10K records
1414
- **🔌 REST Passthrough API Fields** - Comprehensive REST tool configuration with query/header mapping, timeouts, and plugin chains
@@ -21,6 +21,92 @@ This release delivers **REST API Passthrough Capabilities**, **API & UI Paginati
2121
- **🦀 Rust Plugin Framework** - Optional Rust-accelerated plugins with 5-100x performance improvements
2222
- **💻 Admin UI** - Quality of life improvements for admins when managing MCP servers
2323

24+
### ⚠️ BREAKING CHANGES
25+
26+
#### **🗄️ PostgreSQL 17 → 18 Upgrade Required**
27+
28+
**Docker Compose users must run the upgrade utility before starting the stack.**
29+
30+
The default PostgreSQL image has been upgraded from version 17 to 18. This is a **major version upgrade** that requires a one-time data migration using `pg_upgrade`.
31+
32+
**Migration Steps:**
33+
34+
1. **Stop your existing stack:**
35+
```bash
36+
docker compose down
37+
```
38+
39+
2. **Run the automated upgrade utility:**
40+
```bash
41+
make compose-upgrade-pg18
42+
```
43+
44+
This will:
45+
- Prompt for confirmation (⚠️ **backup recommended**)
46+
- Run `pg_upgrade` to migrate data from Postgres 17 → 18
47+
- Automatically copy `pg_hba.conf` to preserve network access settings
48+
- Create a new `pgdata18` volume with upgraded data
49+
50+
3. **Start the upgraded stack:**
51+
```bash
52+
make compose-up
53+
```
54+
55+
4. **(Optional) Run maintenance commands** to update statistics:
56+
```bash
57+
docker compose exec postgres /usr/lib/postgresql/18/bin/vacuumdb --all --analyze-in-stages --missing-stats-only -U postgres
58+
docker compose exec postgres /usr/lib/postgresql/18/bin/vacuumdb --all --analyze-only -U postgres
59+
```
60+
61+
5. **Verify the upgrade:**
62+
```bash
63+
docker compose exec postgres psql -U postgres -c 'SELECT version();'
64+
# Should show: PostgreSQL 18.x
65+
```
66+
67+
6. **(Optional) Clean up old volume** after confirming everything works:
68+
```bash
69+
docker volume rm mcp-context-forge_pgdata
70+
```
71+
72+
**Manual Upgrade (without Make):**
73+
74+
If you prefer not to use the Makefile:
75+
76+
```bash
77+
# Stop stack
78+
docker compose down
79+
80+
# Run upgrade
81+
docker compose -f docker-compose.yml -f compose.upgrade.yml run --rm pg-upgrade
82+
83+
# Copy pg_hba.conf
84+
docker compose -f docker-compose.yml -f compose.upgrade.yml run --rm pg-upgrade \
85+
sh -c "cp /var/lib/postgresql/OLD/pg_hba.conf /var/lib/postgresql/18/docker/pg_hba.conf"
86+
87+
# Start upgraded stack
88+
docker compose up -d
89+
```
90+
91+
**Why This Change:**
92+
93+
- Postgres 18 introduces a new directory structure (`/var/lib/postgresql/18/docker`) for better compatibility with `pg_ctlcluster`
94+
- Enables future upgrades using `pg_upgrade --link` without mount point boundary issues
95+
- Aligns with official PostgreSQL Docker image best practices (see [postgres#1259](https://github.com/docker-library/postgres/pull/1259))
96+
97+
**What Changed:**
98+
99+
- `docker-compose.yml`: Updated from `postgres:17``postgres:18`
100+
- Volume mount: Changed from `pgdata:/var/lib/postgresql/data``pgdata18:/var/lib/postgresql`
101+
- Added `compose.upgrade.yml` for automated upgrade process
102+
- Added `make compose-upgrade-pg18` target for one-command upgrades
103+
104+
**Troubleshooting:**
105+
106+
- **Error: "data checksums mismatch"** - Fixed automatically in upgrade script (disables checksums to match old cluster)
107+
- **Error: "no pg_hba.conf entry"** - Fixed automatically by copying old `pg_hba.conf` during upgrade
108+
- **Error: "Invalid cross-device link"** - Upgrade uses copy mode (not `--link`) to work across different Docker volumes
109+
24110
### Added
25111

26112
#### **📄 REST API and UI Pagination** (#1224, #1277)
@@ -101,7 +187,32 @@ This release delivers **REST API Passthrough Capabilities**, **API & UI Paginati
101187
* **Keycloak Integration** (#1217, #1216, #1109) - Full Keycloak support with application/x-www-form-urlencoded
102188
* **OAuth Timeout Configuration** (#1201) - Configurable `OAUTH_DEFAULT_TIMEOUT` for OAuth providers
103189

104-
#### **🔌 Plugin Framework Enhancements** (#1196, #1198, #1137, #1240, #1289)
190+
#### **� Ed25519 Certificate Signing** - Enhanced certificate validation and integrity verification
191+
* **Digital Certificate Signing** - Sign and verify certificates using Ed25519 cryptographic signatures
192+
- Ensures certificate authenticity and prevents tampering
193+
- Built on proven Ed25519 algorithm (RFC 8032) for high security and performance
194+
- Zero-dependency Python implementation using `cryptography` library
195+
* **Key Generation Utility** - Built-in key generation tool at `mcpgateway/utils/generate_keys.py`
196+
- Generates secure Ed25519 private keys in base64 format
197+
- Simple command-line interface for development and production use
198+
* **Key Rotation Support** - Graceful key rotation with zero downtime
199+
- Configure both current (`ED25519_PRIVATE_KEY`) and previous (`PREV_ED25519_PRIVATE_KEY`) keys
200+
- Automatic fallback to previous key for verification during rotation period
201+
- Supports rolling updates in distributed deployments
202+
* **Environment Variable Configuration** - Three new environment variables for certificate signing
203+
- `ENABLE_ED25519_SIGNING` - Enable/disable signing (default: "false")
204+
- `ED25519_PRIVATE_KEY` - Current signing key (base64-encoded)
205+
- `PREV_ED25519_PRIVATE_KEY` - Previous key for rotation support (base64-encoded)
206+
* **Kubernetes & Helm Support** - Full integration with Helm chart deployment
207+
- Secret management via `values.yaml` configuration
208+
- JSON Schema validation in `values.schema.json`
209+
- External Secrets Operator integration examples
210+
* **Production Ready** - Comprehensive documentation and security best practices
211+
- Complete documentation in main README.md
212+
- Helm chart documentation with Kubernetes examples
213+
- Security guidelines for key storage and rotation
214+
215+
#### **�🔌 Plugin Framework Enhancements** (#1196, #1198, #1137, #1240, #1289)
105216
* **🦀 Rust Plugin Framework** (#1289, #1249) - Optional Rust-accelerated plugins with automatic Python fallback
106217
- Complete PyO3-based framework for building high-performance plugins
107218
- **PII Filter (Rust)**: 5-100x faster than Python implementation with identical functionality

Makefile

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2780,6 +2780,21 @@ compose-validate:
27802780
$(COMPOSE) config --quiet
27812781
@echo "✅ Compose file is valid"
27822782

2783+
compose-upgrade-pg18: compose-validate
2784+
@echo "⚠️ This will upgrade Postgres 17 -> 18"
2785+
@echo "⚠️ Make sure you have a backup!"
2786+
@read -p "Continue? [y/N] " confirm && [ "$$confirm" = "y" ] || exit 1
2787+
@echo "🔄 Running Postgres upgrade..."
2788+
$(COMPOSE) -f $(COMPOSE_FILE) -f compose.upgrade.yml run --rm pg-upgrade
2789+
@echo "🔧 Copying pg_hba.conf from old cluster..."
2790+
@$(COMPOSE) -f $(COMPOSE_FILE) -f compose.upgrade.yml run --rm pg-upgrade sh -c \
2791+
"cp /var/lib/postgresql/OLD/pg_hba.conf /var/lib/postgresql/18/docker/pg_hba.conf && \
2792+
echo '✅ pg_hba.conf copied successfully'"
2793+
@echo "✅ Upgrade complete!"
2794+
@echo "📝 Next steps:"
2795+
@echo " 1. Update docker-compose.yml to use postgres:18"
2796+
@echo " 2. Run: make compose-up"
2797+
27832798
compose-up: compose-validate
27842799
@echo "🚀 Using $(COMPOSE_CMD); starting stack..."
27852800
IMAGE_LOCAL=$(call get_image_name) $(COMPOSE) up -d

README.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1506,6 +1506,76 @@ ContextForge implements **OAuth 2.0 Dynamic Client Registration (RFC 7591)** and
15061506
> Documentation endpoints (`/docs`, `/redoc`, `/openapi.json`) are always protected by authentication.
15071507
> By default, they require Bearer token authentication. Setting `DOCS_ALLOW_BASIC_AUTH=true` enables HTTP Basic Authentication as an additional method using the same credentials as `BASIC_AUTH_USER` and `BASIC_AUTH_PASSWORD`.
15081508

1509+
### Ed25519 Certificate Signing
1510+
1511+
MCP Gateway supports **Ed25519 digital signatures** for certificate validation and integrity verification. This cryptographic signing mechanism ensures that CA certificates used by the gateway are authentic and haven't been tampered with.
1512+
1513+
| Setting | Description | Default | Options |
1514+
| --------------------------- | ------------------------------------------------ | ------- | ------- |
1515+
| `ENABLE_ED25519_SIGNING` | Enable Ed25519 signing for certificates | `false` | bool |
1516+
| `ED25519_PRIVATE_KEY` | Ed25519 private key for signing (PEM format) | (none) | string |
1517+
| `PREV_ED25519_PRIVATE_KEY` | Previous Ed25519 private key for key rotation | (none) | string |
1518+
1519+
**How It Works:**
1520+
1521+
1. **Certificate Signing** - When `ENABLE_ED25519_SIGNING=true`, the gateway signs the CA certificate of each MCP server/gateway using the Ed25519 private key.
1522+
1523+
2. **Certificate Validation** - Before using a CA certificate for subsequent calls, the gateway validates its signature to ensure authenticity and integrity.
1524+
1525+
3. **Disabled Mode** - When `ENABLE_ED25519_SIGNING=false`, certificates are neither signed nor validated (default behavior).
1526+
1527+
**Key Generation:**
1528+
1529+
```bash
1530+
# Generate a new Ed25519 key pair
1531+
python mcpgateway/utils/generate_keys.py
1532+
1533+
# Output will show:
1534+
# - Private key (set this to ED25519_PRIVATE_KEY)
1535+
```
1536+
1537+
**Key Rotation:**
1538+
1539+
To rotate keys without invalidating existing signed certificates:
1540+
1541+
1. Move the current `ED25519_PRIVATE_KEY` value to `PREV_ED25519_PRIVATE_KEY`
1542+
2. Generate a new key pair using the command above
1543+
3. Set the new private key to `ED25519_PRIVATE_KEY`
1544+
4. The gateway will automatically re-sign valid certificates at the point of key change
1545+
1546+
**Example Configuration:**
1547+
1548+
```bash
1549+
# Enable Ed25519 signing
1550+
ENABLE_ED25519_SIGNING=true
1551+
1552+
# Current signing key (PEM format)
1553+
ED25519_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----
1554+
MC4CAQAwBQYDK2VwBCIEIJ5pW... (your key here)
1555+
-----END PRIVATE KEY-----"
1556+
1557+
# Previous key for rotation (optional)
1558+
PREV_ED25519_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----
1559+
MC4CAQAwBQYDK2VwBCIEIOld... (old key here)
1560+
-----END PRIVATE KEY-----"
1561+
```
1562+
1563+
> 🔐 **Security Best Practices:**
1564+
> - Store private keys securely (use secrets management tools like Vault, AWS Secrets Manager, etc.)
1565+
> - Rotate keys periodically (recommended: every 90-180 days)
1566+
> - Never commit private keys to version control
1567+
> - Use environment variables or encrypted config files
1568+
>
1569+
> 🔑 **Public Key Derivation:**
1570+
> - Public keys are automatically derived from private keys
1571+
> - No need to configure public keys separately
1572+
> - Both `ED25519_PUBLIC_KEY` and `PREV_ED25519_PUBLIC_KEY` are computed at startup
1573+
>
1574+
> **Performance:**
1575+
> - Ed25519 signing is extremely fast (~64 microseconds per signature)
1576+
> - Minimal impact on gateway performance
1577+
> - Recommended for production deployments requiring certificate integrity
1578+
15091579
### Response Compression
15101580

15111581
MCP Gateway includes automatic response compression middleware that reduces bandwidth usage by 30-70% for text-based responses (JSON, HTML, CSS, JS). Compression is negotiated automatically based on client `Accept-Encoding` headers with algorithm priority: **Brotli** (best compression) > **Zstd** (fastest) > **GZip** (universal fallback).

charts/mcp-stack/CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
66

77
---
88

9+
## [0.9.0] - 2025-11-05
10+
11+
### Added
12+
* **Ed25519 Certificate Signing** - Digital certificate signing and verification using Ed25519 cryptographic signatures:
13+
- `ENABLE_ED25519_SIGNING` - Enable/disable Ed25519 certificate signing (default: "false")
14+
- `ED25519_PRIVATE_KEY` - Current Ed25519 private key for signing (base64-encoded)
15+
- `PREV_ED25519_PRIVATE_KEY` - Previous Ed25519 private key for graceful key rotation (base64-encoded)
16+
- Built-in key generation utility at `mcpgateway/utils/generate_keys.py`
17+
- Zero-downtime key rotation support with automatic fallback verification
18+
- Complete documentation with Kubernetes examples and External Secrets integration
19+
- Ensures certificate authenticity and prevents tampering using RFC 8032 algorithm
20+
21+
### Changed
22+
* **Chart version** - Bumped to 0.9.0 for Ed25519 certificate signing support
23+
924
## [0.6.1] - 2025-09-01
1025

1126
### Added

0 commit comments

Comments
 (0)