ContextForge Memory (v0)

Lightweight external memory layer providing store, search, and embed APIs for multi-repo reuse (Brainstorm, Cursor, and others).

🚨 CRITICAL: Virtual Environment Required

ALL Python operations MUST be performed in a virtual environment to prevent system corruption. Never install dependencies directly on the host system.

Python Version Requirements

Python 3.12+ Required: This project requires Python 3.12 or higher. Python 3.11 and earlier are not supported.

Features

• Versioned API: /v0/ with store, search, embed.
• Deterministic, minimal embedding (hash-based) for baseline functionality.
• Pluggable storage (default: JSONL on disk).
• v1 features with in-memory vector index and ephemeral session store.
• Automatic backfill system for indexing existing data on startup.

⚠️ Production Considerations

This service uses in-memory storage for the vector index and session store by default. Data will be lost on restart, making this approach unsuitable for production use.

For production deployments, consider:

• Persisting the vector index to disk or a vector database (e.g., Pinecone, Weaviate)
• Storing session state in a persistent store (e.g., Redis, PostgreSQL)
• Implementing proper backup and recovery mechanisms

Quickstart

Environment Configuration

First, set up your environment variables:

# Copy the example environment file
cp .env.example .env

# Edit with your configuration
nano .env

Required for production:

• CF_API_KEY: Set to a secure random string for API authentication
• CF_OPENAI_API_KEY: Optional, for OpenAI-based embeddings and summarization

For secure key generation: Use cryptographically secure methods like openssl rand -hex 32 or your cloud provider's key management service. See Security Documentation for complete key management and rotation guidance.

For development: Environment variables are automatically configured for testing.

See Configuration Guide for complete variable reference.

Production Setup

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn src.contextforge_memory.main:app --host 0.0.0.0 --port 8085

Development Setup

python -m venv .venv && source .venv/bin/activate
make setup  # Installs both production and development dependencies
make run    # Start development server

Optional Dependencies

For enhanced functionality, install optional dependency groups:

# OpenAI integration (embeddings and summarization)
pip install -e ".[openai]"

# Development tools (linting, testing, formatting)
pip install -e ".[dev]"

# Sentence-Transformers support (install separately)
pip install sentence-transformers

Notes:

• If sentence-transformers is not installed, constructing the SentenceTransformersProvider raises: RuntimeError: Optional dependency 'sentence-transformers' is not installed.
• In non-strict mode, missing optional dependencies for providers cause a fallback to the hash provider and emit an ERROR log; strict mode raises.

Prerequisites for type checking:

• Node.js and npm must be installed for make type-check (pyright is Node.js-based)
• Install Node.js from nodejs.org or via your package manager

Development Commands

Command	Description
make setup	Install all dependencies (production + development)
make run	Start development server
make test	Run all test suites
make test-api	Run API contract tests only
make test-integration	Run integration tests only
make format	Format code with black
make lint	Lint code with ruff
make type-check	Run type checking with pyright (requires Node.js)
make precommit	Run pre-commit hooks
make smoke	Quick smoke test

PR Automation

Process CodeRabbit and other bot suggestions from PR comments with intelligent multi-option handling:

# Fetch PR comments (requires OWNER, REPO, PR environment variables)
# Automatically filters out resolved comments
make pr_comments

# Preview all suggestions without applying changes
# Automatically detects and handles multiple resolution options
make pr_suggest_preview

# Review auto-selected options (if multiple options detected)
cat .cr/options_config.json

# Apply safe line-range replacements
# Tracks successfully applied comments to avoid re-processing
make pr_suggest_apply

# Apply unified diffs via git apply
make pr_suggest_gitapply

# Resolve applied comments automatically
make pr_suggest_resolve_dryrun  # Preview what would be resolved
make pr_suggest_resolve         # Actually resolve the comments

# Clean up tracking files (optional)
make pr_cleanup

New Multi-Option Features:

• 🤖 AI Auto-Selection: Intelligently chooses the best option from multiple CodeRabbit suggestions
• ⚙️ User Override: Edit .cr/options_config.json to customize selections
• 📊 Enhanced Reporting: Clear visibility into multi-option scenarios
• 🔄 Backward Compatible: Single-option comments work exactly as before

Resolved Comment Filtering:

• 🚫 Skips resolved comments: Automatically filters out resolved comments during fetch
• 🔄 Safe multiple runs: Won't re-apply suggestions that were already successfully applied
• 📁 Per-PR tracking: Each PR has its own tracking file (automatically cleaned after 30 days)
• 🧹 Multiple PR support: Handles multiple PRs independently

See PR Automation Documentation for detailed usage and safety features.

Dependency Management

Dependencies are automatically updated by Renovate. Check the Dependency Dashboard for pending updates and status.

• Weekly updates: Patch and minor versions auto-merge after CI passes
• Major updates: Require manual review
• Security updates: Created immediately, require review
• Configuration: See renovate.json and Renovate Documentation
• Branch protection: All CI checks must pass for auto-merge (lint, test, security, pre-commit, semgrep, typescript-client)

Pre-commit Hooks

This project uses pre-commit hooks for code quality and security:

# Install pre-commit hooks (done by make setup)
pre-commit install
pre-commit install -t commit-msg -t pre-push

# Run hooks manually
make precommit

⚠️ Pre-push Hook Prerequisites: The pre-push hooks require pytest and pip-audit to be available in your system PATH. See Development Guide for installation options.

Hooks include: black (formatting), ruff (linting), prettier (docs), detect-secrets (security), commitizen (commit messages), and pre-push tests.

API Endpoints

• GET /v0/health
• POST /v0/store
• POST /v0/search
• POST /v0/embed

v1 (enabled by default; set CF_ENABLE_V1=false to disable)

• POST /v1/embed (provider-backed)
• POST /v1/store (accepts optional vectors, updates vector index)
• POST /v1/search (vector search)
• POST /v1/checkpoint (ephemeral session checkpoint)
• POST /v1/restore (stitched context: recent checkpoints + relevant memory)

See openapi/openapi-v0.yaml for schemas.

Authentication (v1)

• All v1 endpoints require an API key via the x-api-key header.
• The OpenAPI spec (openapi/openapi-v1.yaml) declares a top-level security requirement using ApiKeyAuth (type: apiKey, in: header, name: x-api-key).

Example curl:

curl -X POST http://localhost:8085/v1/search \
  -H 'content-type: application/json' \
  -H 'x-api-key: YOUR_API_KEY' \
  -d '{"namespace":"acme:project:dev","project_id":"project","query":"hello"}'

Request constraints

• EmbedRequest.texts: minItems=1, maxItems=100; each string up to 64KB.
• EmbedResponse.vectors: maxItems=100 matches EmbedRequest.texts upper bound.
• CheckpointRequest.phase: one of planning, execution, review.

Clients

• Python: clients/python/contextforge_client.py - Full-featured Python client with retry logic
• TypeScript: clients/typescript/contextforgeClient.ts - TypeScript client with type definitions

Both clients expose v0 and v1 helpers with robust error handling and timeout support.

Python Client Example

from contextforge_client import ContextForgeClient

client = ContextForgeClient(
    base_url="http://localhost:8085",
    api_key="your-api-key",  # Required for v1 endpoints  # pragma: allowlist secret
    timeout=30.0,
    max_retries=3
)

# v0 endpoints (no authentication required)
result = client.v0_store(namespace="test", project_id="test", items=[])
result = client.v0_search(namespace="test", project_id="test", query="test")

# v1 endpoints (authentication required)
result = client.v1_store(namespace="test", project_id="test", items=[])
result = client.v1_search(namespace="test", project_id="test", query="test")

Testing

This project uses a comprehensive dual-test-suite architecture to ensure both user-facing behavior and internal functionality are thoroughly validated.

Test Architecture

Public API Tests (test_backfill_scanning.py):

• Tests user-facing FastAPI endpoints
• Validates authentication, rate limiting, and response formats
• Production-aligned testing that matches real client interactions
• Fast execution with no async complexity

Integration Tests (test_backfill_integration.py):

• Tests complete backfill functionality by calling internal functions
• Validates embedding generation, batching, and data processing
• Comprehensive coverage of internal logic and edge cases
• Proper async handling with isolated execution

Running Tests

Quick API validation (recommended for development):

make test-api
# or
pytest tests/test_backfill_scanning.py -v

Comprehensive functionality testing:

make test-integration
# or
pytest tests/test_backfill_integration.py -v

Full test suite (for CI/CD):

make test
# or
pytest tests/test_backfill_scanning.py && pytest tests/test_backfill_integration.py

Test Coverage

• 13 total tests across both suites
• 100% functionality coverage
• Production-aligned behavior validation
• No async conflicts between test suites
• Clean separation of concerns

See Testing Strategy Documentation for detailed information about the testing approach, architecture, and development guidelines.

CI/CD

GitHub Actions Workflows

• pre-commit.yml: Main CI pipeline with code quality checks, testing, and security scanning
• pip-audit-scheduled.yml: Bi-weekly security vulnerability scanning
• pre-commit-autoupdate.yml: Weekly pre-commit hook updates
• semgrep.yml: Static application security testing

Security Features

• Least-privilege permissions in all workflows
• Pinned tool versions for reproducible builds
• Automated vulnerability scanning with pip-audit and semgrep
• Secrets detection with detect-secrets
• Fail-fast principles - no silent failures

See CI/CD Documentation for detailed workflow information.

Documentation

• Development Guide: Quick start guide for developers
• Configuration Guide: Environment variables and dependencies
• Security Documentation: Authentication and security practices
• CI/CD Documentation: GitHub Actions workflows and automation
• Enhancement Summary: Recent improvements and features
• Contributing Guide: How to contribute to the project

Examples

Run the comprehensive example script to see all features in action:

python examples/example_usage.py

This script demonstrates:

• v0 and v1 API usage
• Client library integration
• Authentication patterns
• Error handling
• All endpoint types