|
| 1 | +# API Key Configuration Guide |
| 2 | + |
| 3 | +With the new genai backend, sqlite-rembed offers multiple flexible ways to configure API keys directly through SQL, eliminating the need to set environment variables. |
| 4 | + |
| 5 | +## 🔑 API Key Configuration Methods |
| 6 | + |
| 7 | +### Method 1: Simple Provider:Key Format |
| 8 | +The easiest way - just use `provider:your-api-key`: |
| 9 | + |
| 10 | +```sql |
| 11 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 12 | + ('my-openai', 'openai:sk-proj-abc123...'), |
| 13 | + ('my-gemini', 'gemini:AIza...'), |
| 14 | + ('my-groq', 'groq:gsk_abc123...'); |
| 15 | +``` |
| 16 | + |
| 17 | +### Method 2: JSON Configuration |
| 18 | +More explicit with JSON format: |
| 19 | + |
| 20 | +```sql |
| 21 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 22 | + ('my-client', '{"provider": "openai", "api_key": "sk-proj-abc123..."}'); |
| 23 | + |
| 24 | +-- Or specify the full model |
| 25 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 26 | + ('my-client', '{"model": "openai::text-embedding-3-large", "key": "sk-proj-abc123..."}'); |
| 27 | +``` |
| 28 | + |
| 29 | +### Method 3: Using rembed_client_options |
| 30 | +The most flexible approach: |
| 31 | + |
| 32 | +```sql |
| 33 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 34 | + ('my-client', |
| 35 | + rembed_client_options( |
| 36 | + 'format', 'openai', |
| 37 | + 'model', 'text-embedding-3-small', |
| 38 | + 'key', 'sk-proj-abc123...' |
| 39 | + ) |
| 40 | + ); |
| 41 | +``` |
| 42 | + |
| 43 | +### Method 4: Environment Variables (Still Supported) |
| 44 | +For production deployments, you can still use environment variables: |
| 45 | + |
| 46 | +```bash |
| 47 | +export OPENAI_API_KEY="sk-proj-abc123..." |
| 48 | +export GEMINI_API_KEY="AIza..." |
| 49 | +``` |
| 50 | + |
| 51 | +Then register without keys in SQL: |
| 52 | +```sql |
| 53 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 54 | + ('my-openai', 'openai::text-embedding-3-small'); |
| 55 | +``` |
| 56 | + |
| 57 | +## 🎯 Complete Examples |
| 58 | + |
| 59 | +### OpenAI with API Key |
| 60 | +```sql |
| 61 | +-- Simple format |
| 62 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 63 | + ('openai-embed', 'openai:sk-proj-your-key-here'); |
| 64 | + |
| 65 | +-- JSON format |
| 66 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 67 | + ('openai-embed', '{"provider": "openai", "api_key": "sk-proj-your-key-here"}'); |
| 68 | + |
| 69 | +-- Use it |
| 70 | +SELECT rembed('openai-embed', 'Hello, world!'); |
| 71 | +``` |
| 72 | + |
| 73 | +### Multiple Providers with Keys |
| 74 | +```sql |
| 75 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 76 | + -- OpenAI |
| 77 | + ('gpt-small', 'openai:sk-proj-abc123'), |
| 78 | + ('gpt-large', '{"model": "openai::text-embedding-3-large", "key": "sk-proj-abc123"}'), |
| 79 | + |
| 80 | + -- Gemini |
| 81 | + ('gemini', 'gemini:AIzaSy...'), |
| 82 | + |
| 83 | + -- Anthropic |
| 84 | + ('claude', '{"provider": "anthropic", "api_key": "sk-ant-..."}'), |
| 85 | + |
| 86 | + -- Local models (no key needed) |
| 87 | + ('local-llama', 'ollama::llama2'), |
| 88 | + ('local-nomic', 'ollama::nomic-embed-text'); |
| 89 | +``` |
| 90 | + |
| 91 | +### Dynamic Key Management |
| 92 | +```sql |
| 93 | +-- Create a table to store API keys |
| 94 | +CREATE TABLE api_keys ( |
| 95 | + provider TEXT PRIMARY KEY, |
| 96 | + key TEXT NOT NULL, |
| 97 | + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP |
| 98 | +); |
| 99 | + |
| 100 | +-- Store keys securely |
| 101 | +INSERT INTO api_keys (provider, key) VALUES |
| 102 | + ('openai', 'sk-proj-...'), |
| 103 | + ('gemini', 'AIza...'); |
| 104 | + |
| 105 | +-- Register clients using stored keys |
| 106 | +INSERT INTO temp.rembed_clients(name, options) |
| 107 | +SELECT |
| 108 | + provider || '-client', |
| 109 | + provider || ':' || key |
| 110 | +FROM api_keys; |
| 111 | +``` |
| 112 | + |
| 113 | +## 🔒 Security Considerations |
| 114 | + |
| 115 | +### Development vs Production |
| 116 | + |
| 117 | +**Development** - API keys in SQL are convenient: |
| 118 | +```sql |
| 119 | +-- Quick testing with inline keys |
| 120 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 121 | + ('test', 'openai:sk-test-key'); |
| 122 | +``` |
| 123 | + |
| 124 | +**Production** - Use environment variables: |
| 125 | +```bash |
| 126 | +# Set in environment |
| 127 | +export OPENAI_API_KEY="sk-proj-production-key" |
| 128 | +``` |
| 129 | + |
| 130 | +```sql |
| 131 | +-- Reference without exposing key |
| 132 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 133 | + ('prod', 'openai::text-embedding-3-small'); |
| 134 | +``` |
| 135 | + |
| 136 | +### Best Practices |
| 137 | + |
| 138 | +1. **Never commit API keys** to version control |
| 139 | +2. **Use environment variables** in production |
| 140 | +3. **Rotate keys regularly** |
| 141 | +4. **Use restricted keys** when possible (limited scope/permissions) |
| 142 | +5. **Store keys encrypted** if persisting in database |
| 143 | + |
| 144 | +## 🎨 Provider-Specific Formats |
| 145 | + |
| 146 | +| Provider | Simple Format | Environment Variable | |
| 147 | +|----------|--------------|---------------------| |
| 148 | +| OpenAI | `openai:sk-proj-...` | `OPENAI_API_KEY` | |
| 149 | +| Gemini | `gemini:AIza...` | `GEMINI_API_KEY` | |
| 150 | +| Anthropic | `anthropic:sk-ant-...` | `ANTHROPIC_API_KEY` | |
| 151 | +| Groq | `groq:gsk_...` | `GROQ_API_KEY` | |
| 152 | +| Cohere | `cohere:co-...` | `CO_API_KEY` | |
| 153 | +| DeepSeek | `deepseek:sk-...` | `DEEPSEEK_API_KEY` | |
| 154 | +| Mistral | `mistral:...` | `MISTRAL_API_KEY` | |
| 155 | +| Ollama | `ollama::model` | None (local) | |
| 156 | + |
| 157 | +## 🚀 Quick Start |
| 158 | + |
| 159 | +```sql |
| 160 | +-- Load the extension |
| 161 | +.load ./rembed0 |
| 162 | + |
| 163 | +-- Register OpenAI with inline key (development) |
| 164 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 165 | + ('embedder', 'openai:sk-proj-your-key-here'); |
| 166 | + |
| 167 | +-- Generate embeddings |
| 168 | +SELECT length(rembed('embedder', 'Hello, world!')); |
| 169 | + |
| 170 | +-- Register multiple providers |
| 171 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 172 | + ('fast', 'openai:sk-proj-key1'), |
| 173 | + ('accurate', '{"model": "openai::text-embedding-3-large", "key": "sk-proj-key1"}'), |
| 174 | + ('free', 'ollama::nomic-embed-text'); |
| 175 | + |
| 176 | +-- Use different models |
| 177 | +SELECT rembed('fast', 'Quick embedding'); |
| 178 | +SELECT rembed('accurate', 'Precise embedding'); |
| 179 | +SELECT rembed('free', 'Local embedding'); |
| 180 | +``` |
| 181 | + |
| 182 | +## 🎭 Migration from Environment Variables |
| 183 | + |
| 184 | +If you're currently using environment variables and want to switch to SQL-based keys: |
| 185 | + |
| 186 | +```sql |
| 187 | +-- Before (requires OPENAI_API_KEY env var) |
| 188 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 189 | + ('my-client', 'openai'); |
| 190 | + |
| 191 | +-- After (self-contained) |
| 192 | +INSERT INTO temp.rembed_clients(name, options) VALUES |
| 193 | + ('my-client', 'openai:sk-proj-your-key-here'); |
| 194 | +``` |
| 195 | + |
| 196 | +Both methods continue to work, giving you flexibility in deployment! |
0 commit comments