|
| 1 | +# LeetCode Automation Strategy with Azure OpenAI |
| 2 | + |
| 3 | +## Overview |
| 4 | +This document outlines the strategy for automating the generation of the remaining ~3,500 LeetCode problems using Azure OpenAI API integrated with Claude Code. |
| 5 | + |
| 6 | +## Architecture |
| 7 | + |
| 8 | +### 1. **Hybrid AI Approach** |
| 9 | +- **Claude Code**: Orchestration, validation, and quality control |
| 10 | +- **Azure OpenAI**: Mass code generation at scale |
| 11 | +- **Division of Labor**: |
| 12 | + - Claude handles complex problems requiring deep reasoning |
| 13 | + - Azure GPT-4 handles standard algorithmic implementations |
| 14 | + - Both systems cross-validate each other's work |
| 15 | + |
| 16 | +### 2. **Component Structure** |
| 17 | + |
| 18 | +``` |
| 19 | +rust-leetcode/ |
| 20 | +├── src/ |
| 21 | +│ ├── automation/ |
| 22 | +│ │ ├── azure_client.rs # Azure OpenAI API integration |
| 23 | +│ │ ├── generator.rs # Problem generation pipeline |
| 24 | +│ │ ├── validator.rs # Code validation & testing |
| 25 | +│ │ └── orchestrator.rs # Workflow coordination |
| 26 | +│ └── bin/ |
| 27 | +│ └── generate_problems.rs # CLI tool |
| 28 | +``` |
| 29 | + |
| 30 | +## Implementation Pipeline |
| 31 | + |
| 32 | +### Phase 1: Infrastructure Setup (Completed ✅) |
| 33 | +1. Azure OpenAI client with rate limiting |
| 34 | +2. Problem generation pipeline |
| 35 | +3. CLI tool for batch processing |
| 36 | +4. Validation framework |
| 37 | + |
| 38 | +### Phase 2: Template Extraction |
| 39 | +```rust |
| 40 | +// Extract patterns from existing 150 problems |
| 41 | +let templates = TemplateExtractor::analyze_existing_solutions(); |
| 42 | +// Categories: DP, Graph, Tree, LinkedList, etc. |
| 43 | +``` |
| 44 | + |
| 45 | +### Phase 3: Batch Generation Strategy |
| 46 | + |
| 47 | +#### Prioritization Algorithm |
| 48 | +```rust |
| 49 | +Priority = (Frequency × Difficulty_Weight × Company_Score) / Implementation_Complexity |
| 50 | + |
| 51 | +Where: |
| 52 | +- Frequency: How often asked in interviews |
| 53 | +- Difficulty_Weight: Easy=1, Medium=2, Hard=3 |
| 54 | +- Company_Score: FAANG=3, Top-tier=2, Others=1 |
| 55 | +- Implementation_Complexity: Estimated LOC / 100 |
| 56 | +``` |
| 57 | + |
| 58 | +#### Batch Processing Schedule |
| 59 | +- **Batch Size**: 50 problems per run |
| 60 | +- **Daily Target**: 200 problems |
| 61 | +- **Validation**: Every 10 problems |
| 62 | +- **Full Test Suite**: Every 50 problems |
| 63 | + |
| 64 | +### Phase 4: Quality Assurance |
| 65 | + |
| 66 | +#### Multi-Layer Validation |
| 67 | +1. **Syntax Check**: Rust compiler |
| 68 | +2. **Logic Validation**: Test cases pass |
| 69 | +3. **Performance Check**: Complexity requirements met |
| 70 | +4. **Style Consistency**: Format and idioms |
| 71 | +5. **Cross-Validation**: Multiple AI solutions compared |
| 72 | + |
| 73 | +## Azure OpenAI Configuration |
| 74 | + |
| 75 | +### Optimal Settings |
| 76 | +```json |
| 77 | +{ |
| 78 | + "model": "gpt-4-turbo", |
| 79 | + "temperature": 0.3, |
| 80 | + "max_tokens": 4000, |
| 81 | + "top_p": 0.95, |
| 82 | + "frequency_penalty": 0.0, |
| 83 | + "presence_penalty": 0.0, |
| 84 | + "system_prompt": "Expert Rust programmer creating LeetCode solutions..." |
| 85 | +} |
| 86 | +``` |
| 87 | + |
| 88 | +### Rate Limiting Strategy |
| 89 | +- **Requests/minute**: 10 (adjustable based on tier) |
| 90 | +- **Tokens/minute**: 40,000 |
| 91 | +- **Retry logic**: Exponential backoff with jitter |
| 92 | +- **Circuit breaker**: After 5 consecutive failures |
| 93 | + |
| 94 | +## Execution Commands |
| 95 | + |
| 96 | +### Basic Generation |
| 97 | +```bash |
| 98 | +# Generate 10 easy problems |
| 99 | +cargo run --bin generate_problems -- --batch-size 10 --difficulty easy |
| 100 | + |
| 101 | +# Generate medium problems with specific topics |
| 102 | +cargo run --bin generate_problems -- --batch-size 20 --difficulty medium --topics "Dynamic Programming,Graph" |
| 103 | + |
| 104 | +# Continue from specific problem ID |
| 105 | +cargo run --bin generate_problems -- --start-from 500 --batch-size 50 |
| 106 | +``` |
| 107 | + |
| 108 | +### Advanced Automation |
| 109 | +```bash |
| 110 | +# Run with parallel processing |
| 111 | +cargo run --bin generate_problems -- --parallel --max-concurrent 5 --batch-size 100 |
| 112 | + |
| 113 | +# Dry run to see what would be generated |
| 114 | +cargo run --bin generate_problems -- --dry-run --batch-size 50 |
| 115 | + |
| 116 | +# Generate without validation (faster, less safe) |
| 117 | +cargo run --bin generate_problems -- --no-validate --batch-size 100 |
| 118 | +``` |
| 119 | + |
| 120 | +## Cost Optimization |
| 121 | + |
| 122 | +### Token Usage Estimation |
| 123 | +- **Average problem**: ~2,000 input tokens + ~2,000 output tokens |
| 124 | +- **Cost per problem**: ~$0.12 (GPT-4 pricing) |
| 125 | +- **Total estimated cost**: ~$420 for 3,500 problems |
| 126 | + |
| 127 | +### Optimization Strategies |
| 128 | +1. **Caching**: Store problem descriptions locally |
| 129 | +2. **Batching**: Group similar problems |
| 130 | +3. **Template Reuse**: Use generated patterns for similar problems |
| 131 | +4. **Incremental Generation**: Start with main solution, add alternatives later |
| 132 | + |
| 133 | +## Monitoring & Metrics |
| 134 | + |
| 135 | +### Key Metrics to Track |
| 136 | +```rust |
| 137 | +struct GenerationMetrics { |
| 138 | + problems_generated: usize, |
| 139 | + success_rate: f64, |
| 140 | + avg_generation_time: Duration, |
| 141 | + avg_tokens_used: usize, |
| 142 | + test_pass_rate: f64, |
| 143 | + compilation_success_rate: f64, |
| 144 | +} |
| 145 | +``` |
| 146 | + |
| 147 | +### Dashboard Output |
| 148 | +``` |
| 149 | +=== Generation Progress === |
| 150 | +Total Problems: 3,661 |
| 151 | +Generated: 150 (4.1%) |
| 152 | +Remaining: 3,511 |
| 153 | +Today's Progress: 45/200 (22.5%) |
| 154 | +Success Rate: 94.3% |
| 155 | +Est. Completion: 18 days |
| 156 | +``` |
| 157 | + |
| 158 | +## Error Handling & Recovery |
| 159 | + |
| 160 | +### Common Issues & Solutions |
| 161 | + |
| 162 | +1. **API Rate Limits** |
| 163 | + - Solution: Automatic backoff and queue management |
| 164 | + - Fallback: Switch to backup API key or different region |
| 165 | + |
| 166 | +2. **Code Compilation Failures** |
| 167 | + - Solution: Retry with adjusted prompt |
| 168 | + - Fallback: Mark for manual review |
| 169 | + |
| 170 | +3. **Test Failures** |
| 171 | + - Solution: Regenerate with test cases in prompt |
| 172 | + - Fallback: Generate alternative approach |
| 173 | + |
| 174 | +4. **Token Limit Exceeded** |
| 175 | + - Solution: Split problem into smaller parts |
| 176 | + - Fallback: Use simpler implementation |
| 177 | + |
| 178 | +## Parallel Execution with Claude Code |
| 179 | + |
| 180 | +### Orchestration Strategy |
| 181 | +```bash |
| 182 | +# Terminal 1: Claude Code generates complex problems |
| 183 | +claude-code generate --type complex --batch 10 |
| 184 | + |
| 185 | +# Terminal 2: Azure handles standard problems |
| 186 | +cargo run --bin generate_problems -- --difficulty medium --batch-size 50 |
| 187 | + |
| 188 | +# Terminal 3: Validation pipeline |
| 189 | +cargo watch -x "test --lib" |
| 190 | +``` |
| 191 | + |
| 192 | +### Integration Points |
| 193 | +1. **Shared Database**: Track progress across both systems |
| 194 | +2. **Message Queue**: Coordinate work distribution |
| 195 | +3. **Git Hooks**: Auto-validate on commit |
| 196 | +4. **CI/CD**: GitHub Actions for continuous validation |
| 197 | + |
| 198 | +## Scaling Strategy |
| 199 | + |
| 200 | +### Progressive Automation Levels |
| 201 | + |
| 202 | +#### Level 1: Semi-Automated (Current) |
| 203 | +- Manual triggering |
| 204 | +- Batch generation |
| 205 | +- Manual review |
| 206 | + |
| 207 | +#### Level 2: Scheduled Automation |
| 208 | +```yaml |
| 209 | +# .github/workflows/generate.yml |
| 210 | +schedule: |
| 211 | + - cron: '0 */6 * * *' # Every 6 hours |
| 212 | +``` |
| 213 | +
|
| 214 | +#### Level 3: Fully Automated |
| 215 | +- Continuous generation |
| 216 | +- Auto-PR creation |
| 217 | +- Automated merging after tests pass |
| 218 | +
|
| 219 | +## Success Criteria |
| 220 | +
|
| 221 | +### Completion Targets |
| 222 | +- **Week 1**: 500 problems (Easy focus) |
| 223 | +- **Week 2**: 1000 problems (Medium focus) |
| 224 | +- **Week 3**: 1000 problems (Mixed difficulties) |
| 225 | +- **Week 4**: 1000 problems (Hard focus) |
| 226 | +- **Week 5**: Remaining + optimizations |
| 227 | +
|
| 228 | +### Quality Metrics |
| 229 | +- **Compilation Rate**: >95% |
| 230 | +- **Test Pass Rate**: >90% |
| 231 | +- **Performance**: Within O(n) of optimal |
| 232 | +- **Code Quality**: Idiomatic Rust score >8/10 |
| 233 | +
|
| 234 | +## Next Steps |
| 235 | +
|
| 236 | +1. **Set up Azure credentials**: |
| 237 | + ```bash |
| 238 | + export AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com |
| 239 | + export AZURE_OPENAI_API_KEY=your-api-key |
| 240 | + export AZURE_OPENAI_DEPLOYMENT=gpt-4 |
| 241 | + ``` |
| 242 | + |
| 243 | +2. **Run initial test batch**: |
| 244 | + ```bash |
| 245 | + cargo run --bin generate_problems -- --batch-size 5 --dry-run |
| 246 | + ``` |
| 247 | + |
| 248 | +3. **Start generation**: |
| 249 | + ```bash |
| 250 | + cargo run --bin generate_problems -- --batch-size 50 --difficulty easy |
| 251 | + ``` |
| 252 | + |
| 253 | +4. **Monitor progress**: |
| 254 | + ```bash |
| 255 | + watch -n 60 'cargo test --lib 2>&1 | grep "test result"' |
| 256 | + ``` |
| 257 | + |
| 258 | +## Risk Mitigation |
| 259 | + |
| 260 | +### Potential Risks |
| 261 | +1. **API Costs**: Implement spending limits |
| 262 | +2. **Code Quality**: Multiple validation layers |
| 263 | +3. **Duplicate Problems**: Hash-based detection |
| 264 | +4. **Rate Limiting**: Distributed API keys |
| 265 | +5. **Model Hallucinations**: Test-driven validation |
| 266 | + |
| 267 | +### Backup Plans |
| 268 | +1. Use GPT-3.5 for simpler problems |
| 269 | +2. Leverage open-source models (CodeLlama) |
| 270 | +3. Community contributions for complex problems |
| 271 | +4. Manual implementation for critical problems |
| 272 | + |
| 273 | +## Conclusion |
| 274 | + |
| 275 | +This automation strategy can generate all 3,500+ remaining LeetCode problems in approximately 3-4 weeks with proper orchestration between Claude Code and Azure OpenAI. The hybrid approach ensures both quality and speed, with comprehensive validation at every step. |
| 276 | + |
| 277 | +**Estimated Timeline**: |
| 278 | +- Setup: 1 day |
| 279 | +- Testing: 2 days |
| 280 | +- Full generation: 18-25 days |
| 281 | +- Validation & cleanup: 3-5 days |
| 282 | + |
| 283 | +**Total: ~1 month to complete entire LeetCode problem set** |
0 commit comments