Infrastructure
Inference, models, fallback chain, and review automation. Each section has its own Save button — presets do not persist until saved.
read-only — power user or admin required to edit
inference-mode
Where reviews run. Local and hybrid use Ollama; cloud uses Groq (free) or OpenRouter — determined by the cloud model name you set.
Ollama compute profile
default-models
Local model (Ollama)
Ollama offline — type slug manually
Cloud primary (quality)
Used for large PRs, security/deep reviews, and sensitive paths. Set GROQ_API_KEY and/or OPENROUTER_API_KEY on API
Cloud fast (volume)
Small/medium routine PRs when cloud fast routing is on (≤ threshold lines).
Recommended fast: llama-3.1-8b-instant · toggle in system-config
cloud-fallback
On Groq HTTP 429, try the next cloud model in the chain. Each entry is a Groq or OpenRouter slug — not Ollama. Use a preset, edit the chain, then Save section below.
Effective order on 429
- 1. Picked model (primary or fast)
- 2. llama-3.1-8b-instant
- 3. gemma2-9b-it
llama-3.1-8b-instant → gemma2-9b-it
Fallback chain
Top 5 Groq models for rate limits
llama-3.1-8b-instant
Volume workhorse
- Why
- 14.4K RPD — highest request budget on free tier; fastest latency.
qwen/qwen3-32b
Balanced primary
- Why
- 60 RPM (2× burst), 500K TPD, better reasoning than 8B; use with 8B fallback.
gemma2-9b-it
Volume backup
- Why
- Same 14.4K RPD / 500K TPD as 8B — second step in fallback chain.
llama-3.3-70b-versatile
Security / deep only
- Why
- 12K TPM, best Llama quality — but only 1K RPD; never as global primary.
qwen/qwen3.6-27b
Medium PRs
- Why
- 8K TPM headroom when 6K TPM models choke on large chunks.
| Model | Role | Why |
|---|---|---|
| llama-3.1-8b-instant | Volume workhorse | 14.4K RPD — highest request budget on free tier; fastest latency. |
| qwen/qwen3-32b | Balanced primary | 60 RPM (2× burst), 500K TPD, better reasoning than 8B; use with 8B fallback. |
| gemma2-9b-it | Volume backup | Same 14.4K RPD / 500K TPD as 8B — second step in fallback chain. |
| llama-3.3-70b-versatile | Security / deep only | 12K TPM, best Llama quality — but only 1K RPD; never as global primary. |
| qwen/qwen3.6-27b | Medium PRs | 8K TPM headroom when 6K TPM models choke on large chunks. |
Qwen 32B vs Llama: Qwen = 60 RPM + better reasoning; Llama 8B = 14.4K RPD (best volume); 70B = security/deep only.
review-actions
Per-action model routing and prompts. Cloud tier picks fast vs primary by diff size (see Infrastructure). Tier auto uses inference mode and diff heuristics; force local or cloud to override. Edit prompts
ai-models
Live Ollama, Groq, and OpenRouter connectivity. Default models are set above.
Preferred:
Model:
Set GROQ_API_KEY on API
Model:
Per-action routing in the table below. Repo overrides in Routing rules.
model-catalog
Loading model catalog…
system-config
Chunk and routing settings affect Groq RPM/TPM usage. With many repos, keep cloud_fast_routing on and use 8B for fast tier.