Free AI Gateway™ routes Groq, Gemini, and OpenRouter free tiers through a single high-availability proxy — with local SQLite quota accounting, automatic failover, streaming, and Prometheus metrics.
Drop-in OpenAI-compatible base URL. Atomic quota reservations. Full-jitter backoff. Production health probes. Zero telemetry by default.
Groq → Gemini → OpenRouter with circular fallbacks. Transient errors retry with Retry-After honouring.
SQLite WAL tracks RPM + RPD per provider. BEGIN IMMEDIATE prevents overshoot under concurrency.
Sync and asyncio clients with stream support. Same API surface as OpenAI SDK.
Prometheus /metrics, liveliness / readiness / startup probes, deep upstream checks.
Master-key required. Non-root Docker, systemd hardening, metrics locked to private nets.
Ship-ready tool-using agent that routes every call through the free gateway.
Model names are gateway aliases. Upstream free limits are tracked locally so you stay inside the envelope.
| Gateway model | Upstream | RPM | RPD |
|---|---|---|---|
free-llm-router |
Groq · llama-3.3-70b-versatile | 30 | 1,000 |
free-llm-router-fallback-1 |
Gemini · gemini-2.5-flash-lite | ~15 | ~1,000 |
free-llm-router-fallback-2 |
OpenRouter · llama-3.3-70b:free | 20 | 50 / 1,000* |
* OpenRouter free RPD rises to 1,000 after a $10 credit purchase.
# clone & bootstrap git clone https://github.com/Mourad-Soltani/free-ai-gateway cd free-ai-gateway && ./install.sh # add keys + generate master key nano .env python -c "import secrets; print('sk-' + secrets.token_urlsafe(32))" # launch proxy on :4000 ./run.sh # query or run the agent python client.py "Explain SQLite WAL in two bullets" python agents/agent.py --task "Summarise today's free-tier status"
The included agent uses the Free AI Gateway for every LLM call. It supports multi-step reasoning, quota-aware routing, and simple tools (time, stats, calculator).
Live mode calls /v1/chat/completions on your gateway (CORS must allow this origin).
Local default: start with ./run.sh, connect to http://127.0.0.1:4000.
CLI agent: python agents/agent.py --interactive
The software is MIT-licensed. Trademarks remain with Mourad Soltani Technologies™ @MST. Support and turnkey setup are available.
Self-host the open core
Priority support · 48h SLA
One-time install & hardening
Clone the repo, drop in three free API keys, and point your OpenAI SDK at localhost:4000.
Star & clone Free AI Gateway →