[P] Open source LLM gateway in Rust looking for feedback and contributors
A Rust-based gateway handles retries, caching, and PII redaction for multiple AI providers.
Deep Dive
Developers have built Sentinel, an open-source LLM gateway written in Rust. It provides a single OpenAI-compatible endpoint to route requests to providers like OpenAI and Anthropic. Key features include automatic failover, exponential backoff retries, exact-match caching, PII redaction, and SQLite audit logging with cost tracking. Users can deploy it locally to simplify managing multiple LLM APIs in production applications, replacing custom glue code.
Why It Matters
It reduces boilerplate code for production AI apps, letting teams focus on core logic instead of API management.