Overview
AI Circuit Breaker is an LLM cost firewall for autonomous agents. The drop-in reverse proxy inspects requests in volatile memory, detects repeated prompt patterns, and checks configured spend ceilings before forwarding traffic to an upstream provider.
Key Features
- Stops recursive tool loops when a prompt fingerprint crosses the repetition threshold.
- Enforces hourly and daily account spending caps before upstream requests are sent.
- Supports HTTP/2 and server-sent event traffic through an OpenAI-compatible endpoint.
- Returns machine-readable errors for detected loops, exceeded budgets, and upstream failures.
- Targets less than 35 milliseconds of proxy overhead, with actual performance dependent on deployment and network conditions.
Setup and Compatibility
Developers create a protected key, replace their client's OpenAI base endpoint, and use that key for gateway authentication. Requests use the familiar chat completions format, so existing SDK-based applications can retain their normal request structure.
The proxy works with Python and TypeScript OpenAI clients, direct REST requests, and frameworks that accept a custom OpenAI-compatible endpoint. Documented examples include CrewAI, LangChain, LangGraph, and AutoGen.
Security and Data Handling
Prompt bodies are processed in memory for loop detection and are not written to disk. Usage and operational metadata are logged without prompt text, while upstream credentials are encrypted at rest with AES-256 using Fernet. Protected keys are hashed with SHA-256 for lookup, and full protected secrets are shown once.
Pricing and Limits
The Developer plan is free and monitors up to $15 per month of spend, with three-request sliding-window loop detection plus hourly and daily caps. The Pro plan costs $29 per month and adds unlimited monitored spend, $50 hourly and $200 daily burst ceilings, emergency Slack and Discord webhooks, priority European routing, and configurable production ceilings according to plan.