Ballast adds circuit breakers, backpressure, and cost-aware fallbacks to any AI agent pipeline — and chaos testing to prove they work. One Python decorator. No orchestrator rewrite.
Multi-agent systems ship with almost none of the hardening microservices learned over 15 years. The failure modes are old; the ecosystem is new.
One degraded tool makes every agent retry. Retries exhaust connections, budgets, and patience — then the whole pipeline fails, not just the sick dependency.
A task spawns 50 agents; 50 agents hit one API at once. Nothing queues, nothing sheds, nothing throttles — until the API does it for you, with 429s and bans.
Every retry is a token bill. Without cost-aware routing, an incident isn't just downtime — it's the most expensive hour of your month.
Decorate any call — a tool, an LLM API, a database query. That's the entire
integration. Sync or async def, auto-detected. Works with LangGraph,
CrewAI, AutoGen, or a plain agent loop.
from ballast import guarded @guarded(dependency="openai_api", timeout=10) async def call_llm(prompt: str) -> str: return await client.chat.completions.create(...)
Every dependency gets a circuit breaker: a rolling window of failures and latency versus a learned healthy baseline. When an API degrades, the breaker trips in milliseconds — and stops your swarm from hammering a dying service.
breaker_trip openai_api failure_rate=0.55, cooldown=10s breaker_half_open openai_api probing recovery… breaker_close openai_api probes_succeeded
While the breaker is open, calls don't fail — they walk a fallback chain: a cached answer, a cheaper model when a classifier says it's safe, or a static response. Your pipeline keeps moving, and it costs less during the incident, not more.
@guarded( dependency="openai_api", cache_ttl_s=300, # rung 1: recent identical answer cheaper=call_small_model, # rung 2: downgrade simple prompts fallback="degraded", # rung 3: static last resort cost_fn=lambda r: r.cost, # meter spend vs. your budget ) def call_llm(prompt: str) -> str: ...
Trip on failure rate or p95 latency vs. a learned healthy baseline — one that excludes anomalies, so a degrading dependency can't drag its own baseline up and mask the degradation. Half-open probing, exponential-backoff cooldowns, manual overrides.
A concurrency ceiling with strict-FIFO queueing — no barging, sync and async in the same queue. Excess load queues, then sheds. Your system degrades instead of collapsing.
Tried in order, automatically, when a call can't reach its dependency:
Rolling-hour budgets with soft and hard ceilings. Past the soft ceiling, simple requests route to a cheaper model automatically — a rules-based, explainable classifier decides, and it's pluggable.
Every trip, reroute, shed, budget crossing, and chaos event flows through one bus into SQLite — queryable, exportable, and streamed live to the dashboard.
Everyone says their pipeline survives an outage. Ballast lets you kill a dependency on purpose — in dev, time-boxed, against the real code path — and watch the system detect, reroute, and recover. This is the actual output:
Breaker cards with cooldown countdowns, traffic and cost-vs-budget charts, and a coalescing event feed — streamed over WebSocket. This is a real recorded incident from the built-in demo (chaos at 0:08, recovery at 0:17):
docker compose up → localhost:8080,
or python -m ballast.dashboard. The demo needs no API keys.
Answer five questions; copy the result into your project. That's the whole integration.
Gateways proxy your LLM traffic and can retry or reroute API requests. Ballast lives in-process with your orchestrator, so it can do what a proxy can't: throttle agent fan-out with backpressure, guard non-LLM dependencies (databases, tools), and chaos-test the whole pipeline. They compose — many teams will want both.
A lock-protected counter check (backpressure), a breaker state read, and a window append per call — microseconds, entirely in-process, no network hop. With chaos disabled the injector is a plain pass-through.
Yes. @guarded detects async def functions automatically —
backpressure waits without blocking the event loop, fallbacks can be async
callables, and timeout becomes a real cancelling timeout
(unlike the sync flavor, where a thread can't be safely killed).
aguard() is the async context-manager form.
It's a well-tested v1 (96 deterministic tests, CI on Linux and Windows across Python 3.11–3.13) that's honest about its limits: single-process in-memory state and a localhost dashboard with no auth. Read the security section in the README before deploying anything.
Install it, run the built-in incident demo, and watch a breaker trip and recover in your browser — no API keys required.