Ballast adds circuit breakers, backpressure, and cost-aware fallbacks to any AI agent pipeline — and chaos testing to prove they work. One Python decorator. No orchestrator rewrite.
Multi-agent systems ship with almost none of the hardening microservices learned over 15 years. The failure modes are old; the ecosystem is new.
One degraded tool makes every agent retry. Retries exhaust connections, budgets, and patience — then the whole pipeline fails, not just the sick dependency.
A task spawns 50 agents; 50 agents hit one API at once. Nothing queues, nothing sheds, nothing throttles — until the API does it for you, with 429s and bans.
Every retry is a token bill. Without cost-aware routing, an incident isn't just downtime — it's the most expensive hour of your month.
Decorate any call — a tool, an LLM API, a database query. That's the entire integration. Works with LangGraph, CrewAI, AutoGen, or a plain agent loop.
from ballast import guarded @guarded(dependency="openai_api", timeout=10) def call_llm(prompt: str) -> str: return client.chat.completions.create(...)
Every dependency gets a circuit breaker: a rolling window of failures and latency versus a learned healthy baseline. When an API degrades, the breaker trips in milliseconds — and stops your swarm from hammering a dying service.
# the event stream, live: breaker_trip openai_api failure_rate=0.55, cooldown=10s breaker_half_open openai_api probing recovery… breaker_close openai_api probes_succeeded
While the breaker is open, calls don't fail — they walk a fallback chain: a cached answer, a cheaper model (when a rules-based classifier says it's safe), or a static response. Your pipeline keeps moving, and it costs less during the incident, not more.
@guarded( dependency="openai_api", cache_ttl_s=300, # rung 1: recent identical answer cheaper=call_small_model, # rung 2: downgrade simple prompts fallback="degraded", # rung 3: static last resort cost_fn=lambda r: r.cost, # meter spend vs. your budget ) def call_llm(prompt: str) -> str: ...
Trip on failure rate or p95 latency vs. a learned baseline. Half-open probing, exponential-backoff cooldowns, manual overrides.
A concurrency ceiling with strict-FIFO queueing. Excess load queues, then sheds — your system degrades instead of collapsing.
Cache → cheaper model → static value → explicit error. Configured per dependency, served automatically when the breaker opens.
Rolling-hour budgets with soft and hard ceilings. Past the soft ceiling, simple requests route to a cheaper model automatically.
Time-boxed failure, latency, and corruption rules against your real code path. Your resilience demos double as CI tests.
Breaker states, traffic and cost charts, a coalescing event feed, and a one-click built-in incident demo — over WebSocket, on localhost.
Everyone says their pipeline survives an outage. Ballast lets you kill a dependency on purpose — in dev, time-boxed, against the real code path — and watch the system detect, reroute, and recover. This is the actual output:
Breaker cards with cooldown countdowns, traffic and cost-vs-budget charts, and a coalescing event feed — streamed over WebSocket. This is a real recorded incident from the built-in demo (chaos at 0:08, recovery at 0:17):
docker compose up → localhost:8080,
or python -m ballast.dashboard. The demo needs no API keys.
Answer five questions; copy the result into your project. That's the whole integration.
Gateways proxy your LLM traffic and can retry or reroute API requests. Ballast lives in-process with your orchestrator, so it can do what a proxy can't: throttle agent fan-out with backpressure, guard non-LLM dependencies (databases, tools), and chaos-test the whole pipeline. They compose — many teams will want both.
A lock-protected counter check (backpressure), a breaker state read, and a window append per call — microseconds, entirely in-process, no network hop. With chaos disabled the injector is a plain pass-through.
Yes. @guarded detects async def functions automatically —
backpressure waits without blocking the event loop, fallbacks can be async
callables, and timeout becomes a real cancelling timeout
(unlike the sync flavor, where a thread can't be safely killed).
aguard() is the async context-manager form.
It's a well-tested v1 (96 deterministic tests, CI on Linux and Windows across Python 3.11–3.13) that's honest about its limits: single-process in-memory state and a localhost dashboard with no auth. Read the security section in the README before deploying anything.
Clone it, run the built-in incident demo, and watch a breaker trip and recover in your browser — no API keys required.
$ pip install "ballast-agents[dashboard] @ git+https://github.com/s3ak6i-dev/Ballast" && python -m ballast.dashboard