How to run an always-on agent for near zero

My agent went down in the small hours and stayed down for the rest of the night, and the annoying part is that it did exactly what I had configured. The model it was holding had been retired weeks earlier. The provider was up the whole time — 404 on the model ID, 200 on the endpoint — but my client read every non-200 as transient, backed off, retried, backed off, twelve times per turn, turn after turn, until it hit the restart limit. Nothing was on fire. The chain was pointing at a model that no longer existed, and the failover logic had no way to tell "this will work in a minute" from "this will never work again."

What follows is the shape the setup has now. Most of it started as notes I wrote up on dev.to — the "7 things that actually work" post is the longer version of this one.

A scout that never calls a model

Once every two hours, a cron job walks every free route I hold a key for. It doesn't ask whether the provider is up; that question is almost never the interesting one. It asks whether the route can call tools — a route that happily completes prose but refuses a tool schema is dead weight to an agent that has to actually do things — and then it ranks what answered and writes the result to a map file. No LLM in that loop. It's a curl script with a bit of plumbing around it.

Every turn after that, the agent picks from the top of the map rather than from a hardcoded primary. That's the whole trick of the architecture: the thing that decides where traffic goes is a file that a dumb script rewrote recently, not a model I trusted months ago.

The reason it's a scheduled probe and not a table I maintain by hand is that free tier lists rot. A provider announces a generous free tier, and by the time I've wired it into anything the model behind it has been retired or the rate limit is nowhere near what the announcement promised. Any list I copy into a config file is stale the moment I commit it. The map is only ever as good as the last probe.

Free legs in front, one paid leg at the end

The chain builder reads that map and writes the fallback order: best live free route first, next-best after that, and paid legs only at the very end.

chain:
  - free-route-1   # whatever ranked first two hours ago
  - free-route-2
  - free-route-3
  - paid-route     # last resort; logged every time it fires

The paid leg is the safety net for the few turns a day that actually matter, not the default path. What I watch is the log of when it fires: if it's firing all day, my free routes are worse than the map claims, and something in the ranking needs another look.

Retired is not the same as dead

That night's outage was a classification problem, not an availability problem. The provider was up. The model was gone. My client was retrying a permanent condition as though it were temporary.

The fix is a small plugin in the failover hook that classifies errors before they reach the retry logic. "Model retired", "free offer ended", "that model ID no longer exists" — all tagged do-not-retry, so the chain moves to the next leg on the first attempt instead of burning its budget on a door that has been walled up. It reads as obvious written down. It was not obvious at the time, because most client libraries treat every non-200 the same way.

After that I re-probed every free API I could get a key for, and the result still shapes how I think about this: a retired model is almost never a dead provider. Same endpoint, same auth, one rotated model string away from working again. Which is why the scout tests routes, not vendors.

Retries are a budget

I keep the retry budget at twelve per turn. That's a setting, not a measurement, and it took me a while to stop treating it as a fact about the world. Every retry a retired model eats is a retry the next model in the chain doesn't get. Set it too low and a busy afternoon becomes restart-limit-exceeded for something a single retry would have fixed. Set it too high and one bad leg can consume an entire turn before the chain moves on.

A restart is the most expensive failure mode I have, so reloads are drain-aware — SIGUSR1, let the turn in flight finish, swap the config underneath, keep going. Rebuilding state I already had is the thing I most want to avoid doing at the moment I'm least awake.

The whole thing is cron, a map file, and a plugin short enough to read in one sitting. Inference was never the expensive part. My assumptions about what "the model is down" means were.

I run my own chains through https://cleosnine.com/r — that's a referral link, and I'd rather say that plainly than dress it up as a discovery.