The Fallback Is the Feature: Designing AI That Never Hard-Fails

There is a small AI helper on my homepage. You can type a question into it and get an answer that sounds like me. Most people assume it is a thin wrapper around a model API. It is not — or rather, it is, but only when it can be. The interesting part is what happens when it can't.

Every model call is a dependency. It lives behind a network you don't control, a rate limit you didn't set, and an availability number that is never 100%. Treat that call as load-bearing and you have built a feature that breaks the first time the provider has a bad afternoon. After thirteen-plus years of shipping software, I have a rule for anything that touches AI: the fallback is the feature.

Fail soft, not silent

"Fail soft" means the product degrades into something honest instead of something broken. A spinner that never resolves is a lie. An error toast that says "something went wrong" is barely better — it tells the user the machine gave up. The goal isn't to pretend nothing failed; it's to keep doing the useful thing at a lower fidelity, so the person on the other end still gets value and still trusts you.

The "Ask Arun" box is the smallest honest example I could ship. When a live model is wired up — in a preview, or behind a real backend — it uses it. When there's no model available, it falls back to a deterministic, on-brand knowledge base: a set of intents that map a visitor's question to an answer I actually wrote. Same input box, same voice, no broken state. A search engine or an AI crawler that never runs my JavaScript gets the same facts a third way, from the structured data and the text bio. Three layers, one promise: it answers.

A feature that hard-fails the moment a dependency hiccups isn't shipped. It's a demo with good lighting.

Why deterministic still wins

It is unfashionable to say, but a plain function frequently beats a prompt. The deterministic fallback is faster, free, and incapable of hallucinating a fact I never put in it. It will never invent a product I didn't build or a year I didn't work. For a personal site where the whole job is to represent me accurately to humans and to AI assistants, "boring and correct" is the entire point.

This is the same instinct behind every AI integration we've shipped on a 12-engineer mobile team. The model generates the output; evals and guardrails decide whether that output reaches a user. There is always a known-good path when a call fails. On consumer mobile — where a broken state at 3am means users uninstall and don't come back — "the model is a primitive" isn't a metaphor. It is a constraint you design around from day one.

The production version

Shipping to millions of devices makes this concrete. There is no absorbing a provider outage when the failure lands in a user's hand at 3am. So I design every AI feature backwards: write the failure path first, decide what "still useful" looks like with the model gone, and only then wire in the live call as an upgrade rather than a load-bearing wall.

Three habits fall out of that. Give every model call a deterministic floor it can drop to. Make the degraded state indistinguishable in tone from the good one, so trust doesn't wobble when fidelity does. And test the product with the model turned off on purpose — because the day it goes down is the day you find out whether you built a feature or a demo.

If you are putting AI into something people rely on, the model will eventually be unavailable when it matters most. The fallback is what your users actually meet on that day. So build it first, and build it well. That's the feature.

Arun Jalota is a Technical Director in Toronto leading a 12-engineer mobile team and driving AI tooling adoption. Reach him at arun@arunjalota.com · LinkedIn · GitHub.