When one policy change can shut down every AI workflow overnight, resilience stops being optional.
Lorem ipsum dolor sit amet, consectetur adipiscing elit lobortis arcu enim urna adipiscing praesent velit viverra sit semper lorem eu cursus vel hendrerit elementum morbi curabitur etiam nibh justo, lorem aliquet donec sed sit mi dignissim at ante massa mattis egestas.
Vitae congue eu consequat ac felis placerat vestibulum lectus mauris ultrices cursus sit amet dictum sit amet justo donec enim diam porttitor lacus luctus accumsan tortor posuere praesent tristique magna sit amet purus gravida quis blandit turpis.
At risus viverra adipiscing at in tellus integer feugiat nisl pretium fusce id velit ut tortor sagittis orci a scelerisque purus semper eget at lectus urna duis convallis porta nibh venenatis cras sed felis eget neque laoreet suspendisse interdum consectetur libero id faucibus nisl donec pretium vulputate sapien nec sagittis aliquam nunc lobortis mattis aliquam faucibus purus in.
Nisi quis eleifend quam adipiscing vitae aliquet bibendum enim facilisis gravida neque velit euismod in pellentesque massa placerat volutpat lacus laoreet non curabitur gravida odio aenean sed adipiscing diam donec adipiscing tristique risu amet est placerat in egestas erat imperdiet sed euismod nisi.
“Nisi quis eleifend quam adipiscing vitae aliquet bibendum enim facilisis gravida neque velit euismod in pellentesque massa placerat volutpat lacus.”
Eget lorem dolor sed viverra ipsum nunc aliquet bibendum felis donec et odio pellentesque diam volutpat commodo sed egestas aliquam sem fringilla ut morbi tincidunt augue interdum velit euismod eu tincidunt tortor aliquam nulla facilisi aenean sed adipiscing diam donec adipiscing ut lectus arcu bibendum at varius vel pharetra nibh venenatis cras sed felis eget.
By: Krish Garg, Mithilesh Gaurihar
Bottom line: the model, the tool, and the pricing plan can all move overnight. The only thing that doesn’t have to is the runtime layer above them. That’s what this post is about.
The US Commerce Department sent Anthropic a letter on a Friday evening. By midnight, Fable 5 and Mythos 5 were offline for every user on the planet. Not just foreign nationals, but everyone, because Anthropic had no way to verify citizenship at scale in real time. AWS Bedrock, Google Cloud, Microsoft Foundry, and the direct Claude API all went dark simultaneously. Teams woke up the next morning to broken pipelines and no warning.
Read Anthropic’s official statement →
Around the same time, GitHub Copilot ended flat-rate pricing and moved every plan to usage-based billing. The AI coding subsidy era ended with an email and a new line item on the bill.
Read the GitHub announcement →
And if you think that’s isolated to models and billing, SpaceX has agreed to acquire Cursor-maker Anysphere for $60 billion pending regulatory approval. Same pattern, different layer.
Different companies. Different layers. Same pattern: rented ground.
Most AI system engineering goes into the model layer. Prompt design, tool calling, eval pipelines, output parsing. All of it tuned against a specific model’s behavior. When that model changes, disappears, or gets repriced, the tuning work is either portable or it isn’t. In most production systems, it isn’t.
The layer that makes it portable is the runtime: model routing, observability, and cost tracking sitting above the model itself. Teams that built that layer had options on Friday night. Teams that didn’t were waiting for support tickets.
This isn’t a new idea. Perplexity’s Aravind Srinivas has argued that the model is not the product. Anthropic’s Boris Cherny has written about loop engineering: stop prompting agents directly and design the loops that prompt them. The durable artifact is the loop, not the model.
This news just made that expensive to ignore.
A model-agnostic runtime needs to do three things. Most teams skip all three until they are forced to deal with the consequences.
1. Treat the model as a configuration, not a dependency: Prompts, tool definitions, and output schemas should not assume a specific provider’s API shape. The moment your prompt is written for GPT-4’s output format, or your tool calling assumes Anthropic’s schema, you have a hard dependency masquerading as flexibility.
2. Span the entire run: The runtime needs to see every component, every model call, every handoff in a single trace. When you route across multiple models, the failure is almost never inside one model. It is in the handoff between them. A trace that only covers one provider tells you nothing useful about that failure.
3. Surface cost in real time: Not in a billing dashboard at the end of the month. As the run executes. When you pay per token, the decision of whether to commit a workload to an expensive model needs to happen before you commit it, not after.
Most teams get none of these right because they build the model integration first and the runtime later. Here is what all three look like when they are built correctly.
The model is a node, not a dependency
In RocketRide’s pipeline builder, the model is a component node sitting between your prompt and your output. A Chat source feeds into a Prompt component, passes questions to an LLM node, and returns answers downstream. That LLM node is a configuration entry. Point it at OpenAI, Anthropic, Gemini, Mistral, Qwen, or a local Ollama instance and the rest of the pipeline stays exactly where it is. The connections, the prompt component, the output handler: none of it changes. You are swapping one node, not rewriting the system.
Here is what a pipeline looks like running end to end:
Every run is fully observable
When a pipeline runs, RocketRide streams six views simultaneously: Design, Status, Tokens, Flow, Trace, and Errors. Each scoped to that specific run, not aggregated across all runs. The Status tab streams CPU%, CPU Memory, GPU Memory, and Total Completions per second in real time.
Cost surfaces before you commit
The Tokens tab breaks down what each component consumed: Total Tokens, CPU Usage, CPU Memory, and GPU Memory, per component, not just as a pipeline total.
Repointing a config key is the start of the work, not the end.
If the banned model was the one your prompts, tool-calling schemas, and eval thresholds were tuned against, you will retune. Prompt behavior differs across providers. Tool calling schemas are not universal. Evals that passed on one model will fail on another. That is real effort and we won’t pretend otherwise.
What the runtime gives you is a scoped problem instead of an architectural crisis. The pipeline structure, tracing, and cost visibility stay intact. You fix one layer, not everything. That distinction matters when you are scrambling at midnight because a government directive just took your model offline.
The economics make the effort worth it. Harvey’s multi-model approach points to a consistent pattern: routing cheaper models for lower-complexity steps and premium models where it counts can outperform a single expensive model while costing less. When you pay per token, that gap compounds across every agent run. But those gains only materialize if the runtime layer exists to do the routing and the measurement. Without it, you cannot even see where the cost is going, let alone optimize it.
The obvious pushback: capability beats resilience. Bet on the best model and architectural flexibility is a distraction.
The stronger version is structural. OpenAI’s AgentKit and Anthropic’s Managed Agents are both pulling orchestration, memory, evals, and observability into managed surfaces. VentureBeat reported that Anthropic has explicitly signaled the orchestration layer is where lock-in will live, because tooling infrastructure will outlast interchangeable models.
The platforms you are routing across are actively trying to absorb the runtime themselves.
That is a real risk. It is also the clearest argument for building on open infrastructure. A vendor-owned runtime is the same rented ground problem one layer up. The swap cost has to stay low, which means the runtime cannot be the thing you are locked into. That is why RocketRide Server is open source. Not as a positioning decision. As an architectural one.
The routing logic, tracing, and cost attribution are not novel engineering. What is hard is building the abstraction before you need it. Most teams build the model integration first and the runtime later, if at all. When the model gets banned or repriced, there is no later.
If your loop is model-agnostic, observable, and on open infrastructure, the next ban, acquisition, or repricing is an operational inconvenience rather than an architectural crisis.
If you’ve run a pipeline on RocketRide Cloud, let us know your feedback.
Your IDE was built for code.
AIDE is built for AI.
