Behind

Token-Saving Hacks for AI Agents (Keep the Change!)

Must be nice if your company has an endless budget for AI tokens, but for a scrappy, hungry startup like us? Getting the absolute most bang for our buck (and keeping those credit card bills in check) is practically a survival sport. Sure, top-tier models get what you mean in a heartbeat and deliver killer results—no debate there. But our daily obsession is: How can we get a cheaper model to pull off even more heavy lifting?

The Anatomy of AI Costs

Here’s the simple math behind what our AI agents cost when using LLMs:

Cost per Token (Model) x Token Count (Input + Output) = Total Cost

Most folks already know that the cost per token varies wildly depending on the model. And you probably also know that token count roughly translates to word or character count.

The real kicker, though, is how those tokens actually pile up. You might think, "Hey, if I just keep my prompts and responses short, I'm good, right?" If only it were that simple! It all comes down to how LLMs actually work under the hood. Let's be real for a second: an LLM doesn't have feelings, it doesn't build a "relationship" with you, and it has zero memory of who you are or what your project is about. It simply tokenizes the text you throw at it, runs it through the neural net, and spits out an answer. That's it.

So, to keep the conversation flowing smoothly, AI agents have to bundle up everything—your working environment, project details, and literally everything you just said—and ship it over with every single prompt. You aren't just sending a quick follow-up; you're re-sending the entire history every time. Started the chat complaining about today's weather, wandered into debugging code, and now brainstorming marketing copy? Guess what: that initial chat about the rain—and the AI's reply to it—is still hitching a ride along with your marketing prompt, driving up your bill.

Here’s a simple way to visualize the tokens consumed per question-and-answer round:

Or, to be a bit more precise, it looks something like this:

All that baggage carried over with every single request? That's what we call Context.

Now, modern AI agents are pretty clever. They can compress context, prioritize recent chats, or leverage prompt caching to help trim down the bill. But here's the catch: context compaction itself relies on summarization—which is another computation, and definitely not free. We’ll dive deeper into that in a bit. But fancy optimizations aside, this is fundamentally how the conversation engine runs!

That’s why folks who don't know this yet often dread repeating themselves in a "New Chat." Instead, they keep piling on completely different tasks into a single window until the scrollbar is practically begging for mercy. Sure, you’ll still get an answer, but you're burning through an absurd number of tokens in the process.

We minimize and optimize this context in several ways. You don't have to obsessively check, "How heavy is the context in this session?" every single time—you just need to adopt a few golden rules and turn them into second nature.

Maximizing Prompt Caching

As we covered the basics above, modern AI providers don't tokenize and recompute every single thing from scratch on their servers each time. When a user's conversation stays active, the system simply adds new calculations onto the previously computed results, delivering faster, snappier responses. This magic is called Prompt Caching. Best of all, cached prompts come at a massive discount compared to raw computations. "Wait, does that mean I can just keep chatting away without a care in the world?"

Not so fast—here’s the catch. Prompt caching is notoriously picky about its activation triggers. The interaction usually needs to happen within a 5-minute window, the model or reasoning depth cannot change, and memory or global system instructions must stay untouched. Mess up just one of these, and boom: the entire context is processed raw, taking a bigger bite out of your budget once again.

Flip that around, though, and it means as long as you play by these rules, you reap all the sweet benefits of prompt caching. If you’re juggling multiple tasks with different agents, a great pro-tip is to review things quickly and queue up the next step before the cache expires—wrapping up your work while the engine is still hot!

Tie this principle back to our "context" discussion earlier, and the golden rule becomes crystal clear: open a fresh session whenever the topic shifts to prevent context pollution and wasted cache, and keep up the momentum without long pauses while working on the same topic so your cache never breaks stride.

Isolating Side Quests (Branch Issues)

When you're deep in the zone, you quickly realize problems rarely travel alone. They like to bring friends—backend bugs, auth quirks, user data pipelines, UI snags, testing woes, and unrelated features that just happen to share the same boat. Ignorance was bliss back in the day, but modern AI agents are notoriously helpful snitches: they'll gladly point out every single little flaw they bump into along the way. In that moment, frustration (or the illusion of an "easy 2-minute fix") kicks in, tempting you to detour and knock it out right away.

Don't take the bait! Resolutely park those side quests and keep hammering away at your original target. This simple discipline keeps your context window pure, saves precious tokens from going up in smoke, and protects your sanity. The key is to capture those newly spotted gremlins immediately so they never slip through the cracks.

Here is a quick snapshot of how that workflow looks:

There are plenty of ways to shelve these distractions. You could scribble them into a throwaway .md scratchpad, or do what we love doing: integrate them straight into GitHub Issues. By carving out a neat habit to log side issues in a structured format (a quick snapshot of the current context, potential pitfalls, etc.), you can cleanly resolve the main task first—then dive back into the side quest with a brand-new, squeaky-clean context window.

Hierarchical Document Architecture

As we've chatted about in previous posts, whether it's your runbooks, global rules, or internal docs—hierarchies rule the world. Sure, stuffing a massive, 1,000-line omnibus full of security, billing, ops, coding, and testing rules into your global instructions might technically work. But it also dumps an avalanche of irrelevant info into every prompt, burning through tokens like there's no tomorrow.

Instead, keep your global instructions strictly lean and essential, tucking domain-specific know-how into separate, dedicated docs. To keep your AI (and yourself!) from getting lost, simply build clear signposts in the entry document (your global instructions) that point to where deeper knowledge lives.

Here’s a handy blueprint for how you can structure it:

AI agents are equipped with a clever mechanism called tool calling, which allows them to fetch necessary information on the fly. They start by grasping the essential global guidelines. If a specific task calls for extra documentation, the agent fetches and reads only those relevant files. By accessing just the bare minimum needed for the job at hand, we save precious tokens—and keep the magic running smoothly!

However, keep in mind that invoking tools to read docs also consumes turns and tokens, and whatever is read sticks around in the context window for the rest of the conversation. Our golden rule? Split by frequency: "Keep what's needed almost every time in the main entry doc, and offload the occasionally needed info to external files." For docs used even less frequently, keep them tucked away in /docs and pull them in with an explicit mention only when absolutely necessary.

Periodic Spring Cleaning for Memory Guidelines

When working with AI agents, give an emphatic command or run into recurring hiccups, and the agent might suggest—or automatically add—memory guidelines. This means that memory prompt will hitch a ride on every single conversation from that moment on. While global guidelines are tracked cleanly in docs and version-controlled via commits, memory guidelines can quietly pile up in the background and become major token-drainers if left unchecked.

Leave the naturally flowing ones alone during everyday use, but make sure to run a periodic tune-up! When giving your global instructions a "diet," promote the consistently useful memories to global guidelines, and toss out the one-off memory instructions that have overstayed their welcome.

Splitting Workloads: Router vs. Heavyweight Models

Using top-tier, high-performance models (like fable / astra) feels amazing—you barely have to type, the AI understands your intent effortlessly, makes fewer mistakes, and gets the video edit done just right. But with great power comes a great bill! High performance equals higher costs, leading us to find smarter ways to save.

In reality, when you run an agentic workflow, nearly half of the tasks are routine: fetching files, searching, simple tool calls, quick tweaks, or checking logs. That’s why we set a mid-tier model (like sonnet / terra) as our everyday default, configuring it to automatically hand off complex, heavy-lifting tasks to the powerhouse models. In a way, our default model acts as an intelligent traffic router—saving money while delivering speed and precision!

A mid-tier model might not crack the toughest problems on its own, but it’s definitely smart enough to tell whether a task is a walk in the park or a deep rabbit hole. It can even catch on when a seemingly "simple" task starts getting out of hand! While you could use an ultra-lightweight model strictly for routing, we love mid-tier models because they can actually pull their own weight and handle a decent chunk of the real work, too.

That's where the magic of sub-agents comes in. Think of it like spinning up a brand-new session: the primary agent can create its own "mini-me" session, delegate the gritty work, and just wait for the results. Since an agent can't magically switch its own underlying model mid-session yet, summoning a sub-agent is the ultimate power move to swap brainpower on the fly.

And the perks don't stop at just swapping models. When a sub-agent digs through dozens of files and goes through trial and error, all that chaotic clutter and endless tool-call logs stay trapped inside the sub-agent. The main session stays squeaky clean, receiving only the polished conclusion. It's the ultimate "keep the drama off the main stage" strategy—keeping your primary context window light, focused, and efficient.

We set up a global instruction along these lines: "If a task feels heavy, complex, or security-critical, summon a high-performance sub-agent, hand over the context, and delegate the plan. Wait for its report, then proceed with the implementation according to its guidance." As you fine-tune this in practice, you'll dial in the exact sweet spot for when to pass the baton. Once set, you don't have to micromanage every single prompt—stick with your default model, and let the agents handle the heavy lifting dynamically.

Why You Should Always Go for the Subscription Model

Most API pricing is strictly pay-as-you-go, which can get spicy fast. Subscription plans, on the other hand, might have caps, but those limits are usually way more generous than burning through raw API credits. Providers love building brand loyalty (and counting on a bit of breakage), which works out in your favor. If you want maximum horsepower without the bill shock, subscriptions are an absolute no-brainer.

If you're dealing with structured bulk workloads rather than real-time conversational tasks—think categorizing or summarizing hundreds of data points in one go—batch API processing (outside your regular subscription) can often slash costs by up to 50%. Yes, pay-as-you-go still has plenty of room to save you some serious cash!

Many CLIs offer a one-off execution mode that cleverly routes token consumption through your existing subscription plan (a handy trick that will come in very useful later for custom skills).

This opens up a massive playground of possibilities for workflow skills. For instance, while running a code review, a skill can summon a completely independent model to debate, cross-examine, and double-check the results. Sure, you could build this entirely via standard APIs, but the real magic—and maximum cost-efficiency—comes from steering that consumption under your subscription tier whenever possible.

Wrapping Up

We've walked through the smartest ways to tap into cutting-edge AI models without burning a hole in your pocket. Here’s your quick cheat sheet: