The settings that stopped my AI bots burning tokens
Every message to an AI resends the whole chat plus its setup. These are the settings that kept mine small. The names are from my bot app (Hermes Agent), but most AI agent tools have an equivalent.
1. Cap the chat, not just the window
My bots had a 1,000,000-token window and only summarized old messages at 90% full. Set a hard cap instead:
Summarize at 60,000 tokens, whatever the window size.
Trim old tool output early (I start at 30,000 tokens). Long results from earlier steps are the biggest waste.
2. Load tools only when needed
Each tool's instructions ride along on every message. Keep the everyday ones loaded (files, search, memory) and let the rest load on demand. My tool instructions went from 66 KB to 36 KB per message.
3. Hide unused skills
A bot doesn't need a list of 100 skills it never opens. I kept each bot's own skills plus the ones it used in the last 60 days. The skill list went from about 11 KB to about 1 KB.
4. Check the summarizer works
Summarizing uses its own model. If that model can't run, the cap silently does nothing. Test it once after changing anything.
Result
Fixed setup per message went from about 100 KB to 38 to 64 KB, and no chat can grow past about 60,000 tokens.