I wanted one chat interface that could reach everything I run: my publishing CMS, the web, files, spreadsheets, images, and audio, on either a local GPU or a cloud model. The obvious approach is to hand the model every tool on every request, and it stops working almost immediately. This platform is what I built instead. It's a private admin tool, running in production behind my own site, and it's where most of my MCP servers get their daily use.
The Challenge
Every MCP tool comes with a schema, and schemas cost tokens. Load enough tool groups and the definitions alone fill the model's context before the conversation starts. On a cloud API they run into per-request token limits. On a local model with an 8K context window, even one heavy tool group can crowd out the actual conversation.
The Tool-Overload Problem
More tools should make an assistant more capable. In practice, past a point, every tool you add makes each request slower, more expensive, and more likely to fail. The model also gets worse at picking the right tool when it's choosing from everything.
My Approach: Two-Phase Tool Routing
I split each request into two calls. First, a cheap, fast planner model reads the user's message and a compact one-line-per-group manifest, then returns just the tool groups the request needs, such as web, cms, files, excel, images, or audio. It runs at temperature zero, has a tiny token budget, and has a hard timeout.
Second, the server opens MCP sessions only to the servers in those groups, discovers their tools, and runs a standard tool-calling loop against the chosen model, with a cap on iterations. The model only ever sees the tools it might actually use.
- Per-backend safe sets: each backend has its own list of groups it may load at all
- Group ceilings: a cap on how many groups load at once, sized to the backend's limits
- Deterministic trimming: when the planner asks for too much, groups are dropped in a fixed order, heaviest and narrowest first
- Safe fallbacks: if the planner fails, the request continues with a small default set instead of failing
The Solution
Dual-backend inference
The same chat runs on a cloud model or on a local model served by LM Studio on an NVIDIA RTX 5060 Ti, which the server reaches over a private WireGuard tunnel. A toggle in the interface switches between them, and both stream through the same parser. The local path is built for the realities of one GPU: one model loaded at a time, slow first loads, and a tunnel that can briefly drop when the home server restarts.
Context-aware ceilings
On the local path, the group ceiling comes from the loaded model's context window: one group at 8K, two at 16K, five at 32K and up. If I load a model with a bigger context, the chat can use more tools on the next request without a restart.
Voice dictation
The browser records audio and the server sends it to a Whisper speech-to-text MCP server. The raw transcription tool is deliberately hidden from the model: dictation is a UI feature, not something the assistant can call on its own. A per-server deny list handles the same thing for other UI-only tools.
Security Boundaries
MCP servers require a bearer token. Private tunnel addresses never reach the browser, and API keys never reach the client or the logs. Higher-risk servers, such as shell and full filesystem access, aren't exposed to this chat at all. They're reserved for a separate, more tightly controlled path.
Results & Impact
Right Tools, Every Request
The planner loads only what a request needs, so prompts stay small and tool choice stays sharp.
Local or Cloud
The same chat, the same tools, on owned hardware or a cloud API.
Talk Instead of Type
Whisper dictation is built into the chat.
Fails Small
Planner errors, tunnel blips, and unloaded models degrade gracefully instead of breaking the chat.
Lessons Learned
- Tool selection is its own problem. Asking a small model "which tools?" first turned out to be cheaper and more reliable than making a big model choose from everything.
- Budget per backend, not per app. Cloud and local models fail at different limits, so each needs its own ceiling.
- Make limits follow configuration. Deriving the ceiling from the loaded context window removed a whole class of "forgot to update the constant" bugs.
- Not every tool belongs to the model. Some capabilities are UI features. A deny list keeps that line clear.
Want an Assistant Wired Into Your Systems?
I build agentic chat and tool-routing layers on top of MCP servers, on your infrastructure, with your models, and with security boundaries you can explain to your security team.
Discuss Your Project