Tool definitions are billed
on every request.

Every connected MCP server's tool catalog (names, descriptions, JSON schemas) goes into the model's context at session start. Connect enough servers and that is tens of thousands of tokens of schema before you type anything.

Inference re-processes the whole context on every request: system prompt, tool definitions, conversation so far. So you don't pay for tool definitions once. You pay for them on every request, for the entire session, and in an agent loop every tool call round trip is another request.

Your bill therefore scales with how many tools are connected, not with how much work is being done. Most of that spend buys nothing: any given request uses two or three tools out of a hundred.

Worked example illustrative

These are assumptions, not measurements. Substitute your own numbers.

36,000 tokens × 50 requests

= 1,800,000 tokens

fixed surface  341 × 55        =  18,755
shortlists     1,500 × 160     = 240,000
# a find_tools result stays in the
# conversation and is re-read by every
# later request: 54+43+32+21+10 = 160
total                          = 258,755 tokens

The percentage matters less than the slope. Connect ten more servers and the direct figure doubles to 3,600,000. The gateway figure does not change, because neither the fixed surface nor the shortlist size depends on how many servers are registered.

The gateway is an MCP server to the client and an MCP client to each downstream server.

2.1 · The client-facing surface: four meta-tools

The client's tools/list returns these four tools and nothing else, at any catalog size. Serialized, they come to ~341 tokens (chars/4 estimate). npm run budget fails if the estimate exceeds 2,000.

2.2 · Just-in-time schema injection

2.3 · Routing

The remaining misses are synonyms ("book" meaning create an event, "jot down" meaning append) that lexical matching cannot catch, which is the dense leg's job. The benchmark measures ranking only; it spawns no servers.

2.4 · Lazy sessions

Registered is not running. No downstream server starts when the gateway starts. A server is spawned by the first call that needs it; its session is pooled and closed after 60 seconds idle. Its tool list is cached for the life of the gateway process, so later searches do not respawn it.

2.5 · Schema compression

Shortlisted schemas pass through a compressor before they are returned. light (the default) removes $schema, $id, $ref, title, examples, $comment, and additionalProperties: false, and keeps types, properties, required, and enums. aggressive also removes descriptions. That is smaller, but it can hurt tool selection; how far to go is an open question.

2.6 · What stays constant and what grows

With every server behind the gateway, every credential is used from the gateway. Holding them there, instead of in each client config, is what makes rotation, audit, and revocation possible from one place.

Design design

Secrets are never written to client config, never returned to the model, and never logged; they are injected only into the outbound call they authenticate. Around the vault, the design adds a usage ledger (which server, which tool, when, and the result, with bulk access flagged), a kill switch (revoke every session and freeze the vault in one action, or scope it to a server or a user), and one-place rotation (automated for OAuth refresh and for providers with key-management APIs, guided otherwise). Team deployments get per-org scoping, admin MFA, and pluggable backends such as HashiCorp Vault or a cloud KMS.

Built so far

The design brief names libsodium. The implementation uses Node's built-in crypto to avoid a dependency; heavier backends remain the plan for team deployments.

The gateway runs locally over stdio and proxies a demo server end to end. It is not packaged, not published, and not ready for real credentials.

The 26 tests (npm test), the budget estimate, and the benchmark result were last recorded on 2026-09-03.

Running it locally (development only)

npm install
npm run build
npm start          # gateway on stdio; loads manifests from servers/
npm test           # client -> gateway -> echo server over stdio
npm run budget     # startup-surface size check
npm run bench      # routing accuracy on the offline benchmark

Point an MCP client at node dist/index.js. The only server included is servers/echo.yaml.

Open design questions (core language, which 50 servers, embedding model, secret isolation, compression level, registry sync, rotation policy, anomaly detection, governance) are in the design brief.

Adding a server is one YAML file in servers/ and one PR. No gateway code changes. The gateway loads every *.yaml there at startup and gets the tool list from the server itself via initialize and tools/list.

id: filesystem
name: Filesystem
category: [dev, files]
description: Read, write, and search files under an allowed directory.
transport:
  type: stdio
  command: npx
  args: ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/allowed/dir"]
auth:
  type: none                # none | api_key | oauth2
routing:
  examples:                 # matched against find_tools queries
    - "read the config file in my project"
    - "list the files in the docs folder"

Required fields are id, name, transport.type, and command for stdio or url for streamable-http. A malformed manifest fails at load time with the file name.

Other useful contributions