ToolSession sends a schema when the model asks for it. The contract does not
change: a loaded tool is the same fully typed tool it would have been, validated
against the same Pydantic schema. Nothing routes through a generic
execute(slug, arguments), and no tool is ever offered with a schema it does not
have. Only the timing changes.
ToolSession
The tools of one conversation, and which of their schemas have been sent.
Building a session is cheap: at the default threshold nothing can be resident,
so nothing is sized, and a deferred tool’s schema is not going out this turn
either way.
ToolSearch is
prepare for each tool as it loads it — the
tools the process is about to expose, rather than all of them to expose none.A positive threshold has to size every schema to place it, and sizing one
derives it. On Linear’s 128 that is about 2.5 seconds, synchronously, on
whichever thread constructs it, so construct it at startup rather than per
request.ToolSearch never reaches the model, which from outside looks like a
model that cannot stop searching. So Charter ships the loop rather than writing
that down: run for the OpenAI APIs,
CharterMiddleware for LangChain, and the
MCP server for MCP, where the protocol carries it.
What the model sees
ToolSearch, and nothing else, until it loads something. Its description is
the flat list of every tool by name. Names only: descriptions cost about five
times as much, and the pack is a substring of every qualified name, so +stripe
filters by pack without a grouping API. It takes three query forms.
Then the loaded tools, in the order they were loaded, each with its full
schema and callable like any other.
A tool that has not been loaded is absent from the tool list, not present with an
empty schema. Two earlier designs left them callable and the model called them,
guessing arguments and failing validation three to four times a run.
The order is not cosmetic.
ToolSearch is fixed and comes first, so each turn’s
tool list extends the previous one rather than reordering it, and the cached
prompt prefix survives.
Every tool, however small the pack
Three tools or twenty-three, all of them are deferred. Claude Code keeps a hot set resident, but that set is its own built-in core; every tool from a configured MCP server is deferred, all of them, however few. A Charter tool is a configured tool, so this is the same line in the same place. Keeping the small schemas resident was tried and measured. A resident tool only pays for itself if it saves aToolSearch round trip, and schema size is
anti-correlated with being the tool an agent reaches for first: an entry point
takes query parameters, a delete takes an id. A 200-token threshold, measured across the
eleven packs shipped at the time, kept 25 tools resident that no agent starts with, and deferred 25 of
the 28 list and search tools. On Gmail it kept threads_delete and
labels_delete while deferring messages_list, so the round trip was paid
anyway and the resident set cost 1,355 tokens to avoid nothing.
threshold is still there for a hand-written pack whose tools have a different
shape. progressive=False sends every schema up front.
Deferral changes how many schemas are in the prompt. It does not change what
one costs: once the model searches for a 47,000-token tool it has bought all of
it, on every turn until the end of the run. The other half of the lever is
narrowing the schema itself.
Methods
visible() returns the tools callable this turn, by name. dispatch(name, arguments) routes one call from the model: ToolSearch loads and returns JSON,
anything else goes to Tool.ainvoke. Calling a
deferred tool raises
ToolValidationError with the
select: query that would load it. reset() clears the loaded set to reuse a
session for another conversation.
dispatch also takes a keyword-only client, an httpx.AsyncClient handed
through to ainvoke unchanged, so a loop you drive yourself can hold one
connection pool open across a run instead of paying a TLS handshake per tool
call. ToolSearch makes no request, so a client passed alongside one is inert.
See latency for what it is worth
and where it cannot be reached.
The mode a session resolves
mode resolves every tool’s view at that Mode
instead of the one it was built with, by calling
with_mode on each. None, the default,
leaves every tool exactly as it was given, by identity.
TOOLSETS = {"free": [...], "pro": [...]} dict
and the code choosing between its entries. The pack declares which fields belong
to which surface once, and the deployment names a surface per conversation.
The session mode is in force alongside a mode the tool was built with, not
instead of it. The author’s mode= says which operation a tool is — create
replaces and update merges — and that is not a deployment decision; the
session’s label says which surface the conversation is. So a session label only
ever adds the fields carrying it, on every tool, whatever the pack declared.
You do not have to read a pack to predict what keying it to a tier will do.
It resolves once, here. Credentials resolve per call, through a ContextVar,
because the subject can change between two calls of one conversation. A schema
cannot: it was serialised into a prompt the model is still reading, and
parameters that change under a model already shown them is a tool it will call
wrong. The session is the per-conversation object, so this is where the
per-conversation answer belongs. A surface that has to change means a new
session.
Everything downstream sees the resolved tools: the partition that decides what is
deferred, the ToolSearch catalogue, what a load hands back, what dispatch
validates against, and the egress_map() a reviewer
runs over session.tools. Resolving costs one schema build per tool per mode,
paid at construction like the rest of the session, and tools from one pack share
the store their factory owns, so a surface’s worth of tools at one mode build
their shared types once between them.
A field the mode withheld is not merely absent from the prompt. It is absent from
the type dispatch validates against, which forbids extras, so a tier the
deployment did not buy is refused rather than discouraged.
mode is a label or None, and an empty string is refused rather than read as
None — None is the widest view, so a plan lookup that came back empty would
otherwise be served every tier. A label that withholds a field the API requires is
refused too, since the request would go out without it. Both raise
DeclarationError where the session is built.
See the mode system.
Across the adapters
MCP is the surface where this depends on the client. The server holds the
session and a load emits
notifications/tools/list_changed, but a client has to
re-list on it and hand the new tools to the model at once, and most do not:
Codex keeps the list it started with, and Google ADK reads it once per turn.
charter-mcp sends every schema unless you pass --progressive;
the MCP page records what each client did.
The projections underneath are public — to_openai_tools(session),
to_langchain_tools(session), build_server(session) all take a session and
return what the model can see this turn. Drive them yourself if you own your
loop. Nothing has to.
A plain list is never progressive, on any surface. A list has nowhere to record
what a search loaded, and a ToolSearch whose results evaporate is worse than
none.
run
The OpenAI tool-calling loop, with the tool list rebuilt every turn.
client is duck-typed: anything exposing an awaitable
chat.completions.create(model=..., messages=..., tools=...), which is what
openai.AsyncOpenAI is. Charter takes no dependency on the openai package.
Extra keyword arguments (temperature, tool_choice) pass through unchanged,
and the messages you pass in are not modified.
A CharterError from a tool becomes that
tool’s result, because its message is written for a model to read and correct
itself from: a rejected argument, or a tool it has not loaded yet. Anything else
propagates, so a bug in your own code is not quietly fed to a model as text.
RunResult carries messages (the whole transcript, ready to append the next
user turn to), content (the final assistant text), turns, and stop —
"end" when the model answered, "max_turns" when it ran out.
CharterMiddleware
A session as LangChain agent middleware, for create_agent.
create_agent binds its tool list when the graph is built, and a session’s list
grows. LangChain provides for exactly this — its docs call it runtime tool
registration and name MCP servers as the case — through two hooks, and both are
needed. wrap_model_call puts this turn’s tools on the request; wrap_tool_call
supplies the tool to execute, because the agent has no way to run something that
was not in its original list. The middleware implements the sync and async form
of each.
Build the agent with tools=[] and let the middleware provide them. Anything you
do pass to tools= stays: the middleware only adds, and only claims calls whose
names it recognises.
The agent’s own context reaches every Charter call, for the tools’
FromContext fields: whatever
agent.ainvoke(..., context=...) is given, a dataclass, a TypedDict or a
pydantic model, read call by call, so one agent serves every user. See
passing the host’s context.
Run the agent with await agent.ainvoke(...). A Charter tool executes through
Tool.ainvoke, async down to httpx.AsyncClient,
so there is no blocking form to give LangChain. agent.invoke(...) is refused at
the first model call with a CharterError
saying so, rather than binding the tools, spending the model call, and failing
afterwards inside LangChain.
This one needs the langchain package, not just langchain-core:
schema_tokens
The size of a tool’s parameter schema, measured as serialised-JSON characters
over four, on the JSON that goes on the wire.
threshold and budget are compared against. It is an
estimate, close enough to rank two tools, which is all it is used for.
json_tokens
The rule underneath it: serialised-JSON characters over four, for any value.
schema_tokens and
Tool.paths(by_cost=True) both measure through
this, so a saving reported by one is checkable against the other.