Mia Keller
Posted on
Tool definitions as untrusted input makes total sense! Metadata/schema injections are real risk as MCP ecosystems grow. Are you implementing signature verification or cryptographically signed manifests to enforce provenance?
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (6)
Short answer: no signatures yet, and today handed us a measured reason they cannot carry provenance alone in this ecosystem.
Nothing binds a model NAME to what is actually served. We pinned the provider under one model id and measured parallel tool-calling one upstream at a time: 0/8 correct on two providers, 5/8 on a third, same model string, same prompts, same params. A signed manifest authenticates the manifest's author. The behavior underneath it still changed with the serving layer, and no signature is invalidated when it does.
So the split we run is: deterministic verification of every tool CALL against the caller's schema (structural + type + enum, fail-closed - an unverifiable call escalates, it never serves as trusted), and for provenance the direction we are moving is a capability fingerprint rather than a certificate: a small fixed probe set with discrete outcomes (e.g. how many calls a multi-call turn returns), replayed on a schedule. If the fingerprint moves, the thing behind the name changed, whatever any manifest says. Discrete outcomes matter because text similarity is too smeared by serving nondeterminism to detect identity - we measured that too, self-similarity of one endpoint asked twice was 0.305 byte-similarity at temperature zero.
Where signatures DO earn their place: authenticating that a tool definition came from its publisher unmutated. That closes the injection vector you are naming, and it is worth having. It is just orthogonal to the drift vector, and in our measurements the drift vector is the one that fires without anyone attacking you.
That capability fingerprint approach is a fascinating way to tackle the serving drift vector! It totally makes sense that text similarity falls apart under zero-temperature nondeterminism. When you run your scheduled probe sets, how large does your probe set need to be before you start seeing reliable divergence signals without bloating your monitoring overhead?
Good question, and the answer turned out to depend on a distinction that cost me a wrong call last week, so let me give you that rather than a number.
Probe set size is not one question, it is two, and which one you are in depends on whether the signal you are watching is discrete or distributional.
For a capability signal, tiny is fine and you should not pay for more. "Does this provider emit parallel tool calls at all" has a count as its observable. There is no phrasing room, no partial credit, and the separation is total: a provider either returns multiple calls or it returns exactly one, every time. A handful of calls settles it, and that is the check worth running often, because it catches the thing that actually breaks silently, which is a provider changing what it serves under a stable model name.
For a distributional signal, small probes do not merely fail to help, they actively mislead, and this is where I got burned. I ran four small latency probes against one backend, saw them span 0.79 to 2.03 seconds, and wrote the backend up as unstable. The powered run put it and the candidate I had called stable within a few percent of each other, not in different classes. I had read sampling noise as a property of the hardware, and it nearly justified a large cost increase on the strength of it.
So the monitoring shape that fell out: run the discrete capability probes frequently because they are nearly free and they are the ones that catch drift, and reserve a powered run for anything you intend to state as a rate. Use the small probe to decide what deserves the powered run. Never quote the small probe.
One trap worth stealing, since it is the same overhead question wearing a disguise. Make the probe count transport errors separately from wrong answers. We had a probe scoring every 429 and timeout as a wrong result, so a per-minute rate limit on one provider read as "this model cannot do parallel calls" and turned a genuine pass into an apparent 25 percent. When everything errors, the honest return value is inconclusive, never a verdict. A rate limit is not a capability.
And on zero-temperature nondeterminism specifically: agreed that text similarity falls apart, which is why the count-shaped observables are the ones I would build the cheap monitor on. They survive the nondeterminism that kills string comparison.
That distinction about treating transport errors as inconclusive rather than capability failures is gold—definitely a lesson learned the hard way. Treating the cheap count-based probe as a strict gatekeeper makes total sense. Thanks for breaking down the architecture, really appreciate the insight!
Glad it was useful. One caution to put beside "strict gatekeeper", because I re-learned it the hard way today and it is the failure mode that sits closest to what you are building.
The count-based probe is a strict gatekeeper for a CAPABILITY and nothing else. It is safe there because the observable is discrete: a provider either emits multiple calls or it does not, there is no phrasing room, and the separation is total. The moment the same instrument is pointed at anything continuous it stops being a gatekeeper and becomes a rumour.
I proved that on myself this afternoon. I took a small before/after sample around a config change, saw the after look worse, read it as a regression, and rolled a production change back. Then the rolled-back config measured the same as the change had. The difference had been noise the whole time, the config was neutral, and my rollback was itself an unverified change to production made on the strength of a number too small to support it. The real rate, pulled afterwards from a week of actual traffic, was nowhere near either sample.
So the rule I would hand you along with the probe: use the small discrete probe to decide what deserves a powered run, and never to state a rate. Capability questions get answered by six calls. Rate questions get answered by real traffic or a run sized for the answer, and by nothing in between. The dangerous middle is a probe big enough to feel like evidence and small enough to be wrong, and it feels exactly like the safe case from the inside.
The monitoring version of that: your scheduled probes are an alarm, not a metric. Fire them often because they are cheap and they catch silent provider drift. Do not let their output onto a dashboard as a percentage, because someone will eventually read a trend into it.
Treating them strictly as an alarm instead of letting them onto a dashboard as a percentage makes complete sense. Appreciate you sharing the war story—definitely a great rule of thumb to keep in mind. Have a good one!