Why Equal AI Agent Scores Hide Different Failures: A Law Firm Governance Test

As of October 2026, the practical governance question for a law firm adopting AI agents is not which model scores highest. It is whether the firm can write down what the agent must do in each situation it will meet, and then test those rules one at a time. This article works from a small set of recent sources. Where a point is our reasoning rather than a sourced fact, we label it as a hypothesis or an illustrative example.

The claim worth testing: an average score is not a control

A preprint posted to arXiv on October 16, 2026 introduces SIT-Eval, a framework for evaluating conversational agents by situation. These are author claims from an abstract, not peer-reviewed or independently reproduced results, and we have read only the excerpt.

According to the abstract, the authors formalize desired behavior as situational rules: if the situation is P, follow rules Q. At each turn, an LLM plays an interlocutor that enacts a uniformly sampled situation. A context-aware LLM judge then scores the agent's reply against the rules that apply. The judge is validated against human-verified synthetic dialogues.

The authors argue that static benchmarks underrepresent rare situations, and that at production scale, low-probability events become frequent failures. They benchmarked 8 LLMs on 1,292 conversation turns. The reported finding is that models with similar overall scores varied widely in how well they followed individual rules. The excerpt also says all models struggled most with ambiguity clarification.

For a managing partner, the implication is direct. A vendor summary that says an agent "scored well" tells you little about the one rule you care about most. A single average can combine a harmless stylistic miss with a missed escalation, and the two are not equally costly.

An illustrative rule set for a firm intake agent

Operational example, hypothetical, not measured data from any firm or from AlphaForge. Suppose a firm deploys an agent to answer first-contact questions. Situational rules might look like this:

• If the person mentions an urgent deadline or a recent death in the family, route to a human the same session and do not continue self-service.
• If the person asks whether they have a case or what they should do, do not give legal advice. State that an attorney must review.
• If the request is ambiguous (which property, which relative, which document), ask a clarifying question instead of guessing.
• If the person says they are an existing client, do not treat the contact as a new lead and do not discuss matter details until identity is confirmed through the firm's normal process.
• If the message names another party, flag it for the firm's conflict-check process.

Each rule is a testable statement. The ambiguity rule matters especially, because it is the category the SIT-Eval abstract reports as hardest across the models tested. We cannot say how any particular model performs on legal intake from this paper. The dialogues are synthetic and the judge is itself a model. The method is the useful part: define situations, sample the rare ones deliberately, and score each rule separately.

Every connected tool is a boundary to document

Agents become more consequential when they connect to other systems. LawNext reported on October 19, 2026 that Legora and Blue J launched an integration built on the Model Context Protocol (MCP). Per that report, Legora customers with a Blue J subscription can query Blue J from within Legora and receive answers with inline citations linking to sources in Blue J. The report says Blue J performs the research on its own infrastructure. Blue J's CEO and cofounder, Benjamin Alarie, is quoted in the announcement as saying the point is cited analysis where tax professionals already work, with source checking and professional judgment kept central. That is the companies' framing, reported by a news outlet, not independent evaluation.

The same trade press described the wider direction. LawNext's September 2026 ILTACON round-up said the conference's product announcements were dominated by two themes: agentic AI and MCP. It also reported that Litera describes a Firm AI Search capability that gives lawyers natural-language access to firm matter, client and expertise data through its Lito agent. That is the vendor's description as reported, not a tested result.

We are not saying any of these products has a governance problem. The excerpts do not support that. The point is what a firm should ask whenever an agent gains a new connection. Our hypothesis is that these are the questions a governance review needs answered for each one:

1. Identity: Under whose permissions does the agent's call run, the user's or a shared service account?
2. Outbound content: What text from the firm's work leaves the firm's environment in a query, and where is it processed?
3. Inbound content: What comes back, and is it labeled with a source a lawyer can open and check?
4. Access inheritance: When a search or agent layer sits over matter and client data, does it respect the same restrictions as the underlying system?
5. Logging: Can the firm reconstruct what was asked, what was returned and who acted on it?

We covered the permissions side in an earlier piece on agents that run with broader access than their task needs. The short version is that an agent's reach is set by the access it is given, so scoping that access is a design decision made before launch.

Design for expert review, not for a verdict

A second preprint, also dated October 16, 2026, offers a useful pattern from outside law. The SEER-Repro paper concerns LLM agents that reproduce medical findings from the SEER cancer registry. The authors report a benchmark of 20 published papers and an agent that reproduced 77.4% of 2,190 atomic claims. They also describe a pipeline that identifies analytical choices behind each finding, proposes defensible alternatives, reruns the analysis and flags cases where the conclusion might change. In their results, it flagged potentially consequential choices in 9.5% of fully reproduced findings for expert review. Again, these are author claims from an abstract.

The domain is medical research, not legal work, so treat what follows as an analogy and a hypothesis. The design lesson is that the agent did not issue a final answer. It surfaced the specific decisions that could change the result and sent a small share of them to a person. A law firm agent that summarizes a document set or drafts a client letter can be built the same way: list the assumptions it relied on, mark the ones that could alter the output, and send those to the responsible lawyer. A review queue with a defined trigger is easier to supervise than a general instruction to "have a lawyer check it."

Test the agent you will actually run

One more observation concerns evaluation setup. In a Microsoft Research blog post on Agent Lightning v1.0, researchers note that an agent's capabilities increasingly depend on the harness around the model, meaning the tools and execution environment that coordinate it. They also say that in most agent reinforcement-learning systems, developers must reimplement the agent inside the training framework, so the agent being trained is not quite the agent that gets deployed. That is a research-team description of a training problem, and it is not about law firms.

We draw a narrower hypothesis from it. If the firm's evaluation runs against a simplified stand-in, such as a bare model without the real prompts, tool connections and retrieval settings, the results may not describe the production system. Run the situational tests against the configuration clients and staff will actually reach, and rerun them when the model, the instructions or a connected tool changes.

A working checklist

1. Write the rules first. Draft situational rules (if P, then Q) with a lawyer and an operations lead before choosing or configuring a tool.
2. Sample the rare cases. Include urgent, ambiguous, emotionally charged and out-of-scope situations on purpose. Natural traffic will under-supply them in testing.
3. Score per rule. Set a separate pass threshold for escalation, no-advice and clarification rules. Do not average them with tone or formatting.
4. Check the judge. If a model grades another model, validate that judge against cases a human has already labeled, as the SIT-Eval authors describe doing with synthetic golden dialogues.
5. Map every connection. For each MCP or similar integration, record identity, outbound data, inbound sources and logging.
6. Define review triggers. Decide which outputs go to a lawyer by rule, not by whim.
7. Retest on change. Treat a model update, prompt edit or new connected tool as a reason to rerun the suite.

What we do not know

This article contains no first-party AlphaForge measurements and no data from any law firm. Nothing here shows that a given firm's agent is failing, or that any firm is losing leads or revenue; that would be a hypothesis requiring discovery work and the firm's own records. The two preprints are unreviewed author claims, one using synthetic dialogues and one in a medical setting. The Legora, Blue J and ILTACON items are trade reporting of vendor announcements. None of these sources addresses legal or ethical obligations, and we have not inferred any. What they support is narrower: aggregate scores can mask rule-level differences, tool connections widen what an agent can reach, and a well-defined review queue is a workable control. Whether those hold in a specific firm is an empirical question the firm can answer by writing its rules and testing them.