As of October 2026, a LawNext report dated September 23, 2026 on a practice-management vendor's new connector raises a question managing partners may face: if an AI agent answers "where did our intake stall last quarter?" from the firm's own system of record, should that answer drive a staffing, marketing or process decision? The framing is our inference, not a finding from the report. This article works through one angle: what to verify before an agent's intake and revenue-leakage report is trusted, using that announcement and two adjacent pieces of research as source material.
What was announced, and what the vendor said about its limits
On September 23, 2026, LawNext reported that 8am, the parent company of MyCase, announced an MCP connector that lets a user work in Claude and reach data inside MyCase. These are the company's claims, relayed by a trade publication, not independent measurements:
- The connector supports 73 actions across cases, clients, documents, billing and intake.
- More than 60 firms had been using it since a quiet preliminary launch about a month earlier, most of them firms of five or fewer.
- A separate 8am IQ Firm Agent, which would answer questions across all of a firm's MyCase data, was described as a prototype, with beta expected by mid-October and general availability slated for December.
The most useful part of the report for an operator is the caveats attributed to 8am's principal product manager for AI, Joshua Carter. Carter said results could differ between the two routes, because MyCase's native AI features run on OpenAI's GPT models while the MCP connector uses Claude. Results through the MCP would also lack the citation links to documents and sources that Firm Agent is planned to include. And what a customer can reach through Claude may be restricted by their MyCase subscription tier.
In our reading, that describes a situation where the same underlying data can be queried through two interfaces with different models, different citation behavior and possibly different data access. That is not a criticism of the product. It is, in our view, a description of the conditions under which any number it produces should be checked.
Why this matters for revenue leakage specifically
Intake and revenue leakage questions are usually counting questions: how many inquiries arrived, how many were contacted, how many were scheduled, how many became matters, how much billed revenue followed. Our working hypothesis, not a sourced finding, is that counting questions are where a fluent agent is hardest to catch out, because a confident paragraph and a correct count can look identical on the screen.
One caution on framing. Nothing in the sources above shows that any particular firm is losing leads or revenue. Leakage is a hypothesis. It can only be confirmed or dismissed with the firm's own records, and an agent's summary of those records is not the records.
A lesson from legal research that may transfer to intake reporting
An Artificial Lawyer piece published October 20, 2026 makes an argument about legal research that we think maps onto intake analytics. Its author, writing from Lucio, a legal research vendor describing its own experiment (so treat it as a perspective, not independent evidence), argues that research usually ends when time runs out, not when everything relevant has been found, and that omission is partly invisible: nobody sees the judgment that was never opened. The piece also argues that verification does not scale the way search does, so trust shifts to how a system screened material, and that screening has to be visible. It describes listing discarded sources alongside cited ones as a design choice.
The analogy to intake is ours, not the article's, and neither source tested it. The reasoning is that an agent summarizing intake performance could share the same blind spot: an inquiry that never made it into the CRM cannot appear in any report built from the CRM. A web form that failed silently, a voicemail that was never logged, a referral emailed to an attorney's personal inbox: these are hypothetical intake equivalents of the case nobody opened. If an agent reads only the system of record, its answer would plausibly be bounded by what the system of record captured. The question to ask is the one the article poses for research tools: how much did it read, how did it decide what mattered, and can a person trace each statement back to something they can open?
Benchmarks help, but only for the task they measure
It is tempting to settle the trust question with a leaderboard. A recent preprint illustrates why that is thin. The RefCraft abstract, posted October 19, 2026, describes a benchmark of 300 multimodal webpage-development tasks. According to the authors, across nine frontier models the strongest system scored 52.12 out of 100. That is an author claim in a preprint, not a reproduced result, and it concerns generating webpages from visual references. It says nothing about intake reporting or any law-firm task.
Our inference is narrower: a benchmark describes performance on its own tasks, scored its own way. The authors say they used independently verifiable keypoints alongside programmatic and model-based judges, which we read as a reminder that a good evaluation defines what "correct" means before scoring. A firm would need the same for its own questions. We covered the related problem of equal scores masking different behavior in why similar agent scores hide different failures; the same logic may apply when two interfaces to one dataset are both "working" but disagreeing.
A test plan before anyone acts on an agent's intake numbers
This is a hypothesis-driven protocol, not a validated method. It is designed to be run by an operator with read access, in a few working sessions.
- Build a small first-party ground truth. Pick a bounded window, such as two weeks. Pull inquiries from every channel you can reach independently of the CRM: phone system logs, web form submissions, shared inboxes, referral notes. Reconcile them by hand against CRM intake records. Any gap between the two lists would be first-party evidence worth investigating, and it exists before any agent is involved.
- Ask the same question through each route. If your platform offers a native assistant and an external connector, put identical questions to both: how many new inquiries in the window, how many converted to matters, which practice area had the longest time to first contact. Record both answers verbatim, along with the model and interface used. Where the vendor says results can differ, measure how much.
- Score against the ground truth, not against each other. Agreement between two agents is not accuracy. Compare each answer to your hand-built list and note which record types each missed.
- Require record-level traceability. For every count, ask the agent to list the underlying record identifiers. If an interface cannot show sources, as 8am said would be the case for results via its MCP route compared with the planned Firm Agent, decide in advance whether that interface is acceptable for decisions or only for exploration.
- Make exclusions visible. Ask what the agent filtered out, what it could not access, and whether your subscription tier or permissions limited the data. In our judgment, an answer that cannot state its own boundaries should be treated as a draft.
- Keep read and write separate. The connector described in the report spans actions across several record types. Begin with read-only use. If a workflow later lets an agent send a client message or change a record, log attempted actions and completed actions separately and review the difference.
- Set a re-test trigger. Models, connectors and tiers can change. Re-run the ground-truth comparison when the vendor changes the model, adds capabilities, or when your own intake channels change.
What this evidence does and does not support
The sources support a limited set of statements. The vendor says its connector spans intake among other record types, that a majority of early users are small firms, and that results may differ between its native AI and the Claude route, with citations absent in the latter. The Artificial Lawyer piece supports the general argument that visible screening matters when verification cannot scale, in the context of legal research. The RefCraft abstract supports only the claim that, in one webpage-development benchmark, the authors found a low top score.
They do not show that any connector improves intake performance, recovers revenue, or produces accurate leakage figures. They do not show how often interfaces disagree on the same question; Carter said only that they could. And none of this tells you whether your firm has a leakage problem. That answer lives in your phone logs, forms, calendars and billing records.
The practical takeaway
Our suggestion for a managing partner is to treat an agent's intake summary the way a careful partner treats an associate's first-draft memo: a starting point that must be traceable, with its boundaries stated. Build the ground truth first. Then let the agent compete against it. If the agent's counts match your reconciled list, with identifiers you can open and exclusions it can name, that is a reasonable basis for some confidence in that interface for that question. If they do not, you will have learned something about the tool and, possibly, something about gaps in your own capture process. Either finding is worth having.