Skip to main content
← Back to Blog
AI Visibility & Reputation
AI-assisted · automated editorial review

5 Checks on What AI Agents Can Retrieve About Your Law Firm, Run This Month

|AlphaForge Editorial|7 min read
AI VisibilityLaw Firm OperationsAI AgentsReputation ManagementAI Implementation

As of October 2026, one design change is worth a managing partner's attention. Some AI assistants used by lawyers are being wired to external data through connectors, so an answer may come from a retrieved record rather than from what a model absorbed in training. This article asks one question: if that pattern spreads, what should a firm check about the information agents can reach, and about how they attribute it? The sources below are news reports, a vendor-described product, a project README excerpt, a blog post, a vendor research post and a preprint. They support design lessons. They do not show what any firm's visibility looks like today.

What the reporting actually says

LawNext reported in August 2026 that Trellis, a legal research company, extended access to its state trial court data through a ChatGPT plugin and its own chat tool, in addition to an MCP connector for Claude launched in May. These are company claims, relayed by a news outlet. Trellis says it holds more than 2.5 billion records. It describes connectors that pull filings, rulings and verdicts rather than leaving the model to answer from general training, which the report says does not include that data. It also describes a workflow in which the system plans a strategy, searches records, cross-references results against actual verdicts and motions, and checks citations before responding.

That is vendor description, not independent measurement. Nothing in the excerpt tells us how often the citation checks catch errors. Our inference, not a finding from these sources: when an assistant is connected to a data source, what the source contains and how it is structured may shape the answer alongside the model. We have not tested this.

On the standards side, Simon Willison's 22 October 2026 link post relays Anthropic's announcement that it is donating the Agent Skills standard to the Agentic AI Foundation under the Linux Foundation. Per that post, the Model Context Protocol was donated there earlier. Willison also notes that because their blog content is available to Claude through a datasette-mcp plugin, they found one of their own older posts by prompting Claude to use it. That is a single anecdote about retrieval through a connector, not evidence of how often it works. Our reading, offered as an inference: this is one public signal that connector conventions are moving toward shared governance. It says nothing about adoption among law firms or buyers.

Why this matters for visibility and reputation

Our working hypothesis, not an established finding: as more assistants retrieve instead of recall, a firm's description in an AI answer depends partly on which sources the assistant can reach and which it prefers. Those sources might include the firm's own site, directory profiles, bar listings, review pages and court records. A firm cannot assume that an answer it saw once in one tool represents what other tools say. Whether any of this affects inquiries or revenue for a particular firm is something only that firm's first-party data can show. We make no such claim from public sources.

Pablo Arredondo, who announced a move to Clio as senior vice president, judiciary, with a start date of Aug. 31, offers one related observation in a LawNext interview published in September 2026. In the episode, recorded just ahead of the announcement, Arredondo describes a torrent of material reaching the courts. He says that on Monday, somewhere on the order of three billion tokens are going to hit court dockets. He also discusses how AI is driving a surge in pro se filings. That is one person's characterization in a podcast, not a measured statistic, and the excerpt does not say the volume recurs weekly. Our inference, offered as a hypothesis: public records, one input to retrieval systems, may contain more machine-generated text over time. A firm that appears in dockets, directories or reviews should expect those records to be read by software at some point.

Three design lessons from sources outside the legal sector

1. Attribution is a field, not a feeling

The README for OneMemory, a self-hosted memory store that agents can share over MCP or REST, is a project's description of itself. We have read only an excerpt and have not tested the software. The excerpt makes a useful design distinction. It says each note carries a server-stamped written_by label. That label is the authenticated token's label for the call or, for writers with no token, a fixed marker such as stdio, cli, server or runner, and a client cannot set it. It says the older source field is only what the writer claims, so anyone can put anything in it.

We do not recommend a firm adopt that tool on the strength of a README excerpt. We recommend asking the same question of any assistant that answers questions about you: can you tell whether a statement came from a record you control, a third-party listing, or the model's own inference? If a tool cannot show provenance, you cannot correct the source.

2. Verification works best when the answer can be checked exactly

A 22 October 2026 arXiv preprint on quantum circuit optimization with AI agents describes a method in which a frozen, exact verifier certifies every result. The abstract says the verifier, together with the challenge around it, is built by an agent working with a researcher, and is validated against a solved case, deliberately broken solutions and published results before any solution exists. The authors report cost reductions of up to 73% at the largest sizes tested and say the method applies to problems whose solutions can be verified exactly and efficiently. These are author claims from a preprint, in a domain unrelated to law.

We recommend borrowing the order of operations: build and validate the checker before you trust the output. Facts about a firm, such as practice areas, licensed jurisdictions, office locations and attorney names, are less clean than a circuit cost, but a firm can still write down the correct answers in advance and score responses against them. We recommend including deliberately broken answers, so you know your scoring catches a wrong jurisdiction or a retired attorney's name.

3. A plausible persona is not the same as a measured boundary

Microsoft Research's October 2026 post on StudentSim notes that a language model asked to role-play a struggling student may change its tone while still relying on knowledge and reasoning far beyond that student's level, which can make a tutor appear more effective than it is. The research concerns education and the post is Microsoft's own description of its work. Our inference, by analogy only and not something the post tests: an agent that sounds fluent and confident about your firm has not shown that it knows the boundaries of what is true about you. We recommend treating fluency as a style, not as evidence.

A five-step check a firm can run this month

We recommend running these steps. They are our suggested procedure, not a validated method, and they are designed to produce first-party evidence rather than assumptions.

  1. What can be retrieved about us? We recommend listing the public places that describe the firm: your site, bar and court listings, directories, review profiles and press. Mark which ones you can edit and which you can only request changes to.
  2. What are the correct answers? We recommend writing a short reference sheet before you test anything: services offered and not offered, jurisdictions, attorney roster, office hours, how intake works. Have someone other than the author review it.
  3. What do different interfaces say? We recommend asking the same fixed set of invented prospective-client questions in at least two assistants, on the same day. Use public firm facts and keep client identities, private matter facts and documents outside these tests. Record the date, tool, model or mode if shown, the exact prompt and the full response. Our earlier post on what a firm should test across newly launched legal AI products is a related starting point for that comparison.
  4. Is the answer retrieved or remembered? We recommend noting whether each response cites a source, whether the source opens, and whether it supports the specific claim. An uncited claim is not necessarily wrong, but you cannot trace it.
  5. Who fixes what? We recommend logging, for each discrepancy, the claim, the likely origin if identifiable, the owner and the fix. Repeat the same questions after changes and keep the earlier runs for comparison.

What this approach cannot tell you

A small question set is a sample, not a measurement of how often anything happens. Results vary by tool, by phrasing and by date, so a single clean run proves little. Corrections to a source may not change any given assistant's answer, and we cannot promise that they will. None of the sources here measures whether an AI answer about a firm affects client inquiries. Treat any claim of lost or gained matters as a hypothesis until intake records, referral sources or call logs support it.

Our inference, not a finding from these sources: if assistants answer from connected data, then the accuracy of your public records, a written reference sheet and a repeatable test become part of ordinary reputation management. The first step is simply to know what the sources say about you and who can change them.


AI Visibility & Reputation

Start with the problem in front of you.

The report is public-evidence based. Connecting visibility to operations and revenue requires the Firm Intelligence Audit.

Liked this post?

Get research, operating guides, and implementation notes once a month. No spam.