The Shadow AI Audit: A 5-Step Playbook to Find Agents Your Team Already Built

As of July 2026, ask ten local business owners how many AI agents are running inside their company and most will answer "one — the chatbot on our website" or "zero." The real number is almost always higher, and IT rarely built any of it. This week two stories from the developer world made that gap impossible to ignore: Tines shipped a new product, Tines 3B, built specifically for what its founder called the moment "when everyone builds software" — finance, marketing, and ops staff already stitching together agents and dashboards in tools like Claude Code, with no security review in sight. That launch drew real attention on Hacker News (22 points, discussed by developers who recognized the pattern in their own companies). Meanwhile, a formally verified 3D geometry project made the opposite case: its author trusted a 93-line written specification over 1,000 lines of AI-generated code, and the spec caught what the code got wrong — a reminder that "it runs" and "it's correct" are two different claims.

Why this matters

Put those two stories together and you get the actual risk facing small and mid-size businesses right now: agents are being built faster than they're being checked. Your bookkeeper might have an agent auto-categorizing expenses. Your marketing coordinator might have one drafting and scheduling social posts. Your front desk might be using a voice agent nobody in leadership approved. None of that is inherently bad — agentic tools are genuinely useful, and stopping the practice isn't realistic or desirable. But an agent with no spec, no owner, and no audit trail is a liability wearing a productivity hat. The gap isn't technical skill; it's that adoption moved faster than oversight, and oversight is the part nobody budgets time for.

There's a second cost hiding in the same news cycle. Moonshot AI's Kimi K3, a 2.8 trillion-parameter model, went live on Telnyx Inference this week — and it needed dedicated GPU infrastructure just to serve it, because a model that size can't run on general-purpose hardware. That's the scale conversation we've made before: chasing frontier-model performance in-house means underwriting infrastructure most local businesses will never fully use, let alone maintain. We covered why that math doesn't pencil out for most operators in our breakdown of what a 2.8 trillion-parameter model proves about building your own AI stack. The same logic applies at a smaller scale to shadow agents: every unmanaged tool your team adopts is a small, recurring bet that nobody is tracking the return on, and those bets compound the same way infrastructure costs do.

The 5-step shadow AI audit

1. Inventory every tool with billing or API access. Pull the last 90 days of credit card and corporate card statements and flag every AI-adjacent charge — OpenAI, Anthropic, Telnyx, Tines, Zapier, any line item you don't immediately recognize. Cross-check against your SSO/Okta login list and browser extension inventory. Most shadow agents leave a paper trail in billing long before anyone admits to building them, because someone had to pay for the API calls even if they never told a manager why.
2. Interview each department, not just IT. Ask finance, marketing, and ops directly: "What have you built or automated with AI in the last six months?" Frame it as discovery, not discipline — the Tines 3B pitch is right that most of this activity isn't malicious, it's just unsupervised. You'll get more honest answers if people aren't afraid of getting written up for solving a problem on their own initiative.
3. Write a one-page spec for every agent you find, then check the agent against it. This is the lesson from the verified-CSG project: a short, plain-language spec — what the agent is allowed to touch, what output counts as correct, what happens on failure — will surface mismatches that "it seems to work fine" never will. You don't need Lean 4 or formal verification; you need one page per agent stating what "correct" means, and someone other than the builder pulling the agent's last 20 outputs and checking them line by line against that page.
4. Decide build-vs-buy per agent, not once for the whole company. Some shadow agents are fine to keep in-house — a simple expense-categorizing script has a low blast radius and a cheap failure mode. Others — anything customer-facing, anything touching pricing or legal language, anything that could misfire in public — belong with a team that owns uptime, model costs, and monitoring as a full-time job. Weigh each one against the actual infrastructure cost of running it well, the same way the Kimi K3 story illustrates at a much bigger scale: bigger and more capable isn't automatically cheaper to own.
5. Assign one governance owner and a real update cadence. Someone — not "IT" as an abstraction, one named person — should hold the list of every active agent, its one-page spec, and its last review date. Without a named owner, this audit becomes a document nobody reopens once the initial excitement wears off.
6. Re-run the audit quarterly. New agents get spun up faster than policies get written, especially as no-code and low-code agent builders keep getting easier to use. A quarterly 90-day statement pull and department check-in keeps the inventory from going stale.

Shadow AI audit checklist

• 90 days of card statements pulled and flagged for AI-related charges
• SSO login list and browser extensions cross-checked against billing
• Every department interviewed about tools they've adopted
• One-page spec written for each agent found
• Last 20 outputs of each agent checked against its spec
• Build-vs-buy decision documented per agent, with cost justification
• One named governance owner assigned
• Next review date set on the calendar

Common pitfalls

The most common mistake is assuming that if nobody wrote code, there's no risk — a marketing coordinator chaining together no-code agent tools has the same exposure as a developer who wrote a script, just less visible because there's no repository to point to. The second is treating the audit as a one-time event; agent adoption inside a company moves faster than most audit calendars, so a single sweep goes stale within a quarter and gives false confidence in the meantime. The third is auditing for existence but not correctness — finding the agent isn't the win, checking its last month of outputs against a written spec is, and skipping that step means you've documented risk without actually reducing it. And the fourth is letting a customer-facing agent run without anyone accountable for what it says in public, which is exactly the kind of unmanaged surface that costs businesses trust and, increasingly, visibility when AI assistants like ChatGPT or Perplexity are asked to vouch for a company.

That last point connects directly to how customers already find you. If an AI assistant can't verify what your business actually does — or worse, surfaces something a shadow agent published without review — that's a visibility problem as much as a governance one. Start by finding out where you stand with a free AI Visibility Report.