Skip to main content
← Back to Blog
AI Visibility & Reputation
AI-assisted · automated editorial review

Which Website Requests Should Your Law Firm Allow, Review or Block?

|AlphaForge Editorial|7 min read
AI VisibilityLaw Firm OperationsWebsite SecurityPrivacy

If your intake coordinator reports junk submissions and your web provider flags automated traffic, you may need clearer evidence before changing website access. How should your law firm decide which requests to allow, review or block?

We recommend verifying claimed crawler identities where possible, grouping requests by attempted actions, and tracing form-processing outcomes. Preserve uncertainty in the findings. Base each proposed control on documented activity and the website function it affects.

What the available guidance establishes

The evidence for this article consists of supplied excerpts from Google crawler-verification guidance and two OWASP guidance pages, accessed November 15, 2026. The source class is publisher guidance, including Google's guidance about its own services. Publication dates and firm-specific measurements were unavailable.

Google explains that requests claiming to come from Google can be checked. Its methods include matching the requesting IP address, the network address associated with the request, against published crawler and fetcher address ranges. Google specifically discusses concerns about troublemakers claiming its identity. See Google's guidance on verifying requests from Google crawlers and fetchers.

OWASP explains that application logging provides event context and outcomes that infrastructure records may lack. It also warns against collecting too much or too little. Information arriving from other trust zones can be missing, modified, forged or malicious. These limits appear in the OWASP Logging Cheat Sheet.

OWASP recommends identifying relevant abuse categories and affected website functions before adding controls. Its stated objective includes preserving access for legitimate users and bots while addressing abusive automation. See the OWASP Bot Management and Anti-Automation Cheat Sheet.

Our interpretation is that this guidance could support a structured website audit. Firm-specific records would still be needed to establish traffic patterns, form abuse or effects on inquiries. AI citations and retained matters would require their own evidence.

Verify identity before assigning trust

We recommend treating a crawler name in a traffic report as an identity claim requiring verification. The audit should record how the claimed operator was checked.

We recommend recording four identity states. Use verified through an applicable provider method, claimed but unverified, unknown, and inconsistent with the claimed identity. Record the verification method and date alongside each result.

For requests claiming a Google crawler identity, ask your web provider to apply Google's applicable procedure. Google's guidance distinguishes crawler and fetcher address ranges from its general address list. We recommend documenting which list was used and which identity it supports.

For another named crawler, check whether its operator supplies a verification method. If available documentation cannot substantiate the identity, retain the unverified label. The supplied sources provide Google-specific verification guidance.

OWASP's logging guidance includes events from trusted systems and search robots among those to retain. We recommend reviewing what a verified requester attempted and the recorded outcome alongside its identity.

Group requests by action and outcome

We recommend choosing a defined review period based on available retention and recording any gaps. Ask your provider for relevant website-server records, traffic-filtering records and application events. Have the provider explain which questions each record can answer.

We recommend adjusting the following request categories to the firm's actual website.

  • Public-page retrieval. We recommend grouping requests for practice descriptions, attorney profiles, contact information and other public pages.
  • Form activity. We recommend grouping attempts to submit inquiries, request consultations or trigger other form actions.
  • Restricted-access activity. We recommend grouping requests involving login pages or administrative functions.
  • Unrecognized paths. We recommend recording requests for addresses your provider cannot associate with an intended website function.
  • Unresolved activity. We recommend retaining a category for requests the available records leave unexplained.

We recommend recording identity and action in separate fields. An unknown requester retrieving a public page and a requester repeatedly failing form validation deserve individual review. An unusual path may warrant investigation, with purpose left unresolved until further evidence is available.

Ask your provider to record outcomes such as served, rejected, challenged, limited or failed. Record a served request as an access observation. Understanding, retention or citation by a search or AI system would require additional evidence.

Trace form handling before choosing controls

For intake staff, the practical questions are whether a submission reached the intended review process and whether the evidence supports an abuse finding. A website request record alone may leave both questions unresolved.

We recommend using an interaction identifier, a reference linking events from one interaction. OWASP describes this approach for related events, including form submissions. Consider tracing receipt, validation, filtering and delivery under that reference while keeping inquiry content in its intended system.

Review repeated content, submission frequency and validation failures as signals requiring context. We recommend requiring corroborating evidence before classifying an overnight inquiry or an unfamiliar name as automated abuse.

Consider controls specific to the affected form or website function when records establish a relevant pattern. OWASP recommends rate limiting, which restricts request frequency, and layered defenses. Ask your provider to document proposed limits using observed behavior and the firm's requirements. The supplied guidance provides no universal threshold for law-firm intake.

Keep logging proportionate to the question

An estate or probate inquiry could contain family details beyond what a traffic reviewer needs. We recommend designing the audit around event summaries and processing outcomes, with inquiry-content access handled according to the firm's review process.

OWASP's bot-management guidance calls for masking credentials and personally identifiable information in logs. Its logging guidance says collection should follow the intended purpose and warns against excessive or insufficient collection.

We recommend documenting each collected field, its purpose, who can access it and how long it remains available. Check whether full form contents, information within requested URLs or authentication material appear in routine exports. Ask the firm's responsible reviewer to assess applicable privacy requirements.

We recommend restricting access to evidence copies. Where identity checks require IP addresses, document why those records are needed and how they will be handled.

A practical example of reviewing crawler traffic and form abuse

Illustrative operational example. A probate firm's provider reports unrecognized crawler labels, while intake staff report repeated junk submissions. The observations could involve different requesters. Any connection, including an effect on legitimate inquiries, would require supporting records.

We recommend checking which form submissions were received, rejected and delivered to the staff queue. Verify supported crawler identity claims and review page-access outcomes in parallel. Record any connection between these findings only when the available evidence supports it.

Before changing controls, consider the following proposed acceptance tests. Use synthetic inquiries containing no client information.

  1. Ordinary inquiry. Check whether a representative submission reaches the intended staff queue and produces a traceable outcome.
  2. Repeated automated submission. Check whether the selected control responds as configured and records its decision.
  3. Public-page access. Check whether chosen restrictions preserve access under the provider's documented crawler policy.
  4. Logging review. Check whether test credentials and inquiry text are masked or omitted as intended.

We recommend recording each test's configuration, date, expected behavior, observed result and unresolved exceptions. These proposed checks would need to be performed and reviewed before drawing conclusions about the configuration.

For a related question, we recommend reading our guide to checking what AI agents can retrieve about your law firm.

What to measure next

We recommend a recurring report showing request categories, identity-verification coverage, form-processing outcomes, control decisions and gaps in available records. Assign an owner to unresolved findings and record when each will be reviewed.

Compare equivalent periods and note configuration changes before interpreting shifts. A larger unknown category might reflect missing verification or incomplete records. A decline after filtering might reflect changed collection or enforcement. Each explanation would require checking against the underlying records.

Our interpretation is that this audit could give the firm a clearer basis for website-control decisions. Evaluate crawler access, AI mentions, inquiries, consultations and retained matters using evidence appropriate to each. Connecting a control change to visibility or business outcomes would require further measurement.


AI Visibility & Reputation

Start with the problem in front of you.

The report is public-evidence based. Connecting visibility to operations and revenue requires the Firm Intelligence Audit.

Liked this post?

Get research, operating guides, and implementation notes once a month. No spam.