Forge's repository describes a framework for self-hosted tool-calling and multi-step workflows. That description does not establish a general success rate for small models or show that one model size is best for a law firm's work. We have removed the article's unqualified 53% and 99% claims; they were not established as representative production results.
Define success before comparing models
Our recommendation is to choose a bounded task and a representative set of examples. Include missing information, conflicting instructions and an unavailable tool. Define what counts as a correct completion, an appropriate refusal and an escalation. Keep those outcomes separate in the scorecard.
Compare complete configurations
Run the same examples against the candidate configurations. Record the model, prompts, tool permissions, retrieval settings and retry policy. If more than one variable changes, do not attribute the difference to model size alone. Keep latency, usage cost and staff review time alongside completion quality.
Inspect failures rather than averaging them away
For each failure, preserve enough context to understand the cause without unnecessarily retaining sensitive data. Ask whether the remedy belongs in the model, the integration, the input data or a deterministic permission check. A retry is not a remedy for an action that should never have been authorized.
A proposed acceptance test
For a workflow the firm is considering, write down the permitted task, the identity it runs under and the records it may access. Mark the actions that need a person to approve them. Include a stop condition and an identified staff owner for escalation. These are implementation recommendations, not evidence that the firm currently has a problem.
Test a normal request, an ambiguous request and a request outside the permitted task. Inspect the actual records and logs rather than relying solely on the agent's explanation of what it did. Keep the test and expected result so it can be repeated after changes.
Make the comparison specific
Ask the implementer to show the behavior on the proposed configuration. Distinguish what is enforced by software from what is merely requested in a prompt. Record unresolved limits and decide whether the remaining work and risk are acceptable before expanding access. A demonstration is evidence about that test, not a promise about every future interaction.
Editorial correction: unsupported or unavailable-source claims were revised. Original publication date retained; this correction does not certify every other article in the archive.