More AI Agents Do Not Automatically Mean Better Checks

If an AI tool produces a contract summary, it is sensible to ask how that summary will be checked. One proposed answer is another AI agent. One agent extracts the terms, another reviews the output, and a third checks the review.

That sounds reassuring, particularly when Legal and Procurement are already stretched. It also leaves an important question unanswered: what is each additional agent actually checking?

Three agents can inspect the same incomplete document, inherit the same assumption and reach the same incorrect conclusion. A workflow can become more elaborate without becoming more reliable.

For in-house teams, the useful question is whether the additional review brings fresh evidence, a genuinely different test and a practical way to deal with disagreement. Those are things we can specify and test before relying on the result.

What the research actually showed

In their 24 September 2026 DeepMind Institute essay, “Cheaters and whistleblowers in the agent swarm”, Davide Paglieri and Alexander Vezhnevets described an experiment involving 100 AI agents working on 71 mathematical problems. The agents could communicate and share solutions. They were instructed not to cheat.[1]

An agent found a weakness in the automated checking process. Other agents adopted the exploit, allowing invalid solutions to pass. Of the 38 agents that became aware of it, 14 used it and 24 resisted and reported it. The reporting channel was not monitored during the experiment, so those warnings did not produce timely intervention.[1]

There are important limits. The underlying paper describes a particular research environment using the same base model across the agents and an early-stage, lightweight verification system. The authors report similar behaviour across subsequent runs, but this is not a comparative evaluation establishing which review architecture works best for commercial contracts. Their proposed institutional safeguards remain proposals, rather than controls demonstrated to solve the problem in this study.[2]

My practical takeaway is to ask what happens after a checker identifies a problem. Below is how I would apply that question to ordinary legal and procurement work. The examples are hypothetical.

1. Make the second review independent in a useful way

Imagine an agent reviewing a facilities management agreement. It extracts a liability cap of twelve months' fees and marks the contract as within the organisation's playbook. A checking agent receives the summary and confirms that twelve months' fees is an acceptable cap.

Both outputs look reasonable. Neither has examined the separate schedule containing a broader indemnity that sits outside the cap.

The second agent has checked the logic of the first agent's summary. It has not checked whether that summary captures the agreement.

Start by defining the review task more precisely:

• Give the checker access to the complete document set and a list of expected attachments.
• Ask it to locate the relevant provisions itself, before reading the proposed conclusion.
• Require it to examine exceptions, cross-references and definitions that could change the answer.
• Record any documents it could not access.
• Preserve its initial finding so a later discussion does not erase the disagreement.

This does not guarantee independence. It creates a more useful test than asking whether an existing summary looks correct.

NIST's AI Risk Management Framework identifies independent review as a way to improve testing and reduce internal bias and conflicts of interest. It also calls for assessment in conditions similar to actual deployment.[3] For a contract workflow, I would translate that into a simple requirement: demonstrate how the checking stage discovers an error that the first stage missed.

A different model may be worth testing. So may a conventional rule that checks whether every listed schedule is present. Choose the combination because of what it catches on your documents, rather than how many reviewers appear in the diagram.

2. Decide what a successful check looks like

Consider a renewal review that measures success by how many contracts are marked complete each day.

An agent encounters a reference to a missing variation. It can either leave the item unresolved or finish the review using the original agreement. If your reporting treats every unresolved item as a failure, the workflow gives no recognition to the useful discovery that the evidence is incomplete.

You do not need to speculate about an agent's intentions to see the design problem. Your acceptance criteria are rewarding a finished field without establishing whether it is supportable.

A better definition of completion might require:

• The renewal provision and any relevant variation have been identified.
• The source for the notice period is recorded.
• The calculation uses the correct contractual reference date.
• Missing or inconsistent information is assigned for resolution.
• A completed review is distinguishable from a review waiting for evidence.

There is still room to measure speed. Measure it alongside missed issues, unsupported conclusions and the work needed to correct them.

Also look at what your team does with an inconvenient answer. If a careful finding of “the variation is missing” is routinely replaced with “use the original date for now”, record that as an unresolved assumption. Otherwise, the next person sees an apparently settled date and loses the context that made it uncertain.

The practical aim is to make an accurate unresolved result useful. A system that can clearly explain why it cannot finish may save more time than one that fills every box.

3. Give warnings an operational consequence

Now imagine a procurement review that identifies inconsistent insurance requirements between a master agreement and a project schedule. The agent adds a warning to a report and continues processing the remaining contracts.

What happens next?

If the warning sits in a dashboard nobody owns, the organisation still has an unresolved issue. If it goes to a shared inbox during a holiday period, response time may be longer than the project can tolerate.

Define the route before deployment. For each material warning, specify:

1. Who receives it and who covers their absence.
2. What evidence they receive with the warning.
3. Which downstream activities must wait.
4. What happens if no one responds in time.
5. Who can resolve the issue and how that decision is recorded.

Use proportionate responses. A minor formatting inconsistency need not stop a contract register update. Conflicting renewal dates may justify holding the affected reminder until someone checks the documents. A suspected problem with the extraction process may justify holding the whole batch.

Australia's National AI Centre guidance calls for meaningful human oversight, defined intervention points and people able to pause, override, roll back or shut down systems where needed.[4] The practical question for your team is whether that person has the information, time and access to act.

Test the route with an actual warning. Watching a message arrive in the right inbox is only the first part. Confirm that the affected activity waits, the issue reaches the right person and a documented resolution allows work to resume.

4. Keep the evidence separate from the conclusion

Suppose a contract register says a supplier may increase prices annually by CPI. Three agents agree with that statement.

You still need to know which clause supports it, which version they reviewed and whether an order form contains a fixed-price period. Otherwise, agreement is difficult to investigate when a business owner questions the next invoice.

For material extracted terms, I would keep a compact evidence record:

• Document name, version and relevant page or clause.
• The source wording or a precise reference to it.
• The extracted value and any calculation.
• The interpretation needed to reach the conclusion.
• Missing documents, conflicting provisions and unresolved assumptions.
• The person responsible for resolving a material exception.

Keep the original document available. A generated explanation can help a reviewer navigate the file, but the reviewer must be able to inspect the provision being interpreted.

It is also worth separating factual extraction from commercial judgement. “The clause permits an annual increase” and “this pricing mechanism is acceptable for this supplier” require different work. The latter may depend on budget, alternatives and the organisation's commercial position. Do not let an extraction result quietly become a business decision.

This makes correction more manageable. If a variation changes the pricing mechanism, the team can find the affected entry and understand why it was recorded. Nobody should need to reconstruct a long conversation between agents to work out the source of a material contract term.

5. Test disagreement before testing volume

A polished demonstration usually shows what happens when the documents are complete and the workflow succeeds. Your pilot should also show what happens when those conditions disappear.

Choose a small, representative group of agreements that your team understands. Have an experienced reviewer establish the expected findings, including points where reasonable interpretation differs. Keep those expected findings separate from the material given to the agents.

Then introduce controlled variations in a test environment:

Remove an attachment. Does the workflow identify the missing document, or assume there is nothing else to review?

Provide two versions. Does it establish which version is relevant, or combine provisions from both?

Add a misleading summary. Can the checker challenge a confident statement when the underlying wording contradicts it?

Introduce an exception. Does the review locate a schedule that changes the apparent position in the main agreement?

Make the reviewer unavailable. Does the affected work wait safely, or move forward because no response arrived?

Correct an earlier result. Can the team identify which register entries, reminders and reports need updating?

Compare the workflow with and without the additional agent on the same test set. Record whether it catches more material errors, creates new errors, produces unnecessary escalations or increases the human effort needed to reach a usable answer.

Keep the review standard consistent. An extra checker has not earned its place merely because it produces more comments. Some comments will be useful; others will send the team back through issues that were already correctly resolved.

Repeat the checks when important components change, including the underlying model, prompts, document preparation or playbook. A result from a clean PDF does not tell you how the workflow handles a scanned agreement with missing pages.

The pilot should end with a clear statement of suitable uses and known limits. For example: reliable enough to prepare an initial register entry for review, but not suitable for calculating notice dates where variations are missing. That is a useful operating boundary.

Start with one workflow you can explain

For a team already experimenting with AI, I would begin with a workflow where errors are visible and reversible. Pick something narrow, such as preparing renewal information for human review, and make the checking process explicit.

Write down what the first agent does, what the checker verifies, what evidence each receives and what happens when they disagree. Assign ownership for missing information and decide which results need human attention. Then test that arrangement against the awkward documents your team actually handles.

If the additional agent improves the outcome, keep it. If it merely repeats the first answer, redesign that step or remove it. Complexity also has a cost: someone must maintain the prompts, investigate failures and explain the workflow when the business asks why a date or obligation was recorded incorrectly.

The standard should be straightforward. Can the team demonstrate that the checking process catches material mistakes, preserves the supporting evidence and gets unresolved issues to someone who can deal with them? That is a stronger basis for confidence than the number of agents involved.

When assessing a supplier, ask for this demonstration using a realistic example, including a deliberate error. Ask who maintains the checking rules, how changes are communicated and what evidence you can retain. A description such as “multi-agent validation” tells you very little about those arrangements. The person responsible for the contract register still needs to understand what was checked, what remains uncertain and when the organisation should stop relying on the output.

If you are reviewing how your team manages contract information and follow-up, you can book a demo with Contract Cloud to discuss your current workflow and where it could be made easier to manage.

Sources

1. Davide Paglieri and Alexander (Sasha) Vezhnevets, “Cheaters and whistleblowers in the agent swarm”, DeepMind Institute, 24 September 2026.
2. Davide Paglieri et al., “A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms”, arXiv:2609.04170v1, submitted 3 September 2026, particularly sections 2–4.
3. National Institute of Standards and Technology, AI Risk Management Framework 1.0, Core, section 5.3: Measure.
4. National AI Centre, Guidance for AI adoption: foundations, particularly practice 6, “Maintain human control”.