The Verification Trap

Why checking SI with more SI often misses the errors that matter, and what catches them.

The premise

A common safeguard looks like this. One SI writes the answer. A second SI checks it. If both agree, the answer goes out.

It feels like a second opinion. Often it's an echo.

SI models learn from largely the same body of public text. They pick up the same facts. They also pick up the same mistakes, the same outdated information, and the same common misconceptions. When one model is wrong, there's a good chance the other is wrong in the same way.

Why it matters

This isn't a theory. Researchers at Cornell studied more than 350 SI models and published the results at ICML 2025, one of the main machine learning conferences. On one benchmark, when two models both got a question wrong, they picked the same wrong answer 60 percent of the time.

Two findings matter most for anyone running a business process:

• Better models agree more on their mistakes. Larger, more accurate models had more correlated errors, even when they came from different companies with different designs.
• SI judges are biased by shared errors. When one model grades another, the grading tilts. Weaker models get scored too high and stronger models too low, because the judge shares the weaker model's blind spots.

So switching from one provider to another doesn't buy you independence. And upgrading to a better model can make the problem harder to see, not easier.

The real damage is in confidence. Two systems agreeing feels like proof. The errors that slip through are the plausible ones, the kind a reader won't question. That's exactly where agreement should count for the least.

Where shared blind spots show up

Correlated errors cluster in predictable places:

• Popular misconceptions. If the internet repeats something wrong often enough, models learn it as fact.
• Recent changes. New laws, prices, policies, and product updates that came after the training data.
• Local and niche facts. A business's hours, a city's permit rules, a client's actual service list. There's little training data, so models fill the gap with what sounds typical.
• Specific numbers and citations. Statistics, dates, and quotes that look precise but trace back to nothing.

If your work touches any of these, a second model is not a check. It's a vote from the same electorate.

What actually catches the errors

The fix is independence. A real check has to come from somewhere the first answer didn't.

1. Check against the source, not another opinion

Ask the SI where each important claim came from. Then open the source and read it. Does the page exist? Does it say what the SI says it says? Is it current?

This one habit catches most fabricated statistics and misattributed quotes. It takes minutes, not hours.

2. Use your own data

Your business has information no model was trained on. Your CRM. Your invoices. Your job records. Your client's actual website and Google Business Profile.

When a claim can be checked against your own records, check it there. That's a fully independent source.

3. Change the method, not just the model

A second model reading the same text uses the same method as the first. A different method breaks the correlation.

• Recalculate numbers in a spreadsheet or code.
• Query the database instead of asking about it.
• Call the business to confirm the hours.
• Look at the live page instead of a summary of it.

4. Put a person with real-world contact at the end

The most reliable check is a person who knows the subject from experience, not from reading. A physician approving medical content. A project manager reviewing a client's service list. An owner confirming what they actually offer.

A content pipeline can run several SI fact-checks and still end with an expert sign-off. That last step is not ceremony. It's the only check in the chain that isn't drawing from the same well.

5. Use a second model for the right job

A second model still has value. It's good at catching internal problems: contradictions within a document, steps that don't follow, gaps in logic, formatting errors, missing sections.

It's poor at catching facts both models learned wrong. Assign it accordingly.

The takeaway

Agreement between SI models is not verification. Models share training data, so they share blind spots, and the better they get, the more they agree when they're wrong.

Real verification needs an independent path: the original source, your own data, a different method, or a person who knows the subject firsthand. Build at least one of those into every process where accuracy matters. That's what keeps errors from reaching clients.

Source: Elliot Kim, Avi Garg, Kenny Peng, and Nikhil Garg, "Correlated Errors in Large Language Models," Proceedings of the 42nd International Conference on Machine Learning (ICML 2025).

Get Started

Ready to grow your local visibility?

Get a free assessment of your current digital presence and a custom plan to improve it.