If you're hiring an AI implementation consultant, the first thing I'd ask isn't which model they use. Ask them to show you how their systems catch a wrong answer BEFORE it reaches a customer. If they can't point to a real check inside something they've built, keep looking.
That's the one reason I'd give for hiring Agnostify, and I'd rather show it than claim it. So this post walks through a build of ours that anyone can open, and the checks inside it.
Why is checking the part that matters?
I think most owners already know AI can draft an email or sort a lead list. What they can't tell is when it's wrong. I'd want to see how wrong prices and counting errors get caught.
In my view, getting something you can trust when nobody's watching is the real work, and that's what you're paying a consultant for: picking the one task worth automating, wiring it into the tools you already use, and putting checks around it so mistakes get caught.
What does a real check look like?
Our public example is Who AI Sends Buyers, the weekly board at agnostify.com/ai-picks. Agnostify runs it. Each week it asks ChatGPT, Gemini and Google AI Mode the same buyer questions and records which brands each engine names.
That sounds simple. The hard part is counting correctly, because some brand names are ordinary words. Streak is a CRM, and "streak" is also just a word. Loops is an email tool, and "loops" turns up in plenty of answers that have nothing to do with it.
So the board's published method says, word for word: "Names that are ordinary words, such as Streak, Loops and Instantly, count only when the answer clearly means the product."
A few other checks sit in the same method:
- A reviewed brand list. Names are matched to a list a person has reviewed, and a brand only counts on questions in its own category. A CRM mentioned as an integration in a meeting-notes answer doesn't score on the meeting-notes board.
- A person adds new names. In the method's words, "A name that is not on the list stays off the board until a person adds it." The AI doesn't get to decide by itself that something new is a brand.
- A minimum sample for the engine comparison. Our published ChatGPT vs Gemini face-off shows no engine figure from fewer than 10 runs.
- No fake precision. Where a name sits inside an answer is not treated as a rank.
- Honest limits. The page says plainly that a logged-out session has no memory and no personalization, so someone signed in can see a different answer.
None of these rules are clever. They're the boring ones that stop a counting job from quietly drifting, and boring is what you want around anything that touches your customers.
Do those checks change the numbers?
They decide whether the numbers mean anything. On the Who AI Sends Buyers board as of November 9, 2026, Streak was named in 4% of ChatGPT's scored CRM answers and 20% of Gemini's. Loops sat at 19% on ChatGPT and 5% on Gemini for email and newsletters. (Method: the same 30 buyer questions asked each week in logged-out US sessions, scored as a share of answers over a rolling four weeks.)
Those are exactly the names a lazy script would overcount by searching for the word. A 16 point gap on Streak is only worth reading if each of those mentions really meant the product. The rule is what earns the number its place on the page.
Does Agnostify check anything besides the board?
We also run Context Pill, which publishes a checked list of new AI releases. Our company log notes it put out that list for its 24th scanner day on November 8, 2026.
Two notes from the November 11, 2026 list show the habit. Anthropic says a team of 1,000 Fable 5.2 agents fully formalized Fermat's Last Theorem in Lean in 30 hours; the Context Pill note calls that a data point that "comes only from Anthropic." On a model advertised at 2,100 tokens per second, the note says "The headline 2,100 tokens per second is a vendor peak figure."
That's what I mean by checking. Nobody can promise an AI that's never wrong. What a careful build can do is mark where each claim came from and how much weight it can hold.
What would those checks look like in my business?
Our bet is that the same habits carry straight over to the work a small service business would hand to AI. Here's how I'd translate them. These are illustrations, not client results:
- A lead-response assistant gets a fixed list of what it may quote, like your services and hours, and passes the chat to a person when a question falls outside that list.
- A lead-sorting tool gets a written rule for the ambiguous cases, the way the board handles "Streak", with a handoff to a person when the case is ambiguous.
- A weekly report shows how many records sit behind each number, and holds a figure back when there's too little data.
- Anything a customer will see gets a person who approves it first.
Custom AI builds with us also come with a 30-day warranty, per our Services page as checked on November 11, 2026.
When is Agnostify the wrong fit?
Straight answer: if you need a password manager, a team chat app or anything whose value is its security and its network, keep paying for the established product. Rebuilding those is a bad trade, and I wouldn't advise it.
For almost anything else, from a chatbot that answers your customers to a custom tool built around how your team already works, we can build it. If you've got one in mind, book a call at https://agnostify.com/book and tell us what you want it to do.
What should I ask on a consultant call?
Whoever you talk to, I'd bring these four:
- Show me one thing you built that checks its own output, and walk me through a mistake it caught.
- What happens when the AI isn't sure, and who gets the handoff?
- Which numbers in your reports have a minimum sample behind them?
- Do I own the code and the accounts when we're done?
I'd weigh the answer to the first one most. If they open a live build and point at the check, you've learned how they work.
If you'd like to hear our answers to those four, book a free call at https://agnostify.com/book and bring the one task you'd automate first, so we can talk through the checks we'd put around it.