Before I ship an agent workflow for a client, it goes through one test that has nothing to do with accuracy scores or benchmark numbers. I call it the 11pm test.
The 11pm test
Picture the worst version of your customer, on the worst device, at the worst time. Typing on a cracked phone screen after two drinks, autocorrect fighting them the whole way, message split across three separate texts instead of one clean sentence. If the workflow handles that customer correctly, it’s ready. If it only handles the customer who types in complete sentences and answers questions in the order you expected, it isn’t.
This is exactly the scenario that shaped the WhatsApp order agent built for a retail client. Orders placed at 11pm, no staff online to catch a mistake, real money on the line if the agent got the order wrong. A clean demo environment would never have surfaced the actual failure modes: customers changing their mind mid-message, sending a photo instead of text, asking a question the menu doesn’t answer.
Three things the 11pm test finds that a demo won’t
Ambiguous input handling. Does the workflow ask a clarifying question, or does it guess and hope? Guessing is fine in a demo. It’s expensive in production.
Graceful handoff. When the agent genuinely can’t handle something, does it say so clearly and route to a human, or does it produce a confident answer that happens to be wrong? This is the single biggest difference between a workflow I’d put in front of paying customers and one I wouldn’t.
Recovery from its own mistakes. Real conversations aren’t linear. A customer corrects themselves. Does the workflow track that correction, or does it lock onto the first thing it heard?
Building this into your own evaluation
If you’re commissioning an agent build, or evaluating a tool that claims to handle customer conversations, don’t ask for a demo. Ask for a log of real conversations, including the messy ones, and ask what happened when the agent didn’t know the answer. That log tells you more than any pitch deck will.