In normal software, the same question always gets the same answer. Ask a calculator for 2 + 2 and you get 4, every time. So we write tests that check for exactly 4.
AI doesn’t work like that. Ask it to summarise a customer complaint twice and you get two different summaries. Both might be fine. So how do you test something that never gives the same answer?
You stop checking the exact words, and start checking the qualities.
Say your AI summarises complaints for a support team. A good summary should:
- be under 50 words
- mention the refund amount if there is one
- never invent an order number
Each of those is a simple yes or no check. That list of checks is called an eval. It’s just a test with a different shape.
The habit that carries over from good engineering is this: write the checks first. Before you touch the AI, collect 20 real complaints and decide what a good summary looks like for each one. Then every time you change the prompt or switch to a newer model, run all 20 again. If something that used to pass now fails, you know straight away, instead of hearing it from a customer.
If you wouldn’t ship code without tests, don’t ship an AI feature without evals.