Testing AI-enabled features like production software

AI-enabled features behave differently from deterministic code. The same input can produce different outputs, quality can degrade without any change to the code, and failures are often plausible rather than obvious. Teams that test these features with the habits they use for ordinary code will miss important problems.

Test the behavior, not only the code

Unit tests still matter for the surrounding logic: parsing, validation, permissions and error handling. For the model-driven part, build an evaluation set of representative inputs with expected properties. Score outputs against those properties and track the results over time.

Make changes visible

• Version prompts, retrieval settings and model choices alongside the code that uses them.
• Re-run the evaluation set on every change, including model updates a provider makes behind the same model name.
• Record cost and latency next to quality scores, since a better answer that costs many times more may not be the right trade-off.
• Keep a sample of production outputs for human review each week.

Design for the wrong answer

Assume that some outputs will be wrong. Show sources so people can check them, make correction easy, route low-confidence cases to a person and log feedback. A feature that fails safely earns more trust than one that is occasionally brilliant and never explains itself.