§8.1 · BUILDING · 6 MIN READ · EDITION 1.0
Software testing is not AI evaluation
A functioning interface does not demonstrate that an AI task is suitable for its intended quality use. Evaluate the real task on known, representative evidence.
A golden reference set includes clean cases, difficult cases, borderline cases, incomplete evidence and cases where the correct answer is no finding. Acceptance criteria are agreed before testing so that results cannot be reinterpreted after the fact.
Useful measures include coverage, missed findings, false findings, reviewer agreement, traceability, time-to-verify and failure response. A model benchmark is not a substitute for performance on the organisation's own documents.
PUT THIS INTO PRACTICE
- 01Build a known-correct reference set.
- 02Define false-positive and missed-finding limits in advance.
- 03Test the reviewer workflow as well as the model output.
RELATED INSTRUMENTS
CITE THIS SECTION
§8.1 · Edition 1.0 · Software testing is not AI evaluation