Most clinical AI evaluation goes wrong in the same place. The buyer asks whether the tool works, the vendor answers with a performance figure, and both parties move on as though a number answered the question.
It did not. A performance figure describes what happened in a study, and the study is almost never the setting you are buying for. Here are the questions that actually discriminate.
1. Is it a regulated device, and for what exactly?
Start here, because it determines everything downstream. If it is cleared, the intended use statement defines the boundary: which condition, which patients, which setting, which role. Using it outside that is off-label, and the software will produce output regardless.
If it is not a device ... which covers most physician-facing tools ... then no regulator has looked at it and the entire evidence burden is yours.
2. Was it validated externally?
The single most informative number, and the one most often absent. Internal validation on held-out data from the same source systematically overstates performance, because that data shares the training set's scanners, protocols, population, and labeling conventions.
If the answer is no, the honest position is that the tool has been shown to work in one place.
3. Was it validated prospectively?
Most authorized devices were cleared on retrospective data. That is the regulatory standard, not a scandal, but it means the evidence describes how a model scores on archived studies rather than what happens when it is switched on.
Archived datasets are curated. Poor-quality studies were excluded. Case mix was enriched. The messy inputs that break models in production are exactly what got filtered out.
4. Who was in the validation population, and do they look like your patients?
Ask for the demographic and clinical breakdown, not a summary. Age distribution, sex, race and ethnicity, comorbidity burden, disease severity. Performance differences across groups are common and are rarely volunteered.
5. What equipment was it validated on?
For imaging, this means scanner manufacturers, models, field strengths, and reconstruction protocols. Dataset shift is the dominant cause of real-world performance loss, and its defining feature is silence ... no error, no flag, just quietly worse output.
6. What is the performance at the threshold you will actually run?
Not AUROC. Sensitivity and specificity at the shipped operating point, and whether that threshold is adjustable. The threshold is a choice the vendor made, and it is usually the one that makes the marketing figure look best.
7. What prevalence do the reported predictive values assume?
This question ends more sales conversations than any other. Positive predictive value depends on prevalence, and a model evaluated on an enriched 50/50 dataset has a PPV that will not survive contact with real screening prevalence.
Compute what the false positive volume will be in your population, in cases per week. Then ask who handles them.
8. What does it do when it does not know?
A well-designed clinical tool has an abstain path. It should be able to return insufficient image quality, indeterminate, or outside indicated use rather than producing a confident output on an input it cannot handle.
For generative tools, this is the question. A system that will always produce a fluent answer will produce one when it should not, and a fabrication is indistinguishable from a correct answer in its presentation.
9. How does it fit the workflow, concretely?
Not whether it integrates. Where does the output appear, how many clicks does it cost, and what happens if the service is unavailable. Imaging AI deployments fail on integration far more often than on model quality, and the integration effort is routinely underestimated relative to the evaluation that preceded it.
10. How will you know if it stops working?
Model drift is silent. Scanners get replaced, protocols get updated, referral patterns change, and performance moves without any alarm. A deployment with no monitoring plan is not stable, it is unobserved.
Ask what the vendor monitors, what you can monitor, and what the escalation path is when a metric moves.
11. Can it be updated, and will you know when it was?
If the product operates under a predetermined change control plan, the model can change within pre-agreed bounds without a new submission. That is reasonable, and it means the device in your department may not be the device that was validated. Know which version you are running.
12. What happens to your data?
A business associate agreement permits processing on your behalf. It does not automatically permit the vendor to train its models on your patients' data. That distinction lives in the contract, not in the regulation, and it is worth reading the clause rather than accepting the summary.
The question underneath all twelve
What is the comparator? A great deal of medical AI is evaluated against nothing, or against a strawman. If a device improves on unassisted reading, ask whether it improves on the existing clinical decision rule, the existing workflow, or simply reading the study sooner.
Sometimes the honest answer is that the current process is already adequate and the tool solves a problem you do not have. That is a legitimate evaluation outcome and it is reached far less often than it should be.