Glossary
Internal Validation
Also called: held-out validation, split-sample validation
Evaluating a model on data held out from the same source as its training data.
Internal validation splits one dataset into training and test portions. It detects overfitting and gives a first performance estimate, and it is the minimum acceptable evaluation.
Where This Gets Misread
It systematically overstates real-world performance because the held-out data shares the training data's scanners, protocols, population, and labeling conventions. A model that only reports internal validation has not been shown to work anywhere except where it was built.