10.2 A defensible modelling workflow

1. Define the question, target, unit, and prediction time.
2. Create an eligible analytical population.
3. Separate training and final evaluation data.
4. Develop preprocessing using training data only.
5. Establish a simple baseline.
6. Fit and tune candidate models using appropriate resampling.
7. Compare metrics linked to the decision.
8. Evaluate the selected model on untouched data.
9. Inspect errors and subgroup performance.
10. Document interpretation, limitations, monitoring, and implementation.

For time-dependent data, random splitting may allow future information to influence the past. For repeated observations from the same member, site, or event, group-based splitting may be required to prevent the same entity from appearing in both training and evaluation sets.