10.10 Error analysis and responsible use

After evaluating overall performance, inspect where errors occur. Compare performance across time periods, locations, service types, and groups that matter to the decision. A model that performs poorly for a small facility may be unsuitable for facility-level staffing even if system-wide performance is strong.

Ask whether the model could create unfair treatment, whether a feature acts as a proxy for sensitive information, and whether affected people have a reasonable way to correct inaccurate data or decisions. Prediction does not remove the need for human judgment and governance.

Common mistake: Selecting the model with the best single validation score without considering uncertainty, subgroup performance, interpretability, implementation cost, and the consequence of error.