Chapter 10 Analysis and Machine Learning

A useful model is selected and evaluated for a defined analytical purpose. This chapter reviews regression, classification, trees, random forests, and clustering while emphasizing validation, baseline comparison, error analysis, interpretability, and the limits of what a model can claim in a client-facing decision context.

Learning outcomes

After completing this chapter, you should be able to:

• select an analytical method that matches the question and unit of analysis;
• distinguish description, explanation, prediction, classification, and segmentation;
• construct a defensible training and evaluation workflow;
• explain the uses and limitations of regression, logistic regression, decision trees, random forests, and clustering; and
• connect model performance to the consequence of a decision.

Key terms

• Analytical plan: A documented explanation of the question, data, variables, method, validation, outputs, assumptions, and decision criteria for an analysis.

• Target variable: The outcome a supervised model is intended to predict or explain.

• Feature: A variable used as an input to a model.

• Training data: Observations used to estimate model parameters or learn patterns.

• Validation: Evaluation using observations not used to fit the model, or another defensible procedure that estimates performance beyond the training data.

• Baseline: A simple reference method against which a more complex model is compared.

• Data leakage: The use of information during model development that would not legitimately be available when a prediction is made.

• Classification threshold: The probability cutoff used to convert a predicted probability into a class decision.

• Hyperparameter: A model setting selected outside the ordinary parameter-estimation process, often through resampling or validation.

• Clustering: An unsupervised method that groups observations according to a defined measure of similarity.