10.5 Decision trees

A decision tree repeatedly divides observations into groups using feature-based rules. Trees are attractive because a sequence of splits can be explained in plain language. They can represent nonlinear relationships and interactions without requiring the analyst to specify each one in advance.

An unrestricted tree can fit noise. Control complexity through depth, minimum node size, or cost-complexity pruning, and select settings through validation. A single tree may also be unstable: a modest change in the training data can produce different splits.

Interpretability does not make every tree explanation causal. A split indicates how the fitted model separates observations, not why the outcome occurs.