1.1 Finding Words for Intuitions

Definition 1.1 Machine learning is the study and development of algorithms that improve automatically through experience and data, without being explicitly programmed for each task.

Machine learning is a field that combines data, models, and learning methods to identify patterns and make predictions or decisions — ideally generalizing well to new, unseen situations. Data is the foundation — machine learning aims to discover useful patterns from data without relying heavily on domain expertise.

Definition 1.2 Data are pieces of information collected to describe, measure, or analyze phenomena.

In practice, data is represented numerically, often as vectors, \(\mathbf{x} = \begin{bmatrix}x_1\\ x_2\\ \vdots\\ x_N \end{bmatrix}\). Models describe how data is generated or how inputs map to outputs.

Definition 1.3 A model is a learned representation that maps inputs to outputs based on patterns found in data.

A model learns when its performance improves after processing data. Good models generalize to new, unseen data.

Definition 1.4 Learning is the process of using data to automatically improve a model’s ability to perform a task.

The goal is not just to fit the training data, but to perform well on new examples.

Formally, you can think of an algorithm as a mapping from inputs to outputs, where each step is precise, unambiguous, and executable by a computer.

Definition 1.5 An Algorithm: is a finite sequence of well-defined instructions or steps designed to solve a specific problem or perform a computation.

In the context of machine learning, an algorithm provides a systematic procedure for processing data — either to make predictions (as in a predictive algorithm) or to adjust model parameters (as in a training algorithm). In this way, machine learning involves two overlapping meanings of “algorithm”:

1. A predictor that makes predictions based on data.
   
2. A training procedure that updates the predictor’s parameters to improve future performance.

Understanding the mathematical foundations behind data, models, and learning helps us build, interpret, and improve machine learning systems — and recognize their assumptions and limitations.