Lesson 6 Basis and Orthonormal Basis
6.2 Lecture Notes
6.2.1 Linear Independence
Definition (Linear Independence):
Vectors \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\) in \(\mathbb{R}^n\) are linearly independent if the equation
\[
c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k = \mathbf{0}
\]
has only the trivial solution \(c_1 = c_2 = \cdots = c_k = 0\).
If nontrivial solutions exist, the vectors are linearly dependent.
- Example: Determine whether the following set of vectors in \(\mathbb{R}^3\) is linearly independent:
\[
\mathbf{v}_1 = \begin{bmatrix}1 \\ 0 \\ 2\end{bmatrix}, \quad
\mathbf{v}_2 = \begin{bmatrix}0 \\ 1 \\ -1\end{bmatrix}, \quad
\mathbf{v}_3 = \begin{bmatrix}2 \\ 1 \\ 3\end{bmatrix}
\]
We want to test whether the equation
\[
c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + c_3\mathbf{v}_3 = \mathbf{0}
\]
has only the trivial solution \(c_1 = c_2 = c_3 = 0\).
Write the linear combination: \[ c_1 \begin{bmatrix}1 \\ 0 \\ 2\end{bmatrix} + c_2 \begin{bmatrix}0 \\ 1 \\ -1\end{bmatrix} + c_3 \begin{bmatrix}2 \\ 1 \\ 3\end{bmatrix} = \begin{bmatrix}0 \\ 0 \\ 0\end{bmatrix} \]
Add the vectors: \[ \begin{bmatrix} c_1 + 2c_3 \\ c_2 + c_3 \\ 2c_1 - c_2 + 3c_3 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix} \]
This gives a system of equations: \[ \begin{aligned} &c_1 + 2c_3 = 0 \quad \text{(1)}\\ &c_2 + c_3 = 0 \quad \text{(2)}\\ &2c_1 - c_2 + 3c_3 = 0 \quad \text{(3)} \end{aligned} \]
Solve the system: From (1): \(c_1 = -2c_3\)
From (2): \(c_2 = -c_3\) Substitute into (3): \[ 2(-2c_3) - (-c_3) + 3c_3 = -4c_3 + c_3 + 3c_3 = 0 \]
The equation is satisfied for any value of \(c_3\), so the original equation has infinitely many solutions (not just the trivial one). Thus, these vectors are linearly dependent.
6.2.2 Span
The span of a set of vectors \(\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}\) is the set of all linear combinations of those vectors: \[ \text{Span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\} = \left\{ c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k \mid c_i \in \mathbb{R} \right\} \]
- In \(\mathbb{R}^2\): Two vectors span the plane if they are linearly independent.
- In \(\mathbb{R}^3\): Three vectors span the space if they are linearly independent.
6.2.3 Standard Basis
- In \(\mathbb{R}^2\), the standard basis is: \[ \left\{ \begin{bmatrix}1 \\ 0\end{bmatrix}, \begin{bmatrix}0 \\ 1\end{bmatrix} \right\} \]
- In \(\mathbb{R}^3\), the standard basis is: \[ \left\{ \begin{bmatrix}1 \\ 0 \\ 0\end{bmatrix}, \begin{bmatrix}0 \\ 1 \\ 0\end{bmatrix}, \begin{bmatrix}0 \\ 0 \\ 1\end{bmatrix} \right\} \]
6.2.4 Gram-Schmidt Orthonormalization
Goal: Convert a linearly independent set \(\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}\) into an orthonormal set \(\{\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_k\}\).
Steps:
- Set \(\mathbf{u}_1 = \frac{\mathbf{v}_1}{\|\mathbf{v}_1\|}\)
- For \(j = 2\) to \(k\):
- Subtract projections of \(\mathbf{v}_j\) onto all previous \(\mathbf{u}_i\): \[ \mathbf{w}_j = \mathbf{v}_j - \sum_{i=1}^{j-1} \text{proj}_{\mathbf{u}_i} \mathbf{v}_j \]
- Normalize: \[ \mathbf{u}_j = \frac{\mathbf{w}_j}{\|\mathbf{w}_j\|} \]
- Example:
Given the linearly independent vectors:
\[
\mathbf{v}_1 = \begin{bmatrix}1 \\ 1\end{bmatrix}, \quad
\mathbf{v}_2 = \begin{bmatrix}1 \\ -1\end{bmatrix}
\]
Use the Gram-Schmidt process to find an orthonormal basis.
Step 1: Let \(\mathbf{u}_1 = \mathbf{v}_1\) \[ \mathbf{u}_1 = \begin{bmatrix}1 \\ 1\end{bmatrix} \] Normalize it: \[ \|\mathbf{u}_1\| = \sqrt{1^2 + 1^2} = \sqrt{2}, \quad \mathbf{e}_1 = \frac{\mathbf{u}_1}{\|\mathbf{u}_1\|} = \frac{1}{\sqrt{2}}\begin{bmatrix}1 \\ 1\end{bmatrix} \]
Step 2: Subtract projection from \(\mathbf{v}_2\) \[ \text{proj}_{\mathbf{u}_1} \mathbf{v}_2 = \frac{\mathbf{v}_2 \cdot \mathbf{u}_1}{\|\mathbf{u}_1\|^2} \mathbf{u}_1 = \frac{(1)(1) + (-1)(1)}{2} \begin{bmatrix}1 \\ 1\end{bmatrix} = 0 \cdot \begin{bmatrix}1 \\ 1\end{bmatrix} = \begin{bmatrix}0 \\ 0\end{bmatrix} \] \[ \mathbf{u}_2 = \mathbf{v}_2 - \text{proj}_{\mathbf{u}_1} \mathbf{v}_2 = \begin{bmatrix}1 \\ -1\end{bmatrix} \] Normalize: \[ \|\mathbf{u}_2\| = \sqrt{1^2 + (-1)^2} = \sqrt{2}, \quad \mathbf{e}_2 = \frac{1}{\sqrt{2}} \begin{bmatrix}1 \\ -1\end{bmatrix} \] The final 0rthonormal basis is \[ \mathbf{e}_1 = \frac{1}{\sqrt{2}} \begin{bmatrix}1 \\ 1\end{bmatrix}, \quad \mathbf{e}_2 = \frac{1}{\sqrt{2}} \begin{bmatrix}1 \\ -1\end{bmatrix}. \]
6.2.5 Coordinates Relative to an Ordered Basis
Given a basis \(\mathcal{B} = \{\mathbf{b}_1, \dots, \mathbf{b}_n\}\) for \(\mathbb{R}^n\), any vector \(\mathbf{v}\) can be written as: \[ \mathbf{v} = c_1\mathbf{b}_1 + \cdots + c_n\mathbf{b}_n \]
The coordinates of \(\mathbf{v}\) with respect to \(\mathcal{B}\) are the scalars \((c_1, \dots, c_n)\).
- Example: Let \(\mathbf{v} = \begin{bmatrix} 3 \\ 2 \end{bmatrix}\) be a vector in \(\mathbb{R}^2\).
Let the basis \(B\) be:
\[
\mathbf{b}_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \quad
\mathbf{b}_2 = \begin{bmatrix} 2 \\ 0 \end{bmatrix}
\]
Find the coordinates of \(\mathbf{v}\) with respect to the basis \(B\).
Step 1: Set up the equation
We want to find scalars \(c_1\) and \(c_2\) such that: \[ \mathbf{v} = c_1 \mathbf{b}_1 + c_2 \mathbf{b}_2 = c_1 \begin{bmatrix}1 \\ 1\end{bmatrix} + c_2 \begin{bmatrix}2 \\ 0\end{bmatrix} \]
Add the vectors: \[ \begin{bmatrix}3 \\ 2\end{bmatrix} = \begin{bmatrix}c_1 + 2c_2 \\ c_1\end{bmatrix} \]
Step 2: Solve the system
Equating components:
\[ c_1 + 2c_2 = 3 \quad \text{(1)} \\ c_1 = 2 \quad \text{(2)} \]
Substitute (2) into (1):
\[ 2 + 2c_2 = 3 \Rightarrow c_2 = \frac{1}{2} \] Final Answer
The coordinates of \(\mathbf{v}\) in basis \(B\) are: \[ [\mathbf{v}]_B = \begin{bmatrix} 2 \\ \frac{1}{2} \end{bmatrix} \]
6.2.6 Change of Basis Matrix in \(\mathbb{R}^2\)
To convert coordinates from basis \(\mathcal{B}\) to basis \(\mathcal{C}\):
- Let \(\mathcal{B} = \{\mathbf{b}_1, \mathbf{b}_2\}\), \(\mathcal{C} = \{\mathbf{c}_1, \mathbf{c}_2\}\)
- Write the basis vectors as columns of matrices:
- \(P_{\mathcal{B}} = [\mathbf{b}_1 \ \mathbf{b}_2]\)
- \(P_{\mathcal{C}} = [\mathbf{c}_1 \ \mathbf{c}_2]\)
- Then the change of basis matrix from \(\mathcal{B}\) to \(\mathcal{C}\) is: \[ P_{\mathcal{C} \leftarrow \mathcal{B}} = P_{\mathcal{C}}^{-1} P_{\mathcal{B}} \]
To find the coordinates of \(\mathbf{v}\) in the new basis: \[ [\mathbf{v}]_{\mathcal{C}} = P_{\mathcal{C} \leftarrow \mathcal{B}} \cdot [\mathbf{v}]_{\mathcal{B}} \]
- Let the standard basis be:
\[
E = \left\{ \mathbf{e}_1 = \begin{bmatrix}1 \\ 0\end{bmatrix}, \mathbf{e}_2 = \begin{bmatrix}0 \\ 1\end{bmatrix} \right\}
\]
Let a new basis \(B\) be: \[ B = \left\{ \mathbf{b}_1 = \begin{bmatrix}1 \\ 1\end{bmatrix}, \mathbf{b}_2 = \begin{bmatrix}2 \\ -1\end{bmatrix} \right\} \] We want to change coordinates from the \(B\)-basis to the standard basis \(E\).
Step 1: Construct the Change of Basis Matrix \(P_{B \to E}\)
The matrix \(P_{B \to E}\) is formed by placing the basis vectors of \(B\) as columns:
\[ P_{B \to E} = \begin{bmatrix} \mathbf{b}_1 & \mathbf{b}_2 \end{bmatrix} = \begin{bmatrix} 1 & 2 \\ 1 & -1 \end{bmatrix} \]
Step 2: Change from \(B\)-coordinates to Standard Coordinates
Let a vector have \(B\)-coordinates: \[ [\mathbf{v}]_B = \begin{bmatrix} 3 \\ 1 \end{bmatrix} \]
To convert to standard coordinates: \[ [\mathbf{v}]_E = P_{B \to E} \cdot [\mathbf{v}]_B = \begin{bmatrix} 1 & 2 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} 3 \\ 1 \end{bmatrix} = \begin{bmatrix} 1(3) + 2(1) \\ 1(3) + (-1)(1) \end{bmatrix} = \begin{bmatrix} 5 \\ 2 \end{bmatrix} \]
Final Answer
The vector \(\mathbf{v}\) in standard coordinates is: \[ \mathbf{v} = \begin{bmatrix} 5 \\ 2 \end{bmatrix} \]
6.3 Practice Problems
- Show that
\[ \begin{bmatrix} 1 \\ 1 \end{bmatrix} \quad \text{and} \quad \begin{bmatrix}1 \\ -1 \end{bmatrix} \]
is a basis for \(\mathbb{R}^2\). Find the coordinates of \((2,-3)\) with respect to the basis. - Explain why
\[ \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \quad \text{and} \quad \begin{bmatrix}0 \\ 2 \end{bmatrix} \]
is not a basis for \(\mathbb{R}^2\). - Show that
\[ \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}, \quad \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, \quad \text{and} \quad \begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix} \]
is a basis for \(\mathbb{R}^3\). Find the coordinates of \((4,-1,1)\) with respect to the basis. - Use Gram Schmidt to find an orthonormal basis for
\[ \begin{bmatrix} 1 \\ 1 \end{bmatrix} \quad \text{and} \quad \begin{bmatrix}2 \\ 1 \end{bmatrix}. \] - Use Gram Schmidt to find an orthonormal basis for
\[ \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}, \quad \begin{bmatrix} 0 \\ 1 \\ 1 \\ 1 \end{bmatrix}, \quad \text{and} \quad \begin{bmatrix}0 \\ 0 \\ 1 \\ 1 \end{bmatrix}. \] - Let
\[ B = \left\{\begin{bmatrix} 0 \\ 1 \end{bmatrix}, \begin{bmatrix}1 \\ 0 \end{bmatrix}\right\}, \quad C = \left\{\begin{bmatrix} 3 \\ 1 \end{bmatrix}, \begin{bmatrix}1 \\ 2 \end{bmatrix}\right\}. \]
Find the change of basis matrices from \(B\) to \(C\) and from \(C\) to \(B\) (be sure to label them correctly). - (Challenge) Let
\[ B = \left\{\begin{bmatrix} -5 \\ -3 \end{bmatrix}, \begin{bmatrix}4 \\ 28 \end{bmatrix}\right\}, \quad C = \left\{\begin{bmatrix} 6 \\ 2 \end{bmatrix}, \begin{bmatrix}1 \\ -1 \end{bmatrix}\right\}. \]
Find the change of basis matrices from \(B\) to \(C\) and from \(C\) to \(B\) (be sure to label them correctly).
6.4 Self-Assessment
Time yourself and try to solve the following questions within twenty minutes.
- Let
\[ B = \left\{\begin{bmatrix} 1 \\ 2 \end{bmatrix}, \begin{bmatrix}3 \\ 1 \end{bmatrix} \right\}, \quad s = \begin{bmatrix} 1 \\ 7 \end{bmatrix}. \]
Express \(s\) in terms of basis \(B\). - Show that
\[ \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}, \quad \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \quad \text{and} \quad \begin{bmatrix}2 \\ 0 \\ 0 \end{bmatrix} \]
is a basis for \(\mathbb{R}^3\). - Let
\[ B = \left\{\begin{bmatrix} 0 \\ 1 \end{bmatrix}, \begin{bmatrix}1 \\ 0 \end{bmatrix}\right\}, \quad C = \left\{\begin{bmatrix} 0 \\ 1 \end{bmatrix}, \begin{bmatrix}1 \\ 1 \end{bmatrix}\right\}. \]
Find the change of basis matrices from \(B\) to \(C\) and from \(C\) to \(B\) (be sure to label them correctly). - Use Gram Schmidt to find an orthonormal basis for
\[ \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}, \quad \begin{bmatrix} 1 \\ 1 \\ 3 \end{bmatrix}, \quad \text{and} \quad \begin{bmatrix}2 \\ 1 \\ 1 \end{bmatrix}. \]
6.5 Lesson Checklist
This checklist is designed to help you keep track of what you need to work on. The main goal is to be aware of what you need to focus more attention on. Place an \(X\) in the appropriate box beside the skill below.
\[\begin{align*} &\textbf{Developing (D):} &&\textrm{You still need to work on this skill.}\\ &\textbf{Consistent (CON):} &&\textrm{You use the skill correctly most of the time.}\\ &\textbf{Competent (COM):} &&\textrm{You show mastery of the skill.} \end{align*}\]
| Skill | D | CON | COM |
|---|---|---|---|
| Find the coordinates of a vector, given a basis. | |||
| Prove that a set of vectors is or is not a basis. | |||
| Perform the Gram Schmidt process to get an orthonormal basis. | |||
| Find a change of basis matrix. |