2.4 Vector Spaces

A vector space is a structured set of objects called vectors that can be added together and scaled by real numbers (scalars), while remaining within the same set. To understand this concept, we first introduce groups, which form the foundation for vector space structure.

2.4.1 Groups

Definition 2.21 A group is a set \(G\) with an operation \(\otimes\) that satisfies:

  1. Closure: \(x \otimes y \in G\) for all \(x, y \in G\).
  2. Associativity: \((x \otimes y) \otimes z = x \otimes (y \otimes z)\).
  3. Neutral element: There exists \(e \in G\) such that \(x \otimes e = e \otimes x = x\).
  4. Inverse element: For every \(x \in G\), there exists \(y \in G\) such that \(x \otimes y = e\).

If the operation is also commutative, the group is called Abelian.

Example 2.41

  • \((\mathbb{Z}, +)\) is an Abelian group.
  • \((\mathbb{N}_0, +)\) is not a group (no inverse elements).
  • \((\mathbb{R} \setminus \{0\}, \cdot)\) is an Abelian group.
  • \((\mathbb{R}^{n \times n}, \cdot)\) forms a group only for invertible matrices—called the general linear group, denoted \(\mathrm{GL}(n, \mathbb{R})\). This group is not Abelian because matrix multiplication is not commutative.

Example 2.42 Prove that \(\mathrm{GL}(2,\mathbb{R})\) forms a group under multiplication.

To show that \(\mathrm{GL}(2,\mathbb{R})\) forms a group, we verify each property:

  1. Closure
    Let \(A, B \in \mathrm{GL}(2,\mathbb{R})\). Then both \(A\) and \(B\) are invertible, meaning there exist matrices \(A^{-1}\) and \(B^{-1}\) such that: \[ AA^{-1} = A^{-1}A = I, \quad BB^{-1} = B^{-1}B = I \] We must show that \(AB\) is also invertible. Consider the product \(B^{-1}A^{-1}\). Then: \[ (AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = AIA^{-1} = AA^{-1} = I \] and similarly, \[ (B^{-1}A^{-1})(AB) = B^{-1}(A^{-1}A)B = B^{-1}IB = BB^{-1} = I \] Hence, \(B^{-1}A^{-1}\) is the inverse of \(AB\). Therefore, \(AB\) is invertible and \(AB \in \mathrm{GL}(2,\mathbb{R})\). Closure holds.

  2. Associativity
    Matrix multiplication is associative for all \(2 \times 2\) real matrices. Thus, for any \(A, B, C \in \mathrm{GL}(2,\mathbb{R})\): \[ (AB)C = A(BC) \] Associativity holds.

  3. Identity Element
    The \(2 \times 2\) identity matrix \(I\) is it’s own inverse. It is therefore a \(2 \times 2\) invertible matrix and so is in (2,)).
    Identity exists.

  4. Inverse Element
    By definition, each \(A \in \mathrm{GL}(2,\mathbb{R})\) is invertible. So there exists \(A^{-1}\) such that: \[ AA^{-1} = A^{-1}A = I \] Moreover, since \(A\) is the inverse of \(A^{-1}\), we know that \(A^{-1}\) is also an invertible \(2 \times 2\) matrix and so is in \(\mathrm{GL}(2,\mathbb{R})\). Inverses exist.

Therefore, since all conditions are satisfied, \(\mathrm{GL}(2,\mathbb{R})\) is a group under matrix multiplication.

Definition 2.22 A real-valued vector space \(V = (V, +, \cdot)\) consists of:

  • An inner operation (vector addition) \(+ : V \times V \to V\).
  • An outer operation (scalar multiplication) \(\cdot : \mathbb{R} \times V \to V\).

The operations satisfy:

  1. \((V, +)\) is an Abelian group.
  2. Distributivity:
    • \(\lambda \cdot (\mathbf{x} + \mathbf{y}) = \lambda \cdot \mathbf{x} + \lambda \cdot \mathbf{y}\)
    • \((\lambda + \psi) \cdot \mathbf{x} = \lambda \cdot \mathbf{x} + \psi \cdot \mathbf{x}\)
  3. Associativity: \(\lambda \cdot (\psi \cdot \mathbf{x}) = (\lambda \psi) \cdot \mathbf{x}\)
  4. Neutral element: \(\mathbf{1} \cdot \mathbf{x} = \mathbf{x}\)

The zero vector \(\mathbf{0}\) acts as the neutral element of addition.

Example 2.43

  • \(\mathbb{R}^n\): Vectors added and scaled componentwise.
  • \(\mathbb{R}^{m \times n}\): Matrices added and scaled elementwise.
  • \(\mathbb{C}\): The complex numbers under standard addition and multiplication.

In notation, vectors are typically written as column vectors
\[ \mathbf{x} = \begin{bmatrix} x_1 \\ \vdots \\ x_n \end{bmatrix}, \] and their transposes \(\mathbf{x}^\top\) are row vectors.

2.4.2 Vector Subspaces

Definition 2.23 A vector subspace \(U \subseteq V\) is a subset of a vector space that is itself a vector space under the same operations. To be a subspace, \(U\) must satisfy:

  1. \(\mathbf{0} \in U\)
  2. Closure under scalar multiplication: \(\lambda \mathbf{x} \in U\)
  3. Closure under addition: \(\mathbf{x} + \mathbf{y} \in U\)

Example 2.44

  • The set of all solutions to a homogeneous system \(\mathbf{A}\mathbf{x} = \mathbf{0}\) is a subspace of \(\mathbb{R}^n\).
  • The set of solutions to an inhomogeneous system \(\mathbf{A}\mathbf{x} = \mathbf{b}\) (where \(\mathbf{b} \neq 0\)) is not a subspace.
  • The intersection of any number of subspaces is also a subspace.

Exercises

Exercise 2.45 SHow that each of these is a group:

  • \((\mathbb{Z}, +)\) is an Abelian group.
  • \((\mathbb{N}_0, +)\) is not a group (no inverse elements).
  • \((\mathbb{R} \setminus \{0\}, \cdot)\) is an Abelian group.
  • \((\mathbb{R}^{n \times n}, \cdot)\) forms a group only for invertible matrices—called the general linear group, denoted \(\mathrm{GL}(n, \mathbb{R})\). This group is not Abelian because matrix multiplication is not commutative.

Exercise 2.46 SHow that each of these is a vector space:

  • \(\mathbb{R}^n\): Vectors added and scaled componentwise.
  • \(\mathbb{R}^{m \times n}\): Matrices added and scaled elementwise.
  • \(\mathbb{C}\): The complex numbers under standard addition and multiplication.

Exercise 2.47 Show each of these is a vector subspace:

  • The set of all solutions to a homogeneous system \(\mathbf{A}\mathbf{x} = \mathbf{0}\) is a subspace of \(\mathbb{R}^n\).
  • The set of solutions to an inhomogeneous system \(\mathbf{A}\mathbf{x} = \mathbf{b}\) (where \(\mathbf{b} \neq 0\)) is not a subspace.
  • The intersection of any number of subspaces is also a subspace.

Exercise 2.48 Prove that \(\mathbb{Z}\) is a group under addition.

Exercise 2.49 Prove that \(\mathbb{Z}^+\) is not a group under addition.

Exercise 2.50 Prove that \(\mathbb{R} \setminus \{0\}\) is a group under multiplication.

Exercise 2.51 Let \(G\) be the set of matrices of the form \(\begin{bmatrix}a&b\\0&c \end{bmatrix}\) where \(a,b,c \in \mathbb{R}\) and \(ac \not = 0\). Prove that \(G\) forms a subgroup of \(G(2\mathbb{R})\).

Exercise 2.52 Let \(g\) be an element of a group \(G\). Show that it’s inverse is unique.

Exercise 2.53

Let \(G\) be a group and let \(H\) be a non-empty subset of \(G\). Prove the following are equivalent by proving \(1 \Rightarrow 3 \Rightarrow 2 \Rightarrow 1\).

Exercise 2.54 Prove that \(\mathbb{R}^n\) is a vector space under componentwise addition and scalar multiplication.

Exercise 2.55 Prove that \(\mathbb{R}^{m \times n}\) is a vector space under componentwise addition and scalar multiplication.

Exercise 2.56