13. Vector Subspaces

Vector Subspaces

The concept of a subspace is one of the most foundational and important topics in linear algebra. It bridges pure algebra (linear combinations of column vectors) with geometry (infinite lines, spanned planes, and high-dimensional hyperplanes).

In this lesson, we will explore both the algebraic and geometric perspectives of subspaces, study the formal closure criteria, and define how subspaces relate to their surrounding ambient space.

1. Algebraic Perspective

From an algebraic perspective, a subspace is defined by the set of all vectors that can be created by taking linear combinations of a specific vector or a set of vectors.

Recall that a linear combination of vectors involves multiplying them by real-valued scalars ($\lambda, \beta \in \mathbb{R}$) and adding them together.

Example A: A Subspace from One Vector

Consider a single vector $\mathbf{v}$ in $\mathbb{R}^2$: $$\mathbf{v} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$$

Any other vector that can be expressed as a scaled multiple $\lambda \mathbf{v}$ is in the same subspace as $\mathbf{v}$. For instance:

• $\begin{bmatrix} 4 \\ 6 \end{bmatrix}$ is in the subspace (where $\lambda = 2$).
• $\begin{bmatrix} -2 \\ -3 \end{bmatrix}$ is in the subspace (where $\lambda = -1$).
• $\begin{bmatrix} 0 \\ 0 \end{bmatrix}$ is in the subspace (where $\lambda = 0$).

Conversely, vectors like $\begin{bmatrix} 2 \\ 4 \end{bmatrix}$, $\begin{bmatrix} 0 \\ 3 \end{bmatrix}$, and $\begin{bmatrix} 0 \\ 4 \end{bmatrix}$ are NOT in the subspace:

• For the vector $\begin{bmatrix} 2 \\ 4 \end{bmatrix}$, we would need a scalar $\lambda = 1$ to match the first element ($1 \cdot 2 = 2$). However, for the second element, $1 \cdot 3 = 3 \neq 4$. Since no single scalar $\lambda$ satisfies both components, this vector lies outside the subspace of $\mathbf{v}$.

Example B: A Subspace from Two Vectors

A subspace is not limited to a single vector. We can define a subspace from two vectors $\mathbf{v}$ and $\mathbf{w}$ by considering all of their possible linear combinations: $$\lambda \mathbf{v} + \beta \mathbf{w}$$

For example: $$\mathbf{v} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}, \quad \mathbf{w} = \begin{bmatrix} 0 \\ 4 \end{bmatrix}$$

Let's test if the vector $\begin{bmatrix} 12 \\ 2 \end{bmatrix}$ is in the spanned subspace. We choose scalars $\lambda = 6$ and $\beta = -4$: $$6 \mathbf{v} - 4 \mathbf{w} = 6 \begin{bmatrix} 2 \\ 3 \end{bmatrix} - 4 \begin{bmatrix} 0 \\ 4 \end{bmatrix} = \begin{bmatrix} 12 \\ 18 \end{bmatrix} - \begin{bmatrix} 0 \\ 16 \end{bmatrix} = \begin{bmatrix} 12 \\ 2 \end{bmatrix}$$

Because we can successfully combine $\mathbf{v}$ and $\mathbf{w}$ to produce $\begin{bmatrix} 12 \\ 2 \end{bmatrix}$, this vector is officially a member of the subspace.

3. Subsets vs. Subspaces

A common point of confusion in linear algebra is the difference between a subset and a subspace. While they sound similar, they are very different mathematical objects:

• Subset: Simply any collection of points or vectors in a coordinate space that satisfies any custom condition you define. Subsets do not need to be closed under vector addition or scalar multiplication, nor do they need to contain the origin.
• Subspace: A highly restricted, "closed" subset that must satisfy the closure criteria and contain the zero vector.

All subspaces are subsets, but very few subsets are subspaces. Subspaces are subsets that act as "closed universes"—once you are inside, no linear operation can take you out.

Let's look at three concrete algebraic examples in $\mathbb{R}^2$ to master this distinction.

Example 1: The Boundary Region ($y \ge 2$)

Consider the set of all points in 2D space where the $y$-coordinate is greater than or equal to $2$: $$S_1 = \left\{ \begin{bmatrix} x \\ y \end{bmatrix} \in \mathbb{R}^2 \;\middle|\; y \ge 2 \right\}$$

1. Origin Test: Let's check if the zero vector $\mathbf{0} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}$ is in $S_1$. Since the $y$-coordinate is $0$, and $0 < 2$, the zero vector is not in this set: $$\mathbf{0} \notin S_1$$
2. Scalar Multiplication Test: Let's choose a valid vector in the set, e.g., $\mathbf{u} = \begin{bmatrix} 0 \\ 3 \end{bmatrix} \in S_1$ (since $3 \ge 2$). Let's scale it by a negative factor $\lambda = -1$: $$\lambda \mathbf{u} = -1 \begin{bmatrix} 0 \\ 3 \end{bmatrix} = \begin{bmatrix} 0 \\ -3 \end{bmatrix}$$ Since $-3 < 2$, the scaled vector has escaped the set: $$\lambda \mathbf{u} \notin S_1$$

• Conclusion: $S_1$ is a perfectly valid subset, but it is NOT a subspace because it contains no origin and fails closure under scaling.

Example 2: The Curved Curve ($y = x^2$)

Consider the set of all points that lie on the parabola $y = x^2$: $$S_2 = \left\{ \begin{bmatrix} x \\ y \end{bmatrix} \in \mathbb{R}^2 \;\middle|\; y = x^2 \right\}$$

1. Origin Test: Check if the zero vector $\mathbf{0} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}$ is in $S_2$. Since $0 = 0^2$, the zero vector is successfully included: $$\mathbf{0} \in S_2$$
2. Vector Addition Test: Let's pick two valid points on this parabola: $$\mathbf{u} = \begin{bmatrix} 1 \\ 1 \end{bmatrix} \in S_2 \quad (\text{since } 1 = 1^2)$$ $$\mathbf{w} = \begin{bmatrix} 2 \\ 4 \end{bmatrix} \in S_2 \quad (\text{since } 4 = 2^2)$$ Let's add these two vectors together: $$\mathbf{u} + \mathbf{w} = \begin{bmatrix} 1 \\ 1 \end{bmatrix} + \begin{bmatrix} 2 \\ 4 \end{bmatrix} = \begin{bmatrix} 1 + 2 \\ 1 + 4 \end{bmatrix} = \begin{bmatrix} 3 \\ 5 \end{bmatrix}$$ For this sum to belong to $S_2$, its coordinates must satisfy $y = x^2$. However: $$5 \neq 3^2 = 9 \implies \mathbf{u} + \mathbf{w} \notin S_2$$ Adding the vectors took us completely off the parabola!

• Conclusion: $S_2$ is a valid subset containing the origin, but it is NOT a subspace because it is not closed under addition (curvature breaks linearity).

Example 3: The Linear Equation ($y = 2x$)

Consider the set of all points lying on the line $y = 2x$: $$S_3 = \left\{ \begin{bmatrix} x \\ y \end{bmatrix} \in \mathbb{R}^2 \;\middle|\; y = 2x \right\}$$

1. Origin Test: Since $0 = 2(0)$, the zero vector is included: $$\mathbf{0} \in S_3$$
2. Addition & Scaling Test: Let's take any arbitrary vector $\mathbf{u} = \begin{bmatrix} x_1 \\ 2x_1 \end{bmatrix} \in S_3$ and another vector $\mathbf{w} = \begin{bmatrix} x_2 \\ 2x_2 \end{bmatrix} \in S_3$, and scale them by any real numbers $\lambda$ and $\alpha$: $$\lambda \mathbf{u} + \alpha \mathbf{w} = \lambda \begin{bmatrix} x_1 \\ 2x_1 \end{bmatrix} + \alpha \begin{bmatrix} x_2 \\ 2x_2 \end{bmatrix} = \begin{bmatrix} \lambda x_1 + \alpha x_2 \\ 2(\lambda x_1 + \alpha x_2) \end{bmatrix}$$ The $y$-component of the result is exactly twice its $x$-component. Thus, the linear combination must remain on the line: $$\lambda \mathbf{u} + \alpha \mathbf{w} \in S_3$$

• Conclusion: This subset $S_3$ is closed under all operations and contains the origin. Therefore, it IS a perfectly valid subspace!

4. Geometric Perspective

Geometrically, a subspace is represented as an infinite flat shape passing directly through the origin of the coordinate space.

The 1D Subspace (Line)

A single non-zero vector $\mathbf{v}_1$ in a 3D coordinate space represents a line. By scaling $\mathbf{v}_1$ by all scalars $\lambda$ from $-\infty$ to $+\infty$, we trace out an infinitely long line. This line is a 1D subspace passing through the origin.

The 2D Subspace (Plane)

If we introduce a second vector $\mathbf{v}_2$ that is independent of $\mathbf{v}_1$, they can be combined to reach any point on a flat plane.

Here we see two vectors and their linear combination vector $0.4\mathbf{v}_1 + 1.2\mathbf{v}_2$:

By taking all possible linear combinations of these two vectors, we span an entire two-dimensional plane passing through the origin. This represents a 2D subspace embedded inside a 3D surrounding space:

5. Subspaces vs. Ambient Space

It is crucial to distinguish between a subspace and the surrounding ambient space (also called the ambient dimensionality).

The Ambient Space is the full dimensional space in which our vectors are embedded. For example, if all vectors have three elements ($x, y, z$), the ambient space is $\mathbb{R}^3$.

Inside an ambient 3D space ($\mathbb{R}^3$), there are exactly four possible dimensionalities for subspaces:

Every single one of these subspaces intersects at the origin, because all subspaces must contain the zero vector.

6. Linear Independence and High-Dimensional Collapse

Simply having multiple vectors does not guarantee a high-dimensional subspace. The vectors must be linearly independent (meaning no vector can be expressed as a linear multiple of the others).

2D Collapse (Line instead of Plane)

Suppose we have two vectors: $$\mathbf{v}_1 = \begin{bmatrix} 1 \\ 2 \\ 4 \end{bmatrix}, \quad \mathbf{v}_2 = \begin{bmatrix} 2 \\ 4 \\ 8 \end{bmatrix}$$

Because $\mathbf{v}_2 = 2\mathbf{v}_1$, they lie on the exact same line. Taking all possible linear combinations of $\mathbf{v}_1$ and $\mathbf{v}_2$ will only yield that single line. Thus, despite having two vectors, we only get a 1D line subspace, not a 2D plane.

5D Generalization ($\mathbb{R}^5$)

This principle holds for any arbitrary dimension, such as an ambient 5D space ($\mathbb{R}^5$):

• 0D: The zero vector $[0, 0, 0, 0, 0]$.
• 1D: A single non-zero vector (spans a 5D line).
• 2D: Two independent vectors (span a 5D plane).
• 3D: Three independent vectors (span a 3D cube-like subspace).
• Collapse: If we have three vectors but the third is a scaled multiple of the second, they collapse and only span a 2D plane subspace.

In high-dimensional contexts, mathematicians use the term Hyperplane to describe a high-dimensional subspace embedded inside an even higher-dimensional ambient space.

Key Takeaways

• Subspace: A set of all vectors formed by the linear combinations of a starting set of vectors.
• Closure Criteria: A subspace must be closed under vector addition and scalar multiplication ($\lambda\mathbf{u} + \alpha\mathbf{w} \in V$).
• Zero Vector Rule: Every subspace must contain the origin ($\mathbf{0} \in V$).
• Ambient Space: The parent coordinate space in which the subspace is embedded.
• Linear Independence: Multiple vectors only increase subspace dimensionality if they are independent; dependent vectors collapse into lower-dimensional subspaces.