5. Dot Product Properties

Dot Product Properties: Associative & Distributive

When moving from basic scalar algebra to linear algebra, we must verify if familiar properties—such as the distributive, associative, and commutative properties—still hold. In this lesson, we explore these properties in the context of the vector dot product and prove them using both formal mathematical proofs and empirical Python simulations.

1. The Distributive Property

In standard high school algebra, the distributive property states that multiplying a scalar by the sum of two others is equal to distributing the multiplier:

Vector Distributivity

Does this distributive property hold for the vector dot product?

Yes. The dot product is distributive. For any three column vectors $\mathbf{a}, \mathbf{b}, \mathbf{c}$ of the same dimensionality:

The Algebraic Proof

To prove why this holds, we can rewrite the terms using the algebraic summation definition of the dot product:

Because element-wise products $\mathbf{a}_i$, $\mathbf{b}_i$, and $\mathbf{c}_i$ are just scalars, we can apply standard scalar factoring inside the sum:

This sum represents the element-wise product of $\mathbf{a}$ and the summed vector $(\mathbf{b} + \mathbf{c})$, which is exactly the summation form of $\mathbf{a}^{\text{T}}(\mathbf{b} + \mathbf{c})$. Thus:

2. The Associative Property

In scalar algebra, the associative property states that the grouping of multiplications does not affect the product:

Vector Associativity

Does this associative property hold for the vector dot product?

⚠️ No! The vector dot product is NOT associative.

There are two primary reasons why the dot product violates the associative law:

Reason 1: Different Directions & Scalings

Let's analyze the mathematical objects on each side of the inequality. None of these vectors are the zero vector, and $\mathbf{a}, \mathbf{b}, \mathbf{c}$ are distinct:

1. On the left-hand side $\mathbf{a}^{\text{T}}(\mathbf{b}^{\text{T}}\mathbf{c})$, the term $(\mathbf{b}^{\text{T}}\mathbf{c})$ is a dot product, which evaluates to a scalar (let's call it $\beta$). Therefore, the expression is a scalar-vector multiplication scaling the vector $\mathbf{a}$: $$\mathbf{a}^{\text{T}}(\mathbf{b}^{\text{T}}\mathbf{c}) = \beta \mathbf{a}$$
2. On the right-hand side $(\mathbf{a}^{\text{T}}\mathbf{b})^{\text{T}}\mathbf{c}$, the term $(\mathbf{a}^{\text{T}}\mathbf{b})$ is a dot product that evaluates to a scalar (let's call it $\alpha$). Therefore, the expression scales the vector $\mathbf{c}$: $$(\mathbf{a}^{\text{T}}\mathbf{b})^{\text{T}}\mathbf{c} = \alpha \mathbf{c}$$

Since $\mathbf{a}$ and $\mathbf{c}$ are distinct vectors pointing in entirely different spatial directions, scaling them by scalars $\beta$ and $\alpha$ will yield two completely different vectors in different coordinates. Thus, they cannot be equal.

Reason 2: Dimensionality Constraints & Invalid Operations

Consider a scenario where the vector lengths are mismatched:

• Let $\mathbf{a} \in \mathbb{R}^5$ (a $5$-dimensional vector).
• Let $\mathbf{b}, \mathbf{c} \in \mathbb{R}^3$ (both $3$-dimensional vectors).

If we attempt to evaluate both groupings:

1. Left-hand side $\mathbf{a}(\mathbf{b}^{\text{T}}\mathbf{c})$: The dot product $\mathbf{b}^{\text{T}}\mathbf{c}$ is a valid operation because both are $3$D vectors. It yields a scalar. Multiplying the $5$D vector $\mathbf{a}$ by this scalar is completely valid, returning a scaled $5$D vector.
2. Right-hand side $(\mathbf{a}^{\text{T}}\mathbf{b})\mathbf{c}$: The grouping requires us to compute the dot product $\mathbf{a}^{\text{T}}\mathbf{b}$. However, $\mathbf{a}$ is $5$D and $\mathbf{b}$ is $3$D. Because their dimensionalities mismatch, their dot product is undefined!

Moving the parentheses changes the expression from a valid operation to an illegal operation, which mathematically proves that grouping is not associative.

Special Cases: There are rare special cases where the associative property may hold, such as when one of the vectors is a zeros vector ($\mathbf{0}$), or when specific collinear vector arrangements align the directions perfectly. However, in the general case, associativity is invalid.

3. Matrix Multiplication is Associative!

It is critical to distinguish vector dot product grouping from general matrix multiplication. While the dot product is not associative, matrix multiplication is fully associative:

Do not assume linear algebra entirely rejects associativity! Matrix associativity is a fundamental pillar used to prove advanced theorems later in this course.

4. The Commutative Property (Teaser!)

The commutative property states that order does not matter: $$a \cdot b = b \cdot a$$

Does the vector dot product commute? Is $\mathbf{a}^{\text{T}}\mathbf{b}$ identical to $\mathbf{b}^{\text{T}}\mathbf{a}$?

We are not going to reveal the answer here! There is a dedicated Code Challenge in an upcoming video where you will write code to test and discover this core linear algebra property for yourself.

5. Python Verification

We can verify both the distributive and non-associative properties of the dot product empirically using random vectors in NumPy:

import numpy as np

# ----------------------------------------------------
# 1. Verification of the Distributive Property
# ----------------------------------------------------
n = 10
a = np.random.randn(n)
b = np.random.randn(n)
c = np.random.randn(n)

# Compute both sides of the distributive equation
distrib_lhs = np.dot(a, (b + c))
distrib_rhs = np.dot(a, b) + np.dot(a, c)

print("--- Distributive Property ---")
print(f"LHS (a . (b + c)): {distrib_lhs:.6f}")
print(f"RHS (a.b + a.c):   {distrib_rhs:.6f}")
print(f"Equal?            {np.allclose(distrib_lhs, distrib_rhs)}")

# ----------------------------------------------------
# 2. Verification of the Associative Property
# ----------------------------------------------------
m = 5
x = np.random.randn(m)
y = np.random.randn(m)
z = np.random.randn(m)

# Compute both sides of the associative grouping
# Note: np.dot(y, z) and np.dot(x, y) evaluate to scalars,
# which scale the remaining vectors.
assoc_lhs = np.dot(x, np.dot(y, z))
assoc_rhs = np.dot(np.dot(x, y), z)

print("\n--- Associative Property ---")
print(f"LHS Vector (x * (y.z)):\n{assoc_lhs}")
print(f"RHS Vector ((x.y) * z):\n{assoc_rhs}")
print(f"Equal?                 {np.allclose(assoc_lhs, assoc_rhs)}")

--- Distributive Property ---
LHS (a . (b + c)): 2.960885
RHS (a.b + a.c):   2.960885
Equal?            True

--- Associative Property ---
LHS Vector (x * (y.z)):
[ 0.34168578  0.33926463  1.90327317 -3.75809543 -2.12757716]
RHS Vector ((x.y) * z):
[-0.00241565  0.06984205 -0.01111632 -0.06833576  0.03170845]
Equal?                 False

Key Takeaways

• Distributive ($Yes$): The dot product distributes over addition: $\mathbf{a}^{\text{T}}(\mathbf{b} + \mathbf{c}) = \mathbf{a}^{\text{T}}\mathbf{b} + \mathbf{a}^{\text{T}}\mathbf{c}$.
• Associative ($No$): Parentheses grouping is invalid for vector dot products: $\mathbf{a}^{\text{T}}(\mathbf{b}^{\text{T}}\mathbf{c}) \neq (\mathbf{a}^{\text{T}}\mathbf{b})^{\text{T}}\mathbf{c}$.
• Matrix Contrast: Grouping remains fully valid for pure matrix multiplication: $\mathbf{A}(\mathbf{B}\mathbf{C}) = (\mathbf{A}\mathbf{B})\mathbf{C}$.