8. Hadamard Product

Vector Hadamard Multiplication (Element-Wise)

If you were to ask someone on the street who had never studied linear algebra how to multiply two vectors together, they would likely describe Hadamard Multiplication. Also commonly known as element-wise multiplication, this operation is the most intuitive form of vector multiplication.

In this lesson, we define the Hadamard product, explore its geometric constraint, contrast it with the dot product, and implement it in Python.

1. Mathematical Definition

The Hadamard Product is defined as the multiplication of corresponding elements from two vectors of the same size.

Mathematically, we denote Hadamard multiplication using the circle-dot operator $\odot$ :

\mathbf{c} = \mathbf{a} \odot \mathbf{b}

For two vectors $\mathbf{a}, \mathbf{b} \in \mathbb{R}^n$ , the resulting vector $\mathbf{c} \in \mathbb{R}^n$ is computed element-by-element:

c_i = a_i \cdot b_i \quad \text{for } i = 1, 2, \dots, n

Writing it out in vector form:

\begin{bmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \end{bmatrix} \odot \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_n \end{bmatrix} = \begin{bmatrix} a_1 b_1 \\ a_2 b_2 \\ \vdots \\ a_n b_n \end{bmatrix}

2. The Dimensionality Constraint

Hadamard multiplication has a strict geometric constraint: it is defined only for vectors of the exact same size.

Valid: Multiplying two 3-dimensional vectors.
Invalid: Multiplying a 3-dimensional vector and a 5-dimensional vector.

If the vectors have different shapes, their element-wise corresponding parts do not exist, and the operation is mathematically undefined.

3. Step-by-Step Example

Let's compute the Hadamard product of two vectors $\mathbf{w}_1$ and $\mathbf{w}_2$ in $\mathbb{R}^3$ :

\mathbf{w}_1 = \begin{bmatrix} 1 \\ 3 \\ 5 \end{bmatrix}, \quad \mathbf{w}_2 = \begin{bmatrix} 3 \\ 4 \\ 2 \end{bmatrix}

Multiply the corresponding elements: $\mathbf{w}_1 \odot \mathbf{w}_2 = \begin{bmatrix} 1 \cdot 3 \\ 3 \cdot 4 \\ 5 \cdot 2 \end{bmatrix} = \begin{bmatrix} 3 \\ 12 \\ 10 \end{bmatrix}$

4. Hadamard Product vs. Dot Product

It is crucial not to confuse Hadamard multiplication with the Vector Dot Product:

Feature	Hadamard Product ( $\mathbf{a} \odot \mathbf{b}$ )	Dot Product ( $\mathbf{a}^{\text{T}}\mathbf{b}$ )
Output Type	Vector (same size as inputs)	Scalar (a single number)
Summation	No summation	Sums the products of elements
Notation	Circle-dot ( $\odot$ )	Transpose matrix multiplication or dot ( $\cdot$ )
Core Concept	Component scaling / Masking	Spatial projection / Similarity

If we took the dot product of the example vectors above, we would sum the Hadamard results: $\mathbf{w}_1^{\text{T}}\mathbf{w}_2 = 3 + 12 + 10 = 25$

5. Python Implementation

In Python, NumPy provides two primary ways to compute the Hadamard product:

Method 1: Using `np.multiply()`

The np.multiply() function accepts standard Python lists or arrays and computes their element-wise product:

python

import numpy as np

# Define lists
w1 = [ 1, 3, 5 ]
w2 = [ 3, 4, 2 ]

# Compute Hadamard multiplication
w3 = np.multiply(w1, w2)
print(w3)

Out · stdout

[ 3 12 10]

Method 2: Using the Overloaded `*` Operator

When you convert lists to NumPy arrays, the standard multiplication asterisk operator * is overloaded to perform element-wise (Hadamard) multiplication automatically: