Linear Independence

When no vector in a set can be written as a combination of the others — the condition that makes a set of vectors non-redundant.

Linear dependence — when one vector is a combination of others

w is dependent

w = 1.0·v₁ + 0.8·v₂

w lies in span(v₁, v₂)

v₁ and v₂ are independent

Neither is a multiple of the other — they point in genuinely different directions.

c₁ = 1.0

c₂ = 0.8

Definition

Vectors $\mathbf{v}_1, \ldots, \mathbf{v}_k$ are linearly independent if the only solution to

$c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k = \mathbf{0}$

is $c_1 = c_2 = \cdots = c_k = 0$ (the trivial solution).

If a nontrivial solution exists (some $c_i \neq 0$ ), the vectors are linearly dependent — at least one can be written as a combination of the others.

Geometric interpretation:

In $\mathbb{R}^2$ : two vectors are independent iff they don't point in the same (or opposite) direction
In $\mathbb{R}^3$ : three vectors are independent iff no one lies in the plane spanned by the other two

Key properties

The zero vector alone is always linearly dependent — $c\cdot\mathbf{0}=\mathbf{0}$ has nontrivial solutions
Any subset of a linearly independent set is also linearly independent
Adding the zero vector, or a duplicate vector, to any set makes it dependent
A set of more than $n$ vectors in $\mathbb{R}^n$ is always dependent (pigeonhole on dimension)

Common mistakes

Checking pairs only: three vectors can each be pairwise "not parallel" yet still be dependent as a set (e.g. one lies in the plane spanned by the other two) — independence must be checked for the whole set at once
Confusing independence with orthogonality: independent vectors don't need to be perpendicular — orthogonality is a stronger, separate condition

Checking dependence

Are $(1,2)$ , $(3,6)$ linearly independent?

$c_1(1,2) + c_2(3,6) = (0,0)$ gives $c_1 + 3c_2 = 0$ and $2c_1 + 6c_2 = 0$ . The second equation is $2\times$ the first — infinitely many solutions, e.g. $c_1=3, c_2=-1$ .

Linearly dependent: $(3,6) = 3(1,2)$ — the second is a scalar multiple of the first.

Try it

Are the vectors $(1,0,0)$ , $(0,1,0)$ , $(1,1,0)$ linearly independent?

Solution

$c_1(1,0,0) + c_2(0,1,0) + c_3(1,1,0) = (c_1+c_3, c_2+c_3, 0) = (0,0,0)$

This gives $c_1 + c_3 = 0$ and $c_2 + c_3 = 0$ . Choose $c_3 = 1$ : then $c_1 = -1$ , $c_2 = -1$ .

Linearly dependent: $(1,1,0) = (1,0,0) + (0,1,0)$ . The third vector is the sum of the first two.

Related concepts

Needs first

Vector Spaces

Gaussian Elimination Rank

View in full concept graph →