Pseudoinverse

The Moore-Penrose generalization of a matrix inverse — exists for any matrix, even non-square or singular ones, and gives the least-squares solution.

Pseudoinverse chooses the closest reachable output

Problem

For a tall matrix, Ax often cannot equal b exactly.

Pseudoinverse answer

x_best = A⁺ b

It picks the input whose output lands closest to the target.

Geometry

The leftover error is perpendicular to the reachable output space.

Definition

Only square, invertible matrices have a true inverse $A^{-1}$ . The pseudoinverse $A^+$ (also called the Moore-Penrose inverse) extends the idea of "undoing" a matrix to any matrix — square or not, invertible or not.

When $A$ is actually invertible, $A^+ = A^{-1}$ . When it isn't, $A^+$ gives the best possible substitute: the matrix that turns $Ax = b$ into the closest thing to a solution, even when no exact solution exists or when many solutions do.

Why ordinary inverses aren't enough

Suppose $A$ is $5 \times 2$ — five equations, two unknowns. Generically, $Ax = b$ has no exact solution: you're asking five constraints to be satisfied by only two degrees of freedom.

$A^+$ doesn't pretend a solution exists. Instead, $x = A^+ b$ gives the $x$ that makes $Ax$ as close to $b$ as possible — the least-squares solution.

Try it

If $A$ is square and invertible, what should $A^+$ equal, and why?

Solution

$A^+ = A^{-1}$ . The pseudoinverse is built to generalize the inverse, so whenever the ordinary inverse exists, the pseudoinverse must agree with it — otherwise it wouldn't deserve the name.

Related concepts

Needs first

Matrix Inverse Rank

Gaussian Elimination Linear Regression

View in full concept graph →