Solving non-square linear system with R

Question

How to solve a non-square linear system with R : A X = B ?

(in the case the system has no solution or infinitely many solutions)

Example :

Answer 1

Aim is to solve Ax = b

where A is p by q, x is q by 1 and b is p by 1 for x given A and b.

Approach 1: Generalized Inverse: Moore-Penrose https://en.wikipedia.org/wiki/Generalized_inverse

Multiplying both sides of the equation, we get

A'Ax = A' b

where A' is the transpose of A. Note that A'A is q by q matrix now. One way to solve this now multiply both sides of the equation by the inverse of A'A. Which gives,

x = (A'A)^{-1} A' b

This is the theory behind generalized inverse. Here G = (A'A)^{-1} A' is pseudo-inverse of A.

library(MASS)

ginv(A) %*% B

#          [,1]
#[1,]  2.411765
#[2,] -2.282353
#[3,]  2.152941
#[4,] -3.470588

Approach 2: Generalized Inverse using SVD.

@duffymo used SVD to obtain a pseudoinverse of A.

However, last elements of svd(A)$d may not be quite as small as in this example. So, probably one shouldn't use that method as is. Here's an example where none of the last elements of A is close to zero.

A <- as.matrix(iris[11:13, -5])    
A
#   Sepal.Length Sepal.Width Petal.Length Petal.Width
# 11          5.4         3.7          1.5         0.2
# 12          4.8         3.4          1.6         0.2
# 13          4.8         3.0          1.4         0.1

svd(A)$d
# [1] 10.7820526  0.2630862  0.1677126

One option would be to look as the singular values in cor(A)

svd(cor(A))$d
# [1] 2.904194e+00 1.095806e+00 1.876146e-16 1.155796e-17

Now, it is clear there is only two large singular values are present. So, one now can apply svd on A to get pseudo-inverse as below.

svda <- svd(A)
G = svda$v[, 1:2] %*% diag(1/svda$d[1:2]) %*% t(svda$u[, 1:2])
# to get x
G %*% B