What is the complexity of matrix addition?

Question

I have found some mentions in another question of matrix addition being a quadratic operation. But I think it is linear.

If I double the size of a matrix, I need to calculate double the additions, not quadruple.

The main diverging point seems to be what is the size of the problem. To me, it's the number of elements in the matrix. Others think it is the number of columns or lines, hence the O(n^2) complexity.

Another problem I have with seeing it as a quadratic operation is that that means adding 3-dimensional matrices is cubic, and adding 4-dimensional matrices is O(n^4), etc, even though all of these problems can be reduced to the problem of adding two vectors, which has an obviously linear solution.

Am I right or wrong? If wrong, why?

Answer 1

As you already noted, it depends on your definition of the problem size: is it the total number of elements, or the width/height of the matrix. Which ever is correct actually depends on the larger problem of which the matrix addition is part of.

NB: on some hardware (GPU, vector machines, etc) the addition might run faster than expected (even though complexity is still the same, see discussion below), because the hardware can perform multiple additions in one step. For a bounded problem size (like n < 3) it might even be one step.

Answer 2

It's O(M*N) for a 2-dimensional matrix with M rows and N columns.

Or you can say it's O(L) where L is the total number of elements.

Answer 3

think of the general case implementation:

for 1 : n
 for 1 : m
   c[i][j] = a[i][j] + b[i][j]

if we take the simple square matrix, that is n x n additions

Answer 4

Usually the problem is defined using square matrices "of size N", meaning NxN. By that definition, matrix addition is an O(N^2) since you must visit each of the NxN elements exactly once.

By that same definition, matrix multiplication (using square NxN matrices) is O(N^3) because you need to visit N elements in each of the source matrices to compute each of the NxN elements in the product matrix.

Generally, all matrix operations have a lower bound of O(N^2) simply because you must visit each element at least once to compute anything involving the whole matrix.