Difference between Big-Theta and Big O notation in simple language

Question

While trying to understand the difference between Theta and O notation I came across the following statement :

The Theta-no         


        

Answer 1

I will just quote from Knuth's TAOCP Volume 1 - page 110 (I have the Indian edition). I recommend reading pages 107-110 (section 1.2.11 Asymptotic representations)

People often confuse O-notation by assuming that it gives an exact order of Growth; they use it as if it specifies a lower bound as well as an upper bound. For example, an algorithm might be called inefficient because its running time is O(n^2). But a running time of O(n^2) does not necessarily mean that running time is not also O(n)

On page 107,

1^2 + 2^2 + 3^2 + ... + n^2 = O(n^4) and

1^2 + 2^2 + 3^2 + ... + n^2 = O(n^3) and

1^2 + 2^2 + 3^2 + ... + n^2 = (1/3) n^3 + O(n^2)

Big-Oh is for approximations. It allows you to replace ~ with an equals = sign. In the example above, for very large n, we can be sure that the quantity will stay below n^4 and n^3 and (1/3)n^3 + n^2 [and not simply n^2]

Big Omega is for lower bounds - An algorithm with Omega(n^2) will not be as efficient as one with O(N logN) for large N. However, we do not know at what values of N (in that sense we know approximately)

Big Theta is for exact order of Growth, both lower and upper bound.