Difference between Big-Theta and Big O notation in simple language

Question

While trying to understand the difference between Theta and O notation I came across the following statement :

The Theta-no         


        

Answer 1

Here's my attempt:

A function, f(n) is O(n), if and only if there exists a constant, c, such that f(n) <= c*g(n).

Using this definition, could we say that the function f(2^(n+1)) is O(2^n)?

1. In other words, does a constant 'c' exist such that 2^(n+1) <= c*(2^n)? Note the second function (2^n) is the function after the Big O in the above problem. This confused me at first.

2. So, then use your basic algebra skills to simplify that equation. 2^(n+1) breaks down to 2 * 2^n. Doing so, we're left with:

   2 * 2^n <= c(2^n)

3. Now its easy, the equation holds for any value of c where c >= 2. So, yes, we can say that f(2^(n+1)) is O(2^n).

Big Omega works the same way, except it evaluates f(n) >= c*g(n) for some constant 'c'.

So, simplifying the above functions the same way, we're left with (note the >= now):

2 * 2^n >= c(2^n)

So, the equation works for the range 0 <= c <= 2. So, we can say that f(2^(n+1)) is Big Omega of (2^n).

Now, since BOTH of those hold, we can say the function is Big Theta (2^n). If one of them wouldn't work for a constant of 'c', then its not Big Theta.

The above example was taken from the Algorithm Design Manual by Skiena, which is a fantastic book.

Hope that helps. This really is a hard concept to simplify. Don't get hung up so much on what 'c' is, just break it down into simpler terms and use your basic algebra skills.