Haskell, 126 (without import, it's a function that takes two arguments):
f n p|l>3=showFFloat (Just p) (c n/c 10^(l-w)) [" kMGTPE"!!f]|True=show n where(f,w)=divMod l 3;c=fromIntegral;l=length$show n
Expanded:
import Numeric
doit :: Integer -> Int -> String
doit n p
| l > 3 = showFFloat (Just p) d [" kMGTPE" !! f]
| otherwise = show n
where
d = (fromIntegral n) / fromIntegral (10^(l-w))
(f,w) = divMod l 3
l = length $ show n
