J, 61 63 65 characters
((j.&(1&{)":({.%&1e3{:));{&' kMGTPE'@{.)(([:<.1e3^.{.),{:,{.)
Output:
((j.&(1&{)":({.%&1e3{:));{&' kMGTPE'@{.)(([:<.1e3^.{.),{:,{.) 1500 0
┌─┬─┐
│2│k│
└─┴─┘
((j.&(1&{)":({.%&1e3{:));{&' kMGTPE'@{.)(([:<.1e3^.{.),{:,{.) 987654321987654321 4
┌────────┬─┐
│987.6543│P│
└────────┴─┘
(The reason the output is "boxed" like that is because J doesn't support a list consisting of varying types)
Explanation (from right to left):
(([:<.1000^.{.),{:,{.)
We make a new 3-element list, using , to join ([:<.1000^.{.) (the floored <. base 1000 log ^. of the first param {.. We join it with the second param {: and then the first param {..
So after the first bit, we've transformed say 12345 2 into 1 2 12345
((j.&(1&{)":({.%&1000{:));{&' kMGTPE'@{.) uses ; to join the two halves of the expression together in a box to produce the final output.
The first half is ((j.&(1&{)":({.%&1000{:)) which divides (%) the last input number ({:) by 1000, the first number of times. Then it sets the precision ": using the second number in the input list (1&{).
The second half {&' kMGTPE'@{. - this uses the first number to select ({) the appropriate character from the 0-indexed list of abbreviations.