To sort on the fourth column use just the -k 4,4 selector.
sort -t $'\t' -k 4,4
You might also want -V which sorts numbers more naturally. For example, yielding 1 2 10 rather than 1 10 2 (lexicographic order).
sort -t $'\t' -k 4,4 -V
If you're getting errors about the $'\t' then make sure your shell is bash. Perhaps you're missing #!/bin/bash at the top of your script?