To sort on the fourth column use just the -k 4,4 selector.
sort -t $'\t' -k 4,4 <filename>
You might also want -V which sorts numbers more naturally. For example, yielding 1 2 10 rather than 1 10 2 (lexicographic order).
sort -t $'\t' -k 4,4 -V <filename>
If you're getting errors about the $'\t' then make sure your shell is bash. Perhaps you're missing #!/bin/bash at the top of your script?