I am aware of some the obvious gains of the x64 architecture (higher addressable RAM addresses, etc)... but:
Some OSs or configurations are unable to run 32-bit programs. A minimal Linux without 32-bit libc installed for example. Also IIRC I usually compile out the 32-bit support from the kernel.
If these OSs or configurations are part of your potential user base then yes, you should port it.
If you need more speed, then you should also port it (as others have said, x86-64 has more registers and cool instructions that speed it up).
Or, of course, if you want to mmap() or otherwise map a large file or lots of memory. Then 64-bit helps.