XCode is a real memory monster, and if you trying using one of the "instruments" it eats even more. Depending on the tasks some people do not have that many issues, specially if they have more memory.
Personally I started with 2GB and XCode was just... painful
After upgrading to 8GB I just had to restart every now and then
I have heard that people with 16GB do not complain that much, and just have to restart XCode once or two a day.
It looks quite strange to me that you use all your memory after 2 minutes. I might guess that there is some setting of yours that is making all XCode leaks much worst. Have you tried deleting all your ~/Library/Developer folder and setting files?
