I am currently trying to reverse a program under Linux that has a bunch of anti-debug tricks. I was able to defeat some of them, but I am still fighting against the remaining ones. Sadly since I am mediocre, it is taking me more time than expected. Anyway, the programs runs without any pain in a VM (I tried with VMWare and VBox), so I was thinking about taking a trace of its execution in the VM, then a trace under the debugger (gdb) and diff them to see were the changes are and find out the anti-debug tricks more easily.
However, I did some kernel debugging with vmware a long time ago, it was more or less ok (I remember having access to the linear address...), but here it's a bit different I think.
Do you see an easy way to debug this userland program without going into too much pain ?
I would suggest using Ether, which is a tool for monitoring the execution of a program and is based on the XEN hypervisor. The whole point of the tool is to trace a program's execution without being observable. The first thing to do is go to their website and click on the malware tab, then submit your binary and see if their automated web interface can do it for you. If this fails, you can install it yourself, which is a pain, but doable, and should yield good results, I have been able to install it in the past. They have instructions on the Ether website, but if you I'd suggest you also take a look at these supplemental instructions from Offensive Computing
A couple of other automated analysis sites that could do the trick for you: Eureka by SRI international and Renovo by bitblaze at UC Berkeley
来源：https://stackoverflow.com/questions/4702358/debugging-userland-program-full-of-anti-debug-through-vmware