I want to run my Java app and for a given workload be able to see:
Exactly this is where AOP helps. Aspects can be added/removed without changing code. If you are using Spring; create Aspect including JoinPoint, Advice listing the Classes and public methods that are to evaluate execution time. Add these beans to the Spring config. Else use AspectJ Container for non-spring application