Profiling CherryPy

Question

I've been trying to start profiling my CherryPy webserver, but the documentation is lacking in detail in how this should be set up. I understand that I should be able to use cherrypy.lib.profiler as middleware to mount my initial server. Right now, I have code like the following:

server_app = ServerClass()
cherrypy.tree.mount(server_app, '/', '/path/to/config/file.cfg')
cherrypy.engine.start()
cherrypy.engine.block()

I want to mount the profiling middleware, and it seems that something like the following is required:

from cherrypy.lib import profiler

server_app = ServerClass()
server_cpapp = cherrypy.Application(server_app, '/', '/path/to/config/file.cfg')
server_profile_cpapp = profiler.make_app(server_cpapp, '/home/ken/tmp/cprofile', True)
#cherrypy.tree.mount(server_profile_cpapp)
cherrypy.tree.graft(server_profile_cpapp)

cherrypy.engine.start()
cherrypy.engine.block()

For some reason cherrypy.tree.mount doesn't work, but if I use cherrypy.tree.graft all seems to operate fine (I can make requests to the server as normal)

However, the above code generates a cp_0001.prof file under /home/ken/tmp/cprofile and I am not sure how to interpret it. I have tried using pyprof2calltree to read the data into KCacheGrind, but I get a parsing error. Does what I'm doing seem correct, and if so how do I interpret the output file?

Answer 1

It turns out that the profile files generated by CherryPy can be interpreted using the profiler.py script shipped as part of CherryPy. Simply run profiler.py in the <site-packages>/cherrypy/lib directory as follows:

python profiler.py /directory/containing/prof/files 8080

Then navigate to localhost:8080 in your browser and the profiling results for all .prof files in the target directory will be displayed in a simple text interface.

I would still prefer to be able to export the results into a calltree to profile using KCacheGrind, but this seems to do for basic profiling.

This is documented in the change log for v2.1 of CherryPy when the profiler was introduced (although the other details on that page describing how to set up the profiler has since become deprecated)

Answer 2

I am also trying to get profiling up and running for a cherrypy instance. I used the same code you have in your initial question, which seems to work in that it generates a cp_0001.prof file in the folder.

To answer your question, I am able to open this file in runsnakerun to see the profiling output in a tree view.

The problem I have is that every request I do to the server now fails, with the following output in the log:

[29/May/2013:16:39:32] ENGINE AssertionError('Bad call', ('', 0, 'sleep'), <frame object at 0x08522400>, <frame object at 0x08522030>, <frame object at 0x08156748>, <frame object at 0x06D06F10>)
Traceback (most recent call last):
  File "<path>\packages\cherrypy\wsgiserver\wsgiserver2.py", line 1302, in communicate
    req.respond()
  File "<path>\packages\cherrypy\wsgiserver\wsgiserver2.py", line 831, in respond
    self.server.gateway(self).respond()
  File "<path>\packages\cherrypy\wsgiserver\wsgiserver2.py", line 2115, in respond
    response = self.req.server.wsgi_app(self.env, self.start_response)
  File "<path>\packages\cherrypy\_cptree.py", line 290, in __call__
    return app(environ, start_response)
  File "<path>\packages\cherrypy\lib\profiler.py", line 188, in __call__
    return self.profiler.run(gather)
  File "<path>\packages\cherrypy\lib\profiler.py", line 147, in run
    result = self.profiler.runcall(func, *args)
  File "<path>\python\lib\profile.py", line 472, in runcall
    return func(*args, **kw)
  File "<path>\packages\cherrypy\lib\profiler.py", line 183, in gather
    def gather():
  File "<path>\python\lib\profile.py", line 246, in trace_dispatch_i
    if self.dispatch[event](self, frame, t):
  File "<path>\python\lib\profile.py", line 301, in trace_dispatch_call
     frame, frame.f_back)
AssertionError: ('Bad call', ('', 0, 'sleep'), <frame object at 0x08522400>, <frame object at 0x08522030>, <frame object at 0x08156748>, <frame object at 0x06D06F10>)

I am using python 2.6.6 and cherrypy 3.2.2

Any suggestions?