Python performance characteristics

Question

I\'m in the process of tuning a pet project of mine to improve its performance. I\'ve already busted out the profiler to identify hotspots but I\'m thinking understanding Py

Answer 1

S.Lott is right: the big effects are data structures and algorithms. Also, if you are doing a lot of I/O, how you manage it will make a big difference.

But if you are curious about the compiler internals: it will fold constants, but it will not inline functions or unroll loops. Inlining functions is a hard problem in a dynamic language.

You can see what the compiler does by disassembling some compiled code. Put some sample code in my_file.py, then use:

python -m dis my_file.py

This source:

def foo():
    return "BAR!"

for i in [1,2,3]:
    print i, foo()

produces:

  1           0 LOAD_CONST               0 ()
              3 MAKE_FUNCTION            0
              6 STORE_NAME               0 (foo)

  4           9 SETUP_LOOP              35 (to 47)
             12 LOAD_CONST               1 (1)
             15 LOAD_CONST               2 (2)
             18 LOAD_CONST               3 (3)
             21 BUILD_LIST               3
             24 GET_ITER
        >>   25 FOR_ITER                18 (to 46)
             28 STORE_NAME               1 (i)

  5          31 LOAD_NAME                1 (i)
             34 PRINT_ITEM
             35 LOAD_NAME                0 (foo)
             38 CALL_FUNCTION            0
             41 PRINT_ITEM
             42 PRINT_NEWLINE
             43 JUMP_ABSOLUTE           25
        >>   46 POP_BLOCK
        >>   47 LOAD_CONST               4 (None)
             50 RETURN_VALUE

Notice that only the top-level code in the module is disassembled, you need to write a little more code yourself to recurse through the nested code objects if you want to see the function definitions disassembled as well.