Cheap exception handling in Python?

Question

I read in an earlier answer that exception handling is cheap in Python so we shouldn\'t do pre-conditional checking.

I have not heard of this before, but I\'m relati

Answer 1

Putting aside the performance measurements that others have said, the guiding principle is often structured as "it is easier to ask forgiveness than ask permission" vs. "look before you leap."

Consider these two snippets:

# Look before you leap
if not os.path.exists(filename):
    raise SomeError("Cannot open configuration file")
f = open(filename)

vs.

# Ask forgiveness ...
try:
  f = open(filename)
except IOError:
  raise SomeError("Cannot open configuration file")

Equivalent? Not really. OSes are multi-taking systems. What happens if the file was deleted between the test for 'exists' and 'open' call?

What happens if the file exists but it's not readable? What if it's a directory name instead of a file. There can be many possible failure modes and checking all of them is a lot of work. Especially since the 'open' call already checks and reports all of those possible failures.

The guideline should be to reduce the chance of inconsistent state, and the best way for that is to use exceptions instead of test/call.

Answer 2

What are static versus dynamic calls and returns, and why do you think that calls and returns are any different in Python depending on if you are doing it in a try/except block? Even if you aren't catching an exception, Python still has to handle the call possibly raising something, so it doesn't make a difference to Python in regards to how the calls and returns are handled.

Every function call in Python involves pushing the arguments onto the stack, and invoking the callable. Every single function termination is followed by the caller, in the internal wiring of Python, checking for a successful or exception termination, and handles it accordingly. In other words, if you think that there is some additional handling when you are in a try/except block that is somehow skipped when you are not in one, you are mistaken. I assume that is what you "static" versus "dynamic" distinction was about.

Further, it is a matter of style, and experienced Python developers come to read exception catching well, so that when they see the appropriate try/except around a call, it is more readable than a conditional check.

Answer 3

"Can someone explain this to me?"

Depends.

Here's one explanation, but it's not helpful. Your question stems from your assumptions. Since the real world conflicts with your assumptions, it must mean your assumptions are wrong. Not much of an explanation, but that's why you're asking.

"Exception handling means a dynamic call and a static return, whereas an if statement is static call, static return."

What does "dynamic call" mean? Searching stack frames for a handler? I'm assuming that's what you're talking about. And a "static call" is somehow locating the block after the if statement.

Perhaps this "dynamic call" is not the most costly part of the operation. Perhaps the if-statement expression evaluation is slightly more expensive than the simpler "try-it-and-fail".

Turns out that Python's internal integrity checks are almost the same as your if-statement, and have to be done anyway. Since Python's always going to check, your if-statement is (mostly) redundant.

You can read about low-level exception handling in http://docs.python.org/c-api/intro.html#exceptions.

---

Edit

More to the point: The if vs. except debate doesn't matter.

Since exceptions are cheap, do not label them as a performance problem.

Use what makes your code clear and meaningful. Don't waste time on micro-optimizations like this.

Answer 4

The general message, as S.Lott said, is that try/except doesn't hurt so you should feel free to use it whenever it seems appropriate.

This debate is often called "LBYL vs EAFP" – that's "look before you leap" vs "easier to ask forgiveness than permission". Alex Martelli weighs forth on the subject here: http://mail.python.org/pipermail/python-list/2003-May/205182.html This debate is almost six years old, but I don't think the basic issues have changed very much.

Answer 5

With Python, it is easy to check different possibilities for speed - get to know the timeit module :

... example session (using the command line) that compare the cost of using hasattr() vs. try/except to test for missing and present object attributes.

% timeit.py 'try:' '  str.__nonzero__' 'except AttributeError:' '  pass'
100000 loops, best of 3: 15.7 usec per loop
% timeit.py 'if hasattr(str, "__nonzero__"): pass'
100000 loops, best of 3: 4.26 usec per loop
% timeit.py 'try:' '  int.__nonzero__' 'except AttributeError:' '  pass'
1000000 loops, best of 3: 1.43 usec per loop
% timeit.py 'if hasattr(int, "__nonzero__"): pass'
100000 loops, best of 3: 2.23 usec per loop

These timing results show in the hasattr() case, raising an exception is slow, but performing a test is slower than not raising the exception. So, in terms of running time, using an exception for handling exceptional cases makes sense.

EDIT: The command line option -n will default to a large enough count so that the run time is meaningful. A quote from the manual:

If -n is not given, a suitable number of loops is calculated by trying successive powers of 10 until the total time is at least 0.2 seconds.

Answer 6

You might find this post helpful: Try / Except Performance in Python: A Simple Test where Patrick Altman did some simple testing to see what the performance is in various scenarios pre-conditional checking (specific to dictionary keys in this case) and using only exceptions. Code is provided as well if you want to adapt it to test other conditionals.

The conclusions he came to:

From these results, I think it is fair to quickly determine a number of conclusions:

1. If there is a high likelihood that the element doesn't exist, then you are better off checking for it with has_key.
2. If you are not going to do anything with the Exception if it is raised, then you are better off not putting one have the except
3. If it is likely that the element does exist, then there is a very slight advantage to using a try/except block instead of using has_key, however, the advantage is very slight.