Update
I have posted my solution to this problem as an answer below. It takes a different approach from my first revision.
Ori
I notice that you're deleting the sections from the table first, and then deleting rows.
I know there's a complicated discussion of batch insertion and deletion for UITableViews in the Table View Programming Guide, but it doesn't specifically cover this.
I think what's happening is that deleting the sections is causing the row deletions to refer to the wrong row.
i.e. you want to delete section #2 and row #1 from section #4... but after you've deleted section #2, the old section #4 is now the third section, so you when you delete with the old NSIndexPath of (4, 1) you're deleting some random different row that may not exist.
So I think the fix might be as simple as swapping those two lines of code, so you're deleting the rows first, then the sections.