I have a table view with the potential for each cell to have its own height, thus isn\'t suitable to use rowHeight. Instead, right now I\'m using let indexSet
This behavior appears to be a bug, if for no other reason than it's no longer reproducible on iOS 9. I'm sure that's not much consolation.
The issue primarily derives from having an inaccurate estimate, like @NickCatib said. The best you can do on iOS 8 is to improve the estimation. A technique many have recommended is to cache heights in willDisplayCell and use them on subsequent calls to estimatedRowHeightAtIndexPath.
You might be able to mitigate the behavior by not doing anything to get UITableView to discard its caches, like by modifying the content in a cell directly using cellForRowAtIndexPath rather than using reloading if it's onscreen. However, that won't help if you actually need to change the height of the cell.
I'm afraid to say the bug can't be easily be fixed within a table view, as you don't have control over the layout. The bug can be more easily worked around in a subclass UICollectionViewFlowLayout by changing the contentOffsetAdjustment during invalidation, although that might not be terribly easy.