When I run this example and create a rectangular selection if I zoom or move the plot window around the selection disappears until I deselect the move or zoom tool and click on
If I understand correctly, the rectangle selector should stay visible throughout the process of panning or zooming. This could be achieved by not using blitting,
toggle_selector.RS = RectangleSelector(ax, ..., useblit=False, ...)
A side effect of this is that the plotting may become slow depending on the complexity of the plot, as without blitting, the complete plot is continuously redrawn while using the rectangle selector.