Assume your event handler looks like this:
function(evt) {
// whatever
}
Inside the handler, evt.relatedNode
is the element into which the insertion is being performed. You can switch on it and decide to ignore or process the event.
See it in action.