I have a git repo with tens of remotes which have been merge
You may need to prune your local "cache" of remote branches first. Try running:
git fetch -p origin
before deleting.