You need to use the fetchSignInMethodsForEmail API. It takes an email and returns a promise that resolves with the list of providers linked to that email if it is already registered:
https://firebase.google.com/docs/reference/js/firebase.auth.Auth.html#fetchsigninmethodsforemail