React, browserify and “Attempted import error” if I don't use js extension

Question

I have a file called constants.js. When I import it as import { Foo } from "constants" I get an "Attempted import error: \'Foo\' is