<stdio.h> refers to a header (not a header file)
"stdio.h" refers to a source file.
Headers need not exist phisically in the implementation; the way they are identified is implementation-defined (usually the headers are files on specific directories)
When the directive uses ", the source file is searched in an implementation-defined manner and, if not found, the directive is reprocessed as if it was written with < and > in the first place.