I want to know the exact difference between the dll and exe file.
The .exe is the program. The .dll is a library that a .exe (or another .dll) may call into.
What sakthivignesh says can be true in that one .exe can use another as if it were a library, and this is done (for example) with some COM components. In this case, the "slave" .exe is a separate program (strictly speaking, a separate process - perhaps running on a separate machine), but one that accepts and handles requests from other programs/components/whatever.
However, if you just pick a random .exe and .dll from a folder in your Program Files, odds are that COM isn't relevant - they are just a program and its dynamically-linked libraries.
Using Win32 APIs, a program can load and use a DLL using the LoadLibrary and GetProcAddress API functions, IIRC. There were similar functions in Win16.
COM is in many ways an evolution of the DLL idea, originally concieved as the basis for OLE2, whereas .NET is the descendant of COM. DLLs have been around since Windows 1, IIRC. They were originally a way of sharing binary code (particularly system APIs) between multiple running programs in order to minimise memory use.