A) compiling C# EXE\'s and DLL\'s on the fly are relatively easy.
B) Executing an EXE means that a new application is run. Loading a DLL means that methods and functions
Once you have loaded a DLL into (the default appdomain of) a running process, the file on disk cannot be overwritten until the process is terminated. DLLs cannot be unloaded in managed code like they can be in unmanaged code.
You need to create a new appdomain in your host process and load the newly created DLL assembly into that appdomain. When you are ready to compile a new version of the DLL, you can dispose of the appdomain. This will unload the DLL from memory and release the lock on the DLL file, so that you can compile a new DLL to that same file. You can then construct a new appdomain to load the new DLL into.
The main hazard of using appdomains is that all calls across the appdomain boundary must be marshalled, much like an IPC or network RPC. Try to keep the interface of the objects you need to call across the appdomain boundary to a minimum.
You can also compile the assembly to memory, receiving a byte array or stream as the output, then load that assembly into the separate appdomain. This avoids creating debris on disk that will need to be deleted eventually.
Do not use compile to memory as a workaround for the file lock issue. The core issue is that assemblies cannot be removed from memory when they are loaded into the default appdomain of the process. You MUST create a new appdomain and load the DLL into that appdomain if you want to unload that assembly from memory later in the lifetime of the process.
Here's a rough outline of how to construct an object in the context of another appdomain:
var appdomain = AppDomain.CreateDomain("scratch");
byte[] assemblyBytes = // bytes of the compiled assembly
var assembly = appdomain.Load(assemblyBytes);
object obj = appdomain.CreateInstanceAndUnwrap(assembly.FullName, "mynamespace.myclass");
After this sequence, obj will contain a reference to a proxy that links to the actual object instance inside the appdomain. You can invoke methods on obj using reflection or typecast obj to a common interface type and call methods directly. Be prepared to make adjustments to support RPC marshalling of the method call parameters. (see .NET remoting)
When working with multiple appdomains, you have to be careful how you access types and assemblies because a lot of .NET functions default to operating in the current appdomain of the caller, which is usually not what you want when you have multiple appdomains. compilerResult.CompiledAssembly, for example, internally performs a Load of the generated assembly in the caller's appdomain. What you want is to load the assembly into your other appdomain. You have to do that explicitly.
Update: In your recently added code snippet showing how you load your appdomain, this line is your problem:
_assembly = Assembly.LoadFrom(path);
That loads the DLL into the current appdomain (the caller's appdomain), not into the target appdomain (referenced by _domain in your example). You need to do use _domain.Load() to load the assembly into that appdomain.