I have created an AppDomain with a different base directory. However, I cannot seem to load the currently executing assembly into the other AppDomain without ha
My educated guess is that you missed an important part of the error message:
System.IO.FileNotFoundException was unhandled Message=Could not load file or assembly 'TaskExecuter, Version=1.0.4244.31921, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified. Source=mscorlib
Besides not being able to load your assembly from another location than under your ApplicationBase there is probably some dependend assembly missing from where it could be resolved and loaded.
By the way, if you start loading from bytes you should have a look at the assemblies loaded to your domain. The dependend assembly might be loaded already, but the dependency cannot be resolved automatically. If you have the same assembly loaded twice, its types will be incompatible. You'll get funny CastExceptions saying an object of YourClass cannot be cast to YourClass.
You can try to register an AssemblyResolve event handler to your domain, but with this you end up easily with some black magic conjuring stuff from .dll hell. If everything else fails and you go to .dll hell yourself, meet me here: Need to hookup AssemblyResolve event when DisallowApplicationBaseProbing = true