In Visual Studio, there's a 'properties' section when you have the active file open. For the App.config, it has a property called Copy to Output Directory, the options are
Do Not Copy Copy if Newer Copy Always
You want either the second or the third option.
