How do I need to indicate to the msbuild task in my nant script that it should use .net 4.0 rc?
I think the latest NAnt/NAntContrib defaults to .NET 3.5, so you'll have to change that to 4.0. There is a NAnt property to handle that (), which should go near the top of your NAnt build file.
Next, you'll need to go into your NAnt's configuration file and add the 4.0 node, so that NAnt (and by extension NantContrib) are aware of the new CLR version.