I created a setup project of window form application built in C# 4.0. I tested the setup in my test PC which have Vs 2010 Ultimate installed in it and its working fine in it
I had the same problem. I've been so confused as to what was causing the issue and have been stepping through debugging line by line for a day or so. After Google searching and typing in the right description, it turns out the issue is with line shapes on a form - and likely other shapes. See the links here, hope it saves somebody else some time... you have to read down through the comments, by like the above post it's with line shapes.
http://social.msdn.microsoft.com/Forums/en-US/cb89a159-c989-470f-b74f-df3f61b9dedd/systeminvalidoperationexception-when-closing-a-form?forum=vbpowerpacks
http://channel9.msdn.com/forums/TechOff/520076-VB-2010-Error-PowerPacks-Line-Shape-Dispose/