An airplane mechanic and a surgeon meet in a bar. The mechanic says "you know, we have basically the same job. We take broken stuff out and put new, better parts in." The surgeon says "yeah, but you don't have to keep the plane flying as you're making the repairs!"
Trying to update an application by moving files while the application is running seems about as dangerous as trying to fix an airplane in flight. Possible? Sure. Greatly increased risk of catestrophic crash? Yep.
If the application you are updating is a managed application, consider using ClickOnce Deployment. That way, the next time someone runs the application, if there is a new version available it will be copied down and installed automatically. That's much more safe and pleasant than trying to mess with an application while its still running.