Edit
Please see also How do I properly clean up Excel interop objects?. I recently came across this question, and it provided a lot of insight into
You should call Marshal.ReleaseComObject on every COM object you use in your code, not just the main application object.