What are the limitations of VS2010 C# Express Edition compared with the paid for versions? I was mostly looking into what sort of applications I can build, in VS Express I only
There's a comparison chart here - You'll need VS Professional to develop Office Plugins.
Although previous version, there is a definitive overview of the VS2008 Express limitations in in this SO Question
Wikipedia also explains the differences.
Edit
The comparison chart link is dead. For 2012: