Debugging can be tricky, if it's not working it might be because you're attaching to the wrong code type on the process. Try manually attaching to the process and make sure you've got 'silverlight code' selected instead of 'managed code'. From then on it will probably work all the time again.
