When I attempt to debug a unit test that fails because of an unhandled exception in my code, I expect Visual Studio to break on the unhandled exception so I can inspect the
The reason why is that your exceptions are not unhandled. The unit testing framework is handling the exceptions under the hood and converting them to failures.
What you need to do instead is to break on thrown exceptions. If you combine this with enabling "Just My Code" (on by default) you should get the behavior you are expecting. Visual Studio will only break when exceptions thrown by your code occur.