We have some NUnit tests that access the database. When one of them fails it can leave database in inconsistent state - which is not an issue, since we rebuild database for
One option not mentioned so far is to wrap the test up in a TransactionScope object, so it doesn't matter what happens as the test never commits anything to the DB.
Here's some details on the technique. You can probably find more if you do a search on unit testing and transactionscope (though you are really doing integration testing if you hit a DB). I've used it successfully in the past.
This approach is simple, does not require any cleanup and ensures that tests are isolated.
Edit- I've just noticed Ray Hayes answer is also similar to mine.