So my question is how to delete bundle I created?
You create bundles with this console command:
php app/console generate:bundle --namespace=Test/BlogBund
It is basically the process you have outlined, only in somewhat different order.
If this wasn't installed using a dependency manager - that should be all.