I suggest using a CSV parser instead of trying to parse by yourself.
There are some nuances to parsing CSV correctly, as you have already found out.
There are many third party ones (and several of these are free), and even one built into the Visual Basic namespace - the TextFieldParser in the Microsoft.VisualBasic.FileIO
namespace.