If you have a web server up and running with PHP support, I highly recommend phpMyAdmin.
- Configure it to connect to MySQL
- Create the Database and table
- Click the import tab and you can import a CSV.
If this is a simple one-time import this probably isn't worth the effort. If, on the other hand, you will ever have to work with MySQL again, you will never regret the time to install and configure phpMyAdmin.