MySQL LOAD DATA INFILE Data too long for column exception

Question

I'm using MySQL LOAD DATA INFILE Data command to bulk insert data to a table. Here's how I am doing it :

LOAD DATA INFILE 'MyFile.csv' INTO TABLE `dbname`.`tablename` FIELDS TERMINATED BY '\t' ENCLOSED BY '"' LINES TERMINATED BY '\r\n' ; 

When I run it from our C# project I'm getting a Data too long for column xxx exception for a char(50) column which the provided data for it is less than 50 (but it is in Persian)but when I use a MySql client such as SQLyog it is working fine.

Here's how I am running this command :

private static void RunCommand(string command,params object[] args)
        {
            if (args != null)
                command = string.Format(command, args);
            using (var conn = MySqlClientFactory.Instance.CreateConnection())
            {
                if (conn == null) return;
                conn.ConnectionString =
                    "Server=localhost;Uid=root;Pwd=123456; AutoEnlist=false;Charset=utf8;";
                conn.Open();
                using (var comm = conn.CreateCommand())
                {
                    comm.CommandText = command;
                    comm.ExecuteNonQuery();
                }
            }
        }

I guess that it could be an issue of converting Unicode characters but I can not figure out how can I make it run correctly.

Answer 1

Add CHARACTER SET utf8 parameter to LOAD DATA INFILE... statement:

LOAD DATA INFILE 'MyFile.csv'
 INTO TABLE `dbname`.`tablename`
 CHARACTER SET utf8
 FIELDS TERMINATED BY '\t' ENCLOSED BY '"'
 LINES TERMINATED BY '\r\n';

As stated in documentation it specifies which character set is used in the file:

The character set indicated by the character_set_database system variable is used to interpret the information in the file. SET NAMES and the setting of character_set_client do not affect interpretation of input. If the contents of the input file use a character set that differs from the default, it is usually preferable to specify the character set of the file by using the CHARACTER SET clause, which is available as of MySQL 5.1.17.

If it is not specified then gets default value, for example latin1 and each utf-8 byte is interpreted as character. Because some utf-8 encoded characters has more than one byte you got longer strings.