load-data-infile

mysql duplicates with LOAD DATA INFILE

我与影子孤独终老i 提交于 2019-11-28 13:03:58
When using LOAD DATA INFILE, is there a way to either flag a duplicate row, or dump any/all duplicates into a separate table? From the LOAD DATE INFILE documentation : The REPLACE and IGNORE keywords control handling of input rows that duplicate existing rows on unique key values: If you specify REPLACE, input rows replace existing rows. In other words, rows that have the same value for a primary key or unique index as an existing row. See Section 12.2.7, “REPLACE Syntax” . If you specify IGNORE, input rows that duplicate an existing row on a unique key value are skipped. If you do not specify

LOAD DATA LOCAL INFILE fails - from php, to mysql (on Amazon rds)

别等时光非礼了梦想. 提交于 2019-11-28 08:44:53
We're moving our database from being on the webserver to a separate server (from an Amazon EC2 webserver to an RDS instance.) We have a LOAD DATA INFILE that worked before that is going to need the LOCAL keyword added now that the database will be on a different machine to the webserver. Testing on my dev server, it turns out that it doesn't work: I can still LOAD DATA INFILE from php as I have been I can LOAD DATA LOCAL INFILE from mysql commandline (with --local_infile=1) I can't LOAD DATA LOCAL INFILE from php. Between those 2 things that do work, it rules out: problems with the sql or php

Google Cloud SQL: Alternative to LOAD DATA INFILE

余生颓废 提交于 2019-11-28 08:38:39
问题 I am working in Google App Engine and we have a Python script that dumps data in Google Cloud SQL. One of the data sets we have to dump is huge. We dump around 150K rows of data once a day daily. I know Google Cloud SQL does not support LOAD DATA INFILE , which I would have normally used. My question is, whether there is an alternative to LOAD DATA INFILE that I can use to speed up the process of data dumping. Inserting the data normally, without LOAD DATA INFILE , takes about 5 minutes. 回答1:

MYSQL LOAD DATA INFILE ignore duplicate rows (autoincrement as primary key)

爷,独闯天下 提交于 2019-11-28 07:32:04
I ran into some trouble using LOAD DATA INFILE command as i wanted to ignore the lines that was already in the data base..say if i have a table with data as follows, id |name |age -------------------- 1 |aaaa |22 2 |bbbb |21 3 |bbaa |20 4 |abbb |22 5 |aacc |22 Where id is auto increment value. an the csv file i have contains data as follows, "cccc","14" "ssee","33" "dddd","22" "aaaa","22" "abbb","22" "dhgg","34" "aacc","22" I want to ignore the rows, "aaaa","22" "abbb","22" "aacc","22" and upload the rest to the table. and the query i have yet which uploads everything to the table is as

How to use LOAD DATA INFILE statement when file is another location?

孤街醉人 提交于 2019-11-28 03:26:41
问题 I want to use LOAD DATA INFILE statement to import data in my table. If the file is available on the same system, it works nicely and imports successfully, but when the file is located on another system, it cannot locate the path. Can anyone explain how to use the LOAD DATA INFILE statement to import data into MySQL table from another system, or remotely? 回答1: When you do a LOAD DATA INFILE, the file must be on the system that is running the MySQL database, in the data path. If the file is on

Tool for importing CSV files into MySQL database? [closed]

落爺英雄遲暮 提交于 2019-11-28 02:03:02
问题 As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 7 years ago . I've got several CSV files I want to import into a table. They all contain different numbers of columns, so some data will be absent

Error 1148 MySQL The used command is not allowed with this MySQL version

萝らか妹 提交于 2019-11-27 22:20:13
I am using MySQL LOAD DATA LOCAL INFILE command and I get this error: PDOException: SQLSTATE[42000]: Syntax error or access violation: 1148 The used command is not allowed with this MySQL version: LOAD DATA LOCAL INFILE '/tmp/phpI0ox54' INTO TABLE `dev_tmp` FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES; Array ( ) in dc_real_estate_form_submit() (line 147 of /PATH/TO/PHP/SCRIPT). What setting can we change to allow LOAD DATA LOCAL infile? Here is the Drupal 7 code we are using: $sql = "LOAD DATA LOCAL INFILE '".$file."' INTO TABLE `dev_tmp` FIELDS TERMINATED

MySQL Convert latin1 data to UTF8

天大地大妈咪最大 提交于 2019-11-27 19:55:19
I imported some data using LOAD DATA INFILE into a MySQL Database. The table itself and the columns are using the UTF8 character set, but the default character set of the database is latin 1. Because the default character type of the database is latin1, and I used LOAD DATA INFILE without specifying a character set, it interpreted the file as latin1, even though the data in the file was UTF8. Now I have a bunch of badly encoded data in my UTF8 colum. I found this article which seems to address a similar problem, which is "UTF8 inserted in cp1251", but my problem is "Latin1 inserted in UTF8". I

What is the fastest way to load an XML file into MySQL using C#?

此生再无相见时 提交于 2019-11-27 15:04:52
问题 Question What is the fastest way to dump a large (> 1GB) XML file into a MySQL database? Data The data in question is the StackOverflow Creative Commons Data Dump. Purpose This will be used in an offline StackOverflow viewer I am building, since I am looking to do some studying/coding in places where I will not have access to the internet. I would like to release this to the rest of the StackOverflow membership for their own use when the project is finished. Problem Originally, I was reading

MySql bulk load command line tool

半世苍凉 提交于 2019-11-27 08:30:27
Does MySql have a bulk load command line tool like bcp for SQLServer and sqlldr for Oracle? I know there's a SQL command LOAD INFILE or similar but I sometimes need to bulk load a file that is on a different box to the MySQL database. mysqlimport. takes the same connection parameters as the mysql command line shell. Make sure to use the -L flag to use a file on the local file system, otherwise it will (strangely) assume the file is on the server. There is also an analogous variant to the load data infile command, i.e., load data local infile , according to which the file will be loaded from