bulk-load

What mysql settings affect the speed of LOAD DATA INFILE?

白昼怎懂夜的黑 提交于 2019-12-05 00:26:34
问题 Let me set up the situation. We are trying to insert a modestly high number of rows (roughly 10-20M a day) into a MyISAM table that is modestly wide: +--------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+--------------+------+-----+---------+-------+ | blah1 | varchar(255) | NO | PRI | | | | blah2 | varchar(255) | NO | PRI | | | | blah3 | varchar(5) | NO | PRI | | | | blah4 | varchar(5) | NO | PRI | | | | blah5 | varchar

OPENROWSET BULK Permissions to Shared Folder

余生颓废 提交于 2019-12-04 05:03:21
问题 OBJECTIVE Use the OPENROWSET feature to JOIN data in a query against a text file. ERROR Leveraging the answer from @gbn on this question I am trying to open a row set just like the OP; though the format of the file is a bit different. However, I'm getting the following error trying to access a shared folder: Msg 4861, Level 16, State 1, Line 1 Cannot bulk load because the file "\MACHINENAME\Share\EC04.txt" could not be opened. Operating system error code 5(Access is denied.). BACKGROUND

How to read a ARRAY of types returned from a stored proc using java?

时间秒杀一切 提交于 2019-12-03 21:23:42
This is a continuation of the question posted under the following location: Java program to pass List of Bean to a oracle stored procedure - Pass entire list at one shot rather than appending objects one after the other I have been trying to enhance the stored procedure mentioned in the above link location and am confused in the implementation. Rather than VARCHAR2 as a output from the procedure i now want to return NUM_ARRAY as the output from the procedure. Can you please help me in implementing the logic to read the NUM_ARRAY in my java code. Normally output is returned using Map out =

What mysql settings affect the speed of LOAD DATA INFILE?

怎甘沉沦 提交于 2019-12-03 15:23:41
Let me set up the situation. We are trying to insert a modestly high number of rows (roughly 10-20M a day) into a MyISAM table that is modestly wide: +--------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+--------------+------+-----+---------+-------+ | blah1 | varchar(255) | NO | PRI | | | | blah2 | varchar(255) | NO | PRI | | | | blah3 | varchar(5) | NO | PRI | | | | blah4 | varchar(5) | NO | PRI | | | | blah5 | varchar(2) | NO | PRI | | | | blah6 | varchar(2) | NO | PRI | | | | blah7 | date | NO | PRI | | | | blah8 |

Cannot bulk load because the file could not be opened. Operating System Error Code 3

旧巷老猫 提交于 2019-12-03 01:04:27
I'm trying to set up a Stored Procedure as a SQL Server Agent Job and it's giving me the following error, Cannot bulk load because the file "P:\file.csv" could not be opened. Operating system error code 3(failed to retrieve text for this error. Reason: 15105). [SQLSTATE 42000] (Error 4861) Funny thing is the Stored Procedure works just fine when I execute it manually. The drive P: is a shared drive on Windows SQL Server from LINUX via Samba Share and it was set up by executing the following command, EXEC xp_cmdshell 'net use P: "\lnxusanfsd01\Data" Password /user:username /Persistent:Yes' Any

Spark issues in creating hfiles- Added a key not lexically larger than previous cell

為{幸葍}努か 提交于 2019-12-02 05:57:08
问题 I am trying to create hfiles to do bulk load into Hbase and it keeps throwing the error with the row key even though everything looks fine. I am using the following code: val df = sqlContext.read.format("com.databricks.spark.csv") .option("header", "true") .option("inferSchema", "true") .load("data.csv") import sqlContext.implicits._ val DF2 = df.filter($"company".isNotNull) .dropDuplicates(Array("company")) .sortWithinPartitions("company").sort("company") val rdd = DF2.flatMap(x => { val

Spark issues in creating hfiles- Added a key not lexically larger than previous cell

荒凉一梦 提交于 2019-12-02 01:02:43
I am trying to create hfiles to do bulk load into Hbase and it keeps throwing the error with the row key even though everything looks fine. I am using the following code: val df = sqlContext.read.format("com.databricks.spark.csv") .option("header", "true") .option("inferSchema", "true") .load("data.csv") import sqlContext.implicits._ val DF2 = df.filter($"company".isNotNull) .dropDuplicates(Array("company")) .sortWithinPartitions("company").sort("company") val rdd = DF2.flatMap(x => { val rowKey = Bytes.toBytes(x(0).toString) for (i <- 0 to cols.length - 1) yield { val index = x.fieldIndex(new

Solutions to put different values for a row-key but the same timestamps in hbase?

我们两清 提交于 2019-12-01 22:27:52
问题 I'm new at Hbase. I'm facing a problem when bulk loading data from a text file into Hbase. Assuming I have a following table: Key_id | f1:c1 | f2:c2 row1 'a' 'b' row1 'x' 'y' When I parse 2 records and put it into Hbase at the same time (same timestamps), then only version {row1 'x' 'y'} updated. Here is the explanation: When you put data into HBase, a timestamp is required. The timestamp can be generated automatically by the RegionServer or can be supplied by you. The timestamp must be

Solutions to put different values for a row-key but the same timestamps in hbase?

你离开我真会死。 提交于 2019-12-01 21:20:31
I'm new at Hbase. I'm facing a problem when bulk loading data from a text file into Hbase. Assuming I have a following table: Key_id | f1:c1 | f2:c2 row1 'a' 'b' row1 'x' 'y' When I parse 2 records and put it into Hbase at the same time (same timestamps), then only version {row1 'x' 'y'} updated. Here is the explanation: When you put data into HBase, a timestamp is required. The timestamp can be generated automatically by the RegionServer or can be supplied by you. The timestamp must be unique per version of a given cell, because the timestamp identifies the version. To modify a previous

Bulk request throws error in elasticsearch 6.1.1

时光总嘲笑我的痴心妄想 提交于 2019-12-01 03:32:02
I recently upgraded to elasticsearch version 6.1.1 and now I can't bulk index documents from a json file. Wehn I do it inline, it works fine. Here are the contents of the document: {"index" : {}} {"name": "Carlson Barnes", "age": 34} {"index":{}} {"name": "Sheppard Stein","age": 39} {"index":{}} {"name": "Nixon Singleton","age": 36} {"index":{}} {"name": "Sharron Sosa","age": 33} {"index":{}} {"name": "Kendra Cabrera","age": 24} {"index":{}} {"name": "Young Robinson","age": 20} When I run this command, curl -XPUT 'localhost:9200/subscribers/ppl/_bulk?pretty' -H 'Content-Type: application/json'