csv

Merging csv-files via batch

蓝咒 提交于 2021-02-11 17:11:41
问题 I need to merge a lot small csv-files from different source into separate bigger csv-files Usually I use: for /r E:\MyFolder %%d in (.) do ( copy "%%d\*.csv" "%%d\merge.txt" /B ) This works fine, but for a certain type of csv file, where a new line at the end missing where this won't work correctly. I can use : for %%x in (E:\MyFolder\case1\*.csv) do more "%%x" >>E:\MyFolder\case1\merge.txt But it won't run recursively, only for the top folder, obviously. I tried wrapping the same for loop as

Merging csv-files via batch

余生颓废 提交于 2021-02-11 17:11:24
问题 I need to merge a lot small csv-files from different source into separate bigger csv-files Usually I use: for /r E:\MyFolder %%d in (.) do ( copy "%%d\*.csv" "%%d\merge.txt" /B ) This works fine, but for a certain type of csv file, where a new line at the end missing where this won't work correctly. I can use : for %%x in (E:\MyFolder\case1\*.csv) do more "%%x" >>E:\MyFolder\case1\merge.txt But it won't run recursively, only for the top folder, obviously. I tried wrapping the same for loop as

Merging csv-files via batch

橙三吉。 提交于 2021-02-11 17:10:48
问题 I need to merge a lot small csv-files from different source into separate bigger csv-files Usually I use: for /r E:\MyFolder %%d in (.) do ( copy "%%d\*.csv" "%%d\merge.txt" /B ) This works fine, but for a certain type of csv file, where a new line at the end missing where this won't work correctly. I can use : for %%x in (E:\MyFolder\case1\*.csv) do more "%%x" >>E:\MyFolder\case1\merge.txt But it won't run recursively, only for the top folder, obviously. I tried wrapping the same for loop as

Merging csv-files via batch

十年热恋 提交于 2021-02-11 17:10:46
问题 I need to merge a lot small csv-files from different source into separate bigger csv-files Usually I use: for /r E:\MyFolder %%d in (.) do ( copy "%%d\*.csv" "%%d\merge.txt" /B ) This works fine, but for a certain type of csv file, where a new line at the end missing where this won't work correctly. I can use : for %%x in (E:\MyFolder\case1\*.csv) do more "%%x" >>E:\MyFolder\case1\merge.txt But it won't run recursively, only for the top folder, obviously. I tried wrapping the same for loop as

mongodb 4x slower than sqlite, 2x slower than csv?

风流意气都作罢 提交于 2021-02-11 16:52:20
问题 I am comparing performance of the two dbs, plus csv - data is 1 million row by 5 column float, bulk insert into sqlite/mongodb/csv, done in python. import csv import sqlite3 import pymongo N, M = 1000000, 5 data = np.random.rand(N, M) docs = [{str(j): data[i, j] for j in range(len(data[i]))} for i in range(N)] writing to csv takes 6.7 seconds: %%time with open('test.csv', 'w', newline='') as file: writer = csv.writer(file, delimiter=',') for i in range(N): writer.writerow(data[i]) writing to

mongodb 4x slower than sqlite, 2x slower than csv?

為{幸葍}努か 提交于 2021-02-11 16:51:23
问题 I am comparing performance of the two dbs, plus csv - data is 1 million row by 5 column float, bulk insert into sqlite/mongodb/csv, done in python. import csv import sqlite3 import pymongo N, M = 1000000, 5 data = np.random.rand(N, M) docs = [{str(j): data[i, j] for j in range(len(data[i]))} for i in range(N)] writing to csv takes 6.7 seconds: %%time with open('test.csv', 'w', newline='') as file: writer = csv.writer(file, delimiter=',') for i in range(N): writer.writerow(data[i]) writing to

how to get the column headers for CSV data with Powershell?

帅比萌擦擦* 提交于 2021-02-11 15:49:13
问题 How can I get the column headers for CSV data? Here is the specific data: PS /home/nicholas/powershell/covid> PS /home/nicholas/powershell/covid> $labsURL='http://www.bccdc.ca/Health-Info-Site/Documents/BCCDC_COVID19_Dashboard_Lab_Information.csv' PS /home/nicholas/powershell/covid> PS /home/nicholas/powershell/covid> $labs_csv=Invoke-RestMethod -Method Get -Uri $labsURL -Headers @{"Content-Type" = "text/csv"} PS /home/nicholas/powershell/covid> PS /home/nicholas/powershell/covid> $labs =

CSV to adjacency matrix

你。 提交于 2021-02-11 15:48:41
问题 I am trying to visualize some data I got using python and bokeh, but it is not working out. It is a .csv file and it looks like this, only then with much more rows and values: ;A;B;C;DA;0;0;1;2;B;0;3;0;0;C;0;0;0;1;D;1;0;2;0 I have tried some stuff, but it did not get me far and now I am kind of stuck. I tried to make the list with this: import csv with open('coauth.csv', 'r') as f: reader = csv.reader(f) your_list = list(reader) print(your_list) But then it outputs a nested list like this: [[

how to get the column headers for CSV data with Powershell?

坚强是说给别人听的谎言 提交于 2021-02-11 15:47:24
问题 How can I get the column headers for CSV data? Here is the specific data: PS /home/nicholas/powershell/covid> PS /home/nicholas/powershell/covid> $labsURL='http://www.bccdc.ca/Health-Info-Site/Documents/BCCDC_COVID19_Dashboard_Lab_Information.csv' PS /home/nicholas/powershell/covid> PS /home/nicholas/powershell/covid> $labs_csv=Invoke-RestMethod -Method Get -Uri $labsURL -Headers @{"Content-Type" = "text/csv"} PS /home/nicholas/powershell/covid> PS /home/nicholas/powershell/covid> $labs =

Reading csv file with double quote delimiter in SAP BODS

给你一囗甜甜゛ 提交于 2021-02-11 15:40:56
问题 I am trying to read a csv file in SAP BODS the file delimiter is ,(comma) and text delimiter is "(double quotes) the text withing the "" AND COMMAS also have double quotes in them due to which the column is being split into new column Is there any way i can only use header columns as a reference for the column definition? 来源: https://stackoverflow.com/questions/61456032/reading-csv-file-with-double-quote-delimiter-in-sap-bods