csv

How to use while read line with tail -n

由 拥有回忆 提交于 2020-04-11 07:14:08
问题 Problem: I have a CSV dump file - with excess of 250,000 lines. When I use while read - it takes a while (no pun intended). I would like to go back to the last 10,000 lines to do what I need to do instead of the 250,000 lines. Code Snippet: My current code is this: IFS="," while read line do awk_var=`echo "$line" | awk -F" " '{print $0}'` var_array=($awk_var) read -a var_array <<< "${awk_var}" echo "${var_array[1]}" done </some_directory/directory/file_in_question.csv Question: How can I use

Python: TypeError: unhashable type: 'slice'

由 你离开我真会死。 提交于 2020-04-11 06:28:07
问题 I am reading CSV using python, here is the code. train_csv = open('train.csv') test_csv = open('test.csv') train_data_reader = csv.DictReader(train_csv) test_data_reader = csv.DictReader(test_csv) row=[] for row in train_data_reader: X.append([int(item) for item in row[4:]]) char = row[1] Y.append(charIntConversion(char)) train_id.append(row[0]) prediction.append(row[1]) for row in test_data_reader: test_id.append(row[0]) test_X.append([int(item) for item in row[4:]] when I tried to run the

How can I read multiple csvs and retain the number in the file name for each?

由 心已入冬 提交于 2020-04-10 06:00:12
问题 I have multiple csv files in a folder none of which have a header. I want to preserve the order set out by the number at the end of the file. The file names are "output-1.csv", "output-2.csv" and so on. Is there a way to include the file name of each csv so I know which data corresponds to which file. The answer [here][1] gets close to what I want. library(tidyverse) #' Load the data ---- mydata <- list.files(path = "C:\\Users\\Documents\\Manuscripts\\experiment1\\output", pattern = "*.csv")

Removing rows when reading data D3

由 ∥☆過路亽．° 提交于 2020-04-10 04:10:11
问题 Say I have a sample file sample.csv: row,col,value 1,1,2 1,2,3 1,3,NA When reading data in d3 you do something like: d3.csv("sample.csv", function(data) { data.forEach(function(d) { d.value = +d.value; }); However, for the NA value +d.value will return NaN. How can I exclude NaN values from my data. i.e. read the data, and only take rows which have a number value Thanks! 回答1: You can call isNaN on the data before you try to add it: d3.csv('sample.csv', function(data) { data = data.filter

Removing rows when reading data D3

由 ≡放荡痞女 提交于 2020-04-10 04:09:18
问题 Say I have a sample file sample.csv: row,col,value 1,1,2 1,2,3 1,3,NA When reading data in d3 you do something like: d3.csv("sample.csv", function(data) { data.forEach(function(d) { d.value = +d.value; }); However, for the NA value +d.value will return NaN. How can I exclude NaN values from my data. i.e. read the data, and only take rows which have a number value Thanks! 回答1: You can call isNaN on the data before you try to add it: d3.csv('sample.csv', function(data) { data = data.filter

python3.x 读取csv遇到的bug

由 亡梦爱人 提交于 2020-04-08 12:20:47
1、failed to set main . loader 兴奋地配置好了Python环境，运行hello.py实例就出现这个异常，着实让人扫兴，百度上搜了下没有找到答案。再去Google了下，发现可能是hello.py文件中包含非英文字符，果然将hello.py放到纯英文路径下就没问题了。 对于eclipse下使用PyDev的情况，可以用File->Switch Workspace的方法来切换到一个英文路径工作空间目录 2、_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?) 在用下面的代码处理csv文件时出现这个错误（Python 3) 复制代码 import csv def main(): reader=csv.reader(open('userid.csv', 'rb')) for item in reader: print(item) if name == ' main ': main() 复制代码 经过万能的Google搜索，找到了问题所在：http://bugs.python.org/msg82661 ，下面是部分摘录： 复制代码 Sorry, folks, we've got an understanding problem here.

Export result set on Dbeaver to CSV

由 时光怂恿深爱的人放手 提交于 2020-04-08 08:33:29
问题 Normally I use Dbeaver for windows and always export my result set like this: Run my query --> select the result --> export the result set --> select export to clipboard --> done This step by step puts my result set in my clipboard and I can paste it wherever I want to work with it. The problem is that now I am using dbeaver for mac and this guide is not working. I can go on until the moment that I select my result set like in the image below: But once I go further in the process, in the last

solr curl索引 CSV/Json/xml文件

由 老子叫甜甜 提交于 2020-04-08 08:27:06
　　在windows系统中，用curl命令工具索引文件命令： 　　启动solr 　　在solr-6.6.0\bin的同级目录下的文件夹ImportData下要索引的文件. 　　1、索引 json文件 　　　　curl "http://localhost:8983/solr/mycore/update?commit=true" --data-binary @../ImportData/books.json -H "Content-type:application/json" 　　　　 　　2、索引 csv文件 　　　　curl "http://localhost:8983/solr/mycore/update?commit=true" --data-binary @../ImportData/2017-07-07_info.csv -H "Content-type:application/csv" 　　　　 　　3、索引 xml文件 　　　　curl "http://localhost:8983/solr/mycore/update?commit=true" --data-binary @../ImportData/hd.xml -H "Content-type:application/xml" 　　　　 　　用curl貌似不能索引pdf文件 　　　　curl "http:/

What's the fastest way to merge multiple csv files by column?

由 守給你的承諾、 提交于 2020-04-08 06:41:03
问题 I have about 50 CSV files with 60,000 rows in each, and a varying number of columns. I want to merge all the CSV files by column. I've tried doing this in MATLAB by transposing each csv file and re-saving to disk, and then using the command line to concatenate them. This took my computer over a week and the final result needs to transposed once again! I have to do this again, and I'm looking for a solution that won't take another week. Any help would be appreciated. 回答1: [...] transposing

What's the fastest way to merge multiple csv files by column?

由 孤者浪人 提交于 2020-04-08 06:39:03
问题 I have about 50 CSV files with 60,000 rows in each, and a varying number of columns. I want to merge all the CSV files by column. I've tried doing this in MATLAB by transposing each csv file and re-saving to disk, and then using the command line to concatenate them. This took my computer over a week and the final result needs to transposed once again! I have to do this again, and I'm looking for a solution that won't take another week. Any help would be appreciated. 回答1: [...] transposing
订阅 csv