csv

Read csv with pandas with commented header

可紊 提交于 2021-02-07 13:33:23
问题 I have CSV files that have # in the header line: s = '#one two three\n1 2 3' If I use pd.read_csv the # sign gets into the first header: import pandas as pd from io import StringIO pd.read_csv(StringIO(s), delim_whitespace=True) #one two three 0 1 2 3 If I set the argument comment='#' , then pandas ignores the line completely. Is there an easy way to handle this case? Second issue, related, is how can I handle quoting in this case, it works with no # : s = '"one one" two three\n1 2 3' print

Read csv with pandas with commented header

限于喜欢 提交于 2021-02-07 13:32:34
问题 I have CSV files that have # in the header line: s = '#one two three\n1 2 3' If I use pd.read_csv the # sign gets into the first header: import pandas as pd from io import StringIO pd.read_csv(StringIO(s), delim_whitespace=True) #one two three 0 1 2 3 If I set the argument comment='#' , then pandas ignores the line completely. Is there an easy way to handle this case? Second issue, related, is how can I handle quoting in this case, it works with no # : s = '"one one" two three\n1 2 3' print

How to plot 3D matrix CSV data in Gnuplot with splot using the first row and column as the x y coordinates?

给你一囗甜甜゛ 提交于 2021-02-07 13:21:08
问题 How can I plot (a 3D plot) a matrix in Gnuplot having such data structure. I cannot find a way to use the first row and column as a x and y ticks (or to ignore them) ,5,6,7,8 1,-6.20,-6.35,-6.59,-6.02 2,-6.39,-6.52,-6.31,-6.00 3,-6.36,-6.48,-6.15,-5.90 4,-5.79,-5.91,-5.87,-5.46 Is the splot 'data.csv' matrix the correct parameter to use ? 回答1: You can give using a format specification; here we need to tell the seperator ','. The following works for me: splot 'data.csv' using 1:2:3 '%lf,%lf,

Set content type in S3 when attaching via Paperclip 4?

旧街凉风 提交于 2021-02-07 11:59:15
问题 I'm trying to attach CSV files to a Rails3 model using paperclip 4.1.1, but I'm having trouble getting the content-type as reported by S3 to be text/csv (instead I am getting text/plain ). When I subsequently download the file from S3, the extension is getting changed to match the content-type instead of preserving the original extension (so test.csv is downloaded as test.txt). From what I can see, when you upload a file, the FileAdapter will cache the content-type on creation with whatever

Export to CSV file and open in browser

流过昼夜 提交于 2021-02-07 11:56:17
问题 I am stuck with an issue where I need to export data to a .csv file, but not store the file in file system - instead I need to simply open the file in browser. I have written the below code to write data to .csv file: FileWriter myWriter = new FileWriter("output.csv"); myWriter.append(EmployeeCode); myWriter.append(','); myWriter.append(Band); myWriter.append('\n'); response.setHeader("Content-Disposition", "attachment; filename=output.csv"); response.setContentType("application/ms-excel");

“DataFrame” object has no attribute 'reshape'

依然范特西╮ 提交于 2021-02-07 11:54:32
问题 I want to reshape some data in a CSV file without header but I keep getting this error AttributeError: 'DataFrame' object has no attribute 'reshape' This is my script, I want to reshape the data in 2nd column only import pandas as pd df = pd.read_csv("test.csv", header=None, usecols=[1]) start = 0 for i in range(0, len(df.index)): if (i + 1)%10 == 0: result = df.iloc[start:i+1].reshape(2,5) start = i + 1 print result Here is the CSV 1,52.1 2,32.2 3,44.6 3,99.1 5,12.3 3,43.2 7,79.4 8,45.5 9,56

“DataFrame” object has no attribute 'reshape'

自闭症网瘾萝莉.ら 提交于 2021-02-07 11:54:11
问题 I want to reshape some data in a CSV file without header but I keep getting this error AttributeError: 'DataFrame' object has no attribute 'reshape' This is my script, I want to reshape the data in 2nd column only import pandas as pd df = pd.read_csv("test.csv", header=None, usecols=[1]) start = 0 for i in range(0, len(df.index)): if (i + 1)%10 == 0: result = df.iloc[start:i+1].reshape(2,5) start = i + 1 print result Here is the CSV 1,52.1 2,32.2 3,44.6 3,99.1 5,12.3 3,43.2 7,79.4 8,45.5 9,56

Is it possible to handle fields containing line breaks in awk?

岁酱吖の 提交于 2021-02-07 10:39:35
问题 Suppose I have a text file with records of the following form, where the FS is generally speaking a comma, and the RS is generally speaking a newline. However, the exception to this rule is that if a field is in quotes, it should treat the line breaks and commas as part of the field. "This field contains line breaks and is quoted but it should be treated as a single field",1,2,3,"another field" How can I use awk to parse such a file correctly, where I can still access $1,$2... , as I usually

Is it possible to handle fields containing line breaks in awk?

流过昼夜 提交于 2021-02-07 10:39:21
问题 Suppose I have a text file with records of the following form, where the FS is generally speaking a comma, and the RS is generally speaking a newline. However, the exception to this rule is that if a field is in quotes, it should treat the line breaks and commas as part of the field. "This field contains line breaks and is quoted but it should be treated as a single field",1,2,3,"another field" How can I use awk to parse such a file correctly, where I can still access $1,$2... , as I usually

Python - Extract CSV Files from Multiple Zip Files and Combine the Data

你说的曾经没有我的故事 提交于 2021-02-07 10:20:32
问题 I have a Python script and pandas to combine multiple ZIP files. I am using data hosted in a GitHub repository here: https://github.com/statistikat/coronaDAT The script should take all ZIP files in a folder structure, find the "Bezirke.csv" file in the ZIP file, and combine all the Bezirke.csv files into one large CSV file. However, the code is only grabbing one ZIP file from the folder. Any suggestions on why the it is not taking the data from the other ZIP files in the folder? import glob