csv

pyparsing parsing csv files with semi-colon instead of comma

♀尐吖头ヾ 提交于 2021-02-19 06:20:11
问题 In mainland europe, the csv files are separated through semicolons because numbers have , in them instead of . So, i am trying to write a semicolonSeparatedList same as commaSeparatedList but with ; instead of ,: _semicolonsepitem = Combine(OneOrMore(Word(printables, excludeChars=';') + Optional( Word(" \t") + ~Literal(";") + ~LineEnd() ) ) ).streamline().setName("semicolonItem") semicolonSeparatedList = delimitedList( Optional( quotedString.copy() | _semicolonsepitem, default="") ).setName(

Convert JSON to CSV with jq

懵懂的女人 提交于 2021-02-19 05:50:08
问题 I'm trying to extract the sids, ll, state, name, smry values in my JSON file using jq and export to a csv. JSON File (out.json): { "data": [ { "meta": { "uid": 74529, "ll": [ -66.9333, 47.0667 ], "sids": [ "CA008102500 6" ], "state": "NB", "elev": 1250, "name": "LONG LAKE" }, "smry": [ [ "42", "1955-02-23" ] ] }, { "meta": { "uid": 74534, "ll": [ -67.2333, 45.9667 ], "sids": [ "CA008103425 6" ], "state": "NB", "elev": 150.9, "name": "NACKAWIC" }, "smry": [ [ "40", "1969-02-23" ] ] }, { "meta"

Handling error “TypeError: Expected tuple, got str” loading a CSV to pandas multilevel and multiindex (pandas)

生来就可爱ヽ(ⅴ<●) 提交于 2021-02-19 05:44:09
问题 I'm trying to load a CSV file (this file) to create a multiindex e multilevel dataframe. It has 5(five) indexes and 3(three) levels in columns. How I can do? Here is the code: df = pd.read_csv('./teste.csv' ,index_col=[0,1,2,3,4] ,header=[0,1,2,3] ,skipinitialspace=True ,tupleize_cols=True) df.columns = pd.MultiIndex.from_tuples(df.columns) Expected output: variables u \ level 1 days 1 2 times 00h 06h 12h 18h 00h wsid lat lon start prcp_24 329 -43.969397 -19.883945 2007-03-18 10:00:00 72.0 0

How do I prevent pyspark from interpreting commas as a delimiter in a csv field having JSON object as its value

↘锁芯ラ 提交于 2021-02-19 05:31:36
问题 I am trying to read a comma delimited csv file using pyspark version 2.4.5 and Databrick's spark-csv module. One of the field in the csv file has a json object as its value. The contents of the csv are as below test.csv header_col_1, header_col_2, header_col_3 one, two, three one, {“key1”:“value1",“key2”:“value2",“key3”:“value3”,“key4”:“value4"}, three Other solutions that I found had read options defined as "escape": '"' , and 'delimiter': "," . This seems not to be working as the commas in

Bash processing of csv file with unknown number of columns

只愿长相守 提交于 2021-02-19 05:22:04
问题 I am trying to learn some text processing using bash. How to make a bash script to read and process CSV file with unknown number of columns with first row as column headers? Example input: column1,column2,...,columnn value11,value12,...,value1n value21,value22,...,value2n ... valuem1,valuem2,...,valuemn output: column1: value11 column2: value12 ... columnn: value1n column1: value21 column2: value22 ... columnn: value2n ... column1: valuem1 column2: valuem2 ... columnn: valuemn 回答1: One simple

how to remove quotes from csv in php

那年仲夏 提交于 2021-02-19 05:21:20
问题 I have a array that i am getting from DB. In this project, im converting my array to csv file. But every time i open the file i get double quoetes. I have tried with str_replace and preg_place with no succes. How can i remove quotes this is my csv code $query = "SELECT t.transactiontime, t.restaurant_id, t.transactionid, t.cardid, emd.m_field_id_2, t.pricebefordiscount, t.menucard_cut from transactions as t left join exp_member_data AS emd ON (t.cardid-10000000 = emd.member_id) order by t

plotting polynomial regression in same plot as the real data

梦想的初衷 提交于 2021-02-19 04:24:15
问题 I have some snippets of code that read two csvs and plot them using matplotlib.pyplot and perform polynomial regression on the same two csvs. What I want to be able to do is plot both the data and my polynomial regression on the same graph. import matplotlib.pyplot as plt import csv import numpy as np datax=np.genfromtxt('Delta R.csv') datay=np.genfromtxt('Example R.csv') plt.title ('Test graph ') plt.xlabel('x axis') plt.ylabel('y axis ') plt.plot(datax, datay,'o-') plt.show() and my second

Pandas how to read sub headers

随声附和 提交于 2021-02-18 13:52:56
问题 I'm using python+pandas to process a csv file. The csv file has multiple headers, like Header1 Header2 Date Subheader1-1 Subheader1-2 Subheader2-1 Subheader2-2 And in raw text format, the csv file content looks like ,Header1,,Header2,,... Date,Subheader1-1,Subheader1-2,Subheader2-1,Subheader2-2,... ... My question is, Does Pandas support this sub-header format? If not, is there a way to read this csv into pandas dataframe and do some calculation on it? (The calculation is like extracting

Difference between using and not using pipe in Export-Csv in Powershell

大城市里の小女人 提交于 2021-02-18 13:52:11
问题 This is probably more of a 'how does PowerShell handle variables and piping' rather than a specific programmatical question, but since it seems like strange behaviour (to me) I thought I'd post it here. I just had some difficulties exporting a variable to a CSV using PowerShell and found this Stack question that helped me a lot. However, when fiddling around with the output I got two different results depending on how I called the Export-CSV function. I have a custom PS object that looks

Difference between using and not using pipe in Export-Csv in Powershell

独自空忆成欢 提交于 2021-02-18 13:51:42
问题 This is probably more of a 'how does PowerShell handle variables and piping' rather than a specific programmatical question, but since it seems like strange behaviour (to me) I thought I'd post it here. I just had some difficulties exporting a variable to a CSV using PowerShell and found this Stack question that helped me a lot. However, when fiddling around with the output I got two different results depending on how I called the Export-CSV function. I have a custom PS object that looks