csv

Merge CSVs in Python with different columns

佐手、 提交于 2020-05-09 22:57:04
问题 I have hundreds of large CSV files that I would like to merge into one. However, not all CSV files contain all columns. Therefore, I need to merge files based on column name, not column position. Just to be clear: in the merged CSV, values should be empty for a cell coming from a line which did not have the column of that cell. I cannot use the pandas module, because it makes me run out of memory. Is there a module that can do that, or some easy code? 回答1: The csv.DictReader and csv

Merge CSVs in Python with different columns

十年热恋 提交于 2020-05-09 22:52:37
问题 I have hundreds of large CSV files that I would like to merge into one. However, not all CSV files contain all columns. Therefore, I need to merge files based on column name, not column position. Just to be clear: in the merged CSV, values should be empty for a cell coming from a line which did not have the column of that cell. I cannot use the pandas module, because it makes me run out of memory. Is there a module that can do that, or some easy code? 回答1: The csv.DictReader and csv

How to split the content of one column into two columns of csv in Ruby?

非 Y 不嫁゛ 提交于 2020-05-09 17:22:47
问题 I got an object of CSV, the content is like: full_name | phone_number | Mike Smith | (123)-456-7890| Tony Davis | (213)-564-7890| And I would like to split the column of 'full_name' into two columns, which represents 'first_name' and 'last_name': first_name | last_name | phone_number | Mike | Smith | (123)-456-7890| Tony | Davis | (213)-564-7890| I didn't find a command to do it, how can I split it? Thank you very much! The code for this part is: def to_csv(options = {}) CSV.generate(options)

Defining a new variable in order to make a huge iteration giving me an error

谁都会走 提交于 2020-05-09 07:05:08
问题 I have an endpoint, you can have información about products {{URL_API}}/products/ If i perform a GET method over that endpoint i will obtain the information of every product BUT i can also specify the product that i want to know about, i.e: {{URL_API}}/products/9345TERFER (the last code is the id of the product, called SKU) The problem is that if i want to make a CSV in order to update the information of different products i have to define a variable called sku in the endpoint so i will be

Sending a request without specifying the fields in the body

丶灬走出姿态 提交于 2020-05-09 06:29:29
问题 I'm trying to send several PUT request to a specific endpoint. I am using a CSV file within the columns and values, the name of the different columns reference the different variables inside the body, do you get me? This is the endpoint: {{URL_API}}/products/{{sku}} --sku is the id of the product, i created that variable because i use it to pass the reference This is the body that i use: { "price":"{{price}}", "tax_percentage":"{{tax_percentage}}", "store_code":"{{store_code}}", "markup_top":

Sending a request without specifying the fields in the body

廉价感情. 提交于 2020-05-09 06:29:09
问题 I'm trying to send several PUT request to a specific endpoint. I am using a CSV file within the columns and values, the name of the different columns reference the different variables inside the body, do you get me? This is the endpoint: {{URL_API}}/products/{{sku}} --sku is the id of the product, i created that variable because i use it to pass the reference This is the body that i use: { "price":"{{price}}", "tax_percentage":"{{tax_percentage}}", "store_code":"{{store_code}}", "markup_top":

Memory error using openpyxl and large data excels

邮差的信 提交于 2020-05-09 02:02:07
问题 I have written a script which has to read lot of excel files from a folder (around 10,000). This script loads the excel file (some of them has more than 2,000 rows) and reads one column to count the number of rows (checking stuff). If the number of rows is not equal to a given number, it writes the warning in a log. The problem comes when the script reads more than 1,000 excel files. It's then when it throws memory error, and I don't know where could be the problem. Previously, the script

How to convert json into csv file using jq?

孤街醉人 提交于 2020-05-08 15:49:25
问题 This is my json file: { "ClientCountry": "ca", "ClientASN": 812, "CacheResponseStatus": 404, "CacheResponseBytes": 130756, "CacheCacheStatus": "hit" } { "ClientCountry": "ua", "ClientASN": 206996, "CacheResponseStatus": 301, "CacheResponseBytes": 142, "CacheCacheStatus": "unknown" } { "ClientCountry": "ua", "ClientASN": 206996, "CacheResponseStatus": 0, "CacheResponseBytes": 0, "CacheCacheStatus": "unknown" } I want to convert these json into csv like below. "ClientCountry", "ClientASN",

How to convert json into csv file using jq?

て烟熏妆下的殇ゞ 提交于 2020-05-08 15:47:38
问题 This is my json file: { "ClientCountry": "ca", "ClientASN": 812, "CacheResponseStatus": 404, "CacheResponseBytes": 130756, "CacheCacheStatus": "hit" } { "ClientCountry": "ua", "ClientASN": 206996, "CacheResponseStatus": 301, "CacheResponseBytes": 142, "CacheCacheStatus": "unknown" } { "ClientCountry": "ua", "ClientASN": 206996, "CacheResponseStatus": 0, "CacheResponseBytes": 0, "CacheCacheStatus": "unknown" } I want to convert these json into csv like below. "ClientCountry", "ClientASN",

How to convert json into csv file using jq?

浪子不回头ぞ 提交于 2020-05-08 15:46:25
问题 This is my json file: { "ClientCountry": "ca", "ClientASN": 812, "CacheResponseStatus": 404, "CacheResponseBytes": 130756, "CacheCacheStatus": "hit" } { "ClientCountry": "ua", "ClientASN": 206996, "CacheResponseStatus": 301, "CacheResponseBytes": 142, "CacheCacheStatus": "unknown" } { "ClientCountry": "ua", "ClientASN": 206996, "CacheResponseStatus": 0, "CacheResponseBytes": 0, "CacheCacheStatus": "unknown" } I want to convert these json into csv like below. "ClientCountry", "ClientASN",