csv

R: How to locate and compare a particular element between two CSV text files

那年仲夏 提交于 2021-01-28 21:53:03
问题 I found some similar questions such as this one (about comparing attributes in XML files), this one (about a case where the compared values are numeric) and this one (about getting a number of columns that differ between two files) but nothing about this particular problem. I have two CSV text files on which many, but not all, rows are equal. The files have the same amount of columns with same data type on the columns but they do not have the same amount of rows. The amount of rows on both

Fast way to download a really big (14 million row) csv from a zip file? Unzip and read_csv and read.csv never stop loading

给你一囗甜甜゛ 提交于 2021-01-28 21:14:19
问题 I am trying to download the dataset at the below link. It is about 14,000,000 rows long. I ran this code chunk, and I am stuck at unzip(). The code has been running for a really long time and my computer is hot. I tried a few different ways that don't use unzip, and then I get stuck at the read.csv/vroom/read_csv step. Any ideas? This is a public dataset so anyone can try. library(vroom) temp <- tempfile() download.file("https://files.consumerfinance.gov/hmda-historic-loan-data/hmda_2017

Powershell - find file by 'file name' and rename based on CSV

限于喜欢 提交于 2021-01-28 21:10:58
问题 I have a set of files (OldName) in a Windows directory that I would like to rename (NewName) based on the following CSV file: OldName,NewName Sources/texas play_PGC_Cpgc_entryPoint_Cbp_1.f4v,01 Texas Play.f4v Sources/texas play_PGC_Cpgc_entryPoint_Dbp_1.f4v,02 First Song.f4v Sources/texas play_PGC_Cpgc_entryPoint_Ebp_1.f4v,03 Yellow Rose.f4v I'm not sure how to loop thru the CSV file... finding each file and replacing. Any thoughts would be appreciated. 回答1: First Import Your CSV file into

TypeError: list indices must be integers or slices, not list"

时光毁灭记忆、已成空白 提交于 2021-01-28 19:42:12
问题 I am trying the following code but I have the problem with the indices and having the following error if int( NewList[i][1] ) <120: TypeError: list indices must be integers or slices, not list In this case, the script should delete the the first 2 lists (['0', '30', '100'], ['20', '100', '54']) and how can I replace NewList[i][1] when the conditions don't meet for the first time? In this case, I want to replace NewList[2][1] = 150 by 120. Here is my code: def newdef (): NewList = [['0', '30',

How to remove NUL strings at begining of CSV; VBA

我是研究僧i 提交于 2021-01-28 18:53:42
问题 I have a program that exports a bill of material (see CSV example data lower in post). I wrote a VBA macro that uses FileSystemObject to loop through a user-selected folder, open the CSV files for reading and read the data into memory using ReadAll, then store and sort the data in excel. The problem I have is after exporting the CSV files, I run the macro and the text in the CSV file is stored in my string variable (sTemp in the code below) as several square braces. I copied the value of the

loop on user.csv and then loop on data.csv in jmeter

半城伤御伤魂 提交于 2021-01-28 18:16:14
问题 I need to do something like this: foreach user in a users.csv: foreach data in data.csv: send http request I've took a look on this post and on this page However, as far I've been to figure out, it's NOT performed in parallel. So, it's not posible to simulate for example that during 10 minutes the users are loading data progressively. So, I'd like each user is a thread and then each user perform an HTTP request as many data is in data.csv file. By this way, I figure out it's easiest to

Parse csv to decimal on different cultures using CsvHelper in c#

时光毁灭记忆、已成空白 提交于 2021-01-28 14:23:42
问题 Problem with parsing decimals by CsvHelper in c#. I created a class that get csv file from byte[] istead of file, and it works correctly. public static List<Topic> GetAll(byte[] attachmentBytes) { using (Stream stream = new MemoryStream(attachmentBytes)) { using (var reader = new StreamReader(stream, Encoding.GetEncoding(1252), true)) { using (var csv = new CsvReader(reader)) { csv.Configuration.Delimiter = ";"; csv.Configuration.HasHeaderRecord = true; csv.Configuration.CultureInfo =

Python psql \copy CSV to remote server

人走茶凉 提交于 2021-01-28 13:02:11
问题 I am attempting to copy a csv (which has a header and quote character ") with python 3.6 to a table on a remote postgres 10 server. It is a large CSV (2.5M rows, 800MB) and while I previously imported it into a dataframe and then used dataframe.to_sql, this was very memory intensive so I switched to using COPY. Using COPY with psycopg2 or sqlalchemy would work fine but the remote server does not have access to the local file system. Using psql in the terminal I have successfully run the query

Python psql \copy CSV to remote server

左心房为你撑大大i 提交于 2021-01-28 13:01:37
问题 I am attempting to copy a csv (which has a header and quote character ") with python 3.6 to a table on a remote postgres 10 server. It is a large CSV (2.5M rows, 800MB) and while I previously imported it into a dataframe and then used dataframe.to_sql, this was very memory intensive so I switched to using COPY. Using COPY with psycopg2 or sqlalchemy would work fine but the remote server does not have access to the local file system. Using psql in the terminal I have successfully run the query

Python pandas - merge csv files in directory into one

流过昼夜 提交于 2021-01-28 12:24:18
问题 I have a directory with csv files: frames/df1.csv df2.csv frames are structured like this: df1.csv artist track plays 1 Pearl Jam Jeremy 456 2 The Rolling Stones Heart of Stone 546 df2.csv artist track likes 3 Pearl Jam Jeremy 5673 9 The Rolling Stones Heart of Stone 3456 and I would like to merge all frames into one, ending up with: artist track plays likes 0 Pearl Jam Jeremy 456 5673 1 The Rolling Stones Heart of Stone 546 3456 I've tried: path = 'frames' all_files = glob.glob(path + "/*