csv

Django: generate a CSV file and store it into FileField

风流意气都作罢 提交于 2020-06-16 02:24:31
问题 In my Django View file, I need to generate a CSV file from a list of tuples and store the CSV file into the FileField of my model. class Bill(models.Model): billId = models.IntegerField() bill = models.FileField(upload_to='bills') I searched on this site, and found some posts such as Django - how to create a file and save it to a model's FileField?, but the solutions can help me. I need the CSV file to be stored in 'media/bills/' fold, and I hope the CSV file can be deleted together with the

Reading OneDrive files to R

大憨熊 提交于 2020-06-14 06:06:14
问题 When I read in csv files from Dropbox into R , I right-click the file and click share Dropbox link . I then have a URL something like: https://www.dropbox.com/blahhhhhhhhhh.csv?dl=0 So I change it to: read.csv("http://dl.dropbox.com/blahhhhhhhhhh.csv?dl=0", ...) and it works without the need to use any packages etc. Is there a way to read files from OneDrive in a similar manner? https://onedrive.live.com/blahhhhhhhhhhhhhhhhccsv As when I try to read it into R it doesn't give me the data frame

Python email MIME attachment filename

六眼飞鱼酱① 提交于 2020-06-14 05:18:33
问题 I'm having trouble attaching a CSV file to an email. I can send the email fine using smtplib, and I can attach my CSV file to the email. But I cannot set the name of the attached file, and so I cannot set it to be .csv . Also I can't figure out how to add a text message to the body of the email. This code results in an attachment called AfileName.dat , not the desired testname.csv , or better still attach.csv #!/usr/bin/env python import smtplib from email.mime.multipart import MIMEMultipart

Skip 6 rows before “reading” into powerquery

本小妞迷上赌 提交于 2020-06-13 07:17:33
问题 I am trying to automate a few reports that are built off of CSV exports from Netsuite (our ERP software). The files never import directly into PowerQuery correctly because there are 6 rows that are "header" rows. These header rows do not have the correct amount of commas so PowerQuery only shows 1 column of data. I currently am opening the files with Notepad++ and deleting those 6 rows and then importing the file into PowerQuery. Is there a way to skip the first 6 rows using PowerQuery code

Mongoexport to multiple csv files

佐手、 提交于 2020-06-12 09:12:00
问题 I have a large mongoDB collection. I want to export this collection to CSV so I can then import it in a statistics package to do data analysis. The collection has about 15 GB of documents in it. I would like to split the collection into ~100 equally sized CSV files. Is there any way to achieve this using mongoexport? I could also query the whole collection in pymongo, split it and write to csv files manually, but I guess this would be slower and would require more coding. Thank you for input.

saving csv file to s3 using boto3

依然范特西╮ 提交于 2020-06-12 08:04:52
问题 I am trying to write and save a CSV file to a specific folder in s3 (exist). this is my code: from io import BytesIO import pandas as pd import boto3 s3 = boto3.resource('s3') d = {'col1': [1, 2], 'col2': [3, 4]} df = pd.DataFrame(data=d) csv_buffer = BytesIO() bucket = 'bucketName/folder/' filename = "test3.csv" df.to_csv(csv_buffer) content = csv_buffer.getvalue() def to_s3(bucket,filename,content): s3.Object(bucket,filename).put(Body=content) to_s3(bucket,filename,content) this is the

saving csv file to s3 using boto3

我的梦境 提交于 2020-06-12 08:04:10
问题 I am trying to write and save a CSV file to a specific folder in s3 (exist). this is my code: from io import BytesIO import pandas as pd import boto3 s3 = boto3.resource('s3') d = {'col1': [1, 2], 'col2': [3, 4]} df = pd.DataFrame(data=d) csv_buffer = BytesIO() bucket = 'bucketName/folder/' filename = "test3.csv" df.to_csv(csv_buffer) content = csv_buffer.getvalue() def to_s3(bucket,filename,content): s3.Object(bucket,filename).put(Body=content) to_s3(bucket,filename,content) this is the

pyspark type error on reading a pandas dataframe

可紊 提交于 2020-06-12 07:15:24
问题 I read some CSV file into pandas, nicely preprocessed it and set dtypes to desired values of float, int, category. However, when trying to import it into spark I get the following error: Can not merge type <class 'pyspark.sql.types.DoubleType'> and <class 'pyspark.sql.types.StringType'> After trying to trace it for a while I some source for my troubles -> see the CSV file: "myColumns" "" "A" Red into pandas like: small = pd.read_csv(os.path.expanduser('myCsv.csv')) And failing to import it to

pyspark type error on reading a pandas dataframe

依然范特西╮ 提交于 2020-06-12 07:13:13
问题 I read some CSV file into pandas, nicely preprocessed it and set dtypes to desired values of float, int, category. However, when trying to import it into spark I get the following error: Can not merge type <class 'pyspark.sql.types.DoubleType'> and <class 'pyspark.sql.types.StringType'> After trying to trace it for a while I some source for my troubles -> see the CSV file: "myColumns" "" "A" Red into pandas like: small = pd.read_csv(os.path.expanduser('myCsv.csv')) And failing to import it to

Apache commons csv skip lines

耗尽温柔 提交于 2020-06-12 02:38:45
问题 How to skip lines in input file with apache commons csv. In my file first few lines are garbage useful meta-information like date, etc. Can't find any options for this. private void parse() throws Exception { Iterable<CSVRecord> records = CSVFormat.EXCEL .withQuote('"').withDelimiter(';').parse(new FileReader("example.csv")); for (CSVRecord csvRecord : records) { //do something } } 回答1: Use FileReader.readLine() before starting the for-loop . Your example: private void parse() throws