export-to-csv

Most efficient way of exporting large (3.9 mill obs) data.frames to text file? [duplicate]

由 喜欢而已 提交于 2019-11-28 16:20:42
问题 This question already has an answer here: Speeding up the performance of write.table 6 answers I have a fairly large dataframe in R that I would like to export to SPSS. This file has caused me hours of headaches trying to import it to R in the first place, however I got successful using read.fwf() using the options comment.char="%" (a character not appearing in the file) and fill= TRUE (it was a fixed-width ASCII file with some rows lacking all variables, causing error messages). Anyway, my

Save output from sql function to csv file (COPY) with dynamic filename

由 耗尽温柔 提交于 2019-11-28 11:46:04
问题 I am running Postgres 9.3 on MacOSX. I am trying to add a COPY statement inside a function for an automatized save-to-file process. I am new to any kind of sql coding, so this is what I have so far; CREATE FUNCTION retrieve_info(input_method TEXT, input_species TEXT) RETURNS SETOF retrieve_info_tbl AS $$ SELECT tblA.id, tblA.method, tblA.species, tblA.location FROM tblA WHERE method=input_method AND species=input_species GROUP BY id, method, species ORDER BY location COPY (SELECT * FROM

Write comments in CSV file with pandas

由 自作多情 提交于 2019-11-28 09:48:26
I would like to write some comments in my CSV file created with pandas . I haven't found any option for this in DataFrame.to_csv (even though read_csv can skip comments) neither in the standard csv module. I can open the file, write the comments (line starting with # ) and then pass it to to_csv . Does any body have a better option? df.to_csv accepts a file object. So you can open a file in a mode, write you comments and pass it to the dataframe to_csv function. For example: In [36]: df = pd.DataFrame({'a':[1,2,3], 'b':[1,2,3]}) In [37]: f = open('foo', 'a') In [38]: f.write('# My awesome

Export Pandas data frame with text column containg utf-8 text and URLs to Excel

由 廉价感情. 提交于 2019-11-28 08:29:25
问题 My Pandas data frame consists of Tweets and meta data of each tweet (300.000 rows). Some of my colleagues need to work with this data in Excel which is why I need to export it. I wanted to use either .to_csv or .to_excel which are both provided by Pandas but I can't get it to work properly. When I use .to_csv my problem is that it keeps failing in the text part of the data frame. I've played around with different separators but the file is never 100% aligned. The text column seems to contain

How to export sqlite to CSV in Python without being formatted as a list?

由 一曲冷凌霜 提交于 2019-11-28 07:31:30
Here is what I currently have: conn = sqlite3.connect(dbfile) conn.text_factory = str ## my current (failed) attempt to resolve this cur = conn.cursor() data = cur.execute("SELECT * FROM mytable") f = open('output.csv', 'w') print >> f, "Column1, Column2, Column3, Etc." for row in data: print >> f, row f.close() It creates a CSV file with output that looks like this: Column1, Column2, Column3, Etc. (1, u'2011-05-05 23:42:29',298776684,1448052234,463564768,-1130996322, None, u'2011-05-06 04:44:41') I don't want the rows to be in parentheses nor have quotes nor the 'u' before strings. How do I

Powershell - Export-CSV and Append

由 允我心安 提交于 2019-11-28 07:24:13
问题 I have a script such as the following: $in_file = "C:\Data\Need-Info.csv" $out_file = "C:\Data\Need-Info_Updated.csv" $list = Import-Csv $in_file ForEach ( $user in $list ) { $zID = $user.zID ForEach-Object { Get-QADUser -Service 'domain.local' -SearchRoot 'OU=Users,DC=domain,DC=local' -SizeLimit 75000 -LdapFilter "(&(objectCategory=person)(objectClass=user)(PersonzID=$zID))" | Select-Object DisplayName,samAccountName,@{Name="zID";expression={$zID}} | Export-Csv $out_file -NoTypeInformation

Calling a stored procedure within a stored procedure

由 半世苍凉 提交于 2019-11-28 05:40:05
问题 I am trying to call a function within a function using sql on postgres 9.3. This question is related to another post by me. I have written the below function. So far I have failed to incorporate any kind of save-output (COPY) statement, so I am trying to work around this by creating a nested function print-out function. CREATE FUNCTION retrieve_info(TEXT, TEXT) RETURNS SETOF retrieve_info_tbl AS $$ SELECT tblA.id, tblA.method, tblA.species, tblA.location FROM tblA WHERE method=$1 AND species=

Force SSRS 2008 to use SSRS 2005 CSV rendering

由 家住魔仙堡 提交于 2019-11-28 05:21:47
问题 We are upgrading our report server from SSRS 2005 to SSRS 2008 R2. I have an issue with CSV export rendering for SSRS 2008 where the SUM of columns are appearing on the right side of the detail values in 2008 instead of the left side like in 2005 as shown in the below blocks. 117 and 131 are the sums of Column2 and Column3 respectively. SSRS 2005 CSV Output Column2_1,Column3_1,Column2,Column3 117,131,1,2 117,131,1,2 117,131,60,23 117,131,30,15 117,131,25,89 SSRS 2008 CSV Output Column2

Generate CSV file on an external FTP server in PHP

由 99封情书 提交于 2019-11-28 05:17:42
问题 I have some PHP code that successfully exports a MySQL table to a CSV file. I would like to add to that code, so instead of saving locally the CSL file is exported to/saved on an external FTP server. My current code: //open database connection require ('../database-config.php'); //name the file header('Content-Type: text/csv'); header('Content-Disposition: attachment;filename=exported-data.csv'); //SQL Query for Data $sql = "SELECT * FROM data;"; //Prepare Query, Bind Parameters, Excute Query

Saving to CSV in Excel loses regional date format

由 送分小仙女□ 提交于 2019-11-28 04:14:52
问题 I have a .xls I need to convert to .csv The file contains some date columns. The format on the date is "*14/03/2001" which, according to Excel means the date responds to regional date and time settings specified for the OS. Opening in Excel you see: 20/01/2013 01/05/2013 Save as... CSV Open in notepad: 01/20/2013 05/01/2013 I have temporarily fixed by setting date formats to "14/03/2001" (no *) but even some other custom formats with no *, like "d/mm/yyyy h:mm" get mangled when saved to CSV.
订阅 export-to-csv