dataframe

问题 I have a dataset showing the exchange rate of the Australian Dollar versus the US dollar once a day over a period of about 20 years. I have the data in a data frame, with the first column being the date, and the second column being the exchange rate. Here's a sample from the data: >data V1 V2 1 12/12/1983 0.9175 2 13/12/1983 0.9010 3 14/12/1983 0.9000 4 15/12/1983 0.8978 5 16/12/1983 0.8928 6 19/12/1983 0.8770 7 20/12/1983 0.8795 8 21/12/1983 0.8905 9 22/12/1983 0.9005 10 23/12/1983 0.9005

问题 I want to select a column that equals to a certain value. I am doing this in scala and having a little trouble. Heres my code df.select(df("state")==="TX").show() this returns the state column with boolean values instead of just TX Ive also tried df.select(df("state")=="TX").show() but this doesn't work either. 回答1: I had the same issue, and the following syntax worked for me: df.filter(df("state")==="TX").show() I'm using Spark 1.6. 回答2: There is another simple sql like option. With Spark 1

问题 I want to select a column that equals to a certain value. I am doing this in scala and having a little trouble. Heres my code df.select(df("state")==="TX").show() this returns the state column with boolean values instead of just TX Ive also tried df.select(df("state")=="TX").show() but this doesn't work either. 回答1: I had the same issue, and the following syntax worked for me: df.filter(df("state")==="TX").show() I'm using Spark 1.6. 回答2: There is another simple sql like option. With Spark 1

问题 I want to select a column that equals to a certain value. I am doing this in scala and having a little trouble. Heres my code df.select(df("state")==="TX").show() this returns the state column with boolean values instead of just TX Ive also tried df.select(df("state")=="TX").show() but this doesn't work either. 回答1: I had the same issue, and the following syntax worked for me: df.filter(df("state")==="TX").show() I'm using Spark 1.6. 回答2: There is another simple sql like option. With Spark 1

问题 I would like to replace all values in a dataframe lower than a given threshold minval with NA . What would be the most elegant way to do this? 回答1: Try this: df[df<minval]=NA df < minval creates a boolean matrix, which is used to select the values you want to replace with NA . 来源： https://stackoverflow.com/questions/26468385/r-replace-all-values-in-a-dataframe-lower-than-a-threshold-with-na

问题 I'm not really sure how to ask this, so I apologize if this is a repeat question. I have this data frame that looks something like this: | ID | Attend_x | Attend_y | Attend_z | | 1 | No | No | No | | 2 | No | No | Yes | | 3 | No | Yes | No | | 4 | No | Yes | Yes | I've been trying to figure out the right combination of group_by and count to get it to look like this: | | Yes | No | |Attend_x| 0 | 4 | |Attend_y| 2 | 2 | |Attend_z| 2 | 2 | I'm honestly stumped. So any advice is super appreciated

问题 i'm lookng to find a way to count the number of values in a column and its proving trickier than i originally thought. Percentile Percentile1 Percentile2 Percentile3 0 mediocre contender contender mediocre 69 mediocre bad mediocre mediocre 117 mediocre mediocre mediocre mediocre 144 mediocre none mediocre contender 171 mediocre mediocre contender mediocre i'm trying to create something looking like the following output. It takes the four options and counts them per column. It is essentially a

问题 i'm lookng to find a way to count the number of values in a column and its proving trickier than i originally thought. Percentile Percentile1 Percentile2 Percentile3 0 mediocre contender contender mediocre 69 mediocre bad mediocre mediocre 117 mediocre mediocre mediocre mediocre 144 mediocre none mediocre contender 171 mediocre mediocre contender mediocre i'm trying to create something looking like the following output. It takes the four options and counts them per column. It is essentially a

问题 I have an XML file of size 4 GB. I want to parse it and convert it to a Data Frame to work on it. But because the file size is too large the following code is unable to convert the file to a Pandas Data Frame. The code just keeps loading and does not provide any output. But when I use it for a similar file of smaller size I obtain the correct output. Can anyone suggest any solution to this. Maybe a code that speeds up the process of conversion from XML to Data Frame or splitting of the XML

问题 I have two dataframes each with 10 rows and I am trying to display them side by side using print df, df2 But it is giving output like this Last High High_Perc 170 0.01324000 0.03822200 65.36026372 194 0.00029897 0.00052040 42.54996157 163 0.00033695 0.00058000 41.90517241 130 0.00176639 0.00282100 37.38426090 78 0.00003501 0.00005552 36.94164265 13 0.00009590 0.00014814 35.26393952 58 0.00002149 0.00003228 33.42627014 124 0.00009151 0.00013700 33.20437956 32 0.00059649 0.00089000 32.97865169