data-science

Where do I call the BatchNormalization function in Keras?

由 泪湿孤枕 提交于 2019-11-27 16:37:50
If I want to use the BatchNormalization function in Keras, then do I need to call it once only at the beginning? I read this documentation for it: http://keras.io/layers/normalization/ I don't see where I'm supposed to call it. Below is my code attempting to use it: model = Sequential() keras.layers.normalization.BatchNormalization(epsilon=1e-06, mode=0, momentum=0.9, weights=None) model.add(Dense(64, input_dim=14, init='uniform')) model.add(Activation('tanh')) model.add(Dropout(0.5)) model.add(Dense(64, init='uniform')) model.add(Activation('tanh')) model.add(Dropout(0.5)) model.add(Dense(2,

quantile normalization on pandas dataframe

由 为君一笑 提交于 2019-11-27 14:02:52
问题 Simply speaking, how to apply quantile normalization on a large Pandas dataframe (probably 2,000,000 rows) in Python? PS. I know that there is a package named rpy2 which could run R in subprocess, using quantile normalize in R. But the truth is that R cannot compute the correct result when I use the data set as below: 5.690386092696389541e-05,2.051450375415418849e-05,1.963190184049079707e-05,1.258362869906251862e-04,1.503352476021528139e-04,6.881341586355676286e-06 8.535579139044583634e-05,5

The simplest way to convert a list with various length vectors to a data.frame in R

由 两盒软妹~` 提交于 2019-11-27 09:10:31
Here I have a list with different length vectors. And I'd want to get a data.frame. I've seen lots of posts about it in SO (see ref), but none of them are as simple as I expected because this is really a common task in data preprocessing. Thank you. Here simplest means as.data.frame(aa) if it works. So one function from the base package of R will be great. sapply(aa, "length<-", max(lengths(aa))) has four functions actually. An example is shown below. Input: aa <- list(A=c(1, 3, 4), B=c(3,5,7,7,8)) Output: A B 1 3 3 5 4 7 NA 7 NA 8 A and B are the colnames of the data.frame. One answer is

How to load a model from an HDF5 file in Keras?

由 浪尽此生 提交于 2019-11-27 09:04:25
问题 How to load a model from an HDF5 file in Keras? What I tried: model = Sequential() model.add(Dense(64, input_dim=14, init='uniform')) model.add(LeakyReLU(alpha=0.3)) model.add(BatchNormalization(epsilon=1e-06, mode=0, momentum=0.9, weights=None)) model.add(Dropout(0.5)) model.add(Dense(64, init='uniform')) model.add(LeakyReLU(alpha=0.3)) model.add(BatchNormalization(epsilon=1e-06, mode=0, momentum=0.9, weights=None)) model.add(Dropout(0.5)) model.add(Dense(2, init='uniform')) model.add

Filter pandas dataframe by list

由 烂漫一生 提交于 2019-11-27 08:52:22
问题 I have a dataframe that has a row called "Hybridization REF". I would like to filter so that I only get the data for the items that have the same label as one of the items in my list. Basically, I'd like to do the following: dataframe[dataframe["Hybridization REF'].apply(lambda: x in list)] but that syntax is not correct. 回答1: You can use .loc or column filtering: df = pd.DataFrame(data=np.random.rand(5,5),columns=list('ABCDE'),index=list('abcde')) df A B C D E a 0.460537 0.174788 0.167554 0

Update pandas dataframe based on matching columns of a second dataframe

由 牧云@^-^@ 提交于 2019-11-27 08:25:40
问题 I have two pandas dataframes ( df_1 , df_2 ) with the same columns, but in one dataframe ( df_1 ) some values of one column are missing. So I want to fill in those missing values from df_2 , but only when the the values of two columns match. Here is a little example what my data looks like: df_1: df_2: I tried to add the missing values with: df_1.update(df_2, overwrite=False) But the problem is, that it will fill in the values, even when just one column matches. I want to fill in the value

Sparklyr: Use group_by and then concatenate strings from rows in a group

由 喜欢而已 提交于 2019-11-27 07:06:58
问题 I am trying to use the group_by() and mutate() functions in sparklyr to concatenate rows in a group. Here is a simple example that I think should work but doesn't: library(sparkylr) d <- data.frame(id=c("1", "1", "2", "2", "1", "2"), x=c("200", "200", "200", "201", "201", "201"), y=c("This", "That", "The", "Other", "End", "End")) d_sdf <- copy_to(sc, d, "d") d_sdf %>% group_by(id, x) %>% mutate( y = paste(y, collapse = " ")) What I'd like it to produce is: Source: local data frame [6 x 3]

reshape multi id repeated variable readings from long to wide

由 时光总嘲笑我的痴心妄想 提交于 2019-11-27 06:28:35
问题 This is what I have: id<-c(1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2) measure<-c("speed","weight","time","speed","weight","time","speed","weight","time", "speed","weight","time","speed","weight","time","speed","weight","time") value<-c(1.23,10.3,33,1.44,10.4,31,1.21,10.1,33,4.25,12.5,38,1.74,10.8,31,3.21,10.3,33) testdf<-data.frame(id,measure,value) This is what I want: id<-c(1,1,1,2,2,2) speed<-c(1.23,1.44,1.21,4.25,1.74,3.21) weight<-c(10.3,10.4,10.1,12.5,10.8,10.3) time<-c(33,31,33,37,31,33) res

Predictive Analytics - “why” factor & model interpretability

由 孤街醉人 提交于 2019-11-27 03:50:01
问题 I have the data that contains tons of x variables that are mainly categorical/nominal and my target variable is a multi-class label. I am able to build a couple models around to predict multi-class variables and compare how each of them performed. I have training and testing data. Both the training and testing data gave me good results. Now, I am trying to find out "why" did the model predicted certain Y-variable? Meaning if I have weather data: X Variable: city, state, zip code, temp, year;

Where do I call the BatchNormalization function in Keras?

由 元气小坏坏 提交于 2019-11-26 22:29:24
问题 If I want to use the BatchNormalization function in Keras, then do I need to call it once only at the beginning? I read this documentation for it: http://keras.io/layers/normalization/ I don't see where I'm supposed to call it. Below is my code attempting to use it: model = Sequential() keras.layers.normalization.BatchNormalization(epsilon=1e-06, mode=0, momentum=0.9, weights=None) model.add(Dense(64, input_dim=14, init='uniform')) model.add(Activation('tanh')) model.add(Dropout(0.5)) model
订阅 data-science