imputation

问题 I want to forward fill a column and I want to specify a limit, but I want the limit to be based on the index---not a simple number of rows like limit allows. For example, say I have the dataframe given by: df = pd.DataFrame({ 'data': [0.0, 1.0, np.nan, 3.0, np.nan, 5.0, np.nan, np.nan, np.nan, np.nan], 'group': [0, 0, 0, 1, 1, 0, 0, 0, 1, 1] }) which looks like In [27]: df Out[27]: data group 0 0.0 0 1 1.0 0 2 NaN 0 3 3.0 1 4 NaN 1 5 5.0 0 6 NaN 0 7 NaN 0 8 NaN 1 9 NaN 1 If I group by the

问题 I am researching how to use multiple imputation results. The following is my understanding, and please let me know if there're mistakes. Suppose you have a data set with missing values, and you want to conduct a regression analysis. You may perform multiple imputation for m = 5 times, and for each imputed data set (5 imputed data sets now) you run a regression analysis, then "pool" the coefficient estimates from these m = 5 models via Rubin's rules (or use R package "pool"). My question is

问题 I am researching how to use multiple imputation results. The following is my understanding, and please let me know if there're mistakes. Suppose you have a data set with missing values, and you want to conduct a regression analysis. You may perform multiple imputation for m = 5 times, and for each imputed data set (5 imputed data sets now) you run a regression analysis, then "pool" the coefficient estimates from these m = 5 models via Rubin's rules (or use R package "pool"). My question is

问题 I have used the below tapply function to get the median of Age based on Pclass. Now how can I impute those median values to NA values based on Pclass? tapply(titan_train$Age, titan_train$Pclass, median, na.rm=T) 回答1: Here is another base R approach that uses replace and ave . df1 <- transform(df1, Age = ave(Age, Pclass, FUN = function(x) replace(x, is.na(x), median(x, na.rm = T)))) df1 # Pclass Age # 1 A 1 # 2 A 2 # 3 A 3 # 4 B 4 # 5 B 5 # 6 B 6 # 7 C 7 # 8 C 8 # 9 C 9 Same idea but using

问题 I have this data set with 78 columns and 5707 rows. Almost every column has missing values and I would like to impute them with IterativeImputer. If I understood it correctly, it will make a "smarter" imputation on each column based on the information from other columns. However, when imputing, I do not want the imputed values to be less than the observed minimum or more than the observed maximum. I realize there are max_value and min_value parameters, but I do not want to impose a "global"

问题 I have this data set with 78 columns and 5707 rows. Almost every column has missing values and I would like to impute them with IterativeImputer. If I understood it correctly, it will make a "smarter" imputation on each column based on the information from other columns. However, when imputing, I do not want the imputed values to be less than the observed minimum or more than the observed maximum. I realize there are max_value and min_value parameters, but I do not want to impose a "global"

来源： https://stackoverflow.com/questions/63650987/how-to-fill-nan-values-by-imputation-in-the-titanic-age-column

来源： https://stackoverflow.com/questions/63650987/how-to-fill-nan-values-by-imputation-in-the-titanic-age-column

问题 I try to fill missing data in a pyspark dataframe. The pyspark dataframe looks as such: +---------+---------+-------------------+----+ | latitude|longitude| timestamplast|name| +---------+---------+-------------------+----+ | | 4.905615|2019-08-01 00:00:00| 1| |51.819645| |2019-08-01 00:00:00| 1| | 51.81964| 4.961713|2019-08-01 00:00:00| 2| | | |2019-08-01 00:00:00| 3| | 51.82918| 4.911187| | 3| | 51.82385| 4.901488|2019-08-01 00:00:03| 5| +---------+---------+-------------------+----+ Within

问题 I have a matrix that is half-sparse. Half of all cells are blank (na) so when I try to run the 'mice' it tries to work on all of them. I'm only interested in a subset. Question: In the following code, how do I make "mice" only operate on the first two columns? Is there a clean way to do this using row-lag or row-lead, so that the content of the previous row can help patch holes in the current row? set.seed(1) #domain x <- seq(from=0,to=10,length.out=1000) #ranges y <- sin(x) +sin(x/2) + rnorm