Identifying Outliers using Quartile Range

问题

I have a dataframe which consists of numerical values with 22 columns. When I do summary(df) on it get be details (min,max,mean,median,1 and 3rd quartiles). Now I want to get 1 and 3rd quartiles for each of the column. Anything above or below it would be an Outlier and I would like to replace the Outlier with NA value.

Summary :
 Var 1                 Var2             Var 3                Var 4                             
 Min.   : 0      Min.   :0       Min : 0           Min : -127.00           
 1st Qu.: 1208   1st Qu.: 1150  1st Qu.: 135000   1st Qu.: 98      
 Median : 1400   Median : 1300   Median : 180000   Median : 99      
 Mean   : 1617   Mean   : 2138   Mean   : 211759   Mean   : 96.59      
 3rd Qu.: 1990   3rd Qu.: 2500   3rd Qu.: 250000   3rd Qu.: 100      
 Max.   :10000   Max  :4000   Max.   :40000   Max:9999.

Its not a duplicate question because, we are not fixating on the quartile range explicitly, we are deriving the value from the data itself

回答1:

Long and commented way to do it, there are thousands:

### take the Q1 - Q3 values (you could also use quantile function where you can choose methods to get quantile) 
q1 <- as.numeric(summary(old_vector)[2])
q3 <- as.numeric(summary(old_vector)[5])

new_vector <- vector()
for (value in old_vector) {
  if ( !is.na(value) && (value < q1 || value > q3) ) new_vector <- append(new_vector, NA)
  else new_vector <- append(new_vector, value)
}

EDITED as you commented:

Of course it can work with such structures:

### your DF
df1 <- structure(list(Var1 = c(100.2, 110, 200, 456, 120000), var2 = c(NA, 4545, 45465, 44422, 250000), var3 = c(NA, 210000, 91500, 215000, 250000), var4 = c(0.983, 0.44, 0.983, 0.78, 2.23)), class = "data.frame", row.names = c(NA, -5L))

### declare the function to replace a vector outliers based on IQR boundaries
replace_outliers <- function (old_vector) {
    q1 <- as.numeric(summary(old_vector)[2])
    q3 <- as.numeric(summary(old_vector)[5])
    new_vector <- vector()
    for (value in old_vector) {
      if ( !is.na(value) && (value < q1 || value > q3) ) new_vector <- append(new_vector, NA)
      else new_vector <- append(new_vector, value)
    }
    return(new_vector)
}

### open loop on DF columns
for ( col in colnames(df1) ) {
    ### create new column name
    name_new_col <- paste( col, "_replaced", sep = "" )
    ### put the replaced values in the new column
    df1[,name_new_col] <- replace_outliers(df1[,col])
}

and you will have the DF with the new columns "Var[n]_replaced" with NA instead of IQR outliers

来源：https://stackoverflow.com/questions/56634463/identifying-outliers-using-quartile-range

标签

outliers