subsetting based on number of observations in a factor variable

Question

how do you subset based on the number of observations of the levels of a factor variable? I have a dataset with 1,000,000 rows and nearly 3000 levels, and I want to subset out

Answer 1

I figured it out using the following, as there is no reason to do things twice:

function (df, column, threshold) { 
    size <- nrow(df) 
    if (threshold < 1) threshold <- threshold * size 
    tab <- table(df[[column]]) 
    keep <- names(tab)[tab >  threshold] 
    drop <- names(tab)[tab <= threshold] 
    cat("Keep(",column,")",length(keep),"\n"); print(tab[keep]) 
    cat("Drop(",column,")",length(drop),"\n"); print(tab[drop]) 
    str(df) 
    df <- df[df[[column]] %in% keep, ] 
    str(df) 
    size1 <- nrow(df) 
    cat("Rows:",size,"-->",size1,"(dropped",100*(size-size1)/size,"%)\n") 
    df[[column]] <- factor(df[[column]], levels=keep) 
    df 
}