Extracting unique rows from a data table in R [duplicate]

问题

I'm migrating from data frames and matrices to data tables, but haven't found a solution for extracting the unique rows from a data table. I presume there's something I'm missing about the [,J] notation, though I've not yet found an answer in the FAQ and intro vignettes. How can I extract the unique rows, without converting back to data frames?

Here is an example:

library(data.table)
set.seed(123)
a <- matrix(sample(2, 120, replace = TRUE), ncol = 3)
a <- as.data.frame(a)
b <- as.data.table(a)

# Confirm dimensionality
dim(a) # 40  3
dim(b) # 40  3

# Unique rows using all columns
dim(unique(a))  # 8 3
dim(unique(b))  # 34 3

# Unique rows using only a subset of columns
dim(unique(a[,c("V1","V2")]))   # 4 2
dim(unique(b[,list(V1,V2)]))    # 29 2

Related question: Is this behavior a result of the data being unsorted, as with the Unix uniq function?

回答1:

Before data.table v1.9.8, the default behavior of unique.data.table method was to use the keys in order to determine the columns by which the unique combinations should be returned. If the key was NULL (the default), one would get the original data set back (as in OPs situation).

As of data.table 1.9.8+, unique.data.table method uses all columns by default which is consistent with the unique.data.frame in base R. To have it use the key columns, explicitly pass by = key(DT) into unique (replacing DT in the call to key with the name of the data.table).

Hence, old behavior would be something like

library(data.table) v1.9.7-
set.seed(123)
a <- as.data.frame(matrix(sample(2, 120, replace = TRUE), ncol = 3))
b <- data.table(a, key = names(a))
## key(b)
## [1] "V1" "V2" "V3"
dim(unique(b)) 
## [1] 8 3

While for data.table v1.9.8+, just

b <- data.table(a) 
dim(unique(b)) 
## [1] 8 3
## or dim(unique(b, by = key(b)) # in case you have keys you want to use them

Or without a copy

setDT(a)
dim(unique(a))
## [1] 8 3

回答2:

As mentioned by Seth the data.table package has evolved and now proposes optimized functions for this.

To all the ones who don't want to get into the documentation, here is the fastest and most memory efficient way to do what you want :

uniqueN(a)

And if you only want to choose a subset of columns you could use the 'by' argument :

uniqueN(a,by = c('V1','V2'))

EDIT : As mentioned in the comments this will only gives the count of unique rows. To get the unique values, use unique instead :

unique(a)

And for a subset :

unique(a[c('V1',"V2")], by=c('V1','V2'))

来源：https://stackoverflow.com/questions/7562284/extracting-unique-rows-from-a-data-table-in-r

标签

data.table