Extracting data from a list of lists into its own `data.frame` with `purrr`

Question

Representative sample data (list of lists):

l <- list(structure(list(a = -1.54676469632688, b = \"s\", c = \"T\", 
d = structure(list(id = 5L, label = \"U         


        

Answer 1

You can do this in three steps, first pulling out d, then binding the rows within each element of d, and then binding everything into a single object.

I use bind_rows from dplyr for the within-list row binding. map_df does the final row binding.

library(purrr)
library(dplyr)

l %>%
    map("d") %>%
    map_df(bind_rows)

This is also equivalent:

map_df(l, ~bind_rows(.x[["d"]] ) )

The result looks like:

# A tibble: 12 x 5
      id          label                 link      score externalId
   <int>          <chr>                <chr>      <dbl>      <dbl>
 1     5           Utah          Asia/Anadyr -0.2110459         NA
 2     8 South Carolina       Pacific/Wallis  0.5265409  -6.743544
 3     9       Nebraska America/Scoresbysund  0.2508955  16.425747
 4     8        Georgia         America/Nome  0.5264941   7.915836
 5     2     Washington     America/Shiprock -0.5551864  15.068666
 6     6   North Dakota            Universal  1.0316830         NA
 7     1  New Hampshire      America/Cordoba  1.2158206   9.727642
 8     1         Alaska        Asia/Istanbul -0.2318326         NA
 9     4   Pennsylvania Africa/Dar_es_Salaam  0.5902453         NA
10     3       Delaware       Asia/Samarkand  0.6955771  15.236482
11     4   North Dakota      America/Tortola  1.0306027  -7.216669
12     9       Nebraska      America/Ojinaga -1.1139800  -8.451451

Answer 2

For more information on purrr, I recommend Grolemund and Wickham's "R for Data Science" http://r4ds.had.co.nz/

I think one issue you are facing is that some of the items in l$d are lists of variables with one observation each, ready to be converted to data frames, while other items are lists of such lists.

But I'm not that good at purrr myself. Here's how I would do it:

l <- lapply(l, function(x){x$d}) ## work with the data you need.

list_of_observations <- Filter(function(x) {!is.null(names(x))},l)

list_of_lists <- Filter(function(x) {is.null(names(x))}, l)

another_list_of_observations <- unlist(list_of_lists, recursive=FALSE)

df <- lapply(c(list_of_observations, another_list_of_observations),
             as.data.frame) %>% bind_rows