Inspecting and visualizing gaps/blanks and structure in large dataframes

Question

I have a large dataframe (400000 x 50) that I want to visually inspect for structure and blanks/gaps.

Is there an existing library or ggplot2 function, that can spit

Answer 1

You may want to have a look at the tabplot package. With such a big data.frame it will take a while to load, but it should also correctly identify missing values. More info here.

Here's an image example using the diamond data.frame.

EDIT

I just saw that you said your df has 50 columns. I've used tabplot on df's that size and find the resolution of information limited by the screen breadth. The row count can also be an issue, but I personally find more information is lost if the df is too wide. Thus, may I suggest you parse it into 3 separate df (for example using dplyr) and then run them through the tableplot() function of tabplot or similar.