1. 技术文章
  2. R with tcltk/tcltk2: Improve slow performance when displaying big data.frame with TkTable?

R with tcltk/tcltk2: Improve slow performance when displaying big data.frame with TkTable?

由 情到浓时终转凉″ 提交于 2019-12-06 06:09:24

I have found one possible solution/workaround using Tktable in an "unbound" (command) mode.

With the command option of Tktable you can specify a function that is called each time a cell shall be displayed on the screen. This avoids "loading" all the data from R to Tcl at once improving the "start-up" time and significantly reduces the memory consumption caused by TCL's way of storing arrays and lists.

This way every time you scroll a series of function calls are done to ask for the content of the visible cells.

It works for me even with over 10 mio. rows!

Drawback: Calling an R function that returns a Tcl variable for each cell is still far from being efficient. If you scroll for the first time you can watch the cells being updated. Therefore I am still looking for a bulk data transfer solution between R and Tcl/Tk.

Any suggestions to improve the performance are welcome!

I have implemented a small demo (with 1 mio. rows and 21 columns consuming 1.2 GB of RAM) and added some buttons to test different features (like caching).

Note: The long start-up time is caused by creating the underlying test data, NOT by Tktable!

library(tcltk)
library(data.table)

# Tktable example with -command ("unbound" mode) ---------------------------
# Doc: http://tktable.sourceforge.net/tktable/doc/tkTable.html

NUM.ROWS <- 1E6
NUM.COLS <- 20

# generate a big data.frame - this will take a while but is required for the demo
dt.data <- data.table(ID = 1:NUM.ROWS)

for (i in 1:NUM.COLS) {
  dt.data[, (paste("Col",i)) := paste0("R", 1:NUM.ROWS, " C", i)]
}

# Fill one cell with a long text containing special control characters to test the Tktable behaviour
dt.data[3,3 := "This is a long text with backslash \\ and \"quotes\"!"]

tclRequire("Tktable")

t <- tktoplevel()

tkwm.protocol(t, "WM_DELETE_WINDOW", function() tkdestroy(t))

# Function to return the current row and column as "calculated" value (without an underlying data "model")
calculated.data <- function(C) {
  # Function arguments  for Tcl "substitutions":
  # See:   http://tktable.sourceforge.net/tktable/doc/tkTable.html
  #   %c the column of the triggered cell.
  #   %C A convenience substitution for %r,%c.
  #   %i 0 for a read (get) and 1 for a write (set). Otherwise it is the current cursor position in the cell.
  #   %r the row of the triggered cell.
  return(tclVar(C))  # this does work!
}

# Function to return the content of a data.table for the current row and colum
data.frame.data <- function(r, c) {
  if( r == "0")
    return(tclVar(names(dt.data)[as.integer(c)+1]))             # First row contains the column names
  else
    return(tclVar(as.character(dt.data[as.integer(r)+1, as.integer(c)+1, with = FALSE])))   # Other rows are data rows
}

frame <- ttklabelframe(t, text = "Data:")
# Add the table to the window environment to ensure killing it when the window is closed (= no more phantom calls to the data command handler)
# Cache = TRUE: This greatly enhances speed performance when used with -command but uses extra memory.
t$env$table <- tkwidget(frame, "table", rows = NUM.ROWS, cols = NUM.COLS, titlerows = 1, selecttype = "cell", selectmode = "extended", command = calculated.data, cache = TRUE, yscrollcommand = function(...) tkset(scroll.y, ...), xscrollcommand = function(...) tkset(scroll.x, ...))

scroll.x <- ttkscrollbar(frame, orient = "horizontal", command=function(...) tkxview(t$env$table,...))  # command that performs the scrolling
scroll.y <- ttkscrollbar(frame, orient = "vertical", command=function(...) tkyview(t$env$table,...))  # command that performs the scrolling

buttons <- ttkframe(t)
btn.read.only <- ttkbutton(buttons, text = "make read only", command = function() tkconfigure(t$env$table, state = "disabled"))
btn.read.write <- ttkbutton(buttons, text = "make writable", command = function() tkconfigure(t$env$table, state = "normal"))
btn.clear.cache <- ttkbutton(buttons, text = "clear cache", command = function() tcl(t$env$table, "clear", "cache"))
btn.bind.data.frame <- ttkbutton(buttons, text = "Fill cells from R data.table",
                                 command = function() {
                                   tkconfigure(t$env$table, command = data.frame.data, rows = nrow(dt.data), cols = ncol(dt.data), titlerows = 1)
                                   tcl(t$env$table, "clear", "cache")
                                   tkwm.title(t,"Cells are filled from an R data.table")
                                 })
btn.bind.calc.value <- ttkbutton(buttons, text = "Fill cells with calculated values",
                                 command = function() {
                                   tkconfigure(t$env$table, command = calculated.data, rows = 1E5, cols = 40)
                                   tcl(t$env$table, "clear", "cache")
                                   tkwm.title(t,"Cells are calculated values (to test the highest performance possible)")
                                 })

tkgrid(btn.read.only, row = 0, column = 1)
tkgrid(btn.read.write, row = 0, column = 2)
tkgrid(btn.clear.cache, row = 0, column = 3)
tkgrid(btn.bind.data.frame, row = 0, column = 5)
tkgrid(btn.bind.calc.value, row = 0, column = 6)

tkpack(frame, fill = "both", expand = TRUE)
tkpack(scroll.x, fill = "x", expand = FALSE, side = "bottom")
tkpack(scroll.y, fill = "y", expand = FALSE, side = "right")
tkpack(t$env$table, fill = "both", expand = TRUE, side = "left")
tkpack(buttons, side = "bottom")
标签
r
user-interface
tcl
tk
ttk
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!