Slick 3 - upsert works too slow

Question

I use flowing code for upsert list items

case class Item(id: String, text: String)

class Items(tag: Tag) extends Table[Item](tag, "items"){
    ...
}

val tbl = TableQuery[Items]

def insertItems(items: List[Item]):Future[Int] = {
    val q = DBIO.sequence(items.map(tbl.insertOrUpdate).toSeq).map(_.sum)
    db.run(q)
}

For items list with length 2000, upsert takes ~10 seconds. It's too long...

I think, most part time takes compiling queries.

How should I rewrite insertItems for acceleration it?

Answer 1

Use compiled queries( docs ). AFAIK, Compiled queries for insert are available after slick 2.0.

Also, to insert a list, you should do a batch operation rather than inserting a record one by one.

So, in Slick-3.0, for insert, you should do:

val tblCompiled = Compiled(TableQuery[Items])
tblCompiled ++= items

Then run another query to get the sum of the desired column.

EDIT: I don't think, slick supports bulk insertOrUpdate statements. If the underlying db supports bulk insertOrUpdate, the fastest approach will be to write plain SQL. Otherwise compiled insertOrUpdate query should be decently fast.

The code should be something like

items.map(tblCompiled.insertOrUpdate)