Slick dynamic groupby | 易学教程

问题

I have code like so:

def query(buckets: List[String]): Future[Seq[(List[Option[String]], Option[Double])]] = {
    database.run {
        groupBy(row => buckets.map(bucket => customBucketer(row.metadata, bucket)))
            .map { grouping =>
                val bucket = grouping._1
                val group = grouping._2
                (bucket, group.map(_.value).avg)
            }
            .result
    }
}

private def customBucketer(metadata: Rep[Option[String]], bucket: String): Rep[Option[String]] = {
    ...
}

I am wanting to be able to create queries in slick which groupby and collect on a given list of columns.

Error I am getting upon compilation is:

[error] Slick does not know how to map the given types.
[error] Possible causes: T in Table[T] does not match your * projection,
[error]  you use an unsupported type in a Query (e.g. scala List),
[error]  or you forgot to import a driver api into scope.
[error]   Required level: slick.lifted.FlatShapeLevel
[error]      Source type: List[slick.lifted.Rep[Option[String]]]
[error]    Unpacked type: T
[error]      Packed type: G
[error]                         groupBy(row => buckets.map(bucket => customBucketer(row.metadata, bucket)))
[error]                                ^

回答1:

Here's a workaround for Slick 3.2.3 (and some background on my approach):

You may have noticed dynamically selecting columns is easy as long as you can assume a fixed type, e.g: columnNames = List("col1", "col2") tableQuery.map( r => columnNames.map(name => r.column[String](name)) )

But if you try a similar approach with a groupBy operation, Slick will complain that it "does not know how to map the given types".

So, while this is hardly an elegant solution, you can at least satisfy Slick's type-safety by statically defining both:

groupby column type
Upper/lower bound on the quantity of groupBy columns

A simple way of implementing these two constraints is to again assume a fixed type and to branch the code for all possible quantities of groupBy columns.

Here's the full working Scala REPL session to give you an idea:

import java.io.File

import akka.actor.ActorSystem
import com.typesafe.config.ConfigFactory
import slick.jdbc.H2Profile.api._

import scala.concurrent.{Await, Future}
import scala.concurrent.duration._


val confPath = getClass.getResource("/application.conf")
val config = ConfigFactory.parseFile(new File(confPath.getPath)).resolve()
val db = Database.forConfig("slick.db", config)

implicit val system = ActorSystem("testSystem")
implicit val executionContext = system.dispatcher

case class AnyData(a: String, b: String)
case class GroupByFields(a: Option[String], b: Option[String])

class AnyTable(tag: Tag) extends Table[AnyData](tag, "macro"){
  def a = column[String]("a")
  def b = column[String]("b")
  def * = (a, b) <> ((AnyData.apply _).tupled, AnyData.unapply)
}

val table = TableQuery[AnyTable]

def groupByDynamically(groupBys: Seq[String]): DBIO[Seq[GroupByFields]] = {
  // ensures columns are returned in the right order
  def selectGroups(g: Map[String, Rep[Option[String]]]) = {
    (g.getOrElse("a", Rep.None[String]), g.getOrElse("b", Rep.None[String])).mapTo[GroupByFields]
  }

  val grouped = if (groupBys.lengthCompare(2) == 0) {
    table
      .groupBy( cols => (cols.column[String](groupBys(0)), cols.column[String](groupBys(1))) )
      .map{ case (groups, _) => selectGroups(Map(groupBys(0) -> Rep.Some(groups._1), groupBys(1) -> Rep.Some(groups._2))) }
  }
  else {
    // there should always be at least one group by specified
    table
      .groupBy(cols => cols.column[String](groupBys.head))
      .map{ case (groups, _) => selectGroups(Map(groupBys.head -> Rep.Some(groups))) }
  }

  grouped.result
}

val actions = for {
  _ <- table.schema.create
  _ <- table.map(a => (a.column[String]("a"), a.column[String]("b"))) += ("a1", "b1")
  _ <- table.map(a => (a.column[String]("a"), a.column[String]("b"))) += ("a2", "b2")
  _ <- table.map(a => (a.column[String]("a"), a.column[String]("b"))) += ("a2", "b3")
  queryResult <- groupByDynamically(Seq("b", "a"))
} yield queryResult

val result: Future[Seq[GroupByFields]] = db.run(actions.transactionally)
result.foreach(println)

Await.ready(result, Duration.Inf)

Where this gets ugly is when you can have upwards of a few groupBy columns (i.e. having a separate if branch for 10+ cases would get monotonous). Hopefully someone will swoop in and edit this answer for how to hide that boilerplate behind some syntactic sugar or abstraction layer.

来源：https://stackoverflow.com/questions/49699392/slick-dynamic-groupby

标签

mysql

scala

slick