How to register byte[][] using kryo serialization for spark

前端 未结 2 489
日久生厌
日久生厌 2021-01-02 17:44

I am trying to fully utilize kryo serialization for spark. Setting

.set(\"spark.kryo.registrationRequired\", \"true\")

This will let me kno

相关标签:
2条回答
  • 2021-01-02 18:13

    I know that extremely old question but for those who might look for a solution for any array of class:

    case class Person(name: String)
    
    conf.registerKryoClasses(Array(
       classOf[Array[Array[Person]]]
    ))
    

    Or in case of this specific question:

    conf.registerKryoClasses(Array(
       classOf[Array[Array[Byte]]]
    ))
    
    0 讨论(0)
  • 2021-01-02 18:18
    conf.registerKryoClasses(Array( Class.forName("[[B"))) 
    

    should work

    0 讨论(0)
提交回复
热议问题