scala

问题 I believe my understanding on this is correct but I'd like to check. When creating typeclasses, it feels neater to have them take a single type parameter, like TypeClass[A] . If the typeclass needs to be parameterized in other ways, abstract types can be used, and there is a comparison of the two approaches here: Abstract types versus type parameters So far as I have been able to figure out, one thing which is not mentioned in the link is that if using a type parameter, you can witness that

问题 I believe my understanding on this is correct but I'd like to check. When creating typeclasses, it feels neater to have them take a single type parameter, like TypeClass[A] . If the typeclass needs to be parameterized in other ways, abstract types can be used, and there is a comparison of the two approaches here: Abstract types versus type parameters So far as I have been able to figure out, one thing which is not mentioned in the link is that if using a type parameter, you can witness that

问题 import scala.reflect.runtime.universe import scala.reflect.runtime.universe._ def getType[T: TypeTag](obj: T) = typeOf[T] case class Thing( val id: Int, var name: String ) val thing = Thing(1, "Apple") val dataType = getType(thing).decl(TermName("id")).asTerm.typeSignature dataType match { case t if t =:= typeOf[Int] => println("I am Int") case t if t =:= typeOf[String] => println("String, Do some stuff") case _ => println("Absurd") } Not able to digest why result is Absurd instead of I am

问题 I am new to Spark and working on a simple application to convert XML streams received from Kafka in to JSON format Using: Spark 2.4.5 Scala 2.11.12 In my use case kafka stream is in xml format). The Following is the code that I tried. val spark: SparkSession = SparkSession.builder() .master("local") .appName("Spark Demo") .getOrCreate() spark.sparkContext.setLogLevel("ERROR") val inputStream = spark.readStream .format("kafka") .option("kafka.bootstrap.servers", "localhost:9092") .option(

问题 I am working on scala play framework application. I am trying to call a web service API which takes request payload data as follows { "toID": [ "email1@email.com", "email2@email.com" ], "fromID": "info@test.com", "userID": "ervd12fdsfksdjnfn9832rbjfdsnf", "mailContent": "Dear Sir, ..." } And for this I am using following code ws.url(Utils.messengerServiceUrl + "service/email") .post( Map("userID" -> userID, "mailContent" -> userData.message, "fromID" -> "info@test.com", "toID" -> userData

问题 Have different files in a directory as below f1.txt id FName Lname Adrress sex levelId t1 Girish Hm 10oak m 1111 t2 Kiran Kumar 5wren m 2222 t3 sara chauhan 15nvi f 6666 f2.txt t4 girish hm 11oak m 1111 t5 Kiran Kumar 5wren f 2222 t6 Prakash Jha 18nvi f 3333 f3.txt t7 Kiran Kumar 5wren f 2222 t8 Girish Hm 10oak m 1111 t9 Prakash Jha 18nvi m 3333 f4.txt t10 Kiran Kumar 5wren f 2222 t11 girish hm 10oak m 1111 t12 Prakash Jha 18nvi f 3333 only first name and last name constant here and case

问题 I'm trying to connect two streams, first is persisting in MapValueState : RocksDB save data in checkpoint folder, but after new run, state is empty. I run it locally and in flink cluster with cancel submiting in cluster and simply rerun locally env.setStateBackend(new RocksDBStateBackend(..) env.enableCheckpointing(1000) ... val productDescriptionStream: KeyedStream[ProductDescription, String] = env.addSource(..) .keyBy(_.id) val productStockStream: KeyedStream[ProductStock, String] = env

问题 I'm trying to connect two streams, first is persisting in MapValueState : RocksDB save data in checkpoint folder, but after new run, state is empty. I run it locally and in flink cluster with cancel submiting in cluster and simply rerun locally env.setStateBackend(new RocksDBStateBackend(..) env.enableCheckpointing(1000) ... val productDescriptionStream: KeyedStream[ProductDescription, String] = env.addSource(..) .keyBy(_.id) val productStockStream: KeyedStream[ProductStock, String] = env

问题 I am trying to serialize a json file to parquet format. I have this error : Error:(34, 25) overloaded method foreachBatch with alternatives: (function: org.apache.spark.api.java.function.VoidFunction2[org.apache.spark.sql.Dataset[org.apache.spark.sql.Row],java.lang.Long])org.apache.spark.sql.streaming.DataStreamWriter[org.apache.spark.sql.Row] (function: (org.apache.spark.sql.Dataset[org.apache.spark.sql.Row], scala.Long) => Unit)org.apache.spark.sql.streaming.DataStreamWriter[org.apache

问题 I look for a way to retrieve the first elements of a DStream created as: val dstream = ssc.textFileStream(args(1)).map(x => x.split(",").map(_.toDouble)) Unfortunately, there is no take function (as on RDD) on a dstream // dstream.take(2) !!! Could someone has any idea on how to do it ?! thanks 回答1: You can use transform method in the DStream object then take n elements of the input RDD and save it to a list, then filter the original RDD to be contained in this list. This will return a new