scala

问题 I have the following code: val fsm = TestFSMRef(new SenderCollectorFsm) And do not understand, why do I have to pass to TestFSMRef an instance. Lets look at the definition of TestFSMRef: object TestFSMRef { def apply[S, D, T <: Actor: ClassTag]( factory: => T)(implicit ev: T <:< FSM[S, D], system: ActorSystem): TestFSMRef[S, D, T] = { val impl = system.asInstanceOf[ActorSystemImpl] new TestFSMRef(impl, Props(factory), impl.guardian.asInstanceOf[InternalActorRef], TestActorRef.randomName) } T

问题 I'm totally new to Apache Spark and Scala , and I'm having problems with mapping a .csv file into a key-value (like JSON) structure. What I want to accomplish is to get the .csv file: user, timestamp, event ec79fcac8c76ebe505b76090f03350a2,2015-03-06 13:52:56,USER_PURCHASED ad0e431a69cb3b445ddad7bb97f55665,2015-03-06 13:52:57,USER_SHARED 83b2d8a2c549fbab0713765532b63b54,2015-03-06 13:52:57,USER_SUBSCRIBED ec79fcac8c76ebe505b76090f03350a2,2015-03-06 13:53:01,USER_ADDED_TO_PLAYLIST ... Into a

问题 I'm having trouble importing packages into package objects. It didn't seem to work in Eclipse and so I switched to intellij. At one point the feature seemed to be working and so I created package objects for most packages. Now it doesn't seem to be working at all. Here's a package object in file package.scala, the package file itself compiles fine: package rStrat.rSwing package testSw //Edited for clarity object testSw { import rStrat._ import rSwing.topUI._ } and here's a class file from the

问题 I have the following snippet I need to complete for an assignment. To fulfill the asignment I have to correctly replace the comments /*fulfill ...*/ . However I tried my best and I am still getting an missing parameter type for expanded function The argument types of an anonymous function must be fully known. (SLS 8.5) error. I found similar questions related to this error. However I could not derive a solution for my paticular problem of those answers. So the target is to check whether the

问题 I am trying to connect to Oracle database using TNS. The problem is that TNS alias contains dot, so when I am specifying url like this: jdbc:oracle:thin:@TNS.ALIAS I've got... oracle.net.ns.NetException: Invalid connection string format, a valid format is: "host:port:sid" ...during creation of connection. I know that dot character is a problem, because after removing it from tnsnames.ora file connection to database works. My question is - is it possible to escape dot character somehow? Maybe

问题 In Dotty given the following: object Domain { final case class Create(name: String) extends BaseCreate[Create] { override type Model = Domain override def service[F[_]](client: KeystoneClient[F]): CrudService[F, Domain, Create] = client.domains } } case class Domain(id: String) class CrudService[F[_], Model, Create] final class Domains[F[_]] extends CrudService[F, Domain, Domain.Create] class KeystoneClient[F[_]] { val domains = new Domains[F] } trait BaseCreate[Create <: BaseCreate[Create]]

问题 I am doing this in Scala and Spark. I have and Dataset of Tuple2 as Dataset[(String, Map[String, String])] . Below is and example of the values in the Dataset . (A, {1->100, 2->200, 3->100}) (B, {1->400, 4->300, 5->900}) (C, {6->100, 4->200, 5->100}) (B, {1->500, 9->300, 11->900}) (C, {7->100, 8->200, 5->800}) If you notice, the key or first element of the Tuple can be repeated. Also, the corresponding map of the same Tuples' key can have duplicate keys in the map (second part of Tuple2). I

问题 I want to transform multiple columns to one column using VectorAssembler ,but the data is compressed by default without other options. val arr2= Array((1,2,0,0,0),(1,2,3,0,0),(1,2,4,5,0),(1,2,2,5,6)) val df=sc.parallelize(arr2).toDF("a","b","c","e","f") val colNames=Array("a","b","c","e","f") val assembler = new VectorAssembler() .setInputCols(colNames) .setOutputCol("newCol") val transDF= assembler.transform(df).select(col("newCol")) transDF.show(false) The input is: +---+---+---+---+---+ |

问题 I have a spark job which samples my input data randomly. Then, I generate a bloom filter for the input data. Finally, I apply the filter and join the data with dataset A. Since the sampling is random, it should only be executed only once. But it executes twice even if I persist it. I can see a green cache step in Spark DAG of the first step but the join still starts from data loading and random sampling. I also found the cached data can be evited when workers are running out of memory, which