scala

How to find the index of the maximum value in a vector column?

…衆ロ難τιáo~ 提交于 2021-01-27 07:41:15
问题 I have a Spark DataFrame with the following structure: root |-- distribution: vector (nullable = true) +--------------------+ | topicDistribution| +--------------------+ | [0.1, 0.2] | | [0.3, 0.2] | | [0.5, 0.2] | | [0.1, 0.7] | | [0.1, 0.8] | | [0.1, 0.9] | +--------------------+ My question is: How to add a column with the index of the maximum value for each row? It should be something like this: root |-- distribution: vector (nullable = true) |-- max_index: integer (nullable = true) +----

Make one sbt config depend on another

折月煮酒 提交于 2021-01-27 07:36:16
问题 The sbt documentation shows example of how to declare dependency only between projects. But I'm positive that there are ways to declare one config be dependent on another, just like the Test configuration uses classpath from the Compile configuration. How can I declare my own configuration so that it would depend on the Compile config generated classpath? I take a more close look to suggested solution and bunch of question arose again. So I reopen the question I can not deduce sbt behavior

Make one sbt config depend on another

不想你离开。 提交于 2021-01-27 07:34:17
问题 The sbt documentation shows example of how to declare dependency only between projects. But I'm positive that there are ways to declare one config be dependent on another, just like the Test configuration uses classpath from the Compile configuration. How can I declare my own configuration so that it would depend on the Compile config generated classpath? I take a more close look to suggested solution and bunch of question arose again. So I reopen the question I can not deduce sbt behavior

How to convert two consecutive elements from List to entries in Map?

淺唱寂寞╮ 提交于 2021-01-27 07:27:50
问题 I have a list: List(1,2,3,4,5,6) that I would like to to convert to the following map: Map(1->2,3->4,5->6) How can this be done? 回答1: Mostly resembles @Vakh answer, but with a nicer syntax: val l = List(1,2,3,4,5,6) val m = l.grouped(2).map { case List(key, value) => key -> value}.toMap // Map(1 -> 2, 3 -> 4, 5 -> 6) 回答2: Try: val l = List(1,2,3,4,5,6) val m = l.grouped(2).map(l => (l(0), l(1))).toMap 回答3: if the list is guaranteed to be of even length: val l = List(1,2,3,4,5,6) val m = l

How to convert two consecutive elements from List to entries in Map?

心已入冬 提交于 2021-01-27 07:27:08
问题 I have a list: List(1,2,3,4,5,6) that I would like to to convert to the following map: Map(1->2,3->4,5->6) How can this be done? 回答1: Mostly resembles @Vakh answer, but with a nicer syntax: val l = List(1,2,3,4,5,6) val m = l.grouped(2).map { case List(key, value) => key -> value}.toMap // Map(1 -> 2, 3 -> 4, 5 -> 6) 回答2: Try: val l = List(1,2,3,4,5,6) val m = l.grouped(2).map(l => (l(0), l(1))).toMap 回答3: if the list is guaranteed to be of even length: val l = List(1,2,3,4,5,6) val m = l

How to convert two consecutive elements from List to entries in Map?

我是研究僧i 提交于 2021-01-27 07:27:03
问题 I have a list: List(1,2,3,4,5,6) that I would like to to convert to the following map: Map(1->2,3->4,5->6) How can this be done? 回答1: Mostly resembles @Vakh answer, but with a nicer syntax: val l = List(1,2,3,4,5,6) val m = l.grouped(2).map { case List(key, value) => key -> value}.toMap // Map(1 -> 2, 3 -> 4, 5 -> 6) 回答2: Try: val l = List(1,2,3,4,5,6) val m = l.grouped(2).map(l => (l(0), l(1))).toMap 回答3: if the list is guaranteed to be of even length: val l = List(1,2,3,4,5,6) val m = l

Scala implicit def do not work if the def name is toString

[亡魂溺海] 提交于 2021-01-27 06:34:56
问题 This code fails to compile: object Foo { implicit def toString(i: Int): String = i.toString def foo(x: String) = println(x) foo(23) } Above code fails to compile with following error: error: type mismatch; found : scala.this.Int(23) required: String foo(23) But, this code compiles object Foo { implicit def asString(i: Int): String = i.toString def foo(x: String) = println(x) foo(23) } Why does the name of an implicit def should matter? Note: If the method is named equals , it also does not

Scala implicit def do not work if the def name is toString

让人想犯罪 __ 提交于 2021-01-27 06:34:14
问题 This code fails to compile: object Foo { implicit def toString(i: Int): String = i.toString def foo(x: String) = println(x) foo(23) } Above code fails to compile with following error: error: type mismatch; found : scala.this.Int(23) required: String foo(23) But, this code compiles object Foo { implicit def asString(i: Int): String = i.toString def foo(x: String) = println(x) foo(23) } Why does the name of an implicit def should matter? Note: If the method is named equals , it also does not

Mock partially a class with scalamock

风流意气都作罢 提交于 2021-01-27 06:31:58
问题 I'm trying to test a class Cls with two functions: A and B . A loads a DataFrame and B calls A then does some operations and returns a new DataFrame . For the sake of example: class Cls { def A(dummy: Int): Int = 5 def B(): Int = A(7) + 1 } With Scalamock how can write my test code ? I tried: test("test case") { val f = stub[Cls] f.A _ when 7 returns 5 assert(f.B() == 6) } I expect test passed successfully and I get 0 did not equal 6 (mytestcase.scala:24) (I do understand that that scalamock

Is it possible to write an immutable doubly linked list?

元气小坏坏 提交于 2021-01-27 06:16:34
问题 I feel a little stupid for asking this, but I'm currently learning functional programming and completed an exercise on creating singly linked lists and it just got me thinking, is it even possible to create an immutable doubly linked list? Suppose the list A :: B, at time of construction, A needs to know about B, but B also needs to know about A. I've been doing this in Scala, so I'm not sure if it's specific to Scala, but I can't picture how that would work. I'm not looking for a substitute