scala

How to avoid calling asInstanceOf in Scala

Deadly 提交于 2021-02-08 14:08:52
问题 Here is a simplified version of my code. How can I avoid to call asInstanceOf (because it is a smell for a poorly design solution) ? sealed trait Location final case class Single(bucket: String) extends Location final case class Multi(buckets: Seq[String]) extends Location @SuppressWarnings(Array("org.wartremover.warts.AsInstanceOf")) class Log[L <: Location](location: L, path: String) { // I prefer composition over inheritance // I don't want to pass location to this method because it's a

Scala: abstract type pattern A is unchecked since it is eliminated by erasure

耗尽温柔 提交于 2021-02-08 12:16:35
问题 I am writing the function that can catch exceptions of the certain type only. def myFunc[A <: Exception]() { try { println("Hello world") // or something else } catch { case a: A => // warning: abstract type pattern A is unchecked since it is eliminated by erasure } } What is the corrent way to bypass jvm type erasure in such case? 回答1: You could use ClassTag like in this answer. But I'd prefer this approach: def myFunc(recover: PartialFunction[Throwable, Unit]): Unit = { try { println("Hello

From Scala, access static inner class of Java class

我们两清 提交于 2021-02-08 11:57:28
问题 I have a Java class which I'm working with in Scala. I can instantiate the class, but I don't see how to instantiate a public static class within the outer class from Scala. Can anyone point me to the answer? (I've read lots of posts that say how to use "static" members in Scala code. That is not what I'm looking for.) 回答1: In my case something like that works: Java class package test.java; public class Test { public static class TestInner {} } Scala class package test.scala import test.java

From Scala, access static inner class of Java class

主宰稳场 提交于 2021-02-08 11:57:26
问题 I have a Java class which I'm working with in Scala. I can instantiate the class, but I don't see how to instantiate a public static class within the outer class from Scala. Can anyone point me to the answer? (I've read lots of posts that say how to use "static" members in Scala code. That is not what I'm looking for.) 回答1: In my case something like that works: Java class package test.java; public class Test { public static class TestInner {} } Scala class package test.scala import test.java

Array of struct parsing in Spark dataframe

丶灬走出姿态 提交于 2021-02-08 11:54:14
问题 I have a Dataframe with one struct type column. Sample dataframe schema is: root |-- Data: array (nullable = true) | |-- element: struct (containsNull = true) | | |-- name: string (nullable = true) | | |-- value: string (nullable = true) Field name holds column name and fields value holds column value. Number of elements in Data column is not defined so it can vary. I need to parse that data and get rid of nested structure. (Array Explode will not work in this case because data in one row

scala - how to substring column names after the last dot?

倖福魔咒の 提交于 2021-02-08 11:27:34
问题 After exploding a nested structure I have a DataFrame with column names like this: sales_data.metric1 sales_data.type.metric2 sales_data.type3.metric3 When performing a select I'm getting the error: cannot resolve 'sales_data.metric1' given input columns: [sales_data.metric1, sales_data.type.metric2, sales_data.type3.metric3] How should I select from the DataFrame so the column names are parsed correctly? I've tried the following: the substrings after dots are extracted successfully. But

Scala Gson, cannot serialize Option[Blob]

我与影子孤独终老i 提交于 2021-02-08 11:27:29
问题 I managed to extend Gson to handle Scala's Option and Blob types, but I have trouble with the combined Option[java.sql.Blob] . My test says [info] - should serialize an Option - OK [info] - should serialize a Blob - OK [info] - should serialize an Option[Blob] * FAILED * [info] {"x":{"buf":[97,115,100,102],"len":4,"origLen":4}} was not equal to {"x":"asdf"} Can you please tell me what this {"buf": [...]} object is, and possibly why Gson is doing that? My serializer (below) definitely tells

Gson with Scala causes StackOverflow for Enumerations

青春壹個敷衍的年華 提交于 2021-02-08 11:14:11
问题 I have an enum defined in Scala class as follows // define compression types as enumerator object CompressionType extends Enumeration { type CompressionType = Value val None, Gzip, Snappy, Lz4, Zstd = Value } and I have class that I want to Serialize in JSON case class ProducerConfig(batchNumMessages : Int, lingerMs : Int, messageSize : Int, topic: String, compressionType: CompressionType.Value ) That class includes the Enum object. It seems that using GSON to serialize causes StackOverflow

Does @tailrec affect compiler optimizations?

时光总嘲笑我的痴心妄想 提交于 2021-02-08 11:13:02
问题 I looked at this question trying to better understand @tailrec annotation in scala. What I'm not sure is whether the annotation also hints the compiler to do some optimizations or it's only used for warnings when you mark a method that is not a tail-recursion? More specifically - is this annotation might affect performance in any way? For example, if I don't put this annotation, the compiler will compile a tail-recursive function as non-tail recursive? 回答1: As per the scaladoc: A method

Gson with Scala causes StackOverflow for Enumerations

我怕爱的太早我们不能终老 提交于 2021-02-08 11:12:41
问题 I have an enum defined in Scala class as follows // define compression types as enumerator object CompressionType extends Enumeration { type CompressionType = Value val None, Gzip, Snappy, Lz4, Zstd = Value } and I have class that I want to Serialize in JSON case class ProducerConfig(batchNumMessages : Int, lingerMs : Int, messageSize : Int, topic: String, compressionType: CompressionType.Value ) That class includes the Enum object. It seems that using GSON to serialize causes StackOverflow