scala

问题 (kinda related to How to create dynamic metric in Flink) I have a stream of events(someid:String, name:String) and for monitoring reasons, I need a counter per event ID. In all the Flink documentations and examples, I can see that the counter is , for instance, initialised with a name in the open of a map function. But in my case I can not initialise the counter as I will need one per eventId and I do not know the value in advance. Also, I understand how expensive it would be to create a new

问题 I want to be able to pass the join condition for two data frames as an input string. The idea is to make the join generic enough so that the user could pass on the condition they like. Here's how I am doing it right now. Although it works, I think its not clean. val testInput =Array("a=b", "c=d") val condition: Column = testInput.map(x => testMethod(x)).reduce((a,b) => a.and(b)) firstDataFrame.join(secondDataFrame, condition, "fullouter") Here's the testMethod def testMethod(inputString:

问题 (kinda related to How to create dynamic metric in Flink) I have a stream of events(someid:String, name:String) and for monitoring reasons, I need a counter per event ID. In all the Flink documentations and examples, I can see that the counter is , for instance, initialised with a name in the open of a map function. But in my case I can not initialise the counter as I will need one per eventId and I do not know the value in advance. Also, I understand how expensive it would be to create a new

问题 In a dataframe the columns have the input shown below: | id| priority| status| datetime|data_as_of_Date|Amount|open_close| | 1|Unassigned| Fixed| 10/8/2019 0:00| 2/12/2020 0:00| 40| Closed| | 1|Unassigned| New|2/12/2019 11:00| 2/12/2020 0:00| 20| Open| | 1|Unassigned|Fix in progress|9/12/2019 11:00| 2/12/2020 0:00| 90| Open| | 3| Critical| Removed|5/17/2019 12:00| 2/12/2020 0:00| 33| Closed| | 3|Unassigned|Fix in progress|5/26/2019 10:00| 2/12/2020 0:00| 30| Open| | 3| Critical| New| 5/8/2019

问题 In a dataframe the columns have the input shown below: | id| priority| status| datetime|data_as_of_Date|Amount|open_close| | 1|Unassigned| Fixed| 10/8/2019 0:00| 2/12/2020 0:00| 40| Closed| | 1|Unassigned| New|2/12/2019 11:00| 2/12/2020 0:00| 20| Open| | 1|Unassigned|Fix in progress|9/12/2019 11:00| 2/12/2020 0:00| 90| Open| | 3| Critical| Removed|5/17/2019 12:00| 2/12/2020 0:00| 33| Closed| | 3|Unassigned|Fix in progress|5/26/2019 10:00| 2/12/2020 0:00| 30| Open| | 3| Critical| New| 5/8/2019

问题 I wrote a function which returns a function based on the String argument it receives. The code looks as follow: def getFunction(str:String) : Map[String, String] => String={ val s = str.charAt(0).toString() s matches { case "x" => (arg:Map[String, String]) => arg("random") case _ =>(arg:Map[String, String]) => arg("") } } This is giving a compilation exception Description Resource Path Location Type missing parameter type for expanded function The argument types of an anonymous function must

问题 I wrote a function which returns a function based on the String argument it receives. The code looks as follow: def getFunction(str:String) : Map[String, String] => String={ val s = str.charAt(0).toString() s matches { case "x" => (arg:Map[String, String]) => arg("random") case _ =>(arg:Map[String, String]) => arg("") } } This is giving a compilation exception Description Resource Path Location Type missing parameter type for expanded function The argument types of an anonymous function must

问题 I have a Spark DataFrame , let's say 'df'. I do the following simple aggregation on this DataFrame : df.groupBy().sum() Upon doing so, I get the following exception: java.lang.IllegalArgumentException: requirement failed: Decimal precision 39 exceeds max precision 38 Is there any way I can fix this? My guess is, if I can decrease the decimal precision of all the columns of double type in df, it would solve the problem. 来源： https://stackoverflow.com/questions/46462377/change-decimal-precision

问题 I have a Spark DataFrame , let's say 'df'. I do the following simple aggregation on this DataFrame : df.groupBy().sum() Upon doing so, I get the following exception: java.lang.IllegalArgumentException: requirement failed: Decimal precision 39 exceeds max precision 38 Is there any way I can fix this? My guess is, if I can decrease the decimal precision of all the columns of double type in df, it would solve the problem. 来源： https://stackoverflow.com/questions/46462377/change-decimal-precision

问题 Didn't find the answer in these topics: first, second Have the next problem. I have a case class named Foo: case class Foo(a: Int, b: List[Int]) When I need to make an AST of this class I invoke Extraction.decompose(<instance of Foo>) and get an AST represenation of foo instance. But if I make field b as private case class Foo(a: Int, private val b: List[Int]) I get org.json4s.package$MappingException: Can't find ScalaSig for class java.lang.Object exception. This is only true for private