implicits

I'm new to Spark 2.0 and using datasets in our code base. I'm kinda noticing that I need to import spark.implicits._ everywhere in our code. For example: File A class A { def job(spark: SparkSession) = { import spark.implcits._ //create dataset ds val b = new B(spark) b.doSomething(ds) doSomething(ds) } private def doSomething(ds: Dataset[Foo], spark: SparkSession) = { import spark.implicits._ ds.map(e => 1) } } File B class B(spark: SparkSession) { def doSomething(ds: Dataset[Foo]) = { import spark.implicits._ ds.map(e => "SomeString") } } What I wanted to ask is if there's a cleaner way to

I used =:= as example type lambda for purpose of making simple minimal example. =:= type take two arguments, I'd like to curry one at type level. I take naive implementation type Curry[G] = {type l[L] = L =:= G} but in practical uses it causes errors: type X = Int type Y = Int type CurryInt[T] = T =:= Int type Curry[G] = {type l[L] = L =:= G} type CurrStatic = {type l[L] = L =:= Int} object CurryObj {type l[L] = L =:= Int} trait Apply[P[_], T] implicit def liftApply[P[_], T](implicit ev : P[T]) = new Apply[P,T] {} implicitly[Apply[CurryInt, Y]] // ok implicitly[Apply[Curry[X]#l, Y]] // fails

Scala implicits are very powerfull. I'm curious if they are a new/unique feature of Scala, or the concept already existed in other programming languages. Thanks. EDIT : To clarify my question, yes, I'm talking about this concrete implementation. Having "implicit things" all around seemed strange at first, but having used it for a while and seeing how others use it, I'm impressed by how well it works. Looks like the inspiration was Haskell's type classes. At least one blog article claims that implicits have their origin in Haskell type classes ; the article refers to a 2006 paper by Martin

Having a background in Haskell I am currently trying to get familiar with Scala. I encountered some problems trying to translate a small, extensible expression language from Haskell into Scala. The underlying issue of writing a data type that is extensible with both new data-variants and operations is commonly known as the expression problem . My original solution in Haskell uses type classes and instance declarations with constraints. The base of my expression is defined as follows: module Expr where class Expr e where eval :: e -> Integer data Lit = Lit Integer instance Expr Lit where eval

Please pardon the length of this question. I often need to create some contextual information at one layer of my code, and consume that information elsewhere. I generally find myself using implicit parameters: def foo(params)(implicit cx: MyContextType) = ... implicit val context = makeContext() foo(params) This works, but requires the implicit parameter to be passed around a lot, polluting the method signatures of layer after layout of intervening functions, even if they don't care about it themselves. def foo(params)(implicit cx: MyContextType) = ... bar() ... def bar(params)(implicit cx:

This question already has an answer here: Enforce type difference 7 answers Is there any way to constraint a method so that it only makes sense if two types are proved not to be equal? trait Something[A, B] { // I can only be called if type A is the same as type B def ifEqual(implicit ev: A =:= B) // Now I cannot be called if type A is proven to be the same as type B def ifNotEqual(implicit ev: A ??? B) } Yes. From shapeless , // Type inequalities trait =:!=[A, B] implicit def neq[A, B] : A =:!= B = new =:!=[A, B] {} implicit def neqAmbig1[A] : A =:!= A = ??? implicit def neqAmbig2[A] : A =:!=