“can't existentially abstract over parameterized type…”

Question

I was messing around with Scala 2.8 for fun and trying to define a pimp which adds an "as" method to type constructors, allowing to convert from one functor to another (please overlook the fact that I'm not necessarily dealing with functors here). So for instance, you could use it like this:

val array:Array[T]
val list:List[T] = array.as[List]

So here's what I tried to do:

object Test {
    abstract class NatTrans[F[_], G[_]] {
        def convert[T](f:F[T]):G[T]
    }

    implicit def array2List:NatTrans[Array, List] = new NatTrans[Array, List] { 
        def convert[T](a:Array[T]) = a.toList
    }

    // this next part gets flagged with an error
    implicit def naturalTransformations[T, F[_]](f:F[T]) = new {
        def as[G[_]](implicit n:NatTrans[F, G]) = n convert f
    }
}

however the definition of naturalTransformations is flagged with the error "can't existentially abstract over parameterized type G[T]". To fix this, I can rewrite naturalTransformations along with an additional class Transformable like so:

class Transformable[T, F[_]](f:F[T]) {
    def as[G[_]](implicit n:NatTrans[F, G]) = n convert f
}

implicit def naturalTransformations[T, F[_]](f:F[T]) = new Transformable[T, F](f)

and it appears to work. But it seems like my first attempt should've been equivalent, so I'm curious why it failed and what the error message means.

Answer 1

My hunch would be that this is because due to the following statements in the spec, § 6.11 , blocks:

A locally defined type definition type t = T is bound by the existential clause type t >: T <: T . It is an error if t carries type parameters.

And a structural instance creation expression is evaluated to a block, so

new {def greet{println("hello")}}

is a shorthand for

{ class anon$X extends AnyRef{ def greet = println("hello") }; new anon$X }

so it evaluates to a block expression (according to § 6.10 of the spec), with the aforementioned restriction. Why this restriction is there I do not know, however. The error that is thrown can be found in the Typers class at this location, which seems to confirm that this restriction is the cause of the error that you see. As you mentioned, encoding the function in a class removes the block expression restriction:

scala> class N[M[_]]
defined class N

scala> class Q { def as[M[_]](n:N[M]) = null}
defined class Q

scala> new { def as[M[_]](n:N[M]) = null}       
:7: error: can't existentially abstract over parameterized type M
       new { def as[M[_]](n:N[M]) = null}

Answer 2

To me this sounds like a simplicity against generality case: there could be a new type variable generated every time a block is created capturing some type constructor instantiated with existential type, but that would make error diagnostics more difficult to understand.

Also note that having a class turns the call into a fast INVOKEVIRTUAL, rather than invoking as() method by reflection.