Why is join not possible after show operator?

Question

The following code works fine until I add show after agg. Why is show not possible?

 val tempTableB = tableB.groupBy(\         


        

Answer 1

As @philantrovert has already answered with much detailed explanation. So I shall not explain.

What you can do if you want to see whats in tempTableB then you can do so after it has been assigned as below.

 val tempTableB = tableB.groupBy("idB")
  .agg(first("numB").as("numB")) 

 tempTableB.show

 tableA.join(tempTableB, $"idA" === $"idB", "inner")
 .drop("idA", "numA").show

It should work then