Why my spark application fails when dataframe is referenced after transformation within a loop?

Question

I have a set of spark dataframe transforms which is similar to the below code

df.cache()
for i in [1,2,3]:
  tempdf = df.select([\'key\'])
  df = df.join(tempdf,o