How to fix AssertionError: assertion failed: unsafe symbol Unstable (child of <none>) in runtime reflection universe

拜拜、爱过 提交于 2019-12-04 05:21:56

问题


I am trying to filter a dataframe like below

'

 val industry_df = industry_df0.filter( col("classification_type").===("GICS") )

'

Me getting error

java.lang.AssertionError: assertion failed: unsafe symbol Unstable (child of <none>) in runtime reflection universe
    at scala.reflect.internal.Symbols$Symbol.<init>(Symbols.scala:205)
    at scala.reflect.internal.Symbols$TypeSymbol.<init>(Symbols.scala:3030)
    at scala.reflect.internal.Symbols$ClassSymbol.<init>(Symbols.scala:3222)

1) What am I doing wrong here ? How to fix it?

2) When I try to do $("classification_type").===("GICS")

I am getting $ not found error , what should I include to use $ on dataframe.

3) I am trying to use join query with spark-sql but it is not working , how to debug it ? Ofcourse this query LOGIC working on oracle db.

'
 val result_df =  sparkSession.sql("""
             select a.company_id, a.data_date, a.model_id, b.domicile_country_code, d.two_digit_iso_code, d.capiq_geo_name  
               from model_lnk a join company_filter_vals b on a.company_id=b.company_id 
               join reg_lnk c 
               join country d on c.ctry_geo_code=d.geo_code
               where  a.model_id in (3020,3030)
               and b.model_group_id in (select model_group_id from model where model_id in (3020,3030))
           """  )

         result_df.show(2)

'

来源:https://stackoverflow.com/questions/53727559/how-to-fix-assertionerror-assertion-failed-unsafe-symbol-unstable-child-of-n

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!