How to apply lag function on streaming dataframe?

二次信任 提交于 2019-12-10 19:36:11

问题


I have a streaming dataframe having three columns time,col1,col2. I have to apply lag function on column2. I have tried following query.

val w = org.apache.spark.sql.expressions.Window.oderBy("time")
df.select(col("time"),col("col1"),lag("col3",1).over(w))

But It gives following exception

org.apache.spark.sql.AnalysisException: Non-time-based windows are not supported on streaming DataFrames/Datasets

How can i achieve this?

Thanks in advance.

来源:https://stackoverflow.com/questions/46036845/how-to-apply-lag-function-on-streaming-dataframe

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!