Spark Streaming: stateless overlapping windows vs. keeping state

Question

What would be some considerations for choosing stateless sliding-window operations (e.g. reduceByKeyAndWindow) vs. choosing to keep state (e.g. via updateStateByKey or the new m

Answer 1

I think one of other drawbacks of third approach is that the RDDs are not received chronologically..considering running them on a cluster..

ongoingEventsStream.foreachRDD { /*accumulate state in casssandra*/ }

also what about check-pointing and driver node failure..In that case do u read the whole data again? curious to know how you wanna handle this?

I guess maybe mapwithstate is a better approach why you consider all these scenario..