Calculating duration by subtracting two datetime columns in string format

Question

I have a Spark Dataframe in that consists of a series of dates:

from pyspark.sql import SQLContext
from pyspark.sql import Row
from pyspark.sql.types import          


        

Answer 1

datediff(Column end, Column start)

Returns the number of days from start to end.

https://spark.apache.org/docs/1.6.2/api/java/org/apache/spark/sql/functions.html