substring multiple characters from the last index of a pyspark string column using negative indexing

Question

Closely related to: Spark Dataframe column with last character of other column but I want to extract multiple characters from the -1 index.

---

I hav

Answer 1

This is how you use substring. Your position will be -3 and the length is 3.

pyspark.sql.functions.substring(str, pos, len)

You need to change your substring function call to:

from pyspark.sql.functions import substring
df.select(substring(df['number'], -3, 3), 'event_type').show(2)
#+------------------------+----------+
#|substring(number, -3, 3)|event_type|
#+------------------------+----------+
#|                     022|        11|
#|                     715|        11|
#+------------------------+----------+