Apache Spark - how to set timezone to UTC? currently defaulted to Zulu

Question

In Spark\'s WebUI (port 8080) and on the environment tab there is a setting of the below:

user.timezone Zulu

Do you know how/where I can override this to U

Answer 1

As described in these SPARK bug reports (link, link), the most current SPARK versions (3.0.0 and 2.4.6 at time of writing) do not fully/correctly support setting the timezone for all operations, despite the answers by @Moemars and @Daniel.

I suggest avoiding time operations in SPARK as much as possible, and either perform them yourself after extraction from SPARK or by using UDFs, as used in this question.