Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 23218

Set up timezone for timezone for all nodes in databricks without init script

$
0
0

I have used this question to set up pyspark timezone using configuration of the cluster with

spark.sql.session.timeZone Asia/Hongkong

This, unfortunately does not set timezone for all nodes and therefore calling pythons datetime.datetime.now() will not include timezone set with spark session config.

Databricks is phasing out support of init scripts and therefore I cannot use it to set up timezone for all nodes.

How is it possible to do it during spark config?


Viewing all articles
Browse latest Browse all 23218

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>