Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Date value getting converted in motion

Pipeline reads a timestamp field from a S3 bucket and the value is '2020-01-01 20:00:00' and it's getting automatically converted into '2020-01-02 00:00:00' while in motion.

Before I apply any conversion or processor on the field value is changing. Let me know how to retain the original value from source.

I have the set the property spark.sql.session.timeZone=UTC