Ask Your Question

Timestamp string TO epoch with milliseconds

asked 2018-10-17 17:40:00 -0500

supahcraig gravatar image

My incoming data is formatted as such: dd-MMM-yyyy HH:mm:ss.SSS. My intermediate destination is an external table in hadoop, but I need to preserve the milliseconds.

My initial thought is to simply convert that date to epoch time (in milliseconds) which would actually make using the data much easier for my particular problem. Unfortunately I can't figure out how to do that. I'm able to convert the input string field into a date, but it loses the milliseconds.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2019-05-02 11:27:49 -0500

meash-nrel gravatar image

You can convert an incoming timestamp string to DATETIME with Field Type Converter, and then on to milliseconds epoch with Expression Evaluator. BTW, DATETIME field isn't losing the ms, it just doesn't display them in its default format string.

Field Type Converter: inject a new field "timestamp" converting an existing string value to DATETIME, and input the custom datetime format by picking Date Fromat = Other and entering it in the new Other Date Format property.

Expression Evaluator: in Field Expressions, convert the new "timestamp" field into epoch ms.

/timestamp = ${time:dateTimeToMilliseconds(record:value('/timestamp'))}

You'll see then that the epoch ms isn't ending in "000" (on the second) -- it's preserving the ms internally.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-10-17 17:40:00 -0500

Seen: 800 times

Last updated: May 02 '19