Ask Your Question

Convert date [string] to an ElasticSearch date field?

asked 2019-06-17 13:50:33 -0500

daveh gravatar image

updated 2019-06-17 13:51:11 -0500

Using the Elasticsearch 5.2.0 stage, with a /datetime field formated as 'yyyy-MM-dd HH:mm:ss.SSSZ', I'm trying to convert the /datetime field to an ElasticSearch date field so that I can have Kibana use that as the @timestamp field.

However, using the Field Converter and selecting either the DATETIME/DATE/TIME conversion, SDC sends the converted field as an epoch number which ES recognizes as a number rather than a date/time field

How can have the ElasticSearch stage send a datetime field to ES?

Thanks in advance.

edit retag flag offensive close merge delete


What format is ES expecting for its date fields?

metadaddy gravatar imagemetadaddy ( 2019-06-17 14:06:37 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2019-08-14 16:26:12 -0500

WebMenace gravatar image

updated 2019-08-14 16:30:52 -0500

Elasticsearch can consume just about every kind of standard date, or datetime strings.

datetime examples:

yyyy-MM-dd HH:mm:ss (2019-08-08 17:55:26)
yyyy-MM-dd HH:mm:ss.SSS (2019-08-08 17:55:26.472)
yyyy-MM-ddTHH:mm:ss.SSS (2019-08-08T17:55:26.472-07:00)

It uses Java JodaTime implementation under the hood too.

But right at this moment I'm struggling with the same problem.

No matter how I transform the field that I want to use for the 'Time Basis', the Elasticsearch designation stage will only deliver a unix epoch to elasticsearch.

I think something is messed up, or the documentation is missing a minor detail. (Currently trying to get the entire stack to build in Eclipse so I can debug)

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2019-06-17 13:50:33 -0500

Seen: 170 times

Last updated: Aug 14