Ask Your Question

how the system create the timestamp offset value

asked 2018-08-09 21:21:50 -0500

Yosen gravatar image

updated 2018-08-09 21:42:27 -0500

Hi,expert: We used the jdbc query origin,and use the incremental mode. The offset is timestamp. But the offset value is very strange. I don't know how this value is created. please see the log:

2018-08-10 10:16:26,050 oracle_to_es_with_timestamp/oracletoeswithtimestamp081ed04a-2c0a-4aeb-8955-941b331f2b1c DEBUG Saving offset {$com.streamsets.datacollector.pollsource.offset$=2018-08-09 08:25:34.461} for pipeline oracletoeswithtimestamp081ed04a-2c0a-4aeb-8955-941b331f2b1c ProductionSourceOffsetTracker *admin ProductionPipelineRunnable-oracletoeswithtimestamp081ed04a-2c0a-4aeb-8955-941b331f2b1c-oracle_to_es_with_timestam

Now the first time is our server time, but how the system get the offset value?

Please give us some help ,where is the offset value come from. thanks!

Addition: And if I stop the pipeline and reset it, then run again, I found the offset value is earlier than the last run time, please see the history log:

Aug 9, 2018 8:26:36 PM admin STOPPED The pipeline was stopped. The last committed source offset is '2018-08-09 08:08:54.566'. View Summary... Aug 9, 2018 8:26:33 PM admin STOPPING
Aug 9, 2018 8:24:26 PM admin RUNNING
Aug 9, 2018 8:24:25 PM admin STARTING
Aug 9, 2018 8:22:54 PM admin STOPPED The pipeline was stopped. The last committed source offset is '2018-08-09 08:25:34.461'. View Summary...

Now, we can see that the at Aug 9, 2018 8:22:54(The ui timezone is not as same as server timezone, so the time is different) , we stop the pipeline and the offset value is 2018-08-09 08:25:34.461. Then I reset and start pipeline again, and after few minutes, At Aug 9, 2018 8:26:36 I stop it. The offset is 2018-08-09 08:08:54.566. You can see the offset value(2018-08-09 08:08:54.566) is earlier than the last one(2018-08-09 08:25:34.461).

Sorry, I can't upload file, when I upload a file or a image, there is an error : >10 points required to upload files. So, I must copy the log content.

My question is how the system create the offset value of the timestamp? It not a server system time?


edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-08-13 19:44:25 -0500

Yosen gravatar image

Who can help me?

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-08-09 21:21:50 -0500

Seen: 23 times

Last updated: Aug 13