Ask Your Question

How can I order records in a JDBC query consumer

asked 2019-02-28 13:51:32 -0600

fvarg00 gravatar image

updated 2019-03-01 15:26:19 -0600

I select records using a JDBC query consumer and specifically gave an 'order by' clause.

I perform JDBC lookups and 'stream selector' stages downstream.

I tried writing the processed records to a FS and to Kafka, however find that the order I selected the records from, in the first stage(JDBC query consumer) is NOT the order I got writen to the FS or Kafka.

How do I ensure the ordering is preserved at the destination?

Thank you,

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2019-03-03 23:54:27 -0600

ak47 gravatar image

updated 2019-03-06 23:21:05 -0600

metadaddy gravatar image

Field Order is for ordering fields and does not impact ordering of records.

  1. StreamSelector will impact the ordering of records that end up at the destination
  2. Also writing to partitioned Kafka topic could impact ordering of records. Kafka only guarantees ordering within a partition and not across partitions.
edit flag offensive delete link more
Login/Signup to Answer

Question Tools



Asked: 2019-02-28 13:46:15 -0600

Seen: 346 times

Last updated: Mar 06 '19