Ask Your Question
1

Can JDBC Multitable origin write to multiple Kudu destinations?

asked 2017-10-20 09:51:07 -0600

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

I have a pipeline which uses the JDBC Multitable origin. Currently this origin pulls 5 SQL tables. I need to send the data from each SQL table to 5 different Kudu tables. Can I use the stream selector to build 5 different conditions to write out to 5 Kudu destinations? If so, what would the syntax look like for the stream selector conditions?

edit retag flag offensive close merge delete

Comments

Did you run into any issues with column ordering with Kudu? I have to manually order the fields for it to map in the correct order in Kudu. Did you run into any issues with column ordering?

andytroiano gravatar imageandytroiano ( 2017-10-31 11:16:51 -0600 )edit

Ignore this, I didn't have any issues with the column mapping with multiple tables. It seems to have mapped the columns correctly.

andytroiano gravatar imageandytroiano ( 2017-10-31 11:56:52 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2017-10-20 09:59:46 -0600

jeff gravatar image

updated 2017-10-20 11:28:35 -0600

I believe the Kudu destination supports EL expressions, including record functions. That means you should not need a stream selector. Try to configure the Kudu Table Name field with an expression like ${record:attribute('jdbc.tables')}. This is the default SDC attribute for the table name from which a record was generated out of the JDBC origin.

edit flag offensive delete link more

Comments

That worked. Thanks for the tip!

jshivers gravatar imagejshivers ( 2017-10-20 11:24:33 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-10-20 09:51:07 -0600

Seen: 253 times

Last updated: Oct 20 '17