Ask Your Question
1

How to make JDBC multitable origin fetch all the records of each table at once and in order?

asked 2018-01-29 22:39:25 -0500

Maithri gravatar image

updated 2018-02-11 22:16:36 -0500

Suppose I want to fetch records from table1,tabl2,table3. table1:

          name       id
          a              10
          b              20

table2:

         company    employee
            x                  xname
            y                  yname

table3:

            place           regionid
            xxx                  100
            yyy                  200

I used JDBC multitable orgin and specified these tables.Now ,I want to dump this data as:

    {name:a,id:10}
    {name:b,id:20}
    {company:x,employee:xname}
    {company:y,yname}
    {place:xxx,regionid:100}
    {place:yyy,regionid:200}
edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
1

answered 2018-01-31 00:02:00 -0500

Maithri gravatar image

Setting the JDBC Multitable Consumer configuration

jdbc:

            Per Batch Strategy:Process All Available Rows From The Table

Advanced:

             Transaction isolation:Read committed

             Initial Table Order Strategy:None

gave me the output as required.

https://streamsets.com/documentation/...

as mentioned in the above documentation.

edit flag offensive delete link more
1

answered 2018-01-30 09:24:11 -0500

Roh gravatar image

IMO multitable consumer is not meant for that, The main purpose of this is to read database data from multiple tables in the parallel approach.

If you want to one table after other, I don't have a straight solution but it's worth giving a try and playing with the below JDBC MULTITABLE CONSUMER

BATCH STRATAGY

Strategy to create each batch of data:

Switch Tables - Each thread creates a batch of data from one table, and then switches to the next available table to create the next batch. Define the Result Set Cache Size and the Batches from Result Set properties when you configure a switch tables strategy.

Process All Available Rows from the Table - Each thread creates multiple batches of data from one table, until all available rows are read from that table.

INITIAL TABLE ORDER STRATEGY

initial order used to read the tables: None - Reads the tables in the order that they are listed in the database. Alphabetical - Reads the tables in alphabetical order. Referential Constraints - Reads the tables based on the dependencies between the tables.

edit flag offensive delete link more

Comments

Thank you for the response @Roh

Maithri gravatar imageMaithri ( 2018-01-30 22:16:29 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-01-29 22:39:25 -0500

Seen: 355 times

Last updated: Feb 11