How to put more than 10000 records into bigquery using streamsets when both origin and destination are bigquery?

asked 2018-09-07 05:34:17 -0500

madhura gravatar image

put more than 10000 records into bigquery using streamsets when both origin and destination are bigquery?

edit retag flag offensive close merge delete

Comments

Can you explain what you've already tried and how it worked out?

metadaddy gravatar imagemetadaddy ( 2018-09-07 10:13:22 -0500 )edit

I have pulled more than 10000 records from bigquery. I have used these records to do some processing. But when dumping records back into big query after processing, it only dumps 10000 records into big query & rest of the records go to error. How to loop streamset to dump next 10000 records & so on

madhura gravatar imagemadhura ( 2018-09-12 00:12:23 -0500 )edit