Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

how to load delta delta in postgres

Hello , Firstly Im new in Streamsets. I couldnt find exact scenario on internet. My scneario is that I would like to load delta from one schema in postgres to another schema in the same postgres db. I dont want full load. instead , incremental load with update or delete.

Up to now I tried jdbc query consumer as origin but I couldnt succeed.

thanks in advance

how to load delta delta in postgres

Hello , Firstly Im new in Streamsets. I couldnt find exact scenario on internet. My scneario is that I would like to load delta from one schema in postgres to another schema in the same postgres db. I dont want full load. instead , incremental load with update or delete.

Up to now I tried jdbc query consumer as origin but I couldnt succeed.

In my scenario, I applied a solution. It is ; Origin : JDBC Query Consuemer : select * from postgreg_schema01.view_a Destination : JDBC Query : insert into pgostgre_schema02.table_a on conflict do update set... (upsert) But nearly I have to create 20 different pipeline for 20 different views Is there any way to create one pipeline instead of 20 different pipeline to solve this scenario?

thanks in advance

how to load delta delta in postgres

Hello , Firstly Im new in Streamsets. I couldnt find exact scenario on internet. My scneario is that I would like to load delta from one schema in postgres to another schema in the same postgres db. I dont want full load. instead , incremental load with update or delete.

Up to now I tried jdbc query consumer as origin but I couldnt succeed.

In my scenario, I applied a solution. It is ; Origin : JDBC Query Consuemer : select * from postgreg_schema01.view_a Destination : JDBC Query : insert into pgostgre_schema02.table_a on conflict do update set... (upsert) But nearly I have to create 20 different pipeline for 20 different views Is there any way to create one pipeline instead of 20 different pipeline to solve this scenario?

thanks in advance

how How to run multiple pipelines to load delta delta in postgres

Hello , Firstly Im new in Streamsets. I couldnt find exact My scenario on internet. My scneario is that I would like to load delta from one schema in postgres to another schema in the same postgres db. I dont don't want to do a full load. instead , Instead, an incremental load with update or delete.

Up to now The solution I tried jdbc query consumer as origin but I couldnt succeed.found is:

In my scenario,

    I applied a solution. It is ;
  • Origin : JDBC Query Consuemer Consumer : select * from postgreg_schema01.view_a postgreg_schema01.view_a
  • Destination : JDBC Query Executor : insert into pgostgre_schema02.table_a postgres_schema02.table_a on conflict do update set... (upsert) set... (upsert)

But nearly I have to create 20 different pipeline for 20 different views views. Is there any way to create one pipeline instead of 20 different pipeline to solve this scenario?

thanks in advance