Ask Your Question
0

RPC: How output is written

asked 2018-07-17 08:16:18 -0600

anonymous user

Anonymous

I'm using a SDC RPC and I have a question about its working.

Since I'm reading the data source by batch of 1000 records, does the RPC write its output by batch of 1000 records or is it writing record by record ? I guess it use a batch but since it's quite slow, I would like to be sure.

Thanks,

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2018-07-18 20:51:20 -0600

metadaddy gravatar image

The SDC RPC Destination writes an entire batch of records per HTTP request. The SDC RPC format includes a lot of metadata, so it is slower than other formats. You might want to use JSON or Avro data format if performance is an issue.

edit flag offensive delete link more

Comments

How can I use Avro to send data from one sdc to another ? Should I replace SDC RPC ?

youa gravatar imageyoua ( 2018-07-19 04:56:39 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-07-17 08:16:18 -0600

Seen: 32 times

Last updated: Jul 18