Ask Your Question
0

Salesforce Bulk API query failing via FORCE_37 - Exception reading XML Stream

asked 2019-09-20 08:54:05 -0500

avdsa gravatar image

updated 2019-09-20 14:48:59 -0500

metadaddy gravatar image

Hello,

I have created a pipeline template to ingest Salesforce objects into a Kafka Producer, and I have been experiencing a specific error when attempting to pull Salesforce data using the BULK API option:

ERROR An exception occurred while running the pipeline, com.streamsets.pipeline.api.StageException: FORCE_37 - Exception reading XML Stream

I have attached the error logs from the SDC specific to this pipeline. What is strange is this is not a consistent error. I previously ran this pipeline, pulling the same Salesforce object, and it successfully completed. This error seems to occur when I attempt to run multiple copies of the pipeline querying different Salesforce objects at the same time.

My goal is to be able to setup scheduled batch processes for our Salesforce environment, using the job template I have created and being able to pull multiple Salesforce objects at the same time.

Here is the stack trace of the error:

2019-09-20 00:00:09,296 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Job 750t0000001qUoEAAU in progress
2019-09-20 00:00:09,388 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Waiting 20000 milliseconds for job 750t0000001qUoEAAU
2019-09-20 00:00:29,388 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Job 750t0000001qUoEAAU in progress
2019-09-20 00:00:29,476 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Waiting 20000 milliseconds for job 750t0000001qUoEAAU
2019-09-20 00:00:49,476 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Job 750t0000001qUoEAAU in progress
2019-09-20 00:00:49,572 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Waiting 20000 milliseconds for job 750t0000001qUoEAAU
2019-09-20 00:01:09,572 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Job 750t0000001qUoEAAU in progress
2019-09-20 00:01:09,662 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Waiting 20000 milliseconds for job 750t0000001qUoEAAU
2019-09-20 00:01:29,662 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Job 750t0000001qUoEAAU in progress
2019-09-20 00:01:29,751 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Batch 751t0000003RVFrAAO completed
2019-09-20 00:01:29,837 [user:*10178717@dsa] [pipeline:SF Batch Upload - 2/SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa] [runner:] [thread:ProductionPipelineRunnable-SF Batch Upload__ac888908-5b4a-49dc-8732-fe8dfcb94de6__dsa-SF Batch Upload - 2] INFO  ForceSource - Query results: 752t0000001nmPV
2019-09-20 00:02:07,784 [user:*10178717@dsa] [pipeline:SF ...
(more)
edit retag flag offensive close merge delete

Comments

I don't see any attachments. Could you edit your question and add the stack trace?

metadaddy gravatar imagemetadaddy ( 2019-09-20 10:39:23 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2019-09-20 15:56:19 -0500

metadaddy gravatar image

Thanks for posting that output. Given the lack of activity between the Salesforce origin reading a batch of records at 00:02:24,718 and the next log line, at 00:02:52,083, it looks like the Kafka producer is waiting for the broker to respond, before pronouncing it dead and reconnecting. It could be that this is indicative of a wider network issue, hence the Connection reset error from the Salesforce origin, or it could be that the network connection to Salesforce times out while the pipeline is paused in the destination.

Hard to say what the exact root cause is, but I would check the Kafka logs around that time, and check for any network issues.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-09-20 08:54:05 -0500

Seen: 65 times

Last updated: Sep 20