Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Sub-query in Salesforce Origin

I have upgraded to Streamsets 3.4.0 and am now trying to use a subquery with the salesforce origin stage. Here is the query I would like to use:

"select Id, Name, (select NSB_Account__r.Pseudo_Natural_Key__c from Account_Roles__r) from Account where Name = 'NA' "

In Saleforce itself, I get results that look like this:

Id Name Account_Roles__r ** ** [{"Role_Status__c":"Primary","NSB_Account__r":{"attributes":{"type":"NSB_Account__c","url":"/services/data/v43.0/sobjects/NSB_Account__c/NA"},"Pseudo_Natural_Key__c":"NA"}}]

But in Streamsets, I get this error:

com.streamsets.datacollector.util.PipelineException: PREVIEW_0003 - Encountered error while previewing : java.lang.NullPointerException at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:249) at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer.lambda$start$0(AsyncPreviewer.java:95) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getAllReferences(SobjectRecordCreator.java:301) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:203) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:189) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.buildMetadataCacheFromQuery(SobjectRecordCreator.java:173) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.prepareQuery(ForceSource.java:401) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.soapProduce(ForceSource.java:656) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.produce(ForceSource.java:448) at com.streamsets.pipeline.api.base.configurablestage.DSource.produce(DSource.java:38) at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:283) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:235) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:298) at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:244) at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.runPollSource(PreviewPipelineRunner.java:314) at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.run(PreviewPipelineRunner.java:217) at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:538) at com.streamsets.datacollector.runner.preview.PreviewPipeline.run(PreviewPipeline.java:51) at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:218) ... 14 more Is there something I can do to make this query work?

Sub-query in Salesforce Origin

I have upgraded to Streamsets 3.4.0 and am now trying to use a subquery with the salesforce origin stage. Here is the query I would like to use:

"select Id, Name, (select NSB_Account__r.Pseudo_Natural_Key__c from Account_Roles__r) from Account where Name = 'NA' "

In Saleforce itself, I get results that look like this:

Id Name Account_Roles__r ** ** [{"Role_Status__c":"Primary","NSB_Account__r":{"attributes":{"type":"NSB_Account__c","url":"/services/data/v43.0/sobjects/NSB_Account__c/NA"},"Pseudo_Natural_Key__c":"NA"}}]

But in Streamsets, I get this error:

com.streamsets.datacollector.util.PipelineException: PREVIEW_0003 - Encountered error while previewing : java.lang.NullPointerException at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:249) at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer.lambda$start$0(AsyncPreviewer.java:95) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getAllReferences(SobjectRecordCreator.java:301) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:203) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:189) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.buildMetadataCacheFromQuery(SobjectRecordCreator.java:173) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.prepareQuery(ForceSource.java:401) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.soapProduce(ForceSource.java:656) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.produce(ForceSource.java:448) at com.streamsets.pipeline.api.base.configurablestage.DSource.produce(DSource.java:38) at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:283) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:235) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:298) at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:244) at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.runPollSource(PreviewPipelineRunner.java:314) at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.run(PreviewPipelineRunner.java:217) at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:538) at com.streamsets.datacollector.runner.preview.PreviewPipeline.run(PreviewPipeline.java:51) at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:218) ... 14 more more

Is there something I can do to make this query work?

Sub-query in Salesforce Origin

I have upgraded to Streamsets 3.4.0 and am now trying to use a subquery with the salesforce origin stage. Here is the query I would like to use:

"select Id, Name, (select NSB_Account__r.Pseudo_Natural_Key__c from Account_Roles__r) from Account where Name = 'NA' "

In Saleforce itself, I get results that look like this:

Id Name Account_Roles__r ** ** [{"Role_Status__c":"Primary","NSB_Account__r":{"attributes":{"type":"NSB_Account__c","url":"/services/data/v43.0/sobjects/NSB_Account__c/NA"},"Pseudo_Natural_Key__c":"NA"}}]

But in Streamsets, I get this error:

com.streamsets.datacollector.util.PipelineException: PREVIEW_0003 - Encountered error while previewing : java.lang.NullPointerException at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:249) at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer.lambda$start$0(AsyncPreviewer.java:95) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getAllReferences(SobjectRecordCreator.java:301) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:203) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:189) at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.buildMetadataCacheFromQuery(SobjectRecordCreator.java:173) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.prepareQuery(ForceSource.java:401) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.soapProduce(ForceSource.java:656) at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.produce(ForceSource.java:448) at com.streamsets.pipeline.api.base.configurablestage.DSource.produce(DSource.java:38) at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:283) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:235) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:298) at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:244) at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.runPollSource(PreviewPipelineRunner.java:314) at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.run(PreviewPipelineRunner.java:217) at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:538) at com.streamsets.datacollector.runner.preview.PreviewPipeline.run(PreviewPipeline.java:51) at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:218) ... 14 more

Is there something I can do to make this query work?

edit:

To clarify, it is "NSB_Account__r.Pseudo_Natural_Key__c" in the subquery that is causing the error. Everything else works just fine.

Sub-query in Salesforce Origin

I have upgraded to Streamsets 3.4.0 and am now trying to use a subquery with the salesforce origin stage. Here is the query I would like to use:

"select Id, Name, (select NSB_Account__r.Pseudo_Natural_Key__c from Account_Roles__r) from Account where Name = 'NA' "

In Saleforce itself, I get results that look like this:

Id Name Account_Roles__r ** ** [{"Role_Status__c":"Primary","NSB_Account__r":{"attributes":{"type":"NSB_Account__c","url":"/services/data/v43.0/sobjects/NSB_Account__c/NA"},"Pseudo_Natural_Key__c":"NA"}}]

But in Streamsets, I get this error:

com.streamsets.datacollector.util.PipelineException: PREVIEW_0003 - Encountered error while previewing : java.lang.NullPointerException
at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:249)
at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer.lambda$start$0(AsyncPreviewer.java:95)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getAllReferences(SobjectRecordCreator.java:301)
at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:203)
at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:189)
at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.buildMetadataCacheFromQuery(SobjectRecordCreator.java:173)
at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.prepareQuery(ForceSource.java:401)
at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.soapProduce(ForceSource.java:656)
at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.produce(ForceSource.java:448)
at com.streamsets.pipeline.api.base.configurablestage.DSource.produce(DSource.java:38)
at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:283)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:235)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:298)
at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:244)
at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.runPollSource(PreviewPipelineRunner.java:314)
at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.run(PreviewPipelineRunner.java:217)
at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:538)
at com.streamsets.datacollector.runner.preview.PreviewPipeline.run(PreviewPipeline.java:51)
at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:218)
... 14 more

more

Is there something I can do to make this query work?

edit:

To clarify, it is "NSB_Account__r.Pseudo_Natural_Key__c" in the subquery that is causing the error. Everything else works just fine.

Sub-query in Salesforce Origin

I have upgraded to Streamsets 3.4.0 and am now trying to use a subquery with the salesforce origin stage. Here is the query I would like to use:

"select Id, Name, (select NSB_Account__r.Pseudo_Natural_Key__c from Account_Roles__r) from Account where Name = 'NA' "

In Saleforce itself, I get results that look like this:

Id Name Account_Roles__r ** ** [{"Role_Status__c":"Primary","NSB_Account__r":{"attributes":{"type":"NSB_Account__c","url":"/services/data/v43.0/sobjects/NSB_Account__c/NA"},"Pseudo_Natural_Key__c":"NA"}}]

But in Streamsets, I get this error:

com.streamsets.datacollector.util.PipelineException: PREVIEW_0003 - Encountered error while previewing : java.lang.NullPointerException
at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:249)
at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer.lambda$start$0(AsyncPreviewer.java:95)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getAllReferences(SobjectRecordCreator.java:301)
at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:203)
at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.getReferencesFromFieldList(SobjectRecordCreator.java:189)
at com.streamsets.pipeline.lib.salesforce.SobjectRecordCreator.buildMetadataCacheFromQuery(SobjectRecordCreator.java:173)
at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.prepareQuery(ForceSource.java:401)
at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.soapProduce(ForceSource.java:656)
at com.streamsets.pipeline.stage.origin.salesforce.ForceSource.produce(ForceSource.java:448)
at com.streamsets.pipeline.api.base.configurablestage.DSource.produce(DSource.java:38)
at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:283)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:235)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:298)
at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:244)
at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.runPollSource(PreviewPipelineRunner.java:314)
at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.run(PreviewPipelineRunner.java:217)
at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:538)
at com.streamsets.datacollector.runner.preview.PreviewPipeline.run(PreviewPipeline.java:51)
at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:218)
... 14 more

Is there something I can do to make this query work?

edit:

To clarify, it is "NSB_Account__r.Pseudo_Natural_Key__c" in the subquery that is causing the error. Everything else works just fine.