Ask Your Question
1

Doesn't sdc pipeline with cluster yarn streaming mode support pipeline level parameters?

asked 2018-07-16 02:24:22 -0600

casel.chen gravatar image

I set a pipeline parameter named KAFKA_BROKER_URI and reference it in my pipeline source component. It works fine in standalone mode but after switch to cluster yarn streaming mode, it reported error: Caused by: org.apache.kafka.common.config.ConfigException: Invalid url in bootstrap.servers: ${KAFKA_BROKER_URI}

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2018-07-18 21:14:27 -0600

metadaddy gravatar image

Runtime parameters in cluster mode are not yet supported, but this should be fixed by SDC-6371, which will be delivered in the upcoming StreamSets Data Collector 3.4.0 release.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-07-16 02:24:22 -0600

Seen: 23 times

Last updated: Jul 18