Ask Your Question

Doesn't sdc pipeline with cluster yarn streaming mode support pipeline level parameters?

asked 2018-07-16 02:24:22 -0500

casel.chen gravatar image

I set a pipeline parameter named KAFKA_BROKER_URI and reference it in my pipeline source component. It works fine in standalone mode but after switch to cluster yarn streaming mode, it reported error: Caused by: org.apache.kafka.common.config.ConfigException: Invalid url in bootstrap.servers: ${KAFKA_BROKER_URI}

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-07-18 21:14:27 -0500

metadaddy gravatar image

Runtime parameters in cluster mode are not yet supported, but this should be fixed by SDC-6371, which will be delivered in the upcoming StreamSets Data Collector 3.4.0 release.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-07-16 02:24:22 -0500

Seen: 42 times

Last updated: Jul 18 '18