Ask Your Question
1

MapR Stream Producer Error, could not get TopicInfo, err 13

asked 2017-09-11 10:37:55 -0500

csmykay gravatar image

Hello StreamSet's experts. I am trying to configure a Mapr Stream Producer. The goal is to take the contents of a file and dump it into a stream. But when I preview it I get this error:

"Error getting metadata for topic '/scada/vendor1:scada_vendor1' from MapR Streams. Reason: org.apache.kafka.common.errors.UnknownTopicOrPartitionException: could not get TopicInfo, err 13"

I have a 3 node MapR cluster that is SEPERATE from the StreamSets server. They can all talk to each other. I don't see anywhere to specify my MapR Streams REST hostnames or ports. I ran the mapr setup for the streamsets binary directoy with no errors listed here: https://streamsets.com/documentation/...

Also the stream and topic have been created on the MapR cluster:

ubuntu@ip-172-16-1-30:/mapr/csmykay-us-east-1.cluster.com/scada$ ll vendor1 lr-------- 1 mapr mapr 2 Sep 11 04:10 vendor1 -> mapr::table::2113.50.262370

mapr@ip-172-16-1-30:~$ maprcli stream topic list -path /scada/vendor1 partitions maxlag logicalsize topic consumers physicalsize 1 0 0 scada_vendor1 0 0

This is the full error:

"Error getting metadata for topic '/scada/vendor1:scada_vendor1' from MapR Streams. Reason: org.apache.kafka.common.errors.UnknownTopicOrPartitionException: could not get TopicInfo, err 13: conf.topic"

I feel like this external StreamSets SDC server does not know how to communicate with MapR Streams but can't find any additional topics on how to configure it.

edit retag flag offensive close merge delete

3 Answers

Sort by ยป oldest newest most voted
1

answered 2017-09-21 22:25:54 -0500

mstang gravatar image

Did you add the mapr-kafka package to the streamsets server? (you need this for it to work)

Can you do hadoop fs -ls /scada/vendor1 from the streamsets server?

Does the user streamsets is running as have permissions to write to the stream the topic is in and the topic?

Some other issues could arise if you're not using a common authentication server and your uid's don't match...

edit flag offensive delete link more
1

answered 2017-09-12 22:24:28 -0500

Mufy gravatar image

Hello Chad :-). Nice to bump into you here.

Will you please export your SDC pipeline configuration and attach it here so that I can have a look at how it is that you have set it up? It'd also help if you could send across the SDC logs (after enabling DEBUG and running the pipeline again) to review the resultant stack as well.

Please let me know if you need any help in getting the above the diagnostics.

edit flag offensive delete link more
0

answered 2018-04-26 16:06:38 -0500

junko_urata gravatar image

Hi, StreamSets needs to run one of the MapR nodes.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-09-11 10:37:55 -0500

Seen: 165 times

Last updated: Apr 26