Ask Your Question
0

Mapr FS origin/dest. KerberosPrincipal exception

asked 2018-01-04 14:44:34 -0600

lampshadesdrifter gravatar image

updated 2018-01-04 15:20:47 -0600

I am trying to do a simple data move from a mapr fs origin to a mapr fs destination (this is not my use case, just doing this simple movement for testing purposes). When trying to validate this pipeline, the error message I see in the staging area is:

HADOOPFS_11 - Cannot connect to the filesystem. Check if the Hadoop FS location: 'maprfs:///mapr/mycluster.cluster.local' is valid or not: 'java.io.IOException: Provided Subject must contain a KerberosPrincipal

Tyring different variations of the hadoop fs URI field (eg. mfs:///mapr/mycluster.cluster.local, maprfs:///mycluster.cluster.local) does not seem to help. Looking at the logs after trying to validate, I see

2018-01-04 10:28:56,686     mfs2mfs/mapr2sqlserver850bfbf0-6dc0-4002-8d44-b73e33fcf9b3  INFO    Created source of type: com.streamsets.pipeline.stage.origin.maprfs.ClusterMapRFSSource@16978460    DClusterSourceOffsetCommitter   *admin      preview-pool-1-thread-3

2018-01-04 10:28:56,697     mfs2mfs/mapr2sqlserver850bfbf0-6dc0-4002-8d44-b73e33fcf9b3  INFO    Error connecting to FileSystem: java.io.IOException: Provided Subject must contain a KerberosPrincipal  ClusterHdfsSource   *admin      preview-pool-1-thread-3

java.io.IOException: Provided Subject must contain a KerberosPrincipal
....

2018-01-04 10:20:39,159     mfs2mfs/mapr2mapr850bfbf0-6dc0-4002-8d44-b73e33fcf9b3   INFO    Authentication Config:  ClusterHdfsSource   *admin      preview-pool-1-thread-3

2018-01-04 10:20:39,159     mfs2mfs/mapr2mapr850bfbf0-6dc0-4002-8d44-b73e33fcf9b3   ERROR   Issues: Issue[instance='MapRFS_01' service='null' group='HADOOP_FS' config='null' message='HADOOPFS_11 - Cannot connect to the filesystem. Check if the Hadoop FS location: 'maprfs:///mapr/mycluster.cluster.local' is valid or not: 'java.io.IOException: Provided Subject must contain a KerberosPrincipal'']    ClusterHdfsSource   *admin      preview-pool-1-thread-3

2018-01-04 10:20:39,169     mfs2mfs/mapr2mapr850bfbf0-6dc0-4002-8d44-b73e33fcf9b3   INFO    Validation Error: Failed to configure or connect to the 'maprfs:///mapr/mycluster.cluster.local' Hadoop file system: java.io.IOException: Provided Subject must contain a KerberosPrincipal     HdfsTargetConfigBean    *admin  0   preview-pool-1-thread-3

java.io.IOException: Provided Subject must contain a KerberosPrincipal
....

However, to my knowledge, the system is not running Keberos, so this error message is a bit confusing for me. Does anyone have any idea what is happening and how to fix it? Thanks.

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
0

answered 2018-01-06 00:48:28 -0600

Selva gravatar image

Hi,

Hope below video helps.

https://youtu.be/9-pwAs5AydU

Selva

edit flag offensive delete link more
0

answered 2018-01-05 10:18:59 -0600

rupal gravatar image

Looks like you're trying to connect to a secure MapR cluster with native authentication. Have you enabled this for StreamSets? https://streamsets.com/documentation/...

edit flag offensive delete link more

Comments

Uncommenting #export SDC_JAVA_OPTS="-Dmaprlogin.password.enabled=true ${SDC_JAVA_OPTS}" in the sdc environment variable file did not seem to help the problem (even when reinstalling and commenting this line before running the mapr setup script).

lampshadesdrifter gravatar imagelampshadesdrifter ( 2018-01-05 13:10:33 -0600 )edit

I'm assuming you're starting the service with 'systemctl' Can you add the following line (without quotes) to /usr/lib/systemd/system/sdc.service file and restart the service: "Environment=SDC_JAVA_OPTS=-Dmaprlogin.password.enabled=true"

rupal gravatar imagerupal ( 2018-01-07 22:57:02 -0600 )edit

Doing that, as well as several other things, were what ended up solving the problem. The video link by Selva incorporated these steps and others together.

lampshadesdrifter gravatar imagelampshadesdrifter ( 2018-01-08 14:15:54 -0600 )edit
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2018-01-04 14:44:34 -0600

Seen: 81 times

Last updated: Jan 05