Ask Your Question

Ubuntu User setting on MapR

asked 2018-01-09 05:52:36 -0500

davidha gravatar image

Hello, I installed Streamsets on Ubuntu 14.0LTS. I am trying to pipe data to MapR FS destination. The MapR prerequisite seem to be okay since I am able to find MapR FS origin and Destination in Streamsets and I am able to use Streamsets to pipe data to MapR FS as well. The problem I encounter is that I can only pipe data to directory with 777 permission, for instance, the default /tmp/ directory works fine. For other directories created by "mapr" user, with 755 permission, write permission denied error pop up.

I tried to change the Hadoop FS User parameter to "mapr", seems no significant difference. To make the directory works, I need to change the permission to 777 again. So I would like to know about how the user setting for this use case works?

Another information is that we are starting sdc with "usera" manually (usera$:bin/streamsets dc). So I suspect that the actual pipeline will be executed by "usera" making me failed to pipe data to MapRFS directory created by "mapr" user. However, even I change the MapRFS directory owner "usera", I still got the error. Is there anything I missed or can check ?


edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted

answered 2018-03-15 13:14:45 -0500

rupal gravatar image

The property for HDFS user configuration in the stages is applicable if you've impersonation setup and configured correctly for the user that's starting StreamSets Data Collector service. Please refer to MapR docs to understand how to setup impersonation. You also need to ensure the parent directory (top level dir)also has write permissions for the user.

edit flag offensive delete link more

answered 2018-01-10 22:58:04 -0500

Mufy gravatar image

Keeping SDC aside, what's your observation if you were to try and copy/move a file to the same SDC MapR-FS destination that complaints with the error? Somehow, the behavior rings a bell that I have come across with MapR in the past - with respect to permissions/ACLs not set right on the target object.

Do you have any ACEs applied on the destination on MapR-FS? What do they look like? Please share a sample - both regular permissions and the ACEs.

edit flag offensive delete link more


It happens if I use sudo -u mapr bin/streamsets dc to start services, the I will be able to write data to the destination, so the current situration feels like the Hadoop FS user parameter does not make a difference. The ACE is just as default(u:mapr) for the volume and permission is 755 mapr:mapr

davidha gravatar imagedavidha ( 2018-01-11 19:50:22 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-01-09 05:52:36 -0500

Seen: 131 times

Last updated: Mar 15 '18