Ask Your Question

How to use maprFS metadata executor to change ownership permissions?

asked 2018-03-14 21:13:54 -0500

lampshadesdrifter gravatar image

updated 2018-03-15 15:00:54 -0500

Trying to set up a maprFS event executor to change maprFS file metadata after a pipeline has moved files from a local directory to a maprFS destination.

I am using a maprFS metadata executor connected to the event output of a maprFS destination and trying to change the ownership permissions of the files after they are written, but am getting the error

HDFS_METADATA_000 - Error when applying metadata changes: Could not set owner/group sdcuser/<some other="" group=""> for path <destination path="">

Which actually makes a bit of sense since non-root linux users can't just "give away" their files.

My question then is, when would this feature actually ever be used? It seems as though the underlying actions would be something like sudo chown <new user>:<new group> <file path>, which would require a password to be entered. I seem to be misunderstanding something about when and how the change-ownership features of the executor would be used. Could someone please educate me a bit about this (as the docs don't seem to specify any special permission requirements ([


It appears that, to use the ownership editing feature of the executor, I'd need to ensure the user has permissions to perform the operations in the Metadata Executor or specify a user in the HDFS User configuration that actions can be executed on behalf of the specified user (which would require MapR impersonation).

But I don't know how exactly I would change permissions so that the sdc process owner (say sdcuser) could change the ownership permissions. I have already set up hadoop impersonation as per the streamsets docs and it works fine, but I don't know what to change about the sdcuser's permissions on the MapR volume in question (which is what I am assuming needs to be done). Giving the sdcuser's (secondary) group (which is the mapr group) the admin ability to change the volume's access control settings does not seem to do anything when testing via hadoop fs -R -chown mapr:mapr /path/to/sdcusers/written/files. Still seeing the error message

chown: changing ownership of '/path/to/sdcusers/written/files': Could not set owner/group mapr/mapr for path maprfs:///path/to/sdcusers/written/files

Is there something else that needs to be changed?

edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted

answered 2018-03-15 13:23:28 -0500

rupal gravatar image

Pipelines in Data Collector, will by default execute as the user that started the data collector service. Ensure this user has permissions to perform the operations in the Metadata Executor. The other option is to specify a user in the HDFS User configuration so that actions can be executed on behalf of the specified user. For this, ensure you've setup impersonation correctly with MapR.

edit flag offensive delete link more


Please see the original question which I have updated in response to this.

lampshadesdrifter gravatar imagelampshadesdrifter ( 2018-03-15 15:01:00 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-03-14 21:13:54 -0500

Seen: 220 times

Last updated: Mar 15 '18