Ask Your Question

403 code from ${vault:read()}

asked 2017-05-17 15:30:28 -0600

jwood gravatar image

I'm working on integrating my SDC installation with HashiCorp Vault, and am having trouble getting SDC to authenticate.

  • SDC: 2.3, installed using Cloudera manager
  • For reference, here are the guides I've been using in the SDC docs and the Vault docs.

What I did:

  1. Enabled the app-id vault backend with vault auth-enable app-id
  2. Created vault write auth/app-id/map/app-id/streamsets-dev value=<myPolicy> display_name=streamsets-dev. The policy in use provides read/write access to secret/*
  3. Got the SDC user id with bin/streamsets show-vault-id
  4. Linked the streamsets-dev vault app with the SDC user id vault write auth/app-id/map/user-id/streamsets-dev value=<mySDCGeneratedVaultUserId>
  5. Configured SDC properties vault.addr,, vault.ssl.truststore.file, vault.ssl.truststore.password, and I set the various timeouts to very generous figures.
  6. Executed vault write secret/mySecretThing password=a1b2v3d4e5
  7. Finally I restarted SDC

At this point, I expected the ${vault:read('secret/mySecretThing', 'password')} function to read the secret and return a1b2v3d4e5. However, I'm getting a HTTP 403 from Vault within SDC.

The fact that I'm getting a 403 would indicate that the vault.addr and other configs took effect. I'm able to verify that the SSL setup from SDC to Vault is taking place (from trolling network logs, and again from the 403).

I'm also able to manually authenticate from the SDC machine like this: curl -XPOST "https://my-vault-server:8200/v1/auth/app-id/login" -d '{"app_id":"streamsets-dev", "user_id":"< mySDCGeneratedVaultUserId >"}' --cacert <PEM version of the same ca I gave SDC>, which returns a JSON response, including my new client token.

Appreciate any ideas or troubleshooting tips to get Vault and SDC talking!

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2017-05-18 16:44:25 -0600

jwood gravatar image

updated 2017-09-06 15:29:35 -0600

metadaddy gravatar image

After reviewing the Vault audit logs, it was clear that SDC was correctly handling the authentication process. In my case, the Vault policy applied to the streamsets-dev app-id did not provide access to the correct secret path.

If you're reading this question, check the Vault policy that applies to your SDC app-id!

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-05-17 15:30:28 -0600

Seen: 236 times

Last updated: Sep 06 '17