403 code from ${vault:read()}
I'm working on integrating my SDC installation with HashiCorp Vault, and am having trouble getting SDC to authenticate.
- SDC: 2.3, installed using Cloudera manager
- For reference, here are the guides I've been using in the SDC docs and the Vault docs.
What I did:
- Enabled the app-id vault backend with
vault auth-enable app-id
- Created
vault write auth/app-id/map/app-id/streamsets-dev value=<myPolicy> display_name=streamsets-dev
. The policy in use provides read/write access tosecret/*
- Got the SDC user id with
bin/streamsets show-vault-id
- Linked the
streamsets-dev
vault app with the SDC user idvault write auth/app-id/map/user-id/streamsets-dev value=<mySDCGeneratedVaultUserId>
- Configured SDC properties
vault.addr
,vault.app.id
,vault.ssl.truststore.file
,vault.ssl.truststore.password
, and I set the various timeouts to very generous figures. - Executed
vault write secret/mySecretThing password=a1b2v3d4e5
- Finally I restarted SDC
At this point, I expected the ${vault:read('secret/mySecretThing', 'password')} function to read the secret and return a1b2v3d4e5
. However, I'm getting a HTTP 403
from Vault within SDC.
The fact that I'm getting a 403 would indicate that the vault.addr
and other configs took effect. I'm able to verify that the SSL setup from SDC to Vault is taking place (from trolling network logs, and again from the 403).
I'm also able to manually authenticate from the SDC machine like this:
curl -XPOST "https://my-vault-server:8200/v1/auth/app-id/login" -d '{"app_id":"streamsets-dev", "user_id":"< mySDCGeneratedVaultUserId >"}' --cacert <PEM version of the same ca I gave SDC>
, which returns a JSON response, including my new client token.
Appreciate any ideas or troubleshooting tips to get Vault and SDC talking!