Using Hashicorp Vault with D1 and K1

This guide describes how you can use Hashicorp Vault to manage the Kubernetes secrets used by D1 and K1.

Installing Vault

You will need to install Hashicorp Vault as well as the HashiCorp Vault CSI driver in your Kubernetes cluster. In order to install Vault, you can follow the guide here. When installing the Helm chart, make sure to set the following options:

• injector.enabled: false
• csi.enabled: true

Next, install the Vault CSI by following this guide. Finally, enabled Kubernetes authentication as described here.

Obtaining Secrets From Vault

In the default configuration of the Helm chart, both D1 and K1 obtain their configuration files from a Kubernetes secret. For improved security, you can instead mount them via the HashiCorp Vault CSI driver. We take D1 Storage as an example in the following, but the process is similar for D1 Generic and K1.

You will first need to create an appropriate configuration file for D1 Storage. See the D1 Storage user manual for the details.

Next, create an appropriate policy file. We recommend only giving read capabilities:

path "secret/data/storage-config" {
    capabilities = ["read"]
}

Copy the configuration and policy files to the Vault pod:

kubectl cp storage-config.toml <vault pod>:/tmp/storage-config.toml
kubectl cp storage-policy.hcl <vault pod>:/tmp/storage-policy.hcl

Create the secret, set the policy, and set up the service account authentication:

kubectl exec -it <vault pod> -- /bin/sh -c 'vault kv put -mount=secret storage-config config.toml=@/tmp/storage-config.toml'
kubectl exec -it <vault pod> -- /bin/sh -c 'vault policy write storage-policy /tmp/storage-policy.hcl'
kubectl exec -it <vault pod> -- /bin/sh -c 'vault write auth/kubernetes/role/storage bound_service_account_names=<service account> bound_service_account_namespaces=<service account namespace> policies=storage-policy ttl=20m'

Make sure to clean up the temporary files:

kubectl exec -it <vault pod> -- /bin/sh -c 'rm -f /tmp/storage-config.toml /tmp/storage-config.hcl'

Next, create the following Secret Provider Class in your cluster:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-storage
spec:
  provider: vault
  parameters:
    vaultAddress: "http://vault.default:8200"
    roleName: "storage"
    objects: |
      - objectName: "config.toml"
        secretPath: "secret/data/storage-config"
        secretKey: "config.toml"

Finally, when deploying the D1 Storage Helm chart set the following values:

config:
    enabled: false

  extraVolumes:
    - name: secrets-store-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "vault-storage"
  extraVolumeMounts:
    - name: secrets-store-inline
      mountPath: /config.toml
      subPath: config.toml

Upon startup, the D1 Storage pod will automatically retrieve the configuration file from Hashcorp Vault.