Skip to main content

Getting started

This guide shows you how to easily get started with D1 Storage.

Running D1 Storage locally with Docker Compose​

In this section, you will be guided through how to get started with D1 Storage in Docker. This guide will use gRPCurl to call the gRPC endpoints, but we also provide a Go client.

Requirements​

By default D1 Storage reads its configuration from the TOML file config.toml, but it is possible to provide another configuration file or to overwrite specific configuration options. For more information, see the Configuration section in the user manual.

In the following we show how to use the built in Standalone ID Provider for user management. D1 Storage also supports using an external OIDC provider as an authentication/authorization mechanism. See the User Manual for more details.

Step 1: Start a local Docker Compose instance​

Place the following Docker Compose file next to your config.toml. Note that we use S3 as the IO Provider in this example. For more details on how to configure various IO Providers see the User Manual.

services:
  # D1 Storage
  d1-service-storage:
    image: cybercryptio/d1-service-storage:<version>
    container_name: d1-service-storage
    environment:
      D1_IO_S3_URL: "http://minio:9000"
      D1_IO_S3_BUCKET: "objects"
      D1_IO_S3_ID: "storageid"
      D1_IO_S3_KEY: "storagekey"
    volumes:
      - ${PWD}/config.toml:/config.toml
    ports:
      - 9000:9000
    depends_on:
      minio-init:
        condition: service_completed_successfully
    tty: true

  # Object storage
  minio:
    image: minio/minio:RELEASE.RELEASE.2022-07-08T00-05-23Z
    container_name: minio
    volumes:
      - /data
    environment:
      MINIO_ROOT_USER: "storageid"
      MINIO_ROOT_PASSWORD: "storagekey"
    command: server /data
    healthcheck:
      test: ["CMD", "curl", "-f", "localhost:9000/minio/health/live"]
      timeout: 30s
      interval: 5s
      retries: 5

  minio-init:
    image: minio/mc:RELEASE.2022-07-06T14-54-36Z
    container_name: minio-init
    environment:
      MINIO_ROOT_USER: "storageid"
      MINIO_ROOT_PASSWORD: "storagekey"
      MINIO_DEFAULT_BUCKETS: "objects"
    entrypoint: |
      /bin/sh -c "
      /usr/bin/mc config host add --api s3v4 storage http://minio:9000 $${MINIO_ROOT_USER} $${MINIO_ROOT_PASSWORD};
      /usr/bin/mc mb storage/$${MINIO_DEFAULT_BUCKETS};
      /usr/bin/mc policy set public storage/$${MINIO_DEFAULT_BUCKETS};
      "
    depends_on:
      minio:
        condition: service_healthy

To start the service call

docker compose up --detach

You now have a complete D1 Storage instance inside Docker. The gRPC API is available on localhost:9000. When running the following command,

docker ps

the output will show that two containers, d1-service-storage and minio, are running.

Step 2: Bootstrap an initial user​

docker exec d1-service-storage /d1-service-storage create-user

This user is created without any scopes, since this user should only be used to create other users, and user management does not require any scopes. Optionally you can set an additional parameter to set the scopes for the initial user. For more details on scopes see the the Standalone User Management section in the user manual.

Note down the generated credentials: user_id and password from the output. They should be used to login to D1 Storage.

Step 3: Login as initial user​

Login as the initial user with the user_id and password using the LoginUser endpoint:

grpcurl -plaintext \
-d '{
      "user_id": <user_id>,
      "password": <password>
    }' \
localhost:9000 d1.authn.Authn.LoginUser

Output:

{
  "accessToken": <access token>,
  "expiryTime": "1653039503"
}

When logging in, an access token is returned. The user needs to pass this token as gRPC metadata in all future calls in order to be authorized.

Step 4: Create a second user​

The initial user can create a second user:

grpcurl -plaintext -H "authorization: bearer <access token>" \
-d '{
      "scopes": ["READ", "CREATE", "UPDATE", "DELETE", "GETACCESS", "MODIFYACCESS"]
}' \
localhost:9000 d1.authn.Authn.CreateUser

Output:

{
  "user_id": "44c8fa82-f8ed-46b0-94d1-8921a19c0d62",
  "password": "Vju86gvJTEKK9zBIZAHloa2K0y2Vw_eJC7icmmCP-jc"
}

Step 5: Use D1 Storage to store an object​

The new user can now store an object using the Store endpoint:

grpcurl -plaintext -H "authorization: bearer <access token>" \
-d '{
      "plaintext": "1234",
      "associated_data": "5678"
}' \
localhost:9000 d1.storage.Storage.Store

Output:

{
  "object_id": "a9a909b9-6019-488d-bfb4-b8b4243ae177"
}

The user needs to keep the object_id in order to be able to retrieve the object later.

Step 6: Shut down the D1 Storage instance​

The D1 Storage instance can be shut down by running:

docker compose down