Yarkon Server with a Compatible S3 Provider

This document will walk you through the steps required to set up Yarkon Server with an S3 compatible storage provider, such as CEPH or Wasabi. The document highlights the differences in the set up compared to the generic AWS S3 set up. Make sure to review the Getting Started manual before proceeding with this document.

Depending on the capabilities of your storage provider, some of the features available to AWS S3 users may not be available when using a compatible provider.

Provider Details

The first step is to set up the proper connection information for your provider. In Yarkon Server, this is handled through ENV variables provided to the server.

Set up in EC2

When using an EC2 instance of Yarkon Server, the recommended method to pass ENV variables to the application is using the configuration file used by the pm2 process manager which is running the application. In a standard install of Yarkon Server, this file is named yarkon-server.pm2.json and is located in the folder /var/app/current. Make sure to restart the server after changing this file.

The following sample shows how to update the config file for Wasabi (just replace the sample access key with yours):

{
    "apps": [{
        "name": "aphek",
        "script": "./aphek",
        "watch": false,
        "env": {
            "PORT": 80,
            "PROVIDER_END_POINT": "https://s3.wasabisys.com",
            "PROVIDER_STS_END_POINT": "https://iam.wasabisys.com",
            "PROVIDER_IAM_END_POINT": "https://iam.wasabisys.com",
            "PROVIDER_NAME": "Wasabi",
            "PROVIDER_IMAGE": "data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxMzciIGhlaWdodD0iMTM2IiB2aWV3Qm94PSIwIDAgMTM3IDEzNiI+CiAgPHBhdGggZmlsbD0iIzAzQzczNiIgZmlsbC1ydWxlPSJldmVub2RkIiBkPSJNMTAwLjc3NjQsMTEzLjMzNSBMMTAwLjc3NjQsOTUuNzUzIEwxMTAuNDY5NCwxMDQuNTQxIEMxMDcuNTk1NCwxMDcuODM1IDEwNC4zNDI0LDExMC43OSAxMDAuNzc2NCwxMTMuMzM1IEwxMDAuNzc2NCwxMTMuMzM1IFogTTYwLjE1NzQsMTIzLjAzMiBMODguNTk3NCw5Ni4xNDggTDg4LjU5NzQsMTE5LjkxMSBDODIuMzY1NCwxMjIuMzMgNzUuNTk0NCwxMjMuNjYgNjguNTE4NCwxMjMuNjYgQzY1LjY3NjQsMTIzLjY2IDYyLjg4NTQsMTIzLjQ0NSA2MC4xNTc0LDEyMy4wMzIgTDYwLjE1NzQsMTIzLjAzMiBaIE0xMy4xNjM0LDczLjg0IEw0My4wNDU0LDEwMC42MzcgTDczLjA4NzQsNzIuNDI1IEw4OC41OTc0LDU3Ljc2NCBMODguNTk3NCw3OS4zODkgTDQ2LjU0OTQsMTE5LjEzNyBDMjguNDU3NCwxMTEuMzM1IDE1LjI5NzQsOTQuMjA2IDEzLjE2MzQsNzMuODQgTDEzLjE2MzQsNzMuODQgWiBNMzYuNzg4NCwyMi4yOTYgTDM2LjgwNzQsMzkuOTY4IEwyNi45MDA0LDMxLjA4NCBDMjkuODM0NCwyNy43OCAzMy4xNTM0LDI0LjgyOCAzNi43ODg0LDIyLjI5NiBMMzYuNzg4NCwyMi4yOTYgWiBNMzYuODQ4NCw3OC43MjEgTDEzLjc1NjQsNTguMDE0IEMxNC44NTU0LDUxLjk3OCAxNi45MzE0LDQ2LjI3OSAxOS44MTI0LDQxLjA4NiBMMzYuODI0NCw1Ni4zNDIgTDM2Ljg0ODQsNzguNzIxIFogTTc3LjAyMTQsMTIuOTg5IEw2NC40NTc0LDI0Ljg2NSBMNDguOTg1NCwzOS41MzkgTDQ4Ljk2MTQsMTUuODkgQzU1LjA0ODQsMTMuNTk4IDYxLjYzODQsMTIuMzQgNjguNTE4NCwxMi4zNCBDNzEuNDA4NCwxMi4zNCA3NC4yNDg0LDEyLjU2MiA3Ny4wMjE0LDEyLjk4OSBMNzcuMDIxNCwxMi45ODkgWiBNMTIzLjMxNDQsNzcuODAxIEMxMjIuMjU2NCw4My43MjkgMTIwLjI1NzQsODkuMzM0IDExNy40Nzg0LDk0LjQ1NyBMMTAwLjc3NjQsNzkuMzE0IEwxMDAuNzc2NCw1Ny4zNjggTDEyMy4zMTQ0LDc3LjgwMSBaIE0xMjMuODM1NCw2MS44MzUgTDk0LjYwMDQsMzUuMzMgTDY0LjczNTQsNjMuNTYxIEw0OS4wMjY0LDc4LjMxMyBMNDkuMDAzNCw1Ni4zMDggTDcyLjgzMTQsMzMuNzA5IEw5MC42MDA0LDE2LjkxMiBDMTA4LjUzNzQsMjQuNjk1IDEyMS42MDE0LDQxLjY1NSAxMjMuODM1NCw2MS44MzUgTDEyMy44MzU0LDYxLjgzNSBaIE0xMTYuNDg3NCwyMC4wMyBDMTAzLjY3NDQsNy4yMTcgODYuNjM4NCwwLjE2MSA2OC41MTg0LDAuMTYxIEM1MC4zOTc0LDAuMTYxIDMzLjM2MTQsNy4yMTcgMjAuNTQ4NCwyMC4wMyBDNy43MzU0LDMyLjg0MyAwLjY3OTQsNDkuODc5IDAuNjc5NCw2OCBDMC42Nzk0LDg2LjEyMSA3LjczNTQsMTAzLjE1NyAyMC41NDg0LDExNS45NyBDMzMuMzYxNCwxMjguNzgzIDUwLjM5NzQsMTM1LjgzOSA2OC41MTg0LDEzNS44MzkgQzg2LjYzODQsMTM1LjgzOSAxMDMuNjc0NCwxMjguNzgzIDExNi40ODc0LDExNS45NyBDMTI5LjMwMDQsMTAzLjE1NyAxMzYuMzU3NCw4Ni4xMjEgMTM2LjM1NzQsNjggQzEzNi4zNTc0LDQ5Ljg3OSAxMjkuMzAwNCwzMi44NDMgMTE2LjQ4NzQsMjAuMDMgTDExNi40ODc0LDIwLjAzIFoiLz4KPC9zdmc+Cg==",
            "AWS_ACCESS_KEY_ID": "EXAMPLE30JHVFEXAMPLE",
            "AWS_SECRET_ACCESS_KEY": "exampleMfEP1yElpH9kWEUmSVdbDcyl5Wexample",
            "AWS_REGION": "us-east-1"
        }
    }]
}

The below image shows how the Yarkon Admin Console page would look when set up with Wasabi.

Set up in Docker

When using a dockerized instance of Yarkon Server, the following sample shows how to set it up to run with Wasabi:

version: "3"

services:
  server:
    image: "yarkon/server:latest"  # Use the correct tag here
    ports:
      # Map the port of the host to the one used by Yarkon
      - "80:8000"
    environment:
      # If using an S3 compatible provider, these variables should be provided.
      # In this example, showing Wasabi
      PROVIDER_END_POINT: "https://s3.wasabisys.com"
      PROVIDER_STS_END_POINT: "https://iam.wasabisys.com"   # Will default to PROVIDER_END_POINT if not specified
      PROVIDER_IAM_END_POINT: "https://iam.wasabisys.com"   # Will default to PROVIDER_END_POINT if not specified
      PROVIDER_NAME: "Wasabi"
      PROVIDER_IMAGE: "data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxMzciIGhlaWdodD0iMTM2IiB2aWV3Qm94PSIwIDAgMTM3IDEzNiI+CiAgPHBhdGggZmlsbD0iIzAzQzczNiIgZmlsbC1ydWxlPSJldmVub2RkIiBkPSJNMTAwLjc3NjQsMTEzLjMzNSBMMTAwLjc3NjQsOTUuNzUzIEwxMTAuNDY5NCwxMDQuNTQxIEMxMDcuNTk1NCwxMDcuODM1IDEwNC4zNDI0LDExMC43OSAxMDAuNzc2NCwxMTMuMzM1IEwxMDAuNzc2NCwxMTMuMzM1IFogTTYwLjE1NzQsMTIzLjAzMiBMODguNTk3NCw5Ni4xNDggTDg4LjU5NzQsMTE5LjkxMSBDODIuMzY1NCwxMjIuMzMgNzUuNTk0NCwxMjMuNjYgNjguNTE4NCwxMjMuNjYgQzY1LjY3NjQsMTIzLjY2IDYyLjg4NTQsMTIzLjQ0NSA2MC4xNTc0LDEyMy4wMzIgTDYwLjE1NzQsMTIzLjAzMiBaIE0xMy4xNjM0LDczLjg0IEw0My4wNDU0LDEwMC42MzcgTDczLjA4NzQsNzIuNDI1IEw4OC41OTc0LDU3Ljc2NCBMODguNTk3NCw3OS4zODkgTDQ2LjU0OTQsMTE5LjEzNyBDMjguNDU3NCwxMTEuMzM1IDE1LjI5NzQsOTQuMjA2IDEzLjE2MzQsNzMuODQgTDEzLjE2MzQsNzMuODQgWiBNMzYuNzg4NCwyMi4yOTYgTDM2LjgwNzQsMzkuOTY4IEwyNi45MDA0LDMxLjA4NCBDMjkuODM0NCwyNy43OCAzMy4xNTM0LDI0LjgyOCAzNi43ODg0LDIyLjI5NiBMMzYuNzg4NCwyMi4yOTYgWiBNMzYuODQ4NCw3OC43MjEgTDEzLjc1NjQsNTguMDE0IEMxNC44NTU0LDUxLjk3OCAxNi45MzE0LDQ2LjI3OSAxOS44MTI0LDQxLjA4NiBMMzYuODI0NCw1Ni4zNDIgTDM2Ljg0ODQsNzguNzIxIFogTTc3LjAyMTQsMTIuOTg5IEw2NC40NTc0LDI0Ljg2NSBMNDguOTg1NCwzOS41MzkgTDQ4Ljk2MTQsMTUuODkgQzU1LjA0ODQsMTMuNTk4IDYxLjYzODQsMTIuMzQgNjguNTE4NCwxMi4zNCBDNzEuNDA4NCwxMi4zNCA3NC4yNDg0LDEyLjU2MiA3Ny4wMjE0LDEyLjk4OSBMNzcuMDIxNCwxMi45ODkgWiBNMTIzLjMxNDQsNzcuODAxIEMxMjIuMjU2NCw4My43MjkgMTIwLjI1NzQsODkuMzM0IDExNy40Nzg0LDk0LjQ1NyBMMTAwLjc3NjQsNzkuMzE0IEwxMDAuNzc2NCw1Ny4zNjggTDEyMy4zMTQ0LDc3LjgwMSBaIE0xMjMuODM1NCw2MS44MzUgTDk0LjYwMDQsMzUuMzMgTDY0LjczNTQsNjMuNTYxIEw0OS4wMjY0LDc4LjMxMyBMNDkuMDAzNCw1Ni4zMDggTDcyLjgzMTQsMzMuNzA5IEw5MC42MDA0LDE2LjkxMiBDMTA4LjUzNzQsMjQuNjk1IDEyMS42MDE0LDQxLjY1NSAxMjMuODM1NCw2MS44MzUgTDEyMy44MzU0LDYxLjgzNSBaIE0xMTYuNDg3NCwyMC4wMyBDMTAzLjY3NDQsNy4yMTcgODYuNjM4NCwwLjE2MSA2OC41MTg0LDAuMTYxIEM1MC4zOTc0LDAuMTYxIDMzLjM2MTQsNy4yMTcgMjAuNTQ4NCwyMC4wMyBDNy43MzU0LDMyLjg0MyAwLjY3OTQsNDkuODc5IDAuNjc5NCw2OCBDMC42Nzk0LDg2LjEyMSA3LjczNTQsMTAzLjE1NyAyMC41NDg0LDExNS45NyBDMzMuMzYxNCwxMjguNzgzIDUwLjM5NzQsMTM1LjgzOSA2OC41MTg0LDEzNS44MzkgQzg2LjYzODQsMTM1LjgzOSAxMDMuNjc0NCwxMjguNzgzIDExNi40ODc0LDExNS45NyBDMTI5LjMwMDQsMTAzLjE1NyAxMzYuMzU3NCw4Ni4xMjEgMTM2LjM1NzQsNjggQzEzNi4zNTc0LDQ5Ljg3OSAxMjkuMzAwNCwzMi44NDMgMTE2LjQ4NzQsMjAuMDMgTDExNi40ODc0LDIwLjAzIFoiLz4KPC9zdmc+Cg=="

      # Sample Wasabi credentials
      AWS_ACCESS_KEY_ID: "EXAMPLE30JHVFEXAMPLE"
      AWS_SECRET_ACCESS_KEY: "exampleMfEP1yElpH9kWEUmSVdbDcyl5Wexample"
      AWS_REGION: "us-east-1"

      # You can also specify the signature version. Valid values are v2, v4.
      # The default is v4.
      # "PROVIDER_SIGNATURE_VERSION": "v4"

    volumes:
      - dbdata:/var/app/current/database
      - license:/var/app/current/.lic
      - yarkon:/var/app/current/public/yarkon
      - /var/log:/var/app/current/log # Map the /var/log folder on the host to the log folder
volumes:
  dbdata:
  license:
  yarkon:

Permissions

Follow the instructions provider by your provider to set up the access policies. At the very least, the access policy must be similar to this generic one used with AWS S3:

{
    "Version": "2012-10-17",
    "Statement": [{
            "Sid": "AllowAllS3Actions",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::*"
        }, {
            "Sid": "AllowUIToDisplayIAMOptions",
            "Effect": "Allow",
            "Action": [
                "iam:List*",
                "iam:Get*"
            ],
            "Resource": "arn:aws:iam::<account-number>:*"
        }, {
            "Sid": "AllowTheRoleToGetPermissions",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::<account-number>:role/yarkons3-console-role"
        }, {
            "Sid": "AllowTheRoleToFederate",
            "Effect": "Allow",
            "Action": [
                "sts:GetFederationToken"
            ],
            "Resource": "arn:aws:sts::<account-number>:*"
        }
    ]
}

Make sure to replace the <account-number> with your account number.

Details (see the Sid attributes for reference):

AllowAllS3Actions – allows the Yarkon Server full access to S3. If you want to limit the usage of Yarkon in your organization to a predefined set of buckets, replace the statement with the below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowServerToIterateBuckets",
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Sid": "AllowServerToAccessSpecificBuckets",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::yarkons3-finance",
                "arn:aws:s3:::yarkons3-sales"
            ]
        },
        {
            "Sid": "AllowUserActionsLimitedToSpecificBuckets",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::yarkons3-finance/*",
                "arn:aws:s3:::yarkons3-sales/*"
            ]
        }
    ]
}

AllowUIToDisplayIAMOptions – only required when using Federated or Integrated security models.
The Yarkon Server does not need IAM access when set to use the Shared security model. This setting has no impact on the permissions granted to end users. If you only intend to use the Shared model, you can remove it.

AllowTheRoleToGetPermissions – only required when using the Integrated security model. You can remove it if using any of the other models. Also, the role name specified, yarkons3-console-role assumes this is the name you’d be using for the IAM role required (see below). If you choose a different name, make sure to update here.

AllowTheRoleToFederate – only required when using the Federated security model. You can remove it if using any of the other models.

For more information about the access policies used by Yarkon, please review this document Set up the IAM role and policies.

Features

As mentioned above, some of the features available to users of AWS S3, might not be available when using other storage providers. Use the Features page of the Yarkon Admin Console to turn off features that are not supported.