Reading File Data

Files stored by Vendia can be accessed from your existing cloud setup by executing a set of configuration steps, dependent on the Cloud Service Provider designated for the node when it was created.

AWS Nodes

Granting client permissions to access files in Vendia

In order to view the File stored in your Vendia Uni, you need to grant the client AWS account access to the Vendia Uni. You can do this using the setting the value for aws.s3ReadAccounts within an updateVendia_Settings mutation, providing your AWS Account ID.

mutation updateAwsReadAccounts {
  updateVendia_Settings(
    input: { aws: { s3ReadAccounts: ["YOUR AWS ACCOUNT ID"] } }
    syncMode: ASYNC
  ) {
    transaction {
      _id
    }
  }
}

Note: updateVendia_Settings mutations will overwrite any existing settings prescribed by your request, so be sure to account for the previous state of the node’s settings if necessary.

Note: anyone with access to the AWS Account you provide will have access to the Files in your Uni Node.

Now your AWS Account will be able to access the Files in your Vendia Uni. To do so, you’ll also need your Node’s Bucket name.

Getting your Node’s Bucket

To get the Node’s Bucket information you can access the Uni Dashboard and select your Uni. Under your Node’s Resources will be a property for S3 Bucket ARN. The portion following the arn:aws:s3::: is your bucket’s name.

You can also get this information using the share CLI and executing the command:

share uni get --uni YOUR_UNI_NAME

As part of the output you will see a section called aws_FileStorage with a property name, this is the Node’s bucket name.

Querying File metadata

You can query the node for File metadata, e.g.:

query listFiles {
  listVendia_FileItems {
    Vendia_FileItems {
      createdTime
      destinationKey
      etag
      sourceBucket
      sourceKey
      sourceRegion
      sourceVersion
      temporaryUrl
      _id
    }
  }
}

Retrieving the File contents

To retrieve the File contents, use an S3 client to perform a GetObject request using the node bucket and DestinationKey attributes on the File metadata.

For example, using the AWS CLI,

aws s3 cp s3://my-node-destination-bucket/my-destination-key local-file

If you are using a Free Tier Uni, you need to provide the Requester Pays header as part of any call. All Vendia Free Tier buckets are set up as Requester Pays buckets. We handle all costs except for accesses directly to the Uni File bucket.

For non-Free Tier unis, the File metadata will include a pre-signed temporaryUrl that can be used to retrieve the object directly using any HTTP client.

Azure Nodes

Create a multi-tenant app registration

Open a new terminal session, logging out of any previous Azure CLI session.

az logout

az login

Make note of the tenant you will be operating against.

MY_SESSION=$(az account show)

MY_TENANT_ID=$(echo $MY_SESSION | jq -r .tenantId)

# verify the value was set
echo $MY_TENANT_ID

Run the following command to create a new multi-tenant appplication, replacing <app-registration-name> with the desired name for your app registration.

MY_APPLICATION=$( az ad app create \
    --display-name "<app-registration-name>" \
    --sign-in-audience AzureADMultipleOrgs \
)

Make a note of the appId (client ID) value returned by the command.

MY_APPLICATION_ID=$(echo $MY_APPLICATION | jq -r .appId)

# verify the value was set
echo $MY_APPLICATION_ID

Run the following command to create a client secret for the app registration, saving off the password as a local variable for your terminal session:

# update the options for configuring duration, etc. as you see fit
MY_APPLICATION_CREDENTIAL=$( \
  az ad app credential reset \
    --id $MY_APPLICATION_ID \
    --display-name "Client secret for accessing Vendia Node's storage" \
    --years 1 \
)

MY_APPLICATION_PASSWORD=$(echo $MY_APPLICATION_CREDENTIAL | jq -r .password)

Store the values for client ID and secret securely using Azure best practices (ex. within an Azure Key Vault).

Update your node’s settings

Since more than one application can be allowed to read your Vendia_File storage, make note of the current settings configuration for azure.storageAccountReaders for your node (by default, this value should be null).

query getNodeStorageAccountReaders {
  getVendia_Settings {
    azure {
      storageAccountReaders {
        applicationId
      }
    }
  }
}

{
  "data": {
    "getVendia_Settings": {
      "azure": null
    }
  }
}

Craft a mutation to update the node’s Vendia_Settings at the azure.storageAccountReaders path, using the value from $MY_APPLICATION_ID for the $applicationId GraphQL variable:

mutation updateNodeStorageAccountReaders(
  $applicationId: String = "<value from $MY_APPLICATION_ID>"
) {
  updateVendia_Settings(
    input: {
      azure: { storageAccountReaders: [{ applicationId: $applicationId }] }
    }
    syncMode: NODE_LEDGERED
  ) {
    result {
      azure {
        storageAccountReaders {
          applicationId
        }
      }
    }
  }
}

{
  "data": {
    "updateVendia_Settings": {
      "result": {
        "azure": {
          "storageAccountReaders": [
            {
              "applicationId": "891c1d75-87d9-4bd9-b4d8-abc2cda30a16"
            }
          ]
        }
      }
    }
  }
}

Use your node’s storage container

Log out of any previous Azure CLI session.

az logout

Log into the Vendia Share CLI.

See https://docs.vendia.com/cli/guide for more.

Set up your terminal session with the information for the uni and node for which you are configuring access.

VENDIA_UNI_NAME="<uni-name>"
VENDIA_NODE_NAME="<node-nam>"

Set up your terminal session with the information for the uni and node for which you are configuring access.

VENDIA_NODE_RESOURCES=$( \
  share uni get --uni $VENDIA_UNI_NAME --json | \
  jq --arg node_name "$VENDIA_NODE_NAME" \
    '.nodes[] | select(.name == $node_name) | .resources' \
)

VENDIA_NODE_TENANT_ID=$(echo $VENDIA_NODE_RESOURCES | jq -r .azure_TenantId)

VENDIA_NODE_STORAGE_URL=$(echo $VENDIA_NODE_RESOURCES | jq -r .azure_FileStorage.id)

# verify the value was set
echo $VENDIA_NODE_STORAGE_URL

VENDIA_NODE_STORAGE_ACCOUNT_NAME=$( \
  echo "$VENDIA_NODE_STORAGE_URL" | \
  awk -F '://' '{print $2}' | \
  cut -d '/' -f 1 | \
  cut -d '.' -f 1 \
)

# verify the value was set
echo $VENDIA_NODE_STORAGE_ACCOUNT_NAME

VENDIA_NODE_STORAGE_CONTAINER_NAME=$( \
  echo "$VENDIA_NODE_STORAGE_URL" | \
  awk -F '://' '{print $2}' | \
  cut -d '/' -f 2- | \
  cut -d '?' -f 1 | rev | \
  cut -d '/' -f 1 | rev \
)

# verify the value was set
echo $VENDIA_NODE_STORAGE_CONTAINER_NAME

az login --service-principal -u $MY_APPLICATION_ID -p $MY_APPLICATION_PASSWORD \
  --tenant $VENDIA_NODE_TENANT_ID

Run the following command to list the files within your node’s storage:

az storage blob list \
  --account-name $VENDIA_NODE_STORAGE_ACCOUNT_NAME \
  --container-name $VENDIA_NODE_STORAGE_CONTAINER_NAME \
  --auth-mode login