Google Cloud Storage (GCS) Volumes

This page contains information on configuring a volume that is associated with a repository in Google Cloud storage.

👍
For a step-by-step tutorial on the process of creating and using a volume associated with an AWS S3 bucket, please see the Google Cloud storage tutorial.

Authorization

Authorization for the CGC to read from (and, optionally, write to) a Google Cloud storage bucket associated with a volume is provided by authenticating with a Google Cloud service account that has access to the bucket.

The Google Cloud storage tutorial explains the process of configuring a volume associated with a Google Cloud storage repository in detail. In short, the process is:

Create a Google Cloud service account that has access to the repository associated with your volume. This account should be:
(a) A Reader account for read-only volumes, or or Writer account for read-write volumes;
(b) A Reader account on the objects within those bucket(s).
Submit the Google Cloud service account credentials (email address and private key) in the storage.credentials object in the API request to create the volume. For the syntax of this object, see below.

Google Cloud Storage-specific volume parameters

When making the API request to create a new volume , you must submit a service array. In the case that the volume is associated with Google Cloud storage, the values of the parameters in the service object are as follows:

Google Cloud Platform service parameter	Description of value
`service`.`type`	This should be be set to `gcs`.
`service`.`bucket`	The name of the Google Cloud storage bucket.
`service`.`root_url` default: https://www.googleapis.com/	The endpoint to use when talking to Google Cloud storage.
`service`.`prefix` default: empty	If provided, the value of this parameter will be used to modify any object key before an operation is performed on the bucket. Even though Google Cloud Platform is not truly a folder-based store and allows for almost arbitrarily named keys, the prefix is treated as a folder name. This means that after applying the prefix to the name of the object the resulting key will be normalized to conform to the standard path-based naming schema for files. For example, if you set the `prefix` for a volume to "a10", and import a file with `source`.`location` set to "test.fastq" from the volume to the CGC, then the object that will be referred to by the newly-created alias will be "a10/test.fastq".
`service`.`credentials`.`client_email`	The client email address for the Google Cloud service account to use for operations on this bucket.
`service`.`credentials`.`private_key`	The private key for the Google Cloud service account to use for operations on this bucket.

Additional configuration

In certain situations, however, it may be necessary to set up additional configuration on the bucket itself. For details, please see the documentation on enabling cross-origin resource sharing (CORS).

Updated about 1 year ago