{"_id":"578520cb5ae9c20e00bc2635","category":{"_id":"5785191af3a10c0e009b75b0","version":"55faf11ba62ba1170021a9aa","__v":0,"project":"55faf11ba62ba1170021a9a7","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-07-12T16:21:46.337Z","from_sync":false,"order":25,"slug":"connect-cloud-storage","title":"CONNECT CLOUD STORAGE"},"parentDoc":null,"project":"55faf11ba62ba1170021a9a7","__v":2,"version":{"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","__v":37,"createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055","58ef66d88646742f009a0216","58f5d52d7891630f00fe4e77"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"user":"5613e4f8fdd08f2b00437620","updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-07-12T16:54:35.891Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":4,"body":"This page contains information on configuring a volume that is associated with an S3 bucket in Amazon Web Services' (AWS) cloud storage.\n[block:callout]\n{\n  \"type\": \"success\",\n  \"title\": \"Read more\",\n  \"body\": \"For a step-by-step tutorial on the process of creating and using a volume associated with an AWS S3 bucket, please see the [AWS Cloud storage tutorial](doc:aws-cloud-storage-tutorial).\"\n}\n[/block]\n##Authorization\nAuthorization for the CGC to read from (and, optionally, write to) the S3 bucket associated with a volume is handled using AWS's Identity and Access Management (IAM) service.\n\nThe [AWS Cloud storage tutorial](doc:aws-cloud-storage-tutorial) explains the process of configuring a volume associated with an S3 bucket in detail. In short, the process is:\n1. Create an AWS Identity and Access Management (IAM) user;\n2. Provide this user access to S3 bucket using an appropriate IAM policy. You can use [Seven Bridges'  IAM policy generator](aws-s3-policy-generator) to create the policy.\n3. Submit the IAM user's access credentials in the `storage.credentials` object in the API request to [create the volume](doc:create-a-volume-v2). For the syntax of this object and other objects relevant to AWS S3 volumes, see the next section.\n\n##AWS S3-specific volume parameters\nWhen making the API request to [create a new volume](doc:create-a-volume-v2), you must submit a `service` array. In the case that the volume is associated with an AWS S3 bucket, the values of the parameters in the `service` object are as follows:\n\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"AWS S3 service parameter\",\n    \"h-1\": \"Description of value\",\n    \"0-0\": \"`service`.`type`\\n*Required*\",\n    \"1-0\": \"`service`.`bucket`\\n*Required*\",\n    \"2-0\": \"`service`.`prefix`\\n*default: empty string*\",\n    \"0-1\": \"This must be set to `s3`.\",\n    \"1-1\": \"The name of your AWS S3 bucket.\",\n    \"2-1\": \"If provided, the value of this parameter will be used to modify any object key before an operation is performed on the bucket.\\nEven though AWS S3 is not truly a folder-based store and allows for almost arbitrarily named keys, the prefix is treated as a folder name. This means that after applying the prefix to the name of the object the resulting key will be normalized to conform to the standard path-based naming schema for files.\\nFor example, if you set the `prefix` for a volume to \\\"a10\\\", and import a file with `source`.`location` set to \\\"test.fastq\\\" from the volume to the CGC, then the object that will be referred to by the newly-created alias will be \\\"a10/test.fastq\\\".\",\n    \"3-0\": \"`service`.`endpoint`\\n*default: s3.amazonaws.com*\",\n    \"3-1\": \"The endpoint to use when talking to AWS S3.\\n\\nNote: Volumes associated with buckets hosted on AWS's China (Beijing) region must have `endpoint` set to `s3.cn-north-1.amazonaws.com.cn`. Volumes associated with buckets hosted in any other zone may use the default value.\",\n    \"4-0\": \"`service`.`credentials`.`access_key_id`\\n*Required*\",\n    \"5-0\": \"`service`.`credentials`.`secret_access_key`\\n*Required*\",\n    \"4-1\": \"The access key ID of the IAM user used for operations on this bucket. You will be provided with this when you create the IAM user.\",\n    \"5-1\": \"The secret access key of the IAM user to use for operations on this bucket. You will be provided with this when you create the IAM user.\",\n    \"6-0\": \"`service`.`properties`.`aws_canned_acl`\",\n    \"7-0\": \"`service`.`properties`.`sse_algorithm`\\ndefault: empty\",\n    \"6-1\": \"Specifies an S3 canned ACL to apply when exporting an object to this volume.\\nFor more information on the canned ACLs supported by S3, please see the [list of canned ACLs in the AWS documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl).\",\n    \"7-1\": \"This indicates whether server-side encryption should be enabled. Supported values are:\\n  * null: do not use server-side encryption;\\n  * `AES256`: use Amazon S3-managed keys (SSE-S3).\\nSupport for SSE-KMS and SSE-C will be added in a later release.\\nFor more information on AWS server-side encryption, see the [AWS webpage Protecting Data Using Server-Side Encryption](http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html).\"\n  },\n  \"cols\": 2,\n  \"rows\": 8\n}\n[/block]\n###Additional configuration\nThe IAM user policy is usually sufficient to permit the CGC to access your S3 bucket. In certain situations, however, it may be necessary to set up additional configuration on the bucket itself. Read how to do this by [enabling cross-origin resource sharing (CORS)](doc:enabling-cross-origin-resource-sharing-cors).","excerpt":"","slug":"amazon-web-services-simple-storage-service-aws-s3-volumes","type":"basic","title":"Amazon Web Services' Simple Storage Service (AWS S3) Volumes"}

Amazon Web Services' Simple Storage Service (AWS S3) Volumes


This page contains information on configuring a volume that is associated with an S3 bucket in Amazon Web Services' (AWS) cloud storage. [block:callout] { "type": "success", "title": "Read more", "body": "For a step-by-step tutorial on the process of creating and using a volume associated with an AWS S3 bucket, please see the [AWS Cloud storage tutorial](doc:aws-cloud-storage-tutorial)." } [/block] ##Authorization Authorization for the CGC to read from (and, optionally, write to) the S3 bucket associated with a volume is handled using AWS's Identity and Access Management (IAM) service. The [AWS Cloud storage tutorial](doc:aws-cloud-storage-tutorial) explains the process of configuring a volume associated with an S3 bucket in detail. In short, the process is: 1. Create an AWS Identity and Access Management (IAM) user; 2. Provide this user access to S3 bucket using an appropriate IAM policy. You can use [Seven Bridges' IAM policy generator](aws-s3-policy-generator) to create the policy. 3. Submit the IAM user's access credentials in the `storage.credentials` object in the API request to [create the volume](doc:create-a-volume-v2). For the syntax of this object and other objects relevant to AWS S3 volumes, see the next section. ##AWS S3-specific volume parameters When making the API request to [create a new volume](doc:create-a-volume-v2), you must submit a `service` array. In the case that the volume is associated with an AWS S3 bucket, the values of the parameters in the `service` object are as follows: [block:parameters] { "data": { "h-0": "AWS S3 service parameter", "h-1": "Description of value", "0-0": "`service`.`type`\n*Required*", "1-0": "`service`.`bucket`\n*Required*", "2-0": "`service`.`prefix`\n*default: empty string*", "0-1": "This must be set to `s3`.", "1-1": "The name of your AWS S3 bucket.", "2-1": "If provided, the value of this parameter will be used to modify any object key before an operation is performed on the bucket.\nEven though AWS S3 is not truly a folder-based store and allows for almost arbitrarily named keys, the prefix is treated as a folder name. This means that after applying the prefix to the name of the object the resulting key will be normalized to conform to the standard path-based naming schema for files.\nFor example, if you set the `prefix` for a volume to \"a10\", and import a file with `source`.`location` set to \"test.fastq\" from the volume to the CGC, then the object that will be referred to by the newly-created alias will be \"a10/test.fastq\".", "3-0": "`service`.`endpoint`\n*default: s3.amazonaws.com*", "3-1": "The endpoint to use when talking to AWS S3.\n\nNote: Volumes associated with buckets hosted on AWS's China (Beijing) region must have `endpoint` set to `s3.cn-north-1.amazonaws.com.cn`. Volumes associated with buckets hosted in any other zone may use the default value.", "4-0": "`service`.`credentials`.`access_key_id`\n*Required*", "5-0": "`service`.`credentials`.`secret_access_key`\n*Required*", "4-1": "The access key ID of the IAM user used for operations on this bucket. You will be provided with this when you create the IAM user.", "5-1": "The secret access key of the IAM user to use for operations on this bucket. You will be provided with this when you create the IAM user.", "6-0": "`service`.`properties`.`aws_canned_acl`", "7-0": "`service`.`properties`.`sse_algorithm`\ndefault: empty", "6-1": "Specifies an S3 canned ACL to apply when exporting an object to this volume.\nFor more information on the canned ACLs supported by S3, please see the [list of canned ACLs in the AWS documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl).", "7-1": "This indicates whether server-side encryption should be enabled. Supported values are:\n * null: do not use server-side encryption;\n * `AES256`: use Amazon S3-managed keys (SSE-S3).\nSupport for SSE-KMS and SSE-C will be added in a later release.\nFor more information on AWS server-side encryption, see the [AWS webpage Protecting Data Using Server-Side Encryption](http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html)." }, "cols": 2, "rows": 8 } [/block] ###Additional configuration The IAM user policy is usually sufficient to permit the CGC to access your S3 bucket. In certain situations, however, it may be necessary to set up additional configuration on the bucket itself. Read how to do this by [enabling cross-origin resource sharing (CORS)](doc:enabling-cross-origin-resource-sharing-cors).