{"__v":1,"_id":"57851f3d87c9280e00903902","category":{"project":"55faf11ba62ba1170021a9a7","version":"55faf11ba62ba1170021a9aa","_id":"5785191af3a10c0e009b75b0","__v":0,"sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-07-12T16:21:46.337Z","from_sync":false,"order":24,"slug":"connect-cloud-storage","title":"CONNECT CLOUD STORAGE"},"parentDoc":null,"project":"55faf11ba62ba1170021a9a7","user":"5613e4f8fdd08f2b00437620","version":{"__v":37,"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055","58ef66d88646742f009a0216","58f5d52d7891630f00fe4e77"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-07-12T16:47:57.776Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":2,"body":"[block:callout]\n{\n  \"type\": \"warning\",\n  \"title\": \"On this page\",\n  \"body\": \"* [What is an alias?](doc:aliases#section-what-is-an-alias-)\\n* [Recognizing aliases](doc:aliases#section-recognizing-aliases)\\n* [Creating aliases](doc:aliases#section-creating-aliases)  \\n * [Importing from a volume to the CGC](aliases#section-importing-from-a-volume-to-the-cgc)\\n * [Exporting from the CGC to a volume](aliases#section-exporting-from-the-cgc-to-a-volume) \\n* [Differences between aliases and CGC files](doc:aliases#section-differences-between-aliases-and-cgc-files)\\n * [Storage charges](doc:aliases#section-storage-charges)\\n * [Change tracking](doc:aliases#section-change-tracking)\\n * ['Broken' aliases](doc:aliases#section--broken-aliases)\\n * [Deleting alises](doc:aliases#section-deleting-alises)\\n * [Exporting alises](doc:aliases#section-exporting-alises)\\n * [Updating an alias's bucket, prefix, or location](doc:aliases#section-updating-an-alias-s-bucket-prefix-or-location)\\n * [Aliases and archiving](doc:aliases#section-aliases-and-archiving)\"\n}\n[/block]\n##What is an alias?\nAn **alias** is a pointer, on the CGC, that refers to a file in a cloud storage repository (AWS S3 bucket, or Google Cloud storage). Aliases enable files in cloud storage repositories outside the CGC to be manipulated on the CGC, for instance, to input them to computational tasks. \n\nAn alias for a file can be used in much the same way as a file on the CGC: you can set its metadata, copy it between projects, download it, and use it as an input to tasks. Operations like these affect the way the target file is used on the CGC, but do not edit the content of the file itself. There are, however, some differences between CGC files, and external files referred to by aliases. These are described in detail below.\n\n###Recognizing aliases\nIn the visual interface of the CGC, files that are aliases are marked with an icon of a bucket and a small black arrow, as shown below:\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/X4sPsifNReiN2xTcDfCb_Screen%20Shot%202016-06-15%20at%2010.20.52%20PM.png\",\n        \"Screen Shot 2016-06-15 at 10.20.52 PM.png\",\n        \"700\",\n        \"280\",\n        \"#37699a\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\n##Creating aliases\nAccess to cloud storage repositories from the CGC is mediated by volumes: to use a file in your cloud storage on the CGC, you must first create a volume that authorizes the CGC to access the objects in a specific cloud storage repository. Once you've created a volume associated with the cloud storage repository, there are two ways to create an alias: by importing a file from a volume to the CGC, or by exporting a file from the CGC to a volume. Both of these operations can be carried out using the API.\n\n###Importing from a volume to the CGC\nIf the cloud storage account used for your volume contains files, you can [import](doc:start-an-import-job-v2) files from the volume to the Platform. Importing a file creates an alias for the file on the Platform, which can then be used in computation. A walk-through of the procedure in the case of Amazon Web Services' cloud storage is given in the [AWS Cloud storage tutorial](doc:aws-cloud-storage-tutorial).\n\n###Exporting from the CGC to a volume\nYou can add files from the CGC to your cloud storage by [exporting](doc:start-an-export-job-v2)  them to a volume created using the cloud storage account. This operation will do the following:\n1. Copy the content of the CGC file to the cloud storage that you used to create the volume;\n2. Replace the CGC file with an alias that refers to the copied file's content.\n\nIn other words, the original project file will become an alias for the newly exported object on the volume; the file will appear in your project as before, but will be only a pointer to the exported file now stored elsewhere.\n\nIf there are no copies of the exported file anywhere else on the CGC (for instance, in another project), then the original file will be deleted from the CGC storage, and so will no longer count towards your total storage costs.\n\nExporting takes advantage of the fastest copying procedure applicable for the cloud storage(s). The copying takes place asynchronously, so you can check on its progress using the API request to list imports.\n\n##Differences between aliases and CGC files\nSince aliases are pointers to files not hosted on the CGC, there are some differences between their behavior and that of regular CGC files.\n\n###Storage charges\nAliases do not count towards your storage costs.\n\n###Change tracking\nThe CGC cannot track changes to the file an alias points to. Since aliases refer to files external to the CGC by their file names (or keys, depending on the cloud service), an alias remains operational as long as there is an object matching the file name (or key) in the cloud storage used to set up the volume. This remains true if the contents of the file in question change.\n\nConsequently, we recommend that you maintain control over the contents of the cloud storage account used for the volume, particularly objects in it that have been brought onto the CGC as aliases. This consideration is essential to guarantee the reproducibility of tasks on the CGC using aliases.\n\n###'Broken' aliases\nA volume is required to create an alias. If that volume is deleted or made unavailable, an alias will continue to exist on the CGC as a \"broken reference\" to the file that was stored on it. Although the content of the alias will not be readable by the CGC, the alias will maintain its metadata defined on the CGC as well as its association with tasks in which it was used or created.\n\nSince aliases are defined in terms of the name of the volume that they are stored in, a 'broken' alias may become readable again by creating a new volume that has the same name as the original volume and is associated with the same cloud storage repository as it.\n\n###Deleting alises\nDeleting an alias on the CGC never results in the corresponding object being deleted from the connected cloud storage. To do so, you must delete objects manually from your cloud storage account, using your cloud account credentials.\n\n###Exporting alises\nExporting to a volume is an operation that copies a file hosted on the CGC to the cloud storage account associated with the volume. This operation will not work on a CGC file that is itself an alias.\n\n###Updating an alias's bucket, prefix, or location\nAn alias will always point to the same location on its original cloud storage. Changes to the originating volume, like updating its prefix or the cloud storage account the volume is associated with, do not affect the alias after it has been created.\n\nThe only exception to this rule are changes to the volume's access configuration. If a volume is deleted, renamed, made inactive or its credentials have been revoked or invalidated in any way, then any aliases to cease functioning.\n\n###Aliases and archiving\nAliases cannot be archived using the CGC's [Archive feature](doc:archive-a-file).","excerpt":"","slug":"aliases","type":"basic","title":"Aliases"}
[block:callout] { "type": "warning", "title": "On this page", "body": "* [What is an alias?](doc:aliases#section-what-is-an-alias-)\n* [Recognizing aliases](doc:aliases#section-recognizing-aliases)\n* [Creating aliases](doc:aliases#section-creating-aliases) \n * [Importing from a volume to the CGC](aliases#section-importing-from-a-volume-to-the-cgc)\n * [Exporting from the CGC to a volume](aliases#section-exporting-from-the-cgc-to-a-volume) \n* [Differences between aliases and CGC files](doc:aliases#section-differences-between-aliases-and-cgc-files)\n * [Storage charges](doc:aliases#section-storage-charges)\n * [Change tracking](doc:aliases#section-change-tracking)\n * ['Broken' aliases](doc:aliases#section--broken-aliases)\n * [Deleting alises](doc:aliases#section-deleting-alises)\n * [Exporting alises](doc:aliases#section-exporting-alises)\n * [Updating an alias's bucket, prefix, or location](doc:aliases#section-updating-an-alias-s-bucket-prefix-or-location)\n * [Aliases and archiving](doc:aliases#section-aliases-and-archiving)" } [/block] ##What is an alias? An **alias** is a pointer, on the CGC, that refers to a file in a cloud storage repository (AWS S3 bucket, or Google Cloud storage). Aliases enable files in cloud storage repositories outside the CGC to be manipulated on the CGC, for instance, to input them to computational tasks. An alias for a file can be used in much the same way as a file on the CGC: you can set its metadata, copy it between projects, download it, and use it as an input to tasks. Operations like these affect the way the target file is used on the CGC, but do not edit the content of the file itself. There are, however, some differences between CGC files, and external files referred to by aliases. These are described in detail below. ###Recognizing aliases In the visual interface of the CGC, files that are aliases are marked with an icon of a bucket and a small black arrow, as shown below: [block:image] { "images": [ { "image": [ "https://files.readme.io/X4sPsifNReiN2xTcDfCb_Screen%20Shot%202016-06-15%20at%2010.20.52%20PM.png", "Screen Shot 2016-06-15 at 10.20.52 PM.png", "700", "280", "#37699a", "" ] } ] } [/block] ##Creating aliases Access to cloud storage repositories from the CGC is mediated by volumes: to use a file in your cloud storage on the CGC, you must first create a volume that authorizes the CGC to access the objects in a specific cloud storage repository. Once you've created a volume associated with the cloud storage repository, there are two ways to create an alias: by importing a file from a volume to the CGC, or by exporting a file from the CGC to a volume. Both of these operations can be carried out using the API. ###Importing from a volume to the CGC If the cloud storage account used for your volume contains files, you can [import](doc:start-an-import-job-v2) files from the volume to the Platform. Importing a file creates an alias for the file on the Platform, which can then be used in computation. A walk-through of the procedure in the case of Amazon Web Services' cloud storage is given in the [AWS Cloud storage tutorial](doc:aws-cloud-storage-tutorial). ###Exporting from the CGC to a volume You can add files from the CGC to your cloud storage by [exporting](doc:start-an-export-job-v2) them to a volume created using the cloud storage account. This operation will do the following: 1. Copy the content of the CGC file to the cloud storage that you used to create the volume; 2. Replace the CGC file with an alias that refers to the copied file's content. In other words, the original project file will become an alias for the newly exported object on the volume; the file will appear in your project as before, but will be only a pointer to the exported file now stored elsewhere. If there are no copies of the exported file anywhere else on the CGC (for instance, in another project), then the original file will be deleted from the CGC storage, and so will no longer count towards your total storage costs. Exporting takes advantage of the fastest copying procedure applicable for the cloud storage(s). The copying takes place asynchronously, so you can check on its progress using the API request to list imports. ##Differences between aliases and CGC files Since aliases are pointers to files not hosted on the CGC, there are some differences between their behavior and that of regular CGC files. ###Storage charges Aliases do not count towards your storage costs. ###Change tracking The CGC cannot track changes to the file an alias points to. Since aliases refer to files external to the CGC by their file names (or keys, depending on the cloud service), an alias remains operational as long as there is an object matching the file name (or key) in the cloud storage used to set up the volume. This remains true if the contents of the file in question change. Consequently, we recommend that you maintain control over the contents of the cloud storage account used for the volume, particularly objects in it that have been brought onto the CGC as aliases. This consideration is essential to guarantee the reproducibility of tasks on the CGC using aliases. ###'Broken' aliases A volume is required to create an alias. If that volume is deleted or made unavailable, an alias will continue to exist on the CGC as a "broken reference" to the file that was stored on it. Although the content of the alias will not be readable by the CGC, the alias will maintain its metadata defined on the CGC as well as its association with tasks in which it was used or created. Since aliases are defined in terms of the name of the volume that they are stored in, a 'broken' alias may become readable again by creating a new volume that has the same name as the original volume and is associated with the same cloud storage repository as it. ###Deleting alises Deleting an alias on the CGC never results in the corresponding object being deleted from the connected cloud storage. To do so, you must delete objects manually from your cloud storage account, using your cloud account credentials. ###Exporting alises Exporting to a volume is an operation that copies a file hosted on the CGC to the cloud storage account associated with the volume. This operation will not work on a CGC file that is itself an alias. ###Updating an alias's bucket, prefix, or location An alias will always point to the same location on its original cloud storage. Changes to the originating volume, like updating its prefix or the cloud storage account the volume is associated with, do not affect the alias after it has been created. The only exception to this rule are changes to the volume's access configuration. If a volume is deleted, renamed, made inactive or its credentials have been revoked or invalidated in any way, then any aliases to cease functioning. ###Aliases and archiving Aliases cannot be archived using the CGC's [Archive feature](doc:archive-a-file).