{"_id":"5de1038fc7610000596aa8aa","project":"55faf11ba62ba1170021a9a7","version":{"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","__v":46,"createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055","58ef66d88646742f009a0216","58f5d52d7891630f00fe4e77","59a555bccdbd85001bfb1442","5a2a81f688574d001e9934f5","5b080c8d7833b20003ddbb6f","5c222bed4bc358002f21459a","5c22412594a2a5005cc9e919","5c41ae1c33592700190a291e","5c8a525e2ba7b2003f9b153c","5cbf14d58c79c700ef2b502e","5db6f03a6e187c006f667fa4"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"category":{"_id":"55faf932a8a7770d00c2c0bf","pages":["56268a69b1c2630d00b112b0","56268a85c2781f0d00364bbc","56268a92c2781f0d00364bbe","5637e0a0cfaa870d00cdeb6a","5637e0c3fbe1c50d008cb06a","5637e164f7e3990d007b2c41"],"version":"55faf11ba62ba1170021a9aa","__v":6,"project":"55faf11ba62ba1170021a9a7","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-09-17T17:32:34.286Z","from_sync":false,"order":8,"slug":"bring-your-private-data","title":"BRING DATA TO THE CGC"},"user":"5767bc73bb15f40e00a28777","__v":0,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2019-11-29T11:39:59.651Z","link_external":false,"link_url":"","sync_unique":"","hidden":false,"api":{"settings":"","results":{"codes":[]},"auth":"required","params":[],"url":""},"isReference":false,"order":999,"body":"[block:callout]\n{\n  \"type\": \"info\",\n  \"title\": \"\",\n  \"body\": \"Currently available version of ICDC data on the CGC corresponds to the initial ICDC release. For more information about the timeline and contents of each ICDC release, please see the ICDC website.\"\n}\n[/block]\n## About ICDC\n\nIntegrated Canine Data Commons (ICDC), a cloud-based repository of canine cancer data and was established to further research on human cancers by enabling comparative analysis with canine cancer. The data in the ICDC is sourced from multiple different programs and projects; all focused on canine subjects. The data is harmonized into an integrated data model and then made available to the research community.\n\nThe process of importing files from ICDC to the Cancer Genomics Cloud (CGC) consists of the following two stages:\n\n* Downloading a manifest file from the ICDC website.\n* Importing files to the CGC based on the downloaded manifest file.\n\nPlease note that the Seven Bridges team strives to keep the data available for import on the CGC aligned with ICDC data updates. However, ICDC updates are not instantly available for import on the CGC. Find out the [currently available release](page:icdc-data#section-currently-available-release-on-the-cgc) of ICDC data on the CGC and see the complete [history of updates](page:icdc-data#section-update-history).\n\n## Downloading manifest files from ICDC\n\nManifest files that are downloaded from ICDC contain information about the data you want to import in the second stage of this process.\n\nTo download a manifest file from ICDC:\n1. Open the ICDC website.\n2. Click **Cases** on the main navigation bar at the top of the page.\n3. In the filters section on the left, select the criteria to narrow down the cases list.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/542f8b4-icdc-import-1.png\",\n        \"icdc-import-1.png\",\n        1422,\n        774,\n        \"#377295\"\n      ]\n    }\n  ]\n}\n[/block]\n4. Scroll down to see the list of cases that match the filtering criteria.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/7bef125-icdc-import-2a.png\",\n        \"icdc-import-2a.png\",\n        1421,\n        774,\n        \"#e8ebec\"\n      ]\n    }\n  ]\n}\n[/block]\n5. Check the boxes next to cases you want to select.\n6. Click **SAVE TO MY CASES** in the bottom-left corner below the list.\n7. Click the **MY CASES** button in the top-right corner of the page. The button should also show a badge with the number of selected cases.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/5bba447-icdc-import-3.png\",\n        \"icdc-import-3.png\",\n        545,\n        471,\n        \"#3a607c\"\n      ]\n    }\n  ]\n}\n[/block]\n8. From the list of your cases, select the ones for which you want to download files.\n9. Click **GO TO FILES** in the bottom-left corner below the list. A list of available files for the selected cases is displayed.\n10. Check the boxes next to the files you want to download.\n11. Click **DOWNLOAD MANIFEST** in the bottom-left corner below the list. \nA CSV file is downloaded to your machine. Save the file as you will need it for the import procedure on the CGC.\n\n## Import files from ICDC to the CGC\n1. [Navigate to a project on the CGC](doc:view-a-project).\n2. Click the **Files** tab.\n3. Click **+ Add files**.\n4. Select the **Import from a manifest file** tab on the right.\n5. In the **Import files from** dropdown, select **Integrated Canine Data Commons (ICDC)**.\n6. Click the **Select file** button and select the downloaded manifest file.\n7. (Optional) In the **Add tags** field add the keywords (tags) that describe the imported items.\n8. **Resolve naming conflicts** - Select the action to be taken if a naming conflict occurs. Available actions are **Skip** (default option) and **Auto-rename**. Read more about [naming conflicts resolution](doc:upload-from-an-ftp-server#section-resolving-naming-conflicts).\n9. Click **Import**. The file import process starts and you are taken to the **Files** tab.\n[block:callout]\n{\n  \"type\": \"info\",\n  \"title\": \"\",\n  \"body\": \"The manifest file does not get imported along with the data files. To upload a manifest file generated for files on the ICDC Website, please use one of our [standard upload methods](doc:upload-to-the-cgc).\"\n}\n[/block]\n### Metadata\n\nThe ICDC API provides easy and open access to ICDC metadata. One of the ways to get ICDC metadata on the CGC is to use [Data Cruncher](doc:about-data-cruncher) to query the ICDC API.\n\n### Reference files\n\nThe [Public Reference Files](https://docs.cancergenomicscloud.org/docs/file-repositories-on-the-cgc) repository on the CGC contains two ICDC-related files that are ready to be used in e.g. RNA-seq alignment workflows with ICDC data provided as inputs. Those files are:\n* `Canis_familiaris.CanFam3.1.dna.toplevel.fa` (reference genome)\n* `Canis_familiaris.CanFam3.1.98.gtf` (gene annotation file)","excerpt":"","slug":"import-from-icdc","type":"basic","title":"Import data from ICDC"}

Import data from ICDC


[block:callout] { "type": "info", "title": "", "body": "Currently available version of ICDC data on the CGC corresponds to the initial ICDC release. For more information about the timeline and contents of each ICDC release, please see the ICDC website." } [/block] ## About ICDC Integrated Canine Data Commons (ICDC), a cloud-based repository of canine cancer data and was established to further research on human cancers by enabling comparative analysis with canine cancer. The data in the ICDC is sourced from multiple different programs and projects; all focused on canine subjects. The data is harmonized into an integrated data model and then made available to the research community. The process of importing files from ICDC to the Cancer Genomics Cloud (CGC) consists of the following two stages: * Downloading a manifest file from the ICDC website. * Importing files to the CGC based on the downloaded manifest file. Please note that the Seven Bridges team strives to keep the data available for import on the CGC aligned with ICDC data updates. However, ICDC updates are not instantly available for import on the CGC. Find out the [currently available release](page:icdc-data#section-currently-available-release-on-the-cgc) of ICDC data on the CGC and see the complete [history of updates](page:icdc-data#section-update-history). ## Downloading manifest files from ICDC Manifest files that are downloaded from ICDC contain information about the data you want to import in the second stage of this process. To download a manifest file from ICDC: 1. Open the ICDC website. 2. Click **Cases** on the main navigation bar at the top of the page. 3. In the filters section on the left, select the criteria to narrow down the cases list. [block:image] { "images": [ { "image": [ "https://files.readme.io/542f8b4-icdc-import-1.png", "icdc-import-1.png", 1422, 774, "#377295" ] } ] } [/block] 4. Scroll down to see the list of cases that match the filtering criteria. [block:image] { "images": [ { "image": [ "https://files.readme.io/7bef125-icdc-import-2a.png", "icdc-import-2a.png", 1421, 774, "#e8ebec" ] } ] } [/block] 5. Check the boxes next to cases you want to select. 6. Click **SAVE TO MY CASES** in the bottom-left corner below the list. 7. Click the **MY CASES** button in the top-right corner of the page. The button should also show a badge with the number of selected cases. [block:image] { "images": [ { "image": [ "https://files.readme.io/5bba447-icdc-import-3.png", "icdc-import-3.png", 545, 471, "#3a607c" ] } ] } [/block] 8. From the list of your cases, select the ones for which you want to download files. 9. Click **GO TO FILES** in the bottom-left corner below the list. A list of available files for the selected cases is displayed. 10. Check the boxes next to the files you want to download. 11. Click **DOWNLOAD MANIFEST** in the bottom-left corner below the list. A CSV file is downloaded to your machine. Save the file as you will need it for the import procedure on the CGC. ## Import files from ICDC to the CGC 1. [Navigate to a project on the CGC](doc:view-a-project). 2. Click the **Files** tab. 3. Click **+ Add files**. 4. Select the **Import from a manifest file** tab on the right. 5. In the **Import files from** dropdown, select **Integrated Canine Data Commons (ICDC)**. 6. Click the **Select file** button and select the downloaded manifest file. 7. (Optional) In the **Add tags** field add the keywords (tags) that describe the imported items. 8. **Resolve naming conflicts** - Select the action to be taken if a naming conflict occurs. Available actions are **Skip** (default option) and **Auto-rename**. Read more about [naming conflicts resolution](doc:upload-from-an-ftp-server#section-resolving-naming-conflicts). 9. Click **Import**. The file import process starts and you are taken to the **Files** tab. [block:callout] { "type": "info", "title": "", "body": "The manifest file does not get imported along with the data files. To upload a manifest file generated for files on the ICDC Website, please use one of our [standard upload methods](doc:upload-to-the-cgc)." } [/block] ### Metadata The ICDC API provides easy and open access to ICDC metadata. One of the ways to get ICDC metadata on the CGC is to use [Data Cruncher](doc:about-data-cruncher) to query the ICDC API. ### Reference files The [Public Reference Files](https://docs.cancergenomicscloud.org/docs/file-repositories-on-the-cgc) repository on the CGC contains two ICDC-related files that are ready to be used in e.g. RNA-seq alignment workflows with ICDC data provided as inputs. Those files are: * `Canis_familiaris.CanFam3.1.dna.toplevel.fa` (reference genome) * `Canis_familiaris.CanFam3.1.98.gtf` (gene annotation file)