{"_id":"579a55a8c2c4d30e00da3933","initVersion":{"_id":"55faf11ba62ba1170021a9aa","version":"1.0"},"user":{"_id":"5613e4f8fdd08f2b00437620","username":"","name":"Emile Young"},"project":"55faf11ba62ba1170021a9a7","__v":0,"createdAt":"2016-07-28T18:57:44.053Z","changelog":[],"body":"Over the last several weeks we’ve added a number of features to make it easier for you to use the CGC  to learn more from your data even more efficiently. We are shifting from announcing changes on a per-version basis to a calendar-based cycle which more accurately reflects our current development process. Below you can find more information about recently-developed features and improvements. Going forward, you’ll see more frequent updates about new functionalities on the CGC. \n\n<hr style=\"margin-left:75px;margin-right:75px;border-top:double;color:#cccccc\">\n##July 25\n###Improvements to batch task validations\nWe’ve extended the validations performed on individual tasks to now include batches of tasks initiated through either the API or GUI. Additionally, usability improvements make the display of warning and error messages on batch tasks more actionable. This improvement will make it easier to avoid common issues when starting large numbers of tasks. In fact, each error now links to documentation explaining the error and suggesting how to troubleshoot it from the visual interface and the API.\n\n###Improvements to long-running queries on the data browser\nTo prevent performance issues associated with very complex or minimally filtered queries, we are introducing a 60 second timeout on the databrowser. Queries that can not be resolved in this time will be stopped and the user prompted to reform the query. Typically, introducing a filter or removing the last added node will resolve the issue. Learn more about the [timeout-error](page:timeout-error).\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##July 11\n###Public files improvements\nWe’ve refreshed the Public Files repository to include the most commonly used reference and test files. Descriptions and file metadata have been updated to make it easier for you to find the files that you need for your analysis.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##July 4\n###App revision notes and provenance\nWe introduced App revision notes to make it easier and more transparent to develop tools and workflows on the platform. This functionality allows researchers to track and communicate changes to apps when they are developing them. Additionally, the new app provenance system allows changes you make in one project to be propagated to copies of your tool in other projects. This is particularly helpful for tool developers who may wish to distribute their tools to their collaborators.Learn more about [App Revision Notes](doc:revision-notes).\n\n###Volumes API\nThe volumes API enables researchers to connect private data resources already existing on AWS or Google Cloud Platform for secure computation on the platform. Additionally, results generated on the platform can now be linked to a researcher’s private storage. Learn more about the [Volumes API](doc:connect-cloud-storage-overview) from our documentation. You can also find a list of API requests [here](doc:volumes-v2).\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##June 27\n###Improved Projects menu\nThe new **Projects **menu on the top navigation bar now includes a search field to make it easier for you to find the project you are looking for. Input keywords to search through project names and descriptions. Learn more about [using the search box](doc:view-a-project) in the **Projects** menu.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##June 20\n###Cancer Cell Line Encyclopedia (CCLE) data\nThe Cancer Cell Line Encyclopedia is a collection of nearly 1,000 cell lines that have been analyzed by multiple next generation sequencing approaches, including RNA Sequencing, Whole Exome Sequencing, and Whole Genome Sequencing. This data is available for analysis under the **Public projects** tab of the top navigation bar. You won’t pay for storage of the raw data files: copy the entire project or select files into your own project on the CGC to take advantage of the CCLE dataset. This dataset also serves as a great place to get started running new analyses. We’ve created a few example tasks to help you get started. Learn more about using the [CCLE public project](doc:ccle).\n\n###Data Browser redesign\nWe've incorporated feedback from our first users to resign the data browser interface from the ground up. The underlying technology remains the same, but we've cleaned up the visuals and made it more intuitive for you to use it to find exactly the data you are interested in. Find out more about the updates (blog:the-data-browser-has-a-new-look) or learn about using the [Data Browser](doc:the-data-browser).\n\n###Tags improvement\nAs a continuation of the previously released Tags functionality, we’ve now introduced the ability to tag files at upload via the graphical uploader, the command line, or via FTP/HTTP. Tags help you organize and communicate about your data which is particularly helpful when working in large groups. Tags are currently viewable only via the visual interface. Learn more about tagging files as you upload them via the [CGC Uploader](doc:upload-via-the-cgc-uploader), via the [command line](doc:upload-via-the-command-line), or from an [FTP or HTTP(S) server](doc:upload-from-an-ftp-server).\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##June 13\n###Batch task improvements\nBatch tasks allow you to run a large number of tasks in just a few clicks. This is particularly helpful when, for example, you need to align all of files from a sequencing experiment using the same workflow. The batch task feature allows you to group files for analysis via a number of different metadata properties including sample and library IDs. Improvements to the visual interface allows you to more easily keep track of tasks initiated in this way. Additionally, overall improvements enhance the stability and number of tasks that may be initiated in parallel. Learn more about [batch tasks](doc:perform-a-batch-analysis).\n\n###Uploads via the API python library\nThe API library now includes support for uploads. This allows you to automate uploads programmatically. The API works best for large numbers of relatively small files. We recommend using the command line uploader for files larger than 10GB as this feature includes additional optimizations not present in the API. Learn more about [uploading your files via the API](doc:upload-files).\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##June 6\n###New loaders\nWe’ve refreshed the visual indications shown when content is being loaded on a page. The global loader is now located under the top navigation bar and component loaders have a new look.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##May 30\n###Contextual documentation\nWe’ve made improvements to the information displayed when you click the question mark on the top navigation bar. You’ll now be shown links directly relevant to the platform page you are currently visiting.\n\n###Improved metadata queries via API\nThe API request to [list files](doc:list-files-in-a-project) now accepts parameter fields. This allows you to specify exactly which metadata fields to include in the API response and should reduce the overall number of queries needed to access the information that you need about your files. \n\n###Improvements to file navigation from the task page\nYou can now more directly access both  input and output files from the Task page. Clicking on the file picker icon allows you to easily edit metadata or directly download files without needing to navigate away from your task. Learn more about [browsing, downloading, and deleting files from the **Task** page](doc:view-output-files#section-view-output-files-from-the-task-page).\n\n###Datasets API\nAs an alternative to the visual interface of the Data Browser or querying using the SPARQL console via the SPARQL query language, you can use the Datasets API to programmatically query and browse TCGA data. You can also use the Datasets API in conjunction with the CGC API to add TCGA data to a project. Learn more about using the [Datasets API](doc:datasets-api-overview).\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##May 23\n###Improvements to task validations - API\nImproved validations and checks now notify users when a task started via API is likely to fail due to improper setup. This is an extension of the recent improvements to tasks initiated via the visual interface and allows users to more quickly identify and solve issues when starting executions programmatically. \n\n###Improvements to the email notification controls\nA number of improvements were made to further improve the usability of the recently released email notification control feature. You can now select from preset options or select to mirror your global settings on a project-by-project basis. \n\n###Username validations\nEnhanced validations on usernames during account creation provide better interoperability with other services such as Docker. \n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##May 16\n###Tagging of files\nTo help you find and organize your data, we’re introducing the ability to tag files within projects and upon upload. Use tags to mark a set of files based on a cohort of patients, a molecular characteristic, or other attributes that you want to remember. For example, if you filter your files based on a specific set of metadata characteristics, you can tag the resulting files so you can remember, find, and analyze them days or weeks later. Some improvements were also made to the data browser interface to accommodate new flows associated with adding tags to sets of files.  This represents a first iteration and additional ways of using tags within projects will be further developed based on user feedback. Note, you can’t yet add tags to files when uploading them via the graphical uploader or interacting with them via API but this will be introduced in subsequent versions. Learn more about [tagging your files](doc:tag-your-files).\n\n###Improvements to task validations\nBefore running a task, the system performs a number of checks to evaluate if it will run properly. These improvements help users avoid issues that could cause a task to fail after starting; for example an error message will be shown if secondary files (like index files) are required but are not available. This improves the robustness of running tasks and reduces cost of task failures due to improper set up.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>","slug":"recent-improvements-to-the-cancer-genomics-cloud","title":"Recent improvements to the Cancer Genomics Cloud"}

Recent improvements to the Cancer Genomics Cloud


Over the last several weeks we’ve added a number of features to make it easier for you to use the CGC to learn more from your data even more efficiently. We are shifting from announcing changes on a per-version basis to a calendar-based cycle which more accurately reflects our current development process. Below you can find more information about recently-developed features and improvements. Going forward, you’ll see more frequent updates about new functionalities on the CGC. <hr style="margin-left:75px;margin-right:75px;border-top:double;color:#cccccc"> ##July 25 ###Improvements to batch task validations We’ve extended the validations performed on individual tasks to now include batches of tasks initiated through either the API or GUI. Additionally, usability improvements make the display of warning and error messages on batch tasks more actionable. This improvement will make it easier to avoid common issues when starting large numbers of tasks. In fact, each error now links to documentation explaining the error and suggesting how to troubleshoot it from the visual interface and the API. ###Improvements to long-running queries on the data browser To prevent performance issues associated with very complex or minimally filtered queries, we are introducing a 60 second timeout on the databrowser. Queries that can not be resolved in this time will be stopped and the user prompted to reform the query. Typically, introducing a filter or removing the last added node will resolve the issue. Learn more about the [timeout-error](page:timeout-error). <div align="right"><a href="#top">top</a></div> ##July 11 ###Public files improvements We’ve refreshed the Public Files repository to include the most commonly used reference and test files. Descriptions and file metadata have been updated to make it easier for you to find the files that you need for your analysis. <div align="right"><a href="#top">top</a></div> ##July 4 ###App revision notes and provenance We introduced App revision notes to make it easier and more transparent to develop tools and workflows on the platform. This functionality allows researchers to track and communicate changes to apps when they are developing them. Additionally, the new app provenance system allows changes you make in one project to be propagated to copies of your tool in other projects. This is particularly helpful for tool developers who may wish to distribute their tools to their collaborators.Learn more about [App Revision Notes](doc:revision-notes). ###Volumes API The volumes API enables researchers to connect private data resources already existing on AWS or Google Cloud Platform for secure computation on the platform. Additionally, results generated on the platform can now be linked to a researcher’s private storage. Learn more about the [Volumes API](doc:connect-cloud-storage-overview) from our documentation. You can also find a list of API requests [here](doc:volumes-v2). <div align="right"><a href="#top">top</a></div> ##June 27 ###Improved Projects menu The new **Projects **menu on the top navigation bar now includes a search field to make it easier for you to find the project you are looking for. Input keywords to search through project names and descriptions. Learn more about [using the search box](doc:view-a-project) in the **Projects** menu. <div align="right"><a href="#top">top</a></div> ##June 20 ###Cancer Cell Line Encyclopedia (CCLE) data The Cancer Cell Line Encyclopedia is a collection of nearly 1,000 cell lines that have been analyzed by multiple next generation sequencing approaches, including RNA Sequencing, Whole Exome Sequencing, and Whole Genome Sequencing. This data is available for analysis under the **Public projects** tab of the top navigation bar. You won’t pay for storage of the raw data files: copy the entire project or select files into your own project on the CGC to take advantage of the CCLE dataset. This dataset also serves as a great place to get started running new analyses. We’ve created a few example tasks to help you get started. Learn more about using the [CCLE public project](doc:ccle). ###Data Browser redesign We've incorporated feedback from our first users to resign the data browser interface from the ground up. The underlying technology remains the same, but we've cleaned up the visuals and made it more intuitive for you to use it to find exactly the data you are interested in. Find out more about the updates (blog:the-data-browser-has-a-new-look) or learn about using the [Data Browser](doc:the-data-browser). ###Tags improvement As a continuation of the previously released Tags functionality, we’ve now introduced the ability to tag files at upload via the graphical uploader, the command line, or via FTP/HTTP. Tags help you organize and communicate about your data which is particularly helpful when working in large groups. Tags are currently viewable only via the visual interface. Learn more about tagging files as you upload them via the [CGC Uploader](doc:upload-via-the-cgc-uploader), via the [command line](doc:upload-via-the-command-line), or from an [FTP or HTTP(S) server](doc:upload-from-an-ftp-server). <div align="right"><a href="#top">top</a></div> ##June 13 ###Batch task improvements Batch tasks allow you to run a large number of tasks in just a few clicks. This is particularly helpful when, for example, you need to align all of files from a sequencing experiment using the same workflow. The batch task feature allows you to group files for analysis via a number of different metadata properties including sample and library IDs. Improvements to the visual interface allows you to more easily keep track of tasks initiated in this way. Additionally, overall improvements enhance the stability and number of tasks that may be initiated in parallel. Learn more about [batch tasks](doc:perform-a-batch-analysis). ###Uploads via the API python library The API library now includes support for uploads. This allows you to automate uploads programmatically. The API works best for large numbers of relatively small files. We recommend using the command line uploader for files larger than 10GB as this feature includes additional optimizations not present in the API. Learn more about [uploading your files via the API](doc:upload-files). <div align="right"><a href="#top">top</a></div> ##June 6 ###New loaders We’ve refreshed the visual indications shown when content is being loaded on a page. The global loader is now located under the top navigation bar and component loaders have a new look. <div align="right"><a href="#top">top</a></div> ##May 30 ###Contextual documentation We’ve made improvements to the information displayed when you click the question mark on the top navigation bar. You’ll now be shown links directly relevant to the platform page you are currently visiting. ###Improved metadata queries via API The API request to [list files](doc:list-files-in-a-project) now accepts parameter fields. This allows you to specify exactly which metadata fields to include in the API response and should reduce the overall number of queries needed to access the information that you need about your files. ###Improvements to file navigation from the task page You can now more directly access both input and output files from the Task page. Clicking on the file picker icon allows you to easily edit metadata or directly download files without needing to navigate away from your task. Learn more about [browsing, downloading, and deleting files from the **Task** page](doc:view-output-files#section-view-output-files-from-the-task-page). ###Datasets API As an alternative to the visual interface of the Data Browser or querying using the SPARQL console via the SPARQL query language, you can use the Datasets API to programmatically query and browse TCGA data. You can also use the Datasets API in conjunction with the CGC API to add TCGA data to a project. Learn more about using the [Datasets API](doc:datasets-api-overview). <div align="right"><a href="#top">top</a></div> ##May 23 ###Improvements to task validations - API Improved validations and checks now notify users when a task started via API is likely to fail due to improper setup. This is an extension of the recent improvements to tasks initiated via the visual interface and allows users to more quickly identify and solve issues when starting executions programmatically. ###Improvements to the email notification controls A number of improvements were made to further improve the usability of the recently released email notification control feature. You can now select from preset options or select to mirror your global settings on a project-by-project basis. ###Username validations Enhanced validations on usernames during account creation provide better interoperability with other services such as Docker. <div align="right"><a href="#top">top</a></div> ##May 16 ###Tagging of files To help you find and organize your data, we’re introducing the ability to tag files within projects and upon upload. Use tags to mark a set of files based on a cohort of patients, a molecular characteristic, or other attributes that you want to remember. For example, if you filter your files based on a specific set of metadata characteristics, you can tag the resulting files so you can remember, find, and analyze them days or weeks later. Some improvements were also made to the data browser interface to accommodate new flows associated with adding tags to sets of files. This represents a first iteration and additional ways of using tags within projects will be further developed based on user feedback. Note, you can’t yet add tags to files when uploading them via the graphical uploader or interacting with them via API but this will be introduced in subsequent versions. Learn more about [tagging your files](doc:tag-your-files). ###Improvements to task validations Before running a task, the system performs a number of checks to evaluate if it will run properly. These improvements help users avoid issues that could cause a task to fail after starting; for example an error message will be shown if secondary files (like index files) are required but are not available. This improves the robustness of running tasks and reduces cost of task failures due to improper set up. <div align="right"><a href="#top">top</a></div>