↳ Browse datasets via the Datasets API

[block:callout] { "type": "danger", "body": "This feature is in our advance access program. This means that, while it is fully operational, it is subject to change.", "title": "Advance Access" } [/block] [block:callout] { "type": "info", "body": "Seven Bridges is committed to providing CGC users with the most up-to-date versions of the public datasets that are available from the NCI Genomic Data Commons (GDC). In keeping with this commitment, the CGC transitioned to hosting the GDC Data Release 14.0 versions of TCGA GRCh38, TARGET GRCh38, TCGA legacy, and CCLE on February 11, 2019.\n\nAs of this date, all files accessible via the Datasets API correspond to Data Release 14.0. Please contact the Support Team at [support@sbgenomics.com](mailto:support@sbgenomics.com) if you have any questions. The CGC Team looks forward to continuing to collaborate with the GDC in the months ahead to ensure the timely availability through the CGC of new data releases for this dataset." } [/block] [block:callout] { "type": "warning", "title": "On this page:", "body": "* [Overview](#section-overview)\n* [Return accessible datasets](#section-return-accessible-datasets)\n* [Return all entities within a dataset](#section-return-all-entities-within-a-dataset)\n* [Return all instances of an entity](#section-return-all-instances-of-an-entity)\n* [Return an entity's metadata schema](#section-return-an-entity-s-metadata-schema)\n* [Next step](#section-next-step)\n* [Resources](#section-resources)" } [/block] ##Overview Browse datasets via the Datasets API by issuing successive `GET` requests. Use these browsing requests individually or in conjunction with [querying](query-via-the-datasets-api). For instance, entities located through browsing are resources which can be the subject of a query. On this page, learn about requests to return: * [datasets you can access](#section-return-accessible-datasets) * [all the entities within a dataset](#section-return-all-entities-within-a-dataset) * [all instances of a single entity](#section-return-all-instances-of-an-entity) * [a single entity's metadata schema](#section-return-an-entity-s-metadata-schema) <div align="right"><a href="#top">top</a></div> ##Return accessible datasets Make the following `GET` request. Be sure to replace the [authentication token](get-your-authentication-token) with your own. [block:code] { "codes": [ { "code": "GET /datasets/ HTTP/1.1\nHost: cgc-datasets-api.sbgenomics.com\nX-SBG-Auth-Token: 7942f56901534434a054dafc3813bc96", "language": "http", "name": "Return accessible datasets" } ] } [/block] This returns a list of accessible datasets, as shown below. [block:code] { "codes": [ { "code": "{\n \"description\": \"Datasets API Advanced Access Program\",\n \"_links\": {\n \"self\": {\n \"href\": \"cgc-datasets-api.sbgenomics.com/datasets/\"\n },\n \"tcga_grch38\": {\n \"href\": \"cgc-datasets-api.sbgenomics.com/datasets/tcga_grch38/v0\",\n \"label\": \"TCGA GRCh38\"\n },\n \"ccle\": {\n \"href\": \"cgc-datasets-api.sbgenomics.com/datasets/ccle/v0\",\n \"label\": \"CCLE Legacy\"\n },\n \"tcga\": {\n \"href\": \"cgc-datasets-api.sbgenomics.com/datasets/tcga/v0\",\n \"label\": \"TCGA Legacy\"\n },\n \"target\": {\n \"href\": \"cgc-datasets-api.sbgenomics.com/datasets/target/v0\",\n \"label\": \"TARGET GRCh38\"\n },\n \"tcia\": {\n \"href\": \"cgc-datasets-api.sbgenomics.com/datasets/tcia/v0\",\n \"label\": \"TCIA\"\n },\n \"cptac\": {\n \"href\": \"cgc-datasets-api.sbgenomics.com/datasets/cptac/v0\",\n \"label\": \"CPTAC\"\n },\n \"icgc\": {\n \"href\": \"cgc-datasets-api.sbgenomics.com/datasets/icgc/v0\",\n \"label\": \"ICGC\" \n }\n }\n}", "language": "json", "name": "Response body" } ] } [/block] The `href` element lists the path for each dataset, such as `https://cgc-datasets-api.sbgenomics.com/datasets/ccle/v0` for CCLE. <div align="right"><a href="#top">top</a></div> ##Return all entities within a dataset Make a `GET` request to the `href` of a dataset to return all of its entities. Learn more about [each dataset's entities](about-metadata-for-datasets) from its metadata page. [block:code] { "codes": [ { "code": "GET /datasets/tcga/v0 HTTP/1.1\nHost: cgc-datasets-api.sbgenomics.com\nX-SBG-Auth-Token: 7942f56901534434a054dafc3813bc96", "language": "http", "name": "Return all entities within a dataset" } ] } [/block] The response contains a list of entities for the dataset you specified. [block:code] { "codes": [ { "code": "{\n \"_links\": {\n \"cases\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/cases/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/cases\"\n },\n \"analytes\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/analytes/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/analytes\"\n },\n \"radiation_therapies\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/radiation_therapies/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/radiation_therapies\"\n },\n \"drug_therapies\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/drug_therapies/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/drug_therapies\"\n },\n \"follow_ups\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/follow_ups/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/follow_ups\"\n },\n \"portions\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/portions/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/portions\"\n },\n \"aliquots\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/aliquots/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/aliquots\"\n },\n \"samples\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/samples/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/samples\"\n },\n \"slides\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/slides/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/slides\"\n },\n \"query\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/query\"\n },\n \"self\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0\"\n },\n \"files\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files\"\n },\n \"new_tumor_events\": {\n \"schema\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/new_tumor_events/schema\",\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/new_tumor_events\"\n }\n }\n}", "language": "json", "name": "Response body" } ] } [/block] Note that two items in the list returned, `query` and `self`, are not dataset entities: * `self` contains the property `href` which is set to the same path issued in making the query. * `query` contains the property `href` which is set to a path that can be used to issue a query into the entities returned, by making a `POST` request. Learn more about [querying with the Datasets API](query-via-the-datasets-api). <div align="right"><a href="#top">top</a></div> ##Return all instances of an entity Make a `GET` request to the `href` of an entity to return all instances of that entity within the specified dataset. For example, to see a list of all TCGA files, make the following request. [block:code] { "codes": [ { "code": "GET /datasets/tcga/v0/files HTTP/1.1\nHost: cgc-datasets-api.sbgenomics.com\nX-SBG-Auth-Token: 7942f56901534434a054dafc3813bc96", "language": "http", "name": "Return all instances of an entity" } ] } [/block] This returns the following response. As you can see, the response contains 100 results (count) per page. You can page through using the paths under `_links`. The resulting files are listed under the `_embedded` section. [block:code] { "codes": [ { "code": "{\n \"total\": 573241,\n \"count\": 100,\n \"_links\": {\n \"self\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files\"\n },\n \"next\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files?page=1\"\n },\n \"last\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files?page=5732\"\n },\n \"first\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files\"\n }\n },\n \"_embedded\": {\n \"files\": [\n {\n \"label\": \"LIGER_p_TCGA_166_180_SNP_N_GenomeWideSNP_6_G03_896632.ismpolish.data.txt\",\n \"id\": \"564a31a5e4b09c884b215ccb\",\n \"_links\": {\n \"self\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files/564a31a5e4b09c884b215ccb\"\n }\n }\n },\n {\n \"label\": \"6055424037_R05C01_Grn.idat\",\n \"id\": \"564a31a6e4b0298dd2c5492c\",\n \"_links\": {\n \"self\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files/564a31a6e4b0298dd2c5492c\"\n }\n }\n },\n {\n \"label\": \"mdanderson.org_SARC.MDA_RPPA_Core.SuperCurve.Level_2.B06C2A68-8B01-41AC-B3B2-20C7BBFA562B.txt\",\n \"id\": \"564a31a6e4b0298dd2c54934\",\n \"_links\": {\n \"self\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files/564a31a6e4b0298dd2c54934\"\n }\n }\n },\n {\n \"label\": \"PULED_p_TCGA_130_157_N_GenomeWideSNP_6_H10_831788.nocnv_hg19.seg.txt\",\n \"id\": \"564a31a6e4b0298dd2c54936\",\n \"_links\": {\n \"self\": {\n \"href\": \"https://cgc-datasets-api.sbgenomics.com/datasets/tcga/v0/files/564a31a6e4b0298dd2c54936\"\n }\n }\n },\n \n \n <snip>\n \n }\n ]\n }\n}", "language": "json", "name": "Response body" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Return an entity's metadata schema Make a `GET` request to an entity's href to obtain its schema. Each entity has its own metadata schema, a list of the metadata fields used to describe the entity and the permissible datatypes (strings, integers, etc) that each field takes. Each metadata field, such as `hasDataType` in the request below, is followed by an object which indicates the type of value for each field, such as `integer`, `string`, or `enum`. If the type is given as `enum`, the object also contains a list of all possible values for the given metadata field. Additionally, note that under `_links`, there is a list of connections to other entities in the dataset. In the example below, these connections include `"hasAliquot"`, `"hasCase"`, `"hasSample"`, `"hasPortion"`, and `"hasAnalyte"`. For example, to see the metadata schema for `files` send the request: [block:code] { "codes": [ { "code": "GET /datasets/tcga/v0/files/schema HTTP/1.1\nHost: cgc-datasets-api.sbgenomics.com\nX-SBG-Auth-Token: 7942f56901534434a054dafc3813bc96", "language": "http", "name": "Return an entity's metadata schema" } ] } [/block] This returns the following response which lists all the properties of the `file` entity: [block:code] { "codes": [ { "code": "{\n \"hasSize\": {\n \"type\": \"integer\"\n },\n \"hasDataType\": {\n \"values\": [\n \"Structural rearrangement\",\n \"Protein expression\",\n \"Gene expression\",\n \"Clinical\",\n \"Simple nucleotide variation\",\n \"DNA methylation\",\n \"Other\",\n \"Copy number variation\",\n \"Not available\",\n \"Raw sequencing data\",\n \"Raw microarray data\"\n ],\n \"type\": \"enum\"\n },\n \"id\": {\n \"type\": \"string\"\n },\n \"hasDataSubmittingCenter\": {\n \"values\": [\n \"MD Anderson - Institute for Applied Cancer Science\",\n \"Johns Hopkins / University of Southern California\",\n \"Baylor College of Medicine\",\n \"HudsonAlpha Institute for Biotechnology\",\n \"Harvard Medical School\",\n \"MD Anderson - RPPA Core Facility (Proteomics)\",\n \"Complete Genomics Inc.\",\n \"Washington University School of Medicine\",\n \"NCH BCR\",\n \"Broad Institute of MIT and Harvard\",\n \"Lawrence Berkeley National Laboratory\",\n \"Memorial Sloan-Kettering Cancer Center\",\n \"Wellcome Trust Sanger Institute\",\n \"University of North Carolina\",\n \"Canada's Michael Smith Genome Sciences Centre\",\n \"Not available\",\n \"Nationwide Children's Hospital BCR\",\n \"University of California, Santa Cruz\"\n ],\n \"type\": \"enum\"\n },\n \"hasDiseaseType\": {\n \"values\": [\n \"Adrenocortical Carcinoma\",\n \"Stomach Adenocarcinoma\",\n \"Colon Adenocarcinoma\",\n \"Cervical Squamous Cell Carcinoma and Endocervical Adenocarcinoma\",\n \"Brain Lower Grade Glioma\",\n \"Bladder Urothelial Carcinoma\",\n \"Sarcoma\",\n \"Lung Adenocarcinoma\",\n \"Testicular Germ Cell Tumors\",\n \"Cholangiocarcinoma\",\n \"Lymphoid Neoplasm Diffuse Large B-cell Lymphoma\",\n \"Thymoma\",\n \"Liver Hepatocellular Carcinoma\",\n \"Thyroid Carcinoma\",\n \"Uveal Melanoma\",\n \"Mesothelioma\",\n \"Kidney Renal Papillary Cell Carcinoma\",\n \"Kidney Renal Clear Cell Carcinoma\",\n \"Kidney Chromophobe\",\n \"Pheochromocytoma and Paraganglioma\",\n \"Acute Myeloid Leukemia\",\n \"Prostate Adenocarcinoma\",\n \"Uterine Corpus Endometrial Carcinoma\",\n \"Esophageal Carcinoma\",\n \"Breast Invasive Carcinoma\",\n \"Skin Cutaneous Melanoma\",\n \"Uterine Carcinosarcoma\",\n \"Pancreatic Adenocarcinoma\",\n \"Glioblastoma Multiforme\",\n \"Ovarian Serous Cystadenocarcinoma\",\n \"Head and Neck Squamous Cell Carcinoma\",\n \"Rectum Adenocarcinoma\",\n \"Lung Squamous Cell Carcinoma\"\n ],\n \"type\": \"enum\"\n },\n \"hasPlatform\": {\n \"values\": [\n \"HG-CGH-244A\",\n \"Ion Torrent PGM\",\n \"BCR Record\",\n \"Not available\",\n \"HG-CGH-415K_G4124A\",\n \"Illumina DNA Methylation OMA003 CPI\",\n \"Illumina HiSeq\",\n \"AgilentG4502A_07_2\",\n \"Complete Genomics\",\n \"AgilentG4502A_07_1\",\n \"H-miRNA_8x15Kv2\",\n \"Hospital Record\",\n \"HiSeq X Ten\",\n \"Illumina DNA Methylation OMA002 CPI\",\n \"LS 454\",\n \"Affymetrix U133 Plus 2\",\n \"Illumina GA\",\n \"AgilentG4502A_07_3\",\n \"MDA_RPPA_Core\",\n \"Illumina HumanHap550\",\n \"ABI capillary sequencer\",\n \"CGH-1x1M_G4447A\",\n \"ABI SOLiD\",\n \"Affymetrix SNP Array 6.0\",\n \"Mixed platforms\",\n \"Illumina MiSeq\",\n \"HT_HG-U133A\",\n \"H-miRNA_8x15K\",\n \"HG-U133_Plus_2\",\n \"HuEx-1_0-st-v2\",\n \"Illumina Human Methylation 27\",\n \"Illumina Human 1M Duo\",\n \"Illumina Human Methylation 450\"\n ],\n \"type\": \"enum\"\n },\n \"hasReferenceGenome\": {\n \"values\": [\n \"GRCh37-lite_WUGSC_variant_2\",\n \"GRCh37\",\n \"HG19\",\n \"GRCh37-lite_WUGSC_variant_1\",\n \"HG18\",\n \"HS37D5\",\n \"GRCh37-lite-+-HPV_Redux-build\",\n \"HG18_Broad_variant\",\n \"NCBI36_BCCAGSC_variant\",\n \"NCBI-human-build36\",\n \"GRCh37_BI_Variant\",\n \"NCBI36_BCM_variant\",\n \"HG19_Broad_variant\",\n \"NCBI36_WUGSC_variant\",\n \"GRCh37-lite\",\n \"Not available\"\n ],\n \"type\": \"enum\"\n },\n \"publishedDate\": {\n \"type\": \"dateTime\"\n },\n \"hasDataSubtype\": {\n \"values\": [\n \"Bisulfite sequence alignment\",\n \"Sequencing tag counts\",\n \"Sequencing tag\",\n \"Isoform expression quantification\",\n \"Genotypes\",\n \"miRNA quantification\",\n \"Copy number variation\",\n \"Simple somatic mutation\",\n \"Raw intensities\",\n \"Protein expression quantification\",\n \"Gene expression summary\",\n \"Methylation percentage\",\n \"Gene expression quantification\",\n \"Structural variation\",\n \"Copy number segmentation\",\n \"Microsattelite instability\",\n \"Not available\",\n \"Intensities Log2Ratio\",\n \"Exon quantification\",\n \"Biospecimen data\",\n \"Probeset summary\",\n \"Simple nucleotide variation\",\n \"Methylation beta value\",\n \"Intensities\",\n \"Exon junction quantification\",\n \"Unaligned reads\",\n \"Copy number estimate\",\n \"LOH\",\n \"Aligned reads\",\n \"Normalized copy numbers\",\n \"Clinical data\",\n \"Normalized intensities\"\n ],\n \"type\": \"enum\"\n },\n \"lastModifiedDate\": {\n \"type\": \"dateTime\"\n },\n \"_links\": {\n \"hasAliquot\": \"anyURI\",\n \"self\": {\n \"href\": \"metadata-api-vayu.sbgenomics.com:9989/datasets/v0/tcga/files/schema\"\n },\n \"hasSample\": \"anyURI\",\n \"hasPortion\": \"anyURI\",\n \"hasAnalyte\": \"anyURI\",\n \"hasCase\": \"anyURI\"\n },\n \"hasAccessLevel\": {\n \"values\": [\n \"Controlled\",\n \"Open\"\n ],\n \"type\": \"enum\"\n },\n \"hasDataFormat\": {\n \"values\": [\n \"BED\",\n \"FA\",\n \"XLSX\",\n \"DGE-TAG\",\n \"IDAT\",\n \"XML\",\n \"TXT\",\n \"BAM\",\n \"BAI\",\n \"DAT\",\n \"CHP\",\n \"MAF\",\n \"CEL\",\n \"TIF\",\n \"TAR\",\n \"FSA\",\n \"Not available\",\n \"GCT\",\n \"VCF\",\n \"TARGZ\"\n ],\n \"type\": \"enum\"\n },\n \"hasInvestigation\": {\n \"values\": [\n \"TCGA-THYM\",\n \"TCGA-KIRC\",\n \"TCGA-SARC\",\n \"TCGA-ESCA\",\n \"TCGA-PCPG\",\n \"TCGA-PRAD\",\n \"TCGA-UCEC\",\n \"TCGA-ACC\",\n \"TCGA-READ\",\n \"TCGA-UVM\",\n \"TCGA-CESC\",\n \"TCGA-COAD\",\n \"Not available\",\n \"TCGA-TGCT\",\n \"TCGA-DLBC\",\n \"TCGA-KICH\",\n \"TCGA-THCA\",\n \"TCGA-HNSC\",\n \"TCGA-UCS\",\n \"TCGA-CHOL\",\n \"TCGA-BLCA\",\n \"TCGA-GBM\",\n \"TCGA-SKCM\",\n \"TCGA-LUSC\",\n \"TCGA-STAD\",\n \"TCGA-LUAD\",\n \"TCGA-LIHC\",\n \"TCGA-KIRP\",\n \"TCGA-BRCA\",\n \"TCGA-MESO\",\n \"TCGA-PAAD\",\n \"TCGA-LAML\",\n \"TCGA-OV\",\n \"TCGA-LGG\"\n ],\n \"type\": \"enum\"\n },\n \"hasStoragePath\": {\n \"type\": \"string\"\n },\n \"hasExperimentalStrategy\": {\n \"values\": [\n \"Exon array\",\n \"CGH array\",\n \"RNA-Seq\",\n \"miRNA-Seq\",\n \"AMPLICON\",\n \"Not available\",\n \"VALIDATION\",\n \"WXS\",\n \"DNA-Seq\",\n \"Total RNA-Seq\",\n \"Gene expression array\",\n \"Protein expression array\",\n \"miRNA expression array\",\n \"Genotyping array\",\n \"Bisulfite-Seq\",\n \"Methylation array\",\n \"WGS\",\n \"MSI-Mono-Dinucleotide Assay\"\n ],\n \"type\": \"enum\"\n },\n \"label\": {\n \"type\": \"string\"\n },\n \"hasSubmitterId\": {\n \"type\": \"string\"\n },\n \"uploadDate\": {\n \"type\": \"dateTime\"\n },\n \"hasGDCFileUUID\": {\n \"type\": \"string\"\n }\n}", "language": "json", "name": "Response body" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Next step Browsing requests can be used in conjunction with querying. For instance, entities located through browsing are resources which can be the subject of a query. Learn more about [querying via the Datasets API](doc:query-via-the-datasets-api). <div align="right"><a href="#top">top</a></div> ##Resources * [Query via the Datasets API](doc:query-via-the-datasets-api) * [About the Datasets API](doc:about-the-datasets-api) * [About metadata for datasets](doc:about-metadata-for-datasets) <div align="right"><a href="#top">top</a></div>

The CGC Knowledge Center

GET STARTED

TUTORIALS

ACCESS THE CGC

MANAGE AN ANALYSIS ENVIRONMENT

COLLABORATE SECURELY

DATASETS HUB

PUBLIC PROJECTS

BRING YOUR PRIVATE DATA

SET METADATA ASSOCIATED WITH A PRIVATE FILE

SEARCH FILES

ADD FILES TO A PROJECT

ORGANIZE FILES

ARCHIVE FILES

FIND APPS

RABIX OVERVIEW

DOCKER

DESCRIBE YOUR TOOL

TOOL WRAPPING TIPS AND TRICKS

EDIT AN APP

ARCHIVE APPS

RUN AN ANALYSIS

TASK EXECUTION

REVIEW A TASK

VIEW ANALYSIS RESULTS

INTERACTIVE ANALYSIS

DOWNLOAD RESULTS

DEVELOPER TOOLS

CONNECT CLOUD STORAGE

API Hub

API CLIENTS

SB Command Line Interface

API Reference

BULK API ACTIONS

WATCH AND LEARN

DATA CRUNCHER

SBFS (Beta)

SBFS Command Reference