I am repeatedly getting errors while trying to upload this 31GB file: [[email protected] login_node ~/software/cgc-uploader]$ ls -lh /scratch/users/nathankw/IndelIndices.tar.gz -rw-r--r-- 1 nathankw ruthm 31G May 13 22:13 /scratch/users/nathankw/IndelIndices.tar.gz [[email protected] login_node ~/software/cgc-uploader]$ ./bin/cgc-uploader.sh -p 0b57a269-388b-4a99-b197-fd9206a00375 /scratch/users/nathankw/IndelIndices.tar.gz Initializing upload... Starting upload of 1 file(s) to 'salzman' 5736ddbfe4b0e5be418274b4 /scratch/users/nathankw/IndelIndices.tar.gz 100.00% Upload failed (FAILED) When I look at the project on CGC that I was uploading to, it looks like the file is all there size-wise. Why do I get the failed error message?

I'd like to test some of my tools on this platform. I was wondering if there was any data that's not TCGA-restricted that everyone can access and use, just for getting familiar with the environment. This could perhaps be 1kG data or cell-line data. I guess the alternative is to upload and use private data but I wanted to check what's recommended. Thanks, Avi

I am wrapping a tool and enter my docker repository (miasteinberg/mirdeep2:0.0.7) in the repository entry field. I click on the Save button. I get the green box saying my tool was successfully updated, but the Docker Repo field is empty now. I even copied and pasted the repo from a previous tool that definitely works to make sure it wasn't a typo. Same thing. After saving, the field becomes blank. I restarted Firefox - same problem. I tried Chrome - same problem. What should I try next?

What is the best way to redirect stderr in SBG? I see the option for stdout that adds a '>' to the command line, but what about when we want to add '2>'? Right now, I have the '2>' added as my last argument, but I get the error... This task ran into a problem during execution and did not finish. Command perl miRDeep2.pl /sbgenomics/Projects/5311483a-0df2-41d1-ba6e-361dc84bc31c/sample.reads.fa /sbgenomics/Projects/5311483a-0df2-41d1-ba6e-361dc84bc31c/hg19_bowtie.fa /sbgenomics/Projects/5311483a-0df2-41d1-ba6e-361dc84bc31c/sample.reads_vs_genome.arf /sbgenomics/Projects/5311483a-0df2-41d1-ba6e-361dc84bc31c/mature.fa none /sbgenomics/Projects/5311483a-0df2-41d1-ba6e-361dc84bc31c/hairpin.fa -P -t Human 2> report.log failed with exit code 2. ...and no error log, so I'm guessing that something is angry at my redirection. Is the shell bash? And if it is not, how do I log my stderr? Also, here is the documentation for the tool: https://www.mdc-berlin.de/36105849/en/research/research_teams/systems_biology_of_gene_regulatory_elements/projects/miRDeep/documentation#mirdeep2

I am attempting to access project output files through the CGC API. I am using the sevenbridges python package. I have run the code provided on the documentation for accessing projects with initialization via environment variables (https://pypi.python.org/pypi/sevenbridges-python). However, an Unauthorized error is returned. I have double checked and the authentication token I am using is correct. What would be the reason for this?

Hi! I used to be able to find my gene of interest (C1ORF61) using the Case Explorer but now I can't find it any more. Is there any gene re-naming in this latest version? Thanks!

Hello I have dockerized Picard in the following image: cgc-images.sbgenomics.com/purkayasthaa/runpicard-2.3.0. In brief, I use a perl script runPicard to run the "AddOrReplaceReadGroups" and "MarkDuplicates" sequentially, on a sorted bam file, within a single container. The image runs as expected on my local machine. However I get an error: "Failed to start runpicard. Internal error: Docker container failed to start. " on CGC. Any ideas on what may be going wrong? Thanks.

Hello, I'm trying to bring a java application (MELTv2.1.4) to the cgc platform. I've made a docker which works perfectly locally and create a tool on CGC. However, maybe there is something wrong with my command line because if I run the tool the task fails with exit code 1. I would like to figure out what is wrong with the command line, but unfortunately job.err.log is totally blank so I cannot understand the problem. For example if I run the application without arguments in my docker ("java -jar MELT.jar Single"), the result is: ` Command Line: MELT.jar Single Start time: Jun 1, 2018 11:13:01 AM Performing MELT analysis... Missing required options: bamfile, h, w, t, n, c usage: java -jar MELT.jar Single <options> MELTv2.1.4 - MELT-Single - Perform transposon analysis on a single sample. -a Reads have been aligned with bwa-aln. [false] -ac Remove ac0 sites from final VCF file. [false] -b <arg> Exclusion list for chromosomes. A '/' seperated list: i.e. to exclude chromosomes 1,2, and 4, put -b 1/2/4. [null] -bamfile <arg> Bam file for MEI analysis. -bowtie <arg> Path to the bowtie2 algorithm if not in PATH [null]. -c <arg> Coverage level of supplied bam file. -cov <arg> Standard deviation cutoff when calling final sites in integer format. [35] -d <arg> Minumum length of chromosome/contig size for calling elements. [1000000] -e <arg> Expected insert size between reads. [500] -h <arg> Path to the reference sequence used to align reads. -j <arg> Total percentage of sites allowed to be no call (in integer form i.e. 25 percent would be -i 25, not .25). [25] -k BAM file(s) have already been processed for discordant pairs (suffixes .fq, .disc, and .disc.bai are already present for the bam file in -l). [false] -n <arg> Path to the genome annotation. -nocleanup Do not cleanup MELT intermediate files after running. [false] -q Alignments are pre Illumina 1.3 Quality encoding. [false] -r <arg> Read length of the supplied bam file(s). [100] -s <arg> Standard deviation cutoff for excluding sites with improper balance of readpairs in double format. [2.0] -sr <arg> Filter sites with less than X SRs during breakpoint ascertainment. Default, -1, is to not filter any such sites. [-1] -t <arg> Path to the transposon ZIP file(s) to be used for this analysis. -w <arg> Path to the working root directory. -z <arg> Maximum reads to load into memory when iterating over sequence files. Setting higher increases run time, but may increase sensitivity in large (>60X coverage) bam files. Setting lower may decrease sensitivity in all bam files. [5000] -help will print this message and exit ` This is not an actual "error message" (like for example "Error: Unable to access jarfile MELTe.jar" that appears in the job.err.log file if I intentionally misspell the name of the app) so, maybe that's the reason why I can't see it in the job.err.log file. Is there a way to get it printed somewhere in order to understand eventual errors in the commandl line? I tried with a trick: ` java -jar MELT.jar Single > debuggy.txt ` This works fine "locally" in the docker; I tried also on CGC setting up a specific output port for the debug file (glob= *.txt); unfortunately there is no downloadable output file (which is very strange). Even more strange is the fact that, according to job.tree.log, the file has been created ` . . [ 89 Jun 01 10:48.16 UTC] cmd.log . [2.3K Jun 01 10:48.17 UTC] deboggy.txt . [ 0 Jun 01 10:48.27 UTC] job.err.log . [3.1K Jun 01 10:48.16 UTC] job.json . [ 0 Jun 01 10:48.27 UTC] job.tree.log 9.5K used in 0 directories, 5 files ` but still is not possible to download it. Can someone help me with this? Thanks a lot! Filippo

I got the HER2, ER, PR status of BRCA patients from the BRCA TCGA publication supplementary: https://www.nature.com/articles/nature11412#supplementary-information filename: TCGA_Supplementary Tables 1-4.csv column: HER2_Final_Status But since it is from 2012, after some months, I downloaded the newer clinical data from cbioportal**: http://www.cbioportal.org/study?id=brca_tcga#clinical filename: data_bcr_clinical_data_patient.txt columns: IHC-Status, HER2 fish status However, I noticed that hundreds patients had different her2 status then they had in the old tcga publication: Barcode IHC-status(cbioportal) Her2 fish status (cbioportal) HER2_Final_Status(tcga publication) TCGA-A1-A0SH Equivocal Negative Negative TCGA-A2-A04U Negative Positive Negative TCGA-A2-A0T2 Negative Not Evaluated Negative TCGA-A8-A06R Positive Positive Equivocal At first, I thought there is a method to convert both ihc and fish statuses into one (final status). But I failed to find such method. In TCGA-A1-A0SH, it seems Fish is prefered. In TCGA-A2-A04U , it seems it is IHC. in TCGA-A8-A06R it is neither. **I verified that for those 4 patients, her2 status is consistent between cbioportal and current TCGA clinical files. (her2_fish_status, her2_status_by_ihc in clinical files) Thanks in advance, Maor

I tried rerunning a task just now, and I almost immediately received the error: This task ran into a problem during execution and did not finish. Execution error occured <sic> I'm assuming this has to do with AWS? Or is it something with my account? It happened 3 times in a row. I'll keep trying every 30 minutes I guess.

As a test of runtimes, I ran one of my dockerized apps without an input file. The app has been written to exit immediately, in the event an input file is not specified, with an error message. On my local machine the app behaves as expected on being run without an input file - immediate exit with a help message. However on CGC, without an input file, the time it took to complete was 3 minutes. Of this , 2m and 14s was spent in the queue and 46 s was spent in running. My question is why did the app run for 46s before exiting? Exit should have been immediate. What is the overhead time being spent in? TIA for any insight. Anjan

Hello, I am trying to run SomaticSniper and have uploaded a Normal BAM, Tumor BAM and Reference Sequence (.fasta). The run failed with exit code 139. Any ideas on how to fix this? Thank you!

SVS files are availble through the website: https://portal.gdc.cancer.gov/cases/9fe336a8-08a7-4fe7-bf45-afd6a8eb9c75 They are also available on the data portal as of May 21 2018: (same page, click on Files, it leads to data portal. You can see svs files for this case) However issue an API call to: https://cgc-datasets-api.sbgenomics.com/datasets/v0/tcga/cases/9FE336A8-08A7-4FE7-BF45-AFD6A8EB9C75/files No svs files are available.

H! I have been creating workflow which consists three main steps ( SAMtools View, SAMtools fadix and SAMtools Mpileup). Firstly, with the SAMtools View I filtered the input bam-file based on a bed-file that contains special regions of the third and fifteenth chromosomes. The input bam-file already aligned and sorted I took from the database. The bed-file was download from my computer and the firs line looks like this (3 193593144 193697811). After that the SAMtools Mpileup took file that contain only necessary chromosomes and the file with the indexed reference (as a reference, I used ucsc.hg19.fasta from the database). At the end of the workflow, I expect to see vcf-file with that contains information about the reference and alternative nucleotide of the third and fifteenth chromosomes. Unfortunately, I get the described file, but there are N in place of the reference allele. Please help me understand what is wrong with my reference file

Hi, I've built a simple tool using R, pushed it to the CGC repository and ran it. However, I have an issue with retrieving the outputs. It seems as the system cannot find them. For now, the tool produces two files: one called model.pdf and another model.txt. I set the glob values (outputs tab in the tool editor) to model.pdf and model.txt since their names are static. My initial thought of the issue was a wrong working directory. I couldn't find more details in the documentation. - How do I know, what is the working directory of the tool for the current analysis? Can I extract this information from the job or self variable? - Should provide information to mount specific directory? - What else could be wrong? For instance, I can successfully retrieve the stdout.txt (caught standard output) file. Also, the job.tree.log file shows stdout.txt available but no model.pdf nor model.txt. Thank you for the help.

I have download TCGA COAD expression data file which have data like shown below. How these values are calculated and what does means by these values. Hybridization REF TCGA-AA-A00E-01A-01R-A002-07 Composite Element REF log2 lowess normalized (cy5/cy3) collapsed by gene symbol ELMO2 -0.201 CREB3L1 2.3005 RPS11 -0.080375 PNMA1 -1.23175

Hi! I have a tool which analyzes .bam files and requires .bai files as secondary files. The secondary files settings are ok as the tool works well when a run on a single file is performed. However, when i try to make a batch task, it doesn't work and the error log says "unable to find index file for example.bam" or something like this. Is it a known bug? is there a way to overcome this problem? Thanks a lot!

Hello, I tried to do my own analysis on BAM files from SGDP project. Due to incompatible fasta file used to generate BAM file all my jobs encounter error. Therefore, I would like to ask you if you could please let me know how can I find the true reference fasta file this project. My error is this: A USER ERROR has occurred: Input files reference and features have incompatible contigs: No overlapping contigs found. reference contigs = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, X, Y, MT] features contigs = [chrM, chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY, chr1_gl000191_random, chr1_gl000192_random, chr4_ctg9_hap1, chr4_gl000193_random, chr4_gl000194_random, chr6_apd_hap1, chr6_cox_hap2, chr6_dbb_hap3, chr6_mann_hap4, chr6_mcf_hap5, chr6_qbl_hap6, chr6_ssto_hap7, chr7_gl000195_random, chr8_gl000196_random, chr8_gl000197_random, chr9_gl000198_random, chr9_gl000199_random, chr9_gl000200_random, chr9_gl000201_random, chr11_gl000202_random, chr17_ctg5_hap1, chr17_gl000203_random, chr17_gl000204_random, chr17_gl000205_random, chr17_gl000206_random, chr18_gl000207_random, chr19_gl000208_random, chr19_gl000209_random, chr21_gl000210_random, chrUn_gl000211, chrUn_gl000212, chrUn_gl000213, chrUn_gl000214, chrUn_gl000215, chrUn_gl000216, chrUn_gl000217, chrUn_gl000218, chrUn_gl000219, chrUn_gl000220, chrUn_gl000221, chrUn_gl000222, chrUn_gl000223, chrUn_gl000224, chrUn_gl000225, chrUn_gl000226, chrUn_gl000227, chrUn_gl000228, chrUn_gl000229, chrUn_gl000230, chrUn_gl000231, chrUn_gl000232, chrUn_gl000233, chrUn_gl000234, chrUn_gl000235, chrUn_gl000236, chrUn_gl000237, chrUn_gl000238, chrUn_gl000239, chrUn_gl000240, chrUn_gl000241, chrUn_gl000242, chrUn_gl000243, chrUn_gl000244, chrUn_gl000245, chrUn_gl000246, chrUn_gl000247, chrUn_gl000248, chrUn_gl000249]

I've set up a Volume to access files from a bucket I have under my own AWS account and copied a file into a project. Does this copy incur storage charges of its own?