2016/11/06 09:52:14: DEBUG Enqueuing command:
#TASK tags=Sample_5
picard -Xmx2g BuildBamIndex \
I=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam \
TMP_DIR=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp
2016/11/06 09:52:20: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 07 seconds
2016/11/06 09:52:14: INFO Starting Job: 8
Cmd is #TASK tags=Sample_5
picard -Xmx2g BuildBamIndex \
I=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam \
TMP_DIR=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp
2016/11/06 09:52:15: ERROR [Sun Nov 06 09:52:15 GST 2016] picard.sam.BuildBamIndex INPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam TMP_DIR=[/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp] VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
2016/11/06 09:52:15: ERROR [Sun Nov 06 09:52:15 GST 2016] Executing as gencore@compute-21-1.local on Linux 3.10.0-327.10.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_92-b15; Picard version: 2.5.0(2c370988aefe41f579920c8a6a678a201c5261c1_1466708365)
2016/11/06 09:52:15: ERROR WARNING: BAM index file /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bai is older than BAM /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam
2016/11/06 09:52:20: ERROR INFO 2016-11-06 09:52:20 BuildBamIndex Successfully wrote bam index file /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bai
[Sun Nov 06 09:52:20 GST 2016] picard.sam.BuildBamIndex done. Elapsed time: 0.08 minutes.
Runtime.totalMemory()=1519386624
2016/11/06 09:52:20: INFO Finishing job 8 with ExitCode 0
2016/11/06 09:52:20: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 06 seconds
2016/11/06 09:52:14: INFO Starting Job: 10
Cmd is #TASK tags=Sample_5
picard -Xmx2g MarkDuplicates \
M=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_markDup_metrics_file.txt \
I=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam \
O=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_mdup.withrg.csorted.cleaned.aligned.bam \
MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=100 \
TMP_DIR=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp
2016/11/06 09:52:15: ERROR [Sun Nov 06 09:52:15 GST 2016] picard.sam.markduplicates.MarkDuplicates MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=100 INPUT=[/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam] OUTPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_mdup.withrg.csorted.cleaned.aligned.bam METRICS_FILE=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_markDup_metrics_file.txt TMP_DIR=[/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp] MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP=50000 SORTING_COLLECTION_SIZE_RATIO=0.25 REMOVE_SEQUENCING_DUPLICATES=false TAGGING_POLICY=DontTag REMOVE_DUPLICATES=false ASSUME_SORTED=false DUPLICATE_SCORING_STRATEGY=SUM_OF_BASE_QUALITIES PROGRAM_RECORD_ID=MarkDuplicates PROGRAM_GROUP_NAME=MarkDuplicates READ_NAME_REGEX=
2016/11/06 09:52:15: ERROR [Sun Nov 06 09:52:15 GST 2016] Executing as gencore@compute-21-1.local on Linux 3.10.0-327.10.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_92-b15; Picard version: 2.5.0(2c370988aefe41f579920c8a6a678a201c5261c1_1466708365)
2016/11/06 09:52:15: ERROR INFO 2016-11-06 09:52:15 MarkDuplicates Start of doWork freeMemory: 1509865608; totalMemory: 1519386624; maxMemory: 1908932608
INFO 2016-11-06 09:52:15 MarkDuplicates Reading input file and constructing read end information.
INFO 2016-11-06 09:52:15 MarkDuplicates Will retain up to 7342048 data points before spilling to disk.
2016/11/06 09:52:21: ERROR INFO 2016-11-06 09:52:21 MarkDuplicates Read 239438 records. 0 pairs never matched.
2016/11/06 09:52:22: ERROR INFO 2016-11-06 09:52:22 MarkDuplicates After buildSortedReadEndLists freeMemory: 1425083440; totalMemory: 1519386624; maxMemory: 1908932608
INFO 2016-11-06 09:52:22 MarkDuplicates Will retain up to 59654144 duplicate indices before spilling to disk.
2016/11/06 09:52:22: ERROR INFO 2016-11-06 09:52:22 MarkDuplicates Traversing read pair information and detecting duplicates.
2016/11/06 09:52:22: ERROR INFO 2016-11-06 09:52:22 MarkDuplicates Traversing fragment information and detecting duplicates.
2016/11/06 09:52:23: ERROR INFO 2016-11-06 09:52:23 MarkDuplicates Sorting list of duplicate records.
2016/11/06 09:52:23: ERROR
2016/11/06 09:52:23: ERROR INFO 2016-11-06 09:52:23 MarkDuplicates After generateDuplicateIndexes freeMemory: 1215927760; totalMemory: 1706557440; maxMemory: 1908932608
2016/11/06 09:52:23: ERROR INFO 2016-11-06 09:52:23 MarkDuplicates Marking 59141 records as duplicates.
INFO 2016-11-06 09:52:23 MarkDuplicates Found 10 optical duplicate clusters.
2016/11/06 09:52:23: ERROR INFO 2016-11-06 09:52:23 MarkDuplicates Reads are assumed to be ordered by: coordinate
2016/11/06 09:52:23: ERROR
2016/11/06 09:52:30: ERROR INFO 2016-11-06 09:52:30 MarkDuplicates Before output close freeMemory: 1692313096; totalMemory: 1706557440; maxMemory: 1908932608
2016/11/06 09:52:30: ERROR INFO 2016-11-06 09:52:30 MarkDuplicates After output close freeMemory: 1692378192; totalMemory: 1706557440; maxMemory: 1908932608
2016/11/06 09:52:30: ERROR [Sun Nov 06 09:52:30 GST 2016] picard.sam.markduplicates.MarkDuplicates done. Elapsed time: 0.25 minutes.
Runtime.totalMemory()=1706557440
2016/11/06 09:52:30: INFO Finishing job 10 with ExitCode -1
2016/11/06 09:52:30: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 16 seconds
2016/11/06 09:51:57: DEBUG Enqueuing command:
#TASK tags=Sample_5
picard -Xmx2g CollectMultipleMetrics \
TMP_DIR=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp \
I=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam \
O=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned \
PROGRAM=CollectAlignmentSummaryMetrics PROGRAM=CollectInsertSizeMetrics PROGRAM=QualityScoreDistribution PROGRAM=MeanQualityByCycle
2016/11/06 09:52:16: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 19 seconds
2016/11/06 09:51:57: DEBUG Enqueuing command:
#TASK tags=Sample_5
picard -Xmx2g AddOrReplaceReadGroups \
I=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam \
O=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam \
SORT_ORDER=coordinate RGID=1 RGLB=1 RGPL=illumina RGPU=unit1 RGSM=Sample_5 RGCN=NYUAD
2016/11/06 09:52:08: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 11 seconds
2016/11/06 09:51:57: INFO Starting Job: 7
Cmd is #TASK tags=Sample_5
picard -Xmx2g AddOrReplaceReadGroups \
I=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam \
O=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam \
SORT_ORDER=coordinate RGID=1 RGLB=1 RGPL=illumina RGPU=unit1 RGSM=Sample_5 RGCN=NYUAD
2016/11/06 09:51:58: ERROR [Sun Nov 06 09:51:58 GST 2016] picard.sam.AddOrReplaceReadGroups INPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam OUTPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam SORT_ORDER=coordinate RGID=1 RGLB=1 RGPL=illumina RGPU=unit1 RGSM=Sample_5 RGCN=NYUAD VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
2016/11/06 09:51:58: ERROR [Sun Nov 06 09:51:58 GST 2016] Executing as gencore@compute-21-1.local on Linux 3.10.0-327.10.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_92-b15; Picard version: 2.5.0(2c370988aefe41f579920c8a6a678a201c5261c1_1466708365)
2016/11/06 09:51:58: ERROR INFO 2016-11-06 09:51:58 AddOrReplaceReadGroups Created read group ID=1 PL=illumina LB=1 SM=Sample_5
2016/11/06 09:52:08: ERROR [Sun Nov 06 09:52:08 GST 2016] picard.sam.AddOrReplaceReadGroups done. Elapsed time: 0.17 minutes.
Runtime.totalMemory()=1519386624
2016/11/06 09:52:08: INFO Finishing job 7 with ExitCode 0
2016/11/06 09:52:08: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 11 seconds
2016/11/06 09:51:57: INFO Starting Job: 9
Cmd is #TASK tags=Sample_5
picard -Xmx2g CollectMultipleMetrics \
TMP_DIR=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp \
I=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam \
O=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned \
PROGRAM=CollectAlignmentSummaryMetrics PROGRAM=CollectInsertSizeMetrics PROGRAM=QualityScoreDistribution PROGRAM=MeanQualityByCycle
2016/11/06 09:51:58: ERROR [Sun Nov 06 09:51:58 GST 2016] picard.analysis.CollectMultipleMetrics INPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam OUTPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned PROGRAM=[CollectAlignmentSummaryMetrics, CollectBaseDistributionByCycle, CollectInsertSizeMetrics, MeanQualityByCycle, QualityScoreDistribution] TMP_DIR=[/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp] ASSUME_SORTED=true STOP_AFTER=0 METRIC_ACCUMULATION_LEVEL=[ALL_READS] VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
2016/11/06 09:51:58: ERROR [Sun Nov 06 09:51:58 GST 2016] Executing as gencore@compute-21-1.local on Linux 3.10.0-327.10.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_92-b15; Picard version: 2.5.0(2c370988aefe41f579920c8a6a678a201c5261c1_1466708365)
2016/11/06 09:52:02: ERROR INFO 2016-11-06 09:52:02 RExecutor Executing R script via command: Rscript /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp/script5169277710540645794.R /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned.base_distribution_by_cycle_metrics /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned.base_distribution_by_cycle.pdf Sample_5_csorted.cleaned.aligned.bam
2016/11/06 09:52:05: ERROR INFO 2016-11-06 09:52:05 ProcessExecutor null device
INFO 2016-11-06 09:52:05 ProcessExecutor 1
2016/11/06 09:52:12: ERROR INFO 2016-11-06 09:52:12 RExecutor Executing R script via command: Rscript /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp/script7011113815525516007.R /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned.insert_size_metrics /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned.insert_size_histogram.pdf Sample_5_csorted.cleaned.aligned.bam
2016/11/06 09:52:14: ERROR INFO 2016-11-06 09:52:14 ProcessExecutor null device
INFO 2016-11-06 09:52:14 ProcessExecutor 1
2016/11/06 09:52:14: ERROR INFO 2016-11-06 09:52:14 RExecutor Executing R script via command: Rscript /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp/script4094347614302844414.R /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned.quality_by_cycle_metrics /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned.quality_by_cycle.pdf Sample_5_csorted.cleaned.aligned.bam
2016/11/06 09:52:15: ERROR INFO 2016-11-06 09:52:15 ProcessExecutor null device
INFO 2016-11-06 09:52:15 ProcessExecutor 1
2016/11/06 09:52:15: ERROR INFO 2016-11-06 09:52:15 RExecutor Executing R script via command: Rscript /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp/script2018843830008339836.R /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned.quality_distribution_metrics /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_collmulmetr.csorted.cleaned.aligned.quality_distribution.pdf Sample_5_csorted.cleaned.aligned.bam
2016/11/06 09:52:16: ERROR INFO 2016-11-06 09:52:16 ProcessExecutor null device
INFO 2016-11-06 09:52:16 ProcessExecutor 1
2016/11/06 09:52:16: ERROR [Sun Nov 06 09:52:16 GST 2016] picard.analysis.CollectMultipleMetrics done. Elapsed time: 0.31 minutes.
Runtime.totalMemory()=1446510592
2016/11/06 09:52:16: INFO Finishing job 9 with ExitCode 0
2016/11/06 09:52:16: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 19 seconds
2016/11/06 09:51:43: DEBUG Enqueuing command:
#TASK tags=Sample_5
picard -Xmx2g SortSam \
INPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_cleaned.aligned.bam \
O=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam \
SO=coordinate \
TMP_DIR=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp
2016/11/06 09:51:52: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 09 seconds
2016/11/06 09:51:43: INFO Starting Job: 4
Cmd is #TASK tags=Sample_5
picard -Xmx2g SortSam \
INPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_cleaned.aligned.bam \
O=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam \
SO=coordinate \
TMP_DIR=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp
2016/11/06 09:51:44: ERROR [Sun Nov 06 09:51:44 GST 2016] picard.sam.SortSam INPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_cleaned.aligned.bam OUTPUT=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_csorted.cleaned.aligned.bam SORT_ORDER=coordinate TMP_DIR=[/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp] VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
2016/11/06 09:51:44: ERROR [Sun Nov 06 09:51:44 GST 2016] Executing as gencore@compute-21-1.local on Linux 3.10.0-327.10.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_92-b15; Picard version: 2.5.0(2c370988aefe41f579920c8a6a678a201c5261c1_1466708365)
2016/11/06 09:51:48: ERROR INFO 2016-11-06 09:51:48 SortSam Finished reading inputs, merging and writing to output now.
2016/11/06 09:51:52: ERROR [Sun Nov 06 09:51:52 GST 2016] picard.sam.SortSam done. Elapsed time: 0.14 minutes.
Runtime.totalMemory()=1519386624
2016/11/06 09:51:52: INFO Finishing job 4 with ExitCode -1
2016/11/06 09:51:52: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 09 seconds
2016/11/06 09:51:23: INFO Starting Job: 11
Cmd is #TASK tags=Sample_5
samtools mpileup -ugf /scratch/Reference_Genomes/Public/Vertebrate_mammalian/Homo_sapiens/ENSEMBL-release-81-GRCh38/Homo_sapiens.GRCh38.dna.toplevel.fa \
/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.bam | bcftools call -vmO z -o \
/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.gz
2016/11/06 09:51:23: ERROR [mpileup] 1 samples in 1 input files
2016/11/06 09:51:23: ERROR Note: Neither –ploidy nor –ploidy-file given, assuming all sites are diploid
2016/11/06 09:51:23: ERROR
2016/11/06 09:55:06: INFO Finishing job 11 with ExitCode 0
2016/11/06 09:55:06: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 03 minutes, 43 seconds