Lab Notebook

015_gatk_realigner_target_creator CMD_015 001

DateTime: 2016-11-06T10:14:35

ProcessID: 23338

SchedulerID: 24604

2016/11/06 10:14:35: ERROR INFO 10:14:34,358 ProgressMeter - CHR_HG2288_HG2289_PATCH:81791701 7.786606808E9 22.0 m 0.0 s 18.1% 2.0 h 99.6 m
2016/11/06 10:15:04: ERROR INFO 10:15:04,379 ProgressMeter - CHR_HG2290_PATCH:148563737 7.990325538E9 22.5 m 0.0 s 18.5% 2.0 h 99.0 m
2016/11/06 10:15:34: ERROR INFO 10:15:34,397 ProgressMeter - CHR_HG2291_PATCH:94447413 8.161519783E9 23.0 m 0.0 s 19.0% 2.0 h 98.4 m
2016/11/06 10:16:04: ERROR INFO 10:16:04,412 ProgressMeter - CHR_HG986_PATCH:19802201 8.391011891E9 23.5 m 0.0 s 19.4% 2.0 h 98.0 m
2016/11/06 10:16:34: ERROR INFO 10:16:34,429 ProgressMeter - CHR_HG986_PATCH:210062001 8.391011891E9 24.0 m 0.0 s 19.8% 2.0 h 97.3 m
2016/11/06 10:17:04: ERROR INFO 10:17:04,459 ProgressMeter - CHR_HSCHR10_1_CTG2:4610201 8.773813252E9 24.5 m 0.0 s 20.2% 2.0 h 96.8 m
2016/11/06 10:17:34: ERROR INFO 10:17:34,498 ProgressMeter - CHR_HSCHR10_1_CTG3:60251601 8.907626737E9 25.0 m 0.0 s 20.7% 2.0 h 96.2 m
2016/11/06 10:18:04: ERROR INFO 10:18:04,519 ProgressMeter - CHR_HSCHR10_1_CTG4:112987701 9.041409955E9 25.5 m 0.0 s 21.1% 2.0 h 95.6 m
2016/11/06 10:18:34: ERROR INFO 10:18:34,559 ProgressMeter - CHR_HSCHR11_1_CTG1_2:23744769 9.318133297E9 26.0 m 0.0 s 21.5% 2.0 h 95.1 m
2016/11/06 10:19:04: ERROR INFO 10:19:04,628 ProgressMeter - CHR_HSCHR11_1_CTG2:73926701 9.445414877E9 26.5 m 0.0 s 21.9% 2.0 h 94.5 m
2016/11/06 10:19:34: ERROR INFO 10:19:34,634 ProgressMeter - CHR_HSCHR11_1_CTG3:123750385 9.649737259E9 27.0 m 0.0 s 22.3% 2.0 h 93.9 m
2016/11/06 10:20:04: ERROR INFO 10:20:04,660 ProgressMeter - CHR_HSCHR11_1_CTG6:40683501 9.850686005E9 27.5 m 0.0 s 22.8% 2.0 h 93.3 m
2016/11/06 10:20:34: ERROR INFO 10:20:34,698 ProgressMeter - CHR_HSCHR11_1_CTG7:96522469 9.988678305E9 28.0 m 0.0 s 23.2% 2.0 h 92.7 m
2016/11/06 10:21:04: ERROR INFO 10:21:04,712 ProgressMeter - CHR_HSCHR11_2_CTG1:5671001 1.0255952254E10 28.5 m 0.0 s 23.6% 2.0 h 92.2 m
2016/11/06 10:21:34: ERROR INFO 10:21:34,729 ProgressMeter - CHR_HSCHR11_2_CTG1_1:54736985 1.0392700482E10 29.0 m 0.0 s 24.1% 2.0 h 91.7 m
2016/11/06 10:22:04: ERROR INFO 10:22:04,747 ProgressMeter - CHR_HSCHR11_3_CTG1:107588101 1.0526129539E10 29.5 m 0.0 s 24.5% 2.0 h 91.1 m
2016/11/06 10:22:34: ERROR INFO 10:22:34,787 ProgressMeter - CHR_HSCHR12_1_CTG2:19459501 1.0794501043E10 30.0 m 0.0 s 24.9% 2.0 h 90.6 m
2016/11/06 10:23:04: ERROR INFO 10:23:04,799 ProgressMeter - CHR_HSCHR12_1_CTG2_1:69489597 1.0985847307E10 30.5 m 0.0 s 25.3% 2.0 h 90.0 m
2016/11/06 10:23:34: ERROR INFO 10:23:34,837 ProgressMeter - CHR_HSCHR12_2_CTG2:124556853 1.1072175231E10 31.0 m 0.0 s 25.8% 2.0 h 89.5 m
2016/11/06 10:24:04: ERROR INFO 10:24:04,883 ProgressMeter - CHR_HSCHR12_3_CTG2:41022629 1.1338994222E10 31.5 m 0.0 s 26.2% 2.0 h 88.9 m
2016/11/06 10:24:34: ERROR INFO 10:24:34,910 ProgressMeter - CHR_HSCHR12_3_CTG2_1:94876901 1.146092806E10 32.0 m 0.0 s 26.6% 2.0 h 88.3 m
2016/11/06 10:25:04: ERROR INFO 10:25:04,961 ProgressMeter - CHR_HSCHR12_4_CTG2_1:7140901 1.1727487128E10 32.5 m 0.0 s 27.0% 2.0 h 87.9 m
2016/11/06 10:25:34: ERROR INFO 10:25:34,998 ProgressMeter - CHR_HSCHR12_5_CTG2:49539901 1.1860768479E10 33.0 m 0.0 s 27.4% 2.0 h 87.4 m
2016/11/06 10:26:05: ERROR INFO 10:26:05,017 ProgressMeter - CHR_HSCHR12_5_CTG2_1:96671401 1.1994043716E10 33.5 m 0.0 s 27.8% 2.0 h 86.9 m
2016/11/06 10:26:35: ERROR INFO 10:26:35,051 ProgressMeter - CHR_HSCHR12_7_CTG2_1:5423701 1.2260598555E10 34.0 m 0.0 s 28.2% 2.0 h 86.5 m
2016/11/06 10:27:05: ERROR INFO 10:27:05,073 ProgressMeter - CHR_HSCHR13_1_CTG1:61145101 1.2393888033E10 34.5 m 0.0 s 28.7% 2.0 h 85.9 m
2016/11/06 10:27:35: ERROR INFO 10:27:35,097 ProgressMeter - CHR_HSCHR13_1_CTG3:15687401 1.262260452E10 35.0 m 0.0 s 29.1% 2.0 h 85.3 m
2016/11/06 10:28:05: ERROR INFO 10:28:05,119 ProgressMeter - CHR_HSCHR13_1_CTG4:90068401 1.2736974746E10 35.5 m 0.0 s 29.5% 2.0 h 84.8 m
2016/11/06 10:28:35: ERROR INFO 10:28:35,156 ProgressMeter - CHR_HSCHR13_1_CTG6:45120301 1.2965708428E10 36.0 m 0.0 s 30.0% 2.0 h 84.2 m
2016/11/06 10:29:05: ERROR INFO 10:29:05,198 ProgressMeter - CHR_HSCHR14_2_CTG1:8542701 1.3187122974E10 36.6 m 0.0 s 30.4% 2.0 h 83.7 m
2016/11/06 10:29:35: ERROR INFO 10:29:35,209 ProgressMeter - CHR_HSCHR14_3_CTG1:82644901 1.3294173836E10 37.1 m 0.0 s 30.8% 2.0 h 83.2 m
2016/11/06 10:30:05: ERROR INFO 10:30:05,248 ProgressMeter - CHR_HSCHR15_1_CTG1:35877593 1.352979965E10 37.6 m 0.0 s 31.2% 2.0 h 82.8 m
2016/11/06 10:30:35: ERROR INFO 10:30:35,278 ProgressMeter - CHR_HSCHR15_1_CTG8:9196601 1.371228039E10 38.1 m 0.0 s 31.6% 2.0 h 82.4 m
2016/11/06 10:31:05: ERROR INFO 10:31:05,318 ProgressMeter - CHR_HSCHR15_2_CTG3:95593301 1.3814283199E10 38.6 m 0.0 s 32.0% 2.0 h 81.8 m

Read More

014_bcftools_stats MAIN

DateTime: 2016-11-06T09:55:20

SchedulerID: 24603

2016/11/06 09:55:20: DEBUG Enqueuing command:

#TASK tags=Sample_5

bcftools stats -F /scratch/Reference_Genomes/Public/Vertebrate_mammalian/Homo_sapiens/ENSEMBL-release-81-GRCh38/Homo_sapiens.GRCh38.dna.toplevel.fa -s - /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.gz \

> /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.stats

2016/11/06 09:55:24: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 04 seconds

013_bcftools_filter CMD_013

DateTime: 2016-11-06T09:55:20

ProcessID: 24080

SchedulerID: 24602

2016/11/06 09:55:20: INFO Starting Job: 13
Cmd is #TASK tags=Sample_5
bcftools filter -O z \
-o /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_filteredQ10.aligned.sorted.vcf.gz \
-s LOWQUAL -i’%QUAL>10’ \
/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.gz
2016/11/06 09:55:20: INFO Finishing job 13 with ExitCode 0
2016/11/06 09:55:20: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 00 seconds

014_bcftools_stats CMD_014

DateTime: 2016-11-06T09:55:20

ProcessID: 24083

SchedulerID: 24603

2016/11/06 09:55:20: INFO Starting Job: 14
Cmd is #TASK tags=Sample_5
bcftools stats -F /scratch/Reference_Genomes/Public/Vertebrate_mammalian/Homo_sapiens/ENSEMBL-release-81-GRCh38/Homo_sapiens.GRCh38.dna.toplevel.fa -s - /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.gz \

> /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.stats

2016/11/06 09:55:24: INFO Finishing job 14 with ExitCode 0
2016/11/06 09:55:24: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 04 seconds

013_bcftools_filter MAIN

DateTime: 2016-11-06T09:55:20

SchedulerID: 24602

2016/11/06 09:55:20: DEBUG Enqueuing command:

#TASK tags=Sample_5

bcftools filter -O z \
-o /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_filteredQ10.aligned.sorted.vcf.gz \
-s LOWQUAL -i’%QUAL>10’ \
/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.gz
2016/11/06 09:55:20: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 00 seconds

012_tabix_index MAIN

DateTime: 2016-11-06T09:55:14

SchedulerID: 24601

2016/11/06 09:55:14: DEBUG Enqueuing command:

#TASK tags=Sample_5

tabix -p vcf /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.gz
2016/11/06 09:55:15: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 01 seconds

012_tabix_index CMD_012

DateTime: 2016-11-06T09:55:14

ProcessID: 23755

SchedulerID: 24601

2016/11/06 09:55:14: INFO Starting Job: 12
Cmd is #TASK tags=Sample_5
tabix -p vcf /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/samtools/Sample_5_aligned.sorted.vcf.gz
2016/11/06 09:55:15: INFO Finishing job 12 with ExitCode 0
2016/11/06 09:55:15: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 01 seconds

015_gatk_realigner_target_creator CMD_015

DateTime: 2016-11-06T09:52:24

ProcessID: 23338

SchedulerID: 24604

2016/11/06 09:52:24: INFO Starting Job: 15
Cmd is #TASK tags=Sample_5
mkdir -p /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/gatk && \
gatk -T RealignerTargetCreator \
-R /scratch/Reference_Genomes/Public/Vertebrate_mammalian/Homo_sapiens/ENSEMBL-release-81-GRCh38/Homo_sapiens.GRCh38.dna.toplevel.fa \
-I /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam \
-o /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/gatk/Sample_5_realignment_targets.list
2016/11/06 09:52:29: ERROR INFO 09:52:29,776 HelpFormatter - ——————————————————————————–
INFO 09:52:29,808 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.5-0-g36282e4, Compiled 2015/11/25 04:03:56
INFO 09:52:29,808 HelpFormatter - Copyright (c) 2010 The Broad Institute
INFO 09:52:29,808 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk
2016/11/06 09:52:29: ERROR INFO 09:52:29,814 HelpFormatter - Program Args: -T RealignerTargetCreator -R /scratch/Reference_Genomes/Public/Vertebrate_mammalian/Homo_sapiens/ENSEMBL-release-81-GRCh38/Homo_sapiens.GRCh38.dna.toplevel.fa -I /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam -o /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/gatk/Sample_5_realignment_targets.list
2016/11/06 09:52:29: ERROR INFO 09:52:29,835 HelpFormatter - Executing as gencore@compute-21-1.local on Linux 3.10.0-327.10.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_92-b15.
INFO 09:52:29,835 HelpFormatter - Date/Time: 2016/11/06 09:52:29
INFO 09:52:29,835 HelpFormatter - ——————————————————————————–
INFO 09:52:29,835 HelpFormatter - ——————————————————————————–
2016/11/06 09:52:30: ERROR INFO 09:52:30,188 GenomeAnalysisEngine - Strictness is SILENT
2016/11/06 09:52:30: ERROR INFO 09:52:30,945 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 1000
2016/11/06 09:52:30: ERROR INFO 09:52:30,984 SAMDataSource$SAMReaders - Initializing SAMRecords in serial
2016/11/06 09:52:31: ERROR INFO 09:52:31,167 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.17
2016/11/06 09:52:31: ERROR INFO 09:52:31,885 GenomeAnalysisEngine - Preparing for traversal over 1 BAM files
2016/11/06 09:52:32: ERROR INFO 09:52:32,156 GenomeAnalysisEngine - Done preparing for traversal
INFO 09:52:32,157 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING]
INFO 09:52:32,157 ProgressMeter - | processed | time | per 1M | | total | remaining
INFO 09:52:32,157 ProgressMeter - Location | sites | elapsed | sites | completed | runtime | runtime
2016/11/06 09:53:02: ERROR INFO 09:53:02,211 ProgressMeter - 1:138905645 1.25190144E8 30.0 s 0.0 s 0.3% 2.6 h 2.6 h
2016/11/06 09:53:32: ERROR INFO 09:53:32,233 ProgressMeter - 10:55160893 3.03646214E8 60.0 s 0.0 s 0.7% 2.4 h 2.4 h
2016/11/06 09:54:02: ERROR INFO 09:54:02,265 ProgressMeter - 11:95766281 4.78108724E8 90.0 s 0.0 s 1.1% 2.3 h 2.2 h
2016/11/06 09:54:32: ERROR INFO 09:54:32,298 ProgressMeter - 13:3006601 6.51115775E8 120.0 s 0.0 s 1.5% 2.2 h 2.2 h
2016/11/06 09:55:02: ERROR INFO 09:55:02,309 ProgressMeter - 14:61400613 8.25986215E8 2.5 m 0.0 s 1.9% 2.2 h 2.1 h
2016/11/06 09:55:32: ERROR INFO 09:55:32,338 ProgressMeter - 16:9951653 9.84394562E8 3.0 m 0.0 s 2.3% 2.2 h 2.2 h
2016/11/06 09:56:02: ERROR INFO 09:56:02,357 ProgressMeter - 18:16212357 1.164232652E9 3.5 m 0.0 s 2.7% 2.2 h 2.1 h
2016/11/06 09:56:32: ERROR INFO 09:56:32,398 ProgressMeter - 2:54156637 1.341234433E9 4.0 m 0.0 s 3.1% 2.2 h 2.1 h
2016/11/06 09:57:02: ERROR INFO 09:57:02,427 ProgressMeter - 2:233872033 1.520950529E9 4.5 m 0.0 s 3.5% 2.1 h 2.1 h

Read More

015_gatk_realigner_target_creator MAIN

DateTime: 2016-11-06T09:52:24

SchedulerID: 24604

2016/11/06 09:52:24: DEBUG Enqueuing command:

#TASK tags=Sample_5

mkdir -p /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/gatk && \
gatk -T RealignerTargetCreator \
-R /scratch/Reference_Genomes/Public/Vertebrate_mammalian/Homo_sapiens/ENSEMBL-release-81-GRCh38/Homo_sapiens.GRCh38.dna.toplevel.fa \
-I /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam \
-o /scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/gatk/Sample_5_realignment_targets.list
2016/11/06 11:51:52: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 119 minutes, 28 seconds

010_picard_markdups MAIN

DateTime: 2016-11-06T09:52:14

SchedulerID: 24599

2016/11/06 09:52:14: DEBUG Enqueuing command:

#TASK tags=Sample_5

picard -Xmx2g MarkDuplicates \
M=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_markDup_metrics_file.txt \
I=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_withrg.csorted.cleaned.aligned.bam \
O=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/picard/Sample_5_mdup.withrg.csorted.cleaned.aligned.bam \
MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=100 \
TMP_DIR=/scratch/gencore/nov_dalma_training/resequencing/data/analysis/Sample_5/bowtie2/tmp
2016/11/06 09:52:30: INFO Total execution time 0 years, 00 months, 0 days, 00 hours, 00 minutes, 16 seconds