今天被人问起如何看懂三代的下机数据，虽然解决了别人的问题，但感觉自己还是没有搞透。
基本的目录结构：
|-- HG002new_O1l_BP_P6_021315b_MB_100pM | |-- D01_1.c60e446d-f276-41fc-9384-ffa937e22683.tar.gz | |-- D01_2.19ee4f13-c420-4974-8262-cb1da56beccd.tar.gz | |-- D01_3.94e34f0a-eef3-4b71-8f1b-c9790dec647e.tar.gz | |-- D01_4.53ef7aed-e91e-46f9-bb71-8b021344b951.tar.gz | |-- D01_5.55b1f7cb-ad44-4afb-bf2b-5c34fcb0a210.tar.gz | `-- D01_6.b9b564dc-b794-4a7f-bc3b-854a7bc32887.tar.gz `-- pacbio_README
解压后的目录结构：
Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.1.bax.h5 Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.1.log Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.1.subreads.fasta Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.1.subreads.fastq Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.2.bax.h5 Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.2.log Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.2.subreads.fasta Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.2.subreads.fastq Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.3.bax.h5 Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.3.log Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.3.subreads.fasta Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.3.subreads.fastq Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.bas.h5 Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.sts.csv Analysis_Results/m140612_020550_42156_c100652082550000001823118110071460_s1_p0.sts.xml m140612_020550_42156_c100652082550000001823118110071460_s1_p0.1.xfer.xml m140612_020550_42156_c100652082550000001823118110071460_s1_p0.2.xfer.xml m140612_020550_42156_c100652082550000001823118110071460_s1_p0.3.xfer.xml m140612_020550_42156_c100652082550000001823118110071460_s1_p0.mcd.h5 m140612_020550_42156_c100652082550000001823118110071460_s1_p0.metadata.xml
可以看到数据是以HDF5的格式存储的，格式介绍：PacBio Sequences的HDF5格式
那么，上面目录和文件名都有哪些含义呢？仔细的看说明文档就会知道
============
Introduction
============
These directories contain data from PacBio sequencing of HG002, HG003, and HG004,
which are the son, father, and mother, respectively, in a trio of Ashkenazim Jewish
ancestry from the Personal Genome Project and are candidate Reference Materials being
characterized by NIST and the Genome in a Bottle Consortium. The coverage is approximately
69X, 32X, and 30X for HG002, HG003, and HG004, respectively. 89.7% of the data is
from P6-C4 chemistry, and the remaining from P5-C3 chemistry. Library preparation was
performed at NIST and sequencing was performed at Mt. Sinai School of Medicine.
Details of the library preparation, sequencing, and data are provided below.
==============================================
Library Preparation (include reagent versions)
==============================================
SMRTbell library preparation and sequencing of HG002, HG003, and HG004 AJ Trio gDNA
DNA library preparation and sequencing was performed according to the manufacturer's
instructions with noted modifications. Following the Pacific Biosciences Protocol,
"20-kb Template Preparation Using Blue Pippin Size-Selection System", library
preparation was performed using the Pacific Biosciences SMRTbell Template Prep Kit 1.0
(PN # 100-259-100). In short, 10 µg of extracted, high-quality, genomic DNA from eac
of HG-002, HG-003, and HG-004, the AJ trio, were used for library preparation. Genomic
DNA extracts were verified with the Life Technologies Qubit 2.0 Fluorometer using the
High Sensitivity dsDNA assay (PN# Q32851) to quantify the mass of double-stranded DNA
present. After quantification, each sample was diluted to 150 µL, using kit provide
EB, yielding a concentration of ~ 66 ng/µL. The 150 µL aliquots were individuall
pipetted into the top chambers of Covaris G-tube (PN# 520079) spin columns and sheared
for 60 seconds at 4500 rpm using an Eppendorf 5424 benchtop centrifuge. Once complete,
the spin columns were flipped after verifying that all DNA was now in the lower chamber.
The columns were spun for another 60 seconds at 4500 rpm to further shear the DNA and
place the aliquot back into the upper chamber. In some cases G-tubes were centrifuged
2-3 times, in both directions to ensure all volume had passed into the appropriate
chamber. Shearing resulted in a ~20,000 bp DNA fragments verified using an Agilent
Bioanlyzer DNA 12000 gel chip (PN# 5067-1508). The sheared DNA isolates were then
purified using a 0.5X AMPure PB magnetic bead purification step (0.5X AMPure PB beads
added, by volume, to each DNA sample, vortexed for 10 minutes at 2,000 rpm, followed
by two washes with 70% alcohol and finally eluted in EB). This AMPure purification
step assures removal of any small fragment and/or biological contaminant. The sheared
DNA concentration was then measured using the Qubit High Sensitivity dsDNA assay.
These values were used to calculate actual input mass for library preparation following
shearing and purification.
After purification, ~8-9 mg of each purified sheared sample went through the following
library preparation process per the aforementioned protocol:
------------------------------------
ExcVII Treatment
(remove) ssDNA ends
|
DNA Fragment Damage Repair
|
DNA Fragment End Repair
|
Purify Blunt-Ended DNA Fragments
|
Blunt End SMRTbell Adapter Ligation
|
Exonuclease Treatment
(remove failed ligation product)
|
Size Selection using BluePippin
|
Clean and Concentrate Final Library
-------------------------------------
All library preparation reaction volumes were scaled to accommodate input mass for a
given sample. Library size selection was performed using the Sage Science BluePippin
0.75% Agarose, Dye Free, PacBio ~20kb templates, S1 cassette (PN# PAC20KB). Size
selections were run overnight to maximize recovered mass. Approximately 2-5 mg of
prepared libraries were size selected using a 10 kb start and 50 kb end in "Range" mode.
This selection is necessary to narrow the library distribution and maximize the SMRTbell
sub-read length for the best de novo assembly possible. Without selection, smaller
2000 - 10,000 bp molecules dominate the zero-mode waveguide loading distribution,
decreasing the sub-read length. Size-selection was confirmed using pre and post size
selected DNA using an Agilent DNA 12000 chip. Final library mass was measured using the
Qubit High Sensitivity dsDNA Assay. Approximately 15-20% of the initial gDNA input mass
resulted after elution from the agarose cassette, which was enough yield to proceed to
primer annealing and DNA sequencing on the PacBio RSII instrument. This entire library
preparation and selection strategy was conducted 7, 2 and 2 times across HG002, HG003,
and HG004 respectively, to provide enough library for the duration of this project.
==================================================
Sequencing (include chemistry/instrument versions)
==================================================
Sequencing reflects the P6-C4 sequencing enzyme and chemistry, respectively. (Note that
10.3 % of the data was collected using the P5-C3 enzyme/chemistry prior to the release
of the P6-C4 enzyme and chemistry.) Primer was annealed to the size-selected SMRTbell
with the full-length libraries (80ºC for 2 minute 30 followed by decreasing thetemperature
by 0.1º/s to 25Cº. To prepare the polymerase-template complex, the SMRTbell template complex
was then bound to the P6 enzyme using the Pacific Biosciences DNA Polymerase Binding Kit
P6 v2 (PN# 100-372-700). A ratio of 10:1, polymerase to SMRTbell at 0.5 nM, was prepared
and incubated for 4 hours at 30ºC and then held at 4ºC until ready for magbead loading
prior to sequencing. The Magnetic bead-loading step was conducted using the Pacific
Biosciences MagBead Kit (PN# 100-133-600) at 4ºC for 60-minutes permanufacturer's guidelines.
The magbead-loaded, polymerase-bound, SMRTbell libraries were placed onto the RSII instrument
at a sequencing concentration of 100 to 40 pM to optimize loading across various SMRTcells.
Sequencing was performed using the C4 chemistry provided in the Pacific Biosciences DNA
Sequence Bundle 4.0 (PN# 100-356-400). The RSII was then configured for 240-minute
continuous sequencing runs.
========================================
Preliminary Analyses and Quality Control
========================================
Assuming a 3.2 Gb human genome, sequencing was conducted to approximately 69X, 32X,
and 30X coverage for HG002, HG003, and HG004 across 292, 139, and 132 SMRTcells,
respectively. 27.4M, 13.2M, and 12.4M subreads were generated resulting in 220.0,
101.6, and 94.9 Gb of sequence data with sub-readlength N50 values of 11,087, 10,728,
and 10,629 basepairs.
================================
File/Directory naming convention
================================
The file/directory naming convention is defined as follows:
[SampleName]/[WellName]_[CollectionNumber].[UUID].tar.gz
Note that SampleName may contain other genomes in the name, but the data directories
only contain data from HG002, HG003, and HG004.
For example, for SampleName of HG002new_O1_BP_P6_021815_MB_105pM,
WellName of A01, and CollectionNumber of 3, you will see a tar.gz file in
HG002new_O1_BP_P6_021815_MB_105pM directory with name A01_3.[UUID].tar.gz
The UUID is currently used for only hashing purpose.
The tar.gz file containts the raw SMRTPortal data including following contents:
tar.gz
| [movie name].1.xfer.xml
| [movie name].2.xfer.xml
| [movie name].3.xfer.xml
| [movie name].mcd.h5
| [movie name].metadata.xml
---Analysis_Results
| [movie name].1.bax.h5
| [movie name].1.log
| [movie name].1.subreads.fasta
| [movie name].1.subreads.fastq
| [movie name].2.bax.h5
| [movie name].2.log
| [movie name].2.subreads.fasta
| [movie name].2.subreads.fastq
| [movie name].3.bax.h5
| [movie name].3.log
| [movie name].3.subreads.fasta
| [movie name].3.subreads.fastq
| [movie name].bas.h5
| [movie name].sts.csv
| [movie name].sts.xml
The metadata.xml contains all the metadata of this particular sample in the xml format;
for example, in the TemplatePrep field you might see "DNA Template Prep Kit 2.0 (3Kb - 10Kb),"
and in the BindingKit field you might see "DNA/Polymerase Binding Kit P6," etc.
For information about bas.h5/bax.h5 files, please see:
http://files.pacb.com/software/instrument/2.0.0/bas.h5%20Reference%20Guide.pdf
For information about subreads, please see:
https://speakerdeck.com/pacbio/track-1-de-novo-assembly