- Research article
- Open Access
Cross-platform comparison of SYBR® Green real-time PCR with TaqMan PCR, microarrays and other gene expression measurement technologies evaluated in the MicroArray Quality Control (MAQC) study
https://doi.org/10.1186/1471-2164-9-328
© Arikawa et al; licensee BioMed Central Ltd. 2008
- Received: 26 November 2007
- Accepted: 11 July 2008
- Published: 11 July 2008
Abstract
Background
The MicroArray Quality Control (MAQC) project evaluated the inter- and intra-platform reproducibility of seven microarray platforms and three quantitative gene expression assays in profiling the expression of two commercially available Reference RNA samples (Nat Biotechnol 24:1115-22, 2006). The tested microarrays were the platforms from Affymetrix, Agilent Technologies, Applied Biosystems, GE Healthcare, Illumina, Eppendorf and the National Cancer Institute, and quantitative gene expression assays included TaqMan® Gene Expression PCR Assay, Standardized (Sta) RT-PCR™ and QuantiGene®. The data showed great consistency in gene expression measurements across different microarray platforms, different technologies and test sites. However, SYBR® Green real-time PCR, another common technique utilized by half of all real-time PCR users for gene expression measurement, was not addressed in the MAQC study. In the present study, we compared the performance of SYBR Green PCR with TaqMan PCR, microarrays and other quantitative technologies using the same two Reference RNA samples as the MAQC project. We assessed SYBR Green real-time PCR using commercially available RT2 Profiler™ PCR Arrays from SuperArray, containing primer pairs that have been experimentally validated to ensure gene-specificity and high amplification efficiency.
Results
The SYBR Green PCR Arrays exhibit good reproducibility among different users, PCR instruments and test sites. In addition, the SYBR Green PCR Arrays have the highest concordance with TaqMan PCR, and a high level of concordance with other quantitative methods and microarrays that were evaluated in this study in terms of fold-change correlation and overlap of lists of differentially expressed genes.
Conclusion
These data demonstrate that SYBR Green real-time PCR delivers highly comparable results in gene expression measurement with TaqMan PCR and other high-density microarrays.
Keywords
- Microarray Platform
- Discordant Gene
- MAQC Project
- Quantitative Platform
- MAQC Study
Background
Gene expression research is a rapidly evolving field with recent advances in technologies aimed at multi-gene expression profiling and high throughput screening. Technologies like high-density DNA microarrays enable one to perform parallel gene expression profiling in the scale of tens of thousands of genes in a single experiment [1, 2]. Quantitative real-time-PCR, though lacking the scale of microarrays, is a rapid, sensitive and less complex method for gene expression analysis and offers an alternative approach for parallel profiling of multiple targets as well as a time-saving means to validate microarray results.
With many different technologies available for gene expression measurement, the need to compare the results obtained from different platforms and technologies and thus the reliability and biological significance of those results becomes evident. Moreover, concerns regarding the reliability and consistency of the microarray technology from different suppliers, different test sites and when using different methods for data processing and normalization have been raised [3–7]. To address those concerns, scientists from the US Food and Drug Administration (FDA) established the MicroArray Quality Control (MAQC) consortium to evaluate the performance of several microarray platforms as well as three quantitative gene expression assays [8–13]. The microarray platforms were from Affymetrix (AFX), Agilent Techonologies (one-color protocol (AG1) or two-color protocol (AGL)), Applied Biosystems (ABI), GE Healthcare (GEH), Illumina (ILM), Eppendorf (EPP) and the National Cancer Institute (NCI), and the three quantitative assays were TaqMan® Gene Expression Assay (Applied Biosystems, Foster City, CA), Standardized (Sta) RT-PCR™ (Gene Express, Inc., Toledo, OH) and QuantiGene® (Panomics, Inc., Fremont, CA).
Reports from the Phase 1 study of the MAQC project, which profiled two standardized reference RNA samples, contain important findings on the performance of different expression measurement technologies and give insights into the level of cross-platform comparability among different technologies [8–13]. The comprehensive data sets generated from this MAQC effort showed that great inter-site and cross-platform consistency can be achieved among different technologies [8, 12]. Importantly, the selection criteria used to define differentially expressed genes has a substantial impact on the overlap of the resulting gene lists, with gene lists generated by fold change ranking being more reproducible than those obtained by t-test P value ranking. With these findings, the MAQC Consortium recommends fold change ranking using a nonstringent P-value cutoff for gene selection. Another important goal attained by this project is to generate a thoroughly characterized reference data set against which new modifications in the existing microarray platforms and other expression measurement technologies can be compared and validated, and laboratory performance can be assessed. This was accomplished by providing the community with two commercially available high-quality human reference RNA samples that can be used as a tool for calibration and quality control as well as for performance assessment and validation of assays. The two RNA samples used in the MAQC project were the Stratagene Universal Human Reference RNA (comprised of RNA from ten different cell lines) and the Ambion Human Brain Reference RNA. Extremely large lots of these two reference RNAs were produced under stringent quality-control procedures. This has allowed researchers to assess the performance of their assays over time using the same RNA samples from identical manufacturing lots and to compare their results with the MAQC data set.
The platforms evaluated in the MAQC project can be categorized into either hybridization-based or PCR-based technologies. Different platforms of microarray and QuantiGene assays belong to hybridization-based technology. For microarrays, RNA samples are labeled with a tag (biotin or a fluorophore) followed by hybridization to immobilized gene-specific probes and fluorophore-based detection. QuantiGene is a sandwich nucleic acid hybridization system that detects RNA directly [14]. Targets are captured through joint hybridization of multiple probes, and the complex is detected by signal amplification through a branched DNA amplifier and chemiluminescence signal production. TaqMan Gene Expression Assays and StaRT-PCR are PCR-based techniques. StaRT-PCR is a competitive end-point PCR-based assay. A standardized mixture of internal standard (SMIS) competitive templates is added to the reverse transcribed products prior to PCR. The individual endpoint StaRT-PCR products are then separated by size and quantified by high-throughput microfluidic electrophoresis [15]. The TaqMan assay is based on real-time PCR using a fluorescent dye to monitor the amplification of target genes by DNA polymerase. It employs a target-specific, dual-labeled, fluorogenic hybridization probe with a quencher on the 3' end to be hydrolyzed by the 5' to 3' exonuclease activity of Taq polymerase during the extension step [16].
Real-time PCR is widely considered the gold standard for gene expression measurement due to its high assay specificity, high detection sensitivity and wide linear dynamic range. In addition to the TaqMan assay, the SYBR® Green PCR assay is another commonly used real-time PCR technique which is employed by half of all real-time PCR users [17]. Despite its widespread use, this technique was surprisingly not included as part of the MAQC project. SYBR Green PCR is widely used because of the ease in designing the assays and its relatively low setup and running costs. Unlike TaqMan fluorescent probes, SYBR Green dye intercalates into double-stranded DNA to monitor the amplification of target gene specifically initiated by gene-specific primers. One drawback of SYBR Green assays, however, is that the dye is non-specific and can generate false positive signals if non-specific products or primer-dimers are present in the assay. Those problems can be addressed by carefully designing the primers and validating the PCR products with dissociation curve analysis immediately after PCR. In addition, other approaches have been practiced to further increase the specificity of SYBR Green detection, such as a "hot start" strategy using a DNA polymerase that requires heat activation, or acquisition of fluorescence signals at a temperature slightly below the melting temperature of the desired amplicon but above which nonspecific primer-dimer related products will denature and produce minimal signals [18, 19].
In the present study, we have evaluated the performance of SuperArray's SYBR Green real-time PCR assays in profiling the same two reference RNA samples analyzed by the MAQC Consortium. Using the MAQC data sets available from the public database [8, 12, 20], we conducted similar analyses for the RT2 Profiler PCR Arrays and compared our expression profiling results with those generated from the three quantitative technologies (TaqMan, StaRT- PCR and QuantiGene) as well as from five of the commercial microarray platforms (AFX, AG1, ABI, GEH and ILM) examined in the MAQC project [8, 12].
Results
Assay Performance of SYBR Green Real-time PCR
Gene list of the custom SYBR Green RT2 Profiler PCR Array showing the overlaps with the other three quantitative gene analysis technologies
Probes present in all three quantitative platforms | Gene Symbol | TaqMan and QuantiGene | Gene Symbol | TaqMan and StaRT-PCR | Gene Symbol |
---|---|---|---|---|---|
1 | ABCD2 | 42 | ABCC2 | 72 | ANXA1 |
2 | ADM | 43 | ABCD1 | 73 | ARHGDIB |
3 | AES | 44 | ABCG5 | 74 | ATP1B2 |
4 | ANXA5 | 45 | AFP | 75 | CDK8 |
5 | BAG1 | 46 | APC | 76 | DAP |
6 | BRCA2 | 47 | APOB | 77 | ERCC5 |
7 | CCNA2 | 48 | APOH | 78 | GSTP1 |
8 | CDK9 | 49 | BAG4 | 79 | GSTT1 |
9 | CDKN1A | 50 | CCL20 | 80 | NTRK3 |
10 | CYP1B1 | 51 | CDK5R1 | 81 | PCNA |
11 | DAD1 | 52 | CHEK1 | 82 | PPP3CA |
12 | DPP4 | 53 | CHGA | 83 | PTCH |
13 | FADD | 54 | COL1A1 | 84 | TP53 |
14 | FGF9 | 55 | CSNK2A2 | ||
15 | FOXA1 | 56 | DRD5 | House Keeping Genes | Gene Symbol |
16 | FURIN | 57 | EDN1 | 85 | B2M |
17 | G6PD | 58 | EGFR | 86 | ACTB |
18 | ICAM1 | 59 | EIF2AK2 | 87 | GAPDH |
19 | IGF2R | 60 | ERBB4 | 88 | HPRT1 |
20 | IGFBP2 | 61 | GFAP | 89 | POLR2A |
21 | IGFBP5 | 62 | IGF2 | 90 | RPL13A |
22 | IL8 | 63 | IGFBP1 | ||
23 | INPPL1 | 64 | JAK2 | ||
24 | JUN | 65 | JUNB | ||
25 | KCNS3 | 66 | KCNC1 | ||
26 | KDR | 67 | MIF | ||
27 | KIT | 68 | MMP1 | ||
28 | LDLR | 69 | RAD51 | ||
29 | MAP2K6 | 70 | STAT4 | ||
30 | MAP3K14 | 71 | TFF1 | ||
31 | MX2 | ||||
32 | MYB | ||||
33 | MYC | ||||
34 | PTGS2 | ||||
35 | RAD52 | ||||
36 | RARA | ||||
37 | RB1 | ||||
38 | SELE | ||||
39 | SLC2A1 | ||||
40 | SOD1 | ||||
41 | TYMS |
Assay precision
Assay precision of SYBR Green RT 2 Profiler PCR Arrays. The plot shows the coefficient of variation for each average CT value. A total of 184 average CT measurements were obtained from Sample A and Sample B. Each average CT value and its coefficient of variation were calculated from the result of 6 replicate assays.
The average standard deviation for different CT value ranges and the percentage of genes in each group (percent frequency)
CT Values | Ave SD | Frequency (%) |
---|---|---|
10–25 | 0.11 | 45 |
25–30 | 0.19 | 41 |
30–35 | 0.40 | 11 |
≥ 35 | 0.96 | 3 |
Not Detectable | 0 | 0 |
Detection sensitivity
PCR Array quantification is determined by CT numbers. A gene is considered absent when the average CT exceeds 35. Among the 90 genes that were assayed with SYBR Green PCR Arrays, seven genes are considered absent in sample B with the average CT > 35. These genes are FOXA1, ABCC2, APOB, APOH, MMP1, RAD51 and TFF1. All of these genes were considered to be either absent or expressed at low levels as measured by TaqMan and QuantiGene [8]. FOXA1 was also noted to be absent by StaRT-PCR while the other six genes were not included in the measurements by this method. All genes are present in Sample A at a level above the limit of quantitation (referred to as the limit of detection (LOD) in the MAQC study [8]) of PCR Arrays with the average CT values smaller than 35. Hence, 83 out of the 90 selected genes (92%) are present in both Sample A and Sample B.
Assay range
The assay range is indicated by the difference in signals on a log10 scale between the highest and the lowest expression as described previously [8]. The assay range for the SYBR Green PCR Arrays is 8.6 with CT values ranging from 6.5 for 18S rRNA to 35 for weakly expressed genes.
Inter-site reproducibility
Inter-site comparison of RT 2 Profiler PCR Arrays. Threshold cycle measurements and fold-change results have been obtained using the two MAQC reference RNAs analyzed on the Human Drug Metabolism RT2 Profiler PCR Array (APH-002) using different real-time PCR thermal cyclers at two different sites. Five replicate arrays were run for each sample at each site, and the average CT value for each assay is compared between the two sites in panels A and B. The results show a high correlation between CT values obtained from the two sites. Panel C shows the inter-site comparison of fold-change results obtained from these data, indicating a correlation coefficient of 0.976.
Cross-platform Comparisons with Other Technologies
To evaluate the concordance of fold changes between SYBR Green PCR Arrays and other technologies evaluated in the MAQC study, we performed regression analyses of fold differences in sample B compared to sample A for all genes common between the custom RT2 Profiler PCR Arrays and another platform. In addition, a list of differentially expressed genes (DEGs) was identified for each platform between the two reference RNA samples using the cut-off criteria of a P value less than 0.05 by an unpaired t-test with a mean difference greater than or equal to 2-fold, and the lists of DEGs from different platforms were compared.
Fold-change correlation between PCR Arrays and other quantitative platforms
Cross-platform comparison between SYBR Green RT 2 Profiler PCR Arrays and the three quantitative platforms. The concordance of fold-changes between the PCR Arrays and the three quantitative platforms was evaluated by regression analysis of fold differences in sample B compared to sample A. Data were normalized against POLR2A for RT2 Profiler PCR Arrays and TaqMan, and against beta-actin for StaRT-PCR. The sample B/sample A (B/A) fold changes (log2) for each gene common between RT2 Profiler PCR Arrays and another platform were subjected to bivariate analysis. The dashed line on each graph represents the ideal slope of 1.0. The solid lines show a linear regression fit.
Fold-change correlation between the four different quantitative gene expression analysis platforms
Correlation (R) | TaqMan | QuantiGene | StaRT-PCR |
---|---|---|---|
(Number of genes in comparison) | |||
RT 2 Profiler PCR Array | 0.97 (86) | 0.93 (76) | 0.91 (57) |
TaqMan | 0.90* (181) | 0.94* (92) | |
QuantiGene | 0.92* (53) |
Fold-change correlation between PCR Arrays and microarray platforms
Cross-platform comparison between SYBR Green RT 2 Profiler PCR Arrays and microarray platoforms. The concordance of fold-changes between the PCR Arrays and microarray platforms was evaluated by regression analysis of fold differences in sample B compared to sample A. Data were normalized against POLR2A for RT2 Profiler PCR Arrays. The sample B/sample A (B/A) fold changes (log2) for each gene common between RT2 Profiler PCR Arrays and another platform were subjected to bivariate analysis. The dashed line on the graph represents the ideal slope of 1.0. Each solid line represents the linear regression fit. The numbers in brackets indicate the number of common genes between RT2 Profiler PCR Arrays and each microarray platform. The individual scatter plot for each comparison is provided in Additional file 3.
Fold-change correlations between SYBR Green RT2 Profiler PCR Arrays or TaqMan assays and the five microarray platforms
Correlation (R) | AFX | AG1 | ABI | GEH | ILM |
---|---|---|---|---|---|
(Number of genes in comparison) | |||||
PCR Array | 0.95 (89) | 0.94 (86) | 0.90 (89) | 0.86 (88) | 0.92 (87) |
TaqMan* | 0.92 (451–472) | 0.91 (532–595) | 0.85 (523–567) | 0.85 (660–670) | 0.91 (484–516) |
Discordant gene analysis
Comparison of fold-change results on the discordant genes between SYBR Green PCR Arrays and other platforms
RT2 Profiler PCR Array | TaqMan | StaRT-PCR | QuantiGene | AFX | ABI | AG1 | GEH | ILM | |
---|---|---|---|---|---|---|---|---|---|
FURIN | 1.16 | -43.98 | -1.49 | -1.42 | -1.21 | 1.04 | -1.38 | N/A | -1.12 |
RB1 | 1.73* | -1.92 | -1.59 | -3.09 | -2.23 | -1.30 | -4.06 | -2.14 | -2.14 |
ABCD1 | -5.44 | -7.11 | N/A | -3.83 | -1.45 | 2.21 | -8.25 | 2.15 | -2.41 |
BAG1 | 1.66 | -1.06 | 1.85 | -1.97 | -2.89 | -1.00 | 1.12 | -1.01 | 1.18 |
BAG4 | 2.66 | 1.01 | N/A | 1.96 | 1.30 | -2.25 | -2.25 | 1.13 | 1.51 |
CDK5R1 | 45.53 | 28.92 | N/A | 21.55 | 18.11 | -1.09 | 1.56 | 2.11 | 14.26 |
IGFBP5 | -1.68 | -2.69 | -1.98 | -2.16 | -2.68 | 4.11 | -3.26 | -2.42 | -2.37 |
JAK2 | 2.11 | 1.31 | N/A | 1.25 | 1.06 | 1.11 | -1.15 | 1.99 | 1.06 |
The other discordant gene, RB1, exhibits a fold change of 1.73 with PCR Arrays but a fold change of -3.09 with QuantiGene. TaqMan and StaRT-PCR indicate fold changes of -1.92 and -1.59, respectively, for RB1. All of the five microarray platforms show negative fold changes with four of the platforms indicating a twofold or greater change. Interestingly, the RB1 gene was originally indicated by PCR Arrays to be absent (CT > 35) in both samples A and B. This finding was in contrast to the results from the other three quantitative platforms where the gene was considered to be present and moderately to highly expressed in both RNA samples [8]. This discrepancy was later found to arise from the probing location of the primers for RB1 in PCR Arrays being based on a former version of the RefSeq accession (NM 000321.1) which had been revised to the current RefSeq release (NM 000321.2) on June 9, 2006. The revised RefSeq accession revealed a two-base mismatch (from C-G to G-C) with the former version in the region where one of the RB1 primers for PCR Arrays happened to recognize. Upon repeating the assays using the RB1 primer with the revised sequence, the RB1 gene is now considered to be present with an average CT value of around 23–25 in both samples. Results generated from this new RB1 assay show a fold change of -1.75 for this gene, which is in agreement with TaqMan and StaRT-PCR, and have replaced the data obtained from the old RB1 assay for the rest of the data analyses in this study.
Six other discordant genes besides RB1 are noted between SYBR Green PCR Arrays and the five microarray platforms (Table 5). Of those six discordant genes, one gene with AFX, four with ABI, two with AG1, and one with GEH, display changes in the opposite direction when compared to the RT2 Profiler PCR Array. BAG1 exhibits a fold change of 1.66 with PCR Arrays but a fold change of -2.89 with AFX while the other platforms indicate the gene to be similarly expressed in both RNA samples with a minimal fold change from -1.97 to 1.85. CDK5R1 and IGFBP5 are shown to be discordant between ABI and PCR Arrays, with the directions of fold-change observed by ABI for these two genes to be opposite to those reported by all other platforms. ABCD1 and BAG4 are discordant within the five microarray platforms while the three quantitative platforms that have measured these two genes, the RT2 Profiler PCR Array, TaqMan and QuantiGene, show fold-change values in a uniform direction. JAK2 displays a fold change of -1.15 by AG1 but 2.11 by PCR Arrays. All other platforms show the same direction of fold-change as PCR Arrays; however, like AG1, all of them indicate no significant difference in JAK2 expression between the two RNA samples with a fold change smaller than two.
Discussion
Although SYBR Geen PCR is a popular gene expression technique used by about half of the real-time PCR users, it was surprisingly not evaluated in the MAQC study. Therefore, in this study, we have assessed the performance of SYBR Green RT2 Profiler PCR Arrays and compared our expression profiling results of those two reference samples with the MAQC results obtained from three other traditional quantitative platforms including TaqMan assays, QuantiGene and StaRT-PCR as well as five commercial microarrays.
The present study demonstrates SYBR Green RT2 Profiler PCR Arrays to be a quantitative platform with high inter-run and inter-laboratory reproducibility. The average CV for the CT values generated from all assays on the custom PCR Array is found to be 0.73% with replicate measurements for CT values below 30 within 0.20 cycle average standard deviation, demonstrating a good inter-run reproducibility. When compared with the results from the other three quantitative platforms studied in the MAQC study (Additional file 1), the transformed data based on the corresponding universal MAQC scale give a median CV for all assays on PCR Arrays of 0.89% and that for assays detecting > 6000 transcript copies of 0.57%. These respective values were reported to be 3.46% and 2.42% for TaqMan, 6.26% and 3.82% for StaRT-PCR, and 2.16% and 2.12% for QuantiGene [8]. As seen with TaqMan assays and StaRT- PCR [8], a trend of increasing CV and SD is observed with increasing CT values (i.e. decreasing amount of transcripts) which is attributable to the stochastic nature of the sample loading of transcript molecules which influences the magnitude of CV [8]. High level of inter-site reproducibility is also demonstrated for SYBR Green PCR Arrays by inter-site comparison of CT values and fold-change results leading to correlation coefficients of 0.97 and 0.98, respectively.
Results obtained from SYBR Green PCR Arrays show that 83 out of the 90 selected genes (92%) are considered to be present in both samples A and B, which is similar to the values reported in the MAQC study for the detection sensitivity of the other three quantitative platforms [8]. Eighty-six percent (86%), 94% and 91% of the tested genes were above the respective LOD of the TaqMan assay (857 out of 997 tested genes), StaRT-PCR (193 out of 205 tested genes) and QuantiGene (223 out of 244 tested genes) in both samples A and B [8]. The assay range of PCR Arrays is 8.6 on a log10 scale, which is comparable to the assay range of 8.1 for TaqMan assay reported in the MAQC project [8] and wider than those of the other two quantitative platforms in the MAQC study, with StaRT-PCR and QuantiGene having an assay range of 6.8 and 4.1, respectively [8].
Cross-platform comparisons of gene profiling results of the two MAQC reference RNA samples illustrate a remarkably good correlation between SYBR Green RT2 Profiler PCR Arrays and other technologies tested in the MAQC studies. Specifically, PCR Arrays and the other two PCR-based methods, TaqMan and StaRT-PCR, exhibit an exceedingly high concordance with values for their correlation coefficients and linear slopes close to 1. This result demonstrates that these three methods report very similar fold changes in gene expression between the two MAQC samples. Although high correlation coefficients are observed between PCR Arrays and the hybridization-based techniques such as QuantiGene and the microarrays, the values for the linear slope are lower, ranging from 0.56 to 0.78. These results are consistent with a compression effect on log2 fold change computed from the hybridization-based techniques which was also observed when comparisons between TaqMan PCR and QuantiGene or microarray platforms were made in the MAQC studies [8, 12].
SYBR Green real-time PCR Arrays demonstrate a good concordance in the differentially expressed gene list with the three quantitative technologies and five microarray platforms examined in the MAQC projects. Great overlaps in the list of DEGs are noted for PCR Arrays and TaqMan as well as the other quantitative platforms, ranging from 81% to 93%. The list of DEGs generated by PCR Arrays is also highly comparable with the five microarray platforms, with overlaps ranging from 73% to 90%. However, disparities in the expression results are observed for two genes between PCR Arrays and quantitative platforms, and for six genes in addition to RB1 between PCR Arrays and the five microarray platforms (Table 5). One possible cause for the disparities was experimentally investigated and found due to the accuracy of sequence information provided by the NCBI RefSeq database. As the design of the primers and probes for gene expression measurement is usually based on the gene sequence information from the RefSeq and non-RefSeq databases, any inaccuracies in those databases or the presence of previously unknown SNPs or splice variants, will potentially affect the accuracy of the assays. This scenario was clearly illustrated by the SYBR Green assay for RB1 gene in this study where the discovery of a two-base mismatch in one of the RB1 primers was made with the current RefSeq accession. Since the RefSeq database is frequently being updated to improve the accuracy of its gene sequence information, it is important to have the sequences of the primers and probes checked periodically against the most updated version of the RefSeq database in order to ensure the accuracy of the assays.
Comparison of the probing locations for the discordant genes between RT2 Profiler PCR Arrays and other gene expression measurement systems in the MAQC study
Gene Length (Bases) | RT2 Profiler PCR Array | TaqMan | StaRT- PCR | QuantiGene | AFX | ABI | AG1 | GEH | ILM | |
---|---|---|---|---|---|---|---|---|---|---|
FURIN | 4180 | 2060 | 4115 | 1382 | 2048 | 65 | 1 | 632 | N/A | 184 |
RB1 | 4740 | 4268 | 2068 | 1742 | 2053 | 46 | 171 | 119 | 1322 | 1471 |
ABCD1 | 3616 | 1139 | 2317 | N/A | 2292 | 104 | 565 | 314 | 355 | 102 |
BAG1 | 3858 | 3143 | 3319 | 2675 | 2750 | 148 | 3381 | 2597 | 2614 | 2718 |
BAG4 | 2182 | 941 | 1618 | N/A | 924 | 584 | 52 | 281 | 1774 | 802 |
CDK5R1 | 3870 | 2823 | 3597 | N/A | 2905 | 41 | 2948 | 2782 | 3403 | 689 |
IGFBP5 | 6316 | 4845 | 5198 | 5519 | 5074 | 411 | 4868 | 418 | 5882 | 4740 |
JAK2 | 5097 | 4761 | 1409 | N/A | 2216 | 36 | 612 | 1164 | 316 | 323 |
Comparison of the probing locations for the discordant genes between RT 2 Profiler PCR Arrays and other gene expression measurement platforms in the MAQC phase 1 study. The diagram illustrates the most 3' interrogative locations of the assays of different platforms for the eight discordant genes discussed in the text and shown in Table 6. The black lines represent, from left to right, the 5' to 3' position of the genes, each indicated with the length of the transcript. The assay location of each platform is indicated by a letter above the black line (A = RT2 Profiler PCR Array; B = TaqMan; C = StaRT-PCR; D = QuantiGene; E = AFX; F = ABI; G = AG1; H = GEH; and I = ILM). Assays with discordant results compared to those of PCR Arrays are highlighted in grey.
The discrepancies in the expression results for the remaining three genes (ABCD1, CDK5R1 and IGFBP5) cannot simply be explained by the difference in probe locations. Interestingly, while these three genes display discordance among the microarray platforms, all four quantitative platforms show comparable results. This has led to the suggestion that disagreements between microarray platforms may be due to cross-hybridization of the probes on the arrays with other targets [8]. As in the case of RB1, the underlying reason for the discordance caused by different detection regions among technologies and platforms may be associated with the degree of accuracy in the sequence information being varied at different regions due to the continuous discoveries of previously unknown SNPs and spliced variants. Hence, even if the primers or probes from different platforms recognize the same region, there is a possibility that they may detect different spliced variants or transcripts with different SNPs. Again, frequently updating the primer and probe design with the most updated releases of RefSeq and non-RefSeq databases is essential for the success of all of these technologies.
Disagreements between different platforms may also arise from low detection signals for some genes. Among the discordant genes noted in Table 5, BAG4 and CDK5R1 are considered by all the five microarray platforms to be either absent or having relatively low expression in at least one of the two RNA samples. This trend with discordance occurring more frequently with low expressers was also noted in the MAQC study [8].
Conclusion
In summary, SYBR Green real-time PCR Arrays produce gene profiling differences between the two MAQC reference RNA samples that are highly concordant with those generated by other quantitative gene expression analysis and microarray platforms. PCR Arrays deliver results comparable to those of high-density microarrays. Moreover, it yields results similar to those of TaqMan Gene Expression Assays, a widely accepted method for validating microarray results, and other more complicated and more expensive quantitative methods tested by the MAQC project. Hence, SYBR Green PCR Array is a quantitative platform suitable for microarray data validation.
Methods
Materials
Sample A (Universal Human Reference RNA) and Sample B (Human Brain Reference RNA), were purchased from Stratagene (Cat# 740000 lot#1130623; La Jolla, CA) and Ambion (Cat# 6050 lot#105P055201A; Austin, TX), respectively. Both of these reference RNA samples are from the identical manufacturing lots as those used in the MAQC study [8, 12]. Reverse transcription kits (Cat# C-01) and SYBR Green real-time PCR master mixes (Cat# PA-012 and Cat# PA-011) were from SuperArray Bioscience (Frederick, MD). The Human Drug Metabolism RT2 Profiler™ PCR Array (Cat# APH-002), that was used for inter-site and inter-instrument comparison, and a custom RT2 Profiler PCR Array (Table 1), that was used for cross-platform comparison, were designed and manufactured at SuperArray with the primer sets for the specified genes pre-dispensed into a 96-well PCR plate.
Cross-platform comparison
The custom PCR Array designed for cross-platform comparison was run in six technical replicates for each of the two MAQC reference RNAs. For each custom PCR array, one microgram (μg) of RNA was reverse transcribed in a 20-μL reaction volume into first-strand cDNA using SuperArray's ReactionReady™ First Strand cDNA Synthesis Kit (Cat# C-01) containing random primers following the instructions provided in the user's manual. After mixing the cDNA with RT2 SYBR Green/ROX qPCR Master Mix (Cat# PA-012, SuperArray), 25 μL of the mixture containing cDNA synthesized from 9 ng total RNA was dispensed to each well of PCR Arrays. This amount of sample input is similar to the 10 ng total RNA sample input per reaction for TaqMan and StaRT-PCR Assays but lower than the 500 ng total RNA input per reaction for QuantiGene performed in the MAQC phase 1 study. Real-time PCR was performed on an ABI 7500 Real-Time PCR System (Applied Biosystems, Foster City, CA) using the following cycling parameters: 10 min at 95°C (heat activation step); 40 cycles of 15 sec at 95°C, 1 min at 60°C. Dissociation curve analyses were performed using the instrument's default setting immediately after each PCR run.
Inter-site comparison
The two MAQC reference RNAs were analyzed on the Human Drug Metabolism RT2 Profiler™ PCR Array (Cat# APH-002) at two different locations with cDNA synthesis and PCR procedures performed as described above. PCR Arrays were performed on an ABI 7500 Real-Time PCR System at Site 1 while the arrays were run on an ABI 7000 at Site 2, with both sites using SuperArray's RT2 SYBR Green/ROX qPCR Master Mix (Cat# PA-012). Five replicate arrays were run for each sample at each site.
Data normalization and analyses
The threshold cycle number (CT) for each PCR reaction is determined by setting the same threshold value across all PCR Arrays. For data normalization in the custom PCR Arrays, POLR2A was selected as the reference endogenous control gene since it was used as the normalizer in the TaqMan Assay in the MAQC study [8]. POLR2A was measured on each array plate in triplicate assays. The comparative CT method was used to calculate relative quantification of gene expression as described previously [21]. The relative amount of transcripts for each gene in Sample A and Sample B was normalized to the reference gene POLR2A and calculated as follows: ΔCT is the log2 difference between the gene and the reference gene, and is obtained by subtracting the average CT of POLR2A from the CT value of the gene on a per array basis; the log2 fold change between the two samples was obtained using the formula: ΔΔCT = the average Δ CT of Sample B – the average Δ CT of Sample A, and their fold difference = 2-ΔΔ CT. For cross-platform comparison, normalized data for both Sample A and Sample B from the other gene expression analysis technologies were directly obtained from published results [8, 12] and from the database accessible from the MAQC website [20]. For each gene, the fold change between the two samples was generated by calculating the ratio of the average of the normalized signals for all sample B replicates to the average of the normalized signals for all sample A replicates. The sample B/sample A (B/A) fold changes (log2) for all genes common between the RT2 Profiler PCR Arrays and another platform were compared and subjected to bivariate regression analysis, and Pearson correlation coefficients (R) were computed for each cross-platform comparison. For inter-site and cross-instrument comparison, data normalization in the Human Drug Metabolism RT2 Profiler™ PCR Array was carried out as instructed in the user's manual using the mean CT of five housekeeping genes as the reference endogenous control for each array plate. The ΔΔCT and fold-change values between the two RNA samples for each gene on the array plate were obtained and compared between the two sites and across different real-time PCR thermal cyclers.
Sensitivity detection and differentially expressed genes (DEG) determination
PCR Array quantification is determined by CT numbers. A gene is considered absent when the average CT exceeds 35. The CT is marked as 35 for the Δ CT calculation when the signal is below the limit of quantitation (this is referred to as the limit of detection (LOD) in the MAQC study [8]). A list of DEGs between the two reference RNA samples was identified using the cut-off criteria of a P value less than 0.05 as assessed by an unpaired t-test with a mean difference greater than or equal to 2-fold.
Declarations
Acknowledgements
This document has been reviewed in accordance with United States Food and Drug Administration (FDA) policy and approved for publication. Approval does not signify that the contents necessarily reflect the position or opinions of the FDA nor does mention of trade names or commercial products constitute endorsement or recommendation for use. The findings and conclusions in this report are those of the author(s) and do not necessarily represent the views of the FDA.
Authors’ Affiliations
References
- Lockhart DJ, Dong H, Byrne MC, Follettie MT, Gallo MV, Chee MS, Mittmann M, Wang C, Kobayashi M, Horton H, Brown EL: Expression monitoring by hybridization to high-density oligonucleotide arrays. Nature biotechnology. 1996, 14 (13): 1675-1680. 10.1038/nbt1296-1675.PubMedView ArticleGoogle Scholar
- Schena M, Shalon D, Davis RW, Brown PO: Quantitative monitoring of gene expression patterns with a complementary DNA microarray. Science (New York, NY). 1995, 270 (5235): 467-470.View ArticleGoogle Scholar
- Tan PK, Downey TJ, Spitznagel EL, Xu P, Fu D, Dimitrov DS, Lempicki RA, Raaka BM, Cam MC: Evaluation of gene expression measurements from commercial microarray platforms. Nucleic acids research. 2003, 31 (19): 5676-5684. 10.1093/nar/gkg763.PubMedView ArticleGoogle Scholar
- Miklos GL, Maleszka R: Microarray reality checks in the context of a complex disease. Nature biotechnology. 2004, 22 (5): 615-621. 10.1038/nbt965.PubMedView ArticleGoogle Scholar
- Marshall E: Getting the noise out of gene arrays. Science (New York, NY). 2004, 306 (5696): 630-631.View ArticleGoogle Scholar
- Ein-Dor L, Zuk O, Domany E: Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer. Proceedings of the National Academy of Sciences of the United States of America. 2006, 103 (15): 5923-5928. 10.1073/pnas.0601231103.PubMedView ArticleGoogle Scholar
- Qin LX, Beyer RP, Hudson FN, Linford NJ, Morris DE, Kerr KF: Evaluation of methods for oligonucleotide array data via quantitative real-time PCR. BMC bioinformatics. 2006, 7: 23-10.1186/1471-2105-7-23.PubMedView ArticleGoogle Scholar
- Canales RD, Luo Y, Willey JC, Austermiller B, Barbacioru CC, Boysen C, Hunkapiller K, Jensen RV, Knight CR, Lee KY, Ma Y, Maqsodi B, Papallo A, Peters EH, Poulter K, Ruppel PL, Samaha RR, Shi L, Yang W, Zhang L, Goodsaid FM: Evaluation of DNA microarray results with quantitative gene expression platforms. Nature biotechnology. 2006, 24 (9): 1115-1122. 10.1038/nbt1236.PubMedView ArticleGoogle Scholar
- Shippy R, Fulmer-Smentek S, Jensen RV, Jones WD, Wolber PK, Johnson CD, Pine PS, Boysen C, Guo X, Chudin E, Sun YA, Willey JC, Thierry-Mieg J, Thierry-Mieg D, Setterquist RA, Wilson M, Lucas AB, Novoradovskaya N, Papallo A, Turpaz Y, Baker SC, Warrington JA, Shi L, Herman D: Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nature biotechnology. 2006, 24 (9): 1123-1131. 10.1038/nbt1241.PubMedView ArticleGoogle Scholar
- Tong W, Lucas AB, Shippy R, Fan X, Fang H, Hong H, Orr MS, Chu TM, Guo X, Collins PJ, Sun YA, Wang SJ, Bao W, Wolfinger RD, Shchegrova S, Guo L, Warrington JA, Shi L: Evaluation of external RNA controls for the assessment of microarray performance. Nature biotechnology. 2006, 24 (9): 1132-1139. 10.1038/nbt1237.PubMedView ArticleGoogle Scholar
- Patterson TA, Lobenhofer EK, Fulmer-Smentek SB, Collins PJ, Chu TM, Bao W, Fang H, Kawasaki ES, Hager J, Tikhonova IR, Walker SJ, Zhang L, Hurban P, de Longueville F, Fuscoe JC, Tong W, Shi L, Wolfinger RD: Performance comparison of one-color and two-color platforms within the MicroArray Quality Control (MAQC) project. Nature biotechnology. 2006, 24 (9): 1140-1150. 10.1038/nbt1242.PubMedView ArticleGoogle Scholar
- Shi L, Reid LH, Jones WD, Shippy R, Warrington JA, Baker SC, Collins PJ, de Longueville F, Kawasaki ES, Lee KY, Luo Y, Sun YA, Willey JC, Setterquist RA, Fischer GM, Tong W, Dragan YP, Dix DJ, Frueh FW, Goodsaid FM, Herman D, Jensen RV, Johnson CD, Lobenhofer EK, Puri RK, Schrf U, Thierry-Mieg J, Wang C, Wilson M, Wolber PK, Zhang L, Amur S, Bao W, Barbacioru CC, Lucas AB, Bertholet V, Boysen C, Bromley B, Brown D, Brunner A, Canales R, Cao XM, Cebula TA, Chen JJ, Cheng J, Chu TM, Chudin E, Corson J, Corton JC, Croner LJ, Davies C, Davison TS, Delenstarr G, Deng X, Dorris D, Eklund AC, Fan XH, Fang H, Fulmer-Smentek S, Fuscoe JC, Gallagher K, Ge W, Guo L, Guo X, Hager J, Haje PK, Han J, Han T, Harbottle HC, Harris SC, Hatchwell E, Hauser CA, Hester S, Hong H, Hurban P, Jackson SA, Ji H, Knight CR, Kuo WP, LeClerc JE, Levy S, Li QZ, Liu C, Liu Y, Lombardi MJ, Ma Y, Magnuson SR, Maqsodi B, McDaniel T, Mei N, Myklebost O, Ning B, Novoradovskaya N, Orr MS, Osborn TW, Papallo A, Patterson TA, Perkins RG, Peters EH, Peterson R, Philips KL, Pine PS, Pusztai L, Qian F, Ren H, Rosen M, Rosenzweig BA, Samaha RR, Schena M, Schroth GP, Shchegrova S, Smith DD, Staedtler F, Su Z, Sun H, Szallasi Z, Tezak Z, Thierry-Mieg D, Thompson KL, Tikhonova I, Turpaz Y, Vallanat B, Van C, Walker SJ, Wang SJ, Wang Y, Wolfinger R, Wong A, Wu J, Xiao C, Xie Q, Xu J, Yang W, Zhang L, Zhong S, Zong Y, Slikker W: The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nature biotechnology. 2006, 24 (9): 1151-1161. 10.1038/nbt1239.PubMedView ArticleGoogle Scholar
- Guo L, Lobenhofer EK, Wang C, Shippy R, Harris SC, Zhang L, Mei N, Chen T, Herman D, Goodsaid FM, Hurban P, Phillips KL, Xu J, Deng X, Sun YA, Tong W, Dragan YP, Shi L: Rat toxicogenomic study reveals analytical consistency across microarray platforms. Nature biotechnology. 2006, 24 (9): 1162-1169. 10.1038/nbt1238.PubMedView ArticleGoogle Scholar
- Flagella M, Bui S, Zheng Z, Nguyen CT, Zhang A, Pastor L, Ma Y, Yang W, Crawford KL, McMaster GK, Witney F, Luo Y: A multiplex branched DNA assay for parallel quantitative gene expression profiling. Analytical biochemistry. 2006, 352 (1): 50-60. 10.1016/j.ab.2006.02.013.PubMedView ArticleGoogle Scholar
- Willey JC, Crawford EL, Knight CR, Warner KA, Motten CA, Herness EA, Zahorchak RJ, Graves TG: Standardized RT-PCR and the standardized expression measurement center. Methods Mol Biol. 2004, 258: 13-41.PubMedGoogle Scholar
- Heid CA, Stevens J, Livak KJ, Williams PM: Real time quantitative PCR. Genome research. 1996, 6 (10): 986-994. 10.1101/gr.6.10.986.PubMedView ArticleGoogle Scholar
- Knudtson KL, Adams PS, Grove DS, Hollingshead DJ, Hunter TC, Shipley GL: The ABRF NARG Real-time PCR Survey 2007: Taking the pulse of the quantitative PCR field. 2007Google Scholar
- Morrison TB, Weis JJ, Wittwer CT: Quantification of low-copy transcripts by continuous SYBR Green I monitoring during amplification. BioTechniques. 1998, 24 (6): 954-8, 960, 962.PubMedGoogle Scholar
- Rajeevan MS, Ranamukhaarachchi DG, Vernon SD, Unger ER: Use of real-time quantitative PCR to validate the results of cDNA array and differential display PCR technologies. Methods (San Diego, Calif. 2001, 25 (4): 443-451.View ArticleGoogle Scholar
- The MicroArray Quality Control Study. [http://www.fda.gov/nctr/science/centers/toxicoinformatics/maqc/index.htm]
- Livak KJ, Schmittgen TD: Analysis of relative gene expression data using real-time quantitative PCR and the 2(-Delta Delta C(T)) Method. Methods. 2001, 25 (4): 402-408. 10.1006/meth.2001.1262.PubMedView ArticleGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.