Large scale real-time PCR validation on gene expression measurements from two commercial long-oligonucleotide microarrays
© Wang et al; licensee BioMed Central Ltd. 2006
Received: 08 November 2005
Accepted: 21 March 2006
Published: 21 March 2006
DNA microarrays are rapidly becoming a fundamental tool in discovery-based genomic and biomedical research. However, the reliability of the microarray results is being challenged due to the existence of different technologies and non-standard methods of data analysis and interpretation. In the absence of a "gold standard"/"reference method" for the gene expression measurements, studies evaluating and comparing the performance of various microarray platforms have often yielded subjective and conflicting conclusions. To address this issue we have conducted a large scale TaqMan® Gene Expression Assay based real-time PCR experiment and used this data set as the reference to evaluate the performance of two representative commercial microarray platforms.
In this study, we analyzed the gene expression profiles of three human tissues: brain, lung, liver and one universal human reference sample (UHR) using two representative commercial long-oligonucleotide microarray platforms: (1) Applied Biosystems Human Genome Survey Microarrays (based on single-color detection); (2) Agilent Whole Human Genome Oligo Microarrays (based on two-color detection). 1,375 genes represented by both microarray platforms and spanning a wide dynamic range in gene expression levels, were selected for TaqMan® Gene Expression Assay based real-time PCR validation. For each platform, four technical replicates were performed on the same total RNA samples according to each manufacturer's standard protocols. For Agilent arrays, comparative hybridization was performed using incorporation of Cy5 for brain/lung/liver RNA and Cy3 for UHR RNA (common reference). Using the TaqMan® Gene Expression Assay based real-time PCR data set as the reference set, the performance of the two microarray platforms was evaluated focusing on the following criteria: (1) Sensitivity and accuracy in detection of expression; (2) Fold change correlation with real-time PCR data in pair-wise tissues as well as in gene expression profiles determined across all tissues; (3) Sensitivity and accuracy in detection of differential expression.
Our study provides one of the largest "reference" data set of gene expression measurements using TaqMan® Gene Expression Assay based real-time PCR technology. This data set allowed us to use an alternative gene expression technology to evaluate the performance of different microarray platforms. We conclude that microarrays are indeed invaluable discovery tools with acceptable reliability for genome-wide gene expression screening, though validation of putative changes in gene expression remains advisable. Our study also characterizes the limitations of microarrays; understanding these limitations will enable researchers to more effectively evaluate microarray results in a more cautious and appropriate manner.
DNA microarray technology provides a powerful tool for characterizing gene expression on a genome scale. While the technology has been widely used in discovery-based medical and basic biological research, its direct application in clinical practice and regulatory decision-making has been questioned [1–4]. A few key issues, including the reproducibility, reliability, compatibility and standardization of microarray analysis and results, must be critically addressed before any routine usage of microarrays in clinical laboratory and regulated areas. Considerable effort has been dedicated to investigate these important issues, most of which focused on the compatibility across different microarray platforms, laboratories and analytical methods. However, in the absence of a "gold standard" or common reference for gene expression measurements, these evaluations and comparisons have often yield subjective and conflicting conclusions [5–11].
Real-time PCR is often referred to as the "gold standard" for gene expression measurements [8, 12], due to its advantages in detection sensitivity, sequence specificity, large dynamic range as well as its high precision and reproducible quantitation compared to other techniques (For recent reviews, see [13–15]). The performance capabilities and ease-of-use of TaqMan based real-time PCR chemistries and instrumentation has led to widespread use of this technology as a preferred method for quantifying gene expression as well as for independent validation of microarray results [3, 12, 16, 17]. TaqMan® Gene Expression Assays (Applied Biosystems, Foster City, CA) utilize the 5' nuclease activity of AmpliTaq Gold® DNA polymerase to hydrolyze a target-specific probe (TaqMan probe) bound to its target amplicon during PCR [see Additional file 1]. Each TaqMan Gene Expression Assay consists of two sequence-specific primers defining the endpoints of the amplicon and a TaqMan probe with a fluorescent reporter dye (FAM™) and a nonfluorescent quencher moiety attached to the 5' and 3' ends. Together, the primer set and the TaqMan probe provide two levels of sequence specificity. Problems associated with DNA contamination are minimized by designing primers that span at least one intron of the genomic sequence whenever possible. During each PCR extension cycle, the Taq DNA polymerase cleaves the reporter dye from the probe and once separated from the quencher. The reporter dye emits its characteristic fluorescence to allow measurement of PCR amplification as it occurs, cycle by cycle, during the highly reproducible exponential phase of PCR. This enables highly accurate and precise quantitation of gene expression over a large dynamic range.
Overview of the two microarray platforms and TaqMan® GeneExpression Assay based real-time PCR
TaqMan Gene Expression Assays
Human Genome Survey Microarray
Human Whole Genome Arrays
TaqMan Expression Assays
5' Nuclease Chemistry & Real Time PCR
TaqMan Primer & Probes
In-situ Ink Jet Printing
Two-color Cy3/Cy5 Fluorescence
One-color FAM Fluorescence
1700 Chemiluminescence Microaray Analyzer Software v 1.1
Feature Extraction A 7.5.1
1375 Selected Targets
Target selection for real-time PCR validation
In order to conduct a comprehensive and unbiased survey of the microarrays' performance, we selected the gene targets for real-time PCR validation based on the following strategies: (1) Ensure a large enough number of validation targets to provide representative overviews of the microarray performance; (2) Select genes with expression levels spanning a wide dynamic range; (3) Select genes that are represented by both microarray platforms. Validation targets were selected across the expression range from the 21,171 genes cross-mapped on both platforms – the "common genes" (See Methods for mapping methodology). Because single-color microarray systems, in general, represent more straightforward signal-abundance relationship than two-color microarray system which based on relative quantification, we used an existing Applied Biosystem data set as a reference for binning by signal. Specifically, average signals from the four technical replicates of the liver sample generated by Applied Biosystem Expression Arrays were sorted and binned into 10 bins with each bin containing equal numbers of genes. 55 genes were selected randomly from each bin yielding 550 gene targets; Another 550 targets were selected in the same fashion from data generated from the UHR sample. Finally, 275 targets were chosen randomly from differentially expressed genes (common between the Applied Biosystem data set and the Agilent data set) across the liver, lung and brain samples based on ANOVA analysis (p < 0.05). As a result, a total of 1375 genes were selected for real-time PCR validation using TaqMan® Gene Expression Assays [see Additional file 2]. For genes with multiple TaqMan assays targeting different exon or exon-exon junctions, one assay was randomly selected without efforts in matching its location to that of corresponding probe on either microarray platform.
Signal detection sensitivity and accuracy
Sensitivity and accuracy in signal detection. TaqMan® Gene Expression Assays calls were used as the "ground truth" to calculate the True Positive, True Negative, False Positive, False Negative, True Positive Rate (TPR) and False Positive Rate (FPR) as described in Methods. Detection thresholds for each platform were defined according to corresponding manufacturer's recommendations and described in Methods. Results are shown as sum of the Brain, Liver and Lung tissues
Specificity (1- FPR)
TaqMan ® Gene Expression Assays
Correlation between real-time PCR and microarrays in fold change measurements of pair-wise tissues
Correlation between real-time PCR and microarrays in expression profiles across all tissues
Sensitivity and accuracy in differential expression analysis
The complete data sets of this study can be accessed from Gene Expression Omnibus (GEO) , accession ID GSE4214.
Although microarrays have been extensively used as discovery tools for biological and biomedical studies, the challenge remains whether this technology can be reliably applied in clinical practice and regulatory decision making, where high precision and accuracy in performance are required. A series of studies have been reported on evaluating performance across various commercial and home-brewed microarray platforms, however, most of these studies focused on evaluating the level of concordance across different microarray platforms. While these analyses emphasized critical issues such as the compatibility across different microarray platforms, they tended to result in conflicting conclusions because the "relative to relative" nature of such approaches. What is lacking in these studies is a "gold standard" data set that allows an evaluation of different microarray platforms based on a common "ground truth". One commonly used approach for setting up such "ground truth" is by spiking in bacterial synthetic transcripts with known concentrations in series of dilutions over a large dynamic range , however, the limitation of this approach is that the information is asserted from very limited transcripts, and it is also very prone to experimental artifacts. An alternative strategy to set up the "ground truth" is using a well accepted reference data set generated by a reliable independent technology, such as real-time PCR for gene expression measurements. In this study, we have constructed a large reference data set of gene expression measurements using TaqMan Gene Expression Assays and real-time PCR technology. We also demonstrated how to use such a data set to evaluate the performance of different microarray platforms.
We first evaluated the detection sensitivity and accuracy of the two selected microarray platforms using TaqMan Gene Expression Assays and real-time PCR data set as the reference. We chose to use the detection thresholds that are recommended by each manufacturer as the base line for comparison. These recommended thresholds are somewhat arbitrary and are not necessarily based on the same parameters, nevertheless, these detection thresholds are widely adopted by researchers and therefore evaluating their effect on detection sensitivity and accuracy can prove useful in further refining them and better interpreting microarray results. Our results showed that both of the microarray platforms can achieve reasonably good sensitivity in signal detection, while the specificity tends to be relatively low, especially for Agilent microarrays, with a ~ 50% false positive rate. It is worth noting that the differences in detection sensitivity and specificity we observed could be caused by less optimal bioinformatics/algorithms used to define the detection thresholds and do not necessarily reflect the inherent qualities or accuracies of the respective platforms. Several strategies could be developed to improve the detection specificity of microarrays, including improving probe design, hybridization conditions which would minimize the effects of cross-hybridization, as well as improving image analysis software/algorithms to facilitate more accurate signal quantification and detection thresh-holding.
This study also evaluated correlation in detecting differential expression between microarray platforms and TaqMan real-time PCR platform (Figure 3 and Figure 5). Our analysis also provided a high-resolution examination of the performance of microarrays in detecting differential expression at different expression levels as well as at different fold changes. We validated that microarrays have acceptable sensitivity and accuracy in detecting differential expression, especially for genes with high and medium expression levels and for detecting > 2-fold changes. These results support the notion that microarrays, as exploratory tools for genome-wide gene expression screening, can achieve acceptable reliability in performance.
Our study also characterized some of the limitations of microarrays, in particular the ratio compression phenomena as shown in Figure 4. A certain level of fold change compression is expected for microarray platforms due to various technical limitations, including limited dynamic range, signal saturations, and cross-hybridizations. The two-color system analyzed in this study (Agilent microarrays), appears to have more severe ratio compression, which could be attributed to several factors: (1) The concentration of the 60mer probes on Agilent microarrays depends on the coupling efficiency of the in-situ oligonucleotide synthesis, and on probe length. Lower efficiency may result in low probe concentration and therefore limit the dynamic range of the platform; (2) Two-color systems such as Agilent arrays, utilize two different fluorescent dyes that have different dynamic ranges and quantum yields. These intrinsic differences may be partially adjusted by intra-array intensity-dependent normalization but may not be completely eliminated. Theoretically, dye swapping experiments may help to further adjust these biases introduced by two different dyes. In reality, however, dye-swapping is not always practical due to cost and limitations in sample amount. Finally, ratio compression can be also introduced by certain data-processing/normalization algorithms that aim to reduce variances (e.g. lowess normalization for Agilent microarrays and RMA method for Affymetrix microarrays). Our analysis suggests that the optimal balance between the two parameters will eventually determine the overall accuracy in detection of differential expression, for a given microarray platform. Other microarray limitations revealed by our study include the significant decrease in overall accuracy of differential expression detection at low expression level (Figure 6) and the relatively poor sensitivity in detecting small fold changes (i.e. < 2-fold). Although these limitations have been previously suspected by many, the large scale "reference" data set provided by our study provides a more quantitative view of these limitations for the first time.
Lastly, it is noteworthy that although TaqMan Gene Expression Assay based real-time PCR is a well accepted "gold standard" for gene expression measurements, we are aware that it has its own limitations and is also affected by experimental errors. In addition, different strategies in probe designs for microarrays (usually 3' biased and targeting a composite of transcripts) and TaqMan Gene Expression Assays (usually without a priori bias and targeting a single or subset of transcripts) may also account for a small percentage of the discordance observed between the microarrays and real-time PCR results. For example, the gene expression profiles of gene NM_003640 measured by both microarray platforms are highly correlated with each other but anti-correlates with the profile measured by TaqMan Gene Expression Assay (Hs00175353_m1, Figure 5A). NM_003640 is a relative long transcript with 5917 bases and 37 exons. While the TaqMan assay was designed against the exon 2 and exon 3 junctions, which is > 4 kb from the 3' end, the microarray probes were usually designed close to 3' end (mostly within 1.5 kb from 3' end for Applied Biosystems probes). In this particular instance, the data suggest that the TaqMan gene expression assay is potentially detecting additional splice variants than the array probes. This difference in probe designs may result in quantifying different population of transcripts (e.g. product of alternative splicing or degradation) by microarrays or TaqMan assays. These factors may change the absolute metrics (i.e. TRP, TFP, and anti-correlation rate); nevertheless, they would not change the general conclusions and trends we observed. We think that most of the discrepancies between TaqMan based real-time PCR and microarrays are due to the sensitivity limits of a PCR based approach vs. a hybridization based approach. It is clear that at high expression levels, there is a much better correlation between the two approaches (Figure 6). These factors may change the absolute metrics (i.e. TRP, TFP, and anti-correlation rate); nevertheless, they would not change the general conclusions and trends we observed. We think that most of the discrepancies between TaqMan based real-time PCR and microarrays are due to the sensitivity limits of a PCR based approach vs. a hybridization based approach. It is clear that at high expression levels, there is a much better correlation between the two approaches (Figure 6).
Our study provides one of the largest "reference" data set of gene expression measurements using TaqMan® Gene Expression Assay based real-time PCR technology. We also provide novel analysis approaches for evaluating different micorarray platforms as well as performing cross-platform correlations. As a result of this study, we recommend using "reference" data sets generated by real-time PCR to evaluate critical aspects of microarray platforms, including signal detection threshold, fold change correlation between pair-wise tissues, profile correlation across multiple tissues, as well as sensitivity and specificity in signal detection and differential expression. We conclude that microarrays are invaluable discovery tools with acceptable reliability for genome-wide gene expression screening. Understanding the limitations of microarrays characterized by our study will help us to better apply this technology and interpret its results more cautiously.
Total RNA samples of human whole brain (# 929565), liver (# 929564), lung (#929566), and the universal human reference sample (UHR, # 929563) were purchased from Stratagene (La Jolla, CA). The quality and integrity of the total RNA was evaluated on the 2100 Bioanalyzer (Agilent Technologies, Palo Alto, CA), and the same samples were divided into individual aliquots for the gene expression analysis on the two different microarray platforms and for the TaqMan Gene Expression Assay based real-time PCR analysis.
Applied Biosystems Expression Array analysis
The Applied Biosystems Human Genome Survey Microarray (P/N 4337467) contains 31,700 60-mer oligonucleotide probes representing 27,868 individual human genes. Digoxigenin-UTP labeled cRNA was generated and amplified from 1 μ g of total RNA from each sample using Applied Biosystems Chemiluminescent RT-IVT Labeling Kit v 1.0 (P/N 4340472) according to the manufacturer's protocol (P/N 4339629). Array hybridization was performed for 16 hrs at 55°C. Chemiluminescence detection, image acquisition and analysis were performed using Applied Biosystems Chemiluminescence Detection Kit (P/N 4342142) and Applied Biosystems 1700 Chemiluminescent Microarray Analyzer (P/N 4338036) following the manufacturer's protocol (P/N 4339629). Images were auto-gridded and the chemiluminescent signals were quantified, background subtracted, and finally, spot- and spatially-normalized using the Applied Biosystems 1700 Chemiluminescent Microarray Analyzer software v 1.1 (P/N 4336391). Four technical replicates were performed on each sample, a total of 16 microarrays were used for the analysis. For inter-array normalization, a global median normalization was applied across all microarrays to achieve the same median signal intensities for each array. Besides using the default global median normalization method, we also investigated several other normalization methods for the Applied Biosystems data set, including Quantile normalization , scale normalization  and Variance Stabilization Normalization (VSN) , using the limma and vsn packages of R/Bioconductor . Marginal difference was observed in outcomes among different normalization methods, including the intra-platform reproducibility [see Additional file 5] and fold change correlation with TaqMan assays [see Additional file 6].
Agilent Whole Human Genome Oligo Microarray analysis
The Agilent Whole Human Genome Oligo Microarray (G4112A) contains 44,000 60-mer oligonucleotide probes representing 41,000 unique genes and transcripts. Probe labeling and hybridization were carried out following the manufacturer's specified protocols. Briefly, amplification and labeling of 5 ug of total RNA was performed using Cy5 for brain/liver/lung RNA and Cy3 for the reference RNA (Stratagene UHR). Hybridization was performed for 16 hrs at 60°C and arrays were scanned on an Agilent DNA microarray scanner. Following the manufacturer's protocol, Agilent's Stabilization and Drying Solution (#5185–5979) was used to protect against the ozone-induced degradation of cyanine dyes on microarray slides during hybridization and processing steps. Images were analyzed and data were extracted, background subtracted and normalized using the standard procedures of Agilent Feature Extraction Software A.7.5.1. Four technical replicates were performed for each pair of RNA samples (brain vs. UHR, liver vs. UHR, and lung vs. UHR), a total of 12 arrays were analyzed. Linear & LOWESS, which is the default normalization method in the Agilent Feature Extraction Software A.7.5.1, were applied for normalizing Agilent microarrays. This method does a linear normalization across the entire range of data, and then applies a non-linear normalization (LOWESS) to the linearized data set.
TaqMan® Gene Expression Assay based real-time PCR
Expression of mRNA for 1375 genes was measured in each of the three human tissues and the UHR total RNA samples by real-time PCR using TaqMan® Gene Expression Assays on ABI PRISM 7900 HT Sequence Detection System (Applied Biosystems, Foster City, CA). ~ 5 ug of total RNA of each sample was used to generate cDNA using the ABI High Capacity cDNA Archiving Kit (Applied Biosystems, Foster City, CA) and the real-time PCR reactions were carried out following the manufacturer's protocol. TaqMan® Gene Expression Assay IDs are listed [see Additional file 3]. Four technical replicates were run for each gene in each sample in a 384-well format plate and a total of 64 plates (16 plates for each sample) were run in this study. On each plate, three endogenous control genes (RPS18, PPIA (Alias: cyclophilin A) and GAPDH) and one no-template-controls (NTC) were also run in quadruplicates. We chose PPIA for normalization across different genes based on that this gene showed the most relatively constant expression in different tissue samples (data not shown).
Cross-mapping between microarray platforms
For a direct comparison of the Applied Biosystems Human Whole Genome Survey Microarrays and the Agilent Whole Human Whole Genome Oligo Microarrays, we identified a set of genes represented on both platforms. The cross-mapping was done by using BLAST to compare Applied Biosystems 60 mer probe sequences to the target transcript sequences interrogated by the probes on Agilent arrays (GEO  platform GPL1708) and only probes with 100% sequence identity were included in the final gene set. When multiple probes from one platform were mapped to one probe from the other platform, a one to one probe pair was randomly selected. As a result, 21171 common genes represented by both platforms were identified [see Additional file 4].
Statistical analyses were performed with the software packages MATLAB® (Mathworks, Natick, MA), R/Bioconductor , and Spotfire Functional Genomic (Spotfire, Göteborg, Sweden).
Signal detection analysis
Detection thresholds are defined according to each platform manufacturer's recommendation. For TaqMan Gene Expression Assays, detection threshold is set as Ct < 35 and Stdev (of 4 technical replicates) < 0.5; for Applied Biosystems Expression Arrays, detection threshold is set as Signal to Noise ratio (S/N) > 3 and quality flag < 5000; for Agilent arrays, detection threshold is set based on multiple parameters, including (1) WellAboveBackground (Signal/Background > 3.0); (2) Positive&Significant vs. Background (p < 0.01); and (3) they are not saturated, non-Uniform or population outliers in signals of feature and background. Detection in each tissue was defined as detectable in 3 out of 4 technical replicates within each platform. Using TaqMan® Gene Expression Assays calls as the reference, contingency tables were constructed against microarray platforms, in which True Positives (TP, detectable by both TaqMan Assay and Microarray), True Negative (TN, not detectable by either TaqMan Assay nor Microarray), False Positive (FP, detectable by Microarrays but not by TaqMan Assays), and False Negative (FN, detectable by TaqMan Assays but not by Microarrays). Based on this matrix, the following statistics were calculated for each microarray platform :
True positive rate,
False positive rate,
False discovery rate,
Gene expression profile correlation between microarray and TaqMan® Gene Expression Assays
To make a more direct comparison on gene expression profiles determined by single-color and two-color platforms, data from Applied Biosystems arrays and TaqMan Assays were transformed using UHR sample as a reference to generate brain vs. UHR, liver vs. UHR and lung vs. UHR ratios. For each individual gene, a median gene expression profile across the three tissue samples (brain, liver, and lung) was determined z-score transformed across the three tissues: , where X is the median expression level (tissue vs. UHR ratio) of the four technical replicates for a given tissue and given gene, is the average expression level across all three tissues for this gene, and σ is the standard deviation of gene expression level across all three tissues for this gene. Using the profile determined by TaqMan® Gene Expression Assays as the reference, a Spearman rank-order correlation coefficient MATLAB® (Mathworks, Natick, MA) was calculated against this reference for each of the two microarray platforms.
To calculate the power for the TaqMan real-time PCR platform for genes with different expression levels, Ct values of all assays in the three tissues were sorted and partitioned into four bins with equal numbers of assays. These bins span Ct intervals [10, 26.8], [26.9, 28.4], [28.5, 30.4] and [30.5, 35] and represent genes with high, medium high, medium low and low different expression levels, respectively. Average standard deviation was calculated for each bin and their power to detect different fold changes with p-value < 0.05 or with p-value < 0.001 (equivalent of FDR < 5%) using four technical replicates was calculated based on methods described previously .
Differential expression analysis
Significantly differentially expressed genes between different pairs of tissues were defined as p-value < 0.05 based on a student's t-test. Using calls from TaqMan® Gene Expression Assays as the reference, contingency tables were constructed against microarray platforms, in which the concordance between microarray platforms and the TaqMan® Gene Expression Assays was determined taking into considerations of both p-value significance and fold change directions (up or down regulation). Specifically, True Positives (TP, p < 0.05 for both TaqMan Assay and Microarray and fold change in the same direction), True Negative (TN, p > 0.05 for both TaqMan Assay and Microarray), False Positive (FP, p > 0.05 for TaqMan Assay and p < 0.05 for Microarray, or p < 0.05 for both TaqMan Assay and Microarray and fold change in opposite direction), and False Negative (FN, p < 0.05 for TaqMan Assay and p > 0.05 for Microarray). Based on this matrix, the TPR, FPR, FDR and accuracy were calculated for each microarray platform as described above . Similar analysis were performed using different criteria for determining differentially expressed genes, including using t-test with FDR correction at 5% (p-value adjusted by Benjamini and Hochberg False Discovery Rate multiple testing corrections) in both microarray and TaqMan reference data sets, or using a fold change cutoff (> 1.2 fold) in TaqMan reference data sets while using the same fold change cutoff and a t-test (p-value < 0.05) in microarray data sets.
We thank Eugene Spier and Karl J Guegler for their contributions in designs of TaqMan Gene Expression Assays and helpful discussions.
- Hackett JL, Lesko LJ: Microarray data--the US FDA, industry and academia. Nat Biotechnol. 2003, 21 (7): 742-743. 10.1038/nbt0703-742.PubMedView Article
- Petricoin EF, Hackett JL, Lesko LJ, Puri RK, Gutman SI, Chumakov K, Woodcock J, Feigal DWJ, Zoon KC, Sistare FD: Medical applications of microarray technologies: a regulatory science perspective. Nat Genet. 2002, 32 Suppl: 474-479. 10.1038/ng1029.PubMedView Article
- Ramaswamy S: Translating cancer genomics into clinical oncology. N Engl J Med. 2004, 350 (18): 1814-1816. 10.1056/NEJMp048059.PubMedView Article
- Shi L, Tong W, Goodsaid F, Frueh FW, Fang H, Han T, Fuscoe JC, Casciano DA: QA/QC: challenges and pitfalls facing the microarray community and regulatory agencies. Expert Rev Mol Diagn. 2004, 4 (6): 761-777. 10.1586/1473718.104.22.1681.PubMedView Article
- Irizarry RA, Warren D, Spencer F, Kim IF, Biswal S, Frank BC, Gabrielson E, Garcia JG, Geoghegan J, Germino G, Griffin C, Hilmer SC, Hoffman E, Jedlicka AE, Kawasaki E, Martinez-Murillo F, Morsberger L, Lee H, Petersen D, Quackenbush J, Scott A, Wilson M, Yang Y, Ye SQ, Yu W: Multiple-laboratory comparison of microarray platforms. Nat Methods. 2005, 2 (5): 345-350. 10.1038/nmeth756.PubMedView Article
- Larkin JE, Frank BC, Gavras H, Sultana R, Quackenbush J: Independence and reproducibility across microarray platforms. Nat Methods. 2005, 2 (5): 337-344. 10.1038/nmeth757.PubMedView Article
- Rogojina AT, Orr WE, Song BK, Geisert EEJ: Comparing the use of Affymetrix to spotted oligonucleotide microarrays using two retinal pigment epithelium cell lines. Mol Vis. 2003, 9: 482-496.PubMedPubMed Central
- Shi L, Tong W, Fang H, Scherf U, Han J, Puri RK, Frueh FW, Goodsaid FM, Guo L, Su Z, Han T, Fuscoe JC, Xu ZA, Patterson TA, Hong H, Xie Q, Perkins RG, Chen JJ, Casciano DA: Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential. BMC Bioinformatics. 2005, 6 Suppl 2: S12-10.1186/1471-2105-6-S2-S12.PubMedView Article
- Shippy R, Sendera TJ, Lockner R, Palaniappan C, Kaysser-Kranich T, Watts G, Alsobrook J: Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics. 2004, 5 (1): 61-10.1186/1471-2164-5-61.PubMedPubMed CentralView Article
- Tan PK, Downey TJ, Spitznagel ELJ, Xu P, Fu D, Dimitrov DS, Lempicki RA, Raaka BM, Cam MC: Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res. 2003, 31 (19): 5676-5684. 10.1093/nar/gkg763.PubMedPubMed CentralView Article
- Yauk CL, Berndt ML, Williams A, Douglas GR: Comprehensive comparison of six microarray technologies. Nucleic Acids Res. 2004, 32 (15): e124-10.1093/nar/gnh123.PubMedPubMed CentralView Article
- Mackay IM, Arden KE, Nitsche A: Real-time PCR in virology. Nucleic Acids Res. 2002, 30 (6): 1292-1305. 10.1093/nar/30.6.1292.PubMedPubMed CentralView Article
- Wong ML, Medrano JF: Real-time PCR for mRNA quantitation. Biotechniques. 2005, 39 (1): 75-85.PubMedView Article
- Arya M, Shergill IS, Williamson M, Gommersall L, Arya N, Patel HR: Basic principles of real-time quantitative PCR. Expert Rev Mol Diagn. 2005, 5 (2): 209-219. 10.1586/1473722.214.171.124.PubMedView Article
- Wilhelm J, Pingoud A: Real-time polymerase chain reaction. Chembiochem. 2003, 4 (11): 1120-1128. 10.1002/cbic.200300662.PubMedView Article
- Pietrzyk MC, Banas B, Wolf K, Rummele P, Woenckhaus M, Hoffmann U, Kramer BK, Fischereder M: Quantitative gene expression analysis of fractalkine using laser microdissection in biopsies from kidney allografts with acute rejection. Transplant Proc. 2004, 36 (9): 2659-2661. 10.1016/j.transproceed.2004.09.029.PubMedView Article
- de Kok JB, Roelofs RW, Giesendorf BA, Pennings JL, Waas ET, Feuth T, Swinkels DW, Span PN: Normalization of gene expression measurements in tumor tissues: comparison of 13 endogenous control genes. Lab Invest. 2005, 85 (1): 154-159.PubMedView Article
- Hughes TR, Mao M, Jones AR, Burchard J, Marton MJ, Shannon KW, Lefkowitz SM, Ziman M, Schelter JM, Meyer MR, Kobayashi S, Davis C, Dai H, He YD, Stephaniants SB, Cavet G, Walker WL, West A, Coffey E, Shoemaker DD, Stoughton R, Blanchard AP, Friend SH, Linsley PS: Expression profiling using microarrays fabricated by an ink-jet oligonucleotide synthesizer. Nat Biotechnol. 2001, 19 (4): 342-347. 10.1038/86730.PubMedView Article
- Stefano GB, Burrill JD, Labur S, Blake J, Cadet P: Regulation of various genes in human leukocytes acutely exposed to morphine: expression microarray analysis. Med Sci Monit. 2005, 11 (5): MS35-42.PubMed
- Zhao JR, Bai YJ, Zhang QH, Wan Y, Li D, Yan XJ: Detection of hepatitis B virus DNA by real-time PCR using TaqMan-MGB probe technology. World J Gastroenterol. 2005, 11 (4): 508-510.PubMedPubMed CentralView Article
- Yang DK, Kweon CH, Kim BH, Lim SI, Kim SH, Kwon JH, Han HR: TaqMan reverse transcription polymerase chain reaction for the detection of Japanese encephalitis virus. J Vet Sci. 2004, 5 (4): 345-351.PubMed
- Rajagopalan D: A comparison of statistical methods for analysis of high density oligonucleotide array data. Bioinformatics. 2003, 19 (12): 1469-1476. 10.1093/bioinformatics/btg202.PubMedView Article
- Bolstad BM, Irizarry RA, Astrand M, Speed TP: A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics. 2003, 19 (2): 185-193. 10.1093/bioinformatics/19.2.185.PubMedView Article
- Smyth GK, Speed T: Normalization of cDNA microarray data. Methods. 2003, 31 (4): 265-273. 10.1016/S1046-2023(03)00155-5.PubMedView Article
- Huber W, von Heydebreck A, Sultmann H, Poustka A, Vingron M: Variance stabilization applied to microarray data calibration and to the quantification of differential expression. Bioinformatics. 2002, 18 Suppl 1: S96-104.PubMedView Article
- Fawcett T: ROC Graphs: Notes and Practical Considerations for Researchers. Tech Report HPL-2003-4, HP Laboratories. 2004, [http://home.comcast.net/~tom.fawcett/public_html/papers/ROC101.pdf]
- Chambers JM, Hastie TH: Statistical Models . 1992, S. Wadsworth & Brooks/Cole, Pacific Grove, California
- Gene Expression Omnibus (GEO). [http://www.ncbi.nlm.nih.gov/projects/geo/]
- R/Bioconductor. [http://www.bioconductor.org]
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.