Skip to main content
  • Research article
  • Open access
  • Published:

Reproducibility of oligonucleotide arrays using small samples

Abstract

Background

Low RNA yields from small tissue samples can limit the use of oligonucleotide microarrays (Affymetrix GeneChipsĀ®). Methods using less cRNA for hybridization or amplifying the cRNA have been reported to reduce the number of transcripts detected, but the effect on realistic experiments designed to detect biological differences has not been analyzed. We systematically explore the effects of using different starting amounts of RNA on the ability to detect differential gene expression.

Results

The standard Affymetrix protocol can be used starting with only 2 micrograms of total RNA, with results equivalent to the recommended 10 micrograms. Biological variability is much greater than the technical variability introduced by this change. A simple amplification protocol described here can be used for samples as small as 0.1 micrograms of total RNA. This amplification protocol allows detection of a substantial fraction of the significant differences found using the standard protocol, despite an increase in variability and the 5' truncation of the transcripts, which prevents detection of a subset of genes.

Conclusions

Biological differences in a typical experiment are much greater than differences resulting from technical manipulations in labeling and hybridization. The standard protocol works well with 2 micrograms of RNA, and with minor modifications could allow the use of samples as small as 1 micrograms. For smaller amounts of starting material, down to 0.1 micrograms RNA, differential gene expression can still be detected using the single cycle amplification protocol. Comparisons of groups of four arrays detect many more significant differences than comparisons of three arrays.

Background

The ability to measure the expression of thousands of genes at once using microarrays has opened new areas of research, including global examination of the effects of perturbations on cells or animals and the classification of tumors by their pattern of gene expression. Microarrays using cDNAs [1, 2] and oligonucleotides [3ā€“5] have both proven valuable.

Commercially available oligonucleotide microarrays provide a standardized tool that allows assay of thousands of mRNAs at one time. Affymetrix GeneChipsĀ® contain pairs of 25-nucleotide sequences (probe pairs) synthesized on silica wafers; one of each pair exactly matches the sequence of interest and the other contains a single mismatching nucleotide in the center [6, 7]. A single sequence is queried by a group of 8 to 16 probe pairs that constitute a probe set. RNA from the sample is converted to double-stranded cDNA and then labeled by in vitro transcription with biotinylated nucleotides. The biotinylated cRNA is hybridized to the GeneChipĀ®, unhybridized material is washed off, and the signal is detected using fluorescein-labeled Streptavidin [8].

The standard Affymetrix protocol [8] uses as starting material 10 Ī¼g of total RNA, from which biotinylated cRNA is synthesized. This can limit the use of this system for small samples from biopsies, laser capture microdissection or tissues from model organisms such as mice. Mahadevappa and Warrington [9] examined the effect of using less biotinylated cRNA in the hybridization. Hybridization reactions that contained the recommended 10 Ī¼g of cRNA (from human endometrium adenocarcinoma cells) detected 35% of the 1779 transcripts on the GeneChipĀ®. Reducing the amount of cRNA in the hybridization to 5 Ī¼g reduced the fraction detected to 30%, and further reducing the cRNA to 2.5 Ī¼g allowed detection of only 27% of the sequences [9]. Ohyama et al. [10] tested a modified protocol for synthesizing biotinylated cRNA from very small amounts of starting material. Total RNA from laser capture microdissected human oral cancer tissues was converted into cDNA and transcribed in vitro; the resulting cRNA was converted into cDNA and transcribed in vitro a second time to generate more cRNA; this cRNA was again converted into cDNA and biotinylated cRNA was synthesized by a third in vitro transcription. This procedure produced 10 Ī¼g of biotinylated cRNA from 0.1 Ī¼g of starting total RNA. Hybridization with 10 Ī¼g of biotinylated cRNA generated by this amplification protocol allowed detection of 30% of the 7000 transcripts on the HuGeneFL GeneChipĀ® [10]. In contrast, hybridization with 10 Ī¼g of biotinylated cRNA generated by the standard protocol from total RNA extracted from similar tissues resulted in detection of 35% of the transcripts being detected [10].

Rather than focusing upon the number of transcripts detected, the real test of a microarray protocol is the extent to which it allows differences in expression levels to be reliably detected. The biological variability inherent in most experimental models, including both genetic and environmental differences between animals or even replicate cell cultures, limits the detection of such differences. Additional variability in the treatment and handling of the models and in the RNA extraction typically occur outside the microarray laboratory, and can be reduced (but not eliminated) by careful experimental design. There is the potential for introducing additional technical variability during the synthesis of biotinylated cRNA. In evaluating a new protocol or comparing existing protocols, measures such as the yield of cRNA or the fraction of probe sets detected can be useful, but the key measure is the extent to which differences in gene expression can be detected in a realistic experiment.

We have systematically explored the use of smaller amounts of starting material (total RNA) in a model experiment that retains the individual-to-individual biological variation of a real experiment. This allowed us to compare technical variability to the biological variability in a typical experiment. We started with total RNA from individual rats exposed to two different nutritional regimens and used serial dilutions of the RNA to simulate experimental systems that provide smaller quantities of total RNA. We used the Rat Genome RGU34A GeneChipĀ® for all of the experiments. Our first goal was to determine a reasonable lower bound for total RNA that could successfully be used in the standard protocol. Second, we wanted to test a modified version of the Ohyama amplification procedure [10] that we thought would be faster, simpler and less likely to skew results due to truncation of the labeled cRNA. We examined the variability and the ability to detect significant changes between animals fed the 2 different dietary regimens when different amounts of starting material and different protocols were used.

Results

Yields and quality of cRNA

The standard protocol uses 10 Ī¼g of total RNA to produce biotinylated cRNA [8]. In our experiment, the average yield of biotinylated cRNA was 97 Ī¼g (Ā± 41 Ī¼g; mean Ā± standard deviation) when we started with 10 Ī¼g of RNA and used one-half of the double stranded cDNA product in the in vitro transcription reaction. Starting with 2 Ī¼g of total RNA, we obtained an average of 50 Ī¼g (Ā± 20 Ī¼g; standard deviation) of biotinylated cRNA after using all of the double stranded cDNA product in the in vitro transcription reaction. The 1 Ī¼g pooled samples produced an average of 5.6 Ī¼g of biotinylated cRNA (yields ranged from 2.4 Ī¼g to 7.7 Ī¼g). No difference in yield was observed between the samples prepared with the ENZO and Epicentre T7 polymerases. This lack of difference led us to select the Epicentre Ampliscribeā„¢ T7 High Yield Kit for the extra in vitro transcription step to decrease expense.

The standard Affymetrix protocol uses 15 Ī¼g of biotinylated cRNA to make a 300 Ī¼l hybridization cocktail, of which 200 Ī¼l is injected into the chip for hybridization. The yield of cRNA from both 10 Ī¼g and 2 Ī¼g of starting RNA (above) were more than sufficient for this. Because yields of cRNA starting from 1 Ī¼g total RNA were too low, we decided to use an additional amplification step for RNA samples less than 1 Ī¼g. Our modified protocol uses the initial in vitro transcribed cRNA as starting material for a second round of cDNA synthesis and in vitro transcription (Methods). The cRNA yields from the 0.5 Ī¼g samples with our protocol averaged 32 Ī¼g (Ā± 13 Ī¼g), more than enough for the standard hybridizations. The cRNA yields from the 0.1 Ī¼g samples using the same protocol averaged 10 Ī¼g with a range from 5 to 21 Ī¼g, so some samples had too little to prepare a hybridization mixture at the same concentration. Therefore, the 0.1 Ī¼g samples were hybridized to GeneChipsĀ® using 5 Ī¼g to 7.5 Ī¼g of cRNA to assess results using these limited amounts.

Aliquots of the biotinylated cRNA samples were analyzed by agarose gel electrophoresis to check the quality and length. The cRNA for both the 10 Ī¼g and 2 Ī¼g samples ranged from 200 to over 2,000 bases (before fragmentation). The cRNA from the 0.5 Ī¼g sample prepared by the amplification protocol ranged from 200 to 850 bases, a considerable decrease in maximum length. The 0.1 Ī¼g samples were too faint to judge their size range.

Changes in sensitivity as measured by detection of probe sets

For a particular tissue or cell type, the percent present and the scaling factors should be similar among all arrays in the same experiment in the absence of variability introduced by preparation, labeling, and handling of the individual samples. Because we created groups of samples diluted from the same individual RNA preparations, any group differences reflect differences in labeling and handling of the samples. We used the percent of probe sets called present and the scaling factor (see Methods) for an initial comparison among the groups (Table 1). The 2 Ī¼g samples were essentially equivalent to the 10 Ī¼g samples by these measures (Table 1). For the amplified samples (0.5 Ī¼g and 0.1 Ī¼g starting material), the percent present was decreased and the scaling factor increased compared to the non-amplified samples. The percent present dropped to 30.2% in the 0.5 Ī¼g group (78% of that detected in the 10 Ī¼g group) and even further, to 24.5%, for the 0.1 Ī¼g samples. Two other quality control measures, noise and background, were similar across all 30 GeneChipsĀ® (data not shown).

Table 1 Quality control measures as a function of starting material.

If the decrease in starting amount of RNA or the differences in protocol had no effect on the outcome, the signals from all of the reduced RNA sample size groups would be distributed similarly to the 10 Ī¼g group. SAS was used to analyze the signals from each RNA sample size group. Probe sets were separated by detection call (absent, present and marginal) and analysis was performed separately for each group (only 2% of the probe sets were called marginal; these were omitted from Table 2). Signals for the 2 Ī¼g samples are distributed similarly to the 10 Ī¼g samples (Table 2). For the amplified samples the range of signals is increased and the distribution is shifted toward higher signal values. The decrease in percent of probe sets called present on the GeneChipsĀ® from the amplified samples has the effect of lowering the average signal on the chip, requiring a higher scaling factor (Table 1). This results in an inflation of the signal values for all of the probe sets on these arrays.

Table 2 Expression levels for probe sets called absent and present by MAS5.

To examine the effects of starting with smaller amounts of RNA on the variability in detection of probe sets, we examined the number of probe sets that changed from present to absent or from absent to present when comparing the 10 Ī¼g sample to the smaller RNA samples from the same animal (Table 3). The average number of probe sets called present on the 10 Ī¼g chip and absent on the 2 Ī¼g chip (P10 to A) and called absent on the 10 Ī¼g chip and present on the 2 Ī¼g chip (A10 to P) changes were similar. The bulk of these changes were for probe sets with lower levels of expression (Table 3). This distribution is consistent with the greater variability seen in probe sets with low signals (see below); there is no significant loss of low-level transcripts in the 2 Ī¼g samples.

Table 3 Variability in the detection of probe sets.

Because the amplified samples (from 0.5 Ī¼g and 0.1 Ī¼g starting material) have a decrease in the percent of probe sets called present (Table 1), the number of probe sets called present in the 10 Ī¼g samples and absent in the amplified sample from the same animal must be greater than the number called absent in the 10 Ī¼g sample and present in the amplified samples (Table 3). Loss of signal is expected in low-level transcripts for the amplified samples because of the decrease in starting material, but probe sets present in the 10 Ī¼g and absent in the amplified samples are not confined to those probe sets with low signals. Forty-three percent of the probes not detected in the amplified samples have a signal greater than 600 in the 10 Ī¼g sample, and 5% have a signal of at least 3200. In comparison, for the 2 Ī¼g samples, only 14% of the probe sets present in the 10 Ī¼g sample and absent in the 2 Ī¼g sample had signals greater than 600, and none had signals over 3200.

The probe sets changing from absent in the 10 Ī¼g samples to present in the amplified samples (Table 3) were mostly those with low signals in the 10 Ī¼g samples, reflecting the greater noise found in probe sets with low signals (Figure 1).

Figure 1
figure 1

Log scale scatter plots of samples labeled by the standard protocol. Lines indicate a 2-fold difference. (a) Two 10 Ī¼g treated samples, all probe sets shown. (b) Same as figure 1a, but limited to the probe sets called present in Sample A (x-axis). (c) 2 Ī¼g vs. 10 Ī¼g from the same animal (Sample A in figures 1a and 1b). (d) Same as figure 1c, but limited to probe sets called present in the 10 Ī¼g sample.

Biological and technical variability

The Affymetrix MAS5 comparison analysis tool was used to compare expression levels between pairs of GeneChipsĀ®. This analysis directly compares two arrays at each individual probe pair rather than merely comparing the signal computed from all of the probe pairs for a probe set [11]. Comparisons among the 10 Ī¼g samples from different animals within each of the treatment groups showed that the biological (between animal) plus random technical variability is considerable (Table 4, 10 vs. 10). There was an average of over 700 apparent differences in expression level between pairs of GeneChipsĀ® within a single treatment group, 12% of which were 2-fold or greater. The number of increases and decreases were comparable, suggesting random rather than systematic changes. Increases and decreases were randomly distributed across probe sets with different levels of expression. This can be visualized as a scatter plot comparing two different 10 Ī¼g samples both from the vitamin-deficient group (Figure 1a and 1b); results are similar for a pair of control samples. There is noticeable scatter from the expected diagonal, and the scatter is greatly exacerbated at low signal levels. This shows variability between animals within a single treatment group (plus the technical variability in handling two samples, even with the same protocol). Figure 1b is the same pair of GeneChipsĀ® but is restricted to probe sets that were called present in the first sample (Sample A, x-axis). Analysis of the signals for all of the 10 Ī¼g arrays shows that 59% of the probe sets have signals below 300, and that 89% of these are called absent. Removing the probe sets called absent from further analyses removes most of the variability seen in this low signal range. The data in Table 4 were limited to probe sets that were called present in the baseline sample for each comparison to avoid the noise of low signal absent and marginal calls (cf. Figure 1).

Table 4 Changes in comparisons between animals within treatment.

GeneChipsĀ® from the lower RNA sample size groups were compared to the 10 Ī¼g chip from the same animal; these comparisons represent technical variation only, because the RNAs were dilutions from the same RNA preparations. To compare variability introduced by the same labeling protocol with different amounts of starting material, in Figure 1c and 1d we compared a 2 Ī¼g sample to a 10 Ī¼g sample from the same animal (sample A, x-axis in a and b). As can be seen, the variation due to differences in sample size plus the technical variability is less than the between-animal variation shown in Figure 1a and 1b. The 2 Ī¼g arrays have an average of 134 decreases and 136 increases when compared to the 10 Ī¼g samples from the same animals (Table 5, 2 vs. 10); this reflects technical variability of samples prepared by the same protocol from different amounts of starting material. Only 4% of these changes had a magnitude of 2-fold or greater, for an average of 10 changes per chip. This variability was much smaller than the biological variation seen between pairs of arrays from different animals (10 Ī¼g arrays, Table 4). There were a balanced number of increases and decreases. The changes in the 2 Ī¼g samples appear random: different probe sets change in different comparisons. Only 8 probe sets changed consistently (in at least 5 of the 6 comparisons).

Table 5 Changes in comparisons between RNA sample size groups.

To examine variability introduced by the amplification protocol, we compared a 0.5 Ī¼g sample to a 10 Ī¼g sample from the same animal (Figures 2a and 2b). This comparison includes both systematic and random variability introduced by the amplification protocol used for the 0.5 Ī¼g sample. There was much more variability between the 10 Ī¼g and 0.5 Ī¼g samples than between 10 Ī¼g and 2 Ī¼g samples or between 10 Ī¼g samples (compare Figures 1 and 2). Only probe sets that were called present in the 10 Ī¼g sample have been plotted in Figure 2b; the variability is still high. This remaining variability extends over a wider range of signals than for the comparisons in Figure 1. More of the probe sets decreased in signal than increased in signal. Many more of these changes are at least 2-fold. The comparison of 0.1 Ī¼g to 10 Ī¼g data (not shown) is very similar to the 0.5 Ī¼g to 10 Ī¼g comparison.

Figure 2
figure 2

Log scale scatter plot of amplified samples. Lines indicate a 2-fold difference. (a) 0.5 Ī¼g sample vs. 10 Ī¼g sample from the same animal as in figure 1c. (b) Same as a, but limited to probe sets called present in the 10 Ī¼g sample. (c) 0.1 Ī¼g vs. 5 Ī¼g from the same animal as in figures 2a and 2b. (d) Same as figure 2c, but limited to probe sets called present in the 0.5 Ī¼g sample.

The number of changes observed from the comparison analysis of the amplified samples to the 10 Ī¼g samples is significantly higher than for the 2 Ī¼g samples, and there are many more decreases than increases (Table 5, 0.5 vs. 10 and 0.1 vs. 10). The number of decreases exceeds 1000 per chip for both amplified groups. For the amplified samples, nearly 2/3 of the changes are 2-fold or greater (Table 5), another indication of the increased variability also seen in Figures 2a and 2b. The changes for the amplified samples were spread evenly across the range of signals, except there were fewer increases seen in the low-level transcripts of the 0.1 Ī¼g samples than for the 0.5 Ī¼g samples, reflecting the loss of more low-level transcripts in the 0.1 Ī¼g samples. Decreases in the amplified samples were more consistent, with 727 and 794 probe sets that decreased in at least 7 of the 8 samples for the 0.5 Ī¼g and 0.1 Ī¼g groups, respectively. Of the 727 probe sets that consistently changed in the 0.5 Ī¼g samples, 712 decreased in at least 6 of the 0.1 Ī¼g samples. This indicates that a group of probe sets is being systematically affected in both of the amplified groups (see below). A percentage of these decreases actually result in loss of detection of probe sets, 33% for the 0.5Ī¼g and 43% for the 0.1 Ī¼g samples.

To measure the level of variability within an amplified group, the 0.5 Ī¼g arrays within a treatment group were compared (Table 4, 0.5 vs. 0.5). The number of changes within the 0.5 Ī¼g arrays was greater than within the 10 Ī¼g arrays, 932 per chip compared to 709 for the 10 Ī¼g. Not only were there more changes, a larger percentage of the changes were 2-fold or greater, 30% vs. 12% for the 10 Ī¼g samples. This indicates that additional noise was introduced by the amplification.

The 0.1 Ī¼g and 0.5 Ī¼g samples from the same animal were similar to each other (Figures 2c and 2d, Table 5). Only 16% of the differences between the 0.1 Ī¼g and 0.5 Ī¼g groups were ā‰„ fold, compared to 65% that were ā‰„ 2 fold when comparing the amplified samples to 10 Ī¼g samples. The variation between the amplified groups (0.1 Ī¼g and 0.5 Ī¼g) is greater than the variation between the non-amplified groups (2 Ī¼g and 10 Ī¼g), and decreases outnumber increases because of the greater loss of signal in the 0.1 Ī¼g samples.

Amplification truncates the 5' ends of the RNA

A likely cause for the consistent decreases of particular probe sets in the amplified samples, not related to low signal level, is the loss of the 5' end of the transcript. Synthesis of cDNA from the cRNA initially prepared is expected to lead to some truncation of the 5' ends of the original mRNA, due to the requirement for priming during synthesis of the second strand. Indeed, the cRNA prepared by amplification from the 0.5Ī¼g samples was noticeably shorter than that prepared by the standard protocol, as detected by agarose gel electrophoresis (above). Another way to detect such potential shortening of the probe sets is to compare the signals from the Affymetrix control probe sets. There are 3 probe sets each for GAPDH and Ī²-actin, designated 3', Middle and 5' based on their relative distance from the 3' end of the transcript. The average 3'/5' ratio for both 10 Ī¼g and 2 Ī¼g samples was 1.7 or below (Table 1), representing good samples http://www.affymetrix.com. The 3'/5' ratios of the amplified samples all exceeded 3 and were as high as 14, with the average near 6 for GAPDH and 8.5 for Ī²-actin (Table 1). These ratios indicate a differential loss of the 5' ends of the transcripts for the amplified samples (see below).

Examination of the Affymetrix comparison analyses for the GAPDH and Ī²-actin probe sets gives an even better picture of the 5' loss. None of the comparisons of the 2 Ī¼g samples to the 10 Ī¼g samples from the same animals showed a significant change in signal for these probe sets. All 8 of the 0.5 Ī¼g and 0.1 Ī¼g samples had significant decreases as compared to the 10 Ī¼g sample from the same animal, the magnitude of which increases as the distance of the probe sequence from the 3' end of the transcript increases (Table 6). The amplified samples both show similar progressive loss of the more 5' sequences. For example, both amplified samples have an average of 33% of the GAPDH 3' signal (mean log2 ratio -1.6), but only 11% as much GAPDH 5' signal (log2 ratio-3.1).

Table 6 Amplified samples show differential loss of 5' sequences.

To determine if truncation of the 5' ends of the RNA may be a significant problem for many of the sequences on the GeneChipĀ®, the distance of the target sequence (from which probe sets were designed) from the 3' end of the transcripts was determined for as many probe sets as possible by BLASTing the target sequence against the nr database (Figure 3). We then compared the percent of probe sets that decreased in signal at different 3' distances. The differences between the 2 Ī¼g and 10 Ī¼g samples were evenly distributed across the 3' distances (Figure 4), reinforcing the idea that these are random differences. In contrast, the amplified samples (0.1 Ī¼g and 0. 5 Ī¼g) show a marked increase in the percent of probe sets that were decreased as the target sequence is moved farther from the 3' end (Figure 4).

Figure 3
figure 3

Distance of the target sequence from the 3' end of the transcript. The target sequence from which the probe sets are designed is aligned to the transcript. The distance in bases from the 3' end is calculated by subtracting the position of the 5' end of the alignment from the length of the transcript.

Figure 4
figure 4

Reduction in signal as a function of the distance from the 3' end. Affymetrix comparison analysis was used to compare the reduced RNA samples to the 10 Ī¼g sample from the same animal. This plot shows the percentage of probe sets called "decreased" as a function of distance from the 3' end of the transcript. Analysis was limited to probe sets called present in the 10 Ī¼g sample.

Both sets of amplified samples were affected in the same manner by this 5' truncation. The decreases seen when comparing the 0.1 Ī¼g to the 0.5 Ī¼g samples were not associated with distance from the 3' end and were not consistent for particular probe sets (only 26 probe sets decreased in at least 7 of the 8 samples). This indicates that the truncation is a result of the single cycle of amplification, rather than the amount of starting material.

Ability to detect differences in expression between treatment groups

The main goal in an experiment comparing two treatment groups is to find genes whose expression differs significantly. To assess whether the lower starting amounts of total RNA can be used successfully, a comparison of results from standard t-tests was performed. Based upon the data in Figure 1, we filter out those probe sets that are not detected in at least one of the treatment groups to be compared before performing statistical comparisons. To be conservative, we only eliminated probe sets that are not called "present" on at least half of the GeneChipsĀ® in either of the treatment groups (rather than demand that a probe set be present in all of the GeneChipsĀ® in a set; others can choose different fractions), and call this a "detection filter." This does not eliminate probe sets that are either turned on or off, because these would be present in one of the two treatment groups.

Table 7 gives the number of probe sets that met our criteria for significance: they passed the "detection filter" and were significant at p ā‰¤ 0.01 for the t-test or at the appropriate level for the Wilcoxon rank sum test (non-parametric) [13]; both tests give generally parallel results. Even though both the 0.5 Ī¼g and 0.1 Ī¼g comparisons are 4 Ɨ 4 comparisons, there is a 50% drop in the number of significant probe sets as compared to the 10 Ī¼g samples. Only part of this drop can be attributed to the decrease in the percent of probe sets that met the "detection filter" for these samples (Table 7); the extra noise introduced by the amplification, as seen in the 0.5 Ī¼g group (cf. Figure 2, Tables 4 and 5), contributes substantially, since the t-test is sensitive to increases in standard deviations of the two groups being tested.

Table 7 Number of significant probe sets for different groups of samples.

Because of the loss of two of the 2 Ī¼g arrays, the table also contains data for the subset of six that match the six remaining 2 Ī¼g arrays. Note the sharp decline in the number of probe sets meeting the p ā‰¤ 0.01 significance criteria (t-test) when the number of samples in the 10 Ī¼g class is reduced from a 4 Ɨ 4 comparison to a 3 Ɨ 3 comparison (Table 7); this attests to the additional power gained by using the additional array. The best (lowest) p-value that can be achieved in the Wilcoxon test with four samples for each treatment group is 0.0143; for three samples per group it is 0.05 [13]. The 2 Ī¼g samples also produced a similar number of significant probe sets for the Wilcoxon with a 0.05 p-value as the 10 Ī¼g samples with the same number of arrays, 585 for the 2 Ī¼g samples and 620 for the corresponding 10 Ī¼g samples. These numbers can be compared to the 869 found in the complete set of 10 Ī¼g samples (sample size of 4) with a p-value of 0.0571 for the Wilcoxon.

As expected, probe sets with low expression level (low signal) were less reproducible in comparisons between the different sample groups, as were probe sets with very low fold-changes. Reproducibility for the 2 Ī¼g samples was best for probe sets with a fold change ā‰„ 1.7 (log2 ratio ā‰„ 0.75). For the amplified samples, good reproducibility was achieved for probe sets with fold changes ā‰„ 2 (log2 ratio ā‰„ 1). Calculated fold changes for the concordant probe sets were reasonably stable across the different groups.

Estimate of technical false positive rate

The 2 Ī¼g and 10 Ī¼g groups were from the same original RNA extractions and were labeled by the same protocol. Because they were so comparable in all of our measures, these 2 groups were used to estimate the number of false positives due to technical variability to be expected using our standard t-tests. The three 2 Ī¼g samples from the normal diet animals were compared to the three 10 Ī¼g samples from the same animals using the t-test, and the three 2 Ī¼g samples from the diet deficient animals were similarly compared to the 10 Ī¼g samples from the same animals. Since both of these comparisons are between samples from the same set of similarly treated animals, one should expect no changes. Therefore, any probe sets that were found to be significantly different between the 2 Ī¼g and 10 Ī¼g samples from the same animals would be false positives. For each comparison, normal and deficient, 14 probe sets were found to be significantly different between the 2 Ī¼g and 10 Ī¼g samples, which is a false positive rate of 0.4% of the probe sets that met the "detection filter." Of these 28, only 3 of the normal group and 4 of the deficient group had a fold-change greater than 1.5 and only one in each group had a fold-change that exceeded 2. Fold-changes larger than 1.5 were seen only in probe sets with average signals < 900. Those probe sets with signals over 900 had smaller fold changes, most less than 1.3.

In comparison, for those probe sets that did not meet the "detection filter", the false positive rate was approximately 1% at a p-value of 0.01. This set of false positives was equally balanced between increases and decreases (53 increases and 58 decreases in the two treatment groups), indicating random noise and not an effect of using 2 Ī¼g instead of 10 Ī¼g of RNA. The fold changes ranged from 1.2 to 12.5. The large fold changes result from very small denominators used in the fold change calculations for these probe sets.

Discussion

This experiment demonstrated that technical variability is much smaller than biological variation when using the standard protocol. The number of differences between samples from the same animal that resulted from using the lower amount of starting material was much smaller than the biological variation between animals treated the same and labeled by the standard protocol. The standard protocol worked well with samples as small as 2 Ī¼g, producing results very similar to those of the 10 Ī¼g sample from the same animal. Although in this experiment, starting with less than 1 Ī¼g of total RNA did not produce enough biotinylated cRNA for hybridization under the normal protocol, a minor change (using vacuum evaporation to concentrate samples before cRNA synthesis, and mixing the minimum 200 Ī¼l hybridization volume) should allow use of 1 Ī¼g samples with the standard protocol. This extends the range of samples that can readily be analyzed on Affymetrix GeneChipsĀ® using the standard protocol.

We have demonstrated here that a single cycle of amplification sufficed to produce cRNA from samples as low as 0.1 Ī¼g of total RNA. The amplification protocol uses the cRNA from the initial protocol as starting material for a second round of cDNA synthesis and in vitro transcription. We hypothesized that each round of cDNA synthesis would lead to some truncation of the molecules, due both to the possibility of priming the second strand from an internal site and to cleavage of the relatively labile RNA. For this reason, we limited our amplification to a single round, rather than using two cycles as had previously been reported [10]. The hypothesized shortening was observed: the cRNA extended to about 850 nt, compared with about 2000 nt for the standard protocol. This shortening was also demonstrated by differential loss of signal from probe sets further away from the 3' end of the RNA, as shown with the Affymetrix control probe sets (Tables 1 and 6), and by the progressive loss in probe sets detected as a function of their distance from the 3' end (Figure 4). The amplification systematically affected over 700 probe sets in at least 7 of 8 samples for both the 0.1 Ī¼g and 0.5 Ī¼g groups. Indeed, of the probe sets with consistent decreases across the 0.1 Ī¼g and 0.5 Ī¼g samples about 90% have target distances greater than 400 nucleotides from the 3' end of the measured transcript, with 26ā€“28% in the 400ā€“600 range and 60ā€“62% over 600. Because the effect of the loss of the 5' end is systematic, it renders a group of probe sets designed from sequences further from the 3' end undetectable. Therefore, we recommend using the standard protocol instead of using the amplification strategy for samples down to 1 Ī¼g of total RNA, and our simplified amplification protocol for smaller samples, at least down to 0.1 Ī¼g. This extends the range of samples that can be usefully analyzed by oligonucleotide microarrays. This experiment used the Affymetrix RGU34A GeneChipĀ®, which was designed using version 34 of Uni Gene for the rat, November 1998 For newer arrays such as the new human U133 GeneChipĀ®, designed using better sequence information and improved probe designs (Affymetrix technical report Array Design for the GeneChipĀ® Human Genome 133 Set), we expect that the problem with loss of the 5' end of the transcript. will be lessened, but not eliminated, because the targets are more likely to be near the real 3' ends of the mRNAs The truncation that results from amplifying samples means that the same protocol should be used for all samples in a given study.

New protocols that increase the production of cDNA [14, 15] using primers attached to the 5' end of the transcript could increase cDNA yields from small amounts of RNA But Iscove [14] states "only a few hundred bases of extreme 3' sequence" are amplified by their method This procedure would be expected to greatly exacerbate the loss of signal due to shortened transcripts.

There was additional variation in the amplified samples (Table 5) that is probably due to the extra steps required in the protocol. This extra noise is partially responsible for the decrease in the number of probe sets that differ at a p-value ā‰¤ 0.01. This, coupled with the decrease in the percent of probe sets present, reduces the ability to find transcripts that differ significantly in expression when using the amplification protocol. The problem is exacerbated for transcripts with small differences in expression (low fold changes). Fold changes ā‰„ 2 were much more likely to be identified in the amplified samples. It may be especially helpful to increase the number of arrays used in these amplification experiments to get more power to detect changes

Our experiment also provided a false positive estimate of technical variability for the t-test with 14 found in each treatment group when comparing the 10 Ī¼g samples to the 2 Ī¼g samples (0.4% of the present probe sets.). These false positives came disproportionately from the genes expressed at lower levels. Genes expressed at these low levels often show high fold-changes because the denominator is so low (often near background), this points out the danger in emphasizing high fold-changes, rather than reproducible changes. Therefore, for genes expressed at lower levels, it might be reasonable to use a more restrictive p-value, none of the false positives had a p-value less than 0.001. Restricting probe sets to those minimally present (our "detection filter") dramatically decreases the number of false positives, from an average of 56 down to 14, restricting analysis to genes called present in a higher fraction of the arrays from one of the comparison groups could further reduce false positives, but at the cost of missing some true positives. The tradeoff can be chosen by an investigator based upon the relative cost of false positives and value of detecting differences in genes expressed at low levels.

The statistical power to detect differences was much reduced when 3 samples per group were analyzed instead of 4. The 2 Ī¼g samples and the matched 10 Ī¼g samples were able to detect 83ā€“86 differences at p ā‰¤ 0.01 as compared to 150 differences when using all of the 10 Ī¼g samples. This decrease was expected, but illustrates that a 25% decrease in expense may result in a much greater loss of information.

Conclusions

This experiment explored the effects of using less than the standard 10 Ī¼g of total RNA for an Affymetnx GeneChipĀ® experiment, and examined how biological and technical variation affect the ability to detect biological differences in a typical experiment comparing gene expression in two conditions. The overall conclusions are that (1) small amounts of RNA can be used effectively in the standard protocol, (2) even very small amounts of RNA (0.1 Ī¼g) can be used with our simplified amplification protocol to detect differential gene expression, (3) biological variation is larger than technical variation, (4) very low-level signals are prone to false positives and to less reliable fold-changes, false positives can be reduced by filtering out probe sets not reliably detected before statistical comparisons, and (5) using 4 independent biological samples is much better than using 3 samples in allowing detection of consistent changes and reducing false positives.

Methods

Labeling test

Total RNA was extracted from the livers of 4 rats fed a normal diet (untreated) and 4 fed a vitamin-deficient diet (treated) using the RNeasyĀ® kit (Qiagen Inc, Valencia, CA). The RNA was resuspended and re-extracted using the same protocol, to reduce DNA contamination. For the labeling test, two pools were created, one treated and one untreated, by mixing equal aliquots from each of the 4 RNA samples. The final concentration was adjusted to 1 Ī¼g/2 Ī¼l, and each pool was divided into 4 aliquots of 1 Ī¼g each. These pooled samples were used to determine the average cRNA yield from 1 Ī¼g of total RNA and to compare the yields of the T7 polymerases from two different in vitro transcription kits. Biotinylated cRNA was prepared using the standard Affymetrix protocol [8] except that the Epicentre AmpliScribeā„¢ T7 polymerase (Epicentre, Madison, WI) was substituted for the ENZO T7 polymerase (BioArray, High Yield RNA Transcript Labeling Kit, ENZO Diagnostics, Inc., Farmingdale, NY) for 2 of the treated and 2 of the untreated pooled samples. The yield of cRNA was estimated from absorbance at 260 nm, using an Amersham Pharmacia Biotech Ultrospec 3100 pro spectrophotometer. This part of the experiment was the only time RNAs from different animals were pooled, the cRNAs from these pooled samples were not hybridized to arrays.

Labeling for microarrays

Aliquots of each of the original 8 samples of total RNA (one from each rat) were treated as individual samples for all hybridization experiments. Each sample was serially diluted to yield 10 Ī¼l samples containing 10 Ī¼g, 2 Ī¼g, 0.5 Ī¼g and 0.1 Ī¼g total RNA (32 samples). For the 10 Ī¼g and 2 Ī¼g samples, cRNA was prepared using the standard Affymetrix protocol [8]. We made slight modifications for the 2 Ī¼g samples to increase the concentration by decreasing added water where that was possible. Because the cRNA yield from the 1 Ī¼g pooled sample test was low, we decided to amplify the smaller samples. The 0.5 Ī¼g and 0.1 Ī¼g samples were amplified by a modification of the protocol of Ohyama et al. [10], using only a single round of amplification. In short, double-stranded cDNA was synthesized from the total RNA using the SuperScript II kit from Invitrogen and the Affymetrix T7-(dT)24 primer, which contains a T7 promoter attached to a poly-dT sequence 5'-GGCCAGTGAATTGTAATACGACTCACTATAGGGAGGCGG-(dT)24-3'. The Epicentre AmpliScribe T7 High Yield Transcription kit was used to produce unlabeled cRNA by in vitro transcription with unbiotinylated NTPs. This in vitro transcription step was followed by a second round of double-stranded cDNA synthesis, and finally by in vitro transcription using the T7 RNA polymerase with the Enzo BioArray, High Yield RNA Transcript Labeling Kit with biotinylated NTPs in the usual manner. Yields of cDNA and cRNA were measured using an Amersham Pharmacia Biotech Ultrospec 3100 pro spectrophotometer in order to adjust concentrations for subsequent steps. Measurements of the amplified samples were made after the second round of cDNA synthesis and in vitro transcription, to limit the loss of sample. After the final round of cRNA synthesis, aliquots of biotinylated cRNA from each sample were electrophoresed on 1% agarose gels in TBE buffer to check them for quality and length, the buffer and gel both contained a 500 ng/ml concentration of ethidium bromide. The Invitrogen 1 Kb Plus DNA ladder was used to provide a relative measure of length.

Since it was not feasible to label this many samples at one time, a balanced experimental design was used processing groups of 10 Ī¼g and 2 Ī¼g samples for the same animal together and always labeling an equal number of normal and vitamin-deficient samples at one time. For the amplification protocol, 0.5 Ī¼g and 0.1 Ī¼g samples from the same animal were processed together for each step, again with balanced numbers of normal and vitamin deficient samples processed together.

Array hybridization

Each sample was hybridized to a separate Affymetrix RGU34A GeneChipĀ®. For most samples, hybridization cocktails of 300 Ī¼l contained 15 Ī¼g of fragmented cRNA, 200 Ī¼l of the cocktail were injected into the GeneChipĀ® for hybridization. One 2 Ī¼g sample had a lower yield so a 200 Ī¼l hybridization cocktail containing 10 Ī¼g of cRNA was made, to keep the concentration fixed. For the 0.1 Ī¼g samples, the cRNA yield was less than 15 Ī¼g so 5ā€“7 5 Ī¼g of cRNA was used for the hybridization cocktail. Data from two of the 2 Ī¼g GeneChipsĀ® (1 from each treatment group) and their hybridization cocktail were unusable due to a bad lot of BSA (used as a blocking factor during hybridization). The analyses for the 2 Ī¼g group were completed using the remaining 6 GeneChipsĀ®. For some comparisons with this group, only the corresponding 6 GeneChipsĀ® from the 10 Ī¼g samples were used.

Scanning and analysis

Each GeneChipĀ® was scanned and analyzed using Affymetrix Microarray Analysis Suite (MAS) version 5.0 [11]. Each sample was scaled to a target intensity of 1000 using the "all probe sets" scaling option, this option scales the trimmed mean target intensity to the specified value [11]. The Affymetrix MAS5 expression report provides statistics for each chip that can be used for quality control purposes. Included in this report are noise, background, the percent of probe sets called present, the scaling factor calculated by the absolute analysis algorithm and the ratio of 3' to 5' signal for GAPDH and Ī²-actin. These measures were used to judge the quality and similarity of data from the various RNA sample size groups.

MAS5 "absolute" and "comparison" expression analyses were performed [11]. The RGU34A chip contains 8799 probe sets (series of probe pairs that query parts of the same gene or EST) [7]. For each probe set, absolute analysis generates a signal value (expression level), a detection call of absent, present, or marginal, and a p-value associated with the detection call [11]. Comparison analysis examines 2 GeneChipsĀ® and indicates for each probe set whether there is a significant difference in the signal between the two arrays. The output is a change call of increase, marginal increase, decrease, marginal decrease or no change, a p-value associated with the change call, and the magnitude of the difference (as the signal log ratio, the log2(log base 2) ratio of the signal from first chip to the signal on the second or baseline chip). Comparisons were made between the vitamin-deficient and normal diet animals in each sample size group using the normal diet samples as the baseline for each comparison. All possible comparisons were made, this equaled 16 comparisons each (4 Ɨ 4) for the 10 Ī¼g, 0.5 Ī¼g and 0.1 Ī¼g groups and 9 comparisons for the 2 Ī¼g samples. Comparisons were also made between each of the 2 Ī¼g, 0.5 Ī¼g and 0.1 Ī¼g samples and the 10 Ī¼g sample from the same animal, to look for apparent differences in gene expression that are really due to the differences in the amount of starting material and sample processing. For these comparisons, the 10 Ī¼g sample was used as the baseline.

In addition, comparisons were made between the 10 Ī¼g arrays within each treatment group to measure variability, both between-animal and technical. This analysis was done for the 0.5 Ī¼g group as well. The 0.1 Ī¼g arrays were compared to the 0.5 Ī¼g arrays from the same animal, to determine how well these two amplified samples corresponded with one another.

Data from each RNA sample size group were exported and each probe set analyzed in the following ways:

1. The number of GeneChipsĀ® on which a probe set was detected within each of the treatment groups (normal vs. deficient diet) was calculated. Each present call was assigned the value 1 and each marginal call assigned 0.51. Only probe sets that are present in at least one-half of the samples from one of the treatment groups (either normal or deficient diet) were used for further analysis, we call this the "detection filter". Note that we did not require the probe set to be present in both treatment groups, just in one. For this experiment, we required a score of at least 2 for the 10 Ī¼g, 0.1 Ī¼g and 0.5 Ī¼g groups (4 GeneChipsĀ® in each treatment group) and 1.51 for the 2 Ī¼g and corresponding 6-sample subset of the 10 Ī¼g groups (3 GeneChipsĀ® in each treatment group).

2. The sum of the calls from the comparison analyses was calculated, using 1 for increase, 0.5 for marginal increase, 0 for no change, -0.5 for marginal decrease, and -1 for decrease.

3. The mean, standard deviation and coefficient of variation of the signal for each probe set within each treatment group (normal and deficient diet) were calculated.

4. Student's t-test for equal means with the assumption of unequal variance [12] was calculated to test for significant differences in signal (expression level) between the normal and vitamin deficient groups. This test was applied separately to the signals and the log2 transformation of the signals.

5. The Wilcoxon rank-sum non-parametric test for equal means between normal and deficient diet groups [13] was performed.

6. The log2 ratio (fold change) of the mean signals for normal vs. deficient diet groups was calculated:

log2 ratio = log2((mean of signal for normal) / (mean of signal for deficient))

Changes within an RNA sample size group between normal and deficient diet groups were assessed as significant if the probe set passed the "detection filter" (#1 above) and the p-value was less than 0.01 for the t-test using either the original signal or log transformed signal (#4 above).

Scatter plots comparing individual samples were created using Microsoft ExcelĀ®. SAS (SAS Institute Inc., Cary, NC) was used for univariate analysis of the signals for each RNA sample size group.

Distance of target from 3' end

To calculate the distance of the target from the 3' end of the interrogated sequence, the target sequence for each probe set (supplied by Affymetrix; http://www.affymetrix.com was BLASTed [16, 17] against the nr (non-redundant) database at NCBI http://www.ncbi.nlm.nih.gov. Using the first (best) alignment retumed, the distance of the 5' end of the target sequence from the 3' end of the mRNA or EST in the nr database was calculated (Figure 3). A few of the alignments were rejected because the first alignment was to a BAC sequence or to the wrong strand.

Abbreviations

cRNA:

complementary RNA

cDNA:

complementary DNA

IVT:

in vitro transcription

MAS5:

microarray analysis suite version 5 from Affymetrix

nr:

non-redundant (nucleotide database at NCBI)

nt:

nucleotides

References

  1. Chu S, DeRisi J, Eisen M, Mulholland J, Botstein D, Brown PO, Herskowitz I: The transcriptional program of sporulation in budding yeast. Science. 1998, 282: 699-705. 10.1126/science.282.5389.699.

    ArticleĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  2. Spellman PT, Sherlock G, Zhang MQ, lyer VR, Anders K, Eisen MB, Brown PO, Botstein D, Futcher B: Comprehensive identification of cell cycle-regulated genes of the yeast Saccharomyces cerevisiae by microarray hybridization. Mol Biol Cell. 1998, 9: 3273-3297.

    ArticleĀ  PubMed CentralĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  3. Beer DG, Kardia SL, Huang CC, Giordano TJ, Levin AM, Misek DE, Lin L, Chen G, Gharib TG, Thomas DG: Gene-expression profiles predict survival of patients with lung adenocarcinoma. Nat Med. 2002, 8: 816-824.

    CASĀ  PubMedĀ  Google ScholarĀ 

  4. Giordano TJ, Shedden KA, Schwartz DR, Kuick R, Taylor JM, Lee N, Misek DE, Greenson JK, Kardia SL, Beer DG: Organ-specific molecular classification of primary lung, colon, and ovarian adenocarcinomas using gene expression profiles. Am J Pathol. 2001, 159: 1231-1238.

    ArticleĀ  PubMed CentralĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  5. Matejuk A, Dwyer J, Zamora A, Vandenbark AA, Offner H: Evaluation of the effects of 17beta-estradiol (17beta-e2) on gene expression in experimental autoimmune encephalomyelitis using DNA microarray. Endocrinology. 2002, 143: 313-319.

    CASĀ  PubMedĀ  Google ScholarĀ 

  6. Lipshutz RJ, Fodor SP, Gingeras TR, Lockhart DJ: High density synthetic oligonucleotide arrays. Nat Genet. 1999, 21: 20-24. 10.1038/4447.

    ArticleĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  7. Lockhart DJ, Dong H, Byrne MC, Follettie MT, Gallo MV, Chee MS, Mittmann M, Wang C, Kobayashi M, Horton H, Brown EL: Expression monitoring by hybridization to high-density oligonucleotide arrays. Nat Biotechnol. 1996, 14: 1675-1680.

    ArticleĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  8. Affymetrix: GeneChipĀ® Expression Analysis Technical Manual. Santa Clara, CA: Affymetrix. 2001

    Google ScholarĀ 

  9. Mahadevappa M, Warrington JA: A high-density probe array sample preparation method using 10- to 100-fold fewer cells. Nat Biotechnol. 1999, 17: 1134-1136. 10.1038/15124.

    ArticleĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  10. Ohyama H, Zhang X, Kohno Y, Alevizos I, Posner M, Wong DT, Todd R: Laser capture microdissection-generated target sample for high-density oligonucleotide array hybridization. Biotechniques. 2000, 29: 530-536.

    CASĀ  PubMedĀ  Google ScholarĀ 

  11. Affymetrix: Affymetrix Microarray Suite User's Guide, version 5.0. Santa Clara, CA; Affymetrix. 2001

    Google ScholarĀ 

  12. Freund JE: Mathematical Statistics. Upper Saddle River, NJ: Prentice Hall. 1992, 462-465. Fifth

    Google ScholarĀ 

  13. Pagano M, Gauvreau K: Principles of Biostatistics. Pacific Grove, CA: Brooks/Cole. 2000, 308-312. Second

    Google ScholarĀ 

  14. Iscove NN, Barbara M, Gu M, Gibson M, Modi C, Winegarden N: Representation is faithfully preserved in global cDNA amplified exponentially from sub-picogram quantities of mRNA. Nat Biotechnol. 2002, 20: 940-943. 10.1038/nbt729.

    ArticleĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  15. Ginsberg SD, Che S: RNA amplification in brain tissues. Neurochem Res. 2002, 27: 981-992. 10.1023/A:1020944502581.

    ArticleĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  16. Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ: Basic local alignment search tool. J Mol Biol. 1990, 215: 403-410. 10.1006/jmbi.1990.9999.

    ArticleĀ  CASĀ  PubMedĀ  Google ScholarĀ 

  17. Altschul SF, Madden TL, Schaffer AA, Zhang J, Zhang Z, Miller W, Lipman DJ: Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res. 1997, 25: 3389-3402. 10.1093/nar/25.17.3389.

    ArticleĀ  PubMed CentralĀ  CASĀ  PubMedĀ  Google ScholarĀ 

Download references

Acknowledgements

This work was supported in part by a grant from the Indiana 21st Century Research and Technology Fund (H.J.E.) and by the Indiana Genomics Initiative. The Indiana Genomics Initiative (INGEN) of Indiana University is supported in part by Lilly Endowment Inc.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Howard J Edenberg.

Additional information

Authors' contributions

JNM participated in the design of the study, performed statistical analysis and drafted the manuscript. REJ and CRN carried out the biochemistry and aided in optimizing the methods. DWC designed and coordinated the original dietary study and provided the total RNA for this work. HJE conceived the study, and participated in its design, coordination, analysis and writing. All authors read and approved the final manuscript.

Authorsā€™ original submitted files for images

Rights and permissions

Reprints and permissions

About this article

Cite this article

McClintick, J.N., Jerome, R.E., Nicholson, C.R. et al. Reproducibility of oligonucleotide arrays using small samples. BMC Genomics 4, 4 (2003). https://doi.org/10.1186/1471-2164-4-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2164-4-4

Keywords