The investigators compared the expression of 579 different proteins. If cell phone radiation did nothing at all, you'd expect 5% of these comparisons to lead to 'statistically significant' differences at a significance level of 5% (the authors state that they did no correction for multiple corrections). Multiply 579 comparisons by 5% -- you'd expect about 28 'statistically significant' changes just by chance. In fact, as the authors note in the Discussion, the paper reports fewer 'significant' effects than this.
It seems to me that the effects (and statistics) presented in the paper are entirely consistent with the hypothesis that cell phone radiation has no effect at all on protein expression, and that the small P values are simply due to testing multiple statistical hypotheses. It certainly is possible that cell phone radiation affects protein expression, but this paper does not show convincing evidence of any such changes.
Competing interests
None declared
Basic knowledge
Alexander Lerchl, Jacobs University Bremen
14 April 2008
I fully agree with Motulsky' comments on the statistical pitfalls of multiple comparisons. This problem though is not new, and reviewers of manuscripts should know it.
Competing interests
none declared
I would not be so hasty...
Dariusz Leszczynski, STUK
16 April 2008
I would not be so hasty. Statistical analysis is important no doubt about it. However it shows only probability, not certainty... We had similar situation in our earlier study (1). We have identified changes in expression of several proteins but the number of affected proteins was small. However, one of the proteins (vimentin) that should be according to statistics a “false positive”, as it “belonged to the pool of false positives”, was not. When this protein’s expression was tested with another method it appeared that it was indeed positive finding, and not only quantitatively but also qualitatively. Therefore, one should be more cautious before automatically throwing out everything for sake of statistical numbers that show probability, not certainty. Hypothetically speaking, what if there would be identified more than the 5% needed spots. Say 40 spots among the 579. Among these 40 spots would be both real and false positives (2) - but statistically speaking we would have positive finding. So, when the statistical analysis is done it is also necessary to look at the proteins themselves and re-confirm whether the change is real or false. This especially applies to weak stimuli that exert weak effects.
1. R. Nylund and D. Leszczynski, Proteomics analysis of human endothelial cell line EA.hy926 after exposure to GSM 900 radiation. Proteomics 4, 1359-1365 (2004).
2. Leszczynski D. Letter to the Editor: Mobile phone radiation and gene expression. Radiation Res. 167, 121 (2007).
Competing interests
Co-author of the commented paper
But which were those 8 proteins?
Miguel A Andrade-Navarro, Max Delbruck Center for Molecular Medicine
24 June 2008
Significant or not, I am curious to know the nature of those eight proteins that were differentially expressed.
Competing interests
No competing interests
Identity of the affected proteins
Dariusz Leszczynski, STUK
25 June 2008
I do agree that independently of the strength of the statistical evidence, it would be of great interest to know the identity of the affected proteins. However, as we have stated in the article, we were unable, for technical reasons, to identify the proteins using Maldi-ToF. We are presently making preparations for a larger study (50 volunteers) in which we will attempt to confirm the results of the published pilot study. In the new study, that will be executed using new technological approach, we should be able to identify the affected proteins (we have learned from our "mistakes"...). However, it will take 2-3 years of work before we will be able to name-the-names of the proteins.
Multiple comparisons
4 March 2008
The investigators compared the expression of 579 different proteins. If cell phone radiation did nothing at all, you'd expect 5% of these comparisons to lead to 'statistically significant' differences at a significance level of 5% (the authors state that they did no correction for multiple corrections). Multiply 579 comparisons by 5% -- you'd expect about 28 'statistically significant' changes just by chance. In fact, as the authors note in the Discussion, the paper reports fewer 'significant' effects than this.
It seems to me that the effects (and statistics) presented in the paper are entirely consistent with the hypothesis that cell phone radiation has no effect at all on protein expression, and that the small P values are simply due to testing multiple statistical hypotheses. It certainly is possible that cell phone radiation affects protein expression, but this paper does not show convincing evidence of any such changes.
Competing interests
None declared
Basic knowledge
14 April 2008
I fully agree with Motulsky' comments on the statistical pitfalls of multiple comparisons. This problem though is not new, and reviewers of manuscripts should know it.
Competing interests
none declared
I would not be so hasty...
16 April 2008
I would not be so hasty. Statistical analysis is important no doubt about it. However it shows only probability, not certainty... We had similar situation in our earlier study (1). We have identified changes in expression of several proteins but the number of affected proteins was small. However, one of the proteins (vimentin) that should be according to statistics a “false positive”, as it “belonged to the pool of false positives”, was not. When this protein’s expression was tested with another method it appeared that it was indeed positive finding, and not only quantitatively but also qualitatively. Therefore, one should be more cautious before automatically throwing out everything for sake of statistical numbers that show probability, not certainty. Hypothetically speaking, what if there would be identified more than the 5% needed spots. Say 40 spots among the 579. Among these 40 spots would be both real and false positives (2) - but statistically speaking we would have positive finding. So, when the statistical analysis is done it is also necessary to look at the proteins themselves and re-confirm whether the change is real or false. This especially applies to weak stimuli that exert weak effects.
1. R. Nylund and D. Leszczynski, Proteomics analysis of human endothelial cell line EA.hy926 after exposure to GSM 900 radiation. Proteomics 4, 1359-1365 (2004).
2. Leszczynski D. Letter to the Editor: Mobile phone radiation and gene expression. Radiation Res. 167, 121 (2007).
Competing interests
Co-author of the commented paper
But which were those 8 proteins?
24 June 2008
Significant or not, I am curious to know the nature of those eight proteins that were differentially expressed.
Competing interests
No competing interests
Identity of the affected proteins
25 June 2008
I do agree that independently of the strength of the statistical evidence, it would be of great interest to know the identity of the affected proteins. However, as we have stated in the article, we were unable, for technical reasons, to identify the proteins using Maldi-ToF. We are presently making preparations for a larger study (50 volunteers) in which we will attempt to confirm the results of the published pilot study. In the new study, that will be executed using new technological approach, we should be able to identify the affected proteins (we have learned from our "mistakes"...). However, it will take 2-3 years of work before we will be able to name-the-names of the proteins.
Competing interests
Co-author of the commented paper.