Skip to main content

Table 2 Comparison of f ^ em and f ^ avg in terms of average relative error

From: A unified approach for allele frequency estimation, SNP detection and association studies based on pooled sequencing data using EM algorithms

 

α= 0.05%

α= 0.1%

α= 0.5%

α= 1%

f

R E f ^ avg

R E f ^ em

R E f ^ avg

R E f ^ em

R E f ^ avg

R E f ^ em

R E f ^ avg

R E f ^ em

0.1%

52.0

9.4

102.0

15.6

502.0

72.0

1000.0

146.0

0.5%

10.3

4.0

20.2

4.9

99.5

13.3

199.0

26.5

1%

5.0

3.3

10.0

3.7

49.2

5.7

98.3

9.5

5%

1.0

5.4

1.9

6.0

9.1

6.7

18.1

6.3

  1. The average RE of f ^ avg and f ^ em for different values of minor allele frequency f and sequencing error rate α.