Skip to main content

Table 3 f ^ em as an estimator of f or ffrac

From: A unified approach for allele frequency estimation, SNP detection and association studies based on pooled sequencing data using EM algorithms

  α= 0.05% α= 0.1% α= 0.5% α= 1%
f MSE Cg MSE Cg MSE Cg MSE Cg
0.1% 9.8e-7 4.3e-8 9.7e-7 1.2e-7 3.2e-6 2.8e-6 1.1e-5 1.1e-5
0.5% 5.5e-6 1.9e-7 5.4e-6 1.9e-7 6.7e-6 1.5e-6 1.0e-5 5.8e-6
1% 1.2e-5 8.6e-7 1.2e-5 9.6e-7 1.3e-5 2.3e-6 1.6e-5 5.6e-6
5% 5.5e-5 1.3e-5 5.9e-5 1.7e-5 8.3e-5 4.3e-5 1.0e-4 7.0e-5
  1. The mean squared errors (MSE) and Cg's of f ^ em as an estimator for f or ffrac for various combinations of f and α, (K, G, n) = (100, 10, 3000).