Skip to main content

Table 1 Comparison of f ^ em and f ^ avg in terms of mean squared error

From: A unified approach for allele frequency estimation, SNP detection and association studies based on pooled sequencing data using EM algorithms

  α= 0.05% α= 0.1% α= 0.5% α= 1%
  MSE Cg MSE Cg MSE Cg MSE Cg
f = 0.1% 0 0 0 0 0 0 0 0
f = 0.5% 9 5 4 3 0 0 0 0
f = 1% 13 10 9 7 0 0 0 0
f = 5% 17 16 17 16 12 12 7 7
  1. Number of scenarios where MSEem > MSEavg or Cgem > Cgavg out of 18 total scenarios for each cell.