Systematic benchmark of state-of-the-art variant calling pipelines identifies major factors affecting accuracy of coding sequence variant discovery

Table 3 Aggregated median statistics of variant caller performance on WES and WGS data

Caller (filtering)^a	Type	SNP F1	SNP Precision	SNP Recall	indel F1	indel Precision	indel Recall
DeepVariant	WGS	0.995794	0.995365	0.996218	0.988316	0.986772	0.992126
Octopus (standard)	WGS	0.987666	0.991172	0.984631	0.979687	0.985000	0.981771
Octopus (forest)	WGS	0.993052	0.990870	0.995244	0.987600	0.977995	0.994595
Strelka2	WGS	0.992320	0.991075	0.9929913	0.983985	0.984252	0.984375
Clair3	WGS	0.991248	0.987123	0.995530	0.984759	0.979798	0.989770
GATK (1D)	WGS	0.991736	0.988720	0.994891	0.977392	0.966921	0.992327
GATK (HF)	WGS	0.983078	0.983781	0.983338	0.969068	0.952618	0.984655
GATK (2D)	WGS	0.991804	0.988431	0.995803	0.981741	0.972010	0.991892
FreeBayes	WGS	0.976158	0.992710	0.960205	0.933873	0.987988	0.884910
DeepVariant	WES	0.995837	0.9972385	0.994441	0.990379	0.989218	0.986523
Octopus (standard)	WES	0.992911	0.992147	0.993656	0.983605	0.981818	0.980392
Octopus (forest)	WES	0.954045	0.997630	0.915830	0.959206	0.988796	0.931507
Strelka2	WES	0.992490	0.992002	0.992391	0.978279	0.975741	0.977961
Clair3	WES	0.991704	0.990249	0.991938	0.975506	0.970350	0.980716
GATK (1D)	WES	0.986426	0.984869	0.987764	0.942208	0.956044	0.922865
GATK (HF)	WES	0.985205	0.987470	0.983273	0.970658	0.958656	0.983193
GATK (2D)	WES	0.747232	0.991695	0.641138	0.914491	0.960000	0.900826
FreeBayes	WES	0.987447	0.991496	0.983301	0.952451	0.976667	0.931507

All values are given with respect to the Novoalign v.4.02.01 read alignment. Bold font corresponds to the best values for WGS and WES data. ^a1D - 1D CNN model in GATK, 2D - 2D CNN model in GATK, HF - hard filtering with recommended parameters

ISSN: 1471-2164