 Research
 Open Access
 Published:
Mathematical modelling for variations of inbreeding populations fitness with single and polygenic traits
BMC Genomics volume 18, Article number: 196 (2017)
Abstract
Background
Inbreeding mating has been widely accepted as the key mechanism to enhance homozygosity which normally will decrease the fitness of the population. Although this result has been validated by a large amount of biological data from the natural populations, a mathematical proof of these experimental discoveries is still not complete. A related question is whether we can extend the wellestablished result regarding the mean fitness from a randomly mating population to inbreeding populations. A confirmative answer may provide insights into the frequent occurrence of selffertilization populations.
Results
This work presents a theoretic proof of the result that, for a large inbreeding population with directional relative genotype fitness, the mean fitness of population increases monotonically. However, it cannot be extended to the case with overdominant genotype fitness. In addition, by employing multiplicative intersection hypothesis, we prove that inbreeding mating does decrease the mean fitness of polygenic population in general, but does not decrease the mean fitness with mixed dominantrecessive genotypes. We also prove a novel result that inbreeding depression depends on not only the mating pattern but also genetic structure of population.
Conclusions
For natural inbreeding populations without serious inbreeding depression, our theoretical analysis suggests the majority of its genotypes should be additive or dominantrecessive genotypes. This result gives a reason to explain why many hermaphroditism populations do not show severe inbreeding depression. In addition, the calculated purging rate shows that inbreeding mating purges the deleterious mutants more efficiently than randomly mating does.
Background
In genetic terminology, inbreeding is the breeding of two individuals who are related to each other. Here inbred is defined as the individual that is generated from inbreeding. The closeness of individuals is important in inbreeding, which has substantial influence on the fitness of offspring individuals. Inbreeding depression refers to the reduced survival and fertility of offspring of related individuals. An important mechanism that leads to inbreeding depression is overdomain in which the heterozygote has greater phenotype value and perhaps is more fit than the homozygous state for either of the alleles that it comprises. Therefore there are two major approaches to increase homozygosity but decrease fitness. The first method is to increase homozygosity for partially recessive detrimental mutations; while the second approach is to increase homozygosity for alleles at loci with heterozygote advantage (overdominance). However, it is still under debate which mechanism is more dominat in nature [1–5]. This is an important question in genetics because two differnet mechanisms will lead to different theories regarding the trait values of the crossbred progeny. If the overdominance theory is valid, the mean trait value of crossed lines will return to the equivalence of the outbred population, as heterozygosity will be restored. On the other hand, the partial dominance theory predicts that the mean trait value will exceed that of the outbred population. In this case, in addition to the restore of heterozygosity, crossbred individuals will be purged from their genetic load [6, 7].
Genetic purging is the process for frequency reduction of a deleterious allele. It has been accepted that purging was achieved when inbreeding depression is caused by deleterious recessive alleles. A fitness rebound in inbred populations provides evidence to support the partial dominance mechanism [8, 9]. The effect of genetic purging have been observed in a number of experimental studies [10, 11], but overall, the evidence for purging in plant and animal populations is still limited. In addition, the combined effect of both inbreeding and purging is important for evaluating the evolutionary consequence of inbreeding and for making recommendations in conservation [12]. Although the inbreedingpurging model has been proposed to predict the evolution of the fitness and inbreeding load [13], a number of model parameters should be estimated from observation data. Thus it is high demanded to explore the role of purging in restoring fitness and calculation of purging speed.
The number of phenotypes produced for a given trait depends on how many genes control the trait. There are two types of trait. The single gene trait is controlled by a single gene that has two alleles. On the other hand, polygenic trait is controlled by two or more genes and has 2 or more alleles. Unlike monogenic traits, polygenic traits do not follow patterns of Mendelian inheritance (discrete categories). Instead, their phonotypes vary along a continuous gradient, such as the height of animals and the colour of human skin. Although we have conducted theoretical study for the single trait [14], the study of inbreeding depression for polygenic traits is more challenging and there is limited theoretical result regarding the variation of mean fitness under inbreeding.
In this work we will conduct novel theoretical studies for inbreeding depression in large population with Mendelian and polygenic traits. First, for single locus, we will explore the conditions for the monotonical increase of the mean fitness. We also derive a formula to calculate the purging rate. In addition, for multiple loci, we will find conditions for inbreeding mating reducing the mean fitness of the population. Furthermore, this work will show mechanisms to determine inbreeding depression. As an application, we present additional reasons for the frequent occurrence of selffertilization populations. The wellknown reason is that selfpollination does not need to be visited by animals to produce seed and hence have ecological advantageous under some circumstances [15].
Methods
Inbreeding of Mendelian traits
A Mendelian trait is a single trait that is regulated by a single locus and shows a simple discrete inheritance pattern. According to Wright’s formula [5, 16], at one autosomal locus of two alleles A and a with frequency x and 1−x, the three diploid genotypes AA,Aa and aa have frequency
respectively, where f is the inbreeding coefficient that is defined as the probability that two homologous alleles in an individual are identical by descent (IBD) [5, 17]. In addition, the fitness of AA,Aa and aa is denoted as w _{11},w _{12},w _{22}, respectively. Thus the change of frequency Δ x=x ^{′}−x is
where x ^{′} is the frequency of A in the next generation, and the mean fitness is given by
For the difference Δ w between the mean fitnesses of two successive generations, we have shown in [14] that
where Δ w=w ^{′}−w,w ^{′} is the mean fitness of the next generation and
Equation (2) suggests that a sufficient condition for Δ w>0 is
Thus, for random mating (f=0), the mean fitness increases monotonically. Similar observation can be found for the case w _{11}=w _{22} even if f≠1. For example, using a different approach, Ziehe and Roberds [18] has shown that, when caused by symmetric homozygous disadvantage at single locus, inbreeding depression is always less than onethird [16]. Here “symmetric” means w _{11}=w _{22}. Thus for the populations considered in [18], the mean fitness increases monotonically.
Inbreeding depression
Now we discuss the variation of mean fitness due to the inbreeding mating. To be consistent with the published literature, from now on, we always make the three genotypes of a population as (1,1−hs,1−s) in the sequel. Thus the mean fitness w _{ o } with an initial frequency x of a randomly mating population is
and the mean fitness w _{ s } of x for the inbreeding mating with inbreeding coefficient f is
where y=1−x. The inbreeding depression is defined by
Based on this definition the inbreeding coefficient is a function of inbreeding coefficient f, frequency x, degree of dominance h and selection coefficient s, namely δ=δ(x,f,h,s). We first have the following theory for the changes of mean fitness.
Proposition 1
Let the genotypes of an inbreeding population be (1,1−hs,1−s). Then inbreeding mating reduces the mean fitness for the following population:

1.
the population with overdominance genotype, or with a partial dominant genotype. That is, inbreeding depression occurs.

2.
the population with underdominance genotype, or with a partial recessive genotype. In this case, no inbreeding depression occurs.
It is well known that an overdominant population has an internal equilibrium. But it is not true for inbreeding populations. In fact, a population with genotypes (1,1−hs,1−s) has an internal equilibrium if and only if the equation
has a solution in (0,1). Thus the following Proposition gives the condition for the nonexistence of the internal equilibrium.
Proposition 2
An inbreeding population with overdominant genotypes (1,1−hs,1−s) (h<0) has no internal equilibrium if and only if
For example, if the degree of dominance satisfies −1<h<0, there would not be any internal equilibrium for a population of selffertilization plants with f=0.5.
Multiplicative interactions for polygenic traits
Genes may interact to or regulate each other in a number of different ways. However, genes normally interact multiplicatively for the fitnessreducing effects of homozygosity for deleterious alleles, namely mutant alleles or alleles at loci with overdominance [19, 20, 24]. Accordingly the relative viability w of an individual over all loci is assumed to be the product of the viability value w _{ A } for individual locus A, namely w=w _{ A } w _{ B } w _{ C }⋯. All the loci are assumed to assort independently, namely they are unlinked. It is also assumed that there is no mutation at these loci over the period of inbreeding. In addition, we assume that the selective advantage s and initial frequency are the same for all loci. Then each w _{ A } can be written as a function of h(≤1/s), given by
Then the overall inbreeding mean fitness is
and that for randomly mating is given by
To proceed an theoretic analysis, we write the values of these mean fitness as
Results
Increase of mean fitness for Mendelian traits
We have described the methods for measuring the fitness of inbreeding population in the previous section. The following theorem gives the condition for the increase of mean fitness.
Theorem 1
For an inbreeding population with relative genotype fitness
where 0≤h,s≤1, the mean fitness increases monotonically for any inbreeding coefficient 0≤f≤1.
Proof
For the difference between the mean fitness of two successive generations (2), we have
To prove Δ w>0, it suffices to show
If 0.5≤h≤1, then the inequality (4) holds since \(\frac {f}{fh+1h}\le 1\). On the other hand, when 0≤h≤0.5, using the identity
we can check the inequality (4) still holds. Thus we have showed that Δ w≥0 for any initial frequency x and any value of h satisfying 0≤h≤1. The proof is completed. □
Figure 1 clearly shows that the mean fitness of both randomly mating and inbreeding populations increase monotonically although at each generation, the mean fitness of randomly mating population is higher than that of inbreeding one. This result is consistent with the wellestablished result for a randommating population [17]. It also gives the reason for the existence of selffertilization plants, even though the most extreme form of inbreeding may reduce the heterozygosity of these plants by 50% per generation. Classically, it was known that selfing has two primary advantages over outcrossing [21], namely the reproductive assurance to ensure seed production and transmission advantage by serving as pollen donors for other individuals and for themselves [15]. Since most genotypes of a selffertilization plant are either partial dominance or partial recessive [8, 22–24], Theorem 1 presents an additional explanation: selffertilization population increases its mean fitness of population by generating steadily. However, Theorem 1 cannot be extended to the inbreeding population with h>1 (overdominance) or h<0 (underdominance). The counter examples are given in Table 1, reference [14] or Fig. 2 below.
On the other hand, the monotonically increasing for mean fitness of a population is not the necessary condition for the population avoiding extinct. Our next goal is to define a quantity which characterize the condition whether a population becomes extinct. To this regard, the mean fitness is denoted as w(x) to emphasize the importance of the initial frequency x. Then we consider the sum of Δ w(x) for all x∈(0,1) of an allele A, where Δ w(x) is defined by Eq. (2). The averaged mean fitness (AMF) of a population is defined by
The biological meaning of AMF is that AMF>0 holds if and only if the population does not become extinct. Thus Theorem 1 in fact shows that AMF for directional inbreeding population is always positive because in this case Δ w(x)>0 for each x≠0. The following theorem gives the value of AMP without any additional condition.
Theorem 2
The AMF of any inbreeding population is positive.
Proof
The proof is given in Appendix section. □
Figure 2 b and 2 d schematically shows this point, where the genotype fitness is (1,1.4,0.8) and inbreeding coefficient f=0.25. From Fig. 2 b, when the population goes to the 8th generation, the frequency x becomes 0.6338 and the mean fitness is 1.0831. Then the mean fitness starts to decrease (that is, Δ w(x)<0) until x reaches to the equilibrium (x ^{∗}≈0.6667 in this case). On other hand, Fig. 2 d clearly shows that the positive area which is above the xaxis (and below the curve) is much larger than the negative area which is below the xaxis and above the curve. These figures show that AMF>0 intuitively.
Inbreeding depression
For inbreeding depression, we give the theorem which provides insights into inbreeding mating.
Theorem 3
Let (1,1−hs,1−s) be the genotypes of an inbreeding population with inbreeding coefficient f. Then we have

1.
If s<1, δ, as a function of x (3), arrives its maximum at \(x^{*}= 1 \frac {1\sqrt {1s}}{s}=\frac {\sqrt {1s}}{1+\sqrt {1s}}\) for \(h< \frac {1}{2}\) (partial dominant or overdominant) and \({\lim }_{x\rightarrow 0}\delta ={\lim }_{x\rightarrow 1}\delta =0\);

2.
If s=1 and h≠1,δ is decreasing and \({\lim }_{x\rightarrow 0}\delta =\frac {f(12h)}{2(1h)}\);

3.
If s=1 and h=1, then \(\frac {w_{s}}{ w_{o}}\) is not continuous at x=0 and \({\lim }_{x\rightarrow 0}\frac {w_{s}}{ w_{o}}= \infty \);

4.
If x≠0, then δ, as a function of h, is decreasing and \({\lim }_{h\rightarrow \infty }\delta = f\);

5.
δ, as a function of s, is strictly increasing;

6.
δ, as a function of f, is strictly increasing.
Remark 1
This result may provide an explanation to the puzzle for the classical strategy of cultivation. When choosing the offspring (i.e. seeds) of a favourable mutation and then inbreeding between the improved offspring, our results suggest that this inbreeding mating greatly increases the mean fitness of the population. This interesting result explains biological observations but is contradict to the belief that inbreeding mating decreases mean fitness. The key point is that in this case the initial frequency of the favourable mutation normally is very low. However, if the first generation of the population still uses the same selection strategy with the favourable mutation, the initial frequency will be very high, because they all already have the favorable mutation. However, the following Theorem 5 shows that the inbreeding mating for the second generation actually reduces the mean fitness.
Purging rate
Although selection against highly recessive alleles become more effective due to the rise of selfing rates [25], the amplified selfing rates have little impact on the alleles that are more nearly additive (i.e. h=0.5). It was wellknown that, for a population with directional selection (i.e., 0≤h≤1), inbreeding mating purges deleterious mutants. Here we propose a formula to determine the value of purging rate. For an inbreeding population with genotype (1,1−hs,1−s) and inbreeding coefficient f, the average time t(x,x _{ t }) required for allele to change the frequency from x to x _{ t } is
where \(a=\left (1f\right)(2h1),\,b=\frac {fh+1h}{a}, K=2fhf2h\) and f satisfies \(\frac {f}{1f}\le h\le 1\). When 0≤h≤1, the purging rate satisfies
Figure 3 gives the calculated time t(x,x _{ t }) based on different genotypes and different frequency x _{ t }. Numerical results show that the random mating population need more time than the inbreeding population from a fixed initial frequency x=0.9 to a new frequency, which suggests that inbreeding mating purges deleterious mutants more effectively than randomly mating. In particular, Fig. 3(c) suggests that the purging strength of the inbreeding population is substantial higher than that of the random mating population. In addition, Comparing the three cases with h=0,0.3,0.8, Fig. 3 suggests that a small value of h leads to a large purging rate. These results may provide a reason for the existence of so many selfing fertilisation plants, though these plants have various degree of inbreeding depression. This may be one of the advantages of selfing fertilisation plants.
Multiplicative interactions for polygenic traits
We first give an example to show that the theory established in the previous section may not be able to explain some experimental observations. For example, for an selfing population with inbreeding coefficient f=0.5, nature selection pressure s=0.05 and h=0 (i.e. a completely dominant mutant), the inbreeding depression at a single locus (3) is δ=0.006. However, the observation in natural populations shows that a completely recessive mutant usually will cost around 20−30% inbreeding depression [9, 23, 24]. This difference suggests that the trait may be determined by multigenes.
In our previous study, we proposed a theorem for the inbreeding mating without proof. The following result is an updated version of Theorem 3.1 in [14].
Proof
For each partial dominance genotype (1,1−hs,1−s) with 0≤h<0.5, there is a partial recessive genotype (1,1−(1−h)s,1−s). It suffices to show
Let A=w _{ o }(0.5)=1−sxy−sy ^{2}=1−sy,a=sxy(1−2h) and z=fsxy(1−2h)=fa. Then we have that
In addition, the logratio of mean fitness is
since A ^{2}≥a ^{2}. Hence the proof is completed.
Theorem 4
Assume that a polygenic trait is determined by the genotypes whose relative fitness is (1,1−hs,1−s), where the selective advantage coefficient s is fixed and the degree of dominance h is uniformly distributed over [0,1] (i.e., the trait is determined by the dominantrecessive genotypes). Then inbreeding mating does not decrease the mean fitness of population. □
Note that the inbreeding depression δ is an allelespecific property. However Theorem 4 shows a genotypespecific property which is independent of the choice of alleles.
From Proposition 1, we immediately have the following results.
Proposition 3
If a polygenic trait is determined by partial dominant or overdominant genotypes, inbreeding mating reduces the mean fitness of the population.
Theorem 5
Assume that a polygenic trait is determined by the genotypes whose relative fitnesses are (1,1−hs,1−s), where the selective advantage coefficient s is fixed and the degree of the dominance h is uniformly distributed over [ −1,1]. Then inbreeding mating reduces the mean fitness of the population for all initial frequency x satisfying
Proof
The proof is given in Appendix section. □
Remark 2
The restriction (6) in Theorem 5 is a sufficient condition, but it can not be omitted. For example, suppose s=1,f=0.5 and a polygenic trait is determined by (1,1−hs,1−s) with h=−1,−0.8,−0.6,…,1. When x=0.1, we have
which suggests that inbreeding mating reduces the fitness. Note that \(\frac {43f}{63f}\approx 0.56\). This means that the above condition (6) is not necessary. However, when x=0.05, we have ln(w _{ s })=−29.77>−30.15= ln(w _{ o }) and hence w _{ s }>w _{ o }. This means that the condition (6) cannot be removed.
Recent studies have showed that there are many overdominant loci in rice [3, 4, 26]. In addition, overdominance and epistasis might play an important role as the genetic basis of heterosis in Brassica rapa [27]. The following theorem investigates the relationship between the number of genotypes and the mean fitness.
Theorem 6
If the set of genotypes which determine a polygenic trait contains enough overdominant genotypes, inbreeding mating reduces the mean fitness of the population for any given initial frequency x≠0, that is, w _{ s }≤w _{ o }.
Proof
The proof is given in Appendix section. □
However, in natural populations, there may not be many overdominance genotypes [28]. Another restriction for over dominance (1,1−hs,1−s) is that h may not be very small. A wellknown example is Sickle cell anaemia which occurs when a particular pair of genes carry the ’sicklecell trait’ which has not been eliminated from the human population by selection. The reason is that there is only one pair of genes carrying the sicklecell trait and individuals (“carrier”) is highly resistant to malaria. On the other hand, a person whose genes do not carry the sicklecell trait is susceptible to malaria. For example, according to the data from the World Health Organization, the carrier frequency ranges from 10 to 40% across equatorial Africa. The genotype’s fitness is 1,1−h and 0 with −1.35≤h≤−1.05. Thus h (≥−2s,s=1) is not very small.
Discussions
We now discuss other factors for determining inbreeding depression. Suppose that initially we have two homozygous, inbred lines P _{1} and P _{2} which are used as parents. Let \(\bar P_{1}>\bar {P}_{2}\), where \(\bar {P}\) is the mean fitness or fitnessrelated trait of P. The genotypes of P _{1} and P _{2} for this gene are A ^{+} A ^{+} and A ^{−} A ^{−} while their hybrid is A ^{+} A ^{−}. Let the average phenotype of the two parents be m and the additive and dominance genetic component of means (the averaged phenotypic value) be \(a_{A} \left (=\bar {P}_{1}m\right)\) and \(d_{A}\left (=\bar {F}_{1}m\right)\), respectively, where F _{1} is the first generation of hybrid offsprings of P _{1} and P _{2} and \(\bar F_{1}\) is the mean of the trait of F _{1}. Suppose d _{ A }>a _{ A }>0, that is, P _{1} is overdominant to P _{2}. If inbreeding mating, the mean value is \(\left (\bar {P}_{1}+\bar {P}_{2}\right)/2=m\). However, for cross mating, the mean value is \(\bar {F}_{1}\) which is greater than m. Consider a natural population consisting of equal number of P _{1} and P _{2}. Then inbreeding reduces the mean value of fitness or fitnessrelated, namely this population exhibits inbreeding depression. On the other hand, Theorem 4 shows that a natural population with few overdominant genotypes does not exhibit severe inbreeding depression. The next result provides a reason to explain this difference.
Theorem 7
If a natural inbreeding population does not show a high level of inbreeding depression (i.e., δ≤0.2), then most fitness or fitness related loci (QTL) of it exhibits additive and dominantrecessive genotypes.
We conclude that inbreeding depression is determined by not only the inbreeding coefficient but also the genetical structure of the population. This result is consistent with the observation that, in many mixedmating plant populations, selfing (≈0.2), outcrossing (≈0.4) and the interm1diate (≈0.4) reaches at an equilibrium [29].
Let us consider an example. Goodwillie and Knight [2] measured inbreeding depression in three populations, namely Lake Hennessey (LH), Wantrup Researve (WR) and Ida Clayton Road (IC) of Leptosiphon, whose mean outcrossing rate are 0.06,0.37 and 0.69, respectively. Significant inbreeding depression was observed for the proportion of fertilized ovules that developed into seeds only occurred in the more outcrossing WR and IC populations. Theorem 7 predicts that most genes in population LH are additive and dominantrecessive and both LH and WR have many overdominant genes.
Another novel contribution of this work is the study for inbreeing depression with polygenic traits. Our theoretical results have shown that inbreeding depression also depends on the structure of the genotypes. The theoretical results support the belief that inbreeding depression is caused more like by overdominant mechanism rather than dominancerecessive mechanism. An application of these results is to explain why Caenorhabditis remanei has demonstrated to suffer severely from inbreeding depression, while its hermaphroditic relative C. elegans has not [22, 30]. In addition, Theorem 5 shows that inbreeding depression is also determined by the initial frequency, which, for cultivation of improved varieties with a favorable mutation, explains why inbreeding mating does not reduce the mean fitness of the first generation, but does reduce the mean fitness of the second generation. These results may have potential application for cultivation of improved varieties.
Conclusions
In this work we developed mathematical approaches to investigate the the variations of inbreeding population fitness under various conditions. For inbreeding population with a single locus, we proved that the mean fitness increases monotonically for directional selection, which extends the existing result for the random mating substantially [17]. Our results showed that, if most genotypes are additive, inbreeding mating (selfing) does not produce inbreeding depression. In addition, we defined the averaged mean fitness (AMF). Using this concept, we have successfully shown monotonical increase of the averaged mean fitness for any inbreeding population. These results can be applied to study the Mendelian trait or unlinked polygenic trait, and provide the reason to explain why inbreeding mating does not have serious impact on the mean fitness of population. For inbreeding depression with single trait, we have presented an explicit formula to calculate purging rate. Computing results suggest that inbreeding mating does purge deleterious mutants more efficiently than randomly mating does.
Appendix: Theoretic Proofs
Theorem 2
The averaged mean fitness (AMF) of any inbreeding population is positive, that is,
Proof
The proof for AMF>0 is equivalent to prove
since min{w(x)x∈[0,1]}>0. We first define
and let w _{11}=1−s,w _{12}=1 and w _{22}=1−t, where 0≤s,t≤1. Then we have that
□
We consider two cases. (1) Let w _{11}+w _{22}≥0.111 (equivalently, t+s≤1.889). In this case, note that (10+40f−12f ^{2}−3f ^{3})>0 and
hold for 0≤f≤1. Thus S(f)>0 hold.
(2) Let w _{11}+w _{22}≤0.111. Then we have that
and 0.889≤s,t≤1 holds.
In addition, let t≥s and write
Then P(f) is a decreasing function of t since
Hence we have, for f≤0.8,
Also note that P(f)_{ t=1} is a decreasing function of f and [P(f)_{ t=1}]_{ f=0.8}>0 for s≥0.89.
Thus we further have
For f≥0.8, we have
and note that the left hand side function of above equation is increasing for f≥0.8 and
if f≤0.85. However, for f≥0.85, we have
and
By repeating this procedure, we can prove S(f)>0 for f<1. Finally, if w _{11}≠w _{22}, it is easy to check that
Note that the unique equilibrium
and the mean fitness reaches its maximum value at
In addition, Δ w(x)<0 holds if and only if \(x\in (\bar x, x^{*})\) with
A similar argument can be applied to the case of underdominance populations. Thus the proof of Theorem 2 is completed.
Theorem 3
Assume that a polygenic trait is determined by the genotypes whose relative fitnesses are (1,1−hs,1−s), where the selective advantage coefficient s is fixed and the degree of the dominance h is uniformly distributed over [−1,1]. Then inbreeding mating reduces the mean fitness of the population for all initial frequency x satisfying
Proof
By the assumption, for each 0≤h≤1, there is an overdominant genotype (1,1+hs,1−s). It suffices to show
for any 0≤h≤1. In addition, we also have
Thus we further have
Hence it suffices to show either
or
Note that the left hand side of the last inequality is
since it is a decreasing function of s.
Let \(\bar y\) be the smaller root of (6−3f)y ^{2}−(8−3f)y+2. Then we have \(\bar y=\frac {2}{63f}\) and (6−3f)y ^{2}−(8−3fh ^{2})y+2 is a decreasing function for \(y\le \bar y\).
We further have that
for all \(y\le \frac {2}{63f}\). Thus
holds and hence w _{ s }≤w _{ o } holds for \(y\le \frac {2}{63f}\) or equivalently for \(x\ge \frac {43f}{63f}\). The proof is completed. □
Theorem 4
If the set of genotypes which determine a polygenic trait contains enough overdominant genotypes, then inbreeding mating reduces the mean fitness of the population for any given initial frequency x≠0, that is, w _{ s }≤w _{ o }.
Proof
It suffices to show lnw _{ s }− lnw _{ o }≤0. In fact, we first have
Note that if h _{1}≤h _{2}≤0, then
In fact, we have that
Thus we have that
Since (lnw _{ s }(0)− lnw _{ o }(0))<0 for x≠0,1 by Proposition 1(2), we have that
for enough large N. Thus lnw _{ s }− lnw _{ o }≤0 holds, that is, lnw _{ s }≤ lnw _{ o } holds. The proof is completed. □
References
 1
Fenster CB, Galloway IF. Inbreeding and ourbreeding depression in nature populations of Chamaecrista fasciculata(Fabaceae). Conserv Biol. 2000; 14:1406–12.
 2
Goodwillie A, Knight MC. Inbreeding depression mixed mating in Leptosiphon jepsonii: a Comparison of three populations. ANN BOTLONDON. 2006; 98:351–60.
 3
Li ZK, Luo LJ, Mei HW, Wang DL, Shu QY, Tabien R, Zhong DB, Ying CS, Stansel JW, Khush GS, Paterson AH. Overdominant epistatic loci are the primary genetic basis of inbreeding depression and heterosis in rice I. Biomass and grain yield. Genetics. 2001; 158:1737–53.
 4
Luo LJ, Li ZK, Mei HW, Shu QY, Tabien R, Zhong DB, Ying CS, Stansel JW, Khush GS, Patterson AH. Overdominant epistatic loci are the primary genetic basis of inbreeding depression and heterosis in rice. II. Grain yield components. Genetics. 2001; 158:1755–71.
 5
Wright S. Coefficients of inbreeding and relationship. AM NAT. 1922; 56:330–8.
 6
Barrett S, Charlesworth D. Effect of a change in the level of inbreeding on the genetic load. Nature. 1991; 352:522–4.
 7
Roff DA. Life History Evolution. Sunderland, MA: Sinauer Associates; 2002.
 8
Carr DE, Dudash MR. Inbreeding depression in two species of Mimulus (Scrophulariaceae) with contrasting mating systems. Am J Bot. 1996; 83:586–93.
 9
Johnston MO, Schoen DJ. Mutation rates and dominance levels of genes affecting fitness in two angiosperm species. Science. 1995; 267:226–9.
 10
Leimu R, Kloss L, Fischer M. Effects of experimental inbreeding on herbivore resistance and plant fitness: the role of history of inbreeding, herbivory and abiotic factors. Ecol Lett. 2008; 11:1101–10.
 11
Leimu R, Fischer M. Betweenpopulation outbreeding affects plant defence. PLoS ONE. 2010; 5:1–8.
 12
LopezCortegano E, Vilas A, Caballero A, GarciaDorada G. Estimation of genetic purging under competitive conditions. Evolution. 2016; 70:1586–870.
 13
GarciaDorado A. Understanding and predicting the fitness decline of shrunk population: inbreeding, purging, mutation, and the standard selection. Genetics. 2012; 190:1461–76.
 14
Sun S, Klebaner F, Tian T. A new mathematical model for inbreeding depression in large population. Lect Notes Bioinforma. 2014; 8492:321–32.
 15
Fisher RA. Average excess and average effect of a gene substitution. Ann Eugenics. 1941; 11:53–63.
 16
Wright S. The interpretation of population structure by FStatistics with special regard to systems of mating. Evolution. 1965; 19:395–420.
 17
Ewens WJ. Mathematical population genetics I, theoretical introduction. 2nd ed. Springer Science & Business Media: Springer: 2004.
 18
Ziehe M, Roberds JH. Inbreeding depression due to overdominance in partially selffertilizing plant populations. Genetics. 1989; 121:861–8.
 19
Charlesworth B, Charlesworth D. The genetic basis of inbreeding depression. Nat Rev Genet. 2009; 10:783–96.
 20
Hedrick PW. Purging inbreeding depression. Heredity. 1994; 73:363–72.
 21
Kubota S, Kameyama Y, Hirao AS, Ohara M. Adaptive significance of selftertiization in a hermaphroditic perennial, Trillium Camschatcense. Am J Bot. 2008; 95:482–9.
 22
Dolgin ES, Charlesworth B, Baird S, Cutter AD. Inbreeding and outbreeding depression in Caenorhabditis Nematodes. Evolution. 2007; 61:1339–52.
 23
Carr DE, Dudash MR. Recent approaches into the genetic basis of inbreeding depression in plants. PHILOS T R SOC B. 2003; 358:1071–84.
 24
Charlesworth D, Charlesworth B. Inbreeding depression and its evolutionary consequences. Annu Rev Ecol Evol S. 1987; 18:237–68.
 25
Dudash MR, Carr DE. Genetics underlying inbreeding depression in Mimulus with contrasting mating systems. Nature. 1998; 393:682–4.
 26
Li Y, Zhang X, Ma Ch, Shen J, Chen Q, Wang T, Fu T, Tu J. QTL and epistatic analyses of heterosis for seed yield and three yield component traits using molecular markers in rapeseed (Brassica napus L.)Russ J Genet. 2012; 48:1001–8.
 27
Dong DK, Cao JS, Shi K, Liu LC. Overdominance and epistasis are important for the genetic basis of heterosis in Brassica rapa. Hortscience. 2007; 42:1207–11.
 28
Fry JD. How common are overdominant mutations?Genetics. 2004; 167:1031–2.
 29
Willis JH. Inbreeding load, average dominance and the mutation rate for mildly Deleterious Alleles in Mimulus guttatus. Genetics. 1999; 153:1885–98.
 30
Bodmer WH, CavalliStorza LL. Genetics, evolution, and man. San francisco: W. H Freeman; 1976.
Acknowledgments
The authors thank the two anonymous reviewers for their insightful comments and constructive suggestions, which are very helpful to improve the manuscript substantially.
Funding
T.T. is supported by the Australian Research Council (ARC) Discovery Projects (DP120104460) which supports the publication cost of this paper. F.K. is supported by the Australian Research Council (ARC) Discovery Projects (DP120102728).
Availability of data and materials
Not Applicable.
Authors’ contributions
SS and TT conceived and conducted the research. SS carried out the theoretical proof. SS, FK and TT interpreted the results and wrote the paper. All authors edited and approved the final version of the manuscript.
Competing interests
The authors declare that they have no competing interests.
Consent for publication
Not applicable.
Ethics approval and consent to participate
Not applicable.
About this supplement
This article has been published as part of BMC Genomics Volume 18 Supplement 2, 2017. Selected articles from the 15th Asia Pacific Bioinformatics Conference (APBC 2017): genomics. The full contents of the supplement are available online http://bmcgenomics.biomedcentral.com/articles/supplements/volume18supplement2.
Author information
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Sun, S., Klebaner, F. & Tian, T. Mathematical modelling for variations of inbreeding populations fitness with single and polygenic traits. BMC Genomics 18, 196 (2017) doi:10.1186/s1286401734921
Published:
Keywords
 Deleterious Mutant
 Inbreeding Depression
 Initial Frequency
 Selfing Rate
 Polygenic Trait