BPLT⁺: A Bayesian-based personalized recommendation model for health care

Abstract

In this paper, we propose an Advanced Bayesian-based Personalized Laboratory Tests recommendation (BPLT⁺) model. Given a patient, we estimate whether a new laboratory test should belong to a "taken" or "not-taken" class. We use the bayesian method to build a weighting function for a laboratory test and the given patient. A higher weight represents that the laboratory test has a higher probability of being "taken" by the patient and lower probability of being "not-taken" by the patient. For the sake of effectiveness and robustness, we further integrate several modified smoothing techniques into the model. In order to evaluate BPLT⁺ model objectively, we propose a framework where the data set is randomly split into a training set, a validation input set and a validation label set. A training matrix is generated from the training data set. Then instead of accessing the training data set repeatedly, we utilize this training matrix to predict the laboratory test on the validation input set. Finally, the recommended ranking list is compared with the validation label set using our proposed metric CorrectRate _M . We conduct experiments on real medical data, and the experimental results show the effectiveness of the proposed BPLT⁺ model.

Background

Large amounts of clinic laboratory test data are collected and stored every day. Therefore, there is an increasing need for analyzing and utilizing the laboratory test data. The problem we are working on in this paper is to recommend laboratory tests for given patients. Health care recommendation problems have drawn researchers' attention for years. However, there are not a lot of studies conducted on the clinic laboratory test recommendation problem.

The medical data we are working on contains several years patients' laboratory test records. Figure 1 shows an example of the data format. Formally, the laboratory test prediction problem can be described as follows [1]: "Given a set of patients P = {p₁, p₂, ..., p _n } and a set of laboratory tests T = {test₁, test₂, ... test _M }, each patient p _j has done tests test_j,1, ..., test _j,kj . If a doctor would like to assign a new test for patient p _j , which test in T should be chosen?"

Figure 1

An example dataset. The format of the laboratory data sets is presented: the attributes from left to right are SDTE (SERVICE DATE), REQ# (REQUISITION NUMBER), PNUM (PATIENT HEALTH CARD#), PNAM (PATIENT NAME), PSEX (PATIENT SEX), BDTE (PATIENT DATE OF BIRTH), TSEQ (TEST SEQUENCE NUMBER), TEST (TEST CODE), DESC (TEST DESCRIPTION), RSLT (TEST RESULT), NORM (NORMAL RANGE), REXP (RESULT EXPECTED Y/N), EXRS (EXTENDED RESULT Y/N). The patient information in this table is fake due to privacy.

The computer systems have been playing for an important role in health care for years [2–8]. Statistic algorithms [9–12] lead an important role in investigating health care data. [13, 14] extracts chemical keywords from a query patent by analyzing word frequency and the word's effect over the data collection. Bayesian learning is a widely used algorithm that shows good performance [15–19]. A semantic-based association rule mining approach is proposed to model the medical query contexts in [20]. Using a novel classifier based on the Bayesian discriminant function, Raymer, M. L. [21] present a hybrid algorithm that employs feature selection and extraction to isolate salient features from large medical and other biological data sets. Martín and Pérez [22] analyze the robustness of the optimal action in a Bayesian decision making problem in the context of health care. [23, 24] studies the association between two words by simulating the impact of words in documents in the context of information retrieval. A probabilistic survival model is derived from the survival analysis theory for measuring aspect novelty of genomics data [25]. A mixture markov model is proposed to investigate user navigation patterns so that a personalized recommendation system for each user can be built [26]. In our previous work [1], we propose a laboratory test prediction model, which would objectively determine whether a laboratory test is associated to a patient. This paper is a significant extension to [1].

Smoothing [27] is a technique to create an approximating function that attempts to capture important patterns in the data, while leaving out noise or other fine-scale structures/rapid phenomena. The smoothing techniques have been used in many realms to improve the accuracy [28]. Based on the basic Bayesian algorithm and smoothing techniques, we propose an Advanced Bayesian-based Personalized Laboratory Tests recommendation (BPLT⁺) model, to investigate the correlation among laboratory tests for each patient. Evaluation is a crucial issue in the health care domain [29]. Some previous health care researchers do evaluation via patient interaction [30] or statistics [31]. We present a metric CorrectRate _X by employing the idea of Mean Average Precision (MAP) [32] in Information Retrieval domain.

Four unique contributions are presented in this paper. Firstly, we learn the associations among laboratory tests and make personalized recommendations to patients without human interaction. Secondly, we integrate modified smoothing technologies to improve the personalized recommendation model and propose the BPLT+ model. Thirdly, we propose a framework to randomly generate a training data set, a validation input set and a validation label set. Fourth, we use a objective evaluation metric for personalized recommendation systems without patient interaction.

Methods

Bayesian-Based personalized laboratory tests recommendation (BPLT) model

Here we assume that the laboratory tests for a patient have associations among each other. For instance, if a patient is suspected to have diabetes, usually the doctor will assign both Hemoglobin test and Glucose Fasting test for this patient. We can see that there exists an association between Hemoglobin and Glucose Fasting with respect to some hidden information, diabetes in this case. On the other hand, if a patient is assigned Hemoglobin test, then it is very likely that this patient should also take Glucose Fasting test. In this section, we build a model for learning the associations of the laboratory tests, inferring the associations between patients and laboratory tests, and therefore recommending new laboratory tests to the patients. We regard the test recommendation problem as a special classification problem, where a test belongs to either a "taken" or "not-taken" class. We use Bayesian classifier as our basic classifier, and modify it to a personalized ranking model.

Basic concept: Bayesian classifier

A classification problem is the following [33]: given a set of training instances, each described with a set of n attributes and each belonging to exactly one of a certain number of possible classes, learn to classify new, unseen objects. In addition, each attribute has a fixed number of possible values. We use naive Bayesian classifier as our basic classifier in this paper, since it evaluates directly the probability of taking a test and the conditional probability among two tests. Moreover, naive Bayesian is easy to construct and has surprisingly good performance in classification, even though the conditional independence assumption is rarely true in real-world applications [34]. The probability model for a classifier is a conditional model

$$\text{Pr}\left(C|F_1,...,F_n\right)$$

(1)

where F₁, ..., F _n are attributes, and C is a class variable. By Bayesian criteria, it equals to

$$\frac{\mathsf{\text{Pr}}\left(C\right)\text{Pr}\left(F_1,...,F_n|C\right)}{\text{Pr}\left(F_1,...,F_n\right)}$$

(2)

The denominator is effectively constant, and the numerator is equivalent to the joint probability model

$$\begin{array}{l}\mathsf{\text{Pr}}\left(C,F_{\mathsf{\text{1}}},...,F_n\right)\\ =\mathsf{\text{ Pr}}\left(C\right)Pr\left(F_{\mathsf{\text{1}}}|C\right)\mathsf{\text{ Pr}}\left(F_{\mathsf{\text{2}}}|C,F_{\mathsf{\text{1}}}\right)\mathsf{\text{ Pr}}\left(F_{\mathsf{\text{3}}}|C,F_{\mathsf{\text{1}}},F_{\mathsf{\text{2}}}\right)...\mathsf{\text{Pr}}\left(F_n|C,F_{\mathsf{\text{1}}},...,F_{n-\mathsf{\text{1}}}\right)\end{array}$$

In naive Bayesian, it assumes the features are conditional independent

$$\mathsf{\text{Pr}}\left(F_i|C,F_{j,0}\right)=\mathsf{\text{Pr}}\left(F_i|C\right),fori\ne j$$

Therefore, the probability of a class C given feature F₁, ..., F _n is

$$\text{Pr}\left(C|F_1,...,F_n\right)=A\text{Pr}\left(C\right){\displaystyle \prod _{i=1}^n}\left(F_i|C\right)$$

(3)

where $A=\frac{1}{\text{Pr}\left(F_1,...,F_n\right)}$ is a constant.

The weighting function of BPLT model

In this Section, we describe the Bayesian-based Personalized Laboratory Tests recommendation (BPLT) model, which was proposed in our previous work [1]. More details are given in this paper. The purpose of BPLT model is to classify the laboratory tests for individual patients by their personal conditions. In the real world, it is often easier to obtain the patients' previous laboratory tests information. Therefore, the BPLT model recommends additional new laboratory tests to patients, given the previous laboratory tests that the patients have taken.

Suppose we have a set of M laboratory tests T = {test₁, test₂, ..., test _M }, and a patient p _j who has taken tests T _j = {test _j ,₁, ..., test _j,kj } where test _j,i ∈ T for all 1 ≤ i ≤ k _j . We denote the events that tests in T _j are taken by p _j as F _j ,₁, F _j ,₂, ...F _j,M . For example, if we have 7 tests in T, and p _j has taken test₃, test₅ and test₇ could be represented as (F _j ,₁, F _j ,₂, ..., F _j ,₇) = (0, 0, 1, 0, 1, 0, 1). Bayesian Classifier is employed to evaluate the association between p _j a new test test₀ where test₀ ∈ T and test₀ ∉ T _j . We use F_j,0 to represent the event of p _j should take t₀, and $F_{j,0}^c$ to represent the event of p _j should not take t₀. By Formula (3), the probability of F _j ,₀ given F _j ,₁, F _j,2 , ...F _j,M is

$$\text{Pr}\left(F_{j,0}|F_{j,1},F_{j,2},\dots F_{j,M}\right)\propto \text{Pr}\left(F_{j,0}\right){\displaystyle \prod _{i=1}^M}\text{Pr}\left(F_{j,i}|F_{j,0}\right)$$

The probability of $F_{j,0}^c$ given F _j ,₁, F _j ,₂, ... F _j,M is

$$\text{Pr}\left(F_{j,0}^c|F_{j,1},F_{j,2},\dots F_{j,M}\right)\propto \text{Pr}\left(F_{j,0}^c\right){\displaystyle \prod _{i=1}^M}\text{Pr}\left(F_{j,i}|F_{j,0}^c\right)$$

In the BPLT model, we reward the tests with high probability of "taken" and low probability of "not-taken". The correlation between a new test test₀ and a given patient p _j is shown in Definition 1 [1].

Definition 1 The correlation between a new test test₀ and a given patient p _j is defined as the log function of the probability of p _j should take test ₀ divided by the probability of p _j should not take test ₀ given F_j,1 , F _j ,₂, ... F _j,M .

$$corr\left(tes{t}_0,p_j\right)=\text{log}\frac{\text{Pr}\left(F_{j,0}|F_{j,1},F_{j,2},\dots F_{j,M}\right)}{\text{Pr}\left(F_{j,0}^c|F_{j,1},F_{j,2},\dots F_{j,M}\right)}$$

(4)

We can see that higher value of corr(test₀, p _j ) indicates that test₀ has more association with p _j . The calculation of corr(test₀, p _j ) can be further simplified as follows

$$\begin{array}{l}corr\left(tes{t}_0,p_j\right)\\ =\text{log}\text{Pr}\left(F_{j,0}|F_{j,1},F_{j,2},\dots F_{j,M}\right)-\text{log}\text{Pr}\left(F_{j,0}^c|F_{j,1},F_{j,2},\dots F_{j,M}\right)\\ =\text{log}\text{Pr}\left(F_{j,0}\right){\displaystyle \prod _{i=1}^M}\text{Pr}\left(F_{j,i}|F_{j,0}\right)-\text{log}\text{Pr}\left(F_{j,0}^c\right){\displaystyle \prod _{i=1}^M}\text{Pr}\left(F_{j,i}|F_{j,0}^c\right)\\ =\text{log}\frac{\text{Pr}\left(F_{j,0}\right)}{\text{Pr}\left(F_{j,0}^c\right)}+{\displaystyle \sum _{i=1}^M}\frac{\text{log}\text{Pr}\left(F_{j,i}|F_{j,0}\right)}{\text{Pr}\left(F_{j,i}|F_{j,0}^c\right)}\end{array}$$

(5)

Moreover, a test either belongs to a "taken" class or a 'not taken" class. Thus, the following two formulas are held.

$$\begin{array}{c}\text{Pr}\left(F_{j,0}\right)+\text{Pr}\left(F_{j,0}^c\right)=1\\ \text{Pr}\left(F_{j,i}|F_{j,0}\right)\text{Pr}\left(F_{j,0}\right)+\text{Pr}\left(F_{j,i}|F_{j,0}^c\right)\text{Pr}\left(F_{j,0}^c\right)=\text{Pr}\left(F_{j,i}\right)\end{array}$$

from which we can obtain $\text{Pr}\left(F_{j,0}^c\right)$ and $\text{Pr}\left(F_{j,i}|F_{j,0}^c\right)$

$$\begin{array}{c}\text{Pr}\left(F_{{}_{j,0}}^c\right)=1-\text{Pr}\left(F_{j,0}\right)\\ \text{Pr}\left(F_{j,i}|F_{j,0}^c\right)=\frac{\text{Pr}\left(F_{j,i}\right)-\text{Pr}\left(F_{j,i}|F_{j,0}\right)\text{Pr}\left(F_{j,0}\right)}{1-\text{Pr}\left(F_{j,0}\right)}\end{array}$$

Thus $\text{Pr}\left(F_{j,0}^c\right)$ and $\text{Pr}\left(F_{j,i}|F_{j,0}^c\right)$ in (5) can be eliminated in corr (test₀, p _j ), as shown below

$$\text{log}\frac{\text{Pr}\left(F_{j,0}\right)}{1-\text{Pr}\left(F_{j,0}\right)}+{\displaystyle \sum _{i=1}^M}\text{log}\frac{\text{Pr}\left(F_{j,i}|F_{j,0}\right)\left(1-\text{Pr}\left(F_{j,0}\right)\right)}{\text{Pr}\left(F_{j,i}\right)-\text{Pr}\left(F_{j,i}|F_{j,0}\right)-\text{Pr}\left(F_{j,0}\right)}$$

A joint probability for patient p _j take both of the tests test _i and test₀ is

$$\text{Pr}\left(F_{j,i},F_{j,0}\right)=\text{Pr}\left(F_{j,i}|F_{j,0}\right)\text{Pr}\left(F_{j,0}\right)$$

The definition of the correlation between test₀ and p _j is

$$\begin{array}{l}corr\left(tes{t}_0,p_j\right)\\ =\text{log}\frac{\text{Pr}\left(F_{j,0}\right)}{1-\text{Pr}\left(F_{j,0}\right)}+{\displaystyle \sum _{i=1}^M}\text{log}\frac{\text{Pr}\left(F_{j,i},F_{j,0}\right)\left(1-\text{Pr}\left(F_{j,0}\right)\right)}{\text{Pr}\left(F_{j,0}\right)\left(\text{Pr}\left(F_{j,i}\right)-\text{Pr}\left(F_{j,i},F_{j,0}\right)\right)}\\ =\left(k-1\right)\cdot \text{log}\frac{1-\text{Pr}\left(F_{j,0}\right)}{\text{Pr}\left(F_{j,0}\right)}+{\displaystyle \sum _{i=1}^M}\text{log}\frac{\text{Pr}\left(F_{j,i}|F_{j,0}\right)}{\text{Pr}\left(F_{j,i}\right)-\text{Pr}\left(F_{j,i}|F_{j,0}\right)}\end{array}$$

which leads to the following Definition 2 [1].

Definition 2 The weighting function for a laboratory test test ₀ for a patient p _j is the simplified correlation between test ₀ and p _j

$$w\left(tes{t}_0,p_j\right)=\left(k-1\right)\cdot \text{log}\frac{1-\alpha }{\alpha }+{\displaystyle \sum _{i=1}^M}\text{log}\frac{{\beta }_{j,i}}{{\gamma }_{j,i}-{\beta }_{j,i}}$$

(6)

where

$$\begin{array}{c}\alpha =\text{Pr}\left(F_{j,0}\right)=\frac{numberofpatientstakentes{t}_0}{numberofpatients}\\ {\gamma }_{j,i}=\text{Pr}\left(F_{j,i}\right)=\frac{numberofpatientsthat{F}_{j,i}holds}{numberofpatients}\\ {\beta }_{j,i}=\text{Pr}\left(F_{j,i}|F_{j,0}\right)=\frac{\text{Pr}\left(F_{j,i},F_{j,0}\right)}{\text{Pr}\left(F_{j,0}\right)}\\ =\frac{1}{\alpha }\frac{numberofpatientsthatboth{F}_{j,0}and{F}_{j,i}holds}{numberofpatients}\end{array}$$

The new laboratory tests will be ranked in a list according to w(test₀, p _j ) for a given patient p _j . In the later section, we will present the evaluation environments for the laboratory test ranking list.

An advanced model: BPLT⁺

To have a more robust and better performance model, we further propose an advanced model, BPLT⁺, by improving the BPLT model using several smoothing techniques. There are two reasons for smoothing BPLT. One reason is that smoothing is a way to deal with noise within the data. Another reason is to avoid the mathematically meaningless. When test⁰ laboratory test has not been observed in the previous visits, which means α = 0, the first part of formula (6) will become an irrational number. Meanwhile, when the joint frequency of two laboratory tests is zero, which means β _j , _i = 0, the second part of (6) will become an irrational number. Therefore, we introduce smoothing technologies to further improve BPLT model.

Smoothing techniques

In statistics, smoothing [27] is a technique to create an approximating function that attempts to capture important patterns in the data, while leaving out noise or other fine-scale structures/rapid phenomena. The main purpose of smoothing in this paper is to assign a non-zero probability to the unseen tests and improve the accuracy of test probability estimation in general.

The smoothing techniques are discussed based on the following definitions of a conditional probability [28].

$$\text{Pr}\left(t|p\right)=\frac{c\left(t;p\right)}{{\sum }_{t\in T}c\left(t;p\right)}$$

(7)

where c(t;p) is the count of a patient taking a test. Here are some commonly used smoothing methods. Since we have defined a ranking problem, which is similar to the problems in Information Retrieval (IR), we use some widely used smoothing methods in language model in IR. The general form of a smoothed model [35] is assumed to be the following:

$$\text{Pr}\left(t|p\right)=\left\{\begin{array}{cc}\underset{t}{\text{Pr}}\mathsf{\text{(}}t|p\mathsf{\text{)}}\hfill & iftesttisobserved\hfill \\ \text{Pr}\left(t|C\right)\hfill & otherwise\hfill \end{array}\right.$$

(8)

where Pr _t (t|p) is the smoothed probability of a test t given the patient with existing tests. Pr(t|C) is the probability of a test t given the whole data set.

A smoothing method may be as simple as adding an extra count to every test, which is called additive or Laplace smoothing, or more sophisticated as in Katz smoothing, where tests of different count are treated differently. Three representative methods that are popular and effective are:

• The Jelinek-Mercer method

  $$\underset{\lambda }{\text{Pr}}\left(t|p\right)=\left(1-\lambda \right)\text{Pr}\left(t|p\right)+\lambda \text{Pr}\left(t|C\right)$$

  (9)

where λ is a balancing parameter ranges from 0 to 1.

• Bayesian Smoothing using Dirichlet Priors

  $$\underset{{\mu }_0}{\text{Pr}}\left(t|p\right)=\frac{c\left(t;p\right)+{\mu }_0\text{Pr}\left(t|C\right)}{{\sum }_{t\in T}c\left(t;p\right)+{\mu }_0}$$

  (10)

where µ₀ is a balancing parameter, and µ₀ >0. The Laplace method is a special case of this technique.

• Absolute Discounting

  $$\underset{\delta }{\text{Pr}}\left(t|p\right)=\frac{max\left(c\left(t;p\right)-\delta ,0\right)}{{\sum }_{t\in T}c\left(t;p\right)}+\sigma p\left(t|C\right)$$

  (11)

where δ ∈ [0, 1] is a discount constant and σ = δ|p| _u /|p|, so that all probabilities sum to one. Here |p| _u is the number of unique terms in document d, and |p| is the total count of words in the documents.

BPLT⁺with smoothing techniques

There are two parts in formula (6) that need smoothing. The first one is the conditional probability β _j , _i = Pr(F _j,i |F_j,0). Its smoothed format is as follows:

• BPLT⁺ with Jelinek-Mercer

  $${\beta }_{j,i}^{\lambda }=\left(1-\lambda \right){\beta }_{j,i}+\lambda {\gamma }_{j,i}$$

  (12)

• BPLT⁺ with dirichlet priors

  $${\beta }_{j,i}^{\mu }=\frac{{\beta }_{j,i}+\mu {\gamma }_{j,i}}{1+\mu }$$

  (13)

• BPLT⁺ with absolute discounting

  $${\beta }_{j,i}^{\delta }=\frac{max\left(c\left(t;p\right)-\delta ,0\right)}{{\sum }_{t\in T}c\left(t;p\right)}+\delta {\gamma }_{j,i}$$

  (14)

In Jelinek-Mercer BPLT⁺ and Absolute Discounting BPLT⁺, we use the existing smoothing method. The smoothing parameters λ, δ are within the range of [0, 1]. In Dirichlet Priors BPLT⁺, we modify the Dirichlet smoothing technique, by divide both the numerator and the denominator in (10) by ${\sum }_{t\in T}c\left(t;p\right)$, and normalize the parameter µ to the range of 0[1], where $\mu =\frac{{\mu }_0}{{\sum }_{t\in T}c\left(t;p\right)}$.

Another part in formula (6) needs smoothing is $\text{log}\frac{\alpha }{1-\alpha }$, which is a simple division that could be smoothed

via Laplace smoothing as

$$\text{log}\left(\frac{\alpha +\theta }{1-\alpha +\theta }\right)$$

(15)

where θ is a tuning parameter ranges from 0 to 1.

Evaluation environments

Datasets

The datasets in our experiment are obtained from Alpha Global IT [1, 36]. Alpha Corporate Group provides laboratory, medical clinic, commercial electronic medical record and practice management software. The data set contains 78 monthly patient's laboratory test results. Our experiments use 6 month results, containing 1,048,575 patients' records, as a key study. Thousands of patients' records and more than 400 laboratory tests are included in our experiments. The data format is the same as the example shown in Figure 1. Our data set contains real patients' information, such as health card ID, age, gender, date of visit, laboratory test ID, laboratory test results. We only use the patient ID and laboratory ID attributes in this paper, and analyze the associations among these laboratory tests. In our future work, we will incorporate more attributes in the laboratory recommendation model.

Validation data and measure

To evaluate BPLT⁺ models objectively, we divide the data set into three components: a training set, a validation input set, and a validation label set. The data set is firstly randomly split into a training set and a validation set. In this step, we split based on the patients and do not split the records from a same patient. Then for the validation set, we randomly remove one test t^* from each patient p _j , and store the t^* in the validation label set. The ranked list returned by BPLT⁺ will be compared with t^* for each patient. To measure such comparison and finally evaluate the effectiveness of BPLT⁺, we use the following defined CorrectRate _X [1]. Suppose the returned laboratory ranking list is $L=t_{1,j}^{\prime },\dots t_{l,j}^{\prime }$, CorrectRate _X validates whether t^* appears in the top ranked tests. The measure is modified from Mean Average Precision (MAP) [32] evaluation metric.

Definition 3 The CorrectRate _X evaluates the accuracy of a laboratory tests prediction system. It is the number of patients with the desired (golden standard) test matching one of the top X tests generated by the system, divided by the total number of the patients.

$$CorrectRat{e}_X=\frac{{\sum }_{j=1}^{n}TO{P}_{j,X}}{n}$$

(16)

where

$$TO{P}_{j,X}=\left\{\begin{array}{c}1if{t}^{*}matchesatestin\left\{{t^{\prime }}_{1,j},\dots {t^{\prime }}_{X,j}\right\}\hfill \\ 0otherwise\hfill \end{array}\right.$$

n is the number of patients, X is a parameter indicating how many top tests are compared to the golden standard test t*, which is set to be 1 or 3 in this paper.

We present an example to show how the CorrectRate _X evaluates the model in Table 1. Suppose the laboratory test sets includes 200 tests and there are 5 patients in the validation set. As we have introduced, the BPLT⁺ model returns a ranked list for each patient. Here ">" represents that the weight of the left-side laboratory test is higher than the weight of the right-side laboratory test. In our example, 2 out of 5 patients have the desired test t^* ranked in the top 1 position of the list, then CorrectRate₁ equals 0.4. And 4 out of 5 patients have t^* appears within the top 3 positions of the returned ranking list, then CorrectRate₃ equals 0.8. We can see that the top 3 positions include the top 1 position, so the following statement is always true: CorrectRate₁ ≤ CorrectRate₃.

Table 1 An example of CorrectRate _X

BPLT⁺System Framework

The framework of BPLT⁺ Model is shown in Figure 2. The data set in this framework is abstracted to contain only patient ID and laboratory test ID. The procedures in the proposed framework are described as follows.

Figure 2

BPLT⁺ System Framework. The procedures for processing the laboratory data and testing the BPLT⁺ model are shown: (1) the rectangles represent the data sets; (2) the rounded rectangles present the implemented procedures; (3) the ovals show the personalized laboratory model; (4) the lines with arrows determine the directions through the framework.

• Split: First the data set is randomly split into a training set and a validation set.

Random Remove a test as label: Since it is hard to objectively evaluate the performance of the BPLT⁺ model, we further randomly remove a test for each visit of the patients from the validation set. These removed tests are regarded as labels of the validation set input. Our ultimate goal is to recommend the missing test for a patient's visit.

• Build training matrix: To avoid duplicate calculating the frequency of a test and the joint frequency between two tests, we build a training matrix out of the training data. This training matrix contains the frequency of co-occurrences of two laboratory tests. For example, if a patient in the training data did test₁ and test₂ together, then add 1 to F₁₂ and F₂₁. We can see that the training matrix is a symmetric matrix.

• BPLT⁺model: The correlation of a given test₀ and a patient is calculated based on formula (6).

• Evaluation via CorrectRate _X : Finally, the evaluation criteria CorrectRate _X evaluates if the model made the correct recommendations.

Results

We first show the overall performance under different training-validation proportion in Table 2[1]. We randomly take 40%, 50% and 60% of the data out of the raw data set as the training data and keep the rest as the validation data. In general, there is higher performance of BPLT⁺ model on a larger training data set. This is because the larger training data set contains more information, and more knowledge can be learned. With the development of computer technology, larger amount of medical data will be available in practice. Therefore, we will use 60% of data as training data in the rest of this paper. As we have discussed before, CorrectRate₃ is always higher than CorrectRate₁. In general, the BPLT⁺ model has promising performance with an accuracy of 0.7074 for CorrectRate₁ and an accuracy of 0.7840 for CorrectRate₃.

Table 2 Performance

Then we investigate how the smoothing parameters affect the effectiveness in detail. We first consider smoothing β _j,i only. There are three smoothing technologies utilized to smooth β _j,i . They are Jelinek-Mercer BPLT⁺, Dirichlet Priors BPLT⁺ and Absolute Discounting BPLT⁺, with the corresponding parameters: λ, µ, δ ∈ [0, 1]. We conduct experiments on these three methods individually. The change of CorrectRate₁ and CorrectRate₃ with respect to the parameters are shown in Figure 3, Figure 4, and Figure 5. We can see from the figures that the curve of CorrectRate₁ is always below the curve of CorrectRate₃, which is consistent as we have discussed Definition 3. With the increasing of parameters from 0.1 to 1, both CorrectRate₁ and CorrectRate₃ become higher at the beginning due to the incorporating of the smoothing portion. After reaching the maximum value, CorrectRate₁ and CorrectRate₃ become lower, since the weighing would tend to be more universal when too much smoothing is incorporated. All the smoothing parameters achieve their best performance at the value of 0.2. Comparing among these three methods, Jelinek-Mercer BPLT⁺ obtains the best performance on both CorrectRate₁ and CorrectRate₃, which are 0.5569 and 0.6167. When it comes to the average value, Dirichlet Priors BPLT⁺'s average performance on CorrectRate₃ is better than the other two, and Jelinek-Mercer BPLT⁺'s average performance on CorrectRate₁ is the best.

Figure 3

Parameter Sensitivity of λ in Jelinek-Mercer BPLT⁺. The influence of parameter λ is investigated: (1) the stars represent the performance of Jelinek-Mercer BPLT⁺ under the evaluation metric CorrectRate₁; (2) the circles represent the performance of Jelinek-Mercer BPLT⁺ under the evaluation metric CorrectRate₃; (3) CorrectRate₃ is always higher than CorrectRate₁; (4) Jelinek-Mercer BPLT⁺ achieves its best performance when λ = 0.2.

Figure 4

Parameter Sensitivity of µ in Dirichlet Priors BPLT⁺. The influence of parameter µ is studied: (1) the stars represent the performance of Dirichlet Priors BPLT⁺ under the evaluation metric CorrectRate₁; (2) the circles represent the performance of Dirichlet Priors BPLT⁺ under the evaluation metric CorrectRate₃; (3) Dirichlet Priors BPLT⁺ achieves its best performance when µ = 0.2.

Figure 5

Parameter Sensitivity of δ in Absolute Discounting BPLT⁺. The influence of parameter δ is investigated: (1) the stars represent the performance of Absolute Discounting BPLT⁺ under the evaluation metric CorrectRate₁; (2) the circles represent the performance of Absolute Discounting BPLT⁺ under the evaluation metric CorrectRate₃; (3) Absolute Discounting BPLT⁺ achieves its best performance when δ = 0.2.

We further discuss to smooth the second part of (6), where the Laplace smoothing parameter is θ. As we have discussed before, Jelinek-Mercer BPLT⁺ has the best performance on both CorrectRate₁ and CorrectRate₃. We focus on investigating the sensitivity of θ by fixing Jelinek-Mercer BPLT⁺ with λ = 0.2. The results are shown in Figure 6. We can see that the CorrectRate₁ increases while θ is increasing, and the CorrectRate₃ decreases a little and then increases. Both of them reach the maximum and tend to be stable when θ is greater than 0.5.

Figure 6

Parameter Sensitivity of θ. The influence of parameter θ is presented: (1) we use the best smoothing technique for the first part in Formula 6, which is Jelinek-Mercer BPLT⁺; (2) the smoothing parameter λ is set to be optimal; (2) the stars represent the results of Jelinek-Mercer BPLT⁺ under evaluation metric CorrectRate₁; (3) the circles represent the results of Jelinek-Mercer BPLT⁺ under evaluation metric CorrectRate₃; (4) both metrics reach the maximum and tend to be stable when θ is greater than 0.5.

Conclusions and future work

An Advanced Bayesian based Personalized Laboratory Tests recommendation (BPLT⁺) model is proposed in this paper. Based on the assumption that hidden association could exist among laboratory tests, we employ a Bayesian approach to build a weighting function for scoring the correlation between a new laboratory test and a patient. To have a more robust and better performance model, we employ several enhanced smoothing technologies into the BPLT⁺ model. The main purpose of smoothing in this paper is to assign a non-zero probability to the unseen laboratory tests and improve the accuracy of test probability estimation. We integrate existing smoothing techniques in the BPLT⁺ model. In particular, we use three techniques, Jelinek-Mercer, Dirichlet Priors and Absolute Discounting approaches, to smooth the conditional probability of observing a patient taking an existing test when a new test test₀ is given (Formula 12-14). Also we use Laplace method to smooth the log function in the BPLT⁺ model (Formula 15). We conducted experiments to discuss the performance of the BPLT⁺ model and the sensitivity of smoothing parameters. We find that BPLT⁺ is able to make accurate recommendations under proper smoothing parameters.

Further, we propose a novel framework for effectively implementing BPLT⁺ model and objectively testing personalized recommendation systems without human interactions, shown in Figure 2. Based on the real patients' laboratory test data, we randomly generate a training data set, a validation input set and a validation label set. A training matrix containing the laboratory test statistics is calculated from the training data set and stored. For new patients (the validation input set), instead of processing the original training set, we utilize this training matrix to predict the laboratory test on the validation input set, and compare the ranking results with the validation label set.

There are a few future directions of this research work. As we can see from the data format in Figure 1, we have not make use of all the attributes. In the future, we would like to conduct a comprehensive investigation for the patients' profiles. For example, we can cluster the patients into groups and investigate the similarities of the patients in the same group. We can also analyze the associations among laboratory test results and therefore further enhance our proposed personalized recommendation model. Moreover, we look forward to testing our proposed models in more real applications.