- Research
- Open Access

# Modelling survival data to account for model uncertainty: a single model or model averaging?

- Sri Astuti Thamrin
^{1, 2}Email author, - James M. McGree
^{1}Email author and - Kerrie L. Mengersen
^{1}

**2**:665

https://doi.org/10.1186/2193-1801-2-665

© Thamrin et al.; licensee Springer. 2013

**Received:**2 July 2013**Accepted:**18 November 2013**Published:**11 December 2013

## Abstract

This study considered the problem of predicting survival, based on three alternative models: a single Weibull, a mixture of Weibulls and a cure model. Instead of the common procedure of choosing a single “best” model, where “best” is defined in terms of goodness of fit to the data, a Bayesian model averaging (BMA) approach was adopted to account for model uncertainty. This was illustrated using a case study in which the aim was the description of lymphoma cancer survival with covariates given by phenotypes and gene expression. The results of this study indicate that if the sample size is sufficiently large, one of the three models emerge as having highest probability given the data, as indicated by the goodness of fit measure; the Bayesian information criterion (BIC). However, when the sample size was reduced, no single model was revealed as “best”, suggesting that a BMA approach would be appropriate. Although a BMA approach can compromise on goodness of fit to the data (when compared to the true model), it can provide robust predictions and facilitate more detailed investigation of the relationships between gene expression and patient survival.

## Keywords

- Bayesian modelling
- Bayesian model averaging
- Cure model
- Markov Chain Monte Carlo
- Mixture model
- Survival analysis
- Weibull distribution

## Introduction

Modelling survival data plays an important role in the application of statistics in medicine and health science. In addition to a nonparametric formulation, there are many parametric models available for describing survival, including models based on a single distribution such as the Exponential and Weibull, mixture models based for example on mixtures of distributions and a mixture of susceptible and insusceptible individuals or so-called cure models which account for a fraction of the patients being cured from the disease. Given the wealth of models, the dilemma that is faced by many practitioners is the choice of a survival model.

The problem of model selection is abundant throughout the literature. This includes both covariate selection and choice of the model itself. Some of the methods are based on a series of significance tests while others fit more comprehensive models; some include prior information; some use analytic or approximate methods of estimation while others use Markov Chain Monte Carlo (MCMC) methods; different approaches use different optimisation or model comparison criteria such as Bayes factors (Raftery 1996). For example, McGrory and Titterington (2007) showed how variational techniques can be used to extend the deviance information criterion (DIC) to include the comparison of mixture models, while Basu and Tiwari (2010) used Bayes factors to compare the various model structures in breast cancer survival data.

Recently, Bonato et al. (2011) proposed Bayesian ensemble methods to obtain better survival prediction in high-dimensional gene expression data. Regardless of the method, the most common approach is to choose a single model based on the adapted optimisation or model choice criterion. However, if a single model is selected, then inferences are conditional on the selected model, and model uncertainty is ignored which often leads to excessively narrow or misleading inferences (Hjort and Claeskens 2003; Raftery et al. 1997). This difficulty can be overcome by combining the information provided by all suitable models into the analysis. The most common way of achieving this is to use a form of model averaging. From a Bayesian point of view, this averaging is applied such that the posterior distribution of the quantity of interest is obtained over the set of suitable models, weighted by the respective posterior model probabilities (Raftery 1996).

Draper (1995) and Raftery (1995) reviewed Bayesian model averaging (BMA) and the cost of ignoring model uncertainty. Madigan and Raftery (1994) also considered BMA by using Occam’s razor and Occam’s window approaches to reduce the number of candidate models. Yuan and Yin (2011) used model averaging procedures to make more robust inferences regarding the dose-finding design for phase I clinical trials. Pramana et al. (2012) focused on the case in which several parametric models are fitted to gene expression data and discussed model averaging techniques for the estimation of dose-response models.

In this paper, we consider the problem of predicting survival, based on three alternatives models; a single Weibull, a mixture of Weibulls and a cure model. The Weibull distribution is a popular parametric distribution for describing survival times (Dodson 1994). Given the variety of shapes that can be described by the probability density function (pdf) and the simple representation of the survival function, the Weibull distribution has been used very effectively for analysing lifetime data, particularly when the data are censored, which is very common in most life testing experiments (Collet 1994; Kundu 2008).

Given the nature of microarray data to describe biological systems and outcomes of patients, and the potential of these covariates to produce more precise inferences about survival, the use of a single parametric distribution to describe survival time may not be adequate. Microarray data may enable the description of several homogeneous subgroups of patients with respect to survival time. This paper therefore also considered a mixture of Weibull models for precise estimation and prediction of survival. Mixture models can be used to describe a population consisting of several disjoint groups, where each group is assigned its own distribution, weighted by the probability of an individual from the overall population belonging to that group. This model thus provides a convenient and flexible mechanism for identification and estimation of distributions which are not well modelled by any standard parametric family (Stephens 1997). In the study considered here, the mixture is assumed to comprise a known number of Weibull distributions, with potentially different parameters. Most approaches to the analysis of time to event data implicitly assume all individuals will experience the event of interest. However, there are situations when a proportion of individuals are not expected to experience the event of interest; that is, those individuals are often referred to as immune, cured or nonsusceptible (Ibrahim et al. 2001). To address this issue, cure rate models are considered, which are survival models incorporating a cure fraction. These models, which can be considered as a form of mixture model with one component degenerating to a point mass, extend the understanding of time to event data by allowing the formulation of more accurate and informative conclusions about the two groups of subjects.

Finally, instead of adopting the usual practice of choosing a single “best” model, where “best” is defined in terms of the probability of the model given the data, a BMA approach was adopted to account for model uncertainty in the prediction of the response. We illustrate the approach using a microarray dataset.

The paper is organised as follows. In Section “Methods”, we define BMA. The three competing models are described in a Bayesian framework in Section “Models”. The computational approach for estimation is also presented in this section. In the Section “Application to gene expression data”, we illustrate the model using a case study. The results are discussed further in Section “Discussion”.

## Methods

*S*models being considered, for

*s*= 1,2,…,

*S*, each with parameter set

*θ*

_{ s }based on data

*D*. Let

*Δ*be the quantity of interest; this could represent, for example, the posterior predictive distribution of

*y*. Hence, the posterior distribution of

*Δ*given data

*D*(Hoeting et al. 1999) is

*p*(

*S*=

*s*∣

*D*) is the posterior probability of a particular model being true, defined as

where $p(D\mid S=s)=\int p(D\mid {\theta}_{s},S=s)p({\theta}_{s}\mid S=s)\mathrm{d}{\theta}_{s}$.

Here, *p*(*D* ∣ *S* = *s*) is the marginal likelihood of the data *D* given model *S* = *s* and *p*(*θ*
_{
s
} ∣ *S* = *s*) is the prior density of *θ*
_{
s
} given model *S* = *s*. *p*(*S* = *s*) is the prior probability that model *s* is the true model (Hoeting et al. 1999).

Given a model selection problem in which we have to choose between two models, the plausibility of the two different models *S*
_{1} and *S*
_{2} is assessed by the Bayes factor as the ratio of posterior model probabilities.

The main detractor from using Bayes factors is that they are, in general, difficult to compute. Raftery (1995) proposed using the Bayesian information criterion (BIC) (Schwarz 1978) as an approximation. Buckland et al. (1997) and Claeskens and Hjort (2008) discussed the utilization of BIC in BMA. Buckland et al. (1997) proposed simpler methods in which weights are based upon the penalised likelihood functions formed from the AIC (Akaike 1973).

*f*represents the density function of the true and unknown model,

*q*represents the density function of the model that is used to approximate

*f*, and

*θ*

_{ s }is a vector of the unknown parameters to be estimated. The notation

*I*(

*f*∣

*q*) denotes the information lost when

*q*is used to approximate

*f*or the distance from

*q*to

*f*. For a given set of models, one can compare the KL information for each model and select the model that minimises the information loss across the considered set of models (Burnham and Anderson 2002;2004). However, in practice

*I*(

*f*∣

*q*) cannot be computed since the true model

*f*is unknown. Schwarz (1978) and Burnham and Anderson (2002) made the link between the KL information and likelihood theory, and showed that the expected KL information can be expressed as

*p*(

*D*∣

*θ*

_{ s }) is the likelihood,

*d*

_{ s }is the number of parameters in the model and

*n*is the number of uncensored observations in a survival context (Volinsky and Raftery 2000). A Laplace approximation, typically the BIC (Schwarz 1978), can be used to approximate

*p*(

*D*∣

*S*=

*s*) (Clyde 2000; Hoeting et al. 1999; Jackson et al. 2009; Yuan and Yin 2011):

Here $\text{log}p(D\mid {\widehat{\theta}}_{s},S=s)$ is the maximised log-likelihood of model *s*, which estimates goodness of fit of the data.

*s*

^{ th }model (Jackson et al. 2009; Yuan and Yin 2011) is therefore given by

The BMA weight can be interpreted as the weight of the evidence that model *s* is true model given a set of *S* models. For the case in which there is no information about prior probabilities, we can let *p*(*S* = *s*) be equal for all candidate models (1/*S*), indicating no prior preference for any of the models (Jackson et al. 2009; Pramana et al. 2012). The model with the largest BMA weight will be considered as the best model. Therefore, *p*(*S* = *s* ∣ *D*) is also an approximation to the posterior probability of the model *s* being correct (Schwarz 1978). A smaller BIC value indicates a better model fit, accounting for model complexity.

*j*

^{ th }simulated observation from the

*s*

^{ th }model. Then the mean of

*f*from the BMA model, (${\stackrel{\u0304}{\phantom{\rule{0.3em}{0ex}}f}}_{\mathit{\text{MA}}}$), can be calculated as follows

where *N* is the number of simulated observations and *w*
_{
s
} = *p*(*S* = *s* ∣ *D*) is the BMA weight, defined previously.

## Models

### Weibull model

In this section, we define the Weibull model for analysing survival of patients in the context of human health. We confine ourselves to survival times that are the difference between a nominated start time and a declared failure (uncensored data) or a nominated end time (censored time). Let *T* be a nonnegative random variable for a person’s survival time and *t* be a realisation of the random variable *T*. Kleinbaum and Klein (2005) give some reasons for the occurrence of right censoring in survival studies, including termination of the study, drop outs, or loss to follow-up. For the censored observations, one could impute the missing survival times or assume that they are event-free. The former is often difficult, especially if the censoring proportion is large, and extreme imputation assumptions (such as all censored cases fail right after the time of censoring) may distort inferences (Leung et al. 1997; Stajduhar et al. 2009). In this study, we treat all censored cases as event-free regardless of observation time.

*t*of patients possibly from a heterogeneous population. The two-parameter Weibull density function for survival time is given by

for *α* > 0 and *γ* > 0, where *α* is a shape parameter and *γ* is a scale parameter (Ibrahim et al. 2001).

*λ*= log(

*γ*) (Ibrahim et al. 2001), so that:

where *t* > 0, *α* > 0 and *γ*>0.

*λ*parameterisation, are as follows:

*n*subjects; y = (

*y*

_{1},…,

*y*

_{ n }) where

*y*

_{ i }= (

*t*

_{ i },

*δ*

_{ i }) and

*δ*

_{ i }is an indicator function such that (Marin et al. 2005a):

*x*

_{ ij }be the

*j*

^{ th }covariate associated with

*t*

_{ i }for

*j*= 1,2,…,

*p*+ 1. In our case study,

*x*

_{ ij }indicates the

*p*gene expressions from DNA microarray data, and

*x*

_{ i 0}indicates the multi-category phenotype covariate. The data structure is as follows:

*λ*(Thamrin et al. 2013). Given that

*λ*must be positive, one option is to include the covariates as follows:

*α*,

*λ*) are independent a priori (Marin et al. 2005a), and assign Gamma distributions. Thus, the priors are now given by:

and we allow Σ to be diagonal with elements ${\sigma}_{j}^{2},j=1,2,\dots ,p$.

Diffuse priors are represented by large positive values for *σ*
^{2}, and small positive values for *u*
_{
α
} and *v*
_{
α
}.

*α*, β) is given by:

MCMC analysis is performed by sampling from the conditional distributions of the parameters. The conditional distribution of *α* does not have an explicit form but can be sampled from MCMC algorithms such as Metropolis Hastings or slice sampling (Gilks et al. 1996).

### Weibull mixture model

*K*Weibull densities (Marin et al. 2005a) is defined by

where α = (*α*
_{1},…, *α*
_{
K
}), γ = (*γ*
_{1},…, *γ*
_{
K
}) are the parameters of each Weibull distribution and w = (*w*
_{1},…, *w*
_{
K
}) is a vector of nonnegative weights which sum to one.

*S*(

*t*∣

*K*, w, α, γ) and hazard function

*h*(

*t*∣

*K*, w, α, γ) are as follows:

We now assume that we observe possibly right-censored data for *n* patients; y = (*y*
_{1},…, *y*
_{
n
}) where *y*
_{
i
} = (*t*
_{
i
}, *δ*
_{
i
}) and *δ*
_{
i
} is an indicator function as described in Section “Weibull model”.

*x*

_{ ij }be the

*j*

^{ th }covariate associated with patient

*i*, for

*j*= 1,2,…,

*p*. In our application,

*x*

_{ ij }could indicate, for example, the gene expressions. The covariates can be included in the model as follows (Farmomeni and Nardi 2010)

where x
_{
i
} = (*x*
_{
i 1},…, *x*
_{
ip
}), *γ*
_{
m
} = (*γ*
_{1m
},…, *γ*
_{
pm
}) and β
_{
m
} = (*β*
_{1m
},…, *β*
_{
pm
}), for *i* = 1,2,…, *n* and *m* = 1,2,…, *K*.

Here, the incomplete information is modelled via the survivor function, which reflects the probability that the patient was alive for duration greater than *t*
_{
i
}.

*γ*

_{ m }.

*u*

_{ α },

*v*

_{ α },

*u*

_{ γ },

*v*

_{ γ }to express vague prior knowledge about these parameters and we set

*ϕ*= 1 (Marin et al. 2005a). For a model with covariates, we employ a multivariate normal prior on β

_{ m }, so that

and we allow Σ to be diagonal with elements ${\sigma}_{j}^{2},j=1,2,\dots ,p$. Again, we express a vaguely informative prior by setting a large positive value for ${\sigma}_{j}^{2}$. The diagonal matrices were used here but this changed recently (Bhadra and Mallick 2013), so one may argue that a non-diagonal variace-covariance matrix may be more appropriate.

The model described in this section can be fitted using MCMC sampling with latent values *Z*
_{
i
} to indicate component membership of the *i*
^{
th
} observation (Diebolt and Robert 1994; Robert and Casella 2000). Since *w*
_{
m
} = *P* *r*(*Z*
_{
i
} = *m*), we can write *Z*
_{
i
} ∼ *M*(*w*
_{1},…, *w*
_{
K
}). In this scheme, the *Z*
_{
i
} are sampled by computing posterior probabilities of membership, and the other parameters are sampled from their full conditional distributions. This was implemented in the WinBUGS software package (Spiegelhalter et al. 2002).

The WinBUGS software (Lunn et al. 2000; Ntzoufras 2009; Spiegelhalter et al. 2002) is an interactive Windows version of the BUGS program for Bayesian analysis of complex statistical models using MCMC techniques.

Label switching, caused by non-identifiability of the mixture components, was dealt with post-MCMC using the reordering algorithm of Marin et al. (2005b). The algorithm proceeded by selecting the permutation of components at each iteration that minimised the vector dot product with the so-called “pivot”, a high density point from the posterior distribution. The MCMC output was then reordered according to each selected permutation. In this paper, the approximate maximum a posteriori (MAP) (i.e. the realization of parameters corresponding to the MCMC iterate that maximised the unnormalised posterior) was chosen as the pivot.

### Cure model

*n*independent subjects, and we let (

*t*

_{ i },

*δ*

_{ i }) denote the observed time and the event indicator for the

*i*-th observation. Let

*S*

_{1}(

*t*) be the survivor function for the entire population,

*S*

^{∗}(

*t*) be the survivor function for the non-cured group in the population, and

*π*be the cure rate function. Then the standard cure rate model is given by:

The commonly used parametric distributions include Exponential and Weibull for *S*
^{∗}(*t*).

*N*denote the number of latent variables. Assume that

*N*has a Poisson distribution with mean

*θ*. Let

*Z*

_{ i },

*i*= 1,…,

*N*denote the random time, where

*Z*

_{ i }are independently and identically distributed (i.i.d.) with a common distribution function

*F*(

*t*) = 1-

*S*(

*t*). Also, assume that

*Z*

_{ i }are independent of

*N*. The time to event can be defined by the random variable

*Y*= min(

*Z*

_{ i },0 ≤

*i*≤

*N*), where

*P*(

*Z*

_{0}=

*∞*) = 1. Hence, the survival function for the population is given by

A corresponding cure fraction in model (8) is ${lim}_{t\to \infty}{S}_{\mathit{\text{pop}}}\left(t\right)=\text{exp}(-\theta )>0$. We also know from (8) that the cure fraction is given by *S*
_{
pop
}(*∞*) = *P*(*N* = 0) = exp(-*θ*). As *θ* → *∞*, the cure fraction tends to 0, whereas as *θ*→0, the cure fraction tends to 1. Corresponding population density and hazard functions are ${f}_{\mathit{\text{pop}}}\left(t\right)=-\frac{d}{\mathit{\text{dt}}}{S}_{\mathit{\text{pop}}}\left(t\right)=\theta f\left(t\right)\text{exp}(-\theta FF(t\left)\right)$ and *h*
_{
pop
}(*t*) = *θ* *f*(*t*), respectively.

*θ*(Chen et al. 1999; Ibrahim et al. 2001). The population survival function (8) can be written as

where ${S}^{\ast}\left(t\right)=\frac{\text{exp}(-\theta F(t\left)\right)-\text{exp}(-\theta )}{1-\text{exp}(-\theta )}$, and $\phantom{\rule{0.3em}{0ex}}{f}^{\ast}\left(t\right)=\frac{\text{exp}(-\theta F(t\left)\right)}{1-\text{exp}(-\theta )}\theta f\left(t\right)$.

Following Chen et al. (1999) and Ibrahim et al. (2001), we construct the likelihood function. Suppose we have *n* subjects and we assume that the *N*
_{
i
} are i.i.d with Poisson distributions with means *θ*
_{
i
}, *i* = 1,…, *n*. Let *Z*
_{
i 1},…, *Z*
_{
iN
} denote the times for the *N*
_{
i
} competing causes, which are unobserved, and which have a cumulative distribution function, *F*(.). In this section, we will specify a parametric form for *F*(.) that is a Weibull distribution. Let ψ = (*α*, *λ*)^{′}, where *α* is the shape parameter and *λ* is the scale parameter. We incorporate covariates for the cure rate model through the cure parameter *θ* and we have a different cure rate parameter, *θ*
_{
i
}, for each subject.

*k*x 1 vector of covariates for the

*i*th subject, and let β = (

*β*

_{1},…,

*β*

_{ k }) denote the corresponding vector of regression coefficients. We relate

*θ*to the covariates by ${\theta}_{i}=\text{exp}(\underset{i}{\overset{\prime}{\mathit{x}}}\mathit{\beta})$. Let

*t*

_{ i }denote the survival time for subject

*i*, which is right censored, let

*C*

_{ i }be the censoring time, and let

*δ*

_{ i }be the censoring indicator, assuming 1 if

*T*

_{ i }is a failure time and 0 if it is right censored. The observed data are

*D*= (

*n*, t, δ, X), where t = (

*t*

_{1},…,

*t*

_{ n })

^{′}, δ = (

*δ*

_{1},…,

*δ*

_{ n })

^{′}and X = (

*x*

_{1},…,

*x*

_{ n })

^{′}. The complete data are given by

*D*

_{ c }= (

*n*, t, δ, X, N), where N = (

*N*

_{1},…,

*N*

_{ n })

^{′}. The complete-data likelihood function of the parameter (ψ, β) can be written as

Again, we assume independent priors for *β* and *ψ*, where *α*∼*G* *amma*(*a*
_{
α
}, *b*
_{
α
}), *λ* ∼ *N*(μ
_{
λ
}, Σ
_{
λ
}) and β ∼ *N*(μ
_{
β
}, Σ
_{
β
}). We also assume *p*(*α*, *λ*) = *p*(*α* ∣ *δ*
_{0}, *τ*
_{0})*p*(*λ*), $p(\alpha \mid {\delta}_{0},{\tau}_{0})\propto {\alpha}^{{\delta}_{0}-1}\text{exp}(-{\tau}_{0}\alpha )$, and the hyperparameters (*δ*
_{0}, *τ*
_{0}) are specified (Chen et al. 1999; Ibrahim et al. 2001).

*α*,

*λ*, β) becomes

The joint posterior density of (*α*, *λ*, β) in equation (10) is analytically intractable because the integration of the joint posterior density is not easy to perform. Hence, inferences are based on MCMC simulation methods. We can use, for example, the Metropolis-Hastings algorithms or slice sampling to simulate samples of *α*, *λ* and β. MCMC computations were implemented using the WinBUGS system (Spiegelhalter et al. 2002).

## Application to gene expression data

### DLBCL dataset

We applied the proposed method of model averaging across the three candidate survival models to a dataset containing gene expression of Diffuse Large B-cell Lymphoma (DLBCL). The dataset comprises gene expression measurements and survival times of patients with DLBCL (Rosenwald et al. 2002). DLBCL (Lenz et al. 2008) is a type of cancer of the lymphatic system in adults which can be cured by anthracycline-based chemotherapy in only 35 to 40 percent of patients (Rosenwald et al. 2002). In general, types of this disease are very diverse and their biological properties are largely unknown, meaning that this is a relatively difficult cancer to cure and prevent. Rosenwald et al. (2002) proposed that there are three phenotypes subgroups of patients of DLBCL: activated B-like DLBCL, germinal centre (GC)-B like and type III DLBCL. The GC B-like DLBCL is less dangerous than the others in the progression of the tumour; the activated B-like DLBCL is more active than the others and the type III DLBCL is the most dangerous in the progression of tumour (Alizadeh et al. 2000). These groups were defined using microarray experiments and hierarchical clustering. The authors showed that these phenotypes subgroups were differentiated from each other by distinct gene expressions of hundreds of different genes and had different survival time patterns. This dataset contains 219 patients with DLBCL, including 138 patient deaths during follow-up. Patients with missing values for a particular microarray element were excluded from all analyses involving that element.

Based on patterns of gene expression in biopsy specimens of the lymphoma, Rosenwald et al. (2002) analysed this dataset to predict the likelihood of patients’ survival after chemotherapy for DLBCL. By using a Cox proportional-hazards model, Rosenwald et al. (2002) identified five individual gene expressions which correlated with the survival after chemotherapy. These gene expressions are germinal center B-cell (GC-B), lymphoma node, proliferation, BMP6 and MHC. In this study, these five gene expressions are used as covariates for estimating survival times based on the three competing models in Section “Models”.

### Results

*β*

_{1}), lymphoma node (

*β*

_{2}) and MHC (

*β*

_{5}). These three genes have a negative effect on the expected survival time. For the mixture model, GC-B (

*β*

_{1}), lymphoma node (

*β*

_{2}) and proliferation (

*β*

_{3}) accounted for patients’ survival times in the first component. In the second component, GC-B (

*β*

_{1}), lymphoma node (

*β*

_{2}) and MHC signature (

*β*

_{5}) substantially explained patients’ survival times. All these genes have negative effects on the expected survival time for their respective component. For the cure model, four of these genes substantially describe patients’ survival times, namely GC-B (

*β*

_{1}), lymphoma node (

*β*

_{2}), BMP6 (

*β*

_{4}) and MHC (

*β*

_{5}) signature. Three of these, namely GC-B (

*β*

_{1}), lymphoma node (

*β*

_{2}) and MHC signature (

*β*

_{5}), have a negative effect on the expected survival time. Under the cure model, approximately 33.8% of the patients are cured of DLBCL (Figure 2).

**The estimated posterior mean of the parameters, the 95% credible intervals (CI), the BIC values and the BMA weights for each of the fitted models for the full DLBCL dataset**

Model | Parameter | Mean | 95% CI | BIC | Weight |
---|---|---|---|---|---|

Weibull |
| 0.7305 | (0.626,0.840) | 687.0953 | 0.0009 |

| -1.578 | (-1.84, -1.33) | |||

| -0.3446 | (-0.516, -0.172) | |||

| -0.2844 | (-0.454, -0.116) | |||

| 0.2097 | (-0.049, 0.468) | |||

| 0.3292 | (0.115, 0.537) | |||

| -0.3019 | (-0.488, -0.112) | |||

Mixture |
| 4.029 | (2.411, 6.631) | 734.0054 | ≈0 |

| 0.7707 | (0.662, 0.885) | |||

| 6.857 | (5.479, 8.205) | |||

| -1.724 | (-2.007, -1.457) | |||

| -11.62 | (-12.88, -10.35) | |||

| -0.3956 | (-0.575, -0.216) | |||

| -2.087 | (-3.54, -0.689) | |||

| -0.3172 | (-0.495, -0.143) | |||

| -2.241 | (-3.425, -1.059) | |||

| 0.1972 | (-0.064, 0.461) | |||

| -0.2849 | (-1.434, 0.854) | |||

| 0.3594 | (0.141, 0.574) | |||

| -0.7928 | (-2.107, 0.477) | |||

| -0.3102 | (-0.500, -0.115) | |||

| 0.01992 | (0.002, 0.053) | |||

| 0.9801 | (0.946, 0.997) | |||

Cure |
| 0.9884 | (0.828, 1.145) | 673.1359 | 0.9991 |

| 0.1611 | (-0.124, 0.560) | |||

| -0.3151 | (-0.484, -0.144) | |||

| -0.2821 | (-0.451, -0.115) | |||

| 0.189 | (-0.070, 0.442) | |||

| 0.3303 | (0.118, 0.539) | |||

| -0.3039 | (-0.490, -0.112) |

This is clearly exhibited in Table 1, which shows that the cure model has the largest posterior model probability (or BMA weight). To evaluate the model fit, a comparison of predicted values under the models and of the observed data was carried out.

**The estimated posterior mean of parameters, the 95% CI, BIC values and the BMA weights for each of the models based on phenotype for the DLBCL dataset**

Phenotype | Model | Variable | Parameter | Mean | 95% CI | BIC | Weight |
---|---|---|---|---|---|---|---|

GCB | Weibull |
| 0.692 | (0.5365, 0.8595) | 341.212 | 0.497 | |

Intercept |
| -1.649 | (-2.185, -1.17) | ||||

GCB |
| -0.179 | (-0.5859, 0.239) | ||||

Lymphoma |
| -0.118 | (-0.3958, 0.1607) | ||||

Proliferation |
| 0.459 | (-0.0306, 0.934) | ||||

BMP6 |
| 0.414 | (0.01773, 0.809) | ||||

MHC |
| -0.325 | (-0.6389, -0.01228) | ||||

Mixture |
| 4.252 | (2.591, 7.175) | 377.759 | ≈0 | ||

| 0.816 | (0.6209, 1.032) | |||||

Intercept |
| 6.491 | (5.246, 7.781) | ||||

| -2.152 | (-2.798, -1.567) | |||||

GCB |
| -11.81 | (-13.05, -10.53) | ||||

| -0.030 | (-0.5104, 0.4592) | |||||

Lymphoma |
| -1.839 | (-3.082, -0.6744) | ||||

| -0.134 | (-0.48, 0.2254) | |||||

Proliferation |
| -2.165 | (-3.313, -0.9932) | ||||

| 0.588 | (-0.07796, 1.407) | |||||

BMP6 |
| -0.242 | (-1.357, 0.8482) | ||||

| 0.654 | (0.17, 1.161) | |||||

MHC |
| -0.629 | (-1.993, 0.5117) | ||||

| -0.382 | (-0.7687, -0.002227) | |||||

| 0.090 | (0.02007, 0.1863) | |||||

| 0.91 | (0.8137, 0.9799) | |||||

Cure |
| 0.845 | (0.6075, 1.1) | 341.188 | 0.503 | ||

Intercept |
| 0.604 | (-0.3556, 3.394) | ||||

GCB |
| -0.173 | (-0.5754, 0.2402) | ||||

Lymphoma |
| -0.116 | (-0.3891, 0.1579) | ||||

Proliferation |
| 0.433 | (-0.0522, 0.9041) | ||||

BMP6 |
| 0.396 | (-0.0007, 0.788) | ||||

MHC |
| -0.330 | (-0.6422, -0.0209) | ||||

ABC | Weibull |
| 0.894 | (0.695, 1.115) | 215.564 | 0.013 | |

Intercept |
| -1.86 | (-2.562, -1.217) | ||||

GCB |
| -0.509 | (-0.9948, -0.03871) | ||||

Lymphoma |
| -0.626 | (-0.9568, -0.3099) | ||||

Proliferation |
| -0.487 | (-1.118, 0.1422) | ||||

BMP6 |
| 0.645 | (0.2725, 1.021) | ||||

MHC |
| -0.479 | (-0.7955, -0.1598) | ||||

Mixture |
| 2.427 | (1.083, 4.152) | 256.552 ‘ | ≈0 | ||

| 0.960 | (0.7525, 1.189) | |||||

Intercept |
| 6.636 | (5.301, 7.959) | ||||

| -2.572 | (-3.346, -1.865) | |||||

GCB |
| -12.11 | (-13.36, -10.86) | ||||

| -0.925 | (-1.438, -0.4356) | |||||

Lymphoma |
| -3.155 | (-4.578, -1.75) | ||||

| -0.768 | (-1.114, -0.4341) | |||||

Proliferation |
| -2.377 | (-3.561, -1.188) | ||||

| -0.480 | (-1.099, 0.1353) | |||||

BMP6 |
| 0.079 | (-1.064, 1.232) | ||||

| 0.690 | (0.3249, 1.061) | |||||

MHC |
| -0.644 | (-1.919, 0.6499) | ||||

| -0.515 | (-0.8176, -0.2047) | |||||

| 0.037 | (0.0046, 0.09883) | |||||

| 0.963 | (0.9012, 0.9953) | |||||

Cure |
| 1.189 | (0.8906, 1.483) | 206.961 | 0.987 | ||

Intercept |
| 0.019 | (-0.6417, 0.7362) | ||||

GCB |
| -0.432 | (-0.8874, 0.01376) | ||||

Lymphoma |
| -0.587 | (-0.905, -0.2867) | ||||

Proliferation |
| -0.484 | (-1.076, 0.1012) | ||||

BMP6 |
| 0.607 | (0.2557, 0.9631) | ||||

MHC |
| -0.446 | (-0.7481, -0.1346) | ||||

Type III | Weibull |
| 0.834 | (0.5958, 1.101) | 162.27 | 0.538 | |

Intercept |
| -1.75 | (-2.736, -0.9093) | ||||

GCB |
| -0.404 | (-1.028, -0.19) | ||||

Lymphoma |
| -0.274 | (-0.7404, 0.1644) | ||||

Proliferation |
| 0.506 | (-0.0897, 1.084) | ||||

BMP6 |
| 0.017 | (-0.5301, 0.5206) | ||||

MHC |
| -0.199 | (-0.6839, 0.3098) | ||||

Mixture |
| 11.82 | (8.609, 15.14) | 196.271 | ≈0 | ||

| 0.596 | (0.43, 0.7757) | |||||

Intercept |
| 6.002 | (3.682, 8.336) | ||||

| -5.005 | (-7.19, -2.812) | |||||

GCB |
| -9.32 | (-12.02, -6.611) | ||||

| 0.564 | (0.1829, 1.004) | |||||

Lymphoma |
| -2.913 | (-5.716, -0.02716) | ||||

| -0.558 | (-1.015, -0.1525) | |||||

Proliferation |
| -2.021 | (-4.547, 0.455) | ||||

| 0.893 | (0.3455, 1.515) | |||||

BMP6 |
| 0.320 | (-2.466, 3.373) | ||||

| 0.140 | (-0.2735, 0.5384) | |||||

MHC |
| -0.336 | (-2.733, 2.323) | ||||

| -0.293 | (-0.7741, 0.1504) | |||||

| 0.072 | (0.0108, 0.1805) | |||||

| 0.928 | (0.8195, 0.9891) | |||||

Cure |
| 0.969 | (0.6534, 1.339) | 162.578 | 0.462 | ||

Intercept |
| 0.989 | (-0.5077, 4.153) | ||||

GCB |
| -0.349 | (-0.973, -0.25) | ||||

Lymphoma |
| -0.269 | (-0.7375, 0.1687) | ||||

Proliferation |
| 0.502 | (-0.0955, 1.084) | ||||

BMP6 |
| 0.046 | (-0.4801, -0.1801) | ||||

MHC |
| -0.183 | (-0.6625, 0.3207) |

From Tables 1 and 2, we can see that the Weibull model is better than a two-component Weibull mixture model.

**The percentage of observed values that lay in the corresponding 95% posterior prediction interval for the individual models and BMA model based on the full DLBCL dataset and each of the three phenotypes**

Model | All DLBCL | GCB | ABC | Type III |
---|---|---|---|---|

Weibull | 87.5 | 90 | 89.1 | 89.3 |

Mixture | 85.9 | 88 | 86.9 | 82.1 |

Cure | 94.3 | 92 | 91.3 | 85.7 |

BMA | 91.9 | 94 | 93.4 | 92.8 |

Furthermore, in the GCB phenotype, the genes corresponding to the BMP6 (*β*
_{4}) and MHC signature (*β*
_{5}) in the Weibull model and MHC signature (*β*
_{5}) in the cure model substantially affect patients’ survival time. In the ABC phenotype, in the Weibull model, with the exception of proliferation (*β*
_{3}), all genes were involved substantially in the description of patients’ survival and lymphoma node (*β*
_{2}), BMP6 (*β*
_{4}) and MHC signature (*β*
_{5}) are potentially important prognostic factors for predicting survival in the cure model. For the type III phenotype, the GC-B gene (*β*
_{1}) in both models and only the BMP6 gene (*β*
_{4}) in the cure model are substantial in explaining the survival times of the patients.

Under the cure model, in the GCB phenotype, approximately 33.2% of the patients are estimated to be cured of DLBCL. In the ABC and type III phenotypes, the respective cure rates are approximately 26.6% and 18.7% (Figure 2).

The results of the posterior densities prediction for the individual models and the model averaged prediction based on these three phenotypes are presented in Figure 3. In comparison to other models, the mixture model fitted the data poorly for each phenotype. In detail, using model averaging, for the GCB phenotype, 94% of the observed survival times in the dataset lie in the respective 95% prediction intervals. For the other two phenotypes, namely the ABC and the type III, 93.4% and 92.8% of the observed survival times in the dataset are in the corresponding 95% prediction intervals, respectively (Table 3).

## Discussion

This study has adopted a Bayesian model averaging approach to account for model uncertainty in the prediction of survival. The case study that we considered involved lymphoma cancer survival, with covariates given by phenotypes and gene expressions. Here, we proposed three competing models and used BMA to combine these models to account for model uncertainty.

Overall, the results of this study indicate that if using the full dataset without further grouping, selecting a single model that best fits the data was adequate. The reason is that there is clear support for one model (i.e. only one model has a relatively larger BIC value and dominates based on this criterion). However, the results were different when the model selection process took into account the phenotype subgroups of the patients. A single model appeared to be inadequate. This was due to the fact that the values of BIC for the Weibull and the cure had nearly equal weight, indicating the absence of a dominant model based on this criterion and the presence of uncertainty issues in the model selection. As suggested and shown in this study, BMA was used to address this problem. The applicability of BMA was also associated with the smaller sample size in each phenotype subgroup (Annest et al. 2009; Volinsky et al. 1997; Yeung et al. 2005).

This study also revealed that in each phenotype, the expression and number of predictor genes substantially describing the survival times of the patients varied across models. Overall, in both of the favourable models, none of the genes were identified consistently as substantial predictors for the patients’ survival. For example, in the Weibull model, the MHC and BMP genes in the GCB and ABC phenotypes and the GCB genes in the ABC and Type III phenotypes were important predictors of survival. In contrast, in the cure model, BMP was substantially associated with predicted survival in the ABC and Type III phenotypes. For both models, only three genes i.e. lymphoma node, BMP6 and MHC signature in the ABC phenotype were highly associated with the survival times of the patients.

This study has indicated that the application of BMA to combine competing models overcomes the problem of model uncertainty. Comparison of different survival models has allowed the identification and analysis of more detailed relationships between gene expressions in given phenotypes and the survival times of the patients. An advantage of BMA is more accurate and precise prediction of patient survival. However, this study only involved three candidate models. More models can be obviously included in the analysis. This study has also focused on the marginal likelihood *p*(*D* ∣ *Q*
_{
s
}) estimation methods based on the Laplace approximation. However, other approaches are also possible. Indeed marginal likelihood estimation is possible using nested sampling (Skilling 2006), where the marginal likelihood is viewed as the expectation, with respect to the prior, of the likelihood. Another generic approach is Chib’s method (Chib 1995), which can be applied to output from the Gibbs sampler. Applying BMA to other datasets or other applications is desired to obtain robust predictions.

## Declarations

### Acknowledgements

The first author acknowledges the Ministry of National Education Indonesia (DIKTI) for providing funding for this research as well as Dr. Christopher Drovandi from Queensland University of Technology for his generous feedback.

## Authors’ Affiliations

## References

- Akaike H: Information theory and an extension of the maximum likelihood principle. In
*Second Int Symp Inform Theory 1*Edited by: Petrov BN, Csaki F. 1973, 267-281.Google Scholar - Alizadeh AA, Eisen MB, Davis RE: Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling.
*Nature*2000, 403: 503-511. 10.1038/35000501View ArticleGoogle Scholar - Annest A, Bumgarner RE, Raftery AE, Yeung KE: Iterative Bayesian model averaging: a method for the application of survival analysis to high-dimensional microarray data.
*BMC Bioinformatics*2009, 10(72):1471-2105.Google Scholar - Bhadra A, Mallick BK: Joint high dimensional Bayesian variable and covariance selection with an application to eQTL analysis.
*Biometrics*2013, 69: 447-457. 10.1111/biom.12021View ArticleGoogle Scholar - Basu S, Tiwari RC: Breast cancer survival, competing risk and mixture cure model: a Bayesian analysis.
*J R Stat Soc*2010, 173(2):307-329. 10.1111/j.1467-985X.2009.00618.xView ArticleGoogle Scholar - Bonato V, Baladandayuthapani V, Broom BM, Sulman EP, Aldape KD, Do KA: Bayesian Ensemble methods for survival prediction in gene expression data.
*Bioinformatics*2011, 27(3):359-367. 10.1093/bioinformatics/btq660View ArticleGoogle Scholar - Buckland S, Burnham K, Augustin N: Model selection: an integral part of inference.
*Biometrics*1997, 53: 603-618. 10.2307/2533961View ArticleGoogle Scholar - Burnham K, Anderson D:
*Model selection and multimodel inference: a practical information-theoretic approach*. Springer, New-York; 2002.Google Scholar - Burnham K, Anderson D: Multimodel inference: understanding AIC and BIC in model selection.
*Sociol Methods Res*2004, 33: 261-304. 10.1177/0049124104268644View ArticleGoogle Scholar - Chen MH, Ibrahim JG, Sinha D: A new Bayesian model for survival data with a surviving fraction.
*J Am Stat Assoc*1999, 94: 909-919. 10.1080/01621459.1999.10474196View ArticleGoogle Scholar - Chib S: Marginal likelihood from the Gibbs output.
*J Am Stat Assoc*1995, 90: 1313-1321. 10.1080/01621459.1995.10476635View ArticleGoogle Scholar - Claeskens G, Hjort NL:
*Model selection and model averaging*. Cambridge University Press, New York; 2008.View ArticleGoogle Scholar - Clyde M: Model uncertainty and health effect studies for particulate matter.
*Environmetrics*2000, 11: 745-763. 10.1002/1099-095X(200011/12)11:6<745::AID-ENV431>3.0.CO;2-NView ArticleGoogle Scholar - Collet D:
*Modelling survival data in medical research,*. Chapman and Hall, Florida; 1994.View ArticleGoogle Scholar - Diebolt J, Robert CP: Estimation of finite mixture distributions through Bayesian sampling.
*J R Stat Soc Ser B*1994, 56: 363-375.Google Scholar - Dodson B:
*Weibull Analysis*. American Society Quality, Milwaukee; 1994.Google Scholar - Draper D: Assessment and propagation of model uncertainty.
*J R Stat Soc Series B*1995, 57: 45-97.Google Scholar - Farmomeni A, Nardi A: A two-component Weibull mixture to model early and late mortality in a Bayesian framework.
*Comput Stat Data Anal*2010, 54: 416-428. 10.1016/j.csda.2009.09.007View ArticleGoogle Scholar - Gilks WR, Richardson S, Spiegelhalter DJ:
*Markov chain monte carlo in practice*. Chapman and Hall, Florida; 1996.Google Scholar - Hjort NL, Claeskens G: Frequentist model average estimators (with discussion).
*J Am Stat Assoc*2003, 98: 879-899. 10.1198/016214503000000828View ArticleGoogle Scholar - Hoeting J, Madigan D, Raftery AE, Volinsky C: Bayesian model averaging: a tutorial.
*Stat Sci*1999, 14: 382-417. 10.1214/ss/1009212519View ArticleGoogle Scholar - Ibrahim JG, Chen MH, Sinha D:
*Bayesian survival analysis*. Springer, New York; 2001.View ArticleGoogle Scholar - Jackson CH, Thompson SG, Sharples LD: Accounting for uncertainty in health economic decision models by using model averaging.
*J R Stat Soc Ser A*2009, 172(2):383-404. 10.1111/j.1467-985X.2008.00573.xView ArticleGoogle Scholar - Kleinbaum DG, Klein M:
*Survival analysis: a self-learning text*. Springer, New York; 2005.Google Scholar - Kundu D: Bayesian inference and life testing plan for the Weibull distribution in presence of pregressive censoring.
*Technometrics*2008, 50(2):144-154. 10.1198/004017008000000217View ArticleGoogle Scholar - Lenz G, Wright GW, Emre NT, Kohlhammer H, Dave SS, Davis RE, Carty S, Lam LT, Shaffer AL, Xiao W, Powell J, Rosenwald A, Ott G, Muller-Hermelink HK, Gascoyne RD, Connors JM, Campo E, Jaffe ES, Delabiei J, Smeland EB, Rimsza LM, Fisher RI, Weisenburger DD, Chan WC, Staudt LM: Molecular subtypes of diffuse large B-cell lymphoma arise by distinct genetic pathways.
*Proc Natl Acad Sci USA*2008, 105(36):13520-13525. 10.1073/pnas.0804295105View ArticleGoogle Scholar - Leung KM, Elashoff RM, Afifi AA: Censoring issues in survival analysis.
*Ann Rev Public Health*1997, 18: 83-104. 10.1146/annurev.publhealth.18.1.83View ArticleGoogle Scholar - Lunn DJ, Thomas A, Best N, Spiegelhalter D: WinBUGS – A Bayesian modelling framework: concepts, structure, and extensibility.
*Stat Comput*2000, 10: 325-337. 10.1023/A:1008929526011View ArticleGoogle Scholar - Madigan D, Raftery AE: Model selection and accounting for model uncertainty in graphical models using Occam’s window.
*J Am Stat Assoc*1994, 89: 1535-1546. 10.1080/01621459.1994.10476894View ArticleGoogle Scholar - Marin JM, Bernal MR, Wiper MP: Using Weibull mixture distributions to model heterogeneous survival.
*Commun Stat Simul Comput*2005a, 34(3):673-684. 10.1081/SAC-200068372View ArticleGoogle Scholar - Marin JM, Mengersen K, Robert CP: Bayesian modelling and inference on mixtures of distributions. In
*Handbook of statistics*. Edited by: Dey D, Rao CR. Elsevier-Sciences, Amsterdam; 2005b.Google Scholar - McGrory CA, Titterington DM: Variational approximations in Bayesian model selection for finite mixture distributions.
*Comput Stat Data Anal*2007, 51(11):5352-5367. 10.1016/j.csda.2006.07.020View ArticleGoogle Scholar - Ntzoufras I:
*Bayesian modelling using WinBUGS*. Wiley, New Jersey; 2009.View ArticleGoogle Scholar - Pramana S, Shkedy Z, Göehlmann HW, Talloen W, Bondt AD, Straetemans R, Lin D, Pinheiro J: Model-based parametric approaches. In
*Modeling dose-response microarray data in early drug development experiments using R,*. Edited by: Lin D, Shkedy Z, Yekutieli D, Amaratunga D, Bijnens L. Springer, New York; 2012:231-249.Google Scholar - Raftery AE: Bayesian model selection in social research (with discussion).
*Soc Methodol*1995, 25: 111-163.View ArticleGoogle Scholar - Raftery AE: Approximate Bayes factors and accounting for model uncertainty in generalized linear models.
*Biometrika*1996, 83: 251-266. 10.1093/biomet/83.2.251View ArticleGoogle Scholar - Raftery AE, Madigan D, Hoeting JA: Bayesian model averaging for linear regression models.
*J Am Stat Assoc*1997, 92(437):179-191. 10.1080/01621459.1997.10473615View ArticleGoogle Scholar - Robert CP, Casella G:
*Monte Carlo statistical methods*. Springer, New York; 2000.Google Scholar - Rosenwald A, Wright G, Wiestner A, Chan WC, Connors JM, Campo E, Gascoyne RD, Grogan TM, Muller HK, Smeland EB, Chiorazzi M, Giltnane JM, Hurt EM, Zhao H, Averett L, Henrickson S, Yang L, Powell J, Wilson WH, Jaffe ES, Simon R, Klausner RD, Montserrat E, Bosch F, Greiner TC, Weisenburger DD, Sanger WG, Dave BJ, Lynch JC, Vose J,
*et al*.: The use Of molecular profiling to predict survival after chemotherapy for diffuse large B-cell lymphoma.*N Engl J Med*2002, 346(25):1937-1947. 10.1056/NEJMoa012914View ArticleGoogle Scholar - Schwarz GE: Estimating the dimension of a model.
*Ann Stat*1978, 6(2):461-464. 10.1214/aos/1176344136View ArticleGoogle Scholar - Skilling J: Nested sampling for general Bayesian computation.
*Bayesian Anal*2006, 1: 833-860. 10.1214/06-BA127View ArticleGoogle Scholar - Spiegelhalter D, Best N, Carlin B, VanderLinde A: Bayesian measures of model complexity and fit.
*J R Stat Soc Ser B*2002, 64(4):583-639. 10.1111/1467-9868.00353View ArticleGoogle Scholar - Stephens M:
*Bayesian methods for mixtures of normal distributions*. PhD thesis, The University of Oxford; 1997.Google Scholar - Stajduhar I, Basic BD, Bogunovic N: Impact of censoring on learning Bayesian networks in survival modelling.
*Artif Intell Med*2009, 47: 199-217. 10.1016/j.artmed.2009.08.001View ArticleGoogle Scholar - Thamrin SA, McGree JM, Mengersen KL: Bayesian Weibull Survival Model for gene expression data. In
*Case studies in Bayesian statistical modelling and analysis,*. Edited by: Alston CL, Mengersen KL, Pettitt AN. Wiley, UK; 2013:171-185.Google Scholar - Volinsky CT, Raftery AE: Bayesian information criterion for censored survival models.
*Biometrics*2000, 56: 256-262. 10.1111/j.0006-341X.2000.00256.xView ArticleGoogle Scholar - Volinsky C, Madigan D, Raftery AE, Kronmal R: Bayesian model averaging in proportional hazard models: assessing the risk of a stroke.
*Appl Stat*1997, 46(4):443-448.Google Scholar - Yakovlev AY, Tsodikov AD:
*Stochastic models of tumor latency and their biostatistical applications*. World Scientific, Singapore; 1996.Google Scholar - Yeung KE, Bumgarner R, Raftery AE: Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray.
*Bioinformatics*2005, 21: 2394-2402. 10.1093/bioinformatics/bti319View ArticleGoogle Scholar - Yuan Y, Yin G: Robust EM continual reassessment method in oncology dose finding.
*J Am Stat Assoc*2011, 108(495):818-831.View ArticleGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.