B. Haibe-Kains, C. Desmedt, C. Sotiriou and G. Bontempi

A comparative study of survival models for breast cancer

prognostication based on microarray data: a single gene beat them all?

B. Haibe-Kains, C. Desmedt, C. Sotiriou and G. Bontempi

BIOINFORMATICS Vol. 24 no. 19 2008, pages 2200-2208

Outline

• Introduction to microarray• Introduction

– Motivation– Purpose– Results– Difficulties

• Risk prediction methods• Performance assessment• Results

Introduction to microarray

• http://www.bio.davidson.edu/courses/genomics/chip/chip.html

• 清華大學 郝旭昶

Introduction - Motivation

• Survival prediction of breast cancer (BC) patients, independently of treatment, also known as prognostication, is a complex task.– Clinically similar breast tumors and molecularly

heterogeneous– Several clinical and pathological indicators such as

histological grade, tumor size and lymph node have been used for the survival prediction of breast cancer.

– Although BC prognostication has been the object of intense research, a still open challenge is how to detect patient who needs adjuvant systemic therapy.

Introduction - Motivation

• The advent of array-based technology and the sequencing of the human genome brought new insights into breast cancer biology and prognosis.

• Several research teams conducted comprehensive genome-wide assessments of gene expression profiling and identified prognostic gene expression signatures.

• With respect to clinical guidelines, these signatures were shown to correctly identify a larger group of low-risk patient not requiring treatment.

Introduction - Motivation

• In fact, clinicians encounter problems when confronted with patient with intermediate-grade tumors (Grade 2). These tumors, which represent 30-60% of cases, are a major source of inter-observer discrepancy and may display intermediate phenotype and survival, making treatment decisions for the patients a great challenge, with subsequent under- or over-treatment.

Introduction - Purpose

• The aim of this work is to assess quantitatively the accuracy of prediction obtained with state-of-the-art data analysis techniques for BC microarray data through an independent and thorough framework.

• Compare the prediction accuracy of these methods in several BC microarray prognostication tasks to elucidate the key characteristics of a successful risk prediction method and to bring additional insights into BC biology.

Introduction - Results

• Complex prediction methods are highly exposed to performance degradation despite the use of cross-validation techniques for the following reasons:– The large number of variables– The reduced amount of samples– The high degree of noise

• Our analysis shows that the most complex methods are not significantly better than the simplest one, a univariate model relying on a single proliferation gene.

Introduction - Results

• This result suggests the proliferation might be the most relevant biological process for BC prognostication.

• The loss of interpretability deriving from the use of overcomplex methods may be not sufficiently counterbalabed by an improvement of the quality of prediction.

Introduction - Difficulties

• Censored information cannot be exploited by traditional supervised classification the regression methods, but demands the adoption of specific survival analysis techniques, like the semi-parametric Cox’s proportional hazards model.

• When the number of explanatory variables exceeds by far the number of petients in the sample cohort, overfitting of naively applied data mining methods and overoptimistic performance assessment lie in wait. At the same time, it is very difficult to select the most relevant variables for prediction, because of their interdependency.

Introduction - Difficulties

• The lack of standards in performance assessment for risk prediction models.

• The validation and the comparison of BC microarray prognostication methods are made difficult due to the lack of independent data.

Risk prediction methods

• In this work, we compare the performance of 13 risk prediction methods on more than 1000 patients.

• The first risk prediction method is also the simplest one and defines the risk score as the expression of a single proliferation gene (AURKA).

• The following 10 methods (from 2 to 11) are characterized by the type of observed genotype (input data), the dimension reduction strategy, the structure of the model, the learning algorithm and the predicted phenotype (outcome variable).

Risk prediction methods

Risk prediction methods

• Genotype:– It can be the expression of

• Single proliferation gene (AURKA)• Biologically driven selection of genes of interest (BD)• The whole genome (GW)

– AURKA and the small set of genes in BD were selected to represent several biological processes in BC. The selected genes were AURKA, PLAU, STAT1, VEFG, GASP3, ESR1 and ERBB2, representing the proliferation, tumor invasion/metastasis, immune response, agiogenesis, apoptosis phenotypes and the ER and HER2 signaling, respectively.

Risk prediction methods

• Dimension reduction strategy:– A simple univariate ranking (RANK) of the k

most relevant features– A selection of the first k principal components

(PCA)

• Structure of the model:– Multivariate (MULTIV) model– a linear combination of univariate modes

(COMBUNIV)

Risk prediction methods

• Learning algorithm:– The linear combination of gene expressions weighted

by the significance computed from the Wilconxon rank sum test (WILCOSON)

– The multivariate linear regression model (LM)– The linear combination of gene expressions weighted

by the significance computed from the univariate Cox’s proportional hazards model (COX)

– The multivariate Cox’s model with L1 regularization (RCOX)

Risk prediction methods

• Phenotype:– The binary class defined by histological

grades 1 and 3 (HG)– The censored survival data (SURA)– The time of events (TOE), i.e, the times from

diagnosis until the patient experienced an event.

Risk prediction methods

• The last two model : – GENE76 (Wang et al., 2005), and GGI(Sotiriou et al.,

2006b)– The GENE76 model is defined as hierarchical model

using two linear combinations of the top gene expressions with respect to a ranking based on Cox’s proportional hazards mode.

– The GGI model consists of a linear combination of the expressions of the top probes ranked according to their standardized mean difference between patients with histological grades 1 and 3 tumors.

Performance assessment

• In order to assess the performance of the risk prediction methods, we used five accuracy measures:– Time-dependent ROC Curve:

• A standard technique for assessing the performance of a continuous variable for binary classification.

– Sensitivity and specificity:• A widely used performance criterion for a clinical test is the

pair {sensitivity, specificity}• For risk score prediction, we estimated the specificity for a

sensitivity of 90% in accordance with the St Gallen and National Institutes of Health.

• The larger the sensitivity and the specificity, the better is the predictability of time to event (TTE).

Performance assessment

– Concordance index:• The concordance index (C-index) computes the probability

that, for a pair of randomly chosen comparable patients, the patient with the higher risk prediction will experience an event before the lower risk patient.

• ri and rj stand for the risk predictions of the i-th and the j-th patient, respectively.

is the set of all the pairs of patient {i,j} who meet one of the following conditions:

– Both patient i and j experienced an event and time ti<tj

– Only patient I experienced an event and ti< tj.• The larger C-index, the better is the predictability of TTE.

Performance assessment

– Brier score:• The Brier score, denoted by BSC, is defined as the

squared difference between an event occurrence and its predicted probabilities at time t.

• The lower the BSC, the better is the predictability of TTE at time t.

– Hazard ration:• The larger the HR, the larger is the difference in

survival probabilities between the groups of patients, and consequently the better is the discrimination between low- and high-risk groups.

Results

• Breast cancer datasets– Four large microarray BC datasets

• VDX– Includes the gene expressions of 286 untreated node-negative

BC patients and war used to build GENE76 and to validate GGI.

• TBG– Used as an official validation of GENE76 and GGI.

• TAM– Homogeneously treated by tamoxifen therapy.

• UPP– Treated with heterogeneous therapies.

– These datasets are publicly available from the GEO databasehttp://www.ncbi.nlm.nih.gov/geo/

Results

• Risk score prediction

The most complex models

Results

Discussion and conclusions

• The loss of interpretability deriving from the use of overcomplex methods in survival analysis of BC microarray data might be not sufficiently counterbalanced by an improvement in the quality of prediction.