Modelling survival after treatment of intraocular melanoma using artificial neural networks and Bayes theorem

INSTITUTE OF PHYSICS PUBLISHING PHYSICS IN MEDICINE AND BIOLOGY

Phys. Med. Biol. 49 (2004) 87–98 PII: S0031-9155(04)59746-4

Modelling survival after treatment of intraocularmelanoma using artificial neural networks andBayes theorem

Azzam F G Taktak1, Anthony C Fisher1 and Bertil E Damato2

1 Department of Clinical Engineering, Duncan Building, Royal Liverpool University Hospital,Liverpool L7 8XP, UK2 Department of Ophthalmology, Royal Liverpool University Hospital, Liverpool L7 8XP, UK

E-mail: [email protected]

Received 18 February 2003, in final form 7 November 2003Published 15 December 2003Online at stacks.iop.org/PMB/49/87 (DOI: 10.1088/0031-9155/49/1/006)

AbstractThis paper describes the development of an artificial intelligence (AI) systemfor survival prediction from intraocular melanoma. The system used artificialneural networks (ANNs) with five input parameters: coronal and sagittaltumour location, anterior tumour margin, largest basal tumour diameter andthe cell type. After excluding records with missing data, 2331 patients wereincluded in the study. These were split randomly into training and test sets.Date censorship was applied to the records to deal with patients who werelost to follow-up and patients who died from general causes. Bayes theoremwas then applied to the ANN output to construct survival probability curves.A validation set with 34 patients unseen to both training and test sets wasused to compare the AI system with Cox’s regression (CR) and Kaplan–Meier(KM) analyses. Results showed large differences in the mean 5 year survivalprobability figures when the number of records with matching characteristicswas small. However, as the number of matches increased to >100 the systemtended to agree with CR and KM. The validation set was also used to comparethe system with a clinical expert in predicting time to metastatic death. Therms error was 3.7 years for the system and 4.3 years for the clinical expertfor 15 years survival. For <10 years survival, these figures were 2.7 and 4.2,respectively. We concluded that the AI system can match if not better theclinical expert’s prediction. There were significant differences with CR andKM analyses when the number of records was small, but it was not knownwhich model is more accurate.

0031-9155/04/010087+12$30.00 © 2004 IOP Publishing Ltd Printed in the UK 87

88 A F G Taktak et al

1. Introduction

Survival modelling is important in oncology, because it provides an indication of prognosis,enables special measures to be targeted at high-risk individuals and enhances the evaluation ofclinical procedures. Traditionally, survival data have been analysed using statistical methodssuch as Cox’s regression (CR) analysis to construct survival prediction functions in varioustypes of cancer (Cintin et al 2002, Gamel and Jones 1993, Pollack et al 2002, Richter et al2002, Seagard and Kock 1995, Shields 2000). These methods require previous knowledge ofthe correct functional relationship between predictive variables, which may not be available(Lisboa 2002). Another problem is uncertainty caused by missing data (Frize et al 2001).

Artificial neural networks (ANNs) have been used in conjunction with statistical methodsto model survival in cancer (Ripley and Ripley 2001). As with statistical methods, ANNs allowthe mixture of categorical and continuous variables. ANNs may offer advantages over linearstatistical models. They allow (1) arbitrary nonlinear relationships between independent anddependent variables and (2) all possible interactions between dependent variables. Moreover,ANNs do not require explicit distributional assumption. Their main disadvantage is their‘black box’ nature making it difficult to get an insight into the problem. A study comparingANNs with statistical regression analysis has found that for smaller sample sizes (n < 2000)the ANNs tended to outperform regression (Sargent 2001). The study however could not ruleout the possibility of publication bias.

A common problem in survival modelling is loss to follow-up of the subjects (Lisboa 2002)that could be due to changing address or death from an unrelated cause. These subjects canonly be included during the time period they were observed and omitted (censored) afterwards(Ravdin and Clark 1992). The application of ANNs to censored data provides potentialadvantages over traditional linear models based on standard assumptions, e.g. proportionalhazards. ANNs deal with the censorship issue by including time as one of the covariates.By including each patient only for the time intervals where the outcome is observed andomitting them when it is unknown, the network weights are optimized to a partial log-likelihood. This is described thoroughly in the literature as a partial logistic artificial neuralnetwork (PLANN) model (Biganzoli et al 1998). The combination of ANNs with probabilisticalgorithms has been explored and proved to be a powerful technique in survival predictionstudies (Le Goff et al 2000, Lisboa et al 2003, Sierra and Larranaga 1998). In this paperhowever, we describe a new method for combining ANNs with Bayes theorem to modelsurvival from intraocular melanoma.

Intraocular melanoma is a highly malignant tumour, which threatens the patient withirreversible visual deficit, pain, loss of the eye and metastatic death (Damato 2000). Previously,the standard form of treatment was enucleation, whereas today the treatment is usually aimedat conserving the eye with as much vision as possible. This is achieved by a variety oftreatment modalities, which include proton beam radiotherapy, plaque radiotherapy, trans-scleral local resection, trans-retinal local resection (i.e. endoresection) and transpupillarythermotherapy. Approximately 50% of all patients ultimately develop metastatic disease,which is invariably fatal. Features predictive of metastatic disease are large tumourdiameter, ciliary body involvement, epithelioid cell type, presence of closed loops and variouscytogenetic abnormalities, particularly monosomy 3 (Scholes et al 2003).

It is common clinical practice to present survival information to the patient as theprobability of survival at specified times rather than estimating time to death. Therefore,in order to conform to clinical practice, the ANN output was transformed into probabilityfigures. The objective of this study is to create an artificial intelligence (AI) system to predictsurvival in patients with intraocular melanoma to support the clinicians in their diagnosis in

Artificial neural networks in intraocular melanoma 89

Table 1. Time intervals in years since initial diagnosis.

Interval Time (years) Number died

T1 [0–1.5) 66T2 [1.5–2.5) 75T3 [2.5–4) 79T4 [4 –7) 86T5 [7–15] 54

modelling the information buried in their database. The aim was for the system to mimic theexpert’s knowledge and provide consistent prognosis in their absence.

2. Patients and methods

At the initial presentation to the ocular oncology clinic, patients underwent a full ocular andsystemic examination. Tumour dimensions were measured by B-scan ultrasonography. Thetreatment modalities included plaque or proton beam radiotherapy, trans-scleral or trans-retinallocal resection, photocoagulation or transpupillary thermotherapy and enucleation, the lastbeing performed at the oncology centre or at the referring hospital. Histological examinationwas performed by staining paraffin-fixed, wax-embedded sections with haematoxylin andeosin.

Patients in whom the eye was conserved were reviewed within a month of their initialtreatment, then approximately every six months for about five years, and then annually. Thesefollow-up assessments were performed at the oncology centre and at the referring hospital, inan alternating fashion, until the risk of complications was small (i.e. about 1%), when furthermonitoring was performed only at the referring hospital.

The database originated from patients treated in Glasgow and Liverpool, UK, between1969 and 2001 by the same consultant clinician. Patients were included in the study if they werediagnosed as having an intraocular melanoma, clinically or histologically or both. Patientswere excluded if their tumour was entirely extraocular, that is, involving only conjunctiva oreyelids.

The largest basal tumour diameter was categorized as small (i.e. <10 mm), medium (i.e.10–15 mm) and large (i.e. >15 mm) using the same cut-offs as in the TNM clinical classification(Campbell and Sobin 1998). Tumour location was categorized as nasal or temporal and assuperior or inferior, with respect to the fovea. Anterior tumour margin was defined as anterioror posterior with respect to the ora serrata. Histologically, tumours were defined as spindle ormixed/epithelioid. All these parameters were used as inputs to the ANNs.

The clinical and pathological data were stored prospectively in a customized computerizeddatabase. Data were analysed using SPSS (version 10.0). Survival analysis was carried outusing CR and Kaplan–Meier (KM) analyses.

The final observation in the database was 15 years. This time period was divided intofive time intervals each containing a roughly equal number of events. These time intervals areshown in table 1 where the expression [x, y) represents a time period from x years inclusive toy years exclusive. The number of tumour specific deaths against time since initial diagnosisand the KM curves are shown in figure 1.

ANN analysis was carried out using a program written in Matlab (The Mathworks, Inc.,Natick, MA) using the Neural Network Toolbox. For each time interval, a three-layer feed-forward network with one hidden layer was constructed and trained by back propagation.

90 A F G Taktak et al

0

20

40

60

80

0 5 10 15Years

No.

TS

D1

Years151050

Cum

Sur

viva

l

1.0

.8

.6

.4

.2

0.0

(a)

(b)

Figure 1. Intraocular melanoma study: (a) number of tumour specific deaths (TSD1) from initialdiagnosis; (b) the Kaplan–Meier curve for the population included in the study.

The learning rate was varied from 0.01 to 0.1. The output layer contained one node whichgenerated an output value ranging from 0 representing very high chance of survival to 1representing very low chance of survival for that time interval. The ANN architecture wasdetermined experimentally following the guidelines proposed by Demuth and Beale (2001).The hyperbolic tangent and logarithmic sigmoid transfer functions were used for the hiddenand output layers, respectively. The number of nodes in the hidden layer was varied from 1 to10. In each case, a receiver-operating characteristic (ROC) curve was plotted to determine theoptimal number of nodes. An ROC curve is a plot of sensitivity versus 1 − specificity for eachpossible cut-off. The area under the ROC curve (AUROC) is a measure of the model’s abilityto discriminate between two groups whereby an AUROC figure of 0.5 represents effectively nodiscrimination and 1 is maximum discrimination. The maximum number of training epochswas limited to 10 000 with the network output evaluated every 100 to reduce the possibility ofoverfitting.

The records in each time interval were divided into training and test sets. In order toeliminate any bias, the selection of the two groups was random and easily reproducible for thepurpose of double blind analysis. This was achieved by dividing the records based on age atthe time of initial diagnosis. The training set included patients whose age had an odd numberat the time of treatment and tested on patients whose age had an even number.

The ANN output was transformed into a survival function using Bayes theorem asdescribed below. If the ANN score at time interval say [Ti−1, Ti) was above a certain cut-offlevel (�i), this indicated a low chance of survival beyond Ti. If, on the other hand, the ANNscore was <�i, the record was presented to the subsequent network for time [Ti, Ti+1).

Artificial neural networks in intraocular melanoma 91

The probability figures are calculated as follows. Let

di = the number of patients who died from the tumour during the time interval [i − 1, i);ci = the number of patients lost to follow-up or those who died from general causes duringthe time interval [i − 1, i);ni = the total number of patients observed during the time interval [i, i+1);

then P(D)i+1, the probability of death at the end of the time interval [i, i+1), can be calculatedas

P(D)i+1 = 1 − di

ni+1 − ci − di

. (1)

Now let

T | di = the number of patients who died specifically from the tumour and had an ANNscore value � �i;t | di = the number of patients who died specifically from the tumour and had an ANNscore value < �i;T | ni = all the patients who had an ANN score � �i;t | ni = all the patients who had an ANN score < �i.

If S is the syndrome, represented in this case by a high ANN score, the probability of deathgiven syndrome presence P(D|S) is calculated from Bayes as follows:

P(D|S)i = P(D)i × P(S|D)i

P (S)i. (2)

Similarly, the probability of death given no syndrome presence (i.e. low ANN score) P(D|S)is

P(D|S)i = P(D)i × P(S|D)i

P (S)i(3)

where

P(S|D)i = T |di

di

(4)

P(S)i = T |ni

ni

(5)

P(S|D)i = t |di

di

(6)

P(S)i = t |ni

ni

. (7)

For time T = 0, di = ci = 0 and ni = 2331, the total number of records included in the study.The survival function SF can be represented by the following sequence:

SFt =

∣∣∣∣∣∣∣

1 for t = 0

1 − P(D|S)t for t � i − 1 if ANN score for [i − 1, i) � �i

1 − P(D|S̄)t for t < i if ANN score for [i − 1, i) < �i.

(8)

92 A F G Taktak et al

Figure 2. The AI system graphical user interface using a Microsoft Visual Basic client for userinterface and a Matlab engine server for the processing.

3. The AI system

An AI system was created by combining the mathematical power of Matlab with MicrosoftVisual Basic (MS VB) in a user-friendly graphical user interface with MS VB running as aserver to Matlab Active X (figure 2). The system used the combination of ANNs and Bayesas described above and plotted a survival curve. Six prognostic categories were assigned: (1)very poor, (2) poor, (3) poor/medium, (4) medium, (5) good and (other) which contained amixture of patients who survived more than 15 years and unclassified records. Records inthe test set were ranked according to their actual survival time and divided into six categoriesusing the time intervals in table 1. The AI system was then used to predict the category theyfitted in.

To demonstrate the validity of the system, a set of 34 patients who died from metastaticmelanoma unseen to the training or test sets was used. The AI system was compared to thetraditional CR and KM analyses. The differences of mean probability of survival at 5 yearsbetween the AI system and traditional methods were calculated using the validation set.The difference was plotted against the number of cases in the database with matching inputparameters to observe the effect of increasing the number of learning and test sets.

The AI system was also assessed against a clinical expert in a double blind trial usingthe validation set. The differences between each prediction and the actual survival time wereplotted against the survival time to visualize the performance of the clinical expert and the AIsystem in the short, medium and long-term survival.

4. Results

After excluding 263 patients with missing data, a total number of 2331 patients was includedin the study. These comprised 1163 (50%) females and 1168 (50%) males. The median agewas 59 years (SD = 14.7). A total of 360 patients died of metastatic melanoma. The median

Artificial neural networks in intraocular melanoma 93

Table 2. Univariate analysis of the ANN inputs and their predictive powers using CR analysis.

Input parameter Category Code TSD0a (%) TSD1b (%) p-value

Coronal location Nasal 1 31.6 5.4 0.051Midline 2 16.2 2.6Temporal 3 36 8.2

Sagittal location Superior 1 30.2 5.1 0.549Horizontal 2 19.7 3.75Inferior 3 34 7.3

Anterior margin Choroid 0 58.2 7.8 0.000Ciliary body 1 25.6 8.3

Largest basal diameter <10 mm 1 22.3 1 0.00010–15 mm 2 51 10.5

>15 mm 3 10.6 4.6Cell type Spindle 0 62.8 7.5 0.000

Mixed/epithelioid 1 21 8.7

a Alive, lost to follow-up or died of other causes.b Died of the tumour.

Table 3. The optimum number of nodes in the hidden layer for the ANNs and their correspondingAUROC.

ANN Optimum no nodes AUROC

1 9 0.782 4 0.723 4 0.754 7 0.815 5 0.96

time to death was 3.14 years. The 1971 patients who were lost to follow-up or died of othercauses had a median follow-up time of 2.48 years, exceeding 5 years in 319 patients.

Table 2 shows the results of univariate analysis of risk of metastatic death according tocoronal tumour location, sagittal tumour location, anterior tumour margin, largest basal tumourdiameter and the cell type. The table also shows the predictive power of these parametersusing CR analysis.

The number of nodes in the hidden layer for the optimum network in each time intervalis shown in table 3. The table also shows the area under the ROC curves (AUROC) figure forthese networks. Mean and standard deviation values for all five networks are shown in table 4for patients who died from metastatic disease (µTSD1, σ TSD1)i and those who were alive, lostto follow-up or died of general causes (µTSD1, σ TSD1)i for i = 1, . . . , 5. The cut-off value foreach time interval was calculated as

�i = µTSD1,i + σTSD1,i . (9)

The sensitivity, specificity and accuracy of the ANN at that cut-off value are also shown intable 4. A total of six survival curves, one for each category, were constructed as shown infigure 3. The error bars were derived from the accuracy figures in table 4.

Of the 1149 patients included in the test set, 192 died from metastatic melanoma. Theactual survival time for each one of these records was plotted against the category it wasassigned to by the AI system as shown in figure 4. For each category, the 25th, 50th and

94 A F G Taktak et al

0

0.2

0.4

0.6

0.8

1

0 5 10 15Years

Cu

mu

lati

ve S

urv

ival

CAT1 CAT2 CAT3 CAT4 CAT5

Figure 3. Survival curves generated by the AI system combining the ANN output with Bayes.The curve marked CAT 1 represents low chance of survival beyond time T1, CAT 2 is low chanceof survival beyond time T2 and so on.

Figure 4. Survival results of the test set as predicted by the AI system. The vertical axis representsthe actual survival time in years and the horizontal axis represents the results of the AI systemclassified into one of the six categories. The 25th, 50th and 75th centile limits are shown for eachcategory. Note that these limits were not calculated for class 5 as only one result was classified inthis category. The joined circles represent the actuarial median survival time for each category.

Table 4. The ANN output values with mean (µ) and standard deviation (σ ) values for the non-tumour specific (TSD0) and tumour specific (TSD1) deaths. Non-tumour specific deaths includealive patients, patients lost to follow-up and patients who died of other causes. The sensitivity,specificity and accuracy figures are for cut-off values � = µTSD0 + σTSD0.

ANN TSD0 TSD1 Sensitivity Specificity Accuracy

1 µ = 0.0785 µ = 0.2205 0.8968 0.6368 0.7141σ = 0.0948 σ = 0.1455

2 µ = 0.1331 µ = 0.5338 0.873 0.6701 0.7271σ = 0.1778 σ = 0.3031

3 µ = 0.1382 µ = 0.4999 0.8647 0.6608 0.7197σ = 0.1656 σ = 0.2848

4 µ = 0.1242 µ = 0.4403 0.8838 0.6303 0.7082σ = 0.1538 σ = 0.2828

5 µ = 0.1268 µ = 0.3236 0.9488 0.4872 0.6559σ = 0.1205 σ = 0.0978

75th centile limits were calculated. These limits and the actual median survival curve for eachcategory are also shown in figure 4.

Artificial neural networks in intraocular melanoma 95

-80

-70

-60

-50

-40

-30

-20

-10

0

10

20

0 20 40 60 80 100 120 140

Num Samples

Pro

babi

lity

of S

urvi

val a

fter

5 Y

ears

(%

)

Figure 5. Comparison of the AI system with CR and KM analyses in the validation set. Thevertical axis represents the difference in the probability of survival at 5 years and the horizontalaxis represents the number of cases with matching input parameters in the database. The diamondsare the (AI–CR) results with the solid line representing the best fit line and the squares representthe (AI–KM) results with the dashed line being the best fit.

-15.0

-10.0

-5.0

0.0

5.0

10.0

15.0

0.0 5.0 10.0 15.0

Actual (Years)

Exp

ert

- A

ctu

al (

year

s)

Figure 6. Results of survival prediction in the validation set—actual survival plotted against theactual survival time. The full circles represent the AI system’s results and the open circles representthe clinical expert’s results.

The 5 year survival probability figure generated by the AI system was on average 41%lower than that predicted by the CR method and 37% lower than for KM. However, as can beseen from figure 5, when the number of samples increased for a particular input combination,the differences between the AI system and traditional methods decreased. For an inputcombination with more than 100 matching records, the mean differences were <20%. Insome cases, the CR and KM methods failed to provide a result due to an insufficient numberof samples.

The results of the AI system’s performance versus the clinical expert’s prediction areshown in figure 6. The clinical expert was asked to predict the survival time for 34 patients,who died from intraocular melanoma. This was compared with the 25th centile figure for thecategory predicted by the system. The rms error of the clinical expert’s prediction was 4.3years compared to 3.7 years from the AI system.

5. Discussion

Two important predictor factors in survival analysis are age and sex. These two parametershowever introduce a number of biases due to different patterns of withdrawals from the

96 A F G Taktak et al

study for different age and sex groups. There is a sizeable amount in the literature thatdescribes the problem and suggests methods for dealing with this bias (see, for example,Hakulinen et al 1987). In order to examine tumour specific survival however, these twoparameters have been deliberately left out in this study. Our group is currently workingon including these two parameters to study the relative survival rate. The performance ofthe system might be further improved by excluding parameters with poor predictive power.In this study however we have only excluded parameters with large numbers of missingvalues.

Previous studies combining ANNs with Bayes have already been mentioned in theintroduction, to generate probabilistic networks. In those studies, the database was dividedinto a number of prognostic groups and the theorem was used to calculate the probability ofthe input vector belonging to a particular group. In this study however, Bayes theorem wasused for a different purpose. The theorem was used to transform the output of the ANN modelinto a probability figure to present the probability of survival beyond a certain time interval.The choice of the number of time intervals was therefore an important factor in the studydesign. A large number would result in smoother survival curves and less information lossdue to grouping but at the cost of a smaller number of events in each group making it verydifficult to detect an event. This was evident by the low mean figures shown in table 4. Acompromise therefore had to be made between generating acceptable curves and generatingANN outputs that are above the noise level.

There were a group of patients whose risk could not be categorized by the ANNs as shownin figure 4. These might have been patients who had non-metastatic deaths, or whose deathwas predicted by a variable that was not considered.

The date censorship issue introduces further biasing. Patients who are alive, lost tofollow-up or died of other causes are only included during the time period they wereobserved. Patients who die of the melanoma, on the other hand, can either be includedfor the whole study period or up to time of death. The former solution introduces a biastowards death which has to be rectified by deleting some records at random to match theoriginal database as suggested by Ravdin and Clark (1992). From our experience however,this was found to generate poor results of the networks’ ability to generalize due to loss ofvital information by deleting records with rare combinations. The latter solution was thereforeadopted. This was found to produce better results but might also be the cause of havinga relatively large number of unclassified results. Our future work will aim to address thisproblem.

A study in the use of computer-decision support systems in cancer showed that a significantnumber of clinicians preferred such systems to more traditional methods due to provisionof evidence and explanations to support advice (Emery et al 2000). Such techniques alsoprovide consistency and are not subject to variability introduced by human factors. The roleof the AI system in our study was to support the clinicians in their diagnosis in modellingthe information buried in the database. It was designed to mimic the expert’s knowledgeand provide consistent prognosis in their absence. It was therefore initially hoped that thetechnique would be at least able to match the performance of the clinical expert in predictingsurvival. In fact, the technique exceeded expectations by outperforming the expert especiallyin the short and medium terms. It has been suggested in the literature that cancer survivalmodels are only efficient for 10 years (see, for example, Moshari et al 2001). Taking thisinto account, the rms error figures were re-calculated ignoring cases with >10 years survival.This presented major improvement in the AI system’s performance where the rms error was2.7 years whereas in the clinical expert’s prediction there was no major improvement with anrms of 4.2 years.

Artificial neural networks in intraocular melanoma 97

6. Conclusion

This study showed that combining neural networks with Bayes theorem provided goodrepresentation of survival function in intraocular melanoma, predicting probability of survivalat specified times rather than estimating time to death, hence conforming to usual clinicalpractice. The technique compared well with the traditional methods such as CR and KManalyses when the sample number was large. The technique however must be used withcaution due to biasing issues as outlined in the discussion. Users of such techniques arereferred to the literature describing these issues in detail, such as Ripley and Ripley (2001).

The main strength of this study is that it can provide a valuable contribution in predictingoutcomes for rare combinations of inputs where applying traditional methods is not feasible.It can also help in predicting outcomes for combinations not previously seen due to the abilityof the ANNs to generalize. The main weakness of this study is that for small numbers ofsamples, where the differences between ANN models and statistical models were large, it isnot known which model is more accurate.

This work has shown that there is a huge potential for AI techniques in modelling survivalprediction in cancer. Current developments in data mining, knowledge discovery and otherAI approaches require large computer resources. With the rapid development in computerscience and technology, there are vast resources yet to be tapped.

References

Biganzoli E, Boracchi O, Mariani L and Marubini E 1998 Feed forward neural networks for the analysis of censoredsurvival data: a partial logistic regression approach Stat. Med. 17 1169–86

Campbell R J and Sobin L H 1998 Histological Typing of Tumours of the Eye and its Adnexa 2nd edn (London:Springer) pp 42–3

Cintin C, Johansen J S, Christensen I J, Price P A, Sorensen S and Nielsen H J 2002 High serum YKL-40 level aftersurgery for colorectal carcinoma is related to short survival Cancer 15 267–74

Damato B 2000 Ocular Tumours: Diagnosis and Treatment (Oxford: Butterworth-Heinemann) chapter 7 pp 57–93Demuth H and Beale M 2001 Neural Network Toolbox: User’s Guide Version 4 vol 5 (Natick, MA: The MathWorks,

Inc.) pp 41–2Emery J, Walton R, Murphy M, Austoker J, Yudkin P, Chapman C, Coulson A, Glasspool D and Fox J 2000 Computer

support for interpreting family histories of breast and ovarian cancer in primary care: comparative study withsimulated cases Br. Med. J. 321 28–32

Frize M, Ennett C M, Stevenson M and Trigg H C E 2001 Clinical decision support systems for intensive care units:using artificial neural networks Med. Eng. Phys. 23 217–25

Gamel J W and Jones A S 1993 Squamous carcinoma of the head and neck: cured fraction and median survival timeas functions of age, sex, histologic type, and node status Br. J. Cancer 67 1071–5

Hakulinen T, Tenkanen L, Abeywickrama K and Paivarinta L 1987 Testing equality of relative survival patterns basedon aggregated data Biometrics 43 313–25

Le Goff J M, Lavayssiere L, Rouesse J and Spyratos F 2000 Nonlinear discriminant analysis and prognostic factorclassification in node-negative primary breast cancer using probabilistic neural networks Anticancer Res. 20(Suppl. 3B) 2213–8

Lisboa P J G 2002 A review of evidence health benefit from artificial neural networks in modern intervention NeuralNetw. 15 11–39

Lisboa P J, Wong H, Harris P and Swindell R 2003 A Bayesian neural network approach for modelling censored datawith an application to prognosis after surgery for breast cancer Artif. Intell. Med. 28 1–25

Moshari A, Cheeseman E W and McLean I W 2001 Totally necrotic choroidal and ciliary body melanomas:associations with prognosis, episcleritis, and scleritis Am. J. Ophthalmol. 131 232–6

Naguib R N G and Sherbet G V 2001 Artificial Neural Networks in Cancer Diagnosis, Prognosis, and PatientManagement (Boca Raton, FL: CRC Press)

Pollack A, Zagars G K, Starkschall G, Antolak J A, Lee J J, Huang E, von Eschenbach A C, Kuban D A andRosen I 2002 Prostate cancer radiation dose response: results from the M.D. Anderson phase III randomizedtrial Int. J. Radiat. Oncol. Biol. Phys. 53 1097–105

98 A F G Taktak et al

Ravdin P M and Clark G M 1992 A practical application of neural network analysis for predicting outcome ofindividual breast cancer patients Breast Cancer Res. Treat. 22 285–93

Richter E N, Oevermann K, Buentig N, Storkel S, Dallman I and Atzpodien J 2002 Primary apoptosis as a prognosticindex classification of metastatic renal cell carcinoma J. Urol. 168 460–4

Ripley B D and Ripley R M 2001 Neural networks as statistical methods in survival analysis Clinical Applications ofArtificial Neural Networks ed R Dybowski and V Gant (Cambridge: Cambridge University Press) pp 237–55

Sargent D J 2001 Comparison of artificial neural networks with other statistical approaches Cancer 91 1636–42Scholes A G, Damato B E, Nunn J, Hiscott P, Grierson I and Field J K 2003 Monosomy 3 in uveal melanoma:

correlation with clinical and histologic predictors of survival Invest. Ophthalmol. Vis. Sci. 44 1008–11Seagard S and Kock E 1995 Prognostic indicators following enucleation for posterior uveal melanoma. A multivariate

analysis for long-term survival with minimised loss to follow-up Acta Ophthalmol. Scand. 73 340–4Shields C L 2000 Conjunctival melanoma: risk factors for recurrence, exenteration, metastasis, and death in 150

consecutive patients Trans. Am. Ophthalmol. Soc. 98 471–92Sierra B and Larranaga P 1998 Predicting survival in malignant skin melanoma using Bayesian networks automatically

induced by genetic algorithms. An empirical comparison between different approaches Artif. Intell. Med. 14215–30