Bankruptcy Prediction with Soft Computingscc/DSML/Bankruptcy... · Bankruptcy Prediction with Soft Computing Prof. V. Ravi Head, Center of Excellence in Analytics IDRBT, Hyderabad

Bankruptcy Prediction with Soft Computing

Prof. V. Ravi

Head, Center of Excellence in Analytics

IDRBT, Hyderabad

[email protected]

Outline • About IDRBT & Center of Excellence

• Introduction to Bankruptcy Prediction

• Differential evolution trained wavelet neural networks

• Differential evolution trained Radial Basis Function Network

• Support Vector-WNN hybrid

• Conclusions

About IDRBT

07/04/2017 3

• Set up by RBI in 1996 • Autonomous R&D Institute • Research, Teaching (M.Tech (IT), Ph.D.), Training and Consultancy

Broad Research Areas

• Networks, Grid, Cloud, Virtualization, Social

Media, Wireless Networks, Internet, Web

Technologies, INFINET etc.

Financial Networks and

Applications

• E-Payments, Internet Banking, Mobile Payments,

ATM, PoS, Cash Dispensers, Smart cards, CTS,

MICR, RTGS, NEFT, IMPS etc.

Electronic Payment and

Settlement systems

•Security Models, Bio-metrics, Access Control,

Information Security, Digital Forensics, Cryptology,

Steganography, Image water marking, Cyber

Frauds and Crimes, Ethical Hacking, Digital

forensics, Controls and Standards etc.

Security Technologies for the

Financial sector

• Data Warehousing, Data Mining, CRM, Big Data,

Soft Computing, Financial Engineering, Risk

Management, Software Engineering,

Optimization, IT Management, e-Governance etc.

Financial Information

Systems and Business

Intelligence

• Soft Computing Hybrids Developed Support Vector-Wavelet Neural Network (SV-WNN)

Nonlinear PCA-Threshold Accepting based Logit (NLPCA-TALR)

Differential Evolution trained RBF network (DERBF)

*Differential Evolution Threshold Accepting hybrid optimization algorithm (DETA)

*Differential Evolution trained WNN (DEWNN)

WNN-Fuzzy Rule based Classifier (WNN-FRBC)

*Data Envelopment Analysis-Fuzzy Multi Attribute Decision Making (DEA-FMADM)

Boosting involving CART, SVM and MLP

Threshold Accepting trained WNN (TAWNN)

Research Contributions of CoE to Computer Science

*Threshold Accepting trained Principal Component NN (TAPCNN)

Ensembling MLP, RBF, MARS, SVM and CART *PCA-PNN and ensembling several techniques Modified Greta Deluge Algorithm trained Auto

Associative Network (MGDAAANN) Threshold Accepting based Fuzzy Clustering (TAFC) Improved Differential Evolution (DE-NM-Simplex) SVM-Naïve Bayes Tree (SVM-NBTree) Recurrent Genetic Programming (RGP) TA trained Kernel Principal Component NN (TA-KPCNN) DERBF-Genetic Algorithm Tree (DERBF-GATree) Ant Colony Optimization-Nelder-Mead Simplex (ACONM) SVM-FRBC

Research Contributions of CoE to Computer Science

• Forecasting Software Development Cost • Forecasting Software Reliability • Fast digital Watermark retrieval using Hopfield

Neural Network • Watermark retrieval using Evolutionary

Algorithms • DE-KPCWNN & DE-KBQR • Recurrent GP and recurrent GMDH • PSOAANN for variety of tasks • Firefly Miner • Novel Time Series Mining algorithms

Research Contributions of CoE to Computer Science

• Bankruptcy Prediction in Banks • Analytical CRM in Banking

• Customer Churn Prediction in Credit Cards/CASA • Credit Scoring • Default Prediction • Fraud Detection in Insurance • Customer Lifetime Value Modeling • Data Quality/Imputation in Customer datasets • Privacy Preserving Data Mining • Data Mining Unbalanced data sets

• Fraud Detection in Accounting Statements • Forex Rate prediction • Cash Demand Forecasting in ATMs • Association Rule Mining in Banks using PSO • Profiling of Internet/ Mobile banking users in India • Ranking Indian PSU banks’ Productivity • Predicting Operational Risk from Software perspective • Fuzzy Optimization of Asset Liability Management (ALM), 2007

Research Contributions of CoE to Banking

International Collaborations • Prof. Kalyanmoy Deb, Dept of Elec & Comp Engg, MSU, USA

• Prof. Dirk Van den Poel, Dept of Marketing, University of Ghent,

Ghent, Belgium

• Prof. Anita Prinzie, Dept of Marketing, Univ. of Ghent, Belgium

• Prof. Venu Govindaraju, SUNY Buffalo, USA • Prof. Ajith Abraham, Director, Mir Labs, USA • Prof. Indranil Bose, Business School, University of Hong Kong- now

with IIM Calcutta • Prof. D. Nagesh Kumar, IISc, Bangalore • Prof. Nik Kasabov, Director KEDRI, Auckland Univ.of Technology,

New Zealand.

Research Output

• Published – 150 papers – One edited book – One edited Conference proceedings (FANNCO 2015)

• Better content for CRM EDP programmes – Customized Training to 15 banks – POC conducted on Analytical CRM for 15 banks – Framework on “Holistic CRM and Analytics” , “Data Quality” and

“Digital Banking” developed

• Proof of the concept software

– Data imputation – RUIBA

Research Supervision • Ph.D.

– Graduated • Mr. Mohammad Abdul Haque Farquad (UoH)

– Rule extraction from Support Vector Machine: Applications to Banking and Finance

• Mr. R. Mohanty (Berhampur Univ, Orissa)

– Application of Machine Learning and Soft Computing to Software Engineering

• Mr. N. Naveen (UoH)

– Rule extraction from Neural Nets & Optimization Techniques

– Ongoing • Mr. D. Pradip Kumar (Time Series Data Mining)

• Mr. B. Shravan Kumar (Unstructured Data Mining)

• Mr. K. Ravi (Social Media Analytics and Big Data with Ontologies)

• Mr. G. Jayakrishna (Evolutionary Computing and Data Mining)

• Mr. S. K. Kamruddin (Big Data and Applications)

Research Supervision

• M. Tech (IT)Projects in Soft Computing/Data Mining – More than 35 Projects

– One student won Best Project Award at UoH in 2007.

– One student won IDRBT Award in 2011; 2 in 2012

– One student did Ph.D. in NJIT and one in SUNY, Stonybrook, USA.

– One won Best Paper Award at ICCIC 2013

• Integrated B. Tech, M. Tech/M. Sc from IITs – More than 25 students for summer projects

– One won Best paper Award at MIWAI 2014

Soft Computing/CI Constituents

It comprises intelligent technologies

– Fuzzy Computing

– Neuro Computing

– Evolutionary Computing

– Rough Set theory

– Chaos theory

– Machine Learning

– Probabilistic reasoning (Bayesian Belief Nets)

• SC solutions hybridize two or more of these technologies in

various permutations and combinations

Definition of Soft Computing

• Soft computing differs from conventional (hard) computing in that it is tolerant of imprecision, uncertainty, partial truth and approximation. In effect, the role model for soft computing is the human mind.

- Lotfi Zadeh (1992)

Benefits of Soft Computing

An ultimate goal shared by AI and SC

– the creation and understanding of machine intelligence

Soft Computing (or Computational intelligence)

– For learning and adaptation, SC requires extensive

computation but does not perform much symbolic

manipulation. So it is also called Computational Intelligence

— a discipline that complements classical AI approaches.

Benefits of Soft Computing

In SC paradigm, one can simultaneously

– incorporate and process human knowledge effectively

– deal with imprecision and uncertainty

– learn to adapt to unknown or changing environment for better performance

– Amplify the advantages of the component technologies while nullifying their disadvantages

Guiding principle of Soft Computing

• Exploit

– the tolerance for imprecision, uncertainty, partial truth, and approximation

• to achieve

– tractable, robust and low cost solution.

Data Mining

• The non-trivial process of extracting USEFUL, NON-OBVIOUS AND ACTIONABLE knowledge from huge masses of data.

• A consortium of techniques – Computer Science

– Statistics

– Operations Research

– Data bases and

– Artificial Intelligence

Introduction to DM- Dr. V. Ravi 20/49

Bankruptcy Predn-V. Ravi

Overview of Bankruptcy

Bankruptcy of banks and financial firms is well researched area since 1960s

This is considered a form of Operational Risk,

Some consider this as a fallout of Credit risk

Creditors, auditors, stockholders and senior management are all interested in knowing about bankruptcy as it affects all of them.

Bankruptcy Predn-V. Ravi

Terminology

• The terms failure, insolvency and bankruptcy are used interchangeably. Altman (1983) distinguishes the terms as follows.

• In an economic sense, failure means that the realized rate of return on invested capital, with allowances for risk considerations, is significantly and continually lower than prevailing rates on similar investments. Thus, a company may be an economic failure for many years.

• Insolvency exists when a firm cannot meet its current obligations.

• Bankruptcy occurs when a company files a formal legal document in a federal district court for the purpose of either liquidation or reorganization.

Bankruptcy Predn-V. Ravi

Overview of Bankruptcy

The numerous factors that could lead to companies and banks going bankrupt can be broadly classified into three main categories:

Economic factors: GDP slowing, inflation, interest rate, unemployment rate, recession, depression etc…

Industry factors: Competition, growth/decline rate, profit margin trends, government regulations, trade barriers, import tariff and quotas, taxes etc…

Bankruptcy Predn-V. Ravi

Overview of Bankruptcy

Company factors: Management quality, capital allocation, competitive advantage, operation efficiency, working capital management, inventory management etc...

During the period of 1974-1994, one third of all commercial banks in U.S. have collapsed due to failures and mergers.

Overview of Bankruptcy

According to US Federal Deposit Insurance Corporation Improvement Act 1991, on-site health examination of banks by regulators every 12-18 months was made mandatory

Each bank was given a CAMELS rating to quantify its financial health

– Capital adequacy

– Asset quality

– Management expertise

– Earning strength

– Liquidity

– Sensitivity to Market Risk

Bankruptcy Predn-V. Ravi

Problems with CAMEL ratings

Cole and Gunter (1995) found that the effectiveness of CAMEL rating of troubled banks began to decay as quickly as six months.

On-site financial health monitoring period (12-18 months) is too long to anticipate impending financial problems of banks

Financial experts are scarce and expensive resources.

Bankruptcy Predn-V. Ravi

Overview of Bankruptcy

Hence, off-line methods preferred. More effective methods required to enable regular monitoring of bank’s financial health and advance detection of impending financial troubles

More efficient and cost effective method using computerized systems required

Differential evolution trained wavelet neural networks: Application to bankruptcy prediction in banks

Nikunj Chauhan, V. Ravi *, D. Karthik Chandra

Expert Systems with Applications (2009)

Doi:10.1016/j.eswa.2008.09.019

Introduction • Differential evolution algorithm (DE) is proposed to train a

wavelet neural network (WNN).

• The resulting network is named as differential evolution trained wavelet neural network (DEWNN).

• The efficacy of DEWNN is tested on bankruptcy prediction datasets.

• The whole experimentation is conducted using 10-fold cross validation method.

Wavelet neural networks

• Based on the locally supported basis functions such as Radial Basis Function Networks (RBFNs),a class of neural net works called WNN, which originate from wavelet decomposition in signal processing, have become more popular recently .

• A family of wavelets can be constructed from a function w(x), sometimes known as a ‘‘mother wavelet,’’ which is confined in a finite interval. ‘‘Daughter wavelets’’ u(a,b) (x) are then formed by using translation(b) and dilation (a) parameters. An individual wavelet is:

Wavelet Neural Network (WNN) • Has one input, hidden and output layer each. • All nodes in each layer are fully connected to the nodes

in the next layer. • The output layer contains a single node. • Based on the activation functions (either Gaussian or

Morlet) used in hidden nodes, two variants of WNN are implemented.

WNN Algorithm 1. Select the number of hidden nodes required.

2. Initialize randomly from Uniform (0,1)

i. the dilation and translation parameters for these nodes

ii. the weights for the connections between the input and hidden layer; for the connections between the hidden and output layer.

3. The output of the sample Vk, k=1, . . ., np, where np is the number of samples, is calculated with the following formula:

3. Reduce the error of prediction by adjusting Wj, wij, aj, bj using ΔWj, Δwij, Δaj, Δbj (see below). In the WNN, the gradient descend algorithm is employed.

where the error function E is taken as normalized root mean squared deviation (NRMSE) as follows:

4. Return to step (2), the process is continued until convergence, and the whole training of the WNN is completed.

• Threshold accepting trained WNN (TAWNN):

– Threshold Accepting algorithm, originally proposed by Dueck and Scheuer (1990) is a faster variant of the original simulated annealing algorithm wherein the acceptance of a new move or solution is determined by a deterministic criterion rather than a probabilistic one.

– We used TA to determine the weights of WNN

Metaheuristics used to train WNN

Differential evolution

• Differential Evolution (DE) is a novel approach in evolutionary algorithms. DE algorithm consists mainly of four steps: initialization, mutation, recombination and selection.

• In a population of solutions within an n-dimensional search space, a fixed number of vectors are randomly initialized, then evolved over time to explore the search space and to locate the minima of the objective function. The objective function is here to minimize the error value.

Differential evolution based WNN (DEWNN)

• Application of DE in training WNN basically modifies steps 3 and 4 of the WNN training algorithm for WNN described.

• Weights W, dilation parameters D, translation parameters T and input values X, i.e. Y = f(X,R), where Y is the output values vector and R = (D,T,W,w).

• Vector R consists of (i) Weight values from input nodes to hidden nodes W= {Wij, i = 1,2,. .

., nin, where nin = number of input nodes, j = 1,2, . . .,nhn, where nhn = number of hidden nodes}

(ii) Weight values from hidden nodes to output nodes w = {wjk, j = 1,2, . . .,nhn and k = 1,2,. . .,non, where non = number of output nodes}

(iii) Dilation parameters D = (d1,d2, . . .,dnhn) (iv) Translation parameter T = (t1, t2, . . ., tnhn)

Flow Chart of Differential evolution

Feature selection

Feature selection is a process by which samples in the measurement space is described by a finite and usually smaller set of features.

• Absolute value of each hidden-to-output weight wko, is incorporated into the input to- hidden weights Wij using the following expression:

• For each hidden node j, the sum of weights over all input nodes is equal to the hidden-to-output node weight wjo.

• For each input node, the adjusted weights W ij are summed over all hidden nodes and converted to a percentage of the total for all input nodes.

Literature review • The prediction of bankruptcy for financial firms

especially banks has been the extensively researched area since late 1960s by Altman (1968).

• Bankruptcy prediction research has attracted both statisticians and computer scientists with the result that a number of statistical techniques and more sophisticated non parametric methods like neural networks are applied to solve this problem.

• Bankruptcy prediction problem can also be solved using various other types of classifiers such as case based reasoning, rough sets, support vector machines, case based reasoning, neural network and discriminant analysis and data envelopment analysis[23]to mention a few

Bankruptcy prediction

• To solve bankruptcy prediction problems, Ravi and Pramodh (2008) proposed a threshold accepting based training algorithm for a novel principal component neural network (PCNN), without a formal hidden layer.

• They employed PCNN for bankruptcy prediction problems and reported that PCNN outperformed BPNN, TANN, PCA-BPNN and PCA-TANN in terms of area under receiver operating characteristic curve (AUC) criterion.

Results and discussion

The datasets analyzed by us pertain to three Turkish Banks, Spanish Banks and US Banks datasets and three other benchmark datasets viz., Iris data, wine data and Wisconsin breast cancer data.

Financial ratios of the datasets and the selected features

Average results for 10-fold cross validation with all features

Table-2

Average results of 10FCV for Benchmark datasets with all features

Table-3

Average results for 10FCV with reduced features

Table- 4

The results indicate the overwhelming supremacy of DEWNN in accuracy and sensitivity as compared to TAWNN and original WNN. The results for other benchmark datasets i.e. wine data and Wisconsin breast cancer data with reduced features.

DEWNN once again outperformed the other algorithms. In this case also the robustness of the algorithm is proved and the high accuracies show us the impeccable feature selection done by incorporating Garson’s algorithm into DEWNN

Average results for 10FCV for benchmark datasets with reduced features

Table- 5

• It is concluded that besides being robust, DEWNN is an effective algorithm for solving classification problems occurring in finance.

Comparison of features selected by different techniques

Table- 6

Conclusions • DEWNN, TAWNN are developed and compared

with and the original WNN on benchmark datasets and the results indicate that DEWNN can be a very effective soft computing tool for classification problems.

• In addition, we also adopted the Garson’s feature selection algorithm to WNN, DEWNN and TAWNN the superior performance of DEWNN as compared to TAWNN and the original WNN.

• It is concluded that training WNN with DE solves classification problems with higher accuracy.

•Results show that soft computing hybrids viz., DEWNN and TAWNN outperformed the original WNN in terms of accuracy and sensitivity across all problems. •Furthermore, DEWNN outscored TAWNN in terms of accuracy and sensitivity across all problems except Turkish banks dataset.

References • Dimoulas, C., Kalliris, G., Papanikolaou, G., Petridis, V.,

& Kalampakas, A. (2008).

• Bowel-sound pattern analysis using wavelets and neural networks with application to long-term, unsupervised, gastrointestinal motility monitoring. Expert Systems with Applications, 34, 26–41.

• Dong, L., Xiao, D., Liang, Y., & Liu, Y. (2008). Rough set and fuzzy wavelet neural network integrated with least square weighted fusion algorithm based fault diagnosis research for power transformers. Electric Power Systems Research, 78, 129–136.

• Dueck, G., & Scheuer, T. (1990). Threshold accepting: A general purpose optimization algorithm appearing superior to simulated annealing. Journal of Computational Physics, 90, 161–175.

• Garson, D. G. (1991). Interpreting neural-network connection weights. A! Expert, 47–51. April.

• Grossmann, A., & Morlet, J. (1984). Decomposition of Hardi functions into square integrable wavelets of constant shape. SIAM Journal of Mathematical Analysis, 15, 725–736.

• Guyon, B., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157–1182.

• Ilonen, J., Kamarainen, J.-K., & Lampinen, J. (2003). Differential evolution training algorithm for feed-forward neural networks. Neural Processing Letters, 17(1), 93–105.

• Altman, E. (1968). Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. Journal of Finance, 23, 589–609.

• Avci, E. (2007). An expert system based on wavelet neural network-adaptive norm entropy for scale invariant texture classification. Expert Systems with Applications, 32, 919–926.

• Becerra, V. M., Galvao, R. K. H., & Abou-Seads, M. (2005). Neural and wavelet network model for financial distress classification. Data Mining and Knowledge Discovery, 11, 35–55.

• Bhat, T. R., Venkataramani, D., Ravi, V., & Murty, C. V. S. (2006). Improved differential evolution method for efficient parameter estimation in biofilter modeling. Biochemical Engineering Journal, 28, 167–176.

• Canbas, S., Caubak, B., & Kilic, S. B. (2005). Prediction of commercial bank failure via multivariate statistical analysis of financial structures: The Turkish case. European Journal of Operational Research, 166, 528–546.

• Cheng, C. B., a radial basis function network with logit analysis learChen, C. L., & Fu, C. J. (2006). Financial distress prediction by ning. Computers and Mathematics with Applications, 51, 579–588.

• Cielen, A., Peeters, L., & Vanhoof, K. (2004). Bankruptcy prediction using a data envelopment analysis. European Journal of Operational Research, 154, 526–532.

• Cole, R., & Gunther, J. (1995). A CAMEL rating’s shelf life. Federal Reserve Bank of Dallas Review, 13–20. December.

• Deb, K. (2000). An efficient constraint handling method for genetic algorithms. Computer Methods in Applied Mechanics and Engineering, 186, 311–338.

DE trained RBF (DERBF)

Results of DERBF on benchmark datasets

Flowchart for differential Evolution

SVM-WNN hybrid (SVWNN)

4 SVWNN hybrid system

The hybrid system that involves SVM and WNN working in tandem. In the proposed hybrid system, first support vectors are extracted from SVM and instead of the whole set of samples for training, only support vectors and the corresponding actual output value are fed to the WNN.

7 Results and Discussions • In the case of Spanish banks, the average results for all the

classifiers across ten-folds are presented in Table 2. It may be noted that though FRBC outperformed all the other classifiers, SVWNN did fairly well with AUC of 9,008.5 and accuracy of 90%. SVWNN stood fifth in terms of AUC in comparison to all other classifiers.

• In the case of Turkish banks, the average results for all the classifiers across ten-folds are presented in Table 3. Here, TAWNN outperformed all the classifiers with AUC 10,000 and accuracy of 100%. SVWNN, in the case of Turkish banks too, performed well with an accuracy of 95% and AUC of 9,750, thereby, standing third among the pool of classifiers.

• In case of UK banks, the average results for different classifiers across ten-folds are presented in Table 4. Here, our hybrid SVWNN outperformed all the other classifiers with an accuracy of 81.67% and AUC of 7,900, followed by TANN with an accuracy of 78% and AUC of 7,791.5.

Datasets description and experimental setup

• The datasets analyzed by us in this work are three different bankruptcy prediction data sets viz. Turkish Banks, Spanish Banks and US Banks datasets; German and UK Credit datasets.

• Throughout this study, we performed the 10-foldcross validation method of testing. The results presented in the tables reflect the average results over the 10 folds.