Диссертация (Рандомизированные алгоритмы на основе интервальных узорных структур)

NATIONAL RESEARCH UNIVERSITY HIGHER SCHOOL OF ECONOMICSas a manuscriptMasyutin AlexeyINTERVAL PATTERN STRUCTURES RANDOMIZEDALGORITHMSFOR CLASSIFICATION AND REGRESSION TASKSIN CREDIT RISK MANAGEMENTPhD Dissertationfor the purpose of obtainingPhilosophy Doctor in Computer Science HSEAcademic supervisor:Doctor of Science, Prof. Sergei O.KuznetsovMoscow, 2018Contents1234Introduction . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . .31.1Ph.D. Thesis Relevance . . . . . . . . . . . . . . . . . . .3Overview of Data Analysis Methods in Commercial Banks . . . . .62.1Mathematical Modeling in Commercial Banks. . . . . . . .62.2Neural Networks in Credit Scoring . . . . . . . .

. . . . . .102.3Classification Tasks in Marketing Campaign Management .132.4Loan Default Prediction in Banking: Scorecards . . . . . .15Formal Concept Analysis in Classification Problem . . . . . . . . .183.1Formal Concept Analysis . . . . . . . . . . . . . . . . . . .203.2Lazy Classification with Pattern Structures . . . . . . .

. .213.3Query-Based Classification Algorithm . . . . . . . . . . . .223.4Voting Schemes . . . . . . . . . . . . . . . . . . . . . . . .253.5Experiments with Top-10 Bank Data . . . . . . . . . . . . .313.6Experiments with open data . . . . . . . . .

. . . . . . . .393.7QBCA. Alternative approaches . . . . . . . . . . . . . . . .453.8Interpretability: visualization of premises . . . . . . . . . .493.9Computational time analysis . . . . . . . . . . . . . . . . .59FCA in regression problem . . . . . . .

. . . . . . . . . . . . . . .644.1Problem description . . . . . . . . . . . . . . . . . . . . .644.2Augmented interval pattern structures . . . . . . . . . . . .644.3Query-based regression algorithm with continuous target at-4.4tribute . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .66Data and experiments . . . . . . . . . . . . . . . . . . . . .6815Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .746Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . .75References . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .76Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86211.1IntroductionPh.D. Thesis RelevanceAlthough the recent biggest bank failures are mostly viewed from perspectiveof inability to predict market key factors and lack of banking regulation (financialcrisis 2007-’08) the history knows a number of failures driven by purely credit riskmismanagement. For instance, Long-Term Credit Bank of Japan was one of the topthree banks in Japan responsible for postwar economic growth and in 1989 it wasconsidered the 9th largest company in the world by asset value.

At the time LTCBhad more than $19.2 billion in bad debt. In 1998, the Japanese government nationalized LTCB, then restructured it as a commercial bank named Shinsei Bank. TheBank of New England (BNE), along with its two sister banks, Maine National Bankand Connecticut Bank and Trust, failed on January 6, 1991. BNE was the largestbank in the New England area. With its sister banks, it had assets totaling $21.8billion and deposits totaling $19 billion. Bad loans led to its downfall.

Los Angelesbased IndyMac used to be the largest loan originator in the USA. Founded in 1995as Countrywide Mortgage Investment, IndyMac fueled its aggressive growth throughrisky loan products like Alt-A mortgages, concentrating on inflated real estate markets like California and Florida, and relying heavily on borrowed funds, especiallyfrom the FHLB (Federal Home Loan Bank). As of July 2008, IndyMac had totalassets of $32.01 billion. Moreover, the financial crisis of 2007-2008 in its basis wasactually led by inappropriate credit risk assessment of mortgage loans.As far as Russian financial market is concerned within 2016-2017 several banksshowed inability to control loan portfolio quality [86], [87], [88]. Central Bank ofRussia strengthened its focus on banks assets quality control and implemented itsown management teams within problem banks executive boards. To considerablepart instability is caused by inconsistent risk management which leads to sufficientlosses.

The greatest part of loss in Russian banks ( 70%) is due to credit risk. Creditrisk is risk of the borrowers are not going to repay the granted loan amount in time.The first step to manage credit risk is ability to assess it. In context of credit risk3assessment there are three key parameters: probability of default (PD), loss givendefault (LGD), exposure at default (EAD) [89]. Multiplied all together they providean estimate for expected loss (EL).

Majority of decisions in credit process, such aswhether to grant a loan, sell the loan, initiate legal bankruptcy procedure, are madebased on expected loss estimates.Mathematical models have been widely used in order to make precise predictions on the level of PD, EAD, LGD [1]. Models are usually calibrated on historical data on borrower performance. From data science standpoint PD estimation isa binary classification problem, EAD and LGD estimation is regression problems.As banking industry has begun to be more and more regulated, the requirements onmathematical models development and validation have become more strict and detailed [89].One of the serious trade-offs in credit risk modeling is accuracy of predictionversus model interpretability.

As it will be later shown, some regulators require banksto be able to provide reject reasons for borrowers and also when central banks examine the bank models they are likely to understand economic intuition behind themto prove the models are going to show expected and stable performance. This canbe typically solved given the model is interpretable. At the same time interpretabledata analysis algorithms usually belong to the simplest class such as logistic regression or decision trees which not always can provide the desired accuracy. We willmake an overview of more complex algorithms applications such as neural networksin credit scoring which are capable of describing non-linear interdependencies withinthe data but cannot provide the bank with a reason for reject or acceptance of a loanapplication.As we stated, accurate credit risk estimation is the key tool for risk managementand banks obviously are eager to increase accuracy of algorithms, but keeping theminterpretable.The relevance of this Ph.D.

thesis is that it offers data analysis algorithms thathave accuracy superior to simple algorithms widely adopted within the banks (suchas logisitic regression, decision trees and scorecards) and still maintain the propertyof interpretability in sense that they provide a decision maker with a set of rules4applicable to the borrower creditworthiness assessment.In order to achieve this goal several novelties within the methods of formal concept analysis (FCA) and interval pattern structures ([51]) were introduced.

The reasons why FCA methods are suitable for credit risk assessment under the interpretability requirements will be explained in following sections.The novelty brought to the well developed tools of FCA consists of two parts.The first one is that FCA is adopted to classification problem based on numerical datawith the step of concept lattice construction being omitted (query-based classificationor "lazy" classification). This allows one to work with the datasets with arbitrarynumber of observations which is vital for banks as soon as historical data is typicallylarge.The second is that we introduce a modification to FCA method based on intervalpattern structures which allows one to solve regression problem.

To our knowledgeit is FCA methods were not applicable to such type of data analysis problem before.The crucial difference in regression problem is that the target variable is distributedcontinuously.The goal of this thesis is to provide PD and LGD algorithms of estimation keeping them interpretable. At the same time methods should provide higher accuracythan basic wide spread algorithms in banking industry (such as logistic regression,decision trees and scorecards). One also should note that PD and LGD are the maindrivers of EL as soon as EAD is modeled only for revolving loans such as credit cardsand credit tranches [1].The work consists of a general overview of data analysis algorithms and mathematical models in banking, FCA terms definitions, detailed description of FCA andinterval pattern structures modifications, data, benchmarks and experiments resultsand appendix with an overview of programming implementation of discussed algorithms.52Overview of Data Analysis Methods in CommercialBanks2.1Mathematical Modeling in Commercial Banks.Commercial banks find themselves in conditions of growing rigid competitivestruggle which results in a lower margin the banks can possibly earn.

In order toobtain new customers and to increase the clients portfolio, loans are granted to borrowers with a lower solvency, which is known as the problem of adverse selection[12]. On the one hand, this has led to the problem of increased credit risk, and on theother to the problem of attracting and retaining clients.To solve these problems it is necessary to structure, process and extract regularities (knowledge extraction, rule induction) [7] from a huge data flow about theclient’s profile, its transactions, behavior and interests.