# Introduction to Reason Codes

#### Colin Priest, Sergey Yurgenson, Thakur Raj Anand

#### 2017-04-15

A few questions always asked by business leaders after seeing the results of highly accurate machine learning models are as follows
- Are machine learning models interpretable and transparent?
- How can the results of the model be used to develop a business strategy?
- Can the predictions from the model be used to explain to the regulators why something was rejected or accepted based on model prediction?

DataRobot does provide many diagnostics like partial dependence, feature impact, reason codes to answer the above questions and using those diagnostics predictions can be converted to prescriptions for the business. In this vignette we would be covering reason codes. Partial dependence has been covered in detail in the companion vignette “Interpreting Predictive Models Using Partial Dependence Plots”.

## Introduction

The DataRobot modeling engine is a commercial product that supports the rapid development and evaluation of a large number of different predictive models from a single data source. The open-source R package datarobot allows users of the DataRobot modeling engine to interact with it from R, creating new modeling projects, examining model characteristics, and generating predictions from any of these models for a specified dataset. This vignette illustrates how to interact with DataRobot using datarobot package, build models, make prediction using a model and then use reason codes to explain why a model is predicting high or low. Reason codes can be used to answer the questions mentioned earlier.

## Load the useful libraries

Let’s load datarobot and other useful packages

    library(datarobot)
    library(httr)
    library(knitr)
    library(data.table)

## Connecting to DataRobot

To access the DataRobot modeling engine, it is necessary to establish an authenticated connection, which can be done in either of two ways. In both cases, the necessary information is the endpoint - the URL address of the specific DataRobot server being used - and token, a previously validated access token.

token is unique for each DataRobot modeling engine account and can be accessed using DataRobot webapp in Account profile section.

endpoint depends on DataRobot modeling engine implementation (cloud-based, on-prem…) you are using. Contact your DataRobot admin for endpoint to use and to turn on API access to your account. The endpoint for DataRobot cloud accounts is `https://app.datarobot.com/api/v2`

The first access method uses a YAML configuration file with these two elements - labeled token: and endpoint: - located at $HOME/.config/datarobot/drconfig.yaml. If this file exists when the datarobot package is loaded, a connection to the DataRobot modeling engine is automatically established. It is also possible to establish a connection using this YAML file via the ConnectToDataRobot function, by specifying the configPath parameter.

The second method of establishing a connection to the DataRobot modeling engine is to call the function ConnectToDataRobot with the endpoint and token parameters.

    ConnectToDataRobot(endpoint ='https://app.datarobot.com/api/v2', token='dqmtAG9B7pB7wIuxtmQ81s4BF0mWxZOi')

## Data

We would be using a sample dataset related to credit scoring open sourced by LendingClub (https://www.lendingclub.com/). Below is the information related to the variables.

Id | Min. : 1 | 1st Qu.: 2501 | Median : 5000 | Mean : 5000 | 3rd Qu.: 7500 | Max. :10000 | NA
is_bad | Min. :0.0000 | 1st Qu.:0.0000 | Median :0.0000 | Mean :0.1295 | 3rd Qu.:0.0000 | Max. :1.0000 | NA
emp_title | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
emp_length | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
home_ownership | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
annual_inc | Min. : 2000 | 1st Qu.: 40000 | Median : 58000 | Mean : 68203 | 3rd Qu.: 82000 | Max. :900000 | NA’s :1
verification_status | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
pymnt_plan | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
Notes | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
purpose_cat | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
purpose | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
zip_code | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
addr_state | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
debt_to_income | Min. : 0.00 | 1st Qu.: 8.16 | Median :13.41 | Mean :13.34 | 3rd Qu.:18.69 | Max. :29.99 | NA
delinq_2yrs | Min. : 0.0000 | 1st Qu.: 0.0000 | Median : 0.0000 | Mean : 0.1482 | 3rd Qu.: 0.0000 | Max. :11.0000 | NA’s :5
earliest_cr_line | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
inq_last_6mths | Min. : 0.000 | 1st Qu.: 0.000 | Median : 1.000 | Mean : 1.067 | 3rd Qu.: 2.000 | Max. :25.000 | NA’s :5
mths_since_last_delinq | Min. : 0.00 | 1st Qu.: 18.00 | Median : 34.00 | Mean : 35.89 | 3rd Qu.: 53.00 | Max. :120.00 | NA’s :6316
mths_since_last_record | Min. : 0.00 | 1st Qu.: 0.00 | Median : 86.00 | Mean : 61.65 | 3rd Qu.:101.00 | Max. :119.00 | NA’s :9160
open_acc | Min. : 1.000 | 1st Qu.: 6.000 | Median : 9.000 | Mean : 9.335 | 3rd Qu.:12.000 | Max. :39.000 | NA’s :5
pub_rec | Min. :0.00000 | 1st Qu.:0.00000 | Median :0.00000 | Mean :0.06013 | 3rd Qu.:0.00000 | Max. :3.00000 | NA’s :5
revol_bal | Min. : 0 | 1st Qu.: 3524 | Median : 8646 | Mean : 14271 | 3rd Qu.: 16952 | Max. :1207359 | NA
revol_util | Min. : 0.00 | 1st Qu.: 25.00 | Median : 48.70 | Mean : 48.45 | 3rd Qu.: 71.80 | Max. :100.60 | NA’s :26
total_acc | Min. : 1.00 | 1st Qu.:13.00 | Median :20.00 | Mean :22.01 | 3rd Qu.:29.00 | Max. :90.00 | NA’s :5
initial_list_status | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA
collections_12_mths_ex_med | Min. :0 | 1st Qu.:0 | Median :0 | Mean :0 | 3rd Qu.:0 | Max. :0 | NA’s :32
mths_since_last_major_derog | Min. :1.000 | 1st Qu.:1.000 | Median :2.000 | Mean :2.002 | 3rd Qu.:3.000 | Max. :3.000 | NA
policy_code | Length:10000 | Class :character | Mode :character | NA | NA | NA | NA

## Divide data into train and test and setup the project

Let’s divide our data in train and test. We can use train data to create a datarobot project using SetupProject function and test data to make predictions and generate reason codes. Detailed explanation about creating projects was described in the vignette , “Introduction to the DataRobot R Package.” The specific sequence used here was:

    target = "is_bad"
    projectName = "Credit Scoring"
    numWorkers = 20

    set.seed(1111)
    split <- sample(nrow(Lending), round(0.9*nrow(Lending)), replace = FALSE)
    train <- Lending[split,]
    test <- Lending[-split,]

    project <- SetupProject(dataSource = train,
                           projectName = projectName)
    SetTarget(project = project,
              target = target)

Once the modeling process has completed, the GetAllModels function returns an S3 object of class “listOfModels” that characterizes all of the models in a specified DataRobot project. Calling this function before the modeling process is complete causes a partial result to be returned, with a warning; to avoid this problem, the WaitForAutopilot function is used before calling GetAllModels:

    # increase the number of workers used by this project
    UpdateProject(project = project$projectId,
                  workerCount = numWorkers)
    WaitForAutopilot(project, verbosity = 1, timeout = 999999)

    results <- as.data.frame(GetAllModels(project))
    kable(head(results),longtable=TRUE, booktabs=TRUE,row.names = TRUE)

| modelType | expandedModel | modelId | blueprintId | featurelistName | featurelistId | samplePct | validationMetric
--- | --- | --- | --- | --- | --- | --- | --- | ---
1 | Advanced AVG Blender | Advanced AVG Blender | 58d9501cf2ff94714e225839 | 300fb24a331bd7e8313023faf0db0a4d | Informative Features | 58d94dddfbfa573bd3aa3295 | 63.9929 | 0.34306
2 | ENET Blender | ENET Blender | 58d9501cf2ff94714e22583b | 9df9e8c1439604171b9a025faf88aa5a | Informative Features | 58d94dddfbfa573bd3aa3295 | 63.9929 | 0.34310
3 | AVG Blender | AVG Blender | 58d9501bf2ff94714e225837 | e3aadd9331abf90ea27fa28dcbd72fc7 | Informative Features | 58d94dddfbfa573bd3aa3295 | 63.9929 | 0.34570
4 | ENET Blender | ENET Blender | 58d9501cf2ff94714e22583d | 6ed326ca6e31c0ce27aa7729b256e814 | Informative Features | 58d94dddfbfa573bd3aa3295 | 63.9929 | 0.34572
5 | eXtreme Gradient Boosted Trees Classifier with Early Stopping | eXtreme Gradient Boosted Trees Classifier with Early Stopping::Tree-based Algorithm Preprocessing v21 | 58d94ec5f2ff947071225847 | 8565246f48026082fac7fb3153ed8aa0 | Informative Features | 58d94dddfbfa573bd3aa3295 | 63.9929 | 0.34636
6 | eXtreme Gradient Boosted Trees Classifier with Early Stopping | eXtreme Gradient Boosted Trees Classifier with Early Stopping::Tree-based Algorithm Preprocessing v1 | 58d94ec5f2ff94707122584b | c2cf08be8a3d2c7d91bbb299ac00fe83 | Informative Features | 58d94dddfbfa573bd3aa3295 | 63.9929 | 0.34636

## Generating Model Predictions

The generation of model predictions is a three-step process:
1. Upload dataset for prediction using UploadPredictionDataset.
2. Create a predict job using RequestPredictionsForDataset function, which returns the predictJobId.
3. Pass the predictJobId to GetPredictions along with the projectId for the DataRobot project containing the model. The result returned by this function is a vector of predicted responses; in the case of binary classification projects, the optional type parameter may be used to request a vector of probabilities instead of binary responses; refer to the help files for details.

As a specific example, the following code sequence identifies the model with the best performance, extracts it as bestModel, and generates predictions for it from the test dataframe we created earlier:

    allModels <- GetAllModels(project)
    modelFrame <- as.data.frame(allModels)
    metric <- modelFrame$validationMetric
    bestIndex <- which.min(metric)
    bestModel <- allModels[[bestIndex]]
    dataset <- UploadPredictionDataset(project,test,maxWait=1200)
    bestPredictJobId <- RequestPredictionsForDataset(project, bestModel$modelId, dataset$id)
    bestPredictions <- GetPredictions(project, bestPredictJobId,type='probability')
    testPredictions <- data.frame(original=test$is_bad,prediction=bestPredictions)
    write.csv(testPredictions,file="testPredictions.csv",row.names=FALSE)
    kable(head(testPredictions),longtable=TRUE, booktabs=TRUE,row.names = TRUE)

| original | prediction
--- | --- | ---
1 | 0 | 0.1882636
2 | 0 | 0.0890647
3 | 0 | 0.0888415
4 | 0 | 0.2131941
5 | 0 | 0.0966023
6 | 0 | 0.0956107

## Calculate Feature Impact

We need to generate Feature Impact for the model before we can get reason codes using that model. Feature Impact, which is available for all model types, works by altering input data and observing the effect on a model’s score. It is an on-demand feature, meaning that you must initiate a calculation to see the results. Once you have had DataRobot compute the feature impact for a model, that information is saved with the project (you do not need to recalculate feature impact each time you re-open the project or each time you request reason codes in new data).

Feature Impact for a given column measures how much worse a model’s error score would be if DataRobot made predictions after randomly shuffling that column (while leaving other columns unchanged). This technique is sometimes called Permutation Importance.

    featureImpactJobId <- RequestFeatureImpact(bestModel)
    featureImpact <- GetFeatureImpactForJobId(project, featureImpactJobId, maxWait=1200)
    #Print top 10 features
    kable(featureImpact[1:10,],longtable=TRUE, booktabs=TRUE,row.names = TRUE)

| featureName | impactNormalized | impactUnnormalized
--- | --- | --- | ---
1 | purpose_cat | 1.0000000 | 0.0684926
2 | total_acc | 0.6393432 | 0.0437903
3 | Notes | 0.3740867 | 0.0256222
4 | open_acc | 0.2490012 | 0.0170547
5 | annual_inc | 0.1822035 | 0.0124796
6 | revol_util | 0.1383699 | 0.0094773
7 | inq_last_6mths | 0.1377131 | 0.0094323
8 | emp_title | 0.1308886 | 0.0089649
9 | purpose | 0.0708564 | 0.0048531
10 | emp_length | 0.0614025 | 0.0042056

## Calculate Reason Codes

For each prediction, DataRobot provides an ordered list of reasons; the number of reasons is based on the setting. Each reason is a feature from the dataset and its corresponding value, accompanied by a qualitative indicator of the reason’s strength—strong (+++), medium (++), or weak (+) positive or negative (-) influence.

There are three main inputs you can set for DataRobot to use when computing reason codes
1. maxCodes : #of reasons for each predictions. Default is 3.
2. thresholdLow : Probability threshold below which DataRobot should calculate reason codes.
3. thresholdHigh : probability threshold above which DataRobot should calculate reason codes.

    reasonCodeJobID <- RequestReasonCodesInitialization(bestModel)
    reasonCodeJobIDInitialization <- GetReasonCodesInitializationFromJobId(project,reasonCodeJobID)
    reasonCodeRequest <- RequestReasonCodes(bestModel, dataset$id, maxCodes = 3, thresholdLow = 0.25, thresholdHigh = 0.75)
    reasonCodeRequestMetaData <- GetReasonCodesMetadataFromJobId(project, reasonCodeRequest, maxWait = 1800)
    reasonCodeMetadata <- GetReasonCodesMetadata(project, reasonCodeRequestMetaData$id)
    reasonCodeAsDataFrame <- GetAllReasonCodesRowsAsDataFrame(project,reasonCodeRequestMetaData$id)
    reasonCodeAsDataFrame$rowId <- NULL
    write.csv(reasonCodeAsDataFrame,'reasonCodeAsDataFrame.csv',row.names = FALSE)
    #subset top 3 and bottom 3 predictions
    reasonCodeAsDataFrameTopBottom <- rbind(reasonCodeAsDataFrame[order(reasonCodeAsDataFrame$class1Probability),][1:3,],
                                            reasonCodeAsDataFrame[order(reasonCodeAsDataFrame$class2Probability),][1:3,])
    kable(head(reasonCodeAsDataFrameTopBottom),longtable=TRUE, booktabs=TRUE,row.names = TRUE)

| prediction | class1Label | class1Probability | class2Label | class2Probability | reason1FeatureName | reason1FeatureValue | reason1QualitativeStrength | reason1Strength | reason1Label | reason2FeatureName | reason2FeatureValue | reason2QualitativeStrength | reason2Strength | reason2Label | reason3FeatureName | reason3FeatureValue | reason3QualitativeStrength | reason3Strength | reason3Label
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
590 | 0 | 1 | 0.0152559 | 0 | 0.9847441 | inq_last_6mths | 5 | — | -1.1721930 | 1 | purpose | CDT CARD RAISING RATE FR/17 TO 27 % | – | -0.4290407 | 1 | total_acc | 38 | – | -0.3091894 | 1
823 | 0 | 1 | 0.0170943 | 0 | 0.9829057 | inq_last_6mths | 5 | — | -1.2399844 | 1 | annual_inc | 230000 | – | -0.6646366 | 1 | total_acc | 40 | – | -0.4963727 | 1
596 | 0 | 1 | 0.0196080 | 0 | 0.9803920 | inq_last_6mths | 6 | — | -0.9582067 | 1 | total_acc | 28 | – | -0.3250731 | 1 | annual_inc | 110000 | – | -0.1900297 | 1
600 | 1 | 1 | 0.9692447 | 0 | 0.0307553 | purpose_cat | credit card small business | +++ | 5.1677089 | 1 | inq_last_6mths | 2 | + | 0.2373998 | 1 | revol_util | 84.1 | + | 0.2321250 | 1
414 | 1 | 1 | 0.9651950 | 0 | 0.0348050 | purpose_cat | major purchase small business | +++ | 4.9394082 | 1 | Notes |  | ++ | 0.3794873 | 1 | annual_inc | 38400 | + | 0.1755247 | 1
540 | 1 | 1 | 0.9576436 | 0 | 0.0423564 | purpose_cat | small business small business | +++ | 4.5879754 | 1 | open_acc | 5 | – | -0.8902392 | 1 | earliest_cr_line (Year) | 2005 | - | -0.3356800 | 1

From the example above, you could answer “Why did the model give one of the customers a 97% probability of defaulting?” Top reason explains that purpose_cat of loan was “credit card small business”" and we can also see in above example that whenever model is predicting high probability of default, purpose_cat is related to small business.

Some notes on reasons:
- If the data points are very similar, the reasons can list the same rounded up values.
- It is possible to have a reason state of MISSING if a “missing value” was important in making the prediction.
- Typically, the top reasons for a prediction have the same direction as the outcome, but it’s possible that with interaction effects or correlations among variables a reason could, for instance, have a strong positive impact on a negative prediction.

## Summary

This note has described the Reason Codes which are useful for understanding why model is predicting high or low predictions for a specific case. DataRobot also provides qualitative stregth of each reason. Reason Codes can be used in developing good business strategy by taking prescriptions based on the reasons which are responsible for high or low predictions. They are also useful in explaining the actions taken based on the model predictions to regulatory or compliance department within an organiation.