Problem Statement¶

Context¶

AllLife Bank is a US bank that has a growing customer base. The majority of its customers are liability customers (depositors) with varying sizes of deposits. The number of customers who are also borrowers (asset customers) is quite small, and the bank is interested in expanding this base rapidly to bring in more loan business and in the process, earn more through the interest on loans. In particular, the management wants to explore ways of converting its liability customers to personal loan customers (while retaining them as depositors).

A campaign that the bank ran last year for liability customers showed a healthy conversion rate of over 9% success. This has encouraged the retail marketing department to devise campaigns with better target marketing to increase the success ratio.

You as a Data scientist at AllLife bank have to build a model that will help the marketing department to identify the potential customers who have a higher probability of purchasing the loan.

Objective¶

To predict whether a liability customer will buy personal loans, to understand which customer attributes are most significant in driving purchases, and identify which segment of customers to target more.

Data Dictionary¶

  • ID: Customer ID
  • Age: Customer’s age in completed years
  • Experience: #years of professional experience
  • Income: Annual income of the customer (in thousand dollars)
  • ZIP Code: Home Address ZIP code.
  • Family: the Family size of the customer
  • CCAvg: Average spending on credit cards per month (in thousand dollars)
  • Education: Education Level. 1: Undergrad; 2: Graduate;3: Advanced/Professional
  • Mortgage: Value of house mortgage if any. (in thousand dollars)
  • Personal_Loan: Did this customer accept the personal loan offered in the last campaign? (0: No, 1: Yes)
  • Securities_Account: Does the customer have securities account with the bank? (0: No, 1: Yes)
  • CD_Account: Does the customer have a certificate of deposit (CD) account with the bank? (0: No, 1: Yes)
  • Online: Do customers use internet banking facilities? (0: No, 1: Yes)
  • CreditCard: Does the customer use a credit card issued by any other bank (excluding All life Bank)? (0: No, 1: Yes)

Importing the necessary libraries¶

In [1]:
# Installing the libraries with the specified version.
!pip install numpy==1.25.2 pandas==1.5.3 matplotlib==3.7.1 seaborn==0.13.1 scikit-learn==1.2.2 sklearn-pandas==2.2.0 -q --user

Note:

  1. After running the above cell, kindly restart the notebook kernel (for Jupyter Notebook) or runtime (for Google Colab), write the relevant code for the project from the next cell, and run all cells sequentially from the next cell.

  2. On executing the above line of code, you might see a warning regarding package dependencies. This error message can be ignored as the above code ensures that all necessary libraries and their dependencies are maintained to successfully execute the code in this notebook.

In [1]:
# we import pandas and numpy for loading and manipulating data
import pandas as pd
import numpy as np

# we import matplotlib and seaborn for data visualization
import matplotlib.pyplot as plt
import seaborn as sns

# we use scikit-learn to split data into training and test sets
from sklearn.model_selection import train_test_split

# to build decision tree model for prediction
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree

# to tune different models
from sklearn.model_selection import GridSearchCV

# to compute classification metrics
from sklearn.metrics import (
    confusion_matrix,
    accuracy_score,
    precision_score,
    recall_score,
    f1_score,
)

# to suppress unnecessary warnings
import warnings
warnings.filterwarnings("ignore")

Loading the data from the dataset¶

In [3]:
# mount google drive to current colab notebook
from google.colab import drive
drive.mount('/content/drive')

Mounted at /content/drive
In [4]:
# read the data from the dataset and load it into a pandas dataframe
original_data = pd.read_csv("/content/drive/My Drive/Texas McCombs/Machine Learning/Project #2/loan-modelling-data.csv")
In [23]:
# create a new copy of the data
data = original_data.copy()

Data Overview¶

  • Observations
  • Sanity checks
  • Data type review
  • Summary statistics
In [24]:
# view the first 5 rows of the data
data.head()
Out[24]:
ID Age Experience Income ZIPCode Family CCAvg Education Mortgage Personal_Loan Securities_Account CD_Account Online CreditCard
0 1 25 1 49 91107 4 1.6 1 0 0 1 0 0 0
1 2 45 19 34 90089 3 1.5 1 0 0 1 0 0 0
2 3 39 15 11 94720 1 1.0 1 0 0 0 0 0 0
3 4 35 9 100 94112 1 2.7 2 0 0 0 0 0 0
4 5 35 8 45 91330 4 1.0 2 0 0 0 0 0 1
In [25]:
# view the last 5 rows of the data
data.tail()
Out[25]:
ID Age Experience Income ZIPCode Family CCAvg Education Mortgage Personal_Loan Securities_Account CD_Account Online CreditCard
4995 4996 29 3 40 92697 1 1.9 3 0 0 0 0 1 0
4996 4997 30 4 15 92037 4 0.4 1 85 0 0 0 1 0
4997 4998 63 39 24 93023 2 0.3 3 0 0 0 0 0 0
4998 4999 65 40 49 90034 3 0.5 2 0 0 0 0 1 0
4999 5000 28 4 83 92612 3 0.8 1 0 0 0 0 1 1
In [26]:
# check the shape of the dataset to see the number of rows (observations) and number of columns (features)
data.shape
Out[26]:
(5000, 14)

👉 The dataset has 5,000 rows (samples) and 14 columns.

In [27]:
# check the attributes to see the number of entries, whether or not there are null values, and the data types for each column
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 5000 entries, 0 to 4999
Data columns (total 14 columns):
 #   Column              Non-Null Count  Dtype  
---  ------              --------------  -----  
 0   ID                  5000 non-null   int64  
 1   Age                 5000 non-null   int64  
 2   Experience          5000 non-null   int64  
 3   Income              5000 non-null   int64  
 4   ZIPCode             5000 non-null   int64  
 5   Family              5000 non-null   int64  
 6   CCAvg               5000 non-null   float64
 7   Education           5000 non-null   int64  
 8   Mortgage            5000 non-null   int64  
 9   Personal_Loan       5000 non-null   int64  
 10  Securities_Account  5000 non-null   int64  
 11  CD_Account          5000 non-null   int64  
 12  Online              5000 non-null   int64  
 13  CreditCard          5000 non-null   int64  
dtypes: float64(1), int64(13)
memory usage: 547.0 KB

👉 This is a quality data with no null values in any column. Very clean data.
👉 There are 14 numerical variables and no categorical variable in the data.
👉 ZIPcode although interpreted as numerical, it is a categorical variable. (We will address this later)
👉 Education, Personal_Loan, Securities_Account, CD_Account, Online, and CreditCard, although interpreted here as numerical, are categorical variables that are encoded by default.
👉 Personal Loan is our target attribute.

In [28]:
# we will drop the ID column because it doesn't add value to the analysis
data = data.drop(['ID'], axis=1)

# check the statistical summary or the descriptive statistics for the data
data.describe(include="all").T
Out[28]:
count mean std min 25% 50% 75% max
Age 5000.0 45.338400 11.463166 23.0 35.0 45.0 55.0 67.0
Experience 5000.0 20.104600 11.467954 -3.0 10.0 20.0 30.0 43.0
Income 5000.0 73.774200 46.033729 8.0 39.0 64.0 98.0 224.0
ZIPCode 5000.0 93169.257000 1759.455086 90005.0 91911.0 93437.0 94608.0 96651.0
Family 5000.0 2.396400 1.147663 1.0 1.0 2.0 3.0 4.0
CCAvg 5000.0 1.937938 1.747659 0.0 0.7 1.5 2.5 10.0
Education 5000.0 1.881000 0.839869 1.0 1.0 2.0 3.0 3.0
Mortgage 5000.0 56.498800 101.713802 0.0 0.0 0.0 101.0 635.0
Personal_Loan 5000.0 0.096000 0.294621 0.0 0.0 0.0 0.0 1.0
Securities_Account 5000.0 0.104400 0.305809 0.0 0.0 0.0 0.0 1.0
CD_Account 5000.0 0.060400 0.238250 0.0 0.0 0.0 0.0 1.0
Online 5000.0 0.596800 0.490589 0.0 0.0 1.0 1.0 1.0
CreditCard 5000.0 0.294000 0.455637 0.0 0.0 0.0 1.0 1.0

👉 The average age of our customers is approximately 45, with ages ranging from 23 to 67 years.
👉 The average customer income is 73,774, spanning a wide range from 8K to $224K, with around 50% earning less than 64K.
👉 The average monthly credit card spending is 1,937, some customers spend 0, with ~50% of customers spending less than 1,500 per month.
👉 The average mortgage value of customer homes is approximately 56K, with some having a mortgage value of over 600K.
👉 About 30% of customers are single or live alone, while the second largest group consists of those with only one dependent (26%).
👉 Also, there are some negative values in the "Experience" column, which we will address in the subsequent steps.

In [29]:
# we know there is no null/missing values
# check for duplicate values in the data
data.duplicated().sum()
Out[29]:
0

👉 There are no duplicate values in the data either.

Exploratory Data Analysis¶

Questions:

  1. What is the distribution of mortgage attribute? Are there any noticeable patterns or outliers in the distribution?
  2. How many customers have credit cards?
  3. What are the attributes that have a strong correlation with the target attribute (personal loan)?
  4. How does a customer's interest in purchasing a loan vary with their age?
  5. How does a customer's interest in purchasing a loan vary with their education?

Outlier Detection¶

In [30]:
Q1 = data.quantile(0.25)  # find the 25th percentile and 75th percentile
Q3 = data.quantile(0.75)

IQR = Q3 - Q1  # inter Quantile Range (75th perentile - 25th percentile)

lower = (
    Q1 - 1.5 * IQR
)  # finding lower and upper bounds for all values, all values outside these bounds are outliers
upper = Q3 + 1.5 * IQR
In [31]:
(
    (data.select_dtypes(include=["float64", "int64"]) < lower)
    | (data.select_dtypes(include=["float64", "int64"]) > upper)
).sum() / len(data) * 100
Out[31]:
0
Age 0.00
Experience 0.00
Income 1.92
ZIPCode 0.00
Family 0.00
CCAvg 6.48
Education 0.00
Mortgage 5.82
Personal_Loan 9.60
Securities_Account 10.44
CD_Account 6.04
Online 0.00
CreditCard 0.00

👉 We will not treate the outliers as we would want our model to learn the underlying pattern for all customers.

Check for Anomalous Values¶

In [32]:
# the Experience column has negatives values, we will address that
data["Experience"].unique()
Out[32]:
array([ 1, 19, 15,  9,  8, 13, 27, 24, 10, 39,  5, 23, 32, 41, 30, 14, 18,
       21, 28, 31, 11, 16, 20, 35,  6, 25,  7, 12, 26, 37, 17,  2, 36, 29,
        3, 22, -1, 34,  0, 38, 40, 33,  4, -2, 42, -3, 43])
In [33]:
# see the values that are smaller than zero
data[data["Experience"] < 0]["Experience"].unique()
Out[33]:
array([-1, -2, -3])
In [34]:
# we will assume that these are typos, we will replace them with positive signs
# correcting the experience values
data["Experience"].replace(-1, 1, inplace=True)
data["Experience"].replace(-2, 2, inplace=True)
data["Experience"].replace(-3, 3, inplace=True)

Univariate Analysis¶

In [36]:
# we will create histograms for the numerical variables to understand data distribution

# defining the figure size
plt.figure(figsize=(15, 10))

# defining the list of numerical features to plot
num_features = ['Age', 'Experience', 'Income', 'Family', 'CCAvg', 'Mortgage']

# plotting the histogram for each numerical feature
for i, feature in enumerate(num_features):
    plt.subplot(3, 3, i+1)    # assign subplots in the main plot
    sns.histplot(data=data, x=feature)    # plot the histogram

plt.tight_layout();   # to add spacing between plots

👉 Customer age and professional experience exhibit a fairly symmetrical distribution.
👉 Annual income shows a right-skewed distribution.
👉 Family size is almost symmetrical distribution.
👉 Average monthly credit card spending and mortgage payments are highly right-skewed, with the vast majority of customers having no mortgage payments at all.

In [37]:
# create boxplots to see outliers

# defining the figure size
plt.figure(figsize=(15, 10))

# plotting the boxplot for each numerical feature
for i, feature in enumerate(num_features):
    plt.subplot(3, 3, i+1)    # assign subplots in the main plot
    sns.boxplot(data=data, x=feature)    # plot the box-and-whisker diagram

plt.tight_layout();    # to add spacing between plots

👉 Customer age and professional experience show perfect distribution of the data.
👉 There are outliers in Income, Credit Card payments, and Mortgage payments.
👉 Income, CCAvg, and Mortgage are right-skewed, which is common with many economic-related data sets.
👉 Customers with high value of mortgage can be potential customers.

In [38]:
# checking number of values in categorical attributes
categorical_columns = ['Education', 'Personal_Loan', 'Securities_Account', 'CD_Account', 'Online', 'CreditCard']

# defining the figure size
plt.figure(figsize=(15, 10))

# loop through each categorical variable and plot the count plot
for i, column in enumerate(categorical_columns, 1):
    plt.subplot(3, 3, i)  # assign subplots in the main plot
    ax = sns.countplot(data=data, x=column)

    # Add values on top of each bar
    for p in ax.patches:
        ax.annotate(f'{p.get_height()}', (p.get_x() + p.get_width() / 2, p.get_height()),
                    ha='center', va='bottom')
    plt.tight_layout()

plt.show()

👉 Only 1470 customers use credit cards issued by other banks. The majority of customers do not use credit cards from other banks.
👉 The majority of customers use online banking. However, most do not have a personal loan, securities account, or CD account with us.
👉 Counts for the other attributes can be seen in the graphs above.

Categorize the Education column values¶

In [39]:
data["Education"].replace(1, "Undergraduate", inplace=True)
data["Education"].replace(2, "Graduate", inplace=True)
data["Education"].replace(3, "Advanced", inplace=True)

Bivariant Analysis¶

In [40]:
# we will look into correlation and relationship between multiple variables

# defining the size of the plot
plt.figure(figsize=(12, 7))

# plotting the heatmap for correlation
sns.heatmap(
    data[num_features].corr(),annot=True, vmin=-1, vmax=1, fmt=".2f", cmap="Spectral"
)

plt.title('Heatmap of Correlation')
plt.show()

👉 Customer age and years of professional experience have a strong positive correlation, as expected.
👉 Average credit card spending increases as income rises.
👉 Similarly, morgage payments increases as income rises.
👉 Interestingly, family size and income exhibit a slightly negative correlation.
👉 There is a very weak negative correlation between age and the interest in purchasing a loan.
👉 Also, there is a weak positive correlation between education and the interest in purchasing a loan.
👉 Income, credit card spending, and mortgage payments have a fairly positive correlation with the target attribute (personal loan), in that order.

In [41]:
# we will use scatter plot matrix to visualize relationships between variables

# exclude 'Experience', 'Family', 'Personal_Loan', and 'Education'
num_features_filtered = [col for col in num_features if col not in ['Experience', 'Family', 'Personal_Loan', 'Education']]

# create the pairplot
plt.figure(figsize=(12, 8))
sns.pairplot(data, vars=num_features_filtered, hue='Personal_Loan', diag_kind='kde');

<Figure size 1200x800 with 0 Axes>

👉 Customers with an annual income higher than 100K are more likely to take out a personal loan.
👉 Customers with an average credit card spending of 3K per month and above are more likely to take out a personal loan.
👉 Customers with a mortgage value of 300K and above are more likely to take out a personal loan.

Correlation Pairs¶

In [42]:
# we will define a function to plot category counts and create a stacked bar chart based on the predictor and target variables
def stacked_barplot(data, predictor, target):
    """
    data: dataframe
    predictor: independent variable
    target: target variable
    """
    count = data[predictor].nunique()
    sorter = data[target].value_counts().index[-1]
    tab1 = pd.crosstab(data[predictor], data[target], margins=True).sort_values(
        by=sorter, ascending=False
    )
    print(tab1)
    print("-" * 120)
    tab = pd.crosstab(data[predictor], data[target], normalize="index").sort_values(
        by=sorter, ascending=False
    )
    tab.plot(kind="bar", stacked=True, figsize=(count + 5, 5))
    plt.legend(
        loc="lower left", frameon=False,
    )
    plt.legend(loc="upper left", bbox_to_anchor=(1, 1))
    plt.show()
In [43]:
# Personal_Loan vs Education
stacked_barplot(data, "Education", "Personal_Loan")

Personal_Loan     0    1   All
Education                     
All            4520  480  5000
Advanced       1296  205  1501
Graduate       1221  182  1403
Undergraduate  2003   93  2096
------------------------------------------------------------------------------------------------------------------------

👉 Loan approval rates show a modest increase with higher education levels.

In [44]:
# Personal_Loan vs Family size
stacked_barplot(data, "Family", "Personal_Loan")

Personal_Loan     0    1   All
Family                        
All            4520  480  5000
4              1088  134  1222
3               877  133  1010
1              1365  107  1472
2              1190  106  1296
------------------------------------------------------------------------------------------------------------------------

👉 Loan approval rates show a modest increase with the family size.

In [45]:
# Personal_Loan vs CD_Account
stacked_barplot(data, "CD_Account", "Personal_Loan")

Personal_Loan     0    1   All
CD_Account                    
All            4520  480  5000
0              4358  340  4698
1               162  140   302
------------------------------------------------------------------------------------------------------------------------

👉 Individuals with a CD account have significantly higher personal loan approval rates.

In [46]:
### function to plot distributions wrt target

def distribution_plot_wrt_target(data, predictor, target):

    fig, axs = plt.subplots(2, 2, figsize=(12, 10))

    target_uniq = data[target].unique()

    axs[0, 0].set_title("Distribution of target for target=" + str(target_uniq[0]))
    sns.histplot(
        data=data[data[target] == target_uniq[0]],
        x=predictor,
        kde=True,
        ax=axs[0, 0],
        color="teal",
        stat="density",
    )

    axs[0, 1].set_title("Distribution of target for target=" + str(target_uniq[1]))
    sns.histplot(
        data=data[data[target] == target_uniq[1]],
        x=predictor,
        kde=True,
        ax=axs[0, 1],
        color="orange",
        stat="density",
    )

    axs[1, 0].set_title("Boxplot w.r.t target")
    sns.boxplot(data=data, x=target, y=predictor, ax=axs[1, 0], palette="gist_rainbow")

    axs[1, 1].set_title("Boxplot (without outliers) w.r.t target")
    sns.boxplot(
        data=data,
        x=target,
        y=predictor,
        ax=axs[1, 1],
        showfliers=False,
        palette="gist_rainbow",
    )

    plt.tight_layout()
    plt.show()

Personal Loan vs Age¶

In [47]:
distribution_plot_wrt_target(data, "Age", "Personal_Loan")

👉 The customers who have the requirement of a Personal Loan have a wider range than the ones who do not require a Personal Loan.

Personal Loan vs Income¶

In [48]:
distribution_plot_wrt_target(data, "Income", "Personal_Loan")

👉 Customers who have an income higher than 90k-100k dollars are the potential customers who will take the Personal Loan.
👉 Income seems to be a significant predictor as it provides a good separation between two classes.

Data Preparation for Modeling¶

Data Preprocessing¶

We completed missing value treatment, outlier detection, found no duplicate entries in the detailed EDA section above. Next, we will proceed with feature engineering and preparing our data for modeling.

Feature Engineering¶

👉 We will drop the Experience column as it perfectly correlated with Age.
👉 We will use dummies for ZIPCode and Education columns.

In [49]:
# dropping Experience as it is perfectly correlated with Age
data.drop(["Personal_Loan", "Experience"], axis=1)
Out[49]:
Age Income ZIPCode Family CCAvg Education Mortgage Securities_Account CD_Account Online CreditCard
0 25 49 91107 4 1.6 Undergraduate 0 1 0 0 0
1 45 34 90089 3 1.5 Undergraduate 0 1 0 0 0
2 39 11 94720 1 1.0 Undergraduate 0 0 0 0 0
3 35 100 94112 1 2.7 Graduate 0 0 0 0 0
4 35 45 91330 4 1.0 Graduate 0 0 0 0 1
... ... ... ... ... ... ... ... ... ... ... ...
4995 29 40 92697 1 1.9 Advanced 0 0 0 1 0
4996 30 15 92037 4 0.4 Undergraduate 85 0 0 1 0
4997 63 24 93023 2 0.3 Advanced 0 0 0 0 0
4998 65 49 90034 3 0.5 Graduate 0 0 0 1 0
4999 28 83 92612 3 0.8 Undergraduate 0 0 0 1 1

5000 rows × 11 columns

In [50]:
# checking the number of uniques in the ZipCode column
print(
    "Total number of unique values of ZIPCode: ",
    data["ZIPCode"].nunique()
)

Total number of unique values of ZIPCode:  467

👉 There are 467 unique Zip Codes.

In [51]:
# we will group the Zip codes based on the first two digits to reduce the number of unique values
data["ZIPCode"] = data["ZIPCode"].astype(str)
print(
    "Number of unique values if we take first two digits of ZIPCode: ",
    data["ZIPCode"].str[0:2].nunique(),
)
data["ZIPCode"] = data["ZIPCode"].str[0:2]

Number of unique values if we take first two digits of ZIPCode:  7

👉 Now, we only have 7 unique Zip Codes.

In [52]:
# we will convert all categorical features to 'category' type
cat_cols = [
    "Education",
    "Personal_Loan",
    "Securities_Account",
    "CD_Account",
    "Online",
    "CreditCard",
    "ZIPCode",
]
data[cat_cols] = data[cat_cols].astype("category")

👉 This makes data smaller in the memory and improves model performance

In [53]:
# defining the explanatory (independent) and response (dependent) variables
# dropping Experience as it is perfectly correlated with Age
X = data.drop(["Personal_Loan", "Experience"], axis=1)
y = data["Personal_Loan"]

# creating dummy variables
X = pd.get_dummies(X, columns=["ZIPCode", "Education"], drop_first=True)
# specifying the datatype of the independent variables data frame
X = X.astype(float)

X.head()
Out[53]:
Age Income Family CCAvg Mortgage Securities_Account CD_Account Online CreditCard ZIPCode_91 ZIPCode_92 ZIPCode_93 ZIPCode_94 ZIPCode_95 ZIPCode_96 Education_Graduate Education_Undergraduate
0 25.0 49.0 4.0 1.6 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
1 45.0 34.0 3.0 1.5 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
2 39.0 11.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0
3 35.0 100.0 1.0 2.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0
4 35.0 45.0 4.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0

Splitting Data into Training and Testing Datasets¶

In [54]:
# splitting the data in an 70:30 ratio for train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, stratify=y, random_state=42)
# stratify ensures that the training and test sets have a similar distribution of the response variable
In [55]:
print("Shape of Training set : ", X_train.shape)
print("Shape of Test set : ", X_test.shape)
print("Percentage of classes in Training set:")
print(y_train.value_counts(normalize=True))
print("Percentage of classes in Test set:")
print(y_test.value_counts(normalize=True))

Shape of Training set :  (3500, 17)
Shape of Test set :  (1500, 17)
Percentage of classes in Training set:
Personal_Loan
0    0.904
1    0.096
Name: proportion, dtype: float64
Percentage of classes in Test set:
Personal_Loan
0    0.904
1    0.096
Name: proportion, dtype: float64

👉 70% of the data is in the training set (3500 out of 5000 samples) and 30% in the test set (1500 out of 5000 samples).
👉 90.4% did not accept the loan and 9.6% accepted the loan. Proportions are exactly same becuase we used 'stratify=y'.

Model Building¶

Model Evaluation Criterion¶

The model can make incorrect predictions, such as:

  1. Predicting that a customer will take a personal loan, but in reality, the customer does not take the loan: Loss of resources
  2. Predicting that a customer will not take a personal loan, but in reality, the customer was going to take the loan: Loss of opportunity

Which case is more important?

  • Losing a potential customer by predicting that the customer will not take a personal loan is a FALSE NEGATIVE. We definitely want to avoid these cases.

How can we reduce the False Negative?

  • Our model must maximize Recall to minimize false negatives. The greater the Recall, the higher the chances of reducing false negatives. Therefore, our model must focus on increasing Recall.

Model Building¶

Decision Tree (scikit-learn / sklearn)¶

👉 Create functions to calculate the confusin matrix and the and different metrics.
👉 We will define a utility function to compile all the metrics into a single data frame and another function to visualize the confusion matrix.

In [56]:
# defining a function to compute different metrics to check performance of a classification model built using sklearn
def model_performance_classification(model, predictors, target):
    """
    Function to compute different metrics to check classification model performance

    model: classifier
    predictors: independent variables
    target: dependent variable
    """

    # predicting using the independent variables
    pred = model.predict(predictors)

    acc = accuracy_score(target, pred)  # to compute Accuracy
    recall = recall_score(target, pred)  # to compute Recall
    precision = precision_score(target, pred)  # to compute Precision
    f1 = f1_score(target, pred)  # to compute F1-score

    # creating a dataframe of metrics
    df_perf = pd.DataFrame(
        {"Accuracy": acc, "Recall": recall, "Precision": precision, "F1": f1,},
        index=[0],
    )

    return df_perf
In [57]:
def plot_confusion_matrix(model, predictors, target):
    """
    To plot the confusion_matrix with percentages

    model: classifier
    predictors: independent variables
    target: dependent variable
    """
    # Predict the target values using the provided model and predictors
    y_pred = model.predict(predictors)

    # Compute the confusion matrix comparing the true target values with the predicted values
    cm = confusion_matrix(target, y_pred)

    # Create labels for each cell in the confusion matrix with both count and percentage
    labels = np.asarray(
        [
            ["{0:0.0f}".format(item) + "\n{0:.2%}".format(item / cm.flatten().sum())]
            for item in cm.flatten()
        ]
    ).reshape(2, 2)    # reshaping to a matrix

    # Set the figure size for the plot
    plt.figure(figsize=(6, 4))

    # Plot the confusion matrix as a heatmap with the labels
    sns.heatmap(cm, annot=labels, fmt="")

    # Add a label to the y-axis
    plt.ylabel("True label")

    # Add a label to the x-axis
    plt.xlabel("Predicted label")

👉 We start fitting/training the model

In [58]:
# creating an instance of the decision tree model
dtree1 = DecisionTreeClassifier(criterion="gini", random_state=42)
# random_state sets a seed value and enables reproducibility

# fitting the model to the training data
dtree1.fit(X_train, y_train)
Out[58]:
DecisionTreeClassifier(random_state=42)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
DecisionTreeClassifier(random_state=42)

Model Evaluation¶

Confusion Matrix for Training Data:

In [59]:
plot_confusion_matrix(dtree1, X_train, y_train)

👉 The confusion matrix perfectly shows that there were no instances of false negatives (Type II errors) and false positives (Type I errors).
👉 Decision trees ALWAYS tend to overfit the training data.

In [60]:
dtree1_train_perf = model_performance_classification(
    dtree1, X_train, y_train
)
dtree1_train_perf
Out[60]:
Accuracy Recall Precision F1
0 1.0 1.0 1.0 1.0

👉 These metrics indicate that the model performs exceptionally well, achieving perfect scores across all measures for the training dataset.
👉 A decision tree will continue to grow and classify every single data point correctly if no restrictions are imposed, as it will learn all the patterns in the training data.

Confusion Matrix for Test Data:

In [61]:
plot_confusion_matrix(dtree1, X_test, y_test)

👉 The confusion matrix generated from the test dataset reveals that 1.60% of actual positive instances were incorrectly classified as negative and 0.53% of actual negative instances were misclassified as positive.

In [62]:
dtree1_test_perf = model_performance_classification(
    dtree1, X_test, y_test
)
dtree1_test_perf
Out[62]:
Accuracy Recall Precision F1
0 0.978667 0.944444 0.85 0.894737

👉 While the model performs well overall, the recall is 0.944, which is still quite high but lower than the training data. Since our goal is to maximize Recall, this is a positive result, though there might be room for slight improvement.
👉 While recall remains high on the test data, precision has dropped to 0.85, which indicates some false positives. This is expected when focusing on maximizing recall, as there’s often a trade-off between recall and precision.

Visualizing the Decision Tree¶

In [63]:
# list of feature names in X_train
feature_names = list(X_train.columns)

# set the figure size for the plot
plt.figure(figsize=(20, 20))

# plotting the decision tree
out = tree.plot_tree(
    dtree1,                         # decision tree classifier model
    feature_names=feature_names,    # list of feature names (columns) in the dataset
    filled=True,                    # fill the nodes with colors based on class
    fontsize=9,                     # font size for the node text
    node_ids=False,                 # do not show the ID of each node
    class_names=None,               # whether or not to display class names
)

# add arrows to the decision tree splits if they are missing
for o in out:
    arrow = o.arrow_patch
    if arrow is not None:
        arrow.set_edgecolor("black")    # set arrow color to black
        arrow.set_linewidth(1)          # set arrow linewidth to 1

# displaying the plot
plt.show()

👉 This is a fully-grown decision tree. We can definitely simplify it.

In [64]:
# printing a text report showing the rules of a decision tree
print(
    tree.export_text(
        dtree1,    # specify the model
        feature_names=feature_names,    # specify the feature names
        show_weights=True    # specify whether or not to show the weights associated with the model
    )
)

|--- Income <= 98.50
|   |--- CCAvg <= 2.95
|   |   |--- weights: [2483.00, 0.00] class: 0
|   |--- CCAvg >  2.95
|   |   |--- CD_Account <= 0.50
|   |   |   |--- Age <= 27.00
|   |   |   |   |--- weights: [0.00, 2.00] class: 1
|   |   |   |--- Age >  27.00
|   |   |   |   |--- Income <= 92.50
|   |   |   |   |   |--- CCAvg <= 3.65
|   |   |   |   |   |   |--- Mortgage <= 216.50
|   |   |   |   |   |   |   |--- Income <= 82.50
|   |   |   |   |   |   |   |   |--- Age <= 45.50
|   |   |   |   |   |   |   |   |   |--- Age <= 43.50
|   |   |   |   |   |   |   |   |   |   |--- ZIPCode_94 <= 0.50
|   |   |   |   |   |   |   |   |   |   |   |--- weights: [15.00, 0.00] class: 0
|   |   |   |   |   |   |   |   |   |   |--- ZIPCode_94 >  0.50
|   |   |   |   |   |   |   |   |   |   |   |--- truncated branch of depth 2
|   |   |   |   |   |   |   |   |   |--- Age >  43.50
|   |   |   |   |   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |   |   |   |--- Age >  45.50
|   |   |   |   |   |   |   |   |   |--- weights: [22.00, 0.00] class: 0
|   |   |   |   |   |   |   |--- Income >  82.50
|   |   |   |   |   |   |   |   |--- CCAvg <= 3.05
|   |   |   |   |   |   |   |   |   |--- weights: [6.00, 0.00] class: 0
|   |   |   |   |   |   |   |   |--- CCAvg >  3.05
|   |   |   |   |   |   |   |   |   |--- Online <= 0.50
|   |   |   |   |   |   |   |   |   |   |--- ZIPCode_92 <= 0.50
|   |   |   |   |   |   |   |   |   |   |   |--- weights: [0.00, 3.00] class: 1
|   |   |   |   |   |   |   |   |   |   |--- ZIPCode_92 >  0.50
|   |   |   |   |   |   |   |   |   |   |   |--- truncated branch of depth 2
|   |   |   |   |   |   |   |   |   |--- Online >  0.50
|   |   |   |   |   |   |   |   |   |   |--- Family <= 1.50
|   |   |   |   |   |   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |   |   |   |   |   |--- Family >  1.50
|   |   |   |   |   |   |   |   |   |   |   |--- truncated branch of depth 3
|   |   |   |   |   |   |--- Mortgage >  216.50
|   |   |   |   |   |   |   |--- ZIPCode_93 <= 0.50
|   |   |   |   |   |   |   |   |--- weights: [0.00, 2.00] class: 1
|   |   |   |   |   |   |   |--- ZIPCode_93 >  0.50
|   |   |   |   |   |   |   |   |--- weights: [1.00, 0.00] class: 0
|   |   |   |   |   |--- CCAvg >  3.65
|   |   |   |   |   |   |--- Mortgage <= 89.00
|   |   |   |   |   |   |   |--- weights: [43.00, 0.00] class: 0
|   |   |   |   |   |   |--- Mortgage >  89.00
|   |   |   |   |   |   |   |--- Mortgage <= 99.50
|   |   |   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |   |   |--- Mortgage >  99.50
|   |   |   |   |   |   |   |   |--- weights: [13.00, 0.00] class: 0
|   |   |   |   |--- Income >  92.50
|   |   |   |   |   |--- Family <= 1.50
|   |   |   |   |   |   |--- weights: [6.00, 0.00] class: 0
|   |   |   |   |   |--- Family >  1.50
|   |   |   |   |   |   |--- Age <= 35.50
|   |   |   |   |   |   |   |--- Education_Graduate <= 0.50
|   |   |   |   |   |   |   |   |--- weights: [2.00, 0.00] class: 0
|   |   |   |   |   |   |   |--- Education_Graduate >  0.50
|   |   |   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |   |--- Age >  35.50
|   |   |   |   |   |   |   |--- weights: [0.00, 5.00] class: 1
|   |   |--- CD_Account >  0.50
|   |   |   |--- CCAvg <= 4.25
|   |   |   |   |--- weights: [0.00, 6.00] class: 1
|   |   |   |--- CCAvg >  4.25
|   |   |   |   |--- Mortgage <= 38.00
|   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |--- Mortgage >  38.00
|   |   |   |   |   |--- weights: [3.00, 0.00] class: 0
|--- Income >  98.50
|   |--- Education_Undergraduate <= 0.50
|   |   |--- Income <= 114.50
|   |   |   |--- CCAvg <= 2.45
|   |   |   |   |--- Income <= 106.50
|   |   |   |   |   |--- weights: [28.00, 0.00] class: 0
|   |   |   |   |--- Income >  106.50
|   |   |   |   |   |--- ZIPCode_93 <= 0.50
|   |   |   |   |   |   |--- Income <= 109.50
|   |   |   |   |   |   |   |--- ZIPCode_94 <= 0.50
|   |   |   |   |   |   |   |   |--- weights: [2.00, 0.00] class: 0
|   |   |   |   |   |   |   |--- ZIPCode_94 >  0.50
|   |   |   |   |   |   |   |   |--- weights: [0.00, 2.00] class: 1
|   |   |   |   |   |   |--- Income >  109.50
|   |   |   |   |   |   |   |--- Mortgage <= 321.00
|   |   |   |   |   |   |   |   |--- Age <= 49.00
|   |   |   |   |   |   |   |   |   |--- CCAvg <= 2.35
|   |   |   |   |   |   |   |   |   |   |--- Age <= 33.50
|   |   |   |   |   |   |   |   |   |   |   |--- weights: [10.00, 0.00] class: 0
|   |   |   |   |   |   |   |   |   |   |--- Age >  33.50
|   |   |   |   |   |   |   |   |   |   |   |--- truncated branch of depth 3
|   |   |   |   |   |   |   |   |   |--- CCAvg >  2.35
|   |   |   |   |   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |   |   |   |--- Age >  49.00
|   |   |   |   |   |   |   |   |   |--- weights: [13.00, 0.00] class: 0
|   |   |   |   |   |   |   |--- Mortgage >  321.00
|   |   |   |   |   |   |   |   |--- Education_Graduate <= 0.50
|   |   |   |   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |   |   |   |--- Education_Graduate >  0.50
|   |   |   |   |   |   |   |   |   |--- weights: [1.00, 0.00] class: 0
|   |   |   |   |   |--- ZIPCode_93 >  0.50
|   |   |   |   |   |   |--- CCAvg <= 1.80
|   |   |   |   |   |   |   |--- weights: [0.00, 2.00] class: 1
|   |   |   |   |   |   |--- CCAvg >  1.80
|   |   |   |   |   |   |   |--- weights: [1.00, 0.00] class: 0
|   |   |   |--- CCAvg >  2.45
|   |   |   |   |--- ZIPCode_93 <= 0.50
|   |   |   |   |   |--- Age <= 63.50
|   |   |   |   |   |   |--- Family <= 2.50
|   |   |   |   |   |   |   |--- ZIPCode_94 <= 0.50
|   |   |   |   |   |   |   |   |--- Income <= 106.00
|   |   |   |   |   |   |   |   |   |--- weights: [5.00, 0.00] class: 0
|   |   |   |   |   |   |   |   |--- Income >  106.00
|   |   |   |   |   |   |   |   |   |--- ZIPCode_92 <= 0.50
|   |   |   |   |   |   |   |   |   |   |--- weights: [0.00, 3.00] class: 1
|   |   |   |   |   |   |   |   |   |--- ZIPCode_92 >  0.50
|   |   |   |   |   |   |   |   |   |   |--- weights: [1.00, 0.00] class: 0
|   |   |   |   |   |   |   |--- ZIPCode_94 >  0.50
|   |   |   |   |   |   |   |   |--- CCAvg <= 2.95
|   |   |   |   |   |   |   |   |   |--- weights: [1.00, 0.00] class: 0
|   |   |   |   |   |   |   |   |--- CCAvg >  2.95
|   |   |   |   |   |   |   |   |   |--- weights: [0.00, 7.00] class: 1
|   |   |   |   |   |   |--- Family >  2.50
|   |   |   |   |   |   |   |--- weights: [0.00, 10.00] class: 1
|   |   |   |   |   |--- Age >  63.50
|   |   |   |   |   |   |--- weights: [2.00, 0.00] class: 0
|   |   |   |   |--- ZIPCode_93 >  0.50
|   |   |   |   |   |--- CreditCard <= 0.50
|   |   |   |   |   |   |--- weights: [5.00, 0.00] class: 0
|   |   |   |   |   |--- CreditCard >  0.50
|   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |--- Income >  114.50
|   |   |   |--- Income <= 116.50
|   |   |   |   |--- CCAvg <= 1.10
|   |   |   |   |   |--- CCAvg <= 0.65
|   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |--- CCAvg >  0.65
|   |   |   |   |   |   |--- weights: [2.00, 0.00] class: 0
|   |   |   |   |--- CCAvg >  1.10
|   |   |   |   |   |--- weights: [0.00, 6.00] class: 1
|   |   |   |--- Income >  116.50
|   |   |   |   |--- weights: [0.00, 215.00] class: 1
|   |--- Education_Undergraduate >  0.50
|   |   |--- Family <= 2.50
|   |   |   |--- Income <= 99.50
|   |   |   |   |--- Online <= 0.50
|   |   |   |   |   |--- weights: [0.00, 2.00] class: 1
|   |   |   |   |--- Online >  0.50
|   |   |   |   |   |--- weights: [1.00, 0.00] class: 0
|   |   |   |--- Income >  99.50
|   |   |   |   |--- Income <= 104.50
|   |   |   |   |   |--- CCAvg <= 3.31
|   |   |   |   |   |   |--- weights: [17.00, 0.00] class: 0
|   |   |   |   |   |--- CCAvg >  3.31
|   |   |   |   |   |   |--- Mortgage <= 124.50
|   |   |   |   |   |   |   |--- CCAvg <= 4.25
|   |   |   |   |   |   |   |   |--- weights: [0.00, 3.00] class: 1
|   |   |   |   |   |   |   |--- CCAvg >  4.25
|   |   |   |   |   |   |   |   |--- weights: [1.00, 0.00] class: 0
|   |   |   |   |   |   |--- Mortgage >  124.50
|   |   |   |   |   |   |   |--- weights: [2.00, 0.00] class: 0
|   |   |   |   |--- Income >  104.50
|   |   |   |   |   |--- weights: [449.00, 0.00] class: 0
|   |   |--- Family >  2.50
|   |   |   |--- Income <= 113.50
|   |   |   |   |--- Online <= 0.50
|   |   |   |   |   |--- CCAvg <= 2.05
|   |   |   |   |   |   |--- Family <= 3.50
|   |   |   |   |   |   |   |--- weights: [2.00, 0.00] class: 0
|   |   |   |   |   |   |--- Family >  3.50
|   |   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |--- CCAvg >  2.05
|   |   |   |   |   |   |--- weights: [0.00, 3.00] class: 1
|   |   |   |   |--- Online >  0.50
|   |   |   |   |   |--- CCAvg <= 0.65
|   |   |   |   |   |   |--- weights: [0.00, 1.00] class: 1
|   |   |   |   |   |--- CCAvg >  0.65
|   |   |   |   |   |   |--- weights: [9.00, 0.00] class: 0
|   |   |   |--- Income >  113.50
|   |   |   |   |--- weights: [0.00, 49.00] class: 1
In [65]:
# importance of features in the tree building

importance_df = pd.DataFrame(
    dtree1.feature_importances_, columns=["Imp"], index=X_train.columns
).sort_values(by="Imp", ascending=False)

print(importance_df)

                              Imp
Income                   0.417597
Education_Undergraduate  0.290600
Family                   0.145648
CCAvg                    0.064047
Age                      0.016731
Mortgage                 0.011947
ZIPCode_94               0.010072
CD_Account               0.009784
ZIPCode_93               0.008997
Education_Graduate       0.008670
Online                   0.008500
ZIPCode_92               0.004664
CreditCard               0.002744
Securities_Account       0.000000
ZIPCode_91               0.000000
ZIPCode_95               0.000000
ZIPCode_96               0.000000

Model Performance Improvement¶

Decision Tree (Pre-pruning)¶

Pre-pruning, also known as early stopping, involves halting the growth of a decision tree during training by setting conditions such as maximum depth, minimum samples per leaf, or minimum impurity decrease to prevent the model from becoming too complex and overfitting the training data.

In [78]:
# define the parameters of the tree to iterate over
max_depth_values = np.arange(2, 7, 2)
max_leaf_nodes_values = np.arange(10, 51, 10)
min_samples_split_values = np.arange(10, 51, 10)

# initialize variables to store the best model and its performance
best_estimator = None
best_score_diff = float('inf')
best_test_score = 0.0

# iterate over all combinations of the specified parameter values
for max_depth in max_depth_values:
    for max_leaf_nodes in max_leaf_nodes_values:
        for min_samples_split in min_samples_split_values:

            # initialize the tree with the current set of parameters
            estimator = DecisionTreeClassifier(
                max_depth=max_depth,
                max_leaf_nodes=max_leaf_nodes,
                min_samples_split=min_samples_split,
                class_weight='balanced',
                random_state=42
            )

            # fit the model to the training data
            estimator.fit(X_train, y_train)

            # make predictions on the training and test sets
            y_train_pred = estimator.predict(X_train)
            y_test_pred = estimator.predict(X_test)

            # our metric is Recall !!!
            # calculate Recall scores for training and test sets
            train_recall_score = recall_score(y_train, y_train_pred)
            test_recall_score = recall_score(y_test, y_test_pred)

            # calculate the absolute difference between training and test Recall scores
            score_diff = abs(train_recall_score - test_recall_score)

            # update the best estimator and best score if the current one has a smaller score difference
            if (score_diff < best_score_diff) & (test_recall_score > best_test_score):
                best_score_diff = score_diff
                best_test_score = test_recall_score
                best_estimator = estimator
In [79]:
# creating an instance of the best model
dtree2 = best_estimator

# print the best parameters
print("Best parameters found:")
print(f"Max depth: {best_estimator.max_depth}")
print(f"Max leaf nodes: {best_estimator.max_leaf_nodes}")
print(f"Min samples split: {best_estimator.min_samples_split}")
print(f"Best test Recall score: {best_test_score}")

Best parameters found:
Max depth: 2
Max leaf nodes: 10
Min samples split: 10
Best test Recall score: 1.0

👉 Best test Recall score of 1.0 is impressive! That means no False Nagatives.

In [80]:
# fitting the best model to the training data
dtree2.fit(X_train, y_train)
Out[80]:
DecisionTreeClassifier(class_weight='balanced', max_depth=2, max_leaf_nodes=10,
                       min_samples_split=10, random_state=42)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
DecisionTreeClassifier(class_weight='balanced', max_depth=2, max_leaf_nodes=10,
                       min_samples_split=10, random_state=42)
In [81]:
plot_confusion_matrix(dtree2, X_train, y_train)

👉 Training Data: Perfect Recall, but high rate of False Positives.

In [82]:
dtree2_train_perf = model_performance_classification(
    dtree2, X_train, y_train
)
dtree2_train_perf
Out[82]:
Accuracy Recall Precision F1
0 0.788 1.0 0.311688 0.475248
In [83]:
plot_confusion_matrix(dtree2, X_test, y_test)

👉 Perfect Recall, but high rate of False Positives.
👉 Our purpose was to maximize Recall.
👉 This is a great model for our case!

In [84]:
dtree2_test_perf = model_performance_classification(
    dtree2, X_test, y_test
)
dtree2_test_perf
Out[84]:
Accuracy Recall Precision F1
0 0.784667 1.0 0.308351 0.471358

👉 Recall Improvement: The recall reached the maximum value of 1.0 after pre-pruning, meaning the model successfully captures all positive instances.
👉 Trade-off: Other performance metrics, such as precision or F1, may have declined, but this is an expected outcome given the focus on maximizing recall.
👉 Conclusion: The model meets the objective of prioritizing Recall, making it suitable for our specific purpose despite a decrease in other metrics.

Visualize the Decision Tree one more time¶

In [73]:
# list of feature names in X_train
feature_names = list(X_train.columns)

# set the figure size for the plot
plt.figure(figsize=(20, 20))

# plotting the decision tree
out = tree.plot_tree(
    dtree2,                         # decision tree classifier model
    feature_names=feature_names,    # list of feature names (columns) in the dataset
    filled=True,                    # fill the nodes with colors based on class
    fontsize=9,                     # font size for the node text
    node_ids=False,                 # do not show the ID of each node
    class_names=None,               # whether or not to display class names
)

# add arrows to the decision tree splits if they are missing
for o in out:
    arrow = o.arrow_patch
    if arrow is not None:
        arrow.set_edgecolor("black")    # set arrow color to black
        arrow.set_linewidth(1)          # set arrow linewidth to 1

# displaying the plot
plt.show()

👉 The decision tree is now much simpler and more readable. The leaf values are greater than 1, indicating that each leaf represents multiple samples, suggesting improved generalization and robustness in the model.

In [85]:
importances = dtree1.feature_importances_
indices = np.argsort(importances)

plt.figure(figsize=(8, 8))
plt.title("Feature Importances")
plt.barh(range(len(indices)), importances[indices], color="violet", align="center")
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel("Relative Importance")
plt.show()

👉 Income, Education, Family, and CCAvg are the most important features.
👉 So, the bank should campaign more on people with higher income, more education, and larger family sizes.

Finalizing the Model¶

Comparing the default model with the pre-pruned model:

In [97]:
# testing performance comparison
models_test_comp_df = pd.concat(
    [
        dtree1_test_perf.T,
        dtree2_test_perf.T,
    ],
    axis=1,
)
models_test_comp_df.columns = [
    "Decision Tree (sklearn default)",
    "Decision Tree (Pre-Pruning)",
]
print("Test set performance comparison:")
models_test_comp_df

Test set performance comparison:
Out[97]:
Decision Tree (sklearn default) Decision Tree (Pre-Pruning)
Accuracy 0.978667 0.784667
Recall 0.944444 1.000000
Precision 0.850000 0.308351
F1 0.894737 0.471358

Accuracy: The default decision tree has higher accuracy (0.978) compared to the pre-pruned model (0.785). This indicates that the default tree classifies more instances correctly overall, but it might be overfitting the training data.

Recall: The pre-pruned model achieves perfect recall (1.0), meaning it correctly identifies all positive cases. This aligns with our goal to maximize recall, although other metrics are compromised.

Precision: The default decision tree has much higher precision (0.85) compared to the pre-pruned version (0.308). This suggests the pre-pruned model is more prone to false positives, as it predicts more positives but with less accuracy.

F1 Score: The F1 score, which balances precision and recall, is higher for the default model (0.895) than for the pre-pruned model (0.471). The pre-pruned model sacrifices balance to maximize recall.

In conclusion, while pre-pruning maximizes recall, it significantly lowers precision and overall accuracy. The trade-off here is between identifying all positive cases versus having more false positives.

Prediction time on a single data point¶

In [98]:
%%time
# choosing a data point
applicant_details = X_test.iloc[:1, :]

# making a prediction
approval_prediction = dtree2.predict(applicant_details)

print(approval_prediction)

[0]
CPU times: user 5.28 ms, sys: 0 ns, total: 5.28 ms
Wall time: 5.08 ms

👉 The model was able to predict the approval status in under 6 milliseconds. It indicates that the model makes predictions very quickly and efficiently.

Actionable Insights and Business Recommendations¶

👉 The AllLife bank can deploy this Machine Learning model for the initial screening of personal loan applications.

👉 The model can be used to target high-potential liability customers for personalized marketing campaigns.

👉 It allows the bank to focus resources on customers with the highest probability of converting to personal loans.

👉 The model helps identify key customer attributes that influence loan purchases, guiding tailored product offerings.

👉 The model's fast prediction time makes it suitable for real-time marketing or automated decision-making systems.