Binary logistic regression tutorial for intermediate coders

Nearly 70 percent of all machine learning and data mining projects use classification techniques like logistic regression or linear regression for predicting outcomes. Logistic regression is a popular classification technique used to predict categorical outcomes that are binary in nature.

So, where is logistic regression used? Let’s understand this using an example. One of the statutory requirements in a bank is to identify and report suspicious money laundering activities to the regulator. Historically, this is a manpower-intensive jobs, proving to be a huge expense for banks. With the help of binary logistic regression in machine learning, banks can now monitor their in and out payments to point out suspicious accounts and transactions.

Here is the list of variables (or dataset) that the bank has on each account from third-party sources:

• Gender (1 if the person is female, 0 otherwise)
•  Marital Status (1 if married, 0 otherwise)
• Graduate (1 if graduate, 0 otherwise)
• Employed (1 if employed, 0 otherwise)
• Retired (1 if retired, 0 otherwise)
• Home ownership (1 if own residence, 0 otherwise)
• Sanctioned country (1 if sanctioned, 0 otherwise)
• High-risk payment (1 if yes, 0 otherwise)

SEE ALSO: The role of artificial intelligence in improving data security

Machine learning uses binary logistic regression in data science to build an algorithm considering all these parameters and predicting the results as suspicious or non-suspicious (a binary outcome).

Logistic regression model for coders

With that basic understanding, let’s understand how to calculate logistic function and make predictions using a logistic regression model.

Logistic regression takes real-valued inputs and predicts the probability of the input belonging to the default class. If the probability is >0.5, you can consider the output as a prediction for the default class 0. Otherwise, the prediction is for the other class 1.

Let’s take the case where a dataset has logistic regression with three coefficients. So, in this case, the output = b0 + b1*x1 + b2*x2

The objective is to discover the best values for these coefficients.

Step 1

Transform the output into a probability using the logistic function:

p(class=0) = 1 / (1 + e^(-output))

In your spreadsheet this would be written as:

p(class=0) = 1 / (1 + EXP(-output))

Step 2

Estimate the values of the coefficients by stochastic gradient descent, a simple procedure that is used by various algorithms in machine learning. It calculates the prediction using the present values of the coefficients for each training instance. It then assesses the new coefficient value using the error in the prediction. This process is repeated until the desired accuracy level is reached.

To calculate the prediction, assign 0.0 input value to each coefficient. Then, calculate the probability of the first training instance that belongs to the default class 0.

B0 = 0.0

B1 = 0.0

B2 = 0.0

For the first training instance, x1=2.7810836, x2=2.550537003, Y=0

Using the equation shared above, calculate a prediction as follows –

Prediction = 1 / (1 + e^(-(b0 + b1*x1 + b2*x2)))

Prediction = 1 / (1 + e^(-(0.0 + 0.0*2.7810836 + 0.0*2.550537003)))

Prediction = 0.5

To calculate new coefficients, use a simple update equation as shared below,

b = b + alpha * (y – prediction) * prediction * (1 – prediction) * x

Here, ‘b’ is the coefficient that’s being updated.

Alpha is the learning rate that controls how much the coefficients can change. Let’s consider alpha to be 0.3. Good values of alpha range between 0.1 and 0.3.

‘x’ is the input value for the coefficient.  B0 does not have an input (0.0). This coefficient is referred to as bias and we can assume its  input value to be 1.0. So, updating the equation with  prediction (0.5) and coefficient values (0.0), this is what we get.

b0 = b0 + 0.3 * (0 – 0.5) * 0.5 * (1 – 0.5) * 1.0

b1 = b1 + 0.3 * (0 – 0.5) * 0.5 * (1 – 0.5) * 2.7810836

b2 = b2 + 0.3 * (0 – 0.5) * 0.5 * (1 – 0.5) * 2.550537003

OR

b0 = -0.0375

b1 = -0.104290635

b2 = -0.09564513761

Repeat this process for each training instance, till the accuracy of the model improves and errors are minimized.

SEE ALSO: Ask these 4 important questions before hiring a data scientist

Step 3

Now that you have trained the model, you can use it to make predictions. Calculate the probabilities of each instance belonging to class=0 and convert them into crisp class using this equation –

prediction = IF (output < 0.5) Then 0 Else 1

Finally, calculate the accuracy for the model:

Accuracy = (correct predictions / num predictions made) * 100

Accuracy = (10 /10) * 100

Accuracy = 100%

The objective of this post was to help you use binary logistic regression to make predictions and come up with a machine learning algorithm that performs very well on a wide range of problems. We hope this tutorial has improved your understanding of binomial logistic regression.