Understanding Logistic Regression for Classification: A Practical Example



In the realm of data analytics, logistic regression is a widely used statistical technique for classification problems. By applying logistic regression, we can predict the probability of an instance belonging to a particular class. In this blog post, we will dive into the implementation of logistic regression using Python, specifically on the Social_Network_Ads.csv dataset. We will explore the steps involved, including loading the dataset, splitting it into training and testing sets, fitting the logistic regression model, and computing evaluation metrics such as the confusion matrix. So, let's unravel the world of logistic regression and gain insights into its applications.



Throughout this blog post, we will be using Python and various libraries such as pandas, scikit-learn, and numpy for data manipulation, modeling, and evaluation.


import pandas as pd

from sklearn.linear_model import LogisticRegression

from sklearn.model_selection import train_test_split

from sklearn.metrics import confusion_matrix


# Load the dataset

data = pd.read_csv('Social_Network_Ads.csv')


# Split the data into X and y

X = data.iloc[:, :-1].values

y = data.iloc[:, -1].values


# Split the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)


# Fit the logistic regression model to the training set

classifier = LogisticRegression(random_state=0)

classifier.fit(X_train, y_train)


# Make predictions on the test set

y_pred = classifier.predict(X_test)


# Compute the confusion matrix

cm = confusion_matrix(y_test, y_pred)

print("Confusion Matrix:")



# Compute evaluation metrics

tp = cm[1][1]

fp = cm[0][1]

tn = cm[0][0]

fn = cm[1][0]


accuracy = (tp+tn)/(tp+tn+fp+fn)

error_rate = (fp+fn)/(tp+tn+fp+fn)

precision = tp/(tp+fp)

recall = tp/(tp+fn)


print("Accuracy: {:.2f}".format(accuracy))

print("Error Rate: {:.2f}".format(error_rate))

print("Precision: {:.2f}".format(precision))

print("Recall: {:.2f}".format(recall))


Logistic Regression for Binary Classification


Logistic regression is a powerful algorithm used for binary classification tasks. It aims to predict the probability of an instance belonging to a specific class. In this blog post, we will explore the logistic regression algorithm and demonstrate its implementation using Python. We will use the popular Social_Network_Ads.csv dataset for our classification task.


Data Preparation and Exploration


To begin, we import the necessary libraries and load the dataset. Then, we dive into understanding the dataset's structure and variables. Exploratory data analysis (EDA) techniques are applied to gain valuable insights into the data. Visualizations such as histograms and bar charts are created to help us understand the distribution of variables and any relationships that exist within the dataset.


Preparing Data for Logistic Regression


Before building our logistic regression model, we need to prepare the data appropriately. This involves splitting the dataset into features (X) and the target variable (y). If necessary, we may also need to scale or normalize numerical features and handle categorical features using suitable encoding techniques. Finally, we split the data into training and testing sets to evaluate our model's performance.


Building the Logistic Regression Model


In this section, we delve into the logistic regression model building process. We discuss the logistic regression model's overview, including its hyperparameters. Then, we fit the logistic regression model to the training set and interpret the model coefficients and their significance. We also examine the model's performance using various evaluation metrics such as accuracy, precision, recall, and the F1-score.

TP: Instances that are actually positive and correctly predicted as positive.

TN: Instances that are actually negative and correctly predicted as negative.

FP: Instances that are actually negative but incorrectly predicted as positive.

FN: Instances that are actually positive but incorrectly predicted as negative.


Accuracy is a commonly used evaluation metric in classification tasks. It measures the overall correctness of the model by calculating the ratio of correct predictions (both true positives and true negatives) to the total number of predictions. It provides an overall view of how well the model performs across all classes. However, accuracy alone may not be sufficient when dealing with imbalanced datasets.



Precision is a metric that focuses on the model's ability to correctly identify positive instances (true positives) among the instances it predicted as positive (true positives + false positives). It calculates the ratio of true positives to the total predicted positives. Precision is useful in situations where minimizing false positives is crucial, as it indicates how reliable the positive predictions are.


Error Rate:

Error rate, also known as misclassification rate, is the complement of accuracy. It calculates the ratio of incorrect predictions (false positives and false negatives) to the total number of predictions. Unlike accuracy, which measures correct predictions, error rate measures the percentage of misclassified instances.


F1 Score:

The F1 score is a measure of a model's accuracy that balances precision and recall. It is the harmonic mean of precision and recall.

 Understanding the Confusion Matrix

To assess the performance of our logistic regression model, we rely on the confusion matrix. We explain the definition and components of the confusion matrix, including true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). To aid interpretation, we visualize the confusion matrix using a heatmap. By analyzing the confusion matrix, we gain insights into the model's strengths and weaknesses.


Evaluating the Logistic Regression Model

In this final section, we calculate essential evaluation metrics to assess the logistic regression model's performance. We calculate metrics such as accuracy, error rate, precision, and recall. We explain the significance of each metric and discuss scenarios where different evaluation metrics are relevant. By understanding these evaluation metrics, we can effectively evaluate the performance of our logistic regression model.

Results and Conclusion


Presenting the results of logistic regression on the Social_Network_Ads.csv dataset.

Confusion Matrix:

[[68  0]

 [32  0]]

Accuracy: 0.68000

Error Rate: 0.32000

Precision: nan

Recall: 0.0000000


In this blog post, we explored logistic regression as a powerful tool for classification tasks. We walked through the implementation steps using Python and evaluated the model's performance using the confusion matrix and various evaluation metrics. By understanding logistic regression and its associated evaluation techniques, you are now equipped to apply this algorithm to your own classification problems. Logistic regression serves as a fundamental building block in the realm of data analytics, and mastering it opens doors to a wide range of predictive modeling tasks.


Note: The code snippets in this blog post will be provided for reference and better understanding.

Click here for dataset 

Previous Post Next Post