Feature engineering and data visualization is an essential part of carrying out any Machine learning and Data analytics related work. As it allows developers to analyze their data and find the various outliers and negatively correlated features with the target feature. The idea is to make the dataset as a cleaner as possible so that a robust Machine learning model can be built and replicated by others. To do feature engineering related activities there are many ways like dropping null value columns, replacing certain values in the columns with relevant information, dropping the outliers in the dataset, changing the data type of the columns, and many more.
One such feature in engineering is scaling the metadata of the columns in our dataset. There are mainly two types of scaling techniques that are usually performed by Data scientists and these are Standard Scaling and Normalization. Both these scaling techniques although work on the same principle that is downscaling the features but have a different working mechanism and generate different types of results. Let’s discuss the differences between these two scaling techniques so that we can get a better understanding of when to use what:
Why use Scaling and on which Algorithms?
First of all, we need to understand why do we need scaling techniques to be implemented in our dataset right?? The answer to this is given below:
The machine learning algorithms that depend on gradient descent that is a parabolic curve in which our feature tries to reach the global minima point to update the weight and reduce the error or the cost function. Machine learning algorithms like Linear, Logistic regression, and Deep learning algorithms are based on the concept of gradient descent so here we do need to scale our data. The reason for selecting scaling techniques is that- when we try to achieve the global minimum point by updating the weights through backpropagation, the values of the independent features should be linearly separable and not scattered because this may lead to the case of overfitting and underfitting. Thus, to help these features get linearly separated we need to use scaling techniques.
In tree-based algorithms, the case is completely different because here there is no point to create the best-fit line and then calculating the distances of features from the best fit line and updating the weights accordingly. So tree-based algorithms do not require feature scaling and it adversely affects the efficiency of the model if we apply to scale techniques here.
Normalization
Here we will be discussing what is exactly the meaning of Normalization?
It is a scaling technique that enables users to scale their data between a range of 0 to 1. This scaling technique should be used when the metadata of the features do not follow a Gaussian distribution that is not obeying the bell-shaped curve where the central point s the mean equal to 0 and the standard deviation is equal to 1. So the graph of the dataset if not following Bell curve then we should go with Normalization technique. It is also called the Min-Max Scaling technique and is generally used in Convolutional Neural Networks that is image-based analysis.
The formula for Normalization is given as;
|
Standardization
|
It is a technique that is used when the dataset resembles a bell-shaped curve when visualizing the same through graph and glyphs. This is also called the Gaussian Normal Distribution where all the features are centered on the mean which is equal to 0 and standard deviation equal to 1. The Standardization technique helps users to find outliers in the dataset. The method to find the outliers and converting the data to the standard scale is called the Z Score method and the formula for finding the Z score is given below:
The standard scaling finds it’s the application in many Machine Learning algorithms like Logistic Regression, Support Vector Machine, Linear Regression, and many more.
Normalization vs Standardization
Although we have mentioned the difference between both standardization and normalization in real-world cases it depends upon the users what to use and when as there is no hard and fast rule that we should this technique here and disrespect the other. The choice is totally unbiased and users can use both the techniques and fine-tune their model and see the difference they are getting in the score of the dataset.
How to use Normalization in Python?
from Sklearn.preprocessing import MinMaxScaler Norm= MinMaxScaler() X_new= Norm.fit_transform(X) print(X_new)
How to use Standardization in Python?
from Sklearn.preprocessing import StandardScaler Scaler= StandardScaler() X_new= Scaler.fit_transform(X) print(X_new)
