Easy Way to Understand Normalization in Statistics

Normalization can seem like a scary topic, for example, the definition from Wikipedia isn't the most straightforward:

"In the simplest cases, normalization of ratings means adjusting values measured on different scales to a notionally common scale, often prior to averaging. In more complicated cases, normalization may refer to more sophisticated adjustments where the intention is to bring the entire probability distributions of adjusted values into alignment. In the case of normalization of scores in educational assessment, there may be an intention to align distributions to a normal distribution. A different approach to normalization of probability distributions is quantile normalization, where the quantiles of the different measures are brought into alignment."

And that's in the simplest case... Normalization is a key component in machine learning, especially when it comes to data processing tasks such as feature scaling, therefore it's important to have a grasp on the workflow. So let's take some more straightforward case studies.

To start off, let's imagine that you need to compare temperatues from cities around the world. If you measure NYC, the data will most likely be in fahrenheit. And in Poland, the temperatues will be listed in celsius. At its most basic, the normalization process will take the data from each city and convert the temperatues to use the same unit of measure. I think that's a little more straightforward than the Wikipedia example. But that's simply the base case.

Now let's discuss an important task in machine learning, feature scaling. Feature scaling is a data preprocessing technique that allows for all values in a data set to be converted into a defined range, many times the range is between 0 and 1. This dramatically increases the performance of running various machine learning algorithms, since it limits the range that the algorithm will need to look over. For example, if you had a data set, such as:

Object M1 M2 M3 M4 A 22 220 11 12 B 22 220 11 12 C 22 220 11 12 D 23 230 11.5 13 E 24 240 12 14

It would be converted to:

Object M1 M2 M3 M4 A 0.44 0.44 0.44 0.43 B 0.44 0.44 0.44 0.43 C 0.44 0.44 0.44 0.43 D 0.45 0.45 0.45 0.46 E 0.47 0.47 0.47 0.50


Examples provided by Steve Borgatti in the resources below: