Data Discretization

Data discretization definition:

Discretization is the process of converting continuous data into a set of discrete intervals or categories. This technique can be used for data reduction, simplification, or to make the data more suitable for analysis and it typically applied to very large datasets.

Data discretization examples using Python

Please note that you need to have the necessary Python libraries installed in your Python environment to run the code samples below.

using pandas cut()

One practical example of discretization in Python is using the pandas cut() function. The cut() function allows you to specify the number of bins you want to use and the range of the data, and it returns a new column with the values binned into those intervals. For example:

import pandas as pd

# create a sample dataset
data = pd.DataFrame({'values': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]})

# bin the values into three intervals
data['bins'] = pd.cut(data['values'], bins=3, labels=['Low', 'Medium', 'High'])

# view the binned data
print(data)

This will create a new column called 'bins' with the values binned into three intervals, 'Low', 'Medium', and 'High'. The output will be:

   values    bins
0       1     Low
1       2     Low
2       3  Medium
3       4  Medium
4       5  Medium
5       6    High
6       7    High
7       8    High
8       9    High
9      10    High

Another example of using using the pandas cut() function, if we have a DataFrame df containing a column ages , we can create bins and labels as follows:

import pandas as pd

# create a sample dataset
data = {'ages': [21, 32, 45, 18, 52, 28, 38, 50]}
df = pd.DataFrame(data)

bins = [0, 18, 35, 50, float('inf')]
labels = ['child', 'young adult', 'middle-aged', 'senior']

df['age_group'] = pd.cut(df['ages'], bins=bins, labels=labels)

print(df)

This will create a new column age_group in df containing the categorical labels based on the binning and print out the result:

   ages    age_group
0    21  young adult
1    32  young adult
2    45  middle-aged
3    18        child
4    52       senior
5    28  young adult
6    38  middle-aged
7    50  middle-aged

using scikit-learn

Another example of discretization in Python is using the KBinsDiscretizer class from the scikit-learn library. This class allows you to specify the number of bins, the strategy for dividing the data, and whether to encode the intervals as integers or one-hot vectors. For example:

from sklearn.preprocessing import KBinsDiscretizer

# create a sample dataset
data = [[1], [2], [3], [4], [5], [6], [7], [8], [9], [10]]

# bin the values into three intervals
discretizer = KBinsDiscretizer(n_bins=3, encode='ordinal', strategy='uniform')
data_bins = discretizer.fit_transform(data)

# view the binned data
print(data_bins)

This will create a new array with the values binned into three intervals, and will yield the following:

[[0.]
 [0.]
 [1.]
 [1.]
 [1.]
 [2.]
 [2.]
 [2.]
 [2.]
 [2.]]

In this case, the encode parameter is set to ordinal, which means that the intervals are represented as integers. If it were set to onehot, the intervals would be represented as one-hot vectors.