Definition of data munging
The process of converting unstructured data into a structured format is typically referred to as "data wrangling" or "data munging". Both terms refer to the process of transforming and mapping data from a raw form into another format that allows for more convenient consumption of the data.
However, different people and organizations may use these terms in slightly different ways. For example, some people may use "data munging" to specifically refer to the cleaning and preprocessing of data, while "data wrangling" might be used in a broader sense to include tasks such as gathering and exploring the data.
Other data engineering terms related to
‘Data Transformation’:
Align
Aligning data can mean one of three things: aligning datasets, meeting business rules, or arranging data elements in memory.
Clean or Cleanse
Remove invalid or inconsistent data values, such as empty fields or outliers.
Cluster
Group data points based on similarities or patterns to facilitate analysis and modeling.
Curate
Select, organize, and annotate data to make it more useful for analysis and modeling.
Denoise
Remove noise or artifacts from data to improve its accuracy and quality.
Denormalize
Optimize data for faster read access by reducing the number of joins needed to retrieve related data.
Derive
Extracting, transforming, and generating new data from existing datasets.
Discretize
Transform continuous data into discrete categories or bins to simplify analysis.
ETL
Extract, transform, and load data between different systems.
Encode
Convert categorical variables into numerical representations for ML algorithms.
Filter
Extract a subset of data based on specific criteria or conditions.
Fragment
Break data down into smaller chunks for storage and management purposes.
Homogenize
Make data uniform, consistent, and comparable.
Impute
Fill in missing data values with estimated or imputed values to facilitate analysis.
Linearize
Transforming the relationship between variables to make datasets approximately linear.
Normalize
Standardize data values to facilitate comparison and analysis. Organize data into a consistent format.
Reduce
Convert a large set of data into a smaller, more manageable form without significant loss of information.
Reshape
Change the structure of data to better fit specific analysis or modeling requirements.
Serialize
Convert data into a linear format for efficient storage and processing.
Shred
Break down large datasets into smaller, more manageable pieces for easier processing and analysis.
Skew
An imbalance in the distribution or representation of data.
Split
Divide a dataset into training, validation, and testing sets for machine learning model training.
Standardize
Transform data to a common unit or format to facilitate comparison and analysis.
Tokenize
Convert data into tokens or smaller units to simplify analysis or processing.
Transform
Convert data from one format or structure to another.
Wrangle
Convert unstructured data into a structured format.
