Back to Glossary Index

Dagster Data Engineering Glossary:

Data Munging

See 'wrangle'.

Glossary entry badge for Munge

Definition of data munging

The process of converting unstructured data into a structured format is typically referred to as "data wrangling" or "data munging". Both terms refer to the process of transforming and mapping data from a raw form into another format that allows for more convenient consumption of the data.

However, different people and organizations may use these terms in slightly different ways. For example, some people may use "data munging" to specifically refer to the cleaning and preprocessing of data, while "data wrangling" might be used in a broader sense to include tasks such as gathering and exploring the data.

Jump to the entry for 'wrangle'.

Other data engineering terms related to

‘Data Transformation’:

Align

Aligning data can mean one of three things: aligning datasets, meeting business rules, or arranging data elements in memory.

An image representing the data engineering concept of 'Align'

Clean or Cleanse

Remove invalid or inconsistent data values, such as empty fields or outliers.

An image representing the data engineering concept of 'Clean or Cleanse'

Cluster

Group data points based on similarities or patterns to facilitate analysis and modeling.

An image representing the data engineering concept of 'Cluster'

Curate

Select, organize, and annotate data to make it more useful for analysis and modeling.

An image representing the data engineering concept of 'Curate'

Denoise

Remove noise or artifacts from data to improve its accuracy and quality.

An image representing the data engineering concept of 'Denoise'

Denormalize

Optimize data for faster read access by reducing the number of joins needed to retrieve related data.

An image representing the data engineering concept of 'Denormalize'

Derive

Extracting, transforming, and generating new data from existing datasets.

An image representing the data engineering concept of 'Derive'

Discretize

Transform continuous data into discrete categories or bins to simplify analysis.

An image representing the data engineering concept of 'Discretize'

ETL

Extract, transform, and load data between different systems.

An image representing the data engineering concept of 'ETL'

Encode

Convert categorical variables into numerical representations for ML algorithms.

An image representing the data engineering concept of 'Encode'

Filter

Extract a subset of data based on specific criteria or conditions.

An image representing the data engineering concept of 'Filter'

Fragment

Break data down into smaller chunks for storage and management purposes.

An image representing the data engineering concept of 'Fragment'

Homogenize

Make data uniform, consistent, and comparable.

An image representing the data engineering concept of 'Homogenize'

Impute

Fill in missing data values with estimated or imputed values to facilitate analysis.

An image representing the data engineering concept of 'Impute'

Linearize

Transforming the relationship between variables to make datasets approximately linear.

An image representing the data engineering concept of 'Linearize'

Normalize

Standardize data values to facilitate comparison and analysis. Organize data into a consistent format.

Reduce

Convert a large set of data into a smaller, more manageable form without significant loss of information.

An image representing the data engineering concept of 'Reduce'

Reshape

Change the structure of data to better fit specific analysis or modeling requirements.

An image representing the data engineering concept of 'Reshape'

Serialize

Convert data into a linear format for efficient storage and processing.

An image representing the data engineering concept of 'Serialize'

Shred

Break down large datasets into smaller, more manageable pieces for easier processing and analysis.

Skew

An imbalance in the distribution or representation of data.

Split

Divide a dataset into training, validation, and testing sets for machine learning model training.

Standardize

Transform data to a common unit or format to facilitate comparison and analysis.

Tokenize

Convert data into tokens or smaller units to simplify analysis or processing.

An image representing the data engineering concept of 'Tokenize'

Transform

Convert data from one format or structure to another.

An image representing the data engineering concept of 'Transform'

Wrangle

Convert unstructured data into a structured format.

An image representing the data engineering concept of 'Wrangle'