# Type Hinting in Python

> Type hints help reduce bugs and improve collaboration. Learn how they work in functions, classes, and pipelines.

Source: https://dagster.io/blog/python-type-hinting
Published: 2023-08-11

![](https://dagster.io/blog-assets/37c94d333adfa1d33779.svg)

The following article is part of a series on Python for data engineering aimed at helping data engineers, data scientists, data analysts, Machine Learning engineers, or others who are new to Python master the basics. To date this beginners guide consists of:

- [**Part 1: Python Packages: a Primer for Data People (part 1 of 2)**](https://dagster.io/blog/python-packages-primer-1), explored the basics of Python modules, Python packages and how to import modules into your own projects.
- [**Part 2: Python Packages: a Primer for Data People (part 2 of 2)**](https://dagster.io/blog/python-packages-primer-2), covered dependency management and virtual environments.
- [**Part 3: Best Practices in Structuring Python Projects**](https://dagster.io/blog/python-project-best-practices), covered 9 best practices and examples on structuring your projects.
- [**Part 4: From Python Projects to Dagster Pipelines**](https://dagster.io/blog/data-engineering-in-python), we explore setting up a Dagster project, and the key concept of Data Assets.
- [**Part 5: Environment Variables in Python**](https://dagster.io/blog/python-environment-variables), we cover the importance of environment variables and how to use them.
- [**Part 6: Type Hinting**](https://dagster.io/blog/python-type-hinting), or how type hints reduce errors.
- [**Part 7: Factory Patterns**](https://dagster.io/blog/python-factory-patterns), or learning design patterns, which are reusable solutions to common problems in software design.
- [**Part 8: Write-Audit-Publish in data pipelines**](https://dagster.io/blog/python-write-audit-publish) a design pattern frequently used in ETL to ensure data quality and reliability.
- [**Part 9: CI/CD and Data Pipeline Automation (with Git)**](https://dagster.io/blog/python-ci-cd-automation), learn to automate data pipelines and deployments with Git.
- [**Part 10: High-performance Python for Data Engineering**](https://dagster.io/blog/python-high-performance), learn how to code data pipelines in Python for performance.
- [**Part 11: Breaking Packages in Python**](https://dagster.io/blog/python-breaking-packages), in which we explore the sharp edges of Python’s system of imports, modules, and packages.

[**Sign up for our newsletter**](https://dagster.io/newsletter-signup) to get all the updates! And if you enjoyed this guide check out our [**data engineering glossary**](https://dagster.io/glossary), complete with Python code examples.

One of the powerful tools Python provides to promote clear and reliable code is the concept of 'type hints'. You might wonder, "*Python is a dynamically-typed language, so why should I bother with types?*"

As a data engineer or a Python beginner interested in coding best practices, understanding and applying type hints in your Python code can be a real asset.

In this article, we will delve deeper into type hints, their applications, and their benefits in Python programming. As [Dagster is a type-annotated framework](https://dagster.io/blog/adding-python-types), we’ll also explain how types can be used in data engineering pipelines to improve its readability and make it less error-prone. It's like providing a map to your future self and other developers who may interact with your code - a map that details the types of data flowing in and out of your functions and classes.

## Table of contents

1. [**What is dynamic typing?**](https://dagster.io/blog/python-type-hinting#what-is-dynamic-typing)
2. [**Basic type hints**](https://dagster.io/blog/python-type-hinting#basic-type-hints)
3. [**Built-in types for Python**](https://dagster.io/blog/python-type-hinting#built-in-types-in-python)
4. [**Function annotations**](https://dagster.io/blog/python-type-hinting#function-annotations-1)
5. [**Complex types**](https://dagster.io/blog/python-type-hinting#complex-types)
6. [**User-defined types**](https://dagster.io/blog/python-type-hinting#user-defined-types)
7. [**Generics**](https://dagster.io/blog/python-type-hinting#generics)
8. [**Type checking with `pyright`**](https://dagster.io/blog/python-type-hinting#type-checking-with-pyright)
9. [**Type hints and docstrings**](https://dagster.io/blog/python-type-hinting#type-hints-and-docstrings)
10. [**Appendix: Common Python types**](https://dagster.io/blog/python-type-hinting#appendix-python-types)

## What is dynamic typing?

Python is a dynamically-typed language. In static-typed languages like Java or C++, you have to declare the type of variables before using them. For example, you need to specify whether a variable is an integer, a float, a string, etc. In Python, you can code without giving a second thought to data types until runtime–which is one of the features that make Python particularly beginner-friendly.

For example, you can declare a variable and directly assign a value to it without specifying its type, hence the term 'dynamically-typed'. Python interpreter implicitly binds the value and its type at runtime.

```javascript theme=classic
x = 10  # x is an integer
x = "Hello"  # now x is a string
```

In the first line, `x` is an integer. In the second line, the same `x` becomes a string. Python handles this transition seamlessly thanks to its dynamic typing nature.

However, this dynamic nature can also lead to bugs that are difficult to debug, especially in large codebases or complex data processing pipelines, where the data flow might not be immediately obvious.

Type hints, introduced in Python 3.5 as a part of the standard library via PEP 484, allow you to specify the expected type of a variable, function parameter, or return value.

##### Why use type hints?

While dynamic typing offers flexibility, it also creates room for potential bugs. Here's where type hints come in. They can significantly enhance code readability and prevent type-related errors.

**Improved code readability:** Type hints act as a form of documentation that helps developers understand the types of arguments a function expects and what it returns. This enhanced clarity makes the code more readable and easier to understand.

**Error detection:** Tools like 'pyright' and `mypy` can be used to statically analyze your Python code. It checks the consistency of types in your code based on the type hints and alerts you about type-related errors before runtime. [Learn why the Dagster team recommends skipping `mypy` entirely and just using `pyright`.](https://dagster.io/blog/adding-python-types)

**Better IDE support:** Many Integrated Development Environments (IDEs) and linters can utilize type hints to provide better code completion, error checking, and refactoring.

**Facilitates large-scale projects**: For larger projects with multiple developers, type hints can be very beneficial in understanding the structure and flow of data throughout the codebase. We’ve [published a guide](https://dagster.io/blog/adding-python-types) on how to include and maintain type annotations for public Python projects.

##### Limitations

<strong>Not enforced at runtime: </strong> Python's type hints are not enforced but are merely hints, and the Python interpreter will not raise errors if the provided types do not match the actual values. This might lead to a misconception that type hints can enforce type safety, which they cannot.

**Over-complicated**: For small or simple scripts, type hints might seem like an overkill, and could potentially complicate code that should be straightforward and simple.

**Not flexible:** One of the reasons for Python's popularity is its dynamic nature and type hints can restrict this.

## Basic type hints

Python's `typing` module contains several functions and classes that are used to provide type hints for your Python code. Here's how you can apply type hints in different scenarios.

##### Declare types for variables

To provide type hints for variables, you can use the colon `:` symbol followed by the type. Here's an example:

```javascript theme=classic
age: int = 20
name: str = "Alice"
is_active: bool = True
```

Here, `age` is hinted as an integer, `name` as a string, and `is_active` as a boolean.

##### Function annotations

You can provide type hints for function parameters and return values. This helps other developers understand what types of arguments are expected by the function and what type the function returns.

`def greet(name: str) -> str: \    return f"Hello, {name}`"

In this example, the function `greet` expects `name` to be a string and will return a string.

## Built-in types in Python

Python has several built-in types. The most commonly used are:

- `int`: Represents an integer
- `float`: Represents a floating-point number
- `bool`: Represents a boolean value (True or False)
- `str`: Represents a string

There are also complex types such as lists, tuples, and dictionaries that can be used to provide more detailed type hints that we will look at later on.

You will also find a list of Python's main types in the [appendix](#).

##### Atomic vs. composite types

In Python, there is a distinction between atomic and composite types when it comes to type hinting. Atomic types, such as `int`, `float`, and `str`, are simple and indivisible, and their type annotations can be provided directly using the type itself, like `str`.

```python theme=classic
def my_function(my_string: str) -> int:
    return len(my_string)
```

On the other hand, composite types like **`List`** and **`Dict`** are composed of other types, and before Python 3.9, they often required importing specific definitions from the typing module, such as **`typing.List[int]`** for a list of integers.

```python theme=classic
from typing import List

def my_function(numbers: List[int]) -> int:
    return sum(numbers)
```

[In newer versions of Python](https://peps.python.org/pep-0585/), you can write `list[str]` instead of `typing.List[int]`.

## Function annotations

Type hints can be particularly useful when incorporated into function signatures. This not only allows developers to understand what types of arguments a function expects but also gives them an idea of what the function will return.

##### How to specify argument types and return type of a function

You can specify the types of arguments and the return type of a function using the `:` symbol for the arguments and the`->` symbol for the return type. Here's the general syntax:

```python theme=classic
def function_name(arg1: type1, arg2: type2, ...) -> return_type:
    # function body
```

In this syntax, `arg1`, `arg2`, etc. are the function arguments, and `type1`, `type2`, etc. are the types of these arguments. `return_type` is the type of value the function returns.

##### Examples of using type hints in function signatures

Let's consider a function that calculates the area of a rectangle:

```python theme=classic
def area_rectangle(length: float, breadth: float) -> float:
    return length * breadth
```

In this function, `length` and `breadth` are expected to be floats, and the function also returns a float. The function will still work if you pass integers or even strings that can be converted to a float, but the type hint makes it clear that it's designed to handle floating-point numbers.

Another example can be a function that accepts a list of integers and returns their sum as an integer:

```python theme=classic
def sum_elements(numbers: list[int]) -> int:
    return sum(numbers)
```

`‍`In this example, the `numbers` parameter is hinted as a list of integers, and the return type is an integer.

Note that these type hints do not enforce type checking at runtime. They are hints for developers, and Python will not raise a `TypeError` if the actual types do not match the specified types.

## Complex types

The typing module in Python provides several classes that can be used to provide more complex type hints. Below are some of the most commonly used classes:

### List, dict, tuple, set

The `list`, `dict`, `tuple`, and `set` classes can be used to provide type hints for lists, dictionaries, tuples, and sets respectively. They can be parameterized to provide even more detailed type hints.

```python theme=classic
### A list of integers
numbers: list[int] = [1, 2, 3]

### A dictionary with string keys and float values
weights: dict[str, float] = {"apple": 0.182, "banana": 0.120}

### A tuple with an integer and a string
student: tuple[int, str] = (1, "John")

### A set of strings
flags: set[str] = {"apple", "banana", "cherry"}
```

In these examples, `numbers` is hinted as a list of integers, `weights` is a dictionary with string keys and float values,  `student` is a tuple with integers and a string, and `flags` is a set of strings.

### Optional

The `Optional` type hint can be used to indicate that a variable can be either a specific type or `None`.

```python theme=classic
from typing import Optional

def find_student(student_id: int) -> Optional[dict[str, str]]:
    # If the student is found, return a dictionary containing their data
    # If the student is not found, return None
```

### Union

The `Union` type hint is used to indicate that a variable can be one of several types. For example, if a variable can be either a `str` or an `int`, you can provide a type hint like this:

```python theme=classic
from typing import Union

def process(data: Union[str, int]) -> None:
    # This function can handle either a string or an integer
```

In newer versions of Python, you can use the pipe (|) operator to indicate a type that can be one of several options, replacing the need for `Union`:

```python theme=classic
def process(data: str | int) -> None:
    # This function can handle either a string or an integer
```

### Any

The `Any` class is used to indicate that a variable can be of any type. This is equivalent to not providing a type hint at all.

```python theme=classic
from typing import Any

def process(data: Any) -> None:
    # This function can handle data of any type
```

These tools from the `typing` module can help you provide detailed type hints that make your code easier to understand and debug.

However, remember that Python's type hints are optional and not enforced at runtime. They are intended as a tool for developers, not a way to enforce type safety.

## User-Defined types

In Python, you can define your own types using classes, which is the fundamental mechanism to create custom types. You can use these classes in type hints just like you'd use the built-in types. The `typing` module also provides additional tools for creating more specific types, including `Type` and `NewType`.

### Defining your own types using classes

You can create a class and use it as a type hint. Here's an example:

```python theme=classic
class Student:
    def __init__(self, name: str, age: int):
        self.name = name
        self.age = age

def print_student_details(student: Student) -> None:
    print(student.name, student.age)
```

`Student` is a user-defined type, and it's used as a type hint in the `print_student_details` function.

### Using `Type` for type hinting

The `Type` class from the `typing` module can be used to indicate that a variable will be a class, not an instance of a class. This is commonly used when a function argument is expected to be a class, for example in factory functions.

```python theme=classic
from typing import Type

def create_student(cls: Type[Student], name: str, age: int) -> Student:
    return cls(name, age)
```

In this example, `create_student` expects a `Student` class (or a subclass of `Student`) as its first argument.

### Using `NewType` to create distinct types

`NewType` is used to create distinct types. It's useful when you want to distinguish between two types that would otherwise be the same.

For example, let's say you're dealing with student IDs and course IDs in your program, and you want to make sure you don't mix them up. Both are represented as integers, so you can use `NewType` to create two distinct types:

```python theme=classic
from typing import NewType

StudentID = NewType('StudentID', int)
CourseID = NewType('CourseID', int)

def get_student(student_id: StudentID) -> None:
    # Fetch student data...

def enroll_in_course(student_id: StudentID, course_id: CourseID) -> None:
    # Enroll the student in the course...
```

Even though `StudentID` and `CourseID` are both integers, they are considered distinct types and cannot be used interchangeably. However, remember that this check is not enforced at runtime, but during static type checking using tools like `mypy`.

## Generics

Generics allows you to define a function, class, or data structure that works with different types. The `Generic` class and the `TypeVar` function from the `typing` module are used to define generic types. For example, a list is a generic data structure because it can contain elements of any type.

### TypeVar

TypeVar is used to define a type variable, which can be any type, and the specific type is determined by the client code. Here's an example:

```python theme=classic
from typing import TypeVar

T = TypeVar('T')

def first_element(lst: List[T]) -> T:
    return lst[0]
```

Here, `T` is a type variable that can be any type. The `first_element` function works with a list of any type and returns an element of that type. The specific type of `T` would be determined by the list you pass to the function.

### Generic

`Generic` is used to define generic classes. A generic class can be initialized with a variety of types, and those types are used in type hints within the class.

```python theme=classic
from typing import Generic, TypeVar

T = TypeVar('T')

class Box(Generic[T]):
    def __init__(self, value: T):
        self.value = value

    def get(self) -> T:
        return self.value
```

Here, `Box` is a generic class that works with any type `T`. When you create an instance of `Box`, you can specify the type of `T`, and that type is used in the `value` attribute and the `get` method.

```python theme=classic
box1 = Box[int](10)
box2 = Box[str]("Hello")
```

`box1` is a `Box` that contains an integer, and `box2` is a `Box` that contains a string.

## Type checking with `pyright`

A type checker like `pyright` is a tool used to enforce type hinting in Python. At Dagster, [we really like `pyright`](https://dagster.io/blog/adding-python-types) because it is faster than other alternatives such as `mypy`.

Python itself is a dynamically-typed language, which means type checks happen at runtime and it does not enforce type hinting rules. If you try to perform an operation that's not supported for a given data type, Python will raise an error during runtime. For example, calling an undefined method on an object will only trigger an error during runtime.

However, when developing large or complex systems, enforcing type consistency can help catch potential bugs early. `pyright` performs static type checking, meaning it checks the types of your variables, function arguments, and return values before the code is actually run. It uses the type hints you've provided in your code to do this. It's important to understand that `pyright` does not execute or run your code; it simply reads and analyzes it.

### How to use a type checker to verify your types

To use `pyright`, you first need to install it:

```javascript theme=classic
pip install pyright
```

Then, to check a Python file, you run `pyright` with the file as an argument:

```javascript theme=classic
pyright my_file.py
```

Pyright will then analyze the file and report any type errors it finds.

For example, if you have a function that's annotated to receive a `str` as an argument and you pass an `int`, `pyright` will catch this.

### Static vs dynamic type checking

Static type checking is the process of verifying the type safety of a program based on analysis of a program's text (source code). Static type checking is done at compile-time (before the program is run). Languages that enforce static type checking include C++, Java, and Rust.

Dynamic type checking, on the other hand, is the process of verifying the type safety of a program at runtime. Dynamic type checking occurs while the program is running. Languages that use dynamic type checking include Python, Ruby, and JavaScript.

In static type checking, types are checked before the program runs, which makes it easier to catch and prevent type errors. This makes the program safer to run, as most type-related bugs have been caught at compile-time. However, it also requires the programmer to explicitly declare the types of all variables and function return values, which can be seen as reducing flexibility.

Dynamic type checking provides more flexibility, as you don't have to explicitly declare the type of every variable. However, this also means that type errors can occur at runtime, which could potentially cause the program to crash.

Python is a dynamically-typed language, but it also supports optional static type checking through tools like `pyright` and type hints. This provides Python programmers with a unique flexibility, allowing them to choose when they want the safety of static type checking and when they prefer the flexibility of dynamic typing.

## Type hints and docstrings

Type hints, as we've discussed, indicate the types of variables, function parameters, and return values. They can help other developers understand what types of data your function expects and what it will return.

Docstrings, on the other hand, are used to provide a description of what your function, class, or module does. A docstring can include a description of the function's purpose, its arguments, its return value, and any exceptions it may raise.

Here's an example of how you can use type hints and docstrings together:

```python theme=classic
def filter_and_sort_products(products: list[dict[str, int]], attribute: str, min_value: int) -> list[dict[str, int]]:
    """
    Filters a list of products by a given attribute and minimum value, and then sorts the filtered products by the attribute.

    Args:
        products (list[dict[str, int]]): A list of products represented as dictionaries.
        attribute (str): The attribute to filter and sort by.
        min_value (int): The minimum acceptable value of the specified attribute.

    Returns:
        list[dict[str, int]]: A list of filtered and sorted products.

    Raises:
        KeyError: If the specified attribute is not found in any product.

    Examples:
        >>> products = [{"name": "Apple", "price": 10}, {"name": "Banana", "price": 5}]
        >>> filter_and_sort_products(products, "price", 6)
        [{"name": "Apple", "price": 10}]
    """
    filtered_products = [product for product in products if product[attribute] >= min_value]
    return sorted(filtered_products, key=lambda x: x[attribute])

```

Here, the function signature shows that the function takes a list of dictionaries representing products, a string representing an attribute, and an integer representing a minimum value. It returns a list of filtered and sorted dictionaries.

The doc strings explain the purpose of the function, its parameters, return value, possible exceptions (such as a `KeyError` if the given attribute is not present), and includes an example of how to call the function.

This combination of type hints and docstrings can greatly improve the readability and maintainability of your code.

### Conclusion

Building on Python programming best practices, we’ve looked at how type hints improve the readability and maintainability of your code.

If you have any questions or need further clarification, feel free to join the Dagster Slack and ask the community for help. Thank you for reading!

Our next article builds on these data engineering concepts and explores how [Factory Patterns help you automate steps in your pipeline](https://dagster.io/blog/python-factory-patterns).

[Sign up for our newsletter to stay in the loop](https://dagster.io/newsletter-signup)!

## Appendix: Python Types

To summarize, here are the most common built-in data types in Python. You might also come across or utilize custom data types from external libraries or those defined by other developers.

### 1. Numeric Types

- `int` – Integer\
  *Examples:* `5`, `-3`, `42`
- `float` – Floating-point number\
  *Examples:* `3.14`, `-0.001`, `2.71`
- `complex` – Complex number\
  *Examples:* `3+4j`, `2-5j`

### 2. Text Type

- `str` – String\
  *Examples:* `"Hello, World!"`, `'Python'`

### 3. Sequence Types

- `list` – List\
  *Examples:* `[1, 2, 3]`, `["a", "b", "c"]`
- `tuple` – Tuple\
  *Examples:* `(1, 2, 3)`, `("a", "b", "c")`
- `range` – Range object\
  *Examples:* `range(5)`, `range(0, 5, 2)`

### 4. Mapping Type

- `dict` – Dictionary\
  *Examples:* `{"name": "John", "age": 30}`, `{1: "one", 2: "two"}`

### 5. Set Types

- `set` – Mutable set\
  *Examples:* `{1, 2, 3}`, `{"apple", "banana", "cherry"}`
- `frozenset` – Immutable set\
  *Created with:* `frozenset(["a", "b"])`

### 6. Boolean Type

- `bool` – Boolean\
  *Values:* `True`, `False`

### 7. Binary Types

- `bytes` – Immutable sequence of bytes\
  *Examples:* `b'hello'`, `bytes([65, 66, 67])`
- `bytearray` – Mutable sequence of bytes\
  *Example:* `bytearray([65, 66, 67])`
- `memoryview` – Memory view object\
  *Created with:* `memoryview(b'abc')`

### 8. None Type

- `NoneType` – Represents the absence of a value\
  *Only value:* `None`

## Other Useful Modules & Types

### 1. `datetime` Module

- `datetime.date` – Represents a date
- `datetime.datetime` – Represents date and time
- `datetime.time` – Represents a time of day
- `datetime.timedelta` – Duration or difference between two dates/times
- `datetime.tzinfo` – Base class for time zone info

### 2. `collections` Module

- `namedtuple` – Factory function for creating tuple subclasses with named fields
- `deque` – Double-ended queue
- `Counter` – A dict subclass for counting hashable objects
- `OrderedDict` – Dict that remembers insertion order
- `defaultdict` – Dict that provides default values for missing keys

### 3. `array` Module

- `array.array` – Space-efficient array with type specification

### 4. `struct` Module

- Used for packing and unpacking binary data

### 5. `json` Module

- Tools for encoding and decoding JSON data

### 6. `enum` Module

- `Enum` – Base class for creating enumerated constants
- `IntEnum` – Enumerations that are also subclasses of `int`