Python Generators

Overview

Python generators are a type of iterator in Python, with the important caviat that you can only iterate over them once.

They are created by writing a function what uses the keyword yield rather than return.

Type Annotations

Generators can be annotated with the Generator type that is defined in the typing module:

Generator[yield_type, send_type, return_type]

Code Examples

The following code example chunks up a list:

def chunkify(l: List[Any], n: int) -> Generator[List[Any], None, None]:
    """
    Provides a generator which provides chunks of size n of the list.

    Args:
        l: The list to chunk up.
        n: The number of list elements to include in each chunk. Each chunk will be of length n,
            except for the last chunk, which may be from 1 to length n depending on the number
            of remaining elements.

    Returns:
        A generator which, when iterated over, provides the chunks.
    """

    for i in range(0, len(l), n):
        # Make sure we don't go over the bounds of the iterable on the last chunk
        upper_bound = min(i + n, len(l))
        yield l[i : upper_bound]