Python Generators, Yield, and Yield From

Introduced in PEP 255, Generators are a special type of Python function that return lazy iterators. Python yield‘s generators. A lazy iterator is one that does not hold its contents in memory. This is especially useful when dealing with large amounts of data and for optimizing space complexity. They’re also useful for animations, as we saw in our guide on creating sorting animations.

In this post we’re going to cover:

• What is a Python Generator Function?
• What is Python Yield and Python Yield From?
• When to use Python Generators
• A Summary of Python Generators

What’s a Python Generator Function?

In Python, a generator function is a special type of function that returns a “lazy iterable”. Its contents are not stored in memory. The way to generate the output is stored in memory instead. A generator function is declared just like a regular function, except instead of return, it uses the yield keyword. Here is an example of a generator function and how to access values in it.

def sample_generator(i):
    for j in range(i):
        yield j
 
for value in sample_generator(5):
    print(value)

The other way to access values in a generator function is with the next function like in the code below.

f = sample_generator(6)
print(next(f))
print(next(f))

What is Python Yield and Python Yield From

There are actually two ways to “return” from a generator function in Python. There’s yield and yield from. What’s the difference? yield returns from a simple generator function, yield from allows you to yield from another generator function. This comes in handy when you need to use two functions in a generator such as when we did merge sort in the sorting animations post. This could also come in handy for recursive generators.

Here’s an example of yield and yield from in action. We use the same generator from above, just wrapped inside another generator function. Notice that just like above, we can’t access individual members of the lazy iterator returned by the generator.

def sample_generator(i):
    for j in range(i):
        yield j
def yf_generator(i):
    yield from sample_generator(i)
for value in yf_generator(5):
    print(value)

When to Use a Generator Function?

When should you use a generator function? As stated above, generator functions are helpful for large datasets that you don’t want to hold all of in memory as well as animations. Let’s take a closer look at the animation case.

In the animation case, what are we doing with the generator function? We’re using it to “produce” frames. Generators are often used in a “producer/consumer” pattern. They’re produce an output to a function that consumes that output one at a time. We can see how this would be useful from a memory standpoint. We don’t need to hold as much content in memory but can still access all the data we need.

Summary of Python Generators, Yield, and Yield From

In summary, generators are simply Python functions that return lazy iterables. Instead of using the return keyword, all you have to do is use the yield keyword to make a function a generator. Additionally, we can also use yield from to yield from other generator functions inside of a function.

We can access values in a generator function with a loop or using the next function. They save us memory space and are useful in a producer/consumer construction.

Further Reading

• Python Dotenv for Environment Variables
• Python Multiprocessing with Arguments
• Dijkstra’s Algorithm in Python
• Build an AI Text Summarizer
• Web Scraping with Python, Selenium, and BeautifulSoup 4

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.