Python Generators, Yield, and Yield From

Introduced in PEP 255, Generators are a special type of Python function that return lazy iterators. That means that the iterator does not hold its contents in memory. This comes in useful when dealing with large amounts of data and for optimizing space complexity. They’re also useful for animations, as we saw in our guide on creating sorting animations.

In this post we’re going to cover:

  • What is a Python Generator Function?
  • What is Python Yield and Yield From?
  • When to use Python Generators
  • A Summary of Python Generators

What’s a Python Generator Function?

In Python, a generator function is a special type of function that returns a “lazy iterable”. Its contents are not stored in memory. The way to generate the output is stored in memory instead. A generator function is declared just like a regular function, except instead of return, it uses the yield keyword. Here is an example of a generator function and how to access values in it.

def sample_generator(i):
    for j in range(i):
        yield j
 
for value in sample_generator(5):
    print(value)

The other way to access values in a generator function is with the next function like in the code below.

f = sample_generator(6)
print(next(f))
print(next(f))

What do the Yield and Yield From Keywords in Python do?

There are actually two ways to “return” from a generator function in Python. There’s yield and yield from. What’s the difference? yield returns from a simple generator function, yield from allows you to yield from another generator function. This comes in handy when you need to use two functions in a generator such as when we did merge sort in the sorting animations post. This could also come in handy for recursive generators.

Here’s an example of yield and yield from in action. We use the same generator from above, just wrapped inside another generator function. Notice that just like above, we can’t access individual members of the lazy iterator returned by the generator.

def sample_generator(i):
    for j in range(i):
        yield j
def yf_generator(i):
    yield from sample_generator(i)
for value in yf_generator(5):
    print(value)

When to Use a Generator Function?

When should you use a generator function? As stated above, generator functions are helpful for large datasets that you don’t want to hold all of in memory as well as animations. Let’s take a closer look at the animation case.

In the animation case, what are we doing with the generator function? We’re using it to “produce” frames. Generators are often used in a “producer/consumer” pattern. They’re produce an output to a function that consumes that output one at a time. We can see how this would be useful from a memory standpoint. We don’t need to hold as much content in memory but can still access all the data we need.

Summary of Python Generators

In summary, generators are simply Python functions that return lazy iterables. Instead of using the return keyword, all you have to do is use the yield keyword to make a function a generator. Additionally, we can also use yield from to yield from other generator functions inside of a function. We can access values in a generator function with a loop or using the next function. They save us memory space and are useful in a producer/consumer construction.

Further Reading

Learn More

To learn more, feel free to reach out to me @yujian_tang on Twitter, connect with me on LinkedIn, and join our Discord. Remember to follow the blog to stay updated with cool Python projects and ways to level up your Software and Python skills! If you liked this article, please Tweet it, share it on LinkedIn, or tell your friends!

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

Yujian Tang
Yujian Tang

I started my professional software career interning for IBM in high school after winning ACSL two years in a row. I got into AI/ML in college where I published a first author paper to IEEE Big Data. After college I worked on the AutoML infrastructure at Amazon before leaving to work in startups. I believe I create the highest quality software content so that’s what I’m doing now. Drop a comment to let me know!

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly

One thought on “Python Generators, Yield, and Yield From

Leave a Reply

%d bloggers like this: