Python Iterators

Every for loop in Python uses an iterator behind the scenes. Knowing how iterators work explains why some objects can be looped over only once, and lets you write your own objects that work with for.

Iterable vs iterator

• An iterable is anything you can loop over: a list, string, tuple, dict, set, file or range.
• An iterator is the object that actually hands out the items, one at a time, and remembers its position.

iter() gets an iterator from an iterable, and next() asks it for the next item:

colors = ["red", "green", "blue"]it = iter(colors) print(next(it))print(next(it))print(next(it))

red
green
blue

When there are no items left, next() raises StopIteration:

it = iter([1])print(next(it))print(next(it))

1
Traceback (most recent call last):
  File "main.py", line 3, in <module>
    print(next(it))
          ~~~~^^^^
StopIteration

Pass a default to next() to avoid the error:

it = iter([])print(next(it, "no more items"))

no more items

How a for loop works

A for loop is a while loop that calls iter() once and next() until StopIteration:

items = ["a", "b", "c"] it = iter(items)while True:    try:        item = next(it)    except StopIteration:        break    print(item)

a
b
c

Iterators are used up

An iterator moves forward only. Once it is exhausted it stays empty. A list can be looped over many times because each for loop asks it for a fresh iterator.

squares = map(lambda x: x * x, [1, 2, 3])    # map returns an iteratorprint(list(squares))print(list(squares))      # already used up nums = [1, 2, 3]print(sum(nums), sum(nums))   # a list gives a new iterator each time

[1, 4, 9]
[]
6 6

Write your own iterator

A class becomes an iterator by defining __iter__() (returning itself) and __next__() (returning the next item or raising StopIteration):

class Countdown:    def __init__(self, start):        self.current = start     def __iter__(self):        return self     def __next__(self):        if self.current <= 0:            raise StopIteration        value = self.current        self.current -= 1        return value for n in Countdown(3):    print(n)print(list(Countdown(5)))

3
2
1
[5, 4, 3, 2, 1]

Infinite iterators and itertools

Iterators can be infinite because they produce values only when asked. The itertools module has many ready-made ones:

import itertools counter = itertools.count(start=100, step=10)print([next(counter) for _ in range(3)]) print(list(itertools.islice(itertools.cycle("AB"), 5)))print(list(itertools.chain([1, 2], (3, 4), "56")))print(list(itertools.combinations("ABC", 2)))print(list(itertools.product([0, 1], repeat=2)))

[100, 110, 120]
['A', 'B', 'A', 'B', 'A']
[1, 2, 3, 4, '5', '6']
[('A', 'B'), ('A', 'C'), ('B', 'C')]
[(0, 0), (0, 1), (1, 0), (1, 1)]

Built-ins that return iterators

map(), filter(), zip(), enumerate(), reversed() and open() all return iterators. They are lazy: no work happens until you ask for items.

pairs = zip(["a", "b", "c"], [1, 2, 3])print(next(pairs))print(list(pairs))       # continues from where next() left off

('a', 1)
[('b', 2), ('c', 3)]

Check your understanding

it = iter("abc")next(it)print(list(it), list(it))

['b', 'c'] []

Evens(10)

[0, 2, 4, 6, 8, 10]

class Evens:    def __init__(self, limit):        self.limit = limit        self.n = 0     def __iter__(self):        return self     def __next__(self):        if self.n > self.limit:            raise StopIteration        value = self.n        self.n += 2        return value print(list(Evens(10)))

[0, 2, 4, 6, 8, 10]

Program Practice

Python iterators FAQ