Video summary
The primary distinction between iterables and iterators lies in their specific roles within Python's iteration process. An iterable is defined as any object that can be looped over, such as a list or a generator expression; essentially, it is anything capable of being passed to the built-in `iter()` function. In contrast, an iterator is a specialized object returned by `iter()` that maintains internal state and knows exactly where it currently stands in the sequence. While all iterators are inherently iterable because they can be looped over, not all iterables are iterators; for instance, a standard list is an iterable but fails to work with the built-in `next()` function unless explicitly converted into an iterator first.
To illustrate how these concepts interact practically, consider that when you use a generator expression or call `iter()` on a list, Python creates a new object specifically designed to yield one element at a time upon request via the `next()` function. This iterator acts like a manager for the iteration process, remembering the last index accessed so it can return the subsequent item without needing to recalculate values from scratch every time. A crucial aspect of this mechanism is that modifying the underlying iterable after creating an iterator does not affect the iterator's internal state; if you change elements in the original list while iterating, the iterator will still yield the specific items corresponding to its stored indices rather than reflecting real-time changes to the data structure it was created from.
Behind the scenes of a standard `for` loop, Python automatically handles this complexity by first converting any given iterable into an iterator using the `iter()` function and then wrapping that logic in an indefinite loop similar to a `while` statement. The loop repeatedly calls `next()` on the iterator until it raises a "StopIteration" exception, signaling that all elements have been exhausted at which point the loop terminates naturally. This design allows Python programmers to write concise one-line loops while abstracting away the manual management of state and index tracking required in lower-level languages or when manually implementing iteration with `while` loops and explicit try-except blocks.
Ultimately, understanding this relationship clarifies why generators are so powerful yet distinct from simple lists; they provide a way to create iterators on demand without storing all data in memory at once, adhering strictly to the principle of yielding only the next element when asked. By grasping that iterables serve as containers or sources while iterators act as the active agents driving the loop forward, developers can better utilize built-in functions like `map`, `filter`, and generators to write more efficient code. This foundational knowledge demystifies Python's iteration mechanics, revealing that every time a `for` loop is used in daily programming tasks, it is implicitly relying on these underlying iterator protocols to function correctly.
Read the full video transcript
in this video i'm going to explain to
you
what is the difference between the
concept of an iterator and the concept
of an iterable
those two are often confused it's
actually quite easy to understand a
difference
so i'm going to give you some background
and then it should be straightforward
so let's create a new file and call it
iterators versus iterables
okay so let's go ahead and first
create a list of numbers and we are
always as always going to use the same
number so
11 11 8 5
3 then we have 12 and 2.
6 9 and 10 and 1 and 4.
okay so numbers is a list object
and we said that any object over which
we can loop
is considered an interval okay so
in the so until now let's put it this
way
until now the definition was
an interval
any object over which
we can loop that is our working
definition up to now
so just to prove it to you it's going to
be um
some almost boring if i say four number
in numbers
just to illustrate the point we can loop
over numbers
okay so that's an interval now the
concept of an iterator is different
the concept of an iterator what we have
seen in previous videos
is it is any object that can do that
understands the next concept or it can
work with the
python's built in next function so let's
say
an iterator
is any object
that may be passed
as an argument
to the built-in
next function
so an example of that would be the
following let's go ahead and
get an another example of the concept of
a generator and
also review the generate expression
syntax
so let's go ahead and simply write a
generator with parentheses here
and we're simply going to write n
squared
for n in numbers
and maybe let's also go ahead and say if
n
or if yeah if the number divided by 2
has no rest so if the number is even
that creates a generator object
and a generator object
can be passed to the next built-in so
let's do that and i get
the square of the first number
um here that is even so the first even
number is 8
and the square of that is 64. that is
why we see 64. if i run it
some second time i see 144
because 12 is the next even number and
its square is simply 144.
okay so let's try and see what happens
if i pass
to the next function the numbers list
i get a type error so what do we learn
from that we learn from that
that lists are not iterators
okay so that is already one difference
you know an iterable is not necessarily
an iterator
the contrary of course however is true
so let's go ahead and do the following
let's say for square singular
in chen china is the generator which
creates
squares let's simply go ahead
and print the square and let's do so
of course only on one line
and we see the numbers 4 36
and so on so why 4 well i executed this
cell here already twice
so i will go and do the next so we've
already got
the equivalent the mapped value to 8 and
12
and the next even number would be 2 here
and the square of 2 would be 4. this is
why we see 4.
then we see the next number is 6 the
square of that would be 36
okay so what we what we want to remember
from that here
is that a generator can only go in one
direction
so as we already got the first two
numbers out of it
then the for loop um can only continue
where the generator has
left off before however what we see from
that
code cell here is that we can indeed
loop over an iterator
okay so what we could note down here is
generators are iterable
okay so there is um
of course a connection between the two
and the connection is as follows
an interval is any object over which we
can loop and the iterator
is the thing that makes it loop so let's
go ahead and see what i mean by that
so i told you that an iterable is
anything we can loop over and the
iterator is what makes it loop so let's
go ahead and
create an iterator out of an iterable
so numbers is
as we discussed an interval because i
can loop over it
how can i get an iterator out of numbers
well there is a built-in function that
we have not seen before it is called
iter and before i execute that
let's go to the python documentation
under built-in functions
and see what the documentation says
so it says we are given any object
and it says return an iterator object
and so on
so that is what the interfunction does
so let's do that
and that's let's store that
in a variable let's call it list
iterator
so first of all let's look at the type
of list iterator
and the type of list iterator happens to
be a list iterator so i used the name of
the type
as the variable name so maybe a better
way to do that is to maybe simply call
it it
as a short version and then the type of
it is of course
also a list iterator and so now what can
it
do well it is an iterator so if i say
next and i pass through it it as the
argument i get back to number seven
y7 well obviously
the number 7
is the first number in the list so
let's go ahead and i call next one more
time
and i get the number 11. so indeed the
inter function
takes an interval as its input and it
gives me back an object
that makes the iteration work so that
basically what the it object here is it
is an object
a rule in memory that remembers the last
number you pulled out of the list
and it always remembers the last one and
when you call next with it
then really what's going on is we you
just get the next number
and really to be even more go in into
more detail
what it does it re it simply remembers
the last index or the index of the last
element you pulled out
so let's do something to to play some
tricks with it
so let's maybe go ahead and
now we did the last number i pulled out
was the number 11
so the third number in here is the
number eight which is index
two so let's overwrite that number
with to be 99 for example if i now go
ahead and say
next it then i'm going to see the number
99
okay so in other words the list
and the list iterator are two different
objects
that may be confusing so at some point
so maybe
let's do the following let's go ahead
and
also use python tutor to illustrate that
it is really quite simple but
just to make sure that you get it i put
a list here
and let's go down and create the
iterator
in a separate line here and
let's go ahead and first numbers is a
full list object
and the it object is simply a list
iterator instance it says here
so it's a second object and all the
object does is it remembers where in the
iteration you are
okay the iterator is kind of like the
manager that manages the iteration
process
and if we change the list the underlying
list
then the iterator does not even know
that know about that so just as we saw
in jupyter lab here when i change some
of the underlying elements in the list
the iterator simply returns it in this
in this case here
okay so that's the the
the difference between iterate and
interval so maybe
i give you a better a better definition
and iterable if any function or
any object sorry that may be passed
as an argument
to the built-in
iter function
okay so that is probably the better
definition but
as a beginner when you start out with
python um i don't want to confuse you so
i gave you the easy definition
an interval is any object over which i
can loop that's kind of the approximate
definition
and now the precise definition is simply
an issue with any object
that may be passed to the ether function
which i did
right here and then i get back an
iterator
and what is an iterator well an iterator
is any object
that i can pass to the built in next
function
okay and we have seen a couple of
iterators already we have seen map
objects we have seen filter objects
we have seen generator objects these are
all iterators and now we have a fourth
example
of an iterator which happens to be
the list iterator so maybe to conclude
this video
i will explain to you how the for loop
really works
behind the scenes
so in many other programming languages
the for loop is not as flexible as
python's for loop so why is python's for
loops are flexible
the reason is behind the scenes it does
the following so let's assume i have
given a numbers list
and let's say i want to write the
following four number
in numbers
and then simply print number
and let's put all of that on one line
so that is the for loop and now let's
rewrite that
using a while loop and see what really
goes on behind the scenes and python
so what goes on is as follows python
first goes ahead
and creates an iterator out of the
numbers object
so right here this is the target numbers
is the target of the for loop over which
we loop
so this the target is used as the
argument
behind the scenes and passed to the
inter function this gives me back an
iterator object
and then what is going to happen is the
following python
behind the scenes will initialize or
start
an indefinite loop you remember that
from the chapter on
indefinite loop and on where we talked
about guessing games
so it creates a while loop a while
through loop behind the scenes
and then in the wild trudeau we have a
try statement and you will see why in a
bit
and we are going to go ahead and in this
case because we
we call this variable here number
singular i'm also going to call it right
here number singular
and how do we get number singular well
by simply asking
what is the next element that the
iterator that we created before the
while loop
returns now we know that
the next function at some point will
give us
a so called stop iteration exception
whenever the iterator is exhausted
then we get the uh the red error message
where it says stop iteration
and now this one we have to accept it so
we don't want to see the stop iteration
exception so maybe let's do that
so accept stop iteration here
and what do we want to do when we see
the stop iteration exception
well we want to stop the for loop right
so in other words we want to break out
of the while loop here and then
there is of course also an else clause
the else clause is whatever happens when
no exception is triggered and here we
will simply
put the code to be repeated so maybe
right here code to be repeated
because for loops are all about
repeating code and of course the while
loop is also about
repeating code and let's do it like that
and we see we get the exact same outcome
here
okay and that is not an accident so in
other words
when we in chapter four we talked about
the idea that
the for loop is really just a special
case of a while loop that is what i told
you
and here you see why so um whenever we
initialize a for loop in python what
goes on behind the scenes is really a
while loop
and the python goes ahead and
calls the interfunction with the thing
we want to loop over the interval
and then it uses the iterator to loop
over everything and
implement the loop so um we see here
by writing a for loop i have one line
for managing the iteration
and then i have the line that is
repeated inside the for loop and here i
would have
i don't know two five six it's seven
lines seven lines of code
that do the same thing so really the for
loop is just a priv
an abbreviation for all of the things
you can read here
okay so you don't have to know that this
is something that you will never see in
practice but this is just to
get you give you some further example of
how iterators and
iterables um yeah relate to each other
okay so what we learned from that is an
iterable is anything we can loop over
but more precisely an interval is
anything that gives him back an iterator
and the iterator is the thing the rule
in memory that makes the looping work
and when we use a for loop everything is
done
for us by python but as we saw with the
generator expression
and also the map and filter types we can
also use these
rules in memory ourselves without a for
loop
okay so at the end of the day from day
one in this course you have already
been using iterators all the time every
time you use the for loop you were using
iterators
and now in chapter 8 you learned what
they are and
i know that iterators or generators in
particular
in the beginning they are a bit scary to
to beginners but really
this is just how python is built so
iterators
generally speaking are just rules that
know one thing and one thing only
give me back the next element in the
line
okay so that is the comparison between
iterators and intervals
so you remember that intervals are not
iterators but all iterators are always
iterable
okay because we saw here that we can
loop over an integral and an iterator of
course
okay so um i hope i did not mix up the
two words
um in this video it's quite hard to
speak about that
but yeah the concepts are now i guess
pretty clear
so i will see you in the next video
where we compare the sorted and the
reverse built-ins
because all um some one of them also has
um to do with an iterator behind the
scenes okay
so i will see you in the next video