python - When do you use iterators and when generators? -

i read question , answers differences between iterators , generators. don't understand when should choose 1 on other? know examples (simple, real life ones) when 1 better other? thank you.

iterators provide efficient ways of iterating on existing data structure.

generators provide efficient ways of generating elements of sequence on fly.

iterator example

python's file readers can used iterators. might use process 1 line of file:

with open('file.txt', 'rb') fh:     lines = fh.readlines()  # reads entire file lines,     line in lines:         process(line)       # done on each line 

you can implement more efficiently using iterators

with open('file.txt', 'rb') fh:     line in fh:         # read as needed, each time         process(line) 

the advantage in fact in second example, you're not reading entire file memory, iterating on list of lines. instead, reader (bufferedreader in python3) reading line @ time, every time ask one.

generator example

generators generate elements of sequence on fly. consider following:

def fib():     idx  = 0     vals = [0,1]     while true:         # if need compute new value, on fly         if len(vals) <= idx: vals.append(vals[-1] + vals[-2])         yield vals[idx]         idx += 1 

this example of generator. in case, every time it's "called" produces next number in fibonacci sequence.

i put "called" in scare quotes because method of getting successive values generators different traditional function.

we have 2 main ways values generators:

iterating on it

# print fibonacci sequence until event occurs f in fib():     print(f)     if f > 100: break 

here use in syntax iterate on generator, , print values returned, until value that's greater 100.

output:

 0 1 1 2 3 5 8 13 21 34 55 89 144 

calling next()

we call next on generator (since generators iterators) , (generate and) access values way:

f = fib()  print(next(f))  # 0 print(next(f))  # 1 print(next(f))  # 1 print(next(f))  # 2 print(next(f))  # 3 

there more persuasive examples of generators however. , these come in form of "generator expressions", related concept (pep-289).

consider following:

first = any((expensive_thing(i) in range(100))) 

here, we're creating generator expression:

(expensive_thing(i) in range(100)) 

and passing any built-in function. any return true element of iterable determined true. when pass generator function any, call expensive_thing(i) many times necessary find true-ish value.

compare using list comprehension passed any:

first = any([expensive_thing(i) in range(100)]) 

in case, expensive_thing(i) called all values of i, first, 100-element list of true/false values given any return true if finds true-ish value.

but if expensive_thing(0) returned true, better approach evaluate that, test it, , stop there. generators allow this, whereas list comprehension not.

---

consider following example, illustrating advantage of using generator expression on list comprehension:

import time  def expensive_thing(n):     time.sleep(0.1)     return 10 < n < 20  # find first true value, using generator expression t0 = time.time() print( any((expensive_thing(i) in range(100))) ) t1 = time.time() td1 = t1-t0  # find first true value, using list comprehension t0 = time.time() print( any([expensive_thing(i) in range(100)]) ) t1 = time.time() td2 = t1-t0  print("td 1:", td1)  # td 1:  1.213068962097168 print("td 2:", td2)  # td 2: 10.000572204589844 

the function expensive_thing introduces artificial delay illustrate difference between 2 approaches. second (list comprehension) approach takes longer, because expensive_thing evaluated @ all 100 indices, whereas first calls expensive_thing until finds true values (i=11).