List comprehensions, generator expressions, maps and filters

List comprehensions offer a concise way to express Python operations that involve lists. Although you can express the same logic using standard for loops and if/else conditionals, there's a high-probability you'll eventually encounter or use a list comprehension because of its compact nature. Listing A-20 illustrates a series of list comprehensions, as well as their equivalent standard syntax representation.

Listing A-20 Python list comprehensions

country_codes = ['us','ca','mx','fr','ru']
zipcodes = {90003:'Los Angeles',90802:'Long Beach',91501:'Burbank',92101:'San Diego',
                                                   92139:'San Diego',90071:'Los Angeles'}

# Regular syntax
country_codes_upper_std = []
for country_code in country_codes:
    country_codes_upper_std.append(country_code.upper())

# Equivalent ist comprehension
country_codes_upper_comp = [cc.upper() for cc in country_codes]

# Regular syntax 
zip_codes = []
for zipcode in zipcodes.keys():
    zip_codes.append(zipcode)

# Equivalent ist comprehension 
zip_codes_comp = [zc for zc in zipcodes.keys()]

# Regular syntax
zip_codes_la = []
for zipcode,city in zipcodes.items():
    if city == "Los Angeles":
        zip_codes_la.append(zipcode)

# Equivalent list comprehension 
zip_codes_la_comp = [zc for zc,city in zipcodes.items() if city == "Los Angeles"]

# Regular syntax
one_to_onehundred = []
for i in range(1,101):
    one_to_onehundred.append(i)

# Equivalent  list comprehension
one_to_onehundred_comp = [i for i in range(1,101)]

As you can see in listing A-20, the syntax for list comprehensions is [<item_result> for item in container <optional conditional>]. At first you'll need some time to get accustomed to the syntax, but eventually you'll find yourself using it regularly because it takes less time to write than equivalent regular code blocks.

The last example in listing A-20 produces a list with the numbers 1 to 100, it's an interesting example because it's an inefficient approach for most cases that require a number series. In the past section, you learned how generators can create on-demand sequences requiring little memory vs. ordinary lists that require more memory to store data. While you could use the generator syntax from the previous section, Python also has a short-handed notation for generators named generator expressions, which is illustrated in listing A-21.

Listing A-21 Python generator expressions

one_to_onehundred_genexpression = (i for i in range(1,101))

print(type(one_to_onehundred_genexpression))

# Call built-in next() to advance over the generator 
# Or generator's __next__() works the same
next(one_to_onehundred_genexpression)
one_to_onehundred_genexpression.__next__()

first_fifty_even_numbers = (i for i in range(2, 101, 2))

# Call built-in next() to advance over the generator 
# Or generator's __next__() works the same
next(first_fifty_even_numbers)
first_fifty_even_numbers.__next__()

first_fifty_odd_numbers = (i for i in range(1, 101, 2))

# Call built-in next() to advance over the generator 
# Or generator's __next__() works the same
next(first_fifty_odd_numbers)
first_fifty_odd_numbers.__next__()

As you can see in listing A-21, the syntax is almost identical to list comprehensions, the only difference is generator expressions use parenthesis () as delimiters, instead of brackets []. Once you create a generator expression, you simply call Python's built-in next() method with the reference or the generator's __next__() method to get the next element.

As helpful as list comprehensions are at reducing the amount of written code, there are two other Python constructs that operate like list comprehensions and are helpful for cutting down even more code or for cases where the logic for list comprehensions is too complex to make them understandable.

The map() function can apply a function to all the elements of a container and the filter() function can produce a new container with elements that fit a certain criteria. Listing A-22 presents some of the list comprehension from listing A-20 with the map() and filter() functions.

Listing A-22 Python map() and filter() examples

country_codes = ['us','ca','mx','fr','ru']
zipcodes = {90003:'Los Angeles',90802:'Long Beach',91501:'Burbank',92101:'San Diego',
                                                 92139:'San Diego',90071:'Los Angeles'}
# List comprehension
country_codes_upper_comp = [cc.upper() for cc in country_codes]

# Helper function
def country_to_upper(name):
    return name.upper()

# Map function 
country_codes_upper_map = [*map(country_to_upper,country_codes)]

# List comprehension 
zip_codes_la_comp = [zc for zc,city in zipcodes.items() if city == "Los Angeles"]

# Helper function
def only_la(tuple_item): 
    if tuple_item[1] == "Los Angeles":
       return True

# Filter function
zip_codes_la_filter_tuple_items = [*filter(only_la,zipcodes.items())]
print(zip_codes_la_filter_tuple_items)
zip_codes_la_filter = [tup[0] for tup in zip_codes_la_filter_tuple_items]

As you can see in listing A-22, the syntax for the map() function is map(<method_for_each_element>,<container>). This technique helps keep a cleaner design, maintaining the logic in a separate method and the invocation of said method on every element contained in the map() method itself.

The reason the map() function is prefixed with a * and wrapped in a list [] is because the map() function produces an iterator. If you just use map(country_to_upper,country_codes) the function outputs something like <map object at 0x7f77f5b59810>, but by adding a * the iterator is unpacked -- as described in the * & ** section -- immediately evaluated and added to a list [].

The syntax for filter() is filter(<method_to_evaluate_each_element>,<container>). Here again, the reason the filter() function in listing A-22 is prefixed with a * and wrapped in a list [] is because the filter() function produces an iterator. If you just use filter(only_la,zipcodes.items()) the function outputs something like <filter object at 0x7f77f5b599d0>, but by adding a * the iterator is unpacked, evaluated and the contents are added to a list [].

In listing A-22 you can see the container passed to the filter() function is a dictionary and the only_la() helper function is used to evaluate each container element and return True if the second tuple value (i.e. the dictionary value) is "Los Angeles".

Unlike the map() function which uses the results of its logic method, the filter() function only checks for True or False values. If the logic method on a container element evaluates to True, the element becomes part of the new container as is (i.e. in this case the dictionary item), if the logic method on a container element evaluates to False then the element is discarded. Finally, in listing A-22 you can see that because the filter() function returns tuple items, an additional list comprehension is made to get the desired key values (i.e. zip codes) as a list.