Earlier this week I posted a quiz about how many iterators there are in this code

numbers = [2, 5, 10, 3, 99, 42]
for number in numbers:
    for another_number in numbers:
        print(number, another_number)

…let's explore with the help of this visual aid

A short series

Here's the code again.

The poll gave four options for possible answers for how many iterators there are in this code:
1
2
7
12

Here's the poll, by the way

First, let's just look at the four lines of code and nothing else

There are no iterators visible in the code.
There's one list named `numbers`. This is an iterable

If you're unsure about the difference between iterable and iterator, read on

https://twitter.com/s_gruppetta_ct/status/1679178647423402005

There are two other variables named `number` and `another_number`

These variables represent the integers you're looping through, and integers are neither iterable nor iterators

And that's it, except for the `print` function and the keywords `for` and `in`

So, no iterators in sight in the lines of code themselves

But 0 wasn't an option in the poll

So let's see what happens as the code runs…

A `for` loop needs an iterable, such as a list

The loop creates an iterator from the iterable

Let's start with the first loop, the outer one. It creates an iterator from the list and the iterator is ready to go, "waiting" just before the list's first element

The loop then gets the first item from this outer loop iterator, which is 2. The iterator stops here for now, between 2 and 5


But then it finds the second, inner, `for` loop statement. This is a new `for` loop. Even though it uses the same list, `numbers`, the inner loop creates a new iterator from the iterable (the list)

Let's call the two iterators inner and outer iterators.

The outer iterator is stuck between 2 and 5 for now, the first and second items in the list, while the inner iterator goes through all the elements of the list

When the inner iterator uses up all the elements, it's exhausted. Iterators can only be used once. They're like disposable items.

The outer iterator can now move on the the next value. And the outer loop now reaches the inner loop again for the second time

It needs to create an iterator again. The previous inner iterator is no longer useful

Here's a representation of the first three iterations of the outer loop. You can figure out the rest

The "checkpoints" you see on the left make sense in the article I'll link to soon…


So, by the time the code has gone through the nested `for` loop code, it needs to create seven iterators: One for the outer loop and six for the inner loop.

You can read more about this in the article about iterators:

open.techwriters.info/thepythoncodingstack/iterators-in-python-data-structure-6

And this is Part 6 of a series of article about data structure categories. You may also want to read the first one about iterables

open.techwriters.info/thepythoncodingstack/python-iterable-data-structures

And since you're there, go ahead with 2-5 about sequences, mappings, containers, and collections

Make sure you subscribe to get all new articles directly in your inbox about all sorts of Python topics, as always, written in a more relaxed style than your typical technical article!

open.techwriters.info/thepythoncodingstack


Iterators in nested `for` loops

Yesterday I talked about tuples when they're used as dictionary keys

Tuples are immutable

But their contents don't have to be…

Let's look at this tuple `book_entry` which contains an integer, a couple of strings, and a list as its elements

https://twitter.com/s_gruppetta_ct/status/1678690650160791552

The tuple and its elements are separate objects which have their own unique id

The tuple is immutable. This means the tuple with the id shown above will always have the same four objects within it, the ones with the ids also shown in the above code snippet

But, the fourth of those elements is a list, which is mutable

So, can we still change the contents of the list even though it's a member of an immutable object?

Yes. In fact, there are three volumes in this series of books, the third was published in 1965:

Compare the ids of the tuple and the list, which is an element in the tuple, before and after the change to the list…

They're both the same objects as before. The same tuple contains the same list, but the list has changed!

Lists are mutable, so they can change:

So, in some ways, the tuple has changed too. It's no longer the same as it was before even though it contains the same objects

But there's another omission in this entry. Although these three volumes are Feynman's lectures, the books have two more authors

Let's add them

We can't

Because the third entry in the tuple, the one with the author name, is a string. And strings are immutable

The only way we can modify this is to create a new tuple with the modified values. But if we do this, we're not changing the tuple, we're replacing it with a new one

Because this tuple contains a list, you cannot create a hash value from it

The error message shows you that the list is the culprit in this case!

You can see why being hashable or not is relevant in yesterday's post: https://twitter.com/s_gruppetta_ct/status/1678690650160791552

You'll see all of this and more discussed in an article coming soon on The Python Coding Stack

open.techwriters.info/thepythoncodingstack


Understanding Mutable and Immutable Elements in Tuples

What data types can you use in a dictionary?

There are two different answers, one for the keys and one for the values of a dictionary

Values can be any type you want

But there are some limitations for keys

In the example in the previous post, you can see two similar dictionaries. They represent the positions in a kitchen unit that has a grid of shelves, so we refer to them using a pair of coordinates…

You can use a tuple of coordinates as a key, but you cannot use a list

The error shows that a list is an 'unhashable type`

A key difference between tuples and lists is that tuples are immutable whereas lists are mutable

The concepts of 'immutability' and 'hashability' are closely linked, but they're not the same

An object is hashable if the same object can always be converted to the same unique identifier during the lifetime of a program

This unique identifier is the hash value which is returned by the `__hash__()` special method

An object must also have `__eq__()` defined to be hashable so that objects can be compared to each other for equality

Any object that meets these requirements can be used as a key in a dictionary

This usually means that immutable types can be used as keys in a dictionary

But what about if we have this tuple:

(2, [3, 5])

The tuple itself is immutable, but one of its elements, the list, is mutable

This means that you cannot change which list is in the second place in the tuple, but you can still change the elements in that list

This means that `(2, [3, 5])` is not hashable

I may write a full long-form article about this topic since there's more to say than can fit in a few posts…

What Data Types Can You Use in Dictionaries

There's nothing magical about arithmetic operators

For example:
>>> my_numbers * 2

returns the same as:
>>> my_numbers.__mul__(2)

`.__mul__()` is the special method which defines what multiplication using * does

But, this can be misleading. There's a bit more happening…

For example, you can achieve the same multiplication of a list by reversing the operands:

>>> 2 * my_numbers
[2, 4, 6, 2, 4, 6]

But, this is _not_ the same as calling `.__mul__()` directly

Note there's a space between the `2` and `.__mul__()` to avoid syntax issues

This returns `NotImplemented` and not the same result as
`2 * my_numbers`

The result from `2 * my_numbers` is actually returned by:
`my_numbers.__rmul__(2)`

>>> my_numbers.__rmul__(2)
[2, 4, 6, 2, 4, 6]

Let's look at
`2 * my_numbers`

The reflected multiplication special method `.__rmul__()` of the operand on the right (`my_numbers`) is called if the `.__mul__()` method of the left operand (`2`) returns `NotImplemented`

I've written more about this in the inaugural article on The Python Coding Stack in April

And there are more examples with `.__rmul__()` and `.__radd__()`, and more detailed explanation in the article

You can get to The Stack through my bio here…

Arithmetic Operators in Python

"Why do we use a built-in function, `len()` to find the length of a list but a method, `.count()` to find how often a value occurs?

"Why aren't they either both methods or both built-in functions?"

Great question asked in a session I was running this week…

The discussion led to what seems like an inconsistency

List is not the only data type that has a `.count()` method

And `len()` doesn't work with all data types, either

So, why don't we treat them in the same way?

`list.count()` and `str.count()`, for example, are different methods that perform a similar task

But is it that different when using `len()`?

Although we use the same built-in function `len()` on several different data types, the hard work is done by the data type's `.__len__()` method

`len([2, 4, 6])`
calls
`[2, 4, 6].__len__()`

and

`len("Hello!")`
calls
`"Hello!".__len__()`

So, behind the scenes, finding the length of data structure is handled by a method, too, even if you're using the built-in function `len()`

The `len()` built-in function gives us a convenient was of accessing the `.__len__()` method

So why is there a `len()` built-in function but not one for counting the number of occurrences of an item, say

These are language design decisions

Finding the length of a _sized_ object was considered worth having a built-in function for, but for anything other than the most common actions, methods are good enough!

In fact, there are cases where we can find both options available

For example, if you want to sort a list, you can either use:
`my_list.sort()`
or
`sorted(my_list)`

…although in this case there's an important difference
Do you know what it is?

Another example where you can either use a method or a function is copying a list. You could use the method:

`my_list.copy()`

or the function that's in the `copy` module

`copy.copy(my_list)`

We could try to justify all of these design decisions. But you may be better off just accepting that that's the way it is!

Methods and Built-In Functions

"Why do we use a method to get the index from a value but we use square brackets for the other way round?"

I was asked this question this week while looking at

>>> cities = ["London", "New York", "Berlin", "New Delhi"]
>>> cities.index("New York")
1
>>> cities[1]
'New York'

This can be confusing for those at an early stage of their journey into programming

The two operations seem equivalent: get the value from the index or get the index from the value

Why does one use a method and the other doesn't?

And the answer is that both of them use methods

From that point of view, there is no difference

When you use the square brackets notation, the program calls the method `.__getitem__()`

This happens "behind the scenes"

Therefore, methods are at the heart of fetching a value using the index or retrieving the index of a value

The difference is that there's a "shortcut" for one of these tasks

Python's syntax allows you to use the square brackets notation to make it easier to pass an argument to `.__getitem__()`

Why does one of them get special treatment? That's a language design choice

Let's take the list in this example. The main use case for lists is one where you need to fetch items based on their position in list

Being able to find where an item is within the list is a bonus

You'll find this behaviour in other places in Python once you dig a bit underneath the surface, for example when using arithmetic operators such as + and *

You'll also find it when using built-in functions such as `len()`

More on this tomorrow (since it was also a question asked this week in The Python Coding Programme!)

Why do we use methods sometimes and other notation at other times?

You hail a black cab in London: "42 Python Street, please" and you get in the cab

This journey will meander through Python mappings and sequences…

but first, how much do you know about London cabbies, as London's taxi drivers are called?

"The Knowledge" is the brilliantly-named test London taxi drivers must pass before they can drive one of the iconic black cabs

A research study found that part of the brain used heavily in memory and spatial navigation is larger for London cabbies compared to a control group

Why does this matter? It doesn't really, but I said it anyway…

When you ask a cab driver to go to "42 Python Street", the name "Python Street" acts as a key in a dictionary

The cabbie visualises the street's geographical location in relation to the current location

The mapping between the street name and the visualisation of the geographical location happens automatically in the taxi driver's brain

But when the cab gets to Python Street, the cab driver treats the row of houses as a sequence. He needs to get to the 42nd one

In reality, the house numbers go up in steps of two since the odd numbers are on one side of the road and the even numbers on the other—but let's not let facts get in the way of a good analogy!

You can read more about the taxi driver, mappings and sequences in the article I published on The Python Coding Stack—you can follow the link shown on this cover image for the article

London Cabbies and Python Mappings and Sequences

Let's travel back in time to a pre-Python 3.6 era

The order of items was meaningless

After all, what matters is the key-value association

But things changed in Python 3.6, and again in 3.7, and now the order is maintained

But beware, they're still mappings not sequences

The change in Python 3.6 wasn't specifically designed to make dictionaries ordered, it was a by-product of other updates to the internal workings of dictionaries

Python 3.7 formalised this, so you can now rely on dictionaries being ordered

But as you can see from the code in the first tweet, this has a different meaning to the order in a sequence, say

In a sequence, such as a list, two lists with the same values but different order are _not_ equal

>>> first_list = [2, 4, 6]
>>> second_list = [2, 6, 4]
>>> first_list == second_list
False

But, two dictionaries with the same items in different orders are equal…

>>> first_dict = {"two": 2, "four": 4, "six": 6}
>>> second_dict = {"two": 2, "six": 6, "four": 4}
>>> first_dict == second_dict
True

I've written in more detail about mappings in this article, which is part of a longer series on data structure categories

Follow the link in the image, and you can subscribe to receive new articles in your inbox

Or just look for The Python Coding Stack in that other place…

Python Dictionaries: Does Order Matter?

We often learn about lists, dictionaries, and other data types early on in our journey to learn to code…

…but it’s tempting to ignore the details of categories of data types such as iterable, sequence, collection, and so on

I wrote a series of threads about this recently

And now, I'm upgrading those Twitter threads into a series of articles

In this first article of the series, published today, I explore iterables

I'll then go on to cover sequences, mappings, collections, containers, iterators, and generators

Here's the link to the first article:

https://thepythoncodingstack.substack.com/p/python-iterable-data-structures

Understanding Data Type Categories in Python - Part 1

Over the past 2 weeks I published a 7-day series on classes & OOP

I knew I couldn't talk about everything, but I also wanted to make it reasonably (!) comprehensive

Here are all the threads, from Year 1 to 7 at Hogwarts School of Codecraft and Algorithmancy

It's long

0 of 7

Year 1: The OOP Mindset

https://twitter.com/s_gruppetta_ct/status/1641121936414736385

Year 2: Defining Classes and Understanding Data Attributes

https://twitter.com/s_gruppetta_ct/status/1641474113624760321

Year 3: Defining Methods

https://twitter.com/s_gruppetta_ct/status/1641846615186210817

Year 4: Interaction between classes

https://twitter.com/s_gruppetta_ct/status/1642888327258058754

Year 5: Inheritance

https://twitter.com/s_gruppetta_ct/status/1643648452331700224

Year 6: Special Methods (Dunder Methods)

https://twitter.com/s_gruppetta_ct/status/1644010840469958661

Year 7: Class Methods, Static Methods, and Class Attributes

https://twitter.com/s_gruppetta_ct/status/1644365916950659073