Incremental Assignment of Sequences

The behavior of the incremental assignment operators += and *= depends on their first operand. For simplicity, we will focus on incremental addition (+=), but these concepts apply equally to *= and other incremental operators.

The special method behind += is iadd (used for “in-place addition”). However, if a class does not implement this method, Python will fall back to calling add. Consider the following simple expression:

>>> a += b

If a implements the iadd method, this method will be called. For mutable sequences (such as list, bytearray, and array.array), a will be modified in place, just like calling a.extend(b). However, if a does not implement iadd, the effect of the expression a += b becomes the same as a = a + b: first, a + b is computed, resulting in a new object, which is then assigned to a. In other words, whether the variable name is associated with the new object in this expression entirely depends on whether this type has implemented the iadd method.

In general, mutable sequences typically implement the iadd method, so += is an in-place addition. Immutable sequences do not support this operation at all, and there is no question of implementing this method.

The concepts discussed above regarding += also apply to *=, with the difference being that the corresponding method is __imul__. Below is a small example demonstrating the effect of *= on mutable and immutable sequences:

>>> l = [1, 2, 3]
>>> id(l)
4311953800 ➊
>>> l *= 2
>>> l
[1, 2, 3, 1, 2, 3]
>>> id(l)
4311953800 ➋
>>> t = (1, 2, 3)
>>> id(t)
4312681568 ➌
>>> t *= 2
>>> id(t)
4301348296 ➍

❶ The initial ID of the list.❷ After applying incremental multiplication, the ID of the list remains unchanged, and new elements are appended to the list.❸ The initial ID of the tuple.❹ After applying incremental multiplication, a new tuple is created.

Performing repeated concatenation operations on immutable sequences can be inefficient, as a new object is created each time, and the interpreter needs to copy the elements from the original object to the new object before appending new elements.

str is an exception because using += on strings is so common that CPython has optimized it. When initializing memory for str, the program allocates extra expandable space, so incremental operations do not involve copying the original string to a new location.

Having understood the general usage of +=, let’s look at an interesting edge case. This example highlights what “immutability” really means for tuples.

A Puzzle about +=Read the following code and answer the question: What will the two expressions in Example 2-14 produce? Do not run these expressions in the console before answering. Read the code below and answer the question: What will the two expressions in Example 2-14 produce? Do not run these expressions in the console before answering.Example 2-14　A puzzle

>>> t = (1, 2, [30, 40])
>>> t[2] += [50, 60]

Which of the following four outcomes will occur?a. t becomes (1, 2, [30, 40, 50, 60]).b. A TypeError will be raised because tuples do not support item assignment.c. Neither of the above.d. Both a and b are correct.

When I first saw this question, I confidently chose b, but the actual answer is d, meaning both a and b are correct! Example 2-15 shows the result of running this code, using Python version 3.4, but the result is the same in 2.7.

Example 2-15　An unexpected result: t[2] was modified, but an exception was also raised

>>> t = (1, 2, [30, 40])
>>> t[2] += [50, 60]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> t
(1, 2, [30, 40, 50, 60])
</module></stdin>

Python Tutor (http://www.pythontutor.com) is a tool for visualizing the workings of Python. Figure 2-3 shows two screenshots representing the initial and final states of t in Example 2-15.

Next, let’s take a look at the bytecode generated by Python for the expression s[a] += b in Example 2-16, which may clarify the reason behind this phenomenon.Example 2-16　The bytecode behind s[a] = b

>>> dis.dis('s[a] += b')
  1  0 LOAD_NAME                0 (s)
  3 LOAD_NAME                1 (a)
  6 DUP_TOP_TWO
  7 BINARY_SUBSCR           ➊
  8 LOAD_NAME                2 (b)
 11 INPLACE_ADD             ➋
 12 ROT_THREE
 13 STORE_SUBSCR            ➌
 14 LOAD_CONST               0 (None)
 17 RETURN_VALUE

➊ The value of s[a] is stored in TOS (Top Of Stack). ➋ TOS += b is computed. This step can be completed because TOS points to a mutable object (the list in Example 2-15). ➌ s[a] = TOS assignment fails because s is an immutable tuple (the tuple t in Example 2-15). This is actually a very rare edge case; in my 15 years of Python experience, I have never seen anyone get caught in this situation.Thus, I have learned three lessons.Do not put mutable objects inside tuples.Incremental assignment is not an atomic operation. As we have seen, it raises an exception but still completes the operation.Examining Python’s bytecode is not difficult, and it helps us understand the underlying mechanisms of code execution.After witnessing the subtleties of + and *, we will shift our focus to another important aspect of sequence types: sorting.