Chapter 0: Basics of Python

Chapter Introduction

This is a statistics course, and yet its first chapter contains no statistics at all. The reason is practical. Every technique in Chapters 1 to 6 — a rolling volatility, a hypothesis test, a best-subset regression, an MCMC sampler, a GARCH forecast — is delivered to you as a method on a Python object, called with keyword arguments, returning a container that you then slice, filter and pass to the next method. If df.loc[...], key=lambda …, **kwargs, or a KeyError still looks like magic, then the statistics will look like magic too, because you will not be able to tell whether a strange number came from the model or from a slice that stopped one row early. The purpose of this chapter is to make those mechanics boring, so that from Chapter 1 onwards your attention can go entirely to the statistics.

You already know some Python: loops, lists, numpy arrays. What this chapter adds is not vocabulary but mental models — the small number of rules that explain why Python behaves the way it does when it surprises you. There are only a handful. A name is a label attached to an object, and assignment moves labels rather than copying objects. Some objects can be edited in place and some cannot, and that single property explains why b = a is harmless for an integer and dangerous for a list. Slices and ranges include their start and exclude their stop, one rule that runs unchanged from s[0:4] through range(1, 5) to df.iloc[0:4]. A function’s default argument is evaluated once, when the function is defined, not each time it is called. Once you hold those rules, the “traps” that fill the multiple-choice questions on the companion slides stop being traps and become consequences.

Why should an MSc student of statistics care about traps at all? Because in quantitative work they are expensive. In April 2013 a graduate student at the University of Massachusetts discovered that the spreadsheet behind Reinhart and Rogoff’s Growth in a Time of Debt — the paper that told finance ministries across Europe that public debt above 90 % of GDP came with growth of −0.1 % a year — averaged rows 30 to 44 instead of rows 30 to 49. Five countries silently fell out of the calculation; the corrected number for the same bucket was +2.2 %. That is a slicing error: rows[30:45] where rows[30:50] was intended. Nothing crashed, nothing warned, and the wrong number was cited by the UK Treasury and the US Congress for three years. This chapter will teach you to print len() of every slice, to test every function on a five-row example, and to distrust any code that silences an exception without saying which one.

By the end of the chapter you will be able to predict, before running, what Python prints for arithmetic that mixes integers and floats; slice any sequence — string, list, tuple, later a Series — and say exactly how many elements the slice holds; explain what happens to a when you modify b after b = a, and choose between .copy() and a second name deliberately; pick the right container for a job (a tuple for a fixed record, a set for membership, a dictionary for a lookup); write if/for/while logic and compress the common case into a comprehension; define functions with defaults, *args and **kwargs, and avoid the mutable-default bug; format numbers with f-strings; and convert a crash into a value with try/except. The closing mini-project — counting the most frequent words in a paragraph of raw text — uses every one of those pieces in a dozen lines and is, structurally, the same programme as a bag-of-words sentiment score on an earnings-call transcript.

The chapter follows the four sections of the companion slide deck in the same order, with the same examples and the same printed numbers, so that the two can be read side by side. Each subsection gives the mechanism in prose, then a runnable cell, then an interpretation of what the cell printed. Some cells are wrapped as “predict the output”: commit to an answer on paper before you click reveal, because a prediction that turns out wrong is worth ten that turn out right. Two cells are marked “debug yourself” and contain a bug that does not raise an error — the dangerous kind. Flashcards at the end of each section are for spaced review before the exam.

Python as a Calculator: Numbers, Booleans, Strings
Containers: Lists, Tuples, Sets, Dictionaries
Control Flow and List Comprehensions
Functions, Lambdas, f-strings, and Error Handling

Python as a Calculator: Numbers, Booleans, Strings

You already know arithmetic. What this section is about is where Python’s arithmetic differs from the calculator on your desk: a division operator that always returns a float, a second division operator that rounds down rather than toward zero, a floating-point representation in which 0.1 + 0.2 is not 0.3, a True that is secretly the integer 1, and a text type that behaves like a list you are not allowed to edit. None of these is an obscure corner of the language. Every price, return and p-value in this course is a float; every count of “up days” is a sum of booleans; every ticker and column name is a string. The differences are small, and they are exactly where the wrong number comes from.

Expressions, and the two divisions

An expression is anything Python can evaluate to a value: 1 + 2, (3 - 1) * 5 / 2, 2 ** 10. In a notebook the value of the last expression in a cell is displayed automatically; in a script, or when you want more than one value from a cell, you wrap it in print(). The arithmetic operators are the ones you expect, with two deliberate departures from calculator notation. Exponentiation is **, not ^ — in Python ^ is bitwise exclusive-or, which is a legal operation on integers and therefore fails silently rather than loudly: 2 ^ 10 is 8, not 1024. And the remainder (modulo) operator is %, so 10 % 3 is 1.

The subtler departure is division. Python 3 has two division operators. True division / always returns a float, even when the mathematical answer is a whole number: 4 / 2 is 2.0, not 2. Floor division // returns the largest integer not greater than the exact quotient; for positive operands that is the familiar “integer part”, but for negative operands it rounds toward minus infinity rather than toward zero, so -10 // 3 is -4, not -3. The remainder is defined to be consistent with the floor, so that a == (a // b) * b + a % b always holds; a consequence is that -10 % 3 is 2, and in Python the sign of a % b follows the sign of b.

Why does this matter in finance? Lot sizing. Shares on many exchanges trade in board lots of 100; shares // 100 counts the whole lots and shares % 100 is the odd lot. For a long position of 1,250 shares that gives 12 lots and 50 odd shares, and every language agrees. For a short position represented as −1,250 shares, Python’s floor gives -13 lots and an odd lot of 50, whereas a language that truncates toward zero would report −12 and −50. Neither is “wrong”, but if you split the sizing logic between Python and SQL or C and do not know which convention each one uses, the lot count on the short book is off by one.

Four expressions, four lines. Write down what each prints — paying attention to whether the second line shows 5 or 5.0 — before you reveal the output.

The four lines print 3, 5.0, 1024 and 1. The one that catches people is the second: the exact answer is the whole number five, but because the expression contains a /, the result is the float 5.0. Python never silently turns a float back into an integer, so once a / appears anywhere in a calculation, everything downstream of it is a float, and type() will tell you so.

The first line is 3.3333333333333335 — sixteen significant digits, of which the last is an artefact of binary representation that we return to below. The second is 3 -4 1: floor division of -10 by 3 is -4 because −3.33… lies between −4 and −3 and the floor is the lower of the two. The third line confirms the type rule: / produced a float, // produced an int. The lot-sizing lines print 12 50 for the long position and -13 50 for the short one — the whole-lot count on the short side is −13, not −12, and the remainder is positive because it takes the sign of the divisor.

Which operator do I want?

Use / whenever the answer is a quantity (a return, a ratio, an average). Use // only when the answer is a count of whole units (lots, weeks, pages) and be explicit about the sign convention. If you find yourself writing int(a / b) to get a whole number, you are truncating toward zero — which is fine, but it is not the same as a // b for negative a, and the two will disagree exactly when it matters.

int, float, bool, and what `type()` tells you

Python has four scalar types you will use constantly. An int is a whole number of unbounded size — 2 ** 100 is a perfectly good integer and Python will happily print all thirty-one of its digits. A float is a double-precision binary floating-point number in the IEEE 754 standard: 64 bits, of which 53 carry the significand, giving roughly 15–17 significant decimal digits. A bool is one of True or False. A str is text. The built-in function type(x) returns the type of any value, and it is the first thing to print when a calculation gives a result you did not expect.

Arithmetic between an int and a float promotes the result to float: 42 + 3.14159 is 45.14159, a float. Python never truncates in the other direction, so the danger with mixed arithmetic is not lost precision from the integer but the false confidence of thinking a value is exact when it is a float. The point at which you will feel this most is comparison. Because a float is a binary fraction, most decimal numbers cannot be represented exactly: 0.1 is stored as the nearest 53-bit binary fraction, which is 0.1000000000000000055511151231257827…, and 0.2 likewise. Their sum is the nearest representable number to the sum of the two approximations, and that turns out to be 0.30000000000000004, which is not the same double as the nearest representable number to 0.3. So 0.1 + 0.2 == 0.3 is False.

The rule that follows is absolute: never compare floats with ==. Compare them with a tolerance — abs(x - y) < 1e-9 — or with math.isclose(x, y) which does the same with a relative tolerance, or with numpy.allclose for arrays, or round both sides first. Every price, return, weight, and p-value in this course is a float. A backtest that checks if weight == 0.0 to decide whether a position is closed will one day meet a weight of 1e-17 left over from a subtraction and keep the position open.

Booleans are the smallest surprise and the most useful one. In Python bool is a subclass of int: True behaves as 1 and False as 0 in any arithmetic context. So True + True is 2, True * 3.5 is 3.5, and — the reason you should care — sum(list_of_booleans) counts how many were True. In Chapter 1, sum(returns > 0) counts the up-days in a return series, and (returns > 0).mean() is the fraction of up-days, with no loop and no explicit conversion. The comparison produces booleans, the aggregation treats them as integers, and the arithmetic is exact.

The first three lines show the promotion rule: <class 'int'> 42, <class 'float'> 3.14159, and then <class 'float'> 45.14159 for the sum. The next three are the float trap in full: False, then 0.30000000000000004, then True once the comparison is made with a tolerance of one part in a billion. The last three lines are the boolean-as-integer rule: 1, then <class 'bool'> 2 — note that the type is still bool but the sum is the integer 2 — and finally 3 of 4 conditions were True, which is the idiom you will use for counting.

== on floats, is on anything but None

Two comparison habits will save you from most silent errors in this course. Never use == between floats — use a tolerance. And never use is to compare values at all; is asks whether two names point to the same object, which is a question about memory, not about value. The only idiomatic use of is is x is None. Section 0.2 explains why.

Strings are sequences: indexing and slicing

A string holds text, and in Python it is a sequence: an ordered collection of characters that you can index and slice with exactly the same syntax you will later use on lists, tuples, numpy arrays, and pandas objects. Learn the rule once here and it never changes.

Indexing is s[i]. Positions count from zero, so s[0] is the first character. Negative positions count from the end, so s[-1] is the last character and s[-2] the one before it; this is a convenience Python offers so that you do not have to write s[len(s) - 1]. Slicing is s[start:stop:step], and the rule to memorise is start inclusive, stop exclusive: s[0:4] gives the characters at positions 0, 1, 2 and 3, and not position 4. Any of the three parts may be omitted — s[:4] starts from the beginning, s[5:] runs to the end, s[::2] takes every second character from the start.

The half-open convention looks arbitrary until you see what it buys. The number of elements in s[a:b] is simply b - a, with no +1 to remember. Two adjacent slices s[:k] and s[k:] tile the sequence exactly, with no overlap and no gap, which is precisely what you want when splitting data into a training set and a test set. And s[i:i+n] is always a window of n elements, which is the shape of every rolling calculation in Chapter 1. The Reinhart–Rogoff spreadsheet error was, in Python terms, a window of the wrong length; the half-open rule is what lets you check a window’s length by subtraction.

With s = 'data Analysis', work out what s[0:4] and s[::2] are — write the characters of s with their positions 0 to 12 on paper first — and then reveal all six lines.

The string has 13 characters. s[0] is d and s[-1] is s. s[0:4] and s[:4] are both data — positions 0, 1, 2, 3, and not the space at position 4. s[5:] is Analysis, the eight characters from position 5 to the end. s[::2] is dt nlss: positions 0, 2, 4, 6, 8, 10 and 12, which are d, t, the space, n, l, s, s. The common wrong answer for s[::2] is aaAayi — the characters at the odd positions — which is what you get if you assume the step starts after the first character rather than at it. It does not: a slice always begins at start, which defaults to 0.

A last observation that will matter in Chapter 1: slicing past the end does not raise an error. s[5:100] is still Analysis; Python clamps the stop to the length. That is convenient, and it is also why a window that runs off the end of a price series shrinks silently rather than failing. Printing len() of the slice is the only defence.

Objects have methods, and strings are immutable

Everything in Python is an object, and every object carries functions that belong to it, called methods. You invoke a method with a dot: object.method(arguments). A string has dozens — .strip() removes whitespace from both ends, .upper() and .lower() change case, .replace(old, new) substitutes text, .split(sep) cuts one string into a list of strings at every occurrence of the separator. Even an integer has methods: (10).bit_length() tells you that 10 needs 4 bits in binary. This dot-syntax is the syntax of pandas — df.dropna(), df.groupby("sector").mean() — and it is also how you discover what an object can do: in a notebook, type the name, a dot, and press Tab.

The method to remember from this section is .split(). It turns one string into a list of strings, and it is the first step of every text-processing task, including the mini-project at the end of the chapter: a paragraph becomes a list of words by paragraph.split(), which with no argument splits on any run of whitespace, including newlines. The reverse operation is the string method .join(), called on the separator: ", ".join(["a", "b", "c"]) is "a, b, c".

The property that separates strings from lists is that a string is immutable: once created, it cannot be changed. There is no s[0] = "D" — that is a TypeError. It follows that every string method that appears to modify the string in fact returns a new string and leaves the original untouched. s.upper() on its own does nothing visible: it computes an upper-case copy, and because you did not assign the copy to anything, it is discarded. To keep the result you must write s = s.upper(), which creates the new string and rebinds the name s to it. This is the single most common beginner bug in text cleaning, and it recurs in pandas, where df.dropna() returns a new frame and leaves df with all its missing values until you write df = df.dropna().

The first four lines are hello, world, HELLO, WORLD, hello, Python and ['data', 'science', '101']; note that .upper() and .replace() left the leading and trailing spaces alone because only .strip() removes them. 10 .bit_length() is 4, since 10 in binary is 1010. Then the immutability demonstration: after a bare s.upper() the string is still ' hello, world ' — repr() shows the quotes and the spaces that print() would hide — and only after s = s.upper() does it become ' HELLO, WORLD '. The last line chains two methods: .strip() returns a new string, and .upper() is called on that, giving 'HELLO, WORLD'. Chaining works precisely because each method returns a value; it is the style that Chapter 2 calls method chaining and that pandas is built around.

Why immutability is a feature

An immutable value can be shared freely. If two variables point to the same string, neither can change it under the other’s feet, so you never need to copy a string defensively. Immutability is also what allows strings (and tuples) to be dictionary keys, which Section 0.2 explains. The price is that “modifying” a string means building a new one — negligible for a ticker, noticeable for a million-character transcript, which is why text pipelines build a list of pieces and .join() them once at the end.

10 / 3 is 3.3333333333333335 (a float, always — / returns a float even when the answer is whole). 10 // 3 is 3 (an int). -10 // 3 is -4, not -3: floor division rounds toward minus infinity, so −3.33 floors to −4.

Neither 0.1 nor 0.2 is exactly representable as a binary fraction; the sum of their nearest doubles is 0.30000000000000004, which is not the double nearest 0.3. Compare with a tolerance: abs(x - y) < 1e-9, or math.isclose(x, y), or np.allclose for arrays. Never == on floats.

It shows ' hello ', unchanged. Strings are immutable: .upper() returns a new string, and because the result was not assigned it was discarded. To keep it write s = s.upper(). The same rule governs df.dropna() in pandas.

Containers: Lists, Tuples, Sets, Dictionaries

Python has four built-in ways to hold many values. A list is an ordered, editable sequence; a tuple is an ordered sequence that cannot be edited; a set is an unordered collection of unique values; a dictionary is a mapping from keys to values. Two questions classify them. Does it have positions? Lists and tuples do, so you can index and slice them; sets and dictionaries do not, and bag[0] is an error. Can it be changed after creation? Lists, sets and dictionaries can — they are mutable; tuples cannot. The second question is the important one, because mutability is the property behind the trap that bites every analyst once: what happens to a when you change b.

Lists: ordered, mutable, mixed types

A list is written with square brackets, may contain values of different types, and supports the same indexing and slicing as a string: nums[1:-1] is everything except the first and last elements. What a list adds is a family of methods that edit it in place: .append(x) adds one element at the end, .insert(i, x) puts x at position i and shifts the rest along, .pop() removes and returns the last element (or .pop(i) for position i), .remove(x) deletes the first occurrence of a value, .sort() reorders. None of these returns a new list; they modify the one you called them on and return None (except .pop, which returns the removed item). This is the opposite convention from strings, and mixing the two up produces the classic nums = nums.append(50), after which nums is None.

nums[1:-1] is [20, 30] — positions 1 up to but not including the last. After .append(50) the list is [10, 20, 30, 40, 50]. .insert(1, 15) makes it [10, 15, 20, 30, 40, 50], and .pop() removes the trailing 50, so the third line prints [10, 15, 20, 30, 40] removed: 50. The fourth line — 10 40 [15, 20] — is indexing and slicing again, identical to the string case. The last line shows three operations that you will use on every list: len() is 5, the membership test 30 in nums is True, and nums.index(30) is 3, the position of the first 30.

The aliasing trap: names and objects

Here is the mental model that explains everything else in this section. In Python a variable is not a box that holds a value; it is a name — a label — attached to an object that lives somewhere in memory. The assignment nums = [10, 20, 30, 40] creates a list object and attaches the label nums to it. The assignment alias = nums does not create a second list. It attaches a second label, alias, to the same object. There is still exactly one list, with two names. Assignment never copies.

Now consider alias.append(50). The method is called on the object that alias points to, and it mutates that object. Because nums points to the very same object, nums “sees” the 50 as well — not because Python did anything to nums, but because there was never anything else for nums to be. Two names, one list. If you wanted alias to be independent, you had to ask for a copy: nums.copy(), or equivalently list(nums) or the full slice nums[:], each of which builds a new list object with the same contents.

Why does the same assignment feel harmless for integers? Because integers are immutable. After a = 5; b = a, both names point to the same integer object 5 — exactly as with the list. But there is no method that can change the integer 5 into 6 in place. When you write b = b + 1, Python computes a new integer object 6 and rebinds b to it; a still points to 5. So the difference between “changing b” for an integer and for a list is not in what = does — it does the same thing in both cases — but in whether the object can be mutated afterwards. = rebinds a name; a method like .append() mutates an object. Keep the two operations distinct and aliasing is never a surprise.

The trap has a well-known descendant in pandas. When you take a slice of a DataFrame and then assign into the slice, pandas cannot always tell whether you are modifying a view of the original (an alias) or a copy, and emits the SettingWithCopyWarning to say so. Every time you see that warning in Chapter 2 you are looking at exactly this question — one object or two? — and the fix is the same: .copy() when you want independence.

The first line prints nums: [10, 20, 30, 40, 50] alias: [10, 20, 30, 40, 50]: appending through alias changed the one object both names share. The second line prints nums: [10, 20, 30, 40, 50] copy: [10, 20, 30, 40, 50, 15]: the copy took the 15 and nums did not. The third line — a: 5 b: 6 — is the integer case, and it looks different only because b = b + 1 is a rebinding, not a mutation. Had lists supported += the way integers do, the same distinction would apply; in fact nums += [60] mutates in place for a list, while b += 1 rebinds for an integer, which is one more reason to keep the model in mind rather than the syntax.

A portfolio starts from equal weights and a “tilted” version overweights the first asset. The code runs without error and prints two lines. Run it, look at the first line, and find the bug.

The benchmark has been destroyed: the first line prints [0.4, 0.1, 0.25, 0.25], not [0.25, 0.25, 0.25, 0.25], and the third line confirms same object? True. tilted = base attached a second name to the one list, and the item assignments tilted[0] = 0.40 mutated that list, so the benchmark you intended to compare against no longer exists. Nothing raised, and any later “tilt minus benchmark” calculation would report zero active weight. The fix is one word: tilted = base.copy(). Then the benchmark stays [0.25, 0.25, 0.25, 0.25], the tilt is [0.4, 0.1, 0.25, 0.25], and tilted is base is False. In pandas the same fix reads tilted = base.copy() on a Series.

Shallow and deep copies

nums.copy() is a shallow copy: it builds a new outer list whose elements are the same objects as before. If the elements are themselves lists — a list of portfolios, each a list of weights — then editing an inner list through the copy still affects the original, because the inner lists are shared. For nested structures use copy.deepcopy(nums) from the standard library, which copies all the way down. For flat lists of numbers and strings, .copy() is enough.

`is` asks about identity; `==` asks about value

The names-and-objects model gives two different questions you can ask about two names. Do they point to the same object? That is is, and it compares identities — in CPython, memory addresses, which id(x) reveals. Do the objects they point to have the same contents? That is ==, and it compares values by calling the objects’ own equality rule. A list built separately with the same elements is == to the original but is not the same object; an alias is both.

The rule for practice is that is has exactly one idiomatic use: x is None. None is a singleton — there is only ever one None object — so identity and equality coincide, and is is faster and reads better. For everything else use ==. Using is between strings or integers sometimes appears to work, because CPython caches small integers and short strings so that equal values happen to be the same object; it then stops working for 1000 or for a string read from a file, and the resulting bug depends on which Python you run. It is the definition of code that “works on my machine”.

nums == fresh is True — same contents. nums is fresh is False — two separate list objects, built by two separate literals. alias is nums is True — one object, two names. The id() line says the same thing in terms of addresses: True False. The None line prints True True; x == None happens to work here, but linters flag it and pandas objects override == so that series == None returns a whole Series of booleans rather than a single answer — is None is the form that always means what you want.

Tuples: ordered and immutable

A tuple is written with parentheses (or, in fact, just commas — 1, 2 is a tuple; the parentheses are for readability) and behaves like a list that has been frozen. You can index and slice it, iterate over it, ask for its length, and test membership; you cannot append, insert, pop, or assign to a position. coords[0] = 0 raises TypeError: 'tuple' object does not support item assignment. You may always build a new tuple from an old one, but the old one never changes.

Immutability is not a restriction for its own sake. It makes a tuple the right container for a fixed record whose fields have meaning by position — a (latitude, longitude) pair, a (sharpe, max_drawdown) result, an (open, high, low, close) bar. A reader who sees a tuple knows that the structure is settled and that nothing downstream will have added a fifth field. It also makes tuples hashable, which means they can be keys in a dictionary and members of a set, where lists cannot: {(2024, 1): 0.031, (2024, 2): -0.012} maps (year, month) pairs to returns, a pattern that becomes the MultiIndex of Chapter 2.

The tuple operation you will use most is unpacking: lat, lon = coords assigns the two elements to two names in one line. It works with any sequence of the right length, it is how functions return more than one value — a function that ends with return sharpe, mdd is returning a tuple, and the caller writes sharpe, mdd = PerformanceMeasure(profits) — and it is how for k, v in d.items() walks a dictionary. A mismatch in length raises ValueError: too many values to unpack, which is a helpful error to get early.

The tuple prints as (34.05, -118.25) <class 'tuple'>. The assignment attempt is caught by the try/except (Section 0.4 explains the mechanism) and prints TypeError: 'tuple' object does not support item assignment — the exact wording is worth recognising, because you will see it whenever you try to edit anything immutable, including a string. Unpacking gives 34.05 -118.25, and the last line shows that reading operations — indexing, negative indexing, len(), in — all work exactly as for a list: 34.05 -118.25 2 True.

Sets: unique, unordered

A set is written with curly braces and holds each value at most once; writing {1, 2, 2, 3, 3, 3} gives you a set of three elements. It has no positions — there is no first element — and the order in which Python displays or iterates a set is an implementation detail you must not rely on. What you get in exchange is very fast membership testing and the algebra of sets: .union(), .intersection(), .difference(), and the operator forms |, &, -. Under the hood a set is a hash table, so x in bag takes constant time regardless of the size of bag, whereas x in nums on a list scans every element. For a membership test inside a loop over a million rows, the difference is the difference between a second and an hour.

The two uses that recur in data work are de-duplication and comparison. set(tickers) is the one-line answer to “how many distinct tickers does this file contain?”, and len(set(tickers)) is the number. set(a) - set(b) is the list of tickers in a that are missing from b, which is the first thing to compute when two data sources disagree on the universe. Because sets are unordered, converting back to a list for display should go through sorted(bag), which returns a new list in ascending order; list(bag) also works but gives an order you cannot predict.

{1, 2, 3} 3 — three distinct values from six literals. After .add(4), 3 in bag is True. The union with {3, 5} is {1, 2, 3, 4, 5}; the intersection with {2, 5} is {2}; set(alist) is {1, 2, 3}. Then the trap: bag[0] raises TypeError: 'set' object is not subscriptable, and the polite way to get an ordered view is sorted(bag), which prints [1, 2, 3, 4]. Notice the type of error. A TypeError means “this operation does not make sense for this type of object”, as opposed to an IndexError (“the position is out of range”) — the set has no positions at all, so it is the operation, not the index, that is wrong.

Dictionaries: key → value

A dictionary maps keys to values. Keys must be hashable — strings, numbers and tuples qualify; lists do not — and are unique; values can be anything, including lists and other dictionaries. You read with d[key], write or update with d[key] = value, and remove with del d[key] or d.pop(key). The three views .keys(), .values() and .items() let you iterate over keys, over values, or over (key, value) pairs; since Python 3.7 they come back in insertion order, so a dictionary is also a reliable ordered record. A pandas Series is essentially a dictionary that has learned arithmetic — index labels are the keys — and a DataFrame is a dictionary of columns, which is why df["close"] uses the same square-bracket syntax.

student["name"] is Alex. After the two assignments the dictionary prints as {'name': 'Alex', 'age': 21, 'skills': ['excel', 'python'], 'gpa': 3.7} — the existing key age was updated in place and the new key gpa appended at the end, in insertion order. The keys are ['name', 'age', 'skills', 'gpa'], the values ['Alex', 21, ['excel', 'python'], 3.7], and the for k, v in student.items() loop — tuple unpacking applied to each pair — prints one key -> value line each. That loop is the pattern behind every “iterate over the columns of a DataFrame” idiom in Chapter 2.

The design decision you have to make with dictionaries is what should happen when a key is absent. Square brackets raise KeyError — loudly, immediately, with the missing key in the message. The method .get(key, default) returns default instead (and None if you give no default). Which is right depends on whether absence is normal or a bug. When you are counting words, a word you have not seen yet is normal, and counts.get(w, 0) + 1 is the idiom. When you are reading the "close" column of a price frame, its absence means the file is wrong, and you want the KeyError at 9:31 a.m. rather than a column of None propagating into a risk report. A .get that papers over a missing key is the dictionary equivalent of a bare except: — it converts a loud failure into a quiet wrong answer.

This student has no gpa key yet. For each of the three lines, decide whether it raises or returns, and what — then reveal.

The bracket lookup raises, and the handler prints KeyError: 'gpa'. .get("gpa", 0.0) returns 0.0; .get("gpa") with no default returns None. When df["Volumn"] throws a KeyError in Chapter 2, it is this exact mechanism, and the fix is to correct the typo, not to switch to .get.

Attributes versus methods

An object carries two kinds of things you reach with a dot. An attribute is a property you read: my_list.__class__, and in pandas df.shape, df.columns, df.index, series.values. A method is a function you call, with parentheses: my_list.append(3), text.upper(), df.head(), df.mean(). The distinction matters because forgetting the parentheses is not an error. text.isalpha without () is a perfectly valid expression whose value is the bound method object itself — the function, not its result. Python does not call it for you.

The consequences are two, and both are silent. First, print(df.mean) prints <bound method DataFrame.mean of …> followed by a dump of the frame instead of the column means — annoying but visible. Second, and worse, a method object is truthy: if text.isalpha: is always True, whatever the text contains, because you are testing whether the function exists rather than what it returns. A validation check written that way passes every record. The rule for pandas is mechanical: df.shape, df.columns, df.dtypes take no brackets; df.head(), df.describe(), df.mean() need them.

<class 'list'> is the attribute. text.isalpha() is True — every character of "tiger" is a letter. callable(text.isalpha) is True and its type is builtin_function_or_method: without the parentheses you are holding the method, not the answer. The last line is the silent bug in one print: bool(text.isalpha) and bool("123".isalpha) are both True, even though "123" contains no letters, because a function object is truthy regardless of what it would return.

[10, 20, 30, 40, 50]. Assignment never copies: alias = nums attached a second name to the one list object. .append() mutated that object, so both names see the 50. For an independent list write alias = nums.copy() (or list(nums) or nums[:]).

== compares values (contents); is compares identity (same object in memory). A separately built list with equal contents is == but not is. Use is only for x is None; use == for everything else — small integers and short strings only appear to work with is because CPython caches them.

Use brackets when absence is a bug you want to hear about — a missing price column should raise KeyError immediately. Use .get when absence is normal — a word not yet counted, counts.get(w, 0) + 1. .get on a key that should exist converts a loud failure into a quiet wrong answer.

Control Flow and List Comprehensions

Programs branch and repeat. This section covers the three statements that do it — if, for, while — and the two statements that interrupt a loop, break and continue, before compressing the most common loop shape into a single expression, the list comprehension. The predictable traps are, again, small: the first true branch wins, so the order of elif clauses changes the answer; range(1, 5) has four numbers, not five, by the same half-open rule as slicing; continue skips the rest of the current iteration while break ends the loop; and an if placed after the for in a comprehension filters, while an if … else placed before it labels.

`if` / `elif` / `else`, and why order matters

A conditional statement tests a boolean expression and runs an indented block if it is True. Python has no braces; the block is the indentation, four spaces by convention, and a block ends where the indentation returns to the previous level. elif (else-if) adds further tests, and else catches everything that no test matched. The evaluation rule is simple and its consequence is not: Python tests the branches top to bottom and executes the first one whose condition is True, then skips all the rest — even if a later condition would also have been true, and even if it would have been “more” true in some sense you had in mind.

The practical rule is to order thresholds from strictest to loosest. A grading chain that tests score >= 90 first, then >= 80, then >= 60, assigns each score to the narrowest band it qualifies for. Reverse the order — test >= 60 first — and a score of 95 satisfies the first test, receives a “C”, and never reaches the >= 90 branch that would have given it an “A”. No error is raised; the chain is syntactically fine; it is merely wrong for every score above 60. The same logic governs a risk limit ladder (if drawdown > 0.20: halt must come before if drawdown > 0.10: reduce) and a rating map, and the bug is invisible until someone checks a high score by hand.

The first chain prints 59 D or below: 59 fails all three tests, and the else catches it. The second prints 95 C: 95 satisfies >= 60, the first branch fires, and the >= 90 branch is never examined. Python is not “choosing the best match”; it is choosing the first one.

elif or several ifs?

Use one if/elif/else chain when the outcomes are mutually exclusive and exactly one should happen. Use separate if statements when several conditions can apply independently — for instance, flagging a trade as both large and after-hours. A chain with mutually exclusive tests written as separate ifs runs every test and may apply several outcomes; separate ifs written as a chain stops at the first and drops the rest.

`for` loops, `enumerate`, and `range`

A for loop iterates over the items of any iterable — a list, a tuple, a string, a dictionary, a range, a file, a pandas Series — binding the loop variable to each item in turn and running the body once per item. You do not manage an index; the loop hands you the items themselves. When you also need the position, the built-in enumerate(seq) yields (index, item) pairs, which you unpack directly in the for line: for i, c in enumerate(cities). Its optional second argument sets the starting count, so enumerate(cities, 1) numbers from one — useful for printing ranks. The idiom for i in range(len(cities)): c = cities[i] is legal and is the mark of someone who learned C first; in Python it is both slower and harder to read than enumerate.

range(start, stop, step) produces the integers from start up to but not including stop, in steps of step; start defaults to 0 and step to 1. This is the half-open convention of slicing applied to integers, and it has the same payoffs: range(n) has exactly n elements, range(a, b) has b - a elements, and range(1, 21) — not range(1, 20) — is the integers 1 to 20 inclusive. The most common off-by-one error in student code is writing range(1, 20) for “one to twenty”. A range is not a list; it is a lazy object that produces its integers on demand, which is why range(10**9) is instant, and why you write list(range(1, 5)) when you actually want to see the numbers.

The first loop prints City: NYC, City: LA, City: Chicago; the second prints 0 NYC, 1 LA, 2 Chicago — indices start at zero. list(range(1, 5)) is [1, 2, 3, 4], four numbers; list(range(5)) is [0, 1, 2, 3, 4]; list(range(0, 10, 3)) is [0, 3, 6, 9], stepping by three and stopping before 10. The last line confirms that range(1, 21) has 20 numbers, which is the form you need for “1 to 20 inclusive”.

`while`, `break`, and `continue`

A while loop repeats its body for as long as its condition is True, re-testing the condition before every iteration. Because the loop has no built-in end, the body must do something that eventually makes the condition False — decrement a counter, consume an item, receive a fill — and forgetting to do so gives an infinite loop that a notebook can only escape by interrupting the kernel. The rule for choosing is: use for when you know what you are iterating over (a list, a range, a file), and while only when the number of iterations depends on what happens inside the loop — “keep bidding until the order fills”, “keep halving the step until the change is below tolerance”. In statistical code the second case appears in iterative algorithms (EM, Newton steps, MCMC burn-in until convergence), and a while in such code should always carry a maximum-iteration guard as a second condition.

Two statements alter the flow inside either kind of loop. break ends the loop immediately, skipping any remaining iterations; it is how a search loop exits as soon as it finds what it was looking for, instead of scanning to the end. continue abandons the current iteration — the rest of the body is skipped — and proceeds to the next one; it is how a cleaning loop discards a bad row without nesting the entire body inside an if. The two are easy to confuse in a multiple-choice question and impossible to confuse in practice once you have seen them side by side: break prints one line and stops; continue prints all the lines except the skipped ones.

The while loop adds 5, 4, 3, 2, 1 and prints 15; the test n > 0 becomes False when n reaches 0. The break loop prints First multiple of 7: 7 and stops — it never looks at 14, 21, or the other five multiples up to 50. The continue loop prints 1 2 4 5 7 8 10: 3, 6 and 9 hit the continue and their print is skipped, while the other seven reach it. The end=" " keyword argument to print replaces the default newline with a space, which is why the seven numbers appear on one line.

FizzBuzz: a chain where order is everything

The notebook’s control-flow exercise is the classic FizzBuzz: for the numbers 1 to 20, print Fizz for multiples of 3, Buzz for multiples of 5, FizzBuzz for multiples of both, and the number otherwise. Its entire difficulty is the first-true-branch rule. If you test “multiple of 3” first, then 15 — which is a multiple of both — receives Fizz and never reaches the FizzBuzz branch. The “both” case is the strictest condition, so it goes first; the two single-divisor cases follow; the plain number is the else. Notice, too, that “multiple of both 3 and 5” is the same as “multiple of 15”, so n % 15 == 0 is a one-test alternative to n % 3 == 0 and n % 5 == 0.

The joined line reads 1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 FizzBuzz 16 17 Fizz 19 Buzz, and the counts are 5 3 1: five plain Fizz (3, 6, 9, 12, 18), three plain Buzz (5, 10, 20), and one FizzBuzz at 15. Two small things in the code are worth noticing. The list collects labels as strings — str(n) converts the number — so that " ".join(fizz) can glue them together; join refuses to mix strings with integers. And the loop builds a list and then does something with it, which is the shape that the next subsection compresses into one line.

List comprehensions

Look at the loop that squares the numbers 0 to 5: create an empty list, iterate, compute, append. Four lines, of which only one — the expression x**2 — carries any information; the rest is scaffolding. A list comprehension removes the scaffolding: [x**2 for x in range(6)] reads as mathematicians write set-builder notation, \(\{x^2 : x \in \{0, \dots, 5\}\}\), and produces the identical list. The general pattern is

[expression for item in iterable if condition]

with the if condition optional. Python evaluates it left to right in the order the words appear after the expression: for each item in the iterable, if the condition holds, evaluate the expression and add the result to the new list. It is not merely shorter; because the loop runs inside the interpreter’s C code rather than as bytecode append calls, it is typically faster, and — more importantly — the reader can see at a glance that the code builds a list, transforms every element the same way, and has no side effects.

[0, 1, 4, 9, 16, 25] twice, and True — the comprehension and the loop agree exactly. With the filter, [0, 4, 16, 36, 64]: the trailing if x % 2 == 0 keeps only 0, 2, 4, 6 and 8, and the expression squares those five. Note that the filter is applied before the expression; odd values never reach x**2. The loop version needs a continue to achieve the same, and again the two agree: True.

The trap in comprehensions is the position of the if. An if after the for is a filter: it decides whether an element gets in, cannot have an else, and may shrink the output. An if … else before the for is a conditional expression — Python’s ternary operator, a if condition else b — that computes one value per element, never drops anything, and must have an else (otherwise the expression has no value for the other case). So ["even" if x % 2 == 0 else "odd" for x in range(6)] has six elements, one label per number; [x for x in range(6) if x % 2 == 0] has three. Writing [x if x % 2 == 0 for x in range(6)] is a SyntaxError, which at least fails loudly. The label pattern is the ancestor of np.where(cond, "even", "odd") in Chapter 1, which does the same thing to a whole array at once.

['even', 'odd', 'even', 'odd', 'even', 'odd'] 6 — six labels, one per element, nothing dropped. [0, 2, 4] 3 — the filter form kept half. The upper-casing comprehension prints ['NYC', 'LA', 'CHICAGO']; the same syntax with braces and a key: value expression builds a dictionary, {'NYC': 3, 'LA': 2, 'Chicago': 7}, mapping each city to the length of its name. Dictionary and set comprehensions are the same idea as list comprehensions and you will meet them in Chapter 2 building column-renaming maps.

When should you not use a comprehension? When the body has side effects (printing, writing to a file), when it needs more than one statement, when it nests more than two levels deep, or when the expression is long enough that the line wraps. A comprehension that a colleague has to read twice has cost more than the four lines it saved.

A rolling mean over 20-price windows. The prices are the integers 100 to 129, so the first window (100 to 119) should average 109.5. The code runs cleanly and prints a plausible number. Find the bug.

The first mean prints as 109.0, not 109.5, and the second line reveals why: the window has 19 prices. Under the half-open rule prices[i:i + window - 1] covers positions i to i + window - 2, which is window - 1 elements; the writer added - 1 because they were thinking of the last index of an inclusive range, not the stop of an exclusive one. The window silently dropped its final price, and every mean in means is the mean of the wrong nineteen numbers. The fix is prices[i:i + window], after which len() is 20 and the first mean is 109.5. This is the Reinhart–Rogoff error in five lines: nothing raised, the number was plausible, and only checking the length exposed it. Print len() of a slice before you average it.

'C'. Python tests branches top to bottom and takes the first one that is True; 95 ≥ 60, so the C branch fires and the A branch is never examined. Order thresholds from strictest to loosest. No error is raised.

Four: 1, 2, 3, 4. range is start-inclusive, stop-exclusive, exactly like slicing, so range(a, b) has b - a elements. For 1 to 20 inclusive write range(1, 21).

An if after the for is a filter: it drops elements, cannot take an else, and the result may be shorter than xs. An if … else before the for is a conditional expression: it produces exactly one value per element, never drops anything, and must have the else. The first selects; the second labels.

Functions, Lambdas, f-strings, and Error Handling

A function is a named, reusable block of code with its own scope. Everything you call in this course — pd.read_csv, sm.OLS(...).fit, find_best_arima, PerformanceMeasure — is a function, and the way it accepts arguments, fills in defaults, returns results and reports failures follows rules you can learn from a six-line example. This section covers those rules: how a function is defined and what it returns when you forget return; how default arguments work and the one way they go wrong; how *args and **kwargs let a function accept any number of arguments; why a variable assigned inside a function vanishes outside; what a lambda is for; how f-strings format numbers; and how try/except turns a crash into a value. It ends with the mini-project that uses all of it.

Define, document, call — and what `return` does

def name(parameters): opens a function; the indented body follows; return value sends a value back to the caller and ends the function. The string immediately under the def line, in triple quotes, is the docstring — the function’s documentation, which help(name) displays and which is stored in name.__doc__. Get into the habit of writing one line: what the function returns, given what. help(pd.read_csv) is how you read the fifty keyword arguments of a pandas function without leaving the notebook, and that help text is nothing more than the docstring somebody wrote.

The detail that catches people is what happens without a return. A function that reaches the end of its body without returning — or that executes a bare return — returns the special value None. Printing inside a function is a side effect; it puts text on the screen, but it does not send anything back. So r = show(5) where show merely prints leaves r equal to None, and the next line, r + 1, raises TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'. If you have ever seen None where you expected a number, this is almost always why — the same reason nums = nums.append(50) sets nums to None in Section 0.2.

add(10, 32) returns 42, and add.__doc__ is the docstring Return the sum of a and b.. The call show(5) prints showing 5 as its side effect, and then r is None — the function returned nothing, so r holds None. help(add) prints the signature and the docstring in the standard format; the same command on any library function gives you the same layout, only longer.

Default arguments, and the mutable-default trap

A parameter may be given a default value in the def line: def greet(name, title="Student"). A caller who omits title gets "Student"; a caller who supplies title="Dr." overrides it. Defaults are why pd.read_csv(path) works with one argument and pd.read_csv(path, sep=";", index_col=0) with three — the function exposes sensible defaults for the dozens of options and lets you override the few you need. Parameters with defaults must come after parameters without, and callers may pass them by position or by name; naming them is clearer and is the convention for anything beyond the first one or two.

The trap is in when the default is evaluated. It is evaluated once, when the def statement runs, and the resulting object is stored with the function and reused on every call that omits the argument. For an immutable default — a string, a number, None — this is invisible, because nothing can change the object. For a mutable default — a list, a dictionary, a set — it means every call that omits the argument receives the same list, and any .append() inside the function accumulates across calls. def add_city(c, cities=[]) returns ["NYC"] on the first call and ["NYC", "LA"] on the second, because the second call’s cities is the very list the first call appended to. The names-and-objects model of Section 0.2 explains it exactly: the default is one object, and the parameter is a name bound to it on each call.

The idiom that avoids it is to default to None and create the mutable object inside the body: def add_city_safe(c, cities=None): cities = [] if cities is None else cities. Now a fresh list is built on every call that omits the argument, and callers who pass their own list still get to. You will see something=None as the default for list- and dict-valued parameters throughout pandas and scikit-learn, and this is the reason.

Hello Student Alex! then Hello Dr. Alex! — the default and the override. Then the trap: ['NYC'] on the first call and ['NYC', 'LA'] on the second, with no list ever passed by the caller; the second call inherited the first call’s mutation. The safe version prints ['NYC'] and then ['LA'], because each call that omits cities builds its own empty list. Note the is None test, which is the one idiomatic use of is; if not cities: would also treat an explicitly passed empty list as absent, which is a different and subtler bug.

`*args` and `**kwargs`: any number of arguments

Sometimes a function should accept however many values the caller has. print is the everyday example — print(a), print(a, b, c) — and max, min, zip are others. The mechanism is a parameter prefixed with a single star: def mean(first, *nums). The first positional argument goes to first; every additional positional argument is collected into nums as a tuple. Inside the function nums is an ordinary tuple — you can take len(nums), sum(nums), iterate over it. Requiring first as a separate parameter guarantees that the caller passed at least one value, so the division in a mean can never be by zero; that is a design choice, not a syntactic requirement.

Two stars collect keyword arguments instead: def connect(host, **options). Any keyword argument the caller passes that does not match a named parameter lands in options as a dictionary, key by name. This is how a function forwards styling options it does not itself understand to another function — df.plot(**plot_kwargs) hands a dictionary of matplotlib settings straight through — and how a library adds new options without breaking old callers. Inside the function, options.get("port", 5432) is exactly the dictionary .get with a default from Section 0.2.

mean(10) is 10.0 — nums is the empty tuple, the total is 10 and the count is 1 — and mean(10, 20, 30, 40, 100) is 40.0, the sum 200 over five values. connect("db.local") prints Connecting to db.local:5432, ssl=False, both defaults taken from .get; the second call overrides both and prints Connecting to db.local:6500, ssl=True.

The full signature puts these pieces in a fixed order: required parameters, then defaulted ones, then *args, then keyword-only parameters, then **kwargs. A parameter that appears after *args cannot be filled by position — there is no position left, because *args has absorbed them all — so it must be passed by name. That is a deliberate design tool: scale=1.0 placed after *args forces every caller to write scale=0.5, which is self-documenting, and it is the pattern behind df.sort_values("x", ascending=False), where ascending can only be given by keyword. Working through one call is the best way to fix the order in memory.

For demo(1, 2, 3, 4, scale=0.5), decide which values go to a, b, args, scale and kwargs, compute total, and then reveal all three calls.

demo(1) gives {'total': 11.0, 'options': {}}: a = 1, b takes its default 10, args is empty, scale is 1.0, and the float comes from multiplying by scale. demo(1, 2, 3, 4, scale=0.5) gives {'total': 5.0, 'options': {}}: a = 1 and b = 2 by position, 3 and 4 spill into args, and \((1 + 2 + 7) \times 0.5 = 5.0\). demo(1, theme="dark", scale=2.0) gives {'total': 22.0, 'options': {'theme': 'dark'}}: theme matches no named parameter and is captured in kwargs, while scale does match and is not.

`lambda`: a function with no name

A lambda is a function consisting of a single expression, written inline, with no name and no return keyword: lambda x: x**2 is the same function as def square(x): return x**2. Assigning a lambda to a name, as the first line of the next cell does, is only for illustration — if a function deserves a name it deserves a def. The place a lambda genuinely belongs is as an argument to another function that expects a function: the key= parameter of sorted, max and min; map; and in pandas .apply(lambda row: …) and .assign(col=lambda df: …). In every one of these you need a small rule — “sort by length”, “the larger of two columns” — that will be used once, right here, and giving it a name and a def three lines above would only separate the rule from the place it is used.

The key= argument deserves its own sentence, because it is the idea that turns sorting into a general tool. sorted(cities, key=lambda c: len(c)) does not sort the lengths; it sorts the cities, using the length of each as the thing to compare. The key function is called once per element to produce a sort key, and the elements are then ordered by their keys. Sorting (word, count) pairs by count, key=lambda kv: kv[1], is how the mini-project finds the most frequent words; sorting a list of tickers by their Sharpe ratio held in a dictionary is key=lambda t: sharpe[t].

[1, 4, 9, 16] twice — a comprehension and map are two spellings of “apply this to every element”; the comprehension is the one Python programmers prefer, map survives because it predates comprehensions. sorted(cities, key=lambda c: len(c)) gives ['LA', 'NYC', 'Chicago'], ordered by name length 2, 3, 7. max(cities, key=len) is Chicago; note that a built-in function such as len can be passed directly as the key — the lambda lambda c: len(c) is redundant whenever an existing function already does the job. With reverse=True the order flips to ['Chicago', 'NYC', 'LA']. A lambda holds one expression; if the rule needs two lines, or a loop, or a name that would help the reader, write a def.

Scope: what happens to a variable inside a function

Every function call gets its own local scope: a private namespace for the names assigned inside the body. When the function returns, that namespace is discarded. A name assigned inside the function — a parameter, or any variable on the left of an = — is local, and it shadows any global of the same name for the duration of the call without changing it. So after x = "global" and a function whose body does x = "local"; return x, calling the function returns "local", and printing x afterwards still shows "global". Two different objects, two different names that happen to be spelled the same, in two different scopes.

Reading is different from writing. A function that only reads a global name — return x + "!" with no assignment to x in the body — finds the global and uses it. Python decides whether a name is local by scanning the function body at definition time for assignments: if the name is assigned anywhere in the body, it is local everywhere in the body, which is the source of the one confusing error in this area. A function that first reads x and then assigns to it raises UnboundLocalError: local variable 'x' referenced before assignment, because the assignment made x local, and the read occurred before the local had a value. The global x declaration tells Python to treat x as the module-level name for writing as well; it works, and you should almost never use it, because a function that silently modifies global state is a function whose behaviour depends on what ran before it — precisely what makes code untestable. Pass values in as parameters and return results; let the caller decide what to do with them.

local then global: the function’s x and the module’s x are different objects. read_only() returns global! — it reads the module-level x because it never assigns to x. The last three lines show the pattern that replaces global: pass the counter in, return the new value, and let the caller rebind c, which prints 1. In statistical code the same discipline means a function that computes a rolling volatility takes the series as an argument and returns a new series, rather than reaching out to a global df and editing a column in place.

f-strings: numbers into text

An f-string is a string literal prefixed with f in which any expression inside braces is evaluated and inserted: f"{name} scored {score}". After the expression, a colon introduces a format specification that controls how a number is rendered, and four of its parts cover almost everything you will write in this course. .2f means fixed-point with two decimals. A number before the dot — 8.3f — is a minimum width, and the value is right-aligned within it, padded with spaces, which is how columns of numbers line up in a printed table. .2% multiplies by 100, appends a percent sign and shows two decimals, which is how a return of 0.0345 becomes 3.45%. A comma — :, or :,.0f — inserts thousands separators. The alignment characters <, > and ^ force left, right and centre alignment when combined with a width, and f"{n:<10}{c:>3}" is how the mini-project prints its word table.

Two cautions. Formatting rounds for display only; f"{pi:.2f}" shows 3.14 but pi is unchanged, so use round() when the rounded number itself will be reused. And f-strings are evaluated where they appear, so f"{x:.2f}" with x a string raises ValueError: Unknown format code 'f' for object of type 'str' — a useful error, because it tells you a column you thought was numeric is text.

zip is the small built-in that belongs next to f-strings because the two are so often used together: it walks two (or more) sequences in parallel, yielding a tuple of corresponding elements at each step, and for n, s in zip(names, scores) unpacks each pair. Its trap is that it stops at the shortest input, silently. Zipping three names against two scores yields two pairs and drops the third name without a word — a real hazard when aligning tickers and prices of different lengths. Python 3.10 added zip(..., strict=True), which raises ValueError on a length mismatch, and it is the form to use whenever the inputs are supposed to be the same length.

Maya scored 93.5 — one decimal, rounded for display. 3.14; then ' 3.142', in which repr shows the three leading spaces that pad the value to width 8; then 3. The finance line prints return 3.45%, AUM 1,250,000. The zip lines show [('Ana', 88), ('Bo', 92), ('Cy', 79)] and then, with the shorter score list, [('Ana', 88), ('Bo', 92)] — Cy gone, no warning. The final loop prints a two-column table, Ana 88, Bo 92, Cy 79, with names left-aligned in four characters and scores right-aligned in four.

`try` / `except`: turn a crash into a value

When Python cannot carry out an operation it raises an exception: an object describing what went wrong, which propagates up through the calling functions until something catches it or the programme stops with a traceback. A try block encloses code that might raise; an except SomeError clause catches that specific exception and runs instead, and the programme continues. safe_divide returns a / b normally and float("inf") — the IEEE infinity, larger than every finite number — when b is zero, so that a downstream min() over many results still works instead of crashing on the first zero denominator.

The discipline is to catch the specific exception you expect, and only that one. A bare except: catches everything — including NameError from a misspelt variable, TypeError from a wrong argument, and KeyboardInterrupt when you try to stop a runaway loop — and converts every one of those into silence. A copilot’s favourite fix for an error is try: … except: pass; the correct response is to ask which exception is being caught and what value is returned instead. If the answer is “all of them” and “nothing”, reject the patch. The exceptions you have already met in this chapter are the ones you will meet most often, and their names tell you where to look:

Exception	Raised by	Seen in this chapter
`ZeroDivisionError`	`x / 0`	`safe_divide`
`KeyError`	`d["missing"]`	`student["gpa"]` before it was added
`IndexError`	`lst[99]`	a position past the end of a list
`TypeError`	`t[0] = 1`, `"a" + 1`, `set()[0]`, `None + 1`	tuples, sets, a forgotten `return`
`ValueError`	`int("abc")`, unpacking the wrong length	converting text; `zip(strict=True)`
`NameError`	using an undefined name	a misspelt variable

The distinction between TypeError and ValueError is the one worth learning: a TypeError says the kind of object is wrong for the operation (you cannot index a set at all); a ValueError says the kind is right but this particular value will not do (int("abc") is a legitimate call on a string, but that string is not a number). NameError is almost always a typo, and its message names the missing identifier.

5.0, then inf, then 2.5 — the infinity took part in min() and lost, as it should. to_number converts the two numeric strings and returns None for the two that are not: [3.5, 1000.0, None, None]. That last pattern — attempt the conversion, return a sentinel on failure, decide later what to do with the sentinels — is the core of every data-cleaning pipeline; pandas’ pd.to_numeric(errors="coerce") in Chapter 1 is the same idea applied to a whole column, with NaN as the sentinel.

Defining your own exceptions

An exception is a class, and ZeroDivisionError is one Python provides. You can define your own by subclassing Exception — class PositionLimitError(Exception): pass — and raise it with raise PositionLimitError("gross exposure 1.4 > 1.2"). Callers then catch PositionLimitError specifically, without accidentally swallowing a KeyError from the same block. In the trading-strategy projects, a custom exception for a breached limit is the cleanest way to stop a backtest with a message that says why.

Mini-project: Tiny Text Stats

The notebook’s closing exercise puts every tool from the chapter in one place. Given a short paragraph of raw text, find the three most frequent words. The problem decomposes into the four steps that every text-processing pipeline shares, and each step is one construct from this chapter. Tokenise: paragraph.split() turns the string into a list of words on whitespace, including the newline in the middle. Clean: each word is lower-cased and stripped of trailing punctuation with w.lower().strip(".,!?") — without this step, Data, data and data. are counted as three different words, which is the bug in the notebook’s starter code. Count: a dictionary accumulates counts with counts.get(key, 0) + 1, the .get-with-default idiom from Section 0.2, because a word not yet seen is normal. Rank: sorted(counts.items(), key=lambda kv: kv[1], reverse=True) orders the (word, count) pairs by count, descending, and a slice [:3] takes the top three. An f-string with width and alignment prints the table.

The cleaning function is worth writing as a separate def with a docstring, because it is the part most likely to change — later you will strip more punctuation, drop stop-words, or stem — and a function you can test on ["Data", "data.", "insight.", "Insight"] in isolation is a function you can change with confidence. That is also the answer to the copilot question: the three input/expected-output pairs you should demand before trusting generated code are exactly a test of this function.

The test prints {'data': 2, 'insight': 2}: the four raw tokens collapse to two cleaned words with two occurrences each, which is the behaviour the exercise specified. The paragraph splits into 19 tokens and, after cleaning, 13 distinct words. The top three are data with 4, insight with 3 and good with 2 — data appears as Data, data, data and data, and insight as insight., insight. and Insight, so without cleaning the answer would have been wrong on both counts. The printed table right-aligns the counts in a three-character column beside a ten-character word column.

The same skeleton — tokenise, clean, count, rank — is a bag-of-words sentiment score on an earnings-call transcript once “count every word” becomes “count words on a positive list and words on a negative list”. In Chapter 2 the dictionary becomes a Series (pd.Series(counts).sort_values(ascending=False) replaces the sorted call), and the standard library’s collections.Counter(words) replaces the counting loop with one call and a .most_common(3) method. What you gain in brevity you lose in transparency for a reader who has not seen Counter; for a five-line prototype, either is fine, and for a pipeline other people will maintain, the explicit version with a tested clean_and_count is the one to keep.

Mistakes library: Reinhart–Rogoff, 2010–2013

In Growth in a Time of Debt (2010), Carmen Reinhart and Kenneth Rogoff reported that countries with public debt above 90 % of GDP grew at −0.1 % a year on average — a number cited by the UK Treasury, the European Commission and the US Congress to justify austerity after 2010. In April 2013 Thomas Herndon, a UMass Amherst graduate student, obtained the working spreadsheet for a replication assignment. The averaging formula covered rows 30–44 instead of 30–49: Australia, Austria, Belgium, Canada and Denmark were silently excluded. Combined with a selective country-year sample and an unusual weighting, the corrected average for the >90 % bucket was +2.2 %, not −0.1 % (Herndon, Ash and Pollin, Cambridge Journal of Economics, 2014). The lesson for this chapter: a range that stops one row early is a slicing error, rows[30:45] when you meant rows[30:50]. Print len() of every slice before you average it, and write the calculation as a function you can test on a five-row example.

['NYC', 'LA']. The default list is created once, when def runs, and shared by every call that omits the argument; the first call’s append is still there for the second. Fix: default to None and build the list inside — cities = [] if cities is None else cities.

'local' then 'global'. Assignment inside a function creates a local name that shadows the global for the duration of the call; the global object is untouched. A function may read a global freely, but writing one needs global x, which you should almost never use — pass values in and return results instead.

It catches every exception — NameError from a typo, TypeError from a wrong argument, even KeyboardInterrupt — and turns each into silence, so bugs become quiet wrong answers. Catch the specific class you expect (except ZeroDivisionError:, except ValueError:) and return a value you can reason about, such as float('inf') or None.

Chapter Wrap-up

You can now read a line of Python and say, before running it, what it will print — which is the only skill this chapter set out to teach. Specifically, you can tell / from // and know that the first always produces a float and the second floors toward minus infinity; you know that 0.1 + 0.2 is not 0.3 and compare floats with a tolerance; you know that True is 1 and that sum(condition) counts. You can slice any sequence with the start-inclusive, stop-exclusive rule and compute the length of the slice by subtraction, and you know that range obeys the same rule. You hold the names-and-objects model: assignment attaches a label, mutation changes the object, and so b = a followed by a change to b alters a for a list and not for an integer — with .copy() when you want independence and is reserved for None. You can choose a tuple for a fixed record, a set for membership and de-duplication, and a dictionary for a lookup, and you know when a missing key should raise and when it should default. You can write a branch whose thresholds are ordered strictest-first, a loop that uses enumerate rather than an index, break to stop and continue to skip, and a comprehension in which you can tell a filter from a label. You can define a function with a docstring, know that it returns None without return, avoid the mutable default, accept *args and **kwargs, pass a lambda as a key=, keep state out of globals, format a number with :.2f, :.2% and :,, and catch the specific exception you expect.

Chapter 1 begins with the pandas Series, and the connection is direct: a Series is a dictionary that has learned arithmetic. Its index labels are keys, s["AAPL"] is a dictionary lookup that raises KeyError when the label is missing, s.iloc[0:4] is a four-element slice under the same half-open rule, s > 0 is a Series of booleans that sum() counts, s.apply(lambda x: …) is map with a lambda, s.rolling(20).mean() is the window loop from the debug exercise done correctly and vectorised, and s.dropna() returns a new object exactly as s.upper() did — you must assign it to keep it. Every trap in this chapter reappears there wearing pandas clothing, and every mental model transfers unchanged.

The companion slide deck for this chapter (Chapter 0 in the course slides) follows the same four sections with the same examples, and adds multiple-choice questions on each trap and “your turn” practice cells — FizzBuzz, clean_and_count, the top-three words — that check your answer automatically. Work through them before the Chapter 1 lecture; the ones you get wrong are the ones to revisit here.