Better Python Code A Guide for Aspiring Experts

Move Beyond Python Code That "Mostly Works" to Code That Is Expressive, Robust, and Efficient

Python is arguably the most-used programming language in the world, with applications from primary school education to workaday web development, to the most advanced scientific research institutes. While there are many ways to perform a task in Python, some are wrong, inelegant, or inefficient. Better Python Code is a guide to "Pythonic" programming, a collection of best practices, ways of working, and nuances that are easy to miss, especially when ingrained habits are borrowed from other programming languages.

Author David Mertz presents concrete and concise examples of various misunderstandings, pitfalls, and bad habits in action. He explains why some practices are better than others, based on his 25+ years of experience as an acclaimed contributor to the Python community. Each chapter thoroughly covers related clusters of concepts, with chapters sequenced in ascending order of sophistication.

Whether you are starting out with Python or are an experienced developer pushing through the limitations of your Python code, this book is for all who aspire to be more Pythonic when writing better Python code.

• Use the right kind of loops in Python
• Learn the ins and outs of mutable and immutable objects
• Get expert advice to avoid Python "gotchas"
• Examine advanced Python topics
• Navigate the "attractive nuisances" that exist in Python
• Learn the most useful data structures in Python and how to avoid misusing them
• Avoid security mistakes
• Understand the basics of numeric computation, including floating point numbers and numeric datatypes

"My high expectations for this engaging Python book have been exceeded: it offers a great deal of insight for intermediate or advanced programmers to improve their Python skills, includes copious sharing of precious experience practicing and teaching the language, yet remains concise, easy to read, and conversational."
--From the Foreword by Alex Martelli

Register your book for convenient access to downloads, updates, and/or corrections as they become available. See inside book for details.

Foreword xvii
Preface xix
Acknowledgments xxv
About the Author xxvii

Introduction 1

Chapter 1: Looping Over the Wrong Things 3
1.1 (Rarely) Generate a List for Iteration 3
1.2 Use enumerate() Instead of Looping Over an Index 6
1.3 Don't Iterate Over dict.keys() When You Want dict.items() 8
1.4 Mutating an Object During Iteration 9
1.5 for Loops Are More Idiomatic Than while Loops 12
1.6 The Walrus Operator for "Loop-and-a-Half" Blocks 13
1.7 zip() Simplifies Using Multiple Iterables 15
1.8 zip(strict=True) and itertools.zip_longest() 17
1.9 Wrapping Up 20

Chapter 2: Confusing Equality with Identity 21
2.1 Late Binding of Closures 21
2.2 Overchecking for Boolean Values 25
2.3 Comparing x == None 28
2.4 Misunderstanding Mutable Default Arguments 29
2.5 Copies versus References to Mutable Objects 33
2.6 Confusing is with == (in the Presence of Interning) 35
2.7 Wrapping Up 37

Chapter 3: A Grab Bag of Python Gotchas 39
3.1 Naming Things 39
3.2 Quadratic Behavior of Naive String Concatenation 52
3.3 Use a Context Manager to Open a File 56
3.4 Optional Argument key to .sort() and sorted() 59
3.5 Use dict.get() for Uncertain Keys 62
3.6 Wrapping Up 64

Chapter 4: Advanced Python Usage 67
4.1 Comparing type(x) == type(y) 67
4.2 Naming Things (Revisited) 71
4.3 Keep Less-Used Features in Mind 79
4.4 Type Annotations Are Not Runtime Types 98
4.5 Wrapping Up 105

Chapter 5: Just Because You Can, It Doesn't Mean You Should... 107
5.1 Metaclasses 107
5.2 Monkeypatching 112
5.3 Getters and Setters 115
5.4 It's Easier to Ask for Forgiveness Than Permission 118
5.5 Structural Pattern Matching 121
5.6 Regular Expressions and Catastrophic Backtracking 123
5.7 Wrapping Up 126

Chapter 6: Picking the Right Data Structure 129
6.1 collections.defaultdict 129
6.2 collections.Counter 132
6.3 collections.deque 135
6.4 collections.ChainMap 138
6.5 Dataclasses and Namedtuples 141
6.6 Efficient Concrete Sequences 146
6.7 Wrapping Up 150

Chapter 7: Misusing Data Structures 153
7.1 Quadratic Behavior of Repeated List Search 153
7.2 Deleting or Adding Elements to the Middle of a List 157
7.3 Strings Are Iterables of Strings 163
7.4 (Often) Use enum Rather Than CONSTANT 166
7.5 Learn Less Common Dictionary Methods 169
7.6 JSON Does Not Round-Trip Cleanly to Python 174
7.7 Rolling Your Own Data Structures 178
7.8 Wrapping Up 187

Chapter 8: Security 189
8.1 Kinds of Randomness 190
8.2 Putting Passwords or Other Secrets in "Secure" Source Code 195
8.3 "Rolling Your Own" Security Mechanisms 198
8.4 Use SSL/TLS for Microservices 201
8.5 Using the Third-Party requests Library 205
8.6 SQL Injection Attacks When Not Using DB-API 208
8.7 Don’t Use assert to Check Safety Assumptions 212
8.8 Wrapping Up 215

Chapter 9: Numeric Computation in Python 217
9.1 Understanding IEEE-754 Floating Point Numbers 217
9.2 Numeric Datatypes 228
9.3 Wrapping Up 239

Appendix: Topics for Other Books 241
A.1 Test-Driven Development 241
A.2 Concurrency 242
A.3 Packaging 243
A.4 Type Checking 243
A.5 Numeric and Dataframe Libraries 244

Index 245

David Mertz has been a member of the Python community for 25 years. He has taught Python to scientists, developers coming from other languages, and to programming neophytes. He was a director of the Python Software Foundation (PSF) for six years and continues to serve on, or chair, a variety of PSF working groups. David is the author of several technical books and has given keynote addresses at numerous international programming conferences.