From e560222974ea5d0420a96c3ac2a8b3eac5da84c2 Mon Sep 17 00:00:00 2001 From: shape-warrior-t Date: Wed, 29 Nov 2023 02:23:03 -0500 Subject: [PATCH 1/6] Pythonic Iteration Patterns - first draft --- .../Pythonic-Iteration-Patterns.md | 238 ++++++++++++++++++ 1 file changed, 238 insertions(+) create mode 100644 Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md diff --git a/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md new file mode 100644 index 000000000..59b4b5815 --- /dev/null +++ b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md @@ -0,0 +1,238 @@ +# Pythonic Iteration Patterns + +Python is a language with many built-in conveniences that can greatly improve the straightforwardness of code. +In particular, some of the simple tools that it provides for iterating over things +can make code much more readable and, in some cases, efficient. + +I was inspired to write about this topic due to a thing that happened while developing my team's project. +A fellow team member used an interface of mine in an inefficient and roundabout way to get some data, +when my intended way of getting the same data was a one-liner +that makes simple use of Python's dictionary iteration methods. +Switching to the intended way of getting the data dramatically improved the performance of the code. +This was a lesson in communicating interfaces properly and not making assumptions about familiarity with certain things, +but it also serves as a nice example of how knowing Python's simple iteration tools can greatly improve one's code. + + + +## Basics `for` data types + +First off, lists, tuples, and ranges are not the only data types in Python that you can iterate over. +In particular, some other basic data types that can be iterated over are strings, dictionaries, and sets. +(There are _many, many_ other standard Python types that can also be iterated over, +and you can always define your own custom iterable types, but I'd like to just focus on these three for now.) + + +### Strings + +Iterating over a string means iterating over the individual characters: +```python +for char in 'monad': + print(char) # prints 'm', then 'o', then 'n', then 'a', then 'd' +``` + + +### Dictionaries + +Iterating over a dictionary means iterating over the _keys_: +```python +for key in {'a': 3, 'b': 1, 'c': 4}: + print(key) # prints 'a', then 'b', then 'c' + +# Alternatively: +for key in {'a': 3, 'b': 1, 'c': 4}.keys(): + print(key) +``` +To iterate over the _values_, use the `dict.values` method: +```python +for value in {'a': 3, 'b': 1, 'c': 4}.values(): + print(value) # prints '3', then '1', then '4' +``` +To iterate over both keys _and_ values, use the `dict.items` method: +```python +for key, value in {'a': 3, 'b': 1, 'c': 4}.items(): + print(f'{key} -> {value}') # prints 'a -> 3', then 'b -> 1', then 'c -> 4' +``` +[Starting in Python 3.7, iterating over a dict is guaranteed to produce items in insertion order.]( +https://stackoverflow.com/questions/39980323/are-dictionaries-ordered-in-python-3-6) + + +### Sets + +Lastly, you can also iterate over a set, which produces items in an unspecified order: +```python +for element in {'a', 'b', 'c'}: + print(element) # prints 'a', 'b', and 'c' in some order +``` + + + +## Other tools to look out `for` + +As a rule of thumb, if you find yourself writing code that looks something like this: +```python +for index in indices_of_a_collection: + do_something_with(index, collection[index]) +``` +...there's probably a more idiomatic way to go about things. + +Here are a few of those potential ways to go about things: + + +### `enumerate` +```python +lst = [3, 1, 4] + +# Instead of: +for i in range(len(lst)): + print(f'{i}: {lst[i]}') + +# Write: +for i, x in enumerate(lst): + print(f'{i}: {x}') # prints '0: 3', then '1: 1', '2: 4' +``` +`enumerate` also has an optional `start` argument that allows for counting from numbers other than 0: +```python +for i, char in enumerate('abc', start=1): + print(f'{i}: {char}') # prints '1: a', then '2: b', then '3: c' +``` + + +### `zip` +```python +lst_a = [1, 5, 9] +lst_b = [2, 6, 5] +lst_c = [3, 5, 8] + +# Instead of: +for i in range(len(lst_a)): + print(f'{lst_a[i]} & {lst_b[i]} & {lst_c[i]}') + +# Write: +for a, b, c in zip(lst_a, lst_b, lst_c): + print(f'{a} & {b} & {c}') # prints '1 & 2 & 3', then '5 & 6 & 5', then '9 & 5 & 8' + +# And instead of: +for i in range(len(lst_a)): + print(f'{i}: {lst_a[i]} & {lst_b[i]} & {lst_c[i]}') + +# Write: +for i, (a, b, c) in enumerate(zip(lst_a, lst_b, lst_c)): + print(f'{i}: {a} & {b} & {c}') # prints '0: 1 & 2 & 3', then '1: 5 & 6 & 5', then '2: 9 & 5 & 8' +``` +Note that this is only equivalent if all the arguments to `zip` have the same number of items to iterate over. +By default, iteration cuts off when any of the arguments to `zip` run out of items to iterate over. +`zip` also supports an optional `strict` keyword argument to raise an error +if the given iterables do not have the same number of items to iterate over. +Also see [`zip_longest` from the `itertools` module]( +https://docs.python.org/3/library/itertools.html#itertools.zip_longest). + + +### `reversed` +```python +s = 'defghi' + +# Instead of: +for i in range(len(s) - 1, -1, -1): + print(s[i]) + +# Or (this creates a copy of the string, which can be inefficient): +for char in s[::-1]: + print(char) + +# Write: +for char in reversed(s): + print(char) # prints 'i', then 'h', then 'g', then 'f', then 'e', then 'd' +``` + + +### `pairwise` +```python +s = 'monoid' + +# Instead of: +for i in range(len(s) - 1): + print(f'{s[i]}|{s[i + 1]}') + +# Write: +import itertools + +for char_a, char_b in itertools.pairwise(s): + print(f'{char_a}|{char_b}') # prints 'm|o', then 'o|n', then 'n|o', then 'o|i', then 'i|d' +``` +In general, [the `itertools` module](https://docs.python.org/3/library/itertools.html) +provides quite a few functions that can be helpful when trying to iterate over things. + + + +## Already done `for` you + +Python doesn't just have tools to make it nicer to write for loops and comprehensions -- +it also has a variety of functions that make it so that you don't _need_ to write those. + + +### Constructors: comprehension shortcuts + +First off, instead of writing `[x for x in iterable]` or `{x for x in iterable}`, +you can simply write `list(iterable)` and `set(iterable)`. Similar functionality also exists for `tuple` and `dict`. + + +### Grabbag of convenient functions + +Python comes with functions to compute the sum, product, maximum, or minimum of any iterable: +`sum`, `prod` from the `math` module, `max`, and `min`. +If you want to check if a criterion is satisfied for every item in an iterable, use the `all` function; +for at least one item in an iterable, the `any` function. + + +### `str.join` + +I've seen some variation of the following code from more than one person in relation to my team's project: +```python +result = '' + +for s in ['a', 'monad', 'is', 'a', 'monoid']: + result += s + ' | ' + +# remove the trailing ' | ' +# a better way to accomplish this would be the str.removesuffix method +result = result[:-3] + +# result == 'a | monad | is | a | monoid' +``` +Instead of writing that, simply write: +```python +result = ' | '.join(['a', 'monad', 'is', 'a', 'monoid']) +``` + + +### Removing the square brackets + +When using all of these functions, instead of giving a list comprehension as an argument, consider using a +[generator comprehension](https://stackoverflow.com/questions/364802/how-does-a-generator-comprehension-works) +instead -- instead of generating an entire list up front, +items are computed on demand, making things more memory-efficient. + +For example: +```python +lst = [3, 1, 4, 1, 5, 9] + +# Instead of: +sum([x * 2 for x in lst]) + +# Simply write: +sum(x * 2 for x in lst) +``` + + + +## Closing thoughts + +As you can see, there are a lot of nice little conveniences that Python provides to make one's life a little nicer. +Not all of the things showcased here are even new concepts that need time and effort to understand -- +many are simply little units of functionality that can still have an impact on the readability of your code. + +My knowledge of these tools was built over time as I programmed more and more in Python. +However, there are ways to stumble upon these sooner rather than later. +The way I'd like to highlight is: whenever there's a problem to solve, do some research into the idiomatic solutions. +A solution using previously unknown tools might be far more elegant +than the one you think up yourself using only what's known to you. From e2a1af8424f8ebc6dc3fc788feb947744c400699 Mon Sep 17 00:00:00 2001 From: Bonan Luan <91752863+shape-warrior-t@users.noreply.github.com> Date: Wed, 29 Nov 2023 02:28:31 -0500 Subject: [PATCH 2/6] Pythonic Iteration Patterns - first draft --- .../Pythonic-Iteration-Patterns.md | 238 ++++++++++++++++++ 1 file changed, 238 insertions(+) create mode 100644 Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md diff --git a/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md new file mode 100644 index 000000000..df8f947f8 --- /dev/null +++ b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md @@ -0,0 +1,238 @@ +# Pythonic Iteration Patterns + +Python is a language with many built-in conveniences that can greatly improve the straightforwardness of code. +In particular, some of the simple tools that it provides for iterating over things +can make code much more readable and, in some cases, efficient. + +I was inspired to write about this topic due to a thing that happened while developing my team's project. +A fellow team member used an interface of mine in an inefficient and roundabout way to get some data, +when my intended way of getting the same data was a one-liner +that makes simple use of Python's dictionary iteration methods. +Switching to the intended way of getting the data dramatically improved the performance of the code. +This was a lesson in communicating interfaces properly and not making assumptions about familiarity with certain things, +but it also serves as a nice example of how knowing Python's simple iteration tools can greatly improve one's code. + + + +## Basics `for` data types + +First off, lists, tuples, and ranges are not the only data types in Python that you can iterate over. +In particular, some other basic data types that can be iterated over are strings, dictionaries, and sets. +(There are _many, many_ other standard Python types that can also be iterated over, +and you can always define your own custom iterable types, but I'd like to just focus on these three for now.) + + +### Strings + +Iterating over a string means iterating over the individual characters: +```python +for char in 'monad': + print(char) # prints 'm', then 'o', then 'n', then 'a', then 'd' +``` + + +### Dictionaries + +Iterating over a dictionary means iterating over the _keys_: +```python +for key in {'a': 3, 'b': 1, 'c': 4}: + print(key) # prints 'a', then 'b', then 'c' + +# Alternatively: +for key in {'a': 3, 'b': 1, 'c': 4}.keys(): + print(key) +``` +To iterate over the _values_, use the `dict.values` method: +```python +for value in {'a': 3, 'b': 1, 'c': 4}.values(): + print(value) # prints '3', then '1', then '4' +``` +To iterate over both keys _and_ values, use the `dict.items` method: +```python +for key, value in {'a': 3, 'b': 1, 'c': 4}.items(): + print(f'{key} -> {value}') # prints 'a -> 3', then 'b -> 1', then 'c -> 4' +``` +[Starting in Python 3.7, iterating over a dict is guaranteed to produce items in insertion order.]( +https://stackoverflow.com/questions/39980323/are-dictionaries-ordered-in-python-3-6) + + +### Sets + +Lastly, you can also iterate over a set, which produces items in an unspecified order: +```python +for element in {'a', 'b', 'c'}: + print(element) # prints 'a', 'b', and 'c' in some order +``` + + + +## Other tools to look out `for` + +As a rule of thumb, if you find yourself writing code that looks something like this: +```python +for index in indices_of_a_collection: + do_something_with(index, collection[index]) +``` +...there's probably a more idiomatic way to go about things. + +Here are a few of those potential ways to go about things: + + +### `enumerate` +```python +lst = [3, 1, 4] + +# Instead of: +for i in range(len(lst)): + print(f'{i}: {lst[i]}') + +# Write: +for i, x in enumerate(lst): + print(f'{i}: {x}') # prints '0: 3', then '1: 1', '2: 4' +``` +`enumerate` also has an optional `start` argument that allows for counting from numbers other than 0: +```python +for i, char in enumerate('abc', start=1): + print(f'{i}: {char}') # prints '1: a', then '2: b', then '3: c' +``` + + +### `zip` +```python +lst_a = [1, 5, 9] +lst_b = [2, 6, 5] +lst_c = [3, 5, 8] + +# Instead of: +for i in range(len(lst_a)): + print(f'{lst_a[i]} & {lst_b[i]} & {lst_c[i]}') + +# Write: +for a, b, c in zip(lst_a, lst_b, lst_c): + print(f'{a} & {b} & {c}') # prints '1 & 2 & 3', then '5 & 6 & 5', then '9 & 5 & 8' + +# And instead of: +for i in range(len(lst_a)): + print(f'{i}: {lst_a[i]} & {lst_b[i]} & {lst_c[i]}') + +# Write: +for i, (a, b, c) in enumerate(zip(lst_a, lst_b, lst_c)): + print(f'{i}: {a} & {b} & {c}') # prints '0: 1 & 2 & 3', then '1: 5 & 6 & 5', then '2: 9 & 5 & 8' +``` +Note that this is only equivalent if all the arguments to `zip` have the same number of items to iterate over. +By default, iteration cuts off when any of the arguments to `zip` run out of items to iterate over. +`zip` also supports an optional `strict` keyword argument to raise an error +if the given iterables do not have the same number of items to iterate over. +Also see [`zip_longest` from the `itertools` module]( +https://docs.python.org/3/library/itertools.html#itertools.zip_longest). + + +### `reversed` +```python +s = 'defghi' + +# Instead of: +for i in range(len(s) - 1, -1, -1): + print(s[i]) + +# Or (this creates a copy of the string, which can be inefficient): +for char in s[::-1]: + print(char) + +# Write: +for char in reversed(s): + print(char) # prints 'i', then 'h', then 'g', then 'f', then 'e', then 'd' +``` + + +### `pairwise` +```python +s = 'monoid' + +# Instead of: +for i in range(len(s) - 1): + print(f'{s[i]}|{s[i + 1]}') + +# Write: +import itertools + +for char_a, char_b in itertools.pairwise(s): + print(f'{char_a}|{char_b}') # prints 'm|o', then 'o|n', then 'n|o', then 'o|i', then 'i|d' +``` +In general, [the `itertools` module](https://docs.python.org/3/library/itertools.html) +provides quite a few functions that can be helpful when trying to iterate over things. + + + +## Already done `for` you + +Python doesn't just have tools to make it nicer to write for loops and comprehensions -- +it also has a variety of functions that make it so that you don't _need_ to write those. + + +### Constructors: comprehension shortcuts + +First off, instead of writing `[x for x in iterable]` or `{x for x in iterable}`, +you can simply write `list(iterable)` and `set(iterable)`. Similar functionality also exists for `tuple` and `dict`. + + +### Grabbag of convenient functions + +Python comes with functions to compute the sum, product, maximum, or minimum of any iterable: +`sum`, `prod` from the `math` module, `max`, and `min`. +If you want to check if a criterion is satisfied for every item in an iterable, use the `all` function; +for at least one item in an iterable, the `any` function. + + +### `str.join` + +I've seen some variation of the following code from more than one person in relation to my team's project: +```python +result = '' + +for s in ['a', 'monad', 'is', 'a', 'monoid']: + result += s + ' | ' + +# remove the trailing ' | ' +# a better way to accomplish this would be the str.removesuffix method +result = result[:-3] + +# result == 'a | monad | is | a | monoid' +``` +Instead of writing that, simply write: +```python +result = ' | '.join(['a', 'monad', 'is', 'a', 'monoid']) +``` + + +### Removing the square brackets + +When using all of these functions, instead of giving a list comprehension as an argument, consider using a +[generator comprehension](https://stackoverflow.com/questions/364802/how-does-a-generator-comprehension-works) +instead -- instead of generating an entire list up front, +items are computed on demand, making things more memory-efficient. + +For example: +```python +lst = [3, 1, 4, 1, 5, 9] + +# Instead of: +sum([x * 2 for x in lst]) + +# Simply write: +sum(x * 2 for x in lst) +``` + + + +## Closing thoughts + +As you can see, there are a lot of nice little conveniences that Python provides to make one's life a little nicer. +Not all of the things showcased here are even new concepts that need time and effort to understand -- +many are simply little units of functionality that can still have an impact on the readability of your code. + +My knowledge of these tools was built over time as I programmed more and more in Python. +However, there are ways to stumble upon these sooner rather than later. +The way I'd like to highlight is: whenever there's a problem to solve, do some research into the idiomatic solutions. +A solution using previously unknown tools might be far more elegant +than the one you think up yourself using only what's known to you. From 3a20395db348b32dfebc5cf1500d8b2b56ca5874 Mon Sep 17 00:00:00 2001 From: shape-warrior-t Date: Thu, 30 Nov 2023 05:37:09 -0500 Subject: [PATCH 3/6] Pythonic Iteration Patterns - second draft --- .../Pythonic-Iteration-Patterns.md | 108 +++++++++++++----- 1 file changed, 77 insertions(+), 31 deletions(-) diff --git a/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md index 59b4b5815..15bdde2b9 100644 --- a/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md +++ b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md @@ -78,7 +78,7 @@ for index in indices_of_a_collection: Here are a few of those potential ways to go about things: -### `enumerate` +### [Counting as you iterate: `enumerate`](https://docs.python.org/3/library/functions.html#enumerate) ```python lst = [3, 1, 4] @@ -97,7 +97,7 @@ for i, char in enumerate('abc', start=1): ``` -### `zip` +### [Iteration in parallel: `zip`](https://docs.python.org/3/library/functions.html#zip) ```python lst_a = [1, 5, 9] lst_b = [2, 6, 5] @@ -110,24 +110,31 @@ for i in range(len(lst_a)): # Write: for a, b, c in zip(lst_a, lst_b, lst_c): print(f'{a} & {b} & {c}') # prints '1 & 2 & 3', then '5 & 6 & 5', then '9 & 5 & 8' - -# And instead of: -for i in range(len(lst_a)): - print(f'{i}: {lst_a[i]} & {lst_b[i]} & {lst_c[i]}') - -# Write: -for i, (a, b, c) in enumerate(zip(lst_a, lst_b, lst_c)): - print(f'{i}: {a} & {b} & {c}') # prints '0: 1 & 2 & 3', then '1: 5 & 6 & 5', then '2: 9 & 5 & 8' ``` Note that this is only equivalent if all the arguments to `zip` have the same number of items to iterate over. -By default, iteration cuts off when any of the arguments to `zip` run out of items to iterate over. +By default, iteration cuts off when any of the arguments to `zip` run out of items to iterate over: +```python +for a, b, c in zip([1, 5, 9, 2], [6, 5], [3, 5, 8]): + print(f'{a} & {b} & {c}') # prints '1 & 6 & 3', then '5 & 5 & 5' -- nothing else +``` `zip` also supports an optional `strict` keyword argument to raise an error -if the given iterables do not have the same number of items to iterate over. +if the given iterables do not have the same number of items to iterate over: +```python +for a, b, c in zip([1, 5, 9, 2], [6, 5], [3, 5, 8], strict=True): + # prints '1 & 6 & 3', then '5 & 5 & 5', then raises a ValueError since there are no more items to take from [6, 5] + print(f'{a} & {b} & {c}') +``` Also see [`zip_longest` from the `itertools` module]( -https://docs.python.org/3/library/itertools.html#itertools.zip_longest). +https://docs.python.org/3/library/itertools.html#itertools.zip_longest): +```python +import itertools + +for a, b, c in itertools.zip_longest([1, 5, 9, 2], [6, 5], [3, 5, 8], fillvalue=7): + print(f'{a} & {b} & {c}') # prints '1 & 6 & 3', then '5 & 5 & 5', then '9 & 7 & 8', then '2 & 7 & 7' +``` -### `reversed` +### [Easy, efficient backwards iteration: `reversed`](https://docs.python.org/3/library/functions.html#reversed) ```python s = 'defghi' @@ -145,7 +152,7 @@ for char in reversed(s): ``` -### `pairwise` +### [Iterating over adjacent indices: `itertools.pairwise`](https://docs.python.org/3/library/itertools.html#itertools.pairwise) ```python s = 'monoid' @@ -166,8 +173,8 @@ provides quite a few functions that can be helpful when trying to iterate over t ## Already done `for` you -Python doesn't just have tools to make it nicer to write for loops and comprehensions -- -it also has a variety of functions that make it so that you don't _need_ to write those. +Python doesn't just have tools to make it _nicer_ to write for loops and comprehensions -- +it also has a variety of functions that remove the need to write them in the first place. ### Constructors: comprehension shortcuts @@ -179,12 +186,29 @@ you can simply write `list(iterable)` and `set(iterable)`. Similar functionality ### Grabbag of convenient functions Python comes with functions to compute the sum, product, maximum, or minimum of any iterable: -`sum`, `prod` from the `math` module, `max`, and `min`. -If you want to check if a criterion is satisfied for every item in an iterable, use the `all` function; -for at least one item in an iterable, the `any` function. +[`sum`](https://docs.python.org/3/library/functions.html#sum), +[`prod` from the `math` module](https://docs.python.org/3/library/math.html#math.prod), +[`max`](https://docs.python.org/3/library/functions.html#max), +and [`min`](https://docs.python.org/3/library/functions.html#min). +If you want to check if a criterion is satisfied for every item in an iterable, +use the [`all`](https://docs.python.org/3/library/functions.html#all) function; +for at least one item in an iterable, +the [`any`](https://docs.python.org/3/library/functions.html#any) function. +```python +from math import prod + +sum([3, 1, 4]) # 8 +prod([3, 1, 4]) # 12 +max([3, 1, 4]) # 4 +min([3, 1, 4]) # 1 +all([True, True, True]) # True +all([True, False, True]) # False +any([False, True, False]) # True +any([False, False, False]) # False +``` -### `str.join` +### [Whatever-separated values: `str.join`](https://docs.python.org/3/library/stdtypes.html#str.join) I've seen some variation of the following code from more than one person in relation to my team's project: ```python @@ -205,30 +229,52 @@ result = ' | '.join(['a', 'monad', 'is', 'a', 'monoid']) ``` -### Removing the square brackets -When using all of these functions, instead of giving a list comprehension as an argument, consider using a -[generator comprehension](https://stackoverflow.com/questions/364802/how-does-a-generator-comprehension-works) -instead -- instead of generating an entire list up front, -items are computed on demand, making things more memory-efficient. +## `for` use on any iterable -For example: +With the (unfortunate, but [understandable](https://peps.python.org/pep-0322/#rejected-alternatives)) +exception of `reversed`, all of the functions outlined in this article +take more than just lists and strings as arguments -- they generally accept anything that can be iterated over. +In particular, you can combine various iteration tools together: ```python -lst = [3, 1, 4, 1, 5, 9] +lst_a = [1, 5, 9] +lst_b = [2, 6, 5] +lst_c = [3, 5, 8] # Instead of: -sum([x * 2 for x in lst]) +for i in range(len(lst_a)): + print(f'{i}: {lst_a[i]} & {lst_b[i]} & {lst_c[i]}') + +# Write: +for i, (a, b, c) in enumerate(zip(lst_a, lst_b, lst_c)): + print(f'{i}: {a} & {b} & {c}') # prints '0: 1 & 2 & 3', then '1: 5 & 6 & 5', then '2: 9 & 5 & 8' +``` + +One other notable type of iterable that can be used with the functions outlined in this article +(again, except for `reversed`) is _generator objects_. +Whenever you want to pass a list comprehension as an argument to one of these functions, consider using a +[generator comprehension](https://stackoverflow.com/questions/364802/how-does-a-generator-comprehension-works) instead: +```python +# Instead of: +sum([x ** 2 for x in range(1000)]) # Simply write: -sum(x * 2 for x in lst) +sum(x ** 2 for x in range(1000)) ``` +Instead of generating an entire list `[x ** 2 for x in range(1000)]` up front, +the generator-based approach computes values on demand. +Not only does this give us slightly shorter code, +but it's also more memory efficient when there are many values to iterate over. + +There's a lot more to generators than this, but that's out of scope for this article. For more info, see +[this StackOverflow question](https://stackoverflow.com/questions/1756096/understanding-generators-in-python?rq=3). ## Closing thoughts As you can see, there are a lot of nice little conveniences that Python provides to make one's life a little nicer. -Not all of the things showcased here are even new concepts that need time and effort to understand -- +Not all of the things showcased here are even new concepts that take time and effort to understand -- many are simply little units of functionality that can still have an impact on the readability of your code. My knowledge of these tools was built over time as I programmed more and more in Python. From 1ed905e36d418fad24efed1fc76a74191eb47e1e Mon Sep 17 00:00:00 2001 From: Bonan Luan <91752863+shape-warrior-t@users.noreply.github.com> Date: Thu, 30 Nov 2023 05:40:19 -0500 Subject: [PATCH 4/6] Pythonic Iteration Patterns - second draft --- .../Pythonic-Iteration-Patterns.md | 108 +++++++++++++----- 1 file changed, 77 insertions(+), 31 deletions(-) diff --git a/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md index df8f947f8..fa5ab2070 100644 --- a/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md +++ b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md @@ -78,7 +78,7 @@ for index in indices_of_a_collection: Here are a few of those potential ways to go about things: -### `enumerate` +### [Counting as you iterate: `enumerate`](https://docs.python.org/3/library/functions.html#enumerate) ```python lst = [3, 1, 4] @@ -97,7 +97,7 @@ for i, char in enumerate('abc', start=1): ``` -### `zip` +### [Iteration in parallel: `zip`](https://docs.python.org/3/library/functions.html#zip) ```python lst_a = [1, 5, 9] lst_b = [2, 6, 5] @@ -110,24 +110,31 @@ for i in range(len(lst_a)): # Write: for a, b, c in zip(lst_a, lst_b, lst_c): print(f'{a} & {b} & {c}') # prints '1 & 2 & 3', then '5 & 6 & 5', then '9 & 5 & 8' - -# And instead of: -for i in range(len(lst_a)): - print(f'{i}: {lst_a[i]} & {lst_b[i]} & {lst_c[i]}') - -# Write: -for i, (a, b, c) in enumerate(zip(lst_a, lst_b, lst_c)): - print(f'{i}: {a} & {b} & {c}') # prints '0: 1 & 2 & 3', then '1: 5 & 6 & 5', then '2: 9 & 5 & 8' ``` Note that this is only equivalent if all the arguments to `zip` have the same number of items to iterate over. -By default, iteration cuts off when any of the arguments to `zip` run out of items to iterate over. +By default, iteration cuts off when any of the arguments to `zip` run out of items to iterate over: +```python +for a, b, c in zip([1, 5, 9, 2], [6, 5], [3, 5, 8]): + print(f'{a} & {b} & {c}') # prints '1 & 6 & 3', then '5 & 5 & 5' -- nothing else +``` `zip` also supports an optional `strict` keyword argument to raise an error -if the given iterables do not have the same number of items to iterate over. +if the given iterables do not have the same number of items to iterate over: +```python +for a, b, c in zip([1, 5, 9, 2], [6, 5], [3, 5, 8], strict=True): + # prints '1 & 6 & 3', then '5 & 5 & 5', then raises a ValueError since there are no more items to take from [6, 5] + print(f'{a} & {b} & {c}') +``` Also see [`zip_longest` from the `itertools` module]( -https://docs.python.org/3/library/itertools.html#itertools.zip_longest). +https://docs.python.org/3/library/itertools.html#itertools.zip_longest): +```python +import itertools + +for a, b, c in itertools.zip_longest([1, 5, 9, 2], [6, 5], [3, 5, 8], fillvalue=7): + print(f'{a} & {b} & {c}') # prints '1 & 6 & 3', then '5 & 5 & 5', then '9 & 7 & 8', then '2 & 7 & 7' +``` -### `reversed` +### [Easy, efficient backwards iteration: `reversed`](https://docs.python.org/3/library/functions.html#reversed) ```python s = 'defghi' @@ -145,7 +152,7 @@ for char in reversed(s): ``` -### `pairwise` +### [Iterating over adjacent indices: `itertools.pairwise`](https://docs.python.org/3/library/itertools.html#itertools.pairwise) ```python s = 'monoid' @@ -166,8 +173,8 @@ provides quite a few functions that can be helpful when trying to iterate over t ## Already done `for` you -Python doesn't just have tools to make it nicer to write for loops and comprehensions -- -it also has a variety of functions that make it so that you don't _need_ to write those. +Python doesn't just have tools to make it _nicer_ to write for loops and comprehensions -- +it also has a variety of functions that remove the need to write them in the first place. ### Constructors: comprehension shortcuts @@ -179,12 +186,29 @@ you can simply write `list(iterable)` and `set(iterable)`. Similar functionality ### Grabbag of convenient functions Python comes with functions to compute the sum, product, maximum, or minimum of any iterable: -`sum`, `prod` from the `math` module, `max`, and `min`. -If you want to check if a criterion is satisfied for every item in an iterable, use the `all` function; -for at least one item in an iterable, the `any` function. +[`sum`](https://docs.python.org/3/library/functions.html#sum), +[`prod` from the `math` module](https://docs.python.org/3/library/math.html#math.prod), +[`max`](https://docs.python.org/3/library/functions.html#max), +and [`min`](https://docs.python.org/3/library/functions.html#min). +If you want to check if a criterion is satisfied for every item in an iterable, +use the [`all`](https://docs.python.org/3/library/functions.html#all) function; +for at least one item in an iterable, +the [`any`](https://docs.python.org/3/library/functions.html#any) function. +```python +from math import prod + +sum([3, 1, 4]) # 8 +prod([3, 1, 4]) # 12 +max([3, 1, 4]) # 4 +min([3, 1, 4]) # 1 +all([True, True, True]) # True +all([True, False, True]) # False +any([False, True, False]) # True +any([False, False, False]) # False +``` -### `str.join` +### [Whatever-separated values: `str.join`](https://docs.python.org/3/library/stdtypes.html#str.join) I've seen some variation of the following code from more than one person in relation to my team's project: ```python @@ -205,30 +229,52 @@ result = ' | '.join(['a', 'monad', 'is', 'a', 'monoid']) ``` -### Removing the square brackets -When using all of these functions, instead of giving a list comprehension as an argument, consider using a -[generator comprehension](https://stackoverflow.com/questions/364802/how-does-a-generator-comprehension-works) -instead -- instead of generating an entire list up front, -items are computed on demand, making things more memory-efficient. +## `for` use on any iterable -For example: +With the (unfortunate, but [understandable](https://peps.python.org/pep-0322/#rejected-alternatives)) +exception of `reversed`, all of the functions outlined in this article +take more than just lists and strings as arguments -- they generally accept anything that can be iterated over. +In particular, you can combine various iteration tools together: ```python -lst = [3, 1, 4, 1, 5, 9] +lst_a = [1, 5, 9] +lst_b = [2, 6, 5] +lst_c = [3, 5, 8] # Instead of: -sum([x * 2 for x in lst]) +for i in range(len(lst_a)): + print(f'{i}: {lst_a[i]} & {lst_b[i]} & {lst_c[i]}') + +# Write: +for i, (a, b, c) in enumerate(zip(lst_a, lst_b, lst_c)): + print(f'{i}: {a} & {b} & {c}') # prints '0: 1 & 2 & 3', then '1: 5 & 6 & 5', then '2: 9 & 5 & 8' +``` + +One other notable type of iterable that can be used with the functions outlined in this article +(again, except for `reversed`) is _generator objects_. +Whenever you want to pass a list comprehension as an argument to one of these functions, consider using a +[generator comprehension](https://stackoverflow.com/questions/364802/how-does-a-generator-comprehension-works) instead: +```python +# Instead of: +sum([x ** 2 for x in range(1000)]) # Simply write: -sum(x * 2 for x in lst) +sum(x ** 2 for x in range(1000)) ``` +Instead of generating an entire list `[x ** 2 for x in range(1000)]` up front, +the generator-based approach computes values on demand. +Not only does this give us slightly shorter code, +but it's also more memory efficient when there are many values to iterate over. + +There's a lot more to generators than this, but that's out of scope for this article. For more info, see +[this StackOverflow question](https://stackoverflow.com/questions/1756096/understanding-generators-in-python?rq=3). ## Closing thoughts As you can see, there are a lot of nice little conveniences that Python provides to make one's life a little nicer. -Not all of the things showcased here are even new concepts that need time and effort to understand -- +Not all of the things showcased here are even new concepts that take time and effort to understand -- many are simply little units of functionality that can still have an impact on the readability of your code. My knowledge of these tools was built over time as I programmed more and more in Python. From cfac62952184b76517eca03fb8be8144190beb14 Mon Sep 17 00:00:00 2001 From: shape-warrior-t Date: Thu, 30 Nov 2023 11:48:01 -0500 Subject: [PATCH 5/6] Pythonic Iteration Patterns - third draft --- Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md index 15bdde2b9..d9cb9d626 100644 --- a/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md +++ b/Topics/Tech_Stacks/Pythonic-Iteration-Patterns.md @@ -12,6 +12,8 @@ Switching to the intended way of getting the data dramatically improved the perf This was a lesson in communicating interfaces properly and not making assumptions about familiarity with certain things, but it also serves as a nice example of how knowing Python's simple iteration tools can greatly improve one's code. +Note: this article assumes a Python version of 3.10 or greater. + ## Basics `for` data types @@ -74,8 +76,7 @@ for index in indices_of_a_collection: do_something_with(index, collection[index]) ``` ...there's probably a more idiomatic way to go about things. - -Here are a few of those potential ways to go about things: +Here are a few of the iteration tools used in more idiomatic Python code: ### [Counting as you iterate: `enumerate`](https://docs.python.org/3/library/functions.html#enumerate) From 3b8ad262f47aa20da847cc1dc418c939121d7071 Mon Sep 17 00:00:00 2001 From: shape-warrior-t Date: Thu, 30 Nov 2023 12:13:57 -0500 Subject: [PATCH 6/6] Add topic to readme --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 6a1eccb1c..b36a4eca0 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,9 @@ Potential Topics-- 1. Set up 8. Unity 1. Introduction to Unity Basics - 2. A Beginner's Guide for Unity UI Design + 2. A Beginner's Guide for Unity UI Design + 9. Python + 1. Pythonic iteration patterns - Software Tools 1. Git