checkpoint

README.md

@@ -4,29 +4,38 @@
 
 * [The Missing Semester of Your CS Education](https://missing.csail.mit.edu)
 * [Every Programmer Should Know 🤔](https://github.com/mtdvio/every-programmer-should-know)
+* [Designing Data-Intesive Applications tree](https://anvaka.github.io/greview/ddia/1/)
 
 ## Table of Contents
 
-* [Languages Reference](languages-reference) -- Python, TypeScript, Go, Scala, and Haskell
-
+* [Software Engineering](softeng)
+* [Languages Reference](languages-reference) -- Python, TypeScript, Rust, Scala, and Haskell
+* [etc](etc/) -- everything that doesn't fit in one of these other categories
 * [Algorithms](algorithms/)
 * [Bash](bash/)
-* [Command-line](cmdline/)
+* [Command-line](cmdline/) -- *nix command line tools
 * [Domain-driven design](ddd/)
 * [Docker](docker/)
-* [etc](etc/)
 * [GDAL](gdal/)
 * [Git](git/)
-* [Make](make/)
+* [Insights](insights/)
 * [Nginx](nginx/)
 * [Python](python/)
+* [RESTful APIs](rest/)
 * [Scala](scala/)
 * [Vim](vim/)
 
 ## Other assemblages
 
 * [Project Awesome](https://project-awesome.org/)
 
+## Great Works
+
+* [Simple Made Easy](https://www.youtube.com/watch?v=oytL881p-nQ) by Rich Hickey (2012)
+* [How To Abstract Over Context](https://www.youtube.com/watch?v=uiorT754IwA) by Martin Odersky (2018)
+* [Inventing on Principle](http://worrydream.com/#!/InventingOnPrinciple) by Bret Victor (2012)
+* [RustConf 2020 Opening Keynote](https://www.youtube.com/watch?v=IwPRu5FhfIQ) by various (2020)
+
 ## License
 
 [LICENSE](LICENSE)

algorithms/README.md

@@ -35,17 +35,26 @@ free throw.
 
 ### More academic/technical
 
-* [Algorithms](https://algs4.cs.princeton.edu/home/) by Robert Sedgewick and Kevin Wayne. Also available on Coursera in [Part 1](https://www.coursera.org/learn/algorithms-part1) and [Part 2](https://www.coursera.org/learn/algorithms-part2)
-* [Introduction to Algorithms](https://mitpress.mit.edu/books/introduction-algorithms-third-edition) by Cormen, Leiserson, Rivest and Stein *a/k/a* CLR *a/k/a* "The Big White Book That Won't Fit on a Shelf"
 * [Algorithms](https://jeffe.cs.illinois.edu/teaching/algorithms/) by Jeff Erickson
 * [Algorithms Notes for Professionals](https://books.goalkicker.com/AlgorithmsBook/)
 * [Advanced Algorithms and Data Structures](https://www.manning.com/books/advanced-algorithms-and-data-structures) by Marcello La Rocca
 * [Data Structures & Algorithm Analysis](https://people.cs.vt.edu/~shaffer/Book/) by Clifford A. Shaffer and the [OpenDSA project](https://opendsa-server.cs.vt.edu/)
 * [The Algorithm Design Manual](https://www.algorist.com/) by Steven Skiena
 * [Algorithms in a Nutshell](https://learning.oreilly.com/library/view/algorithms-in-a/9781491912973/) by George T. Heineman, Gary Pollice, and Stanley Selkow
 * [TheAlgorithms](https://the-algorithms.com/) and [github](https://github.com/TheAlgorithms)
+* [Introduction to Algorithms](https://mitpress.mit.edu/books/introduction-algorithms-third-edition) by Cormen, Leiserson, Rivest and Stein *a/k/a* CLR *a/k/a* "The Big White Book That Won't Fit on a Shelf"
+* [Daniel G. Graham -- Algorithms](https://danielggraham.com/algorithms/)
+
+#### Courses 
+* [Algorithms](https://algs4.cs.princeton.edu/home/) by Robert Sedgewick and Kevin Wayne. Also available on Coursera in [Part 1](https://www.coursera.org/learn/algorithms-part1) and [Part 2](https://www.coursera.org/learn/algorithms-part2) 
 * [Coursera - Data Structures and Algorithms Specialization](https://www.coursera.org/specializations/data-structures-algorithms)
-* [Introduction to Algorithms (MIT SMA 5503)](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-046j-introduction-to-algorithms-sma-5503-fall-2005/)
+
+#### MIT Courses
+
+in prereq order:
+
+* [Mathematics for Computer Science -- MIT 6.042J](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-042j-mathematics-for-computer-science-spring-2015/)
+* [Intro to Algorithms -- MIT 6.006](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-006-introduction-to-algorithms-fall-2011/)
 * [Design and Analysis of Algorithms (MIT 6.046J / 18.410J)](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-046j-design-and-analysis-of-algorithms-spring-2015/)
 * [Advanced Data Structures (MIT 6.851)](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-851-advanced-data-structures-spring-2012/)
 

bash/README.md

@@ -2,29 +2,49 @@
 
 ## Resources
 
-* [Learn Bash the Hard Way](https://leanpub.com/learnbashthehardway) by Ian Miell
-* [Advanced Bash-Scripting Guide](https://tldp.org/LDP/abs/html/) by Mendel Cooper
-* [Classic Shell Scripting](https://learning.oreilly.com/library/view/classic-shell-scripting/0596005954/) by Arnold Robbins and Nelson H. F. Beebe
+- [Learn Bash the Hard Way](https://leanpub.com/learnbashthehardway) by Ian Miell
+- [Advanced Bash-Scripting Guide](https://tldp.org/LDP/abs/html/) by Mendel Cooper
+- [Classic Shell Scripting](https://learning.oreilly.com/library/view/classic-shell-scripting/0596005954/) by Arnold Robbins and Nelson H. F. Beebe
+
+##  Tools
+
+- [shellcheck](http://www.shellcheck.net/)
 
 ## Example
 
 ```sh
 #!/usr/bin/env bash
 set -Eeuo pipefail
-# set -x # print each command before exe
+# set -x # print each command before executing
 
 readonly VAR1="foo"
 
+function help(){
+  cat > /dev/stdout << END
+  Some help.
+END
+}
+
  #!/bin/bash
-while getopts "xy:z" opt
+while getopts "h?vxy:z" opt
 do
   case "$opt" in
+  h|\?) 
+          help
+          exit 0
+          ;;
+  v) VERBOSE=1 ;;
   x) echo '-x';;
   y) echo "-y ${OPTARG}";;
   z) echo '-z';;
   esac
 done
 
+if [[ ${VERBOSE} -gt 0 ]]
+then
+  set -x fi
+fi
+
 # todo
 
 if [[ 1 -lt 2 ]] 
@@ -195,16 +215,36 @@ String comparison: = == != < > -z (== '') -n (!= '')
 
 File test: -e exists
 
-## Scratch 
+## Heredoc
 
+substituted $MYVAR
+
+```bash
+cat > script.sh << END
+  #!/bin/bash
+  echo $MYVAR
+END
+```
+
+literal $MYVAR (END in quotes)
+
+```bash
+cat > script.sh << 'END'
+  #!/bin/bash 
+  echo $MYVAR
+END
 ```
+
+## Scratch 
+
+```bash
 exec 5>&1
 FF=$(echo aaa|tee >(cat - >&5))
 echo $FF
 ```
 
 
-```
+```bash
 if [ -z "${MY_VAR:-}" ]; then
   echo "MY_VAR was not set"
 fi

cloud/aws.md

@@ -4,25 +4,30 @@
 
 ### Include/Exclude 
 
+```sh
 aws s3 sync s3://bucket1/path/ s3://bucket2/path/ --include '*' --exclude '*.xml'
-
 aws s3 rm s3://bucket1/path --exclude '*' --include '*.jpg' --recursive
+```
 
 ### High-throughput copy configuration
 
+```sh
 aws configure set default.s3.max_concurrent_requests 1000
 aws configure set default.s3.max_queue_size 10000
 aws configure set default.s3.multipart_threshold 64MB
 aws configure set default.s3.multipart_chunksize 16MB
+```
 
 ### Accessing Requester Pays Buckets
 
 By default, the S3 storage, request, and data transfer costs are all paid for by the owner of the objects.  However, an owner may choose to enable the Requester Pays feature of S3 to have the request and data transfer costs changed to the person accessing the objects.  This requires a special flag to be added to requests so that the caller knows that they're going to be charged for the request.  
 
+```
 $ aws s3 cp s3://bucket/path/file.tif . 
 fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
 $ aws s3 cp s3://bucket/path/file.tif . --request-payer
 download: s3://bucket/path/file.tif  to ./file.tif
+```
 
 Note that if you get a message saying that the `--request-payer` option is invalid, you need to upgrade to a newer version of the AWS CLI.
 
@@ -61,6 +66,10 @@ There are three aspects of S3 usage that incur cost: Storage, Request, and Data
 
 Storage costs are always paid by the owner of the data.  
 
+### Static Website
+
+[Hosting a static website using Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html)
+
 ## Athena
 
 Create w/ ORC: