Skip to content

Commit

Permalink
Add some ancient text updates
Browse files Browse the repository at this point in the history
These mostly date back to 2017 but just sat in the worktree
uncommitted.

Fixes #6 from the looks of it.
  • Loading branch information
tokenrove committed Nov 23, 2023
1 parent 0609da5 commit 2cd6fa7
Show file tree
Hide file tree
Showing 5 changed files with 48 additions and 4 deletions.
22 changes: 21 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,14 @@ of different approaches.

# Supplementary Material

## Tools

- The [shtepper] is a great resource for understanding shell
execution. You can input an expression and see in excruciating
detail how it should be evaluated.

[shtepper](http://shell.cs.pomona.edu/shtepper)

## Documents

- [Advanced Programming in the Unix Environment] by Stevens covers
Expand Down Expand Up @@ -158,6 +166,8 @@ you're having trouble with a stage they cover:
its utilities in reasonable detail.
- there are [POSIX conformance test suites] but they don't seem to be
available in convenient, non-restricted forms.
- [yash's posix-shell-tests] are only runnable with yash, but the
tests themselves are full of useful ideas.

[A Unix Shell in Ruby]: http://www.jstorimer.com/blogs/workingwithcode/7766107-a-unix-shell-in-ruby
[Advanced Programming in the Unix Environment]: http://www.apuebook.com/
Expand All @@ -173,6 +183,7 @@ you're having trouble with a stage they cover:
[Unix system programming in OCaml]: https://ocaml.github.io/ocamlunix/
[Write a Shell in C]: https://brennan.io/2015/01/16/write-a-shell-in-c/
[POSIX conformance test suites]: https://www.opengroup.org/testing/testsuites/vscpcts2003.htm
[yash's posix-shell-tests]: https://github.com/posix-shell-tests/posix-shell-tests

## Shells to Examine

Expand All @@ -189,6 +200,7 @@ you're having trouble with a stage they cover:
- [oil]: Python and C++; has an extensive test suite.
- [xonsh](http://xon.sh/): Python.
- [oh](https://github.com/michaelmacinnis/oh): Go.
- [yash-rs](https://github.com/magicant/yash-rs): Rust.

[busybox]: https://git.busybox.net/busybox/tree/shell
[cash]: https://github.com/ShamoX/cash
Expand Down Expand Up @@ -234,7 +246,7 @@ There's also [sb-posix] in `sbcl` for the daring.
- use [the unix package](https://hackage.haskell.org/package/unix)
- [Hell] might be a starting point

[Hell]: https://github.com/chrisdone/hell/tree/master/src
[Hell]: https://github.com/chrisdone/hell/

### Java / JVM-based languages

Expand All @@ -247,8 +259,10 @@ There's also [jtux](http://basepath.com/aup/jtux/index.htm).
### Lua

There are a variety of approaches, but [ljsyscall] looks promising.
[luaposix] might be sufficient.

[ljsyscall]: https://github.com/justincormack/ljsyscall
[luaposix]: https://github.com/luaposix/luaposix

### OCaml

Expand All @@ -269,8 +283,14 @@ Although Python provides higher-level abstractions like
[`subprocess`], for the purposes of this workshop you probably want to
use the functions in [`os`].

Please note an important gotcha for stage 2! Since [Python 3.4], fds
have defaulted to non-inheritable, which means you'll need to
explicitly `os.set_inheritable(fd, True)` any file descriptor you
intend to pass down to a child.

[`os`]: https://docs.python.org/3/library/os.html
[`subprocess`]: https://docs.python.org/3/library/subprocess.html
[Python 3.4]: https://peps.python.org/pep-0446/

### Racket

Expand Down
2 changes: 2 additions & 0 deletions next-steps.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ sh`?
- GNU parallel is a very popular tool; is it possible to provide its
major features directly in a shell?

See also [`dgsh`](https://www2.dmst.aueb.gr/dds/sw/dgsh/).

## alternative uses of the shell

- it's often been noted that `make` and `cron`, maybe with a dash of
Expand Down
11 changes: 10 additions & 1 deletion stage_1.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ lists*, respectively. See section 2.9.3, [Lists]. We'll look at

(If you want to add support for compound lists surrounded by braces,
go ahead, but I didn't consider them important enough for an
interactive shell to bother testing them.
interactive shell to bother testing them.)

[Lists]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_03

Expand Down Expand Up @@ -213,6 +213,12 @@ See [Token Recognition] in the standard. Writing a tokenizer that
follows this, ignoring the parts we aren't doing yet, will make
writing your parser easier. I'll expand this section more, shortly.

You're faced with a tricky choice, here: I wrote this originally
trying to put off quoting and other difficulties until later, just
doing the simplest operations to get things going. But after writing
a few shells, I feel that getting proper parsing working is worth the
hassle. It's painful, but it makes everything else much easier.

[Token Recognition]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_03

## Notes
Expand Down Expand Up @@ -261,6 +267,9 @@ See also https://ewontfix.com/7/
Stevens, APUE, chapter 8.
Kerrisk, LPI, chapters 24 through 28.

A fork() in the road: https://www.microsoft.com/en-us/research/publication/a-fork-in-the-road/
Introducing `io_uring_spawn`: https://lwn.net/Articles/908268/

["A much faster popen and system implementation for Linux"]: https://blog.famzah.net/2009/11/20/a-much-faster-popen-and-system-implementation-for-linux/
[FreeBSD's posix_spawn]: https://github.com/freebsd/freebsd/blob/master/lib/libc/gen/posix_spawn.c
[glibc's generic implementation of posix_spawn]: https://github.com/bminor/glibc/blob/master/sysdeps/posix/spawni.c
Expand Down
14 changes: 13 additions & 1 deletion stage_2.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,10 @@ The following table summarizes the fd redirections:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07

Did you know about `<>`? That's something I only learned when I wrote
this workshop. Note there's also `>|` which interacts with the
shell's `noclobber` option, which I haven't bothered to implement.

Note that `!`, which you added in stage 1, may appear only at the
start of a pipeline, and affects the return value of the whole
pipeline.
Expand Down Expand Up @@ -130,7 +134,7 @@ but it's really convenient.
You might have a bunch of files open in your shell, for example, files
for maintaining history or configuration. Your children shouldn't
have to care about these files; file descriptors are a limited
resource and most peolple wouldn't appreciate having that limit
resource and most people wouldn't appreciate having that limit
unnecessarily decreased just because you were lazy.

How can you prevent your children from inheriting fds you don't want
Expand All @@ -145,6 +149,13 @@ about, particularly when writing a library that might open fds.
Recently, Linux [added an `fdmap` syscall] that could be used for
this.

Since I first wrote this, several languages have made `CLOEXEC` the
default, which means you'll have to disable it on the file descriptors
you actually want passed to the child. For example [PEP-446], adopted
in Python 3.4, makes `CLOEXEC` the default for all these reasons, so
you'll need to `os.set_inheritable(fd, True)` explicitly on the file
descriptors you actually want passed to the child.

See Chris Siebenmann's [fork() and closing file descriptors] and
CERT's [FIO22-C] (close files before spawning processes).

Expand All @@ -155,6 +166,7 @@ your shell, with various redirections, and see what happens.
[fork() and closing file descriptors]: https://utcc.utoronto.ca/~cks/space/blog/unix/ForkFDsAndRaces
[FIO22-C]: https://www.securecoding.cert.org/confluence/display/c/FIO22-C.+Close+files+before+spawning+processes
[added an `fdmap` syscall]: https://lwn.net/Articles/734709/
[PEP-446]: https://peps.python.org/pep-0446/

### Builtins

Expand Down
3 changes: 2 additions & 1 deletion stage_4.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,9 +127,10 @@ pipe.
There can actually be multiple heredocs on a line, but if you only
want to implement one for now, I think that's ok.

See [Heredocs].
See [Heredocs], [How to Parse Here Documents].

[Heredocs]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04
[How to Parse Here Documents]: http://www.oilshell.org/blog/2016/10/17.html

### Globbing

Expand Down

0 comments on commit 2cd6fa7

Please sign in to comment.