V0.2 (#1)

Thingy 0.2:
- move to Actix actor system
- add web UI and REST API

.cargo/config

@@ -0,0 +1,2 @@
+#[build]
+#target = "x86_64-unknown-linux-musl"

.dockerignore

@@ -0,0 +1,4 @@
+.idea
+.vscode
+target
+tmp

.vscode/launch.json

@@ -0,0 +1,49 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "type": "lldb",
+            "request": "launch",
+            "name": "Debug executable 'thingy'",
+            "cargo": {
+                "args": [
+                    "build",
+                    "--bin=thingy",
+                    "--package=thingy"
+                ],
+                "filter": {
+                    "name": "thingy",
+                    "kind": "bin"
+                }
+            },
+            "args": ["./tmp"],
+            "env": {
+                "RUST_BACKTRACE": "1",
+                "SERVE_STATIC": "true"
+            },
+            "cwd": "${workspaceFolder}"
+        },
+        {
+            "type": "lldb",
+            "request": "launch",
+            "name": "Debug unit tests in executable 'thingy'",
+            "cargo": {
+                "args": [
+                    "test",
+                    "--no-run",
+                    "--bin=thingy",
+                    "--package=thingy"
+                ],
+                "filter": {
+                    "name": "thingy",
+                    "kind": "bin"
+                }
+            },
+            "args": [],
+            "cwd": "${workspaceFolder}"
+        }
+    ]
+}

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "thingy"
-version = "0.1.1"
+version = "0.2.0"
 authors = ["n-k <[email protected]>"]
 description = "Lightweight build server and thing-doer"
 homepage = "https://github.com/n-k/thingy"
@@ -13,9 +13,13 @@ edition = "2018"
 
 [dependencies]
 serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
 serde_yaml = "0.8"
 git2 = "0.13"
 failure = "0.1"
 chrono = "0.4"
-tempfile = "3.1"
-file-lock = "1.1"
+tempfile = "3"
+actix = "0.10"
+actix-web = "3"
+actix-web-actors = "3"
+actix-files = "0.5.0"

Dockerfile

@@ -0,0 +1,15 @@
+# Build Stage
+FROM clux/muslrust AS builder
+WORKDIR /usr/src/thingy
+COPY . .
+RUN cargo install --target x86_64-unknown-linux-musl --path .
+
+# Bundle Stage
+FROM alpine
+WORKDIR /app
+COPY --from=builder /root/.cargo/bin/thingy .
+ENV LISTEN_ADDRESS=0.0.0.0
+WORKDIR /workspace
+EXPOSE 8080
+USER 1000
+CMD ["/app/thingy", "/workspace"]

README.md

@@ -1,43 +1,91 @@
 # thingy
 Lightweight build server and thing-doer
 
-## Using thingy
-Installation: `cargo install --force thingy`
+## Installation
+`cargo install --force thingy`
 
-Run: `thingy <path/to/workspace/folder/containing thingy.yaml>`
+## Usage
+Run: `thingy path/to/workspace/folder`, then go to `http://localhost:8080/`
 
-Thingy is a command line application and works inside a 'workspace' folder. A thingy workspace is a plain folder with a `thingy.yaml` file in it. This file's structure is based on [this struct](./src/models.rs#L4). This file lists build jobs and configurations.
+### Configuration
+Thingy has few configuration options, which are provided as optional environment variables:
+
+|Environment Variable|Default value| |
+|-|-|-|
+|`LISTEN_ADDRESS`|`127.0.0.1`|Address to bind web server to|
+|`LISTEN_PORT`|`8080`|Port web server listens on|
+
+
+Thingy works inside a 'workspace' folder. A thingy workspace is a plain folder with a `thingy.yaml` file in it. This file's structure is based on [this struct](./src/models.rs#L7). This file lists build jobs and configurations. If this file does not exist, an empty config with no jobs will be created. Jobs can then be added from web UI.
 
 An example of a workspace file:
 ```yaml
 jobs:
-  - name: "test"
+  - name: "test" # names must be unique within workspace
     repo_url: "[email protected]:n-k/thingy.git"
-    branch: "master"
-    build_script: "build.sh" # should be an executable file present in the repository
-    poll_interval_seconds: 300
+    build_script: "build.sh" # should be an executable file present in the repository, see build.sh in this repository for example
+    poll_interval_seconds: 300 # optional
     auth: # optional
-      PrivateKey: # currently only supported method, besides no auth
+      PrivateKey:
         path: "/path/to/your/ssh/private/key"
         passphrase: "optional - if key has passphrase"
-
+  - name: "test2"
+    repo_url: "../../some/path/to/repo.git"
+    build_script: "build.sh"
+    auth: # optional
+      UserPass:
+        username: "username"
+        password: "password"
 ```
-In this example, it is assumed that the repository contains an executable file `build.sh`. When a new commit is being built, thingy will pull the code, set CWD to the checkout directory, and run `build.sh` with a few special envronment variables. See next section for list of additional environment variables.
 
-### List of environment variables provided to build scripts
+In this example, it is assumed that the repository contains an executable file `build.sh`. When a new commit is being built, thingy will pull the code, and run `build.sh` in the checkout directory with a few special envronment variables. See next section for list of additional environment variables.
+
+### List of additional environment variables provided to build scripts
 - `BRANCH`: name of branch being built
 - `COMMIT_HASH`: current commit hash being built
+Any environment variables passed to the thingy executable are also passed to the buld processes.
+
+## Features
+- Multi-branch Git poll/build
+- REST API
+- Simple, but functional web interface
+- Log viewer, with tailing support for running builds
 
 ## Roadmap
-- [x] Single branch Git poll/build
-- [ ] Multi-branch Git poll/build
-- [ ] Web hooks
-- [ ] Secrets (other than auth)
+- Github account support - allow authenticating with github API token, and listing repositories.
+- Support docker builds. It would be good to have more support for docker bulds, but for now, having docker commands in the build scripts works well enough.
+- Secrets. It will be good to have better support for secret management. For now, the thingy.yaml file can have git credentials. This file is not expected to be shared or be public, so at least for my setup, it is safe to put credentials in it. 
+  For other secrets, any environment variables passed to the thingy executable are passed on to build processes, which can be used to, e.g., provide paths to files containing other secrets.
 
 ## FAQ
  1. Why?
  - This has the minimal set of features which I need for my personal projects, and home-lab automation things. Every other alternative seemed overkill for my needs. I also run this on Raspberry Pi's, and this project will always focus on low resource consumption.
  2. Is this going to be maintained? Will you add features?
- - I use this myself, so I will maintain at least the current features and a few more (please see roadmap section). If you would like to see some additional features, please open a Github issue, or send a PR.
+ - I use this myself, so I will maintain at least the current features and a few more (please see roadmap section). If you would like to see some additional features, please open a Github issue, or send a PR - both are very welcome.
  3. Why only Git support?
- - I only have Git repositories. PRs are very welcome for supporting others.
+ - I only have Git repositories.
+
+
+# Design
+Thingy works on top of Actix actors, and a REST API made with Actix-web. Each component in thingy is an actor.
+
+Actors in thingy form a tree, with one root. The organization looks like this:
+
+ - Thingy actor (root)
+    - 1 Actor per job
+      - 1 Actor per branch of the job's repository
+        - Temporary actors for each build
+
+## Structure of workspace directories
+```
+workspace_directory/
+  thingy.yaml (job definitions)
+  job_1/ (directory name is same as job name)
+    branch_1/
+      data.json (saved state for this branch, contains past/ongoing builds, last seen commit hash)
+      build_num.txt (number of latest build to have been started, keeps increasing by 1)
+      1/
+        repo/ (directory where this build cloned the repository)
+          ... files from repo ...
+        log.txt (build logs, both stdout and stderr are captured, and prefixed by [out] or [err])
+```

build.sh

@@ -1,3 +1,10 @@
 #!/bin/bash
+echo "===                       Sample build script                     ==="
+echo "= Note that this can be any executable file, like a python script   ="
 cargo test
 cargo build
+# also make musl build
+mkdir -p ./tmp
+docker build -t thingy:$(BRANCH) .
+docker run --rm -it -v "$PWD"/tmp:/tmp thingy:$(BRANCH) sh -c "cp -fv /app/thingy /tmp/"
+# distribute/deploy builds?