docs: add architecture diagrams

README.md

@@ -16,22 +16,30 @@ in a Kubernetes cluster. The source of the content could be another node in the
 
 #### Important Disclaimer
 
-This is **work in progress** and not yet production ready. We are actively working on this project and would love to
-hear your feedback. Please feel free to open an issue or a pull request.
+**Work in Progress**: We are actively working on this project and would love to hear your feedback.
+Please feel free to open an issue or a pull request.
 
 ## Usage
 
-Peerd is designed to be deployed as a daemonset on every node in a Kubernetes cluster and acts as a registry mirror. It 
-discovers and serves content from other nodes in the cluster, and can also download content from an upstream source.
+**Peer D**aemon is designed to be deployed as a daemonset on every node in a Kubernetes cluster and acts as a registry
+mirror.
+
+* It discovers other nodes in the cluster and establishes a peer-to-peer overlay network in the cluster using the
+  [Kademlia DHT][white paper] protocol.
+
+* It discovers content such as OCI images in the node's containerd content store as well as streamable container files,
+  such as used in [Azure Artifact Streaming][ACR Artifact Streaming], and advertises them to its peers.
+
+* It can serve discovered/cached content to other nodes in the cluster, acting as a mirror for the content.
 
 This is useful in the following scenarios:
 
-1. **Increased Throughput**: For downloading large images or deploying large clusters, the registry can become a 
-   bottleneck. Peerd can be used to download images from other nodes in the cluster that have already downloaded it,
+1. **Increased Throughput**: For downloading large images or deploying large clusters, the container/artifact registry
+   can become a bottleneck. Peerd can be used to download images from other nodes in the cluster that have already downloaded it,
    increasing throughput.
 
-2. **Improved Fault Tolerance**: If the registry is unavailable, Peerd can still serve images from other nodes in the
-   cluster.
+2. **Improved Fault Tolerance**: If the upstream registry is unavailable, Peerd can still serve images from other nodes
+   in the cluster.
 
 3. **Firewall configuration**: Peerd can be used to download images from other nodes in the cluster. This can be useful
     in scenarios where outbound internet access is restricted on some nodes.

assets/mermaid/peerd-dht-topo.md

@@ -0,0 +1,54 @@
+graph TD;
+     subgraph Cluster[DHT Topology in a Kubernetes Cluster]
+        direction LR
+        subgraph peerd-1[Peerd]
+            dht-1(DHT)
+        end
+
+        subgraph peerd-2[Peerd]
+            dht-2(DHT)
+        end
+
+        subgraph peerd-3[Peerd]
+            dht-3(DHT)
+        end
+
+        subgraph Node1[Node A]
+            peerd-1
+        end
+
+        subgraph Node2[Node B]
+            peerd-2(Peerd)
+        end
+
+        subgraph Node3[Node C]
+            peerd-3(Peerd)
+        end
+
+        subgraph k8s-api[K8s API Server]
+            lease-1((("Peerd Leader
+            Lease Resource")))
+        end
+    end
+
+    dht-1 o-.-o |<b style="color:orange">Initialize<br><br></b>| lease-1
+    dht-2 o-.-o |<b style="color:orange">Initialize<br><br></b>| lease-1
+    dht-3 o-.-o |<b style="color:orange">Initialize<br><br></b>| lease-1
+
+    dht-1 <==> |<b style="color:blue">State<br><br></b>| dht-2
+    dht-1 <==> |<b style="color:blue">State<br><br></b>| dht-3
+    dht-2 <==> |<b style="color:blue">State<br><br></b>| dht-3
+
+    classDef cluster fill:#fafafa,stroke:#bbb,stroke-width:2px,color:#326ce5;
+    class Node1,NodeN cluster
+
+    classDef outer fill:#e0f7fa,stroke:#00008b,stroke-width:2px,color:#a9a9a9;
+    class Cluster outer
+
+    subgraph Legend[Legend]
+        direction TB
+        tls[<b style="color:orange">Initialize</b> - TLS connections]
+        mtls[<b style="color:blue">State</b> - mTLS connections]
+    end
+
+    Cluster ~~~ Legend

assets/mermaid/peerd-pull-seq.md

@@ -0,0 +1,40 @@
+sequenceDiagram
+    Title: Peer-to-Peer Image Pulling in a Kubernetes Cluster
+
+    box white Node A
+        participant Nginx Pod
+        participant Containerd Client
+        participant Peerd-A
+    end
+
+    box white Node N
+        participant Peerd-N
+    end
+
+    box white Upstream Registry
+        participant Upstream
+    end
+
+    loop Every layer
+        Containerd Client->>Peerd-A: GET sha256:l1
+        Note over Containerd Client,Peerd-A: 1
+
+        alt peer found
+            Peerd-A->>Peerd-N: GET sha256:l1
+            Note over Peerd-A,Peerd-N: 2
+            activate Peerd-N
+            Peerd-N->>Peerd-A: result
+            Peerd-A->>Containerd Client: result
+        else upstream request
+            Containerd Client->>Upstream: GET sha256:l1
+            Note over Peerd-A,Upstream: 3
+            Upstream->>Containerd Client: result
+        end
+
+        opt Advertise state (async)
+            activate Peerd-A
+            Note right of Peerd-A: Advertise state from containerd content store
+        end
+    end
+
+    Containerd Client-->Nginx Pod: start

assets/mermaid/peerd-pull.md

@@ -0,0 +1,67 @@
+graph TB;
+     subgraph Cluster[Peer-to-Peer Image Pulling in a Kubernetes Cluster]
+        direction LR
+        subgraph app-1[mcr.microsoft.com/nginx:latest]
+            app-pod-1((Pod))
+        end
+
+        subgraph ctr-1[Containerd]
+            client-1{Client}
+            store-1[[Content Store]]
+
+            subgraph hosts-1["Containerd Hosts Configuration"]
+                h-1[(mcr.microsoft.com pull mirror: peerd)]
+            end
+        end
+
+        subgraph peerd-1[Peerd]
+            proxy-1(Proxy)
+            sub-1(((Subscription)))
+        end
+
+        subgraph Node1[Node A]
+            hosts-1
+            app-1
+            peerd-1
+            ctr-1
+        end
+
+        subgraph NodeN[Node N]
+            peerd-n(Peerd)
+        end
+
+    end
+
+    subgraph manifest-1[mcr.microsft.com/nginx@sha256:m1]
+        direction TB
+        c-1[config sha256:c1]
+        l-1[layer sha256:l1]
+        l-2[layer sha256:l2]
+    end
+
+    subgraph Upstream[Upstream Container Registry]
+        acr(mcr.microsoft.com)
+    end
+
+    hosts-1 ~~~ client-1
+    c-1 ~~~ l-1
+    l-1 ~~~ l-2
+
+    client-1 --> |<b style="color:orange">1</b>| proxy-1
+    proxy-1 -.-> |<b style="color:orange">&nbsp&nbsp&nbsp&nbsp2</b>| peerd-n
+    client-1 -.-> |<b style="color:orange">&nbsp&nbsp&nbsp&nbsp3</b>| acr
+    client-1 --o |<b style="color:orange">&nbsp&nbsp&nbsp&nbsp4</b>| app-1
+
+    sub-1 o-.-o store-1
+    sub-1 o-.-o |<b style="color:darkgray">Advertise</b>| peerd-n
+
+    classDef containerd fill:#e0ffff,stroke:#000,stroke-width:4px,color:#000;
+
+    classDef cluster fill:#fafafa,stroke:#bbb,stroke-width:2px,color:#326ce5;
+    class Node1,NodeN cluster
+
+    classDef registry fill:#e0f7fa,stroke:#00008b,stroke-width:2px,color:#326ce5;
+    class acr registry
+
+    classDef outer fill:#e0f7fa,stroke:#00008b,stroke-width:2px,color:#a9a9a9;
+    class Cluster outer

assets/mermaid/peerd-streaming-seq.md

@@ -0,0 +1,55 @@
+sequenceDiagram
+    Title: Peer-to-Peer Artifact Streaming in a Kubernetes Cluster
+
+    box white Node A
+        participant Nginx Pod
+        participant File System
+        participant Overlaybd TCMU
+        participant Peerd-A
+    end
+
+    box white Node N
+        participant Peerd-N
+    end
+
+    box white Upstream Registry
+        participant Upstream
+    end
+
+    Nginx Pod->>File System: Read bytes 30-2000 from data.csv
+    Note over Nginx Pod,File System: 1
+
+    File System->>Overlaybd TCMU: Read bytes 30-2000 from data.csv
+    Note over File System,Overlaybd TCMU: 2
+
+    Overlaybd TCMU->>Peerd-A: Fetch file data.csv 'Range: bytes=30-2000'
+    Note over Overlaybd TCMU,Peerd-A: 3
+    activate Peerd-A
+
+    alt bytes cached
+        Peerd-A->>Overlaybd TCMU: result
+    else peer found
+        Peerd-A->>Peerd-N: Fetch file data.csv 'Range: bytes=30-2000'
+        Note over Peerd-A,Peerd-N: 4
+        activate Peerd-N
+        Peerd-N->>Peerd-A: result
+        Peerd-A->>Overlaybd TCMU: result
+    else upstream request
+        Peerd-A->>Upstream: Fetch file data.csv 'Range: bytes=30-2000'
+        Note over Peerd-A,Upstream: 5
+        Upstream->>Peerd-A: result
+        Peerd-A->>Overlaybd TCMU: result
+    end
+
+    opt Optimistic File Prefetch
+        activate Peerd-A
+        Note right of Peerd-A: Prefetch entire file from peers/upstream
+    end
+
+    opt Advertise state (async)
+        activate Peerd-A
+        Note right of Peerd-A: Advertise state from files cache
+    end
+
+    Overlaybd TCMU->>File System: result
+    File System->>Nginx Pod: result

assets/mermaid/peerd-streaming.md

@@ -0,0 +1,58 @@
+graph RL;
+     subgraph Cluster[Peer-to-Peer Artifact Streaming in a Kubernetes Cluster]
+        direction LR
+        subgraph kernel-1[Kernel]
+            fs-1[Filesystem]
+        end
+
+        subgraph app-1[Nginx]
+            app-pod-1((Pod))
+        end
+
+        subgraph overlaybd-1["(User Space)"]
+            driver-1["Overlaybd
+            TCMU"]
+        end
+
+        subgraph peerd-1[Peerd]
+            proxy-1(Proxy)
+            files-1(("Files
+            Cache"))
+        end
+
+        subgraph Node1[Node A]
+            kernel-1
+            app-1
+            overlaybd-1
+            peerd-1
+        end
+
+        subgraph NodeN[Node N]
+            peerd-n(Peerd)
+        end
+
+        files-1 o-.-o |<b style="color:darkgray"><br>Advertise</b>| peerd-n
+
+        app-pod-1 --> |<b style="color:orange"><br>1</b>| fs-1
+        fs-1 -.-> |<b style="color:orange"><br>2</b>| driver-1
+        driver-1 --> |<b style="color:orange"><br>3</b>| proxy-1
+        proxy-1 <-.-> |<b style="color:orange"><br>4</b>| peerd-n
+    end
+
+    subgraph Upstream[Upstream Container Registry]
+        acr(mcr.microsoft.com)
+    end
+
+    proxy-1 -.-> |<b style="color:orange"><br>5</b>| acr
+
+    classDef userspace fill:#e0ffff,stroke:#000,stroke-width:4px,color:#000;
+    class proxy-1,files-1,driver-1,app-pod-1,peerd-n userspace
+
+    classDef cluster fill:#fafafa,stroke:#bbb,stroke-width:2px,color:#326ce5;
+    class Node1,NodeN cluster
+
+    classDef registry fill:#e0f7fa,stroke:#00008b,stroke-width:2px,color:#326ce5;
+    class acr registry
+
+    classDef outer fill:#e0f7fa,stroke:#00008b,stroke-width:2px,color:#a9a9a9;
+    class Cluster outer

docs/design.md

@@ -1,10 +1,26 @@
 # Peerd Design
 
-![cluster-arch]
-
 The design is inspired from the [Spegel] project, which is a peer to peer proxy for container images that uses libp2p.
 In this section, we describe the design and architecture of `peerd`.
 
+#### **DHT Topology**
+
+|                   |
+| ----------------- |
+| ![peerd-dht-topo] |
+
+#### **Image Pulls Description** 
+
+|               |                   |
+| ------------- | ----------------- |
+| ![peerd-pull] | ![peerd-pull-seq] |
+
+#### **Image Streaming Description**
+
+|                    |                        |
+| ------------------ | ---------------------- |
+| ![peerd-streaming] | ![peerd-streaming-seq] |
+
 ### Background
 
 An OCI image is composed of multiple layers, where each layer is stored as a blob in the registry. When a container
@@ -188,6 +204,10 @@ running this container in p2p vs non-p2p mode on a 3 node AKS cluster with Artif
 
 ---
 
-[cluster-arch]: ../assets/images/cluster.png
 [file-system-layout]: ../assets/images/file-system-layout.png
 [Spegel]: https://github.com/XenitAB/spegel
+[peerd-pull]: ../assets/mermaid/rendered/peerd-pull.png
+[peerd-pull-seq]: ../assets/mermaid/rendered/peerd-pull-seq.png
+[peerd-streaming]: ../assets/mermaid/rendered/peerd-streaming.png
+[peerd-streaming-seq]: ../assets/mermaid/rendered/peerd-streaming-seq.png
+[peerd-dht-topo]: ../assets/mermaid/rendered/peerd-dht-topo.png