Core Streamlit Example (#66) (#67)

.github/workflows/test-examples.yaml

@@ -150,4 +150,4 @@ jobs:
         with:
           name: error-logs
           path: |
-            ${{ github.workspace }}/Tools/*.txt
+            ${{ github.workspace }}/Tools/*.txt

Core/README.md

@@ -0,0 +1,19 @@
+# Edge Core Examples
+
+## About
+
+This GitHub repo holds working examples of apps supported by Enthought Edge.
+
+Each example is a standalone container that inherits from `edm-centos-7`.
+
+The examples are self-contained, with their own README.md and "ci" module that
+contains build commands.  For new users, the recommended workflow for
+development is:
+
+* Make a copy of the example files.
+* Following the instructions in the README file, get to the point where the
+  example app is working locally.
+* Publish the example (possibly with minor changes to the config, such as adding
+  an image name that matches your Docker repository).
+* Ensure you are able to launch the app in Edge.
+* Modify your copy of the example as desired.

Core/Streamlit/.gitignore

@@ -0,0 +1,7 @@
+/dev_settings.json
+/dev_settings.json.disabled
+*.zbundle
+__pycache__/
+.DS_Store
+*.sqlite
+*.sqlite-journal

Core/Streamlit/Dockerfile

@@ -0,0 +1,63 @@
+# This is the Dockerfile for the Enthought Edge "Streamlit" core example.
+# 
+# We use "edm-centos-7" as the base image.  This is a Centos-7 based image
+# which includes EDM.
+#
+# EDM dependencies for the app are brought in via a .zbundle file.  This avoids
+# the need to pass your EDM token and/or a custom edm.yaml into the Dockerfile.
+#
+# We perform a two-stage build, to avoid including the .zbundle in the layers
+# of the published image.
+
+
+# IMPORTANT: Please do not define any of the EDGE_* environment variables, or
+# any of the JUPYTERHUB_* variables, directly in this file. They will be set
+# automatically when running in Edge, or by the "ci" module when running
+# locally.
+
+# First stage
+
+ARG BASE_IMAGE=quay.io/enthought/edm-centos-7:3.4.0
+
+FROM $BASE_IMAGE as stage_one
+
+ARG EDGE_BUNDLE=app_environment.zbundle
+COPY $EDGE_BUNDLE /tmp/app_environment.zbundle
+
+RUN adduser app
+USER app
+WORKDIR /home/app
+
+# Create a default EDM environment using the enthought_edge bundle
+RUN edm env import -f /tmp/app_environment.zbundle edm && edm cache purge --yes
+
+# Add a few 'pip' packages to the application environment
+RUN edm run -- python -m pip install --no-cache-dir \
+    streamlit streamlit-javascript streamlit-extras
+
+# Second stage
+
+FROM $BASE_IMAGE as stage_two
+
+RUN adduser app
+USER app
+WORKDIR /home/app
+
+COPY --from=stage_one --chown=app /home/app/.edm /home/app/.edm
+
+# Make any global changes (yum install, e.g.) in the second stage.
+# Don't change the user, and in particular don't make the user "root".
+
+# Copy startup script and application.
+# Note: the startup script must be placed in /home/app for the base image
+# machinery to pick it up.
+
+COPY --chown=app ./src/startup-script.sh /home/app/startup-script.sh
+RUN chmod +x /home/app/startup-script.sh
+COPY --chown=app ./src/app.py /home/app/app.py
+
+# Copy in the config file for Streamlit
+RUN mkdir /home/app/.streamlit
+COPY --chown=app ./src/config.toml /home/app/.streamlit
+
+CMD ["/home/app/startup-script.sh"]

Core/Streamlit/README.md

@@ -0,0 +1,178 @@
+# Streamlit example
+
+This example shows how to develop an application for Edge using the Streamlit
+library.  You can read more about Streamlit at their official site: 
+https://streamlit.io/.
+
+
+
+## Before you begin
+
+Before starting, ensure you have the following installed:
+
+* [Docker](https://docker.com)
+* [EDM](https://www.enthought.com/edm/), the Enthought Deployment Manager 
+
+Finally, ensure your ``edm.yaml`` file lists ``enthought/edge`` as an egg
+repository, along with ``enthought/free`` and ``enthought/lgpl``.  This will be
+necessary to use EdgeSession in the example.
+
+
+## Quick start
+
+1. Run ``python bootstrap.py``.  This will create a development environment
+   and activate it, dropping you into a development shell.
+
+2. From within the development shell, build the example by running
+   ``python -m ci build``.  This will produce a Docker image.
+
+3. Run the Docker image via ``python -m ci run``.  The app will serve on
+   http://0.0.0.0:9000 in a local development mode.
+
+
+## Modifying the example for your use case
+
+Please note these are the minimal changes necessary to make the example your
+own.  Depending on your use case you may wish to add additional "ci" commands,
+install different items in the Dockerfile, etc.
+
+* In ``ci/__main__.py``, change the constants IMAGE, VERSION, and
+  APP_DEPENDENCIES as appropriate.
+* In ``bootstrap.py``, consider changing the name of the development
+  environment (ENV_NAME) to avoid conflicts with other examples.
+
+
+## Publishing your app
+
+1. Ensure you are logged in (via ``docker login``) to DockerHub, Quay.io, or
+   another Docker registry.
+
+2. Run ``python -m ci publish`` to push your app's image to the registry.
+
+Once you have published your Docker image, you are ready to register that
+version of your app with Edge.
+
+As a quick reminder, in Edge, there is a distinction between an _app_ and an
+_app version_.  Your organization administrator can define an _app_, and then
+developers are free to register _app versions_ associated with that app.  This
+ensures that org administrators have full control over what shows up on the
+dashboard, but gives developers freedom to publish new versions by themselves.
+
+The easiest way to register a new app version is by logging in to Edge, going
+to the gear icon in the upper-right corner, and navigating to the app in
+question. There is a form to fill out which asks for the version, Quay.io or
+DockerHub URL for the image, and some other information.
+
+It is also possible to register app versions programmatically, for example
+from within a GitHub Actions workflow.  See the example at the end of this
+README.
+
+
+## Getting EdgeSession to work in development
+
+When you run your app on Edge, you can create an EdgeSession object simply by
+calling the constructor (``mysession = EdgeSession()``).  This works by
+collecting environment variables set by Edge when the container is launched.
+
+When developing locally, it's also convenient to have an EdgeSession.  You
+can get the "ci" module to inject the appropriate environment variables, so
+that your ``EdgeSession()`` call will work with ``python -m ci run`` and
+``python -m ci preflight``.  
+
+To do so, follow this procedure:
+
+* Copy the "dev_settings.json.example" file to "dev_settings.json".
+* Define EDGE_API_SERVICE_URL in that file.  The typical value here is
+  ``"https://edge.enthought.com/services/api"``.
+* Define EDGE_API_ORG in that file.  This is the "short name" displayed in
+  the URL bar when you log into an organization, for example, ``"default"``.
+* Define EDGE_API_TOKEN.  You can get one of these by going to the
+  ``/hub/token`` endpoint on the Edge server.
+
+Be sure *not* to check the "dev_settings.json" file into source control, as it
+contains your API token.
+
+
+## Viewing console output
+
+When running with ``python -m ci run``, the app's output will be displayed
+on the console where you launched it.
+
+## Guidelines for your Dockerfile
+
+Edge will run your app next to a built-in reverse proxy, which allows
+you to skip a lot of work in the development process.  This includes stripping
+the prefix from requests, handling the OAuth2 login flow, pinging JupyterHub
+for container activity, and more.  But, there are a few guidelines you will
+need to follow in your own Dockerfile.
+
+* Don't change the user to anything other than ``app`` (for example, by the 
+  Dockerfile ``USER`` command). If you need to run ``yum`` for some reason, 
+  use ``sudo``.
+* Your app should bind to ``127.0.0.1``, *not* ``0.0.0.0``, and it should serve
+  on port 9000.  The Edge machinery will respond to requests on port 8888 and 
+  forward them to your app.
+
+
+## Publishing versions from CI (e.g. GitHub Actions)
+
+You can register your app version programmatically.  This is particularly
+convenient during the development process, for automated builds.  A general
+example looks like this, for an app whose ID is ``my-app-id``:
+
+
+```
+from edge.api import EdgeSession
+from edge.apps.application import Application
+from edge.apps.app_version import AppKindEnum, AppVersion
+from edge.apps.server_info import ServerInfo
+
+# Create an Edge session.
+edge = EdgeSession(
+    service_url="https://edge.enthought.com/services/api",
+    organization="<YOUR_ORGANIZATION>",
+    api_token="<YOUR_EDGE_API_TOKEN>"
+)
+
+# Register the application version.
+
+# Icon is a base64-encoded PNG.  
+# It should be square, any size; 64x64 or 128x128 are common.
+ICON = (
+    "iVBORw0KGgoAAAANSUhEUgAAAIAAAACACAMAAAD04JH5AAAAk1BMVEUAAA"
+    "AMfIYAdnwAdnwAd3wAc3kAdntYsb8AdnwAdn0AdXwAdXsAd30Ad3xYsb8A"
+    "d30AdntYsL8Ad30AdnsAd30AdnwAd3wAd30AdnxYssBYsb5Ysr9Zs70Adn"
+    "tYsr9Ysb9Ysb9Ysr8AdnxXs75Ysb4AdXsAd3wAdnxYsb8AeoAAfYMAf4Vb"
+    "t8Zfv84Ad31ixtVevMuy+odXAAAAJ3RSTlMABPr88Shd+4pmkXyDbfB0Xy"
+    "dWTy0h9TLnkYv+IJmX9uigNzOIf5LOQ2WlAAAC/ElEQVR42sXX7VLiMBiG"
+    "4aTBDwQVEJSPXdkF3dAi1fM/uoXp7DyzIzHEm7bvT6Dc16SlDQbObG5anf"
+    "nHx4tpcea7q6sdWQPeXyyIgPc3+7nazQwY3N8sWhLM9uu/+Sd4NmBQvxog"
+    "YH0g4P2qLMFvAwb0WxE8q9+CwJqX96W6muV7U9fB8NfbsRV4u1ubhubHQf"
+    "C5PzQNjZXg/741dY8EdxKAPhTQPhfQPhfQPheQPhcM9wLa54KG+jYoCPZt"
+    "M4Llspn+KlRZr0PvrM7Z/7m9DHVCr19ub87Zd4UEX436hZeA9zPnJTix7z"
+    "IJcN97CU7tey8B70twev9Mgj+HvgQJ/fMIbqq+BCl9CXhfgpS+BLwvQUpf"
+    "At6XIN7nAvWTBOpDgfppgm6uPhGonyaw00LHAIH6qYLJNA8KeD8usF8Irk"
+    "E/SdChAvXbEajfjkD9dgTqtyNQv0XB9aFPBRcRAexzQXo/q1MQ77uyqHEN"
+    "4v2smPby0qUKHoOC1H7eGZtB4VMF47AgtX+hrU4Ngmj/cWzs+QU2rW/qFc"
+    "T7lSA/o6BvbEq/+rpRuuApiwvi/doE0X72pA/VJYj36xXE+7UL+sf7Tn0s"
+    "WPncBQTh51/Z0fV3oiDvBtegU/rQ/eC1OIpz5XRirEkQOFfch/8ulEfNru"
+    "gZcx8QVI+AuECnoGtM8NEc6N8aY0OC7ESB+qdvDdS33xQ8aH9A+0igfj74"
+    "Zh8K9BMEfSxwPgd9KKj6o8S+V58KaJ8LXJk/gD4WeM/6XMD6XDCC558LYJ"
+    "8LYJ8LQB8IYJ8IeD9+eG9LBPH7PxTgPhbQPhfwPhek932gDwTRfnjLyAW0"
+    "zwW8H5+IAPa5gPa5gPa5gPa5gPa5gPa5gPa5gPa5gPa5APa5gPW5YDJBfS"
+    "6YTkmfC1xZukb6NiiAfS4AfSAAfSQoHOpzwes2Q30+fQla6VsJQB8LeJ8L"
+    "wv0B6AMB6AMB6HMB7XMB7XMB7XMB7XMB7XMB7XMB7XMB7XMB7XMB7XMB7H"
+    "NBjvpccH0L+38B6kvWH2wXIe8AAAAASUVORK5CYII="
+)
+version = AppVersion(
+    app_id="my-app-id",  # Specified when the app is created
+    version="1.0.0",  	 # or whatever version you have
+    title="My Application Title",
+    description="This Is An Example Edge Application",
+    icon=ICON,
+    kind=AppKindEnum.Native,
+    link="quay.io/<YOUR_ORGANIZATION>/YOUR_IMAGE_NAME_HERE:TAG",
+    recommended_profile="edge.medium"
+)
+edge.applications.add_app_version(version)
+```

Core/Streamlit/bootstrap.py

@@ -0,0 +1,70 @@
+# Enthought product code
+#
+# (C) Copyright 2010-2022 Enthought, Inc., Austin, TX
+# All rights reserved.
+#
+# This file and its contents are confidential information and NOT open source.
+# Distribution is prohibited.
+
+"""
+    Bootstrap file, which builds the local EDM development environment for this
+    example.
+"""
+
+import argparse
+import subprocess
+
+ENV_NAME = "edge-streamlit-core"
+EDM_DEPS = ["click", "pip", "setuptools"]
+PIP_DEPS = ["sqlalchemy<2", "dockerspawner"]
+
+
+def bootstrap(ci_mode):
+    """Create and populate dev env.
+
+    Will automatically activate the environment, unless ci_mode is True.
+    """
+
+    if ENV_NAME not in _list_edm_envs():
+        print(f"Creating development environment {ENV_NAME}...")
+        cmd = ["edm", "envs", "create", ENV_NAME, "--version", "3.8", "--force"]
+        subprocess.run(cmd, check=True)
+
+        cmd = ["edm", "install", "-e", ENV_NAME, "-y"] + EDM_DEPS
+        subprocess.run(cmd, check=True)
+
+        cmd = ["edm", "run", "-e", ENV_NAME, "--", "pip", "install"] + PIP_DEPS
+        subprocess.run(cmd, check=True)
+
+        print("Bootstrap complete.")
+
+    else:
+        print("Environment already exists; reusing.")
+
+    if not ci_mode:
+        print(f"Activating dev environment {ENV_NAME}")
+        subprocess.run(["edm", "shell", "-e", ENV_NAME])
+
+
+def _list_edm_envs():
+    cmd = ["edm", "envs", "list"]
+    proc = subprocess.run(
+        cmd, check=True, capture_output=True, encoding="utf-8", errors="ignore"
+    )
+    envs = []
+    for line in proc.stdout.split("\n"):
+        parts = line.split()
+        if len(parts) < 6:
+            continue
+        if parts[0] == "*":
+            envs.append(parts[1])
+        else:
+            envs.append(parts[0])
+    return envs
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--ci", action="store_true")
+    args = parser.parse_args()
+    bootstrap(args.ci)