Docker build image and devcontainer config (#147)

* update rates url * Add docker image and devcontainer Only Linux and WSL on x86 are supported at the moment * SSL migration (#143) * Update coins to SSL config * Rename references to SSL coin config * Prefer SSL, but fall back to TCP if not available. `coins_config_ssl` delists coins without SSL support whereas `coins_config_tcp` favours SSL but falls back to TCP if not available. * Sync latest TCP file from coins repo * bump target SDK https://developer.android.com/google/play/requirements/target-sdk Signed-off-by: Kadan Stadelmann <[email protected]> * Bump app build Signed-off-by: Charl (Nitride) <[email protected]> --------- Signed-off-by: Kadan Stadelmann <[email protected]> Signed-off-by: Charl (Nitride) <[email protected]> Co-authored-by: CharlVS <[email protected]> * Replace python script with new dart version Also update dockerfile * Fix issues with new dart script as when used via CLI * Revert "Replace python script with new dart version" This reverts commit 0bde503. * Change from clone to COPY and fix permission issue * Add documentation and fix build directory permission issues * Update devcontainer image and fix permission issues * Fix devcontainer ndk install error Write permission denied to `/opt/android-sdk-linux` * Move changing ownership of android sdk folder to dockerfile * Replace Flutter archive download with git clone of Flutter repo Use a build matching the current architecture rather than downloading a prebuilt archive * Refactor to fetch Defi binaries only from GH releases (#148) * Refactor to fetch Defi binaries from GH releases Refactor to fetch Defi binaries from GH releases since they are built in a secure environment instead of the automated CI builds. * Fix script to use version tag Fix the script to use a version tag instead of a hash, as GH’s API does not support fetching a release tied to a specific commit. In the future, we can investigate if there’s an alternate way to verify/validate that the release build is indeed built from a given hash so we can change from a tag ref to a hash ref. * Fix bug in API fetch script Fix the API binary fetch script Ravioli ravioli * Remove the hardcoded platform from the devcontainer image * Copy over android sdk dockerfile from Cirrus CI Remove references to 3rd party dockerfiles, allow for local image tagging and simplify code review. Credit to https://github.com/cirruslabs/docker-images-android/tree/master * Fix bug in API fetch script Fix bug in API fetch script where fetching would fail if the android lib folders didn’t already exist. * Add script to abstract apk release build * Change devcontainer to docker-compose format Use local tagged image instead of the 3rd party image * Revert "Change devcontainer to docker-compose format" This reverts commit 95ce8bf. * Add note regarding build limitations and the Python script * Update fetch coins scripts * Add komodo-defi-framework build step to apk build image Also standardise base images to ubuntu:22.04 * Fix kdf branch build argument * Remove unnecessary copy Leftover artefact from previous build stage setup * Combine dockerfiles into `android-dev` for Github codespaces * Move API build to postAttach event of devcontainer API was built in image to reduce Codespaces startup time, but ended up increasing the already large image size * Fix cargo install in devcontainer/Codespaces * Set the minimum host requirements for Codespaces 4 cores, 16GB RAM and 32GB storage is the minimum requirements for the API to build successfully from source. --------- Signed-off-by: Kadan Stadelmann <[email protected]> Signed-off-by: Charl (Nitride) <[email protected]> Co-authored-by: Charl <[email protected]> Co-authored-by: smk762 <[email protected]> Co-authored-by: Kadan Stadelmann <[email protected]>
KomodoPlatform · Jul 13, 2024 · fe814ae · fe814ae
1 parent 28790c0
commit fe814ae
Show file tree

Hide file tree

Showing 15 changed files with 659 additions and 3 deletions.
diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
@@ -0,0 +1,17 @@
+{
+    "name": "flutter_docker",
+    "context": "..",
+    "dockerFile": "../.docker/android-dev.dockerfile",
+    "remoteUser": "komodo",
+    "postAttachCommand": "sh .docker/dev-setup.sh",
+    "runArgs": [
+        "--privileged"
+    ],
+    "workspaceMount": "source=${localWorkspaceFolder},target=/home/komodo/workspace,type=bind,consistency=delegated",
+    "workspaceFolder": "/home/komodo/workspace",
+    "hostRequirements": {
+        "cpus": 4,
+        "memory": "16gb",
+        "storage": "32gb"
+    }
+}
diff --git a/.docker/android-apk-build.dockerfile b/.docker/android-apk-build.dockerfile
@@ -0,0 +1,43 @@
+FROM komodo/kdf-android:latest as build 
+
+RUN cd /app && \ 
+    rustup default nightly-2022-10-29 && \
+    rustup target add aarch64-linux-android && \
+    rustup target add armv7-linux-androideabi && \
+    export PATH=$PATH:/android-ndk/bin && \
+    CC_aarch64_linux_android=aarch64-linux-android21-clang CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=aarch64-linux-android21-clang cargo rustc --target=aarch64-linux-android --lib --release --crate-type=staticlib --package mm2_bin_lib && \
+    CC_armv7_linux_androideabi=armv7a-linux-androideabi21-clang CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=armv7a-linux-androideabi21-clang cargo rustc --target=armv7-linux-androideabi --lib --release --crate-type=staticlib --package mm2_bin_lib && \
+    mv target/aarch64-linux-android/release/libmm2lib.a target/aarch64-linux-android/release/libmm2.a &&\
+    mv target/armv7-linux-androideabi/release/libmm2lib.a target/armv7-linux-androideabi/release/libmm2.a
+
+FROM komodo/android-sdk:34 as final
+
+ENV FLUTTER_VERSION="2.8.1"
+ENV FLUTTER_HOME "/home/komodo/.flutter-sdk"
+ENV USER="komodo"
+ENV PATH $PATH:$FLUTTER_HOME/bin
+ENV ANDROID_AARCH64_LIB=android/app/src/main/cpp/libs/arm64-v8a
+ENV ANDROID_AARCH64_LIB_SRC=/app/target/aarch64-linux-android/release/libmm2.a
+ENV ANDROID_ARMV7_LIB=android/app/src/main/cpp/libs/armeabi-v7a
+ENV ANDROID_ARMV7_LIB_SRC=/app/target/armv7-linux-androideabi/release/libmm2.a
+
+WORKDIR /app
+COPY . .
+
+RUN curl -o assets/coins.json https://raw.githubusercontent.com/KomodoPlatform/coins/master/coins && \
+    curl -o assets/coins_config.json https://raw.githubusercontent.com/KomodoPlatform/coins/master/utils/coins_config.json && \
+    mkdir -p android/app/src/main/cpp/libs/armeabi-v7a && \
+    mkdir -p android/app/src/main/cpp/libs/arm64-v8a && \
+    git clone https://github.com/flutter/flutter.git ${FLUTTER_HOME} && \
+    cd ${FLUTTER_HOME} && \
+    git fetch && \
+    git checkout tags/2.8.1 
+
+COPY --from=build --chown=$USER:$USER ${ANDROID_AARCH64_LIB_SRC} ${ANDROID_AARCH64_LIB}
+COPY --from=build --chown=$USER:$USER ${ANDROID_ARMV7_LIB_SRC} ${ANDROID_ARMV7_LIB}
+
+RUN flutter config --no-analytics  \
+    && flutter precache \
+    && yes "y" | flutter doctor --android-licenses \
+    && flutter doctor \
+    && flutter update-packages 
diff --git a/.docker/android-dev.dockerfile b/.docker/android-dev.dockerfile
@@ -0,0 +1,170 @@
+FROM docker.io/ubuntu:22.04
+
+ARG KDF_BRANCH=main
+ENV KDF_DIR=/kdf
+
+# Libz is distributed in the android ndk, but for some unknown reason it is not
+# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
+ENV CARGO_TARGET_X86_64_LINUX_ANDROID_LINKER=x86_64-linux-android-clang \
+    CARGO_TARGET_X86_64_LINUX_ANDROID_RUNNER="qemu-x86_64 -cpu qemu64,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt" \
+    CC_x86_64_linux_android=x86_64-linux-android-clang \
+    CXX_x86_64_linux_android=x86_64-linux-android-clang++ \
+    CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=armv7a-linux-androideabi21-clang \
+    CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_RUNNER=qemu-arm \
+    CC_armv7_linux_androideabi=armv7a-linux-androideabi21-clang \
+    CXX_armv7_linux_androideabi=armv7a-linux-androideabi21-clang++ \
+    CC_aarch64_linux_android=aarch64-linux-android21-clang \
+    CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=aarch64-linux-android21-clang \
+    CC_armv7_linux_androideabi=armv7a-linux-androideabi21-clang \
+    CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=armv7a-linux-androideabi21-clang \
+    DEP_Z_INCLUDE=/android-ndk/sysroot/usr/include/ \
+    OPENSSL_STATIC=1 \
+    OPENSSL_DIR=/openssl \
+    OPENSSL_INCLUDE_DIR=/openssl/include \
+    OPENSSL_LIB_DIR=/openssl/lib \
+    RUST_TEST_THREADS=1 \
+    HOME=/home/komodo/ \
+    TMPDIR=/tmp/ \
+    ANDROID_DATA=/ \
+    ANDROID_DNS_MODE=local \
+    ANDROID_ROOT=/system
+
+ENV FLUTTER_VERSION="2.8.1"
+ENV FLUTTER_HOME "/home/komodo/.flutter-sdk"
+ENV USER="komodo"
+ENV USER_ID=1000
+ENV PATH $PATH:$FLUTTER_HOME/bin
+ENV AR=/usr/bin/llvm-ar-16
+ENV CC=/usr/bin/clang-16
+ENV PATH="$HOME/.cargo/bin:$PATH"
+ENV PATH=$PATH:/android-ndk/bin
+ENV ANDROID_HOME=/opt/android-sdk-linux \
+    LANG=en_US.UTF-8 \
+    LC_ALL=en_US.UTF-8 \
+    LANGUAGE=en_US:en
+
+ENV ANDROID_SDK_ROOT=$ANDROID_HOME \
+    PATH=${PATH}:${ANDROID_HOME}/cmdline-tools/latest/bin:${ANDROID_HOME}/platform-tools:${ANDROID_HOME}/emulator
+
+# comes from https://developer.android.com/studio/#command-tools
+ENV ANDROID_SDK_TOOLS_VERSION 11076708
+
+# https://developer.android.com/studio/releases/build-tools
+ENV ANDROID_PLATFORM_VERSION 34
+ENV ANDROID_BUILD_TOOLS_VERSION 34.0.0
+
+# https://developer.android.com/ndk/downloads
+ENV ANDROID_NDK_VERSION 26.3.11579264
+
+RUN apt update && apt install -y sudo && \
+    useradd -u $USER_ID -m $USER && \ 
+    usermod -aG sudo $USER && \ 
+    echo "$USER ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
+
+USER $USER
+
+RUN sudo apt-get update -y && \
+    sudo apt-get install -y --no-install-recommends \
+    ca-certificates \
+    build-essential 	\
+    libssl-dev \ 
+    cmake \
+    llvm-dev \
+    libclang-dev \ 
+    lld \
+    gcc \
+    libc6-dev \
+    jq \
+    make \
+    pkg-config \
+    git \
+    automake \
+    libtool \
+    m4 \
+    autoconf \
+    make \
+    file \
+    curl \
+    wget \
+    gnupg \
+    software-properties-common 	\
+    lsb-release \
+    libudev-dev \
+    zip unzip \
+    binutils && \
+    sudo apt-get clean
+
+RUN sudo ln -s /usr/bin/python3 /bin/python &&\
+    sudo curl --output llvm.sh https://apt.llvm.org/llvm.sh && \
+    sudo chmod +x llvm.sh && \
+    sudo ./llvm.sh 16 && \
+    sudo rm ./llvm.sh && \
+    sudo ln -s /usr/bin/clang-16 /usr/bin/clang && \
+    PROTOC_ZIP=protoc-25.3-linux-x86_64.zip && \
+    sudo curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v25.3/$PROTOC_ZIP && \
+    sudo unzip -o $PROTOC_ZIP -d /usr/local bin/protoc && \
+    sudo unzip -o $PROTOC_ZIP -d /usr/local 'include/*' && \
+    sudo rm -f $PROTOC_ZIP && \
+    sudo mkdir $KDF_DIR && \
+    sudo chown -R $USER:$USER $KDF_DIR
+
+RUN PATH="$HOME/.cargo/bin:$PATH" && \
+    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && \
+    export PATH="$HOME/.cargo/bin:$PATH" && \ 
+    sudo chown -R $USER:$USER $HOME/.cargo && \
+    rustup toolchain install nightly-2022-10-29 --no-self-update --profile=minimal &&\
+    rustup default nightly-2022-10-29 && \
+    rustup target add aarch64-linux-android && \
+    rustup target add armv7-linux-androideabi && \
+    sudo apt install -y python3 python3-pip git curl nodejs python3-venv sudo && \
+    git clone https://github.com/KomodoPlatform/komodo-defi-framework.git $KDF_DIR && \
+    cd $KDF_DIR && \
+    git fetch --all && \
+    git checkout origin/$KDF_BRANCH && \
+    if [ "$(uname -m)" = "x86_64" ]; then \
+    bash ./scripts/ci/android-ndk.sh x86 23; \
+    elif [ "$(uname -m)" = "aarch64" ]; then \
+    bash ./scripts/ci/android-ndk.sh arm64 23; \
+    else \
+    echo "Unsupported architecture"; \
+    exit 1; \
+    fi
+
+RUN set -e -o xtrace \
+    && cd /opt \
+    && sudo chown -R $USER:$USER /opt \
+    && sudo apt-get update \
+    && sudo apt-get install -y jq \
+    openjdk-17-jdk \
+    wget zip unzip git openssh-client curl bc software-properties-common build-essential \
+    ruby-full ruby-bundler libstdc++6 libpulse0 libglu1-mesa locales lcov \
+    libsqlite3-dev --no-install-recommends \
+    # for x86 emulators
+    libxtst6 libnss3-dev libnspr4 libxss1 libatk-bridge2.0-0 libgtk-3-0 libgdk-pixbuf2.0-0 \
+    && sudo rm -rf /var/lib/apt/lists/* \
+    && sudo sh -c 'echo "en_US.UTF-8 UTF-8" > /etc/locale.gen' \
+    && sudo locale-gen \
+    && sudo update-locale LANG=en_US.UTF-8 \
+    && wget -q https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_TOOLS_VERSION}_latest.zip -O android-sdk-tools.zip \
+    && mkdir -p ${ANDROID_HOME}/cmdline-tools/ \
+    && unzip -q android-sdk-tools.zip -d ${ANDROID_HOME}/cmdline-tools/ \
+    && mv ${ANDROID_HOME}/cmdline-tools/cmdline-tools ${ANDROID_HOME}/cmdline-tools/latest \
+    && sudo chown -R $USER:$USER $ANDROID_HOME \
+    && rm android-sdk-tools.zip \
+    && yes | sdkmanager --licenses \
+    && sdkmanager platform-tools \
+    && git config --global user.email "[email protected]" \
+    && git config --global user.name "Komodo Platform" \
+    && yes | sdkmanager \
+    "platforms;android-$ANDROID_PLATFORM_VERSION" \
+    "build-tools;$ANDROID_BUILD_TOOLS_VERSION"
+
+RUN git clone https://github.com/flutter/flutter.git ${FLUTTER_HOME}  \
+    && cd ${FLUTTER_HOME}  \
+    && git fetch \
+    && git checkout tags/2.8.1 \
+    && flutter config --no-analytics  \
+    && flutter precache \
+    && yes "y" | flutter doctor --android-licenses \
+    && flutter doctor \
+    && flutter update-packages 
diff --git a/.docker/android-sdk.dockerfile b/.docker/android-sdk.dockerfile
@@ -0,0 +1,56 @@
+FROM docker.io/ubuntu:22.04
+
+# Credit to Cirrus Labs for the original Dockerfile
+# LABEL org.opencontainers.image.source=https://github.com/cirruslabs/docker-images-android
+
+USER root
+
+ENV ANDROID_HOME=/opt/android-sdk-linux \
+    LANG=en_US.UTF-8 \
+    LC_ALL=en_US.UTF-8 \
+    LANGUAGE=en_US:en
+
+ENV ANDROID_SDK_ROOT=$ANDROID_HOME \
+    PATH=${PATH}:${ANDROID_HOME}/cmdline-tools/latest/bin:${ANDROID_HOME}/platform-tools:${ANDROID_HOME}/emulator
+
+# comes from https://developer.android.com/studio/#command-tools
+ENV ANDROID_SDK_TOOLS_VERSION 11076708
+
+# https://developer.android.com/studio/releases/build-tools
+ENV ANDROID_PLATFORM_VERSION 34
+ENV ANDROID_BUILD_TOOLS_VERSION 34.0.0
+
+# https://developer.android.com/ndk/downloads
+ENV ANDROID_NDK_VERSION 26.3.11579264
+
+RUN set -o xtrace \
+    && cd /opt \
+    && apt-get update \
+    && apt-get install -y jq \
+    openjdk-17-jdk \
+    sudo wget zip unzip git openssh-client curl bc software-properties-common build-essential ruby-full ruby-bundler libstdc++6 libpulse0 libglu1-mesa locales lcov libsqlite3-dev --no-install-recommends \
+    # for x86 emulators
+    libxtst6 libnss3-dev libnspr4 libxss1 libatk-bridge2.0-0 libgtk-3-0 libgdk-pixbuf2.0-0 \
+    && rm -rf /var/lib/apt/lists/* \
+    && sh -c 'echo "en_US.UTF-8 UTF-8" > /etc/locale.gen' \
+    && locale-gen \
+    && update-locale LANG=en_US.UTF-8 \
+    && wget -q https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_TOOLS_VERSION}_latest.zip -O android-sdk-tools.zip \
+    && mkdir -p ${ANDROID_HOME}/cmdline-tools/ \
+    && unzip -q android-sdk-tools.zip -d ${ANDROID_HOME}/cmdline-tools/ \
+    && mv ${ANDROID_HOME}/cmdline-tools/cmdline-tools ${ANDROID_HOME}/cmdline-tools/latest \
+    && chown -R root:root $ANDROID_HOME \
+    && rm android-sdk-tools.zip \
+    && echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers \
+    && yes | sdkmanager --licenses \
+    && wget -O /usr/bin/android-wait-for-emulator https://raw.githubusercontent.com/travis-ci/travis-cookbooks/master/community-cookbooks/android-sdk/files/default/android-wait-for-emulator \
+    && chmod +x /usr/bin/android-wait-for-emulator \
+    && sdkmanager platform-tools \
+    && mkdir -p /root/.android \
+    && touch /root/.android/repositories.cfg \
+    && git config --global user.email "[email protected]" \
+    && git config --global user.name "Komodo Platform" \
+    && yes | sdkmanager \
+    "platforms;android-$ANDROID_PLATFORM_VERSION" \
+    "build-tools;$ANDROID_BUILD_TOOLS_VERSION" \
+    && yes | sdkmanager "ndk;$ANDROID_NDK_VERSION"
diff --git a/.docker/build_apk_release.sh b/.docker/build_apk_release.sh
@@ -0,0 +1,6 @@
+set -e
+
+docker build -f .docker/kdf-android-build.dockerfile . -t komodo/kdf-android --build-arg KDF_BRANCH=main
+docker build -f .docker/android-sdk.dockerfile . -t komodo/android-sdk:34
+docker build -f .docker/android-apk-build.dockerfile . -t komodo/komodo-wallet-mobile
+docker run --rm -v ./build:/app/build komodo/komodo-wallet-mobile:latest flutter build apk --release
diff --git a/.docker/build_config.json b/.docker/build_config.json
@@ -0,0 +1,28 @@
+{
+    "api": {
+        "release_tag": "v2.0.0-beta",
+        "use_latest_release": false,
+        "github_repository": "https://github.com/KomodoPlatform/komodo-defi-framework",
+        "platforms": {
+            "ios": {
+                "keywords": [
+                    "ios",
+                    "aarch64"
+                ],
+                "path": "ios"
+            },
+            "android-armv7": {
+                "keywords": [
+                    "android-armv7"
+                ],
+                "path": "android/app/src/main/cpp/libs/armeabi-v7a"
+            },
+            "android-aarch64": {
+                "keywords": [
+                    "android-aarch64"
+                ],
+                "path": "android/app/src/main/cpp/libs/arm64-v8a"
+            }
+        }
+    }
+}
diff --git a/.docker/dev-setup.sh b/.docker/dev-setup.sh
@@ -0,0 +1,24 @@
+#!/bin/bash
+
+set -e
+
+sudo git config core.fileMode false
+sudo chmod -R u+rwx /home/komodo/workspace
+sudo chown -R komodo:komodo /home/komodo/workspace
+flutter pub get
+
+curl -o assets/coins.json https://raw.githubusercontent.com/KomodoPlatform/coins/master/coins 
+curl -o assets/coins_config_tcp.json https://raw.githubusercontent.com/KomodoPlatform/coins/master/utils/coins_config_tcp.json 
+mkdir -p android/app/src/main/cpp/libs/armeabi-v7a 
+mkdir -p android/app/src/main/cpp/libs/arm64-v8a 
+
+cd /kdf 
+export PATH="$HOME/.cargo/bin:$PATH" 
+export PATH=$PATH:/android-ndk/bin
+CC_aarch64_linux_android=aarch64-linux-android21-clang CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=aarch64-linux-android21-clang cargo rustc --target=aarch64-linux-android --lib --release --crate-type=staticlib --package mm2_bin_lib
+CC_armv7_linux_androideabi=armv7a-linux-androideabi21-clang CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=armv7a-linux-androideabi21-clang cargo rustc --target=armv7-linux-androideabi --lib --release --crate-type=staticlib --package mm2_bin_lib
+mv target/aarch64-linux-android/release/libmm2lib.a target/aarch64-linux-android/release/libmm2.a 
+mv target/armv7-linux-androideabi/release/libmm2lib.a target/armv7-linux-androideabi/release/libmm2.a
+
+mv /kdf/target/aarch64-linux-android/release/libmm2.a /home/komodo/workspace/android/app/src/main/cpp/libs/arm64-v8a/libmm2.a
+mv /kdf/target/armv7-linux-androideabi/release/libmm2.a /home/komodo/workspace/android/app/src/main/cpp/libs/armeabi-v7a/libmm2.a