Proposal to add github action build test...

[mofosyne]

I got a flow with https://huggingface.co/mofosyne/TinyLLama-v0-5M-F16-llamafile/blob/main/llamafile-creation.sh

wonder if I can consistently do it in a dockerfile...

ARG UBUNTU_VERSION=22.04

FROM ubuntu:$UBUNTU_VERSION as build

RUN apt-get update && \
    apt-get install -y build-essential git

WORKDIR /app

COPY . .

RUN make -j8 

FROM ubuntu:$UBUNTU_VERSION as runtime

COPY --from=build /app/o/llama.cpp/main/main         /main.llamafile
COPY --from=build /app/o/llamafile/zipalign          /zipalign
COPY --from=build /app/models/TinyLLama-5M-F16.gguf  /model.gguf

RUN echo "-m" > .args && \
    echo "model.gguf" >> .args

RUN ./zipalign -j0 main.llamafile model.gguf .args

ENV LC_ALL=C.utf8

RUN ./main.llamafile --version
RUN ./main.llamafile --cli -p "hello world the gruff man said"

# docker build -f main.Dockerfile -t llamafile_test ..
# docker run llamafile_test
ENTRYPOINT [ "./main.llamafile"]

For some reason I cannot get entrypoint working... it's a cosmo binary so for some reason docker cannot run without throwing a exec ./main.llamafile: exec format error

This is what file command see ./main.llamafile: DOS/MBR boot sector; partition 1 : ID=0x7f, active, start-CHS (0x0,0,1), end-CHS (0x3ff,255,63), startsector 0, 4294967295 sectors Removing intermediate container 8484815c0cca so its a valid dos binary...

but anyway at least its able to run the binary inside the 'docker image build' process, so we can still validate the compile and run cycle.

[jart]

We used to have a GA runner until a few weeks ago. Try getting https://github.com/Mozilla-Ocho/llamafile/blob/0bdea60a92b18461ff71433af888fe831bc5c3ed/.github/workflows/ci.yml to work. It's not very useful to me. I have about ten computers in my home that I test llamafile on. I generally do not want to work with computers I can't SSH into. However the GA runner is important for optics and helping developers. So it would be enormously helpful if you could help me keep it maintained. My only rule with CI is it has to be rock solid stable. I believe no CI is better than flaky proprietary CI.

[mofosyne]

I see. Well then it would make sense for the CI to not be as extensive as llama.cpp . Perhaps it would be more about sanity checking that it builds and run a simple prompt like you were trying to do.

I think docker is going to be useful in the sense that developers would also be able to run it locally as a quick sanity check that it would run well online.

Plus it's less annoying than trying to maintain github proprietary CI and we could then minimise that github actions surface area.

[jart]

Why would we need to CI test Docker? We don't have any Docker related code.

[mofosyne]

The idea of using docker to contain the process is that it would allow you to run the check locally and be confident that it would run in github action successfully.

It's not easy to run github actions but turns out there is https://github.com/nektos/act so we can at least use that at least.

Decided to try and give it a shot. Especially with your github action script and hit some issues. Was it working last time you tried?

• docker approach: github: add docker based ci github actions #455
  • Works on desktop, but not on github actions
  • #12 ERROR: process "/bin/sh -c make -j $(nproc)" did not complete successfully: exit code: 2
  • ERROR: failed to solve: process "/bin/sh -c make -j $(nproc)" did not complete successfully: exit code: 2
• pure github actions approach: github: add ci #454
  • Failed on act runner and on github actions
  • Cannot open assembly '.cosmocc/3.3.8/bin/mktemper': File does not contain a valid CIL image.

[mofosyne]

Potential Solution: #454