Skip to content

Commit

Permalink
Add singularity mini lesson
Browse files Browse the repository at this point in the history
  • Loading branch information
multimeric committed Sep 18, 2024
1 parent 0c0fb63 commit 5a80513
Show file tree
Hide file tree
Showing 2 changed files with 66 additions and 17 deletions.
18 changes: 1 addition & 17 deletions episodes/docker-image-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,23 +36,7 @@ Ask @mkuzak to make a PR to add extra for <https://github.com/escience-academy/d

## Using Containers on an HPC Cluster

It is possible to run containers on shared computing systems run by a university or national
computing center. As a researcher, you can build container images and test containers on your own
computer and then run your full-scale computing work on a shared computing
system like a high performance cluster or high throughput grid.

The catch? Most university and national computing centers do not support *running*
containers with Docker commands, and instead use a similar tool called Singularity or
Shifter. However, both of these programs can be used to run containers based on Docker container images,
so often people create their container image as a Docker container image, so they can
run it using either of Docker or Singularity.

There isn't yet a working example of how to use Docker container images on a shared
computing system, partially because each system is slightly different, but the
following resources show what it can look like:

- [Introduction to Singularity](https://carpentries-incubator.github.io/singularity-introduction/): See the episode titled "Running MPI parallel jobs using Singularity containers"
- [Container Workflows at Pawsey](https://pawseysc.github.io/container-workflows/): See the episode titled "Run containers on HPC with Shifter (and Singularity)"
[In this example](../instructors/singularity.md), you will learn about using running containers on HPC clusters.

## Seeking Examples

Expand Down
65 changes: 65 additions & 0 deletions instructors/singularity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: Running Containers on HPC
teaching: 30
exercises: 0
---

::::::::::::::::::::::::::::::::::::::: objectives

- Learn how to convert Docker images to SIF
- Distinguish the `run` and `exec` subcommands

::::::::::::::::::::::::::::::::::::::::::::::::::

:::::::::::::::::::::::::::::::::::::::: questions

- How is singularity different to Docker?
- How do I use my Docker images on a shared HPC?

::::::::::::::::::::::::::::::::::::::::::::::::::

::::::::::::::::::::::::::::::::::::::::: callout

You can find more detail about using Singularity in the [singularity-introduction](https://carpentries-incubator.github.io/singularity-introduction/) Carpentries workshop.

::::::::::::::::::::::::::::::::::::::::::::::::::

Singularity is a container engine, like Docker.
However, unlike Docker, container images are stored as single files called `.sif` (Singularity Image Format).
For a number of reasons, Singularity suits shared High Performance Computing (HPC) environments much better than Docker, so is valuable to learn if you work in these environments.
A related tool called singularity is a fork of Singularity that generally has the same command line interface.

::: challenge
## Singularity Command Line Interface

Like we did with Docker, try to work out what commands Singularity has.
Which one do you think is the equivalent of `docker run`?
:::
::: solution
`singularity run` behaves similarly to `docker run`, but as we will see, the arguments are somewhat different.
:::

## Running Docker Containers

Since Singularity containers have their own file format, if we have a Docker image we want to run, it first has to be converted.
We can do this using `singularity pull`.
For example, we can pull the container we previously pushed to Docker Hub:

```bash
singularity pull docker://alice/alpine-python
```

This creates a file called `alpine_python.sif` in our working directory.
To run this container, we then use `singularity run`:

```bash
singularity run alpine_python.sif
```

## Singularity Exec

If we want to modify the command run in the container, we have to use `singularity exec`.
For example, to make Python add numbers like in our sum example, we could do:
```bash
apptainer exec alpine_python.sif python -c 'print(1 + 1)'
```

0 comments on commit 5a80513

Please sign in to comment.