Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tutorials: add flux mini run tutorial/example #202

Merged
merged 7 commits into from
Feb 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 39 additions & 9 deletions tutorials/commands/flux-mini-submit.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
.. _flux-mini-submit:
.. _flux-mini-run:

==========================
How to Submit Jobs in Flux
Expand All @@ -14,7 +15,7 @@ From within a Flux instance, you can submit your job on the command line with
optional arguments, resource options, per task options, and per resource
options:

.. code-block:: sh
.. code-block:: console

$ flux mini submit --nodes=2 --ntasks=4 --cores-per-task=2 ./my_compute_script.py 120
ƒM5k8m7m
Expand All @@ -28,41 +29,70 @@ for each task. In the second submission, we are asking for our job to start 1
task on just one node with 2 cores allocated for the task.

There are many different options to customize your job submission. For further
details, please see :core:man1:`flux-mini`.
details, please see :core:man1:`flux-mini`.

A :ref:`jobID<fluid>` (e.g., ``ƒSUEFPDH``) is returned for every job submitted. You can view
A :ref:`jobid<fluid>` (e.g., ``ƒSUEFPDH``) is returned for every job submitted. You can view
the status of your running jobs with ``flux jobs``:

.. code-block:: sh
.. code-block:: console

$ flux jobs
JOBID USER NAME ST NTASKS NNODES TIME INFO
ƒSUEFPDH fluxuser my_other_s R 1 1 1.842s
ƒSUEFPDH fluxuser my_other_s R 1 1 1.842s
ƒM5k8m7m fluxuser my_compute R 4 2 3.255s

And that's it! If you have any questions, please
-----------------------
Interactively Run a Job
-----------------------

If you wish to run a job interactively, e.g. see standard output as it runs, you can
use the ``flux mini run`` command. It is identical to ``flux mini submit`` except it
will handle stdio and it will block until the job has finished. For example:

.. code-block:: console

$ flux mini run bash -c "echo start; sleep 5; echo done"
start
done

In the above example, we run a small bash script that will output "start", sleep for 5 seconds,
and then echo "done". Unlike ``flux mini submit``, you'll notice it does not output a jobid.
If we check for the status of this job with ``flux jobs`` after it has run, you will not find the
job listed because it is no longer running. Instead run ``flux jobs -a`` which will list all jobs,
including completed jobs.

.. code-block:: console

$ flux jobs -a
JOBID USER NAME ST NTASKS NNODES TIME INFO
f2HnvmZy achu bash CD 1 1 5.119s catalyst159


You will see that job is in the "CD" state or "Completed" state.

And that's it! If you have any questions, please
`let us know <https://github.com/flux-framework/flux-docs/issues>`_.

-------------------------------------
More Examples of Submitting Flux Jobs
-------------------------------------

.. code-block:: sh
.. code-block:: console

$ flux mini submit --nodes=2 --queue=foo --name=my_special_job ./my_job.py

This submits a job to the `foo` queue across two nodes, and sets a custom name
to the job.

.. code-block:: sh
.. code-block:: console

$ flux mini submit --dry-run ./my_cool_job.py

If you don't want your job to actually run, but you are interested in looking
at the :ref:`jobspec<jobspec>` for your job, include the ``--dry-run`` option
when you submit your job.

.. code-block:: sh
.. code-block:: console

$ flux mini submit --output=job-{{id}}.out ./my_super_cool_job.py
ƒ3D78hc3q
Expand Down
6 changes: 3 additions & 3 deletions tutorials/commands/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ Command Tutorials
Welcome to the Command Tutorials! These tutorials should help you to map specific Flux commands
with your use case, and then see detailed usage.

- ``flux proxy`` (:ref:`ssh-across-clusters`): "Send commands to a flux instance across clusters using ssh"
- ``flux-mini-submit`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance"
- ``flux mini submit/flux mini run`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance"
- ``flux proxy`` (:ref:`ssh-across-clusters`): "Send commands to a Flux instance across clusters using ssh"

This section is currently 🚧️ under construction 🚧️, so please come back later to see more command tutorials!

Expand All @@ -16,5 +16,5 @@ This section is currently 🚧️ under construction 🚧️, so please come bac
:maxdepth: 2
:caption: Command Tutorials

ssh-across-clusters
flux-mini-submit
ssh-across-clusters
4 changes: 2 additions & 2 deletions tutorials/commands/ssh-across-clusters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Create a Flux Instance
----------------------

First, let's create the allocation on the first cluster. We typically want to ask for an allocation,
and run flux start via our job manager. Here we might be on a login node:
and run ``flux start`` via our job manager. Here we might be on a login node:

.. code-block:: console

Expand All @@ -22,7 +22,7 @@ and run flux start via our job manager. Here we might be on a login node:
$ srun -N4 -n4 --pty --mpibind=off flux start

And then we get our allocation! You might adapt this command to be more specific to your resource manager. E.g., slurm uses srun.
After you run flux start, you are inside of a Flux instance on your allocation!
After you run ``flux start``, you are inside of a Flux instance on your allocation!
Let's run a simple job on our allocation. This first example will ask to see the hostnames of your nodes:

.. code-block:: console
Expand Down