Skip to content

Commit

Permalink
fix typos and more cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
ryanday36 committed Mar 29, 2023
1 parent 8d7e122 commit 58a4093
Show file tree
Hide file tree
Showing 10 changed files with 16 additions and 16 deletions.
2 changes: 1 addition & 1 deletion flux/exercises/exercise1.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
layout: tutorial_page
title: "Exercise 1: Starting Flux and getting an allocation"
title: "Exercise 1: Starting Flux"
release_number: LLNL-WEB-822959
author: Ryan Day, Lawrence Livermore National Laboratory
---
Expand Down
6 changes: 3 additions & 3 deletions flux/exercises/exercise2.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ release_number: LLNL-WEB-822959
author: Ryan Day, Lawrence Livermore National Laboratory
---

1. Ensure that you have a flux instance running in an allocation and determine the number of cores available to you in that allocation.
1. Ensure that you have a Flux instance running in an allocation and determine the number of cores available to you in that allocation.
2. Download the [mpi_hellosleep.c](/flux/example_code/mpi_hellosleep.c) file and build the application with `mpicc -o mpi_hellosleep mpi_hellosleep.c`.
3. Run `mpi_hellosleep` as a blocking job step with enough mpi tasks to fill all of the cores available in your flux instance.
4. Submit two non-blocking job steps that will run `mpi_hellosleep` with enough mpi tasks to fill all of the cores available in your flux instance. Verify that only one of the job steps is running.
3. Run `mpi_hellosleep` as a blocking job step with enough mpi tasks to fill all of the cores available in your Flux instance.
4. Submit two non-blocking job steps that will run `mpi_hellosleep` with enough mpi tasks to fill all of the cores available in your Flux instance. Verify that only one of the job steps is running.
5. Cancel the running job step. Confirm that the other job step is running and attach to it to monitor its progress.

### Notes / Solutions
Expand Down
2 changes: 1 addition & 1 deletion flux/exercises/exercise3.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ release_number: LLNL-WEB-822959
author: Ryan Day, Lawrence Livermore National Laboratory
---

1. Ensure that you have a flux instance running in an allocation and determine the number of cores available to you in that allocation.
1. Ensure that you have a Flux instance running in an allocation and determine the number of cores available to you in that allocation.
2. Write a batch script that will run the `mpi_hellosleep` application from [Exercise 2.2](/flux/exercises/exercise2) with four tasks. Ensure that the script logs when it starts and ends and that it is self documenting.

### Notes / Solutions
Expand Down
2 changes: 1 addition & 1 deletion flux/exercises/exercise5.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ $ cd flux-workflow-examples/hierarchical-launching
2. Run the [hierarchical launching](https://flux-framework.readthedocs.io/projects/flux-workflow-examples/en/latest/hierarchical-launching/README.html) example. Review the launcher scripts to understand which `flux` commands are launching Flux instances and which are not.

### Notes / Solutions
2. This workflow example explicitly includes instructions for getting a Slurm allocation and starting flux. See [Section 1](/flux/section1) for general instructions on getting an allocation in flux or starting flux under Slurm.
2. This workflow example explicitly includes instructions for getting a Slurm allocation and starting Flux. See [Section 1](/flux/section1) for general instructions on getting an allocation in Flux or starting Flux under Slurm.

---
[Section 4](/flux/section4) | [Section 5](/flux/section5) | Exercise 5 | [Section 6](/flux/section6)
Expand Down
4 changes: 2 additions & 2 deletions flux/exercises/exercise6.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ release_number: LLNL-WEB-822959
author: Ryan Day, Lawrence Livermore National Laboratory
---

1. Clone the workflow examples from the [flux github](https://github.com/flux-framework/flux-workflow-examples.git):
1. Clone the workflow examples from the [Flux github](https://github.com/flux-framework/flux-workflow-examples.git):
```
$ git clone https://github.com/flux-framework/flux-workflow-examples.git
$ cd flux-workflow-examples/job-submit-api
Expand All @@ -14,7 +14,7 @@ $ cd flux-workflow-examples/job-submit-api
3. Run the [Python Job Submit/Wait](https://flux-framework.readthedocs.io/projects/flux-workflow-examples/en/latest/job-submit-wait/README.html), [Python Asynchronous Bulk Job Submission](https://flux-framework.readthedocs.io/projects/flux-workflow-examples/en/latest/async-bulk-job-submit/README.html), and [Using Flux job status and control API](https://flux-framework.readthedocs.io/projects/flux-workflow-examples/en/latest/job-status-control/README.html) work examples. Review the scripts included in those examples to see how the `FluxExecutor` and `concurrent.futures` are being used.

### Notes / Solutions
2. Most of the examples explicitly include instructions for getting a Slurm allocation and starting flux. See [Section 1](/flux/section1) for general instructions on getting an allocation in flux or starting flux under Slurm.
2. Most of the examples explicitly include instructions for getting a Slurm allocation and starting Flux. See [Section 1](/flux/section1) for general instructions on getting an allocation in Flux or starting Flux under Slurm.
3. See note 2.

---
Expand Down
2 changes: 1 addition & 1 deletion flux/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ author: Ryan Day, Lawrence Livermore National Laboratory

[Introduction](intro)
#### Part I: Simple job submission and interacting with Flux
- [Section 1: Starting Flux and getting an allocation](section1)
- [Section 1: Starting Flux](section1)
-- [Exercise 1](exercises/exercise1) --
- [Section 2: Launching and tracking Flux jobs](section2)
-- [Exercise 2](exercises/exercise2) --
Expand Down
4 changes: 2 additions & 2 deletions flux/section1.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
layout: tutorial_page
title: "Starting Flux and getting an allocation"
title: "Starting Flux"
release_number: LLNL-WEB-822959
author: Ryan Day, Lawrence Livermore National Laboratory
---

Regardless of what resource management software a cluster is running, the first step in running in a multi-user environment is to get an allocation of hardware resources. In Flux, your allocation will take the form of a Flux instance where you can use Flux commands to manage your workload on those resources. This section will tell you where to find Flux and how to start it in an allocation even if it is not the main resource manager on the cluster that you are running on.
Regardless of what resource management software a cluster is running, the first step in running in a multi-user environment is to get an allocation of hardware resources. In Flux, your allocation will take the form of a Flux instance where you can use Flux commands to manage your workload on those resources. This section will tell you where to find Flux and how to start an instance of it in an allocation even if it is not the main resource manager on the cluster that you are running on.
### Finding Flux
Flux is included in the TOSS operating system on LC systems, so should be available in your standard `PATH`. You can check on this with:
```console
Expand Down
4 changes: 2 additions & 2 deletions flux/section2.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ release_number: LLNL-WEB-822959
author: Ryan Day, Lawrence Livermore National Laboratory
---

In the previous section, we learned how to find flux, get an allocation, and query the compute resources in that allocation. Now, we are ready to launch work on those compute resources and get some work done. When you launch work in Flux, that work takes the form of jobs that can be either blocking or non-blocking. Blocking jobs will run to completion before more work can be submitted, whereas non-blocking jobs are enqueued, allowing you to immediately submit more work in the allocation.
In the previous section, we learned how to find Flux, get an allocation, and query the compute resources in that allocation. Now, we are ready to launch work on those compute resources and get some work done. When you launch work in Flux, that work takes the form of jobs that can be either blocking or non-blocking. Blocking jobs will run to completion before control is returned to the terminal, whereas non-blocking jobs are enqueued, allowing you to immediately submit more work in the allocation.

Before we get into submitting and managing Flux jobs, we should also discuss Flux's jobids as they're a bit different than what you'll find in other resource management software. Remember that an allocation in Flux is a fully featured Flux instance. Rather than create sequential numeric ids within each instance, Flux combines the submit time, an id, and sequence number to create identifiers that are unique for each job in a Flux instance. This avoids requiring a central allocator for jobids which improves the scalability of job submission within instances. There are options to display these identifiers in a number of ways, but the default is an 8 character string prepended by an `f`, e.g. `fBsFXaow5` for the job submitted in the example below. For more details on Flux's identifiers, see the [FLUID documentation](https://flux-framework.readthedocs.io/projects/flux-rfc/en/latest/spec_19.html).
Before we get into submitting and managing Flux jobs, we should also discuss Flux's jobids as they're a bit different than what you'll find in other resource management software. Remember that an allocation in Flux is a fully featured Flux instance. Rather than create sequential numeric ids within each instance, Flux combines the submit time, an instance id, and sequence number to create identifiers that are unique for each job in a Flux instance. This avoids requiring a central allocator for jobids which improves the scalability of job submission within instances. There are options to display these identifiers in a number of ways, but the default is an 8 character string prepended by an `f`, e.g. `fBsFXaow5` for the job submitted in the example below. For more details on Flux's identifiers, see the [FLUID documentation](https://flux-framework.readthedocs.io/projects/flux-rfc/en/latest/spec_19.html).
### Submit blocking Flux jobs with `flux run`
If you want your work to block until it completes, the `flux run` command will submit a job and then wait until it is complete before returning. For example, in a two node allocation, we can launch an mpi program with 4 tasks:
```
Expand Down
4 changes: 2 additions & 2 deletions flux/section3.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ sh-4.2$ cat simplescript.sh
#flux: -n 4
date
flux mini run -n4 ./mpi_hellosleep
flux run -n4 ./mpi_hellosleep
echo 'job complete'
sh-4.2$ flux batch ./simplescript.sh
Expand Down Expand Up @@ -115,7 +115,7 @@ corona178
[day36@corona212:~]$
```
### More user facing batch options
The Flux job submission commands have many more options for doing things like running on specific nodes or queues, modifying your job environment, specifying task mappings, and more. See, for example, `man flux-run` for details on all of the options available.
The Flux job submission commands have many more options for doing things like running on specific nodes or queues, modifying your job environment, specifying task mappings, and more. See, for example, `man flux-run` for details on all of the options available. We have also put together a [table of equivalent resource manager flags](https://hpc.llnl.gov/banks-jobs/running-jobs/batch-system-cross-reference-guides).

---
[Section 2](/flux/section2) | Section 3 | [Exercise 3](/flux/exercises/exercise3) | [Section 4](/flux/section4)
Expand Down
2 changes: 1 addition & 1 deletion flux/section5.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ release_number: LLNL-WEB-822959
author: Ryan Day, Lawrence Livermore National Laboratory
---

One of the key innovations of Flux is the ability to easily start flux instances within a parent Flux instances. This allows users to create separate allocations on different subsets of their allocated resources and assign different portions of their workflow to those resources. The basic command line interface for Flux has two commands that create new Flux instances, and you've already been using one of them. The `flux batch` command described in [section 3](/flux/section3) is actually creating a flux instance that the `flux run` commands are running in. Similarly, `flux alloc` can be used to create a new instance, but blocks until its work is complete.
One of the key innovations of Flux is the ability to easily start Flux instances within a parent Flux instances. This allows users to create separate allocations on different subsets of their allocated resources and assign different portions of their workflow to those resources. The basic command line interface for Flux has two commands that create new Flux instances, and you've already been using one of them. The `flux batch` command described in [section 3](/flux/section3) is actually creating a flux instance that the `flux run` commands are running in. Similarly, `flux alloc` can be used to create a new instance, but blocks until its work is complete.
### Creating allocations inside of an allocation
We can use the `flux resource list` and `flux jobs` commands discussed in [section 1](/flux/section1) and [section 2](/flux/section2) to demonstrate the differences between running in an allocation (`flux run` or `flux submit`) and creating a new allocation (`flux batch` or `flux alloc`). We will start with a two node allocation:
```
Expand Down

0 comments on commit 58a4093

Please sign in to comment.