Set of experiments

bin/benchpark

@@ -125,17 +125,19 @@ def benchpark_check_system(arg_str):
 
 def benchpark_setup(subparsers, actions_dict):
     create_parser = subparsers.add_parser(
-        "setup", help="Create a benchmark and prepare it to build/run"
+        "setup", help="Set up an experiment and prepare it to build/run"
     )
 
     create_parser.add_argument(
-        "benchmark", type=str, help="The benchmark/experiment to run"
+        "benchmark", type=str, help="The experiment (benchmark/ProgrammingModel) to run"
+    )
+    create_parser.add_argument(
+        "system", type=str, help="The system on which to run the experiment"
     )
-    create_parser.add_argument("system", type=str, help="The system we are on")
     create_parser.add_argument(
         "workspace_root",
         type=str,
-        help="Where to install packages, store results, etc. (should be empty/nonexistent)",
+        help="Where to install packages, store results, etc. (could be empty/nonexistent)",
     )
 
     actions_dict["setup"] = benchpark_setup_handler
@@ -174,22 +176,22 @@ def symlink_tree(src, dst):
 def benchpark_setup_handler(args):
     """
     workspace_root/
+        spack/
+        ramble/
         <benchmark>/
             <system>/
                 workspace/
                     configs/
                         (everything from source/configs/<system>)
                         (everything from source/experiments/<benchmark>)
-        spack/
-        ramble/
     """
 
     benchmark = args.benchmark
     system = args.system
     workspace_root = pathlib.Path(os.path.abspath(args.workspace_root))
     source_dir = source_location()
     debug_print(f"source_dir = {source_dir}")
-    debug_print(f"specified benchmark = {benchmark}")
+    debug_print(f"specified benchmark/ProgrammingModel = {benchmark}")
     valid_benchmark = benchpark_check_benchmark(benchmark)
     debug_print(f"specified system = {system}")
     valid_system = benchpark_check_system(system)
@@ -261,8 +263,6 @@ def benchpark_setup_handler(args):
     instructions = f"""\
 To complete the benchpark setup, do the following:
 
-    cd {workspace_dir}/workspace
-
     . {spack_location}/share/spack/setup-env.sh
     . {ramble_location}/share/ramble/setup-env.sh
 

docs/4-benchpark-setup.rst

@@ -9,30 +9,28 @@ Also choose the workspace for your experiment::
 
 where:
 
-- ``benchmark/ProgrammingModel``: amg2023/openmp | amg2023/cuda | saxpy/openmp (available choices in benchpark/experiments)
-- ``system``: ats2 | ats4 | cts1 (available choices in benchpark/configs)
+- ``benchmark/ProgrammingModel``: amg2023/openmp | amg2023/cuda | saxpy/openmp (available choices in ``benchpark/experiments``)
+- ``system``: ats2 | ats4 | cts1 (available choices in ``benchpark/configs``)
 
 This command will assemble a Ramble workspace 
 with a configuration for the specified benchmark and system 
 with the following directory structure::
 
     workspace_root/
+        ramble/
+        spack/
         <benchmark>/
             <ProgrammingModel>/
                 <system>/
-                    ramble/
-                    spack/
                     workspace/
                         configs/
                             (everything from source/configs/<system>)
                             (everything from source/experiments/<benchmark>)
 
 ``benchpark setup`` will output instructions to follow::
 
-   cd <workspace_root>/<benchmark/ProgrammingModel>/<system>/workspace
-
-   . <workspace_root>/<benchmark/ProgrammingModel>/<system>/spack/share/spack/setup-env.sh
-   . <workspace_root>/<benchmark/ProgrammingModel>/<system>/ramble/share/ramble/setup-env.sh
+   . <workspace_root>/spack/share/spack/setup-env.sh
+   . <workspace_root>/ramble/share/ramble/setup-env.sh
 
    export SPACK_DISABLE_LOCAL_CONFIG=1
 

docs/5-build-experiment.rst

@@ -12,11 +12,11 @@ The next step is setting up the Ramble workspace and building the code::
 Ramble will build the source code and set up the following workspace directory structure::
 
     workspace_root/
+        ramble/
+        spack/
         <benchmark>/
             <ProgrammingModel>/
                 <system>/
-                    ramble/
-                    spack/
                     workspace/
                         configs/
                             (everything from source/configs/<system>)

docs/index.rst

@@ -21,3 +21,9 @@
    add-a-benchmark
    add-an-experiment
    add-a-system-config
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Benchmark Suites
+
+   set-of-experiments

docs/set-of-experiments.rst

@@ -0,0 +1,36 @@
+==============
+Working with a set of experiments
+==============
+
+To keep the source code consistent while working with 
+a set of ``experiments`` (several ``benchmark/ProgrammingModels`` x ``systems``),
+one can consider working with a single ``Benchpark workspace``, which will ensure the same
+clone of Spack, and the same clone of Ramble, will be used for all of the ``experiments`` in 
+this set.  To work with a set of ``experiments`` based on a single clone of Spack and 
+a single clone of Ramble, set up your experiments in the same /output/path/to/workspace_root::
+
+    benchpark setup benchmark1/ProgrammingModel1 system1 /output/path/to/workspace_root
+    benchpark setup benchmark1/ProgrammingModel2 system2 /output/path/to/workspace_root
+    benchpark setup benchmark2/ProgrammingModel2 system1 /output/path/to/workspace_root
+
+This will result in the following directory structure::
+
+    workspace_root/
+        ramble/
+        spack/
+        benchmark1/
+            ProgrammingModel1/
+                system1/
+                    workspace/
+            ProgrammingModel2/
+                system2/
+                    workspace/
+        benchmark2/
+            ProgrammingModel2/
+                system1/
+                    workspace/
+
+Note that there is a single clone of Ramble, and a single clone of Spack, 
+which all of the ``experiments`` use.
+Each ``experiment`` (``benchmark/ProgrammingModel`` x ``system`` combination)
+has its own ``Ramble workspace``, where the ``experiment`` will be compiled and run.