LLNL · pearce8 · Nov 3, 2023 · Nov 1, 2023 · Nov 1, 2023 · Nov 1, 2023
diff --git a/README.rst b/README.rst
@@ -1,6 +1,8 @@
-=========
-benchpark
-=========
+==================================================
+Benchpark 
+==================================================
+
+You can find detailed documentation in `ReadTheDocs <https://software.llnl.gov/benchpark>`_
 
 Benchpark is an open collaborative repository for reproducible specifications of HPC benchmarks.
 Benchpark enables cross-site collaboration on benchmarking by providing a mechanism for sharing
@@ -26,25 +28,13 @@ Dependencies
 ------------
 Benchpark uses the following open source projects for specifying configurations:
 
-* `Ramble <https://github.com/GoogleCloudPlatform/ramble>`_ ro specify run configurations
+* `Ramble <https://github.com/GoogleCloudPlatform/ramble>`_ to specify run configurations
 * `Spack <https://github.com/spack/spack>`_ to specify build configurations
 
-Documentation
--------------
-1. :doc:`1-getting-started`
-2. :doc:`2-benchpark-list`
-3. (optional) :doc:`3-opt-edit-experiment`
-4. :doc:`4-benchpark-setup`
-5. :doc:`5-build-experiment`
-6. :doc:`6-run-experiment`
-7. :doc:`add-a-benchmark`
-8. :doc:`add-a-system-config`
-9. :doc:`add-an-experiment`
-
 Community
 ---------
 Benchpark is an open source project.  Questions, discussion, and contributions 
-of new benchmarks and system specifications are welcome.
+of new benchmarks, experiments, and system specifications are welcome.
 We use `github discussions <https://github.com/llnl/benchpark/discussions>`_ for Q&A and discussion.
 
 Contributing

diff --git a/bin/benchpark b/bin/benchpark
@@ -251,7 +251,7 @@ def benchpark_setup_handler(args):
     run_command(f"{ramble_exe} repo add --scope=site {source_dir}/repo")
 
     instructions = f"""\
-To build these benchmarks, do the following:
+To complete the benchpark setup, do the following:
 
     cd {workspace_dir}/workspace
 
@@ -260,9 +260,7 @@ To build these benchmarks, do the following:
 
     export SPACK_DISABLE_LOCAL_CONFIG=1
 
-    ramble -P -D . workspace setup
-
-Further steps are needed to run the experiments (ramble -P -D . on)
+Further steps are needed to build the experiments (ramble -P -D . workspace setup) and run them (ramble -P -D . on)
 """
     print(instructions)
 

diff --git a/docs/3-opt-edit-experiment.rst b/docs/3-opt-edit-experiment.rst
@@ -1,6 +1,6 @@
-==============
+==========================
 (optional) Edit experiment
-==============
+==========================
 
 Benchpark configuration files are organized as follows:: 
 
@@ -26,26 +26,26 @@ Benchpark configuration files are organized as follows::
      │  └── package.py 
      └── repo.yaml 
 
-You can edit any of them to change the behaviour of your experiments.
+You can edit any of them to change the behavior of your experiments.
 
 System specification
--------------------
+--------------------
 Files under ``benchpark/configs/${SYSTEM}`` provide the specification
 of the software stack on your system
-(see `Add a system config <add-a-system.rst>`_ for details).
+(see :doc:`add-a-system-config` for details).
 
 Experiment specification
--------------------
+------------------------
 Files under ``benchpark/experiments/${BENCHMARK}/${ProgrammingModel}``
 provide the specifications for the experiments.
 If you would like to make changes to your experiments,  such as enabling 
 specific tools to measure the performance of your experiments, 
 you can manually edit the specifications in ``ramble.yaml``
-(see `Add an experiment <add-an-experiment.rst>`_ for details).
+(see :doc:`add-an-experiment` for details).
 
 Benchmark specification
--------------------
+-----------------------
 If you would like to modify a specification of your benchmark,
 you can do so by upstreaming changes to Spack and/or Ramble,
 or working on your benchmark specification in ``benchpark/repo/${BENCHMARK}`` 
-(see `Add a benchmark <add-a-benchmark.rst>`_ for details).
+(see :doc:`add-a-benchmark` for details).
diff --git a/docs/4-benchpark-setup.rst b/docs/4-benchpark-setup.rst
@@ -18,7 +18,7 @@ with the following directory structure::
 
     workspace_root/
         <benchmark>/
-            <ProgrammingMode>/
+            <ProgrammingModel>/
                 <system>/
                     ramble/
                     spack/
@@ -27,4 +27,13 @@ with the following directory structure::
                             (everything from source/configs/<system>)
                             (everything from source/experiments/<benchmark>)
 
-``benchpark setup`` will output :doc:`5-build-experiment` instructions.
+``benchpark setup`` will output instructions to follow::
+
+   cd <workspace_root>/<benchmark/ProgrammingModel>/<system>/workspace
+
+   . <workspace_root>/<benchmark/ProgrammingModel>/<system>/spack/share/spack/setup-env.sh
+   . <workspace_root>/<benchmark/ProgrammingModel>/<system>/ramble/share/ramble/setup-env.sh
+
+   export SPACK_DISABLE_LOCAL_CONFIG=1
+
+Now you are ready to build your experiments as described in :doc:`5-build-experiment`.
diff --git a/docs/5-build-experiment.rst b/docs/5-build-experiment.rst
@@ -2,19 +2,14 @@
 Build experiment
 ================
 
-``benchpark setup`` will output instructions to follow::
+``benchpark setup`` has set up the directory structure for your experiment.  
+The next step is setting up the Ramble workspace and building the code::
 
    cd <workspace_root>/<benchmark/ProgrammingModel>/<system>/workspace
-
-   . <workspace_root>/<benchmark/ProgrammingModel>/<system>/spack/share/spack/setup-env.sh
-   . <workspace_root>/<benchmark/ProgrammingModel>/<system>/ramble/share/ramble/setup-env.sh
-
-   export SPACK_DISABLE_LOCAL_CONFIG=1
-
    ramble -P -D . workspace setup  
 
 
-which will build the source code and set up the following workspace directory structure::
+Ramble will build the source code and set up the following workspace directory structure::
 
     workspace_root/
         <benchmark>/
@@ -31,3 +26,7 @@ which will build the source code and set up the following workspace directory st
                                 <problem>/
                                     <benchmark>_<ProgrammingModel>_<problem>
                                         execute_experiment
+
+
+If you edit any of the files, see :doc:`FAQ-what-to-rerun` to determine
+whether you need to re-do any of the previous steps.
diff --git a/docs/6-run-experiment.rst b/docs/6-run-experiment.rst
@@ -31,15 +31,9 @@ An output file is generated for each experiment in its unique directory::
 
 An experiment instance can also be executed individually by directly invoking its ``execute_experiment`` script 
 (e.g., ``$workspace/experiments/amg2023/problem1/amg2023_cuda11.8.0_problem1_1_8_2_2_2_10_10_10/execute_experiment``).
-Note that re-running the experiment may overwrite any existing output files in the directory.
-
-Analyze the experiment results 
-------------------------------
-Once the experiments have been run, the command::
-
-  ramble -P -D . workspace analyze 
-
-can be used to analyze figures of merit and evaluate 
-`success/failure <https://googlecloudplatform.github.io/ramble/success_criteria.html>`_ 
-of the experiments. Ramble generates a summary results file in ``$workspace``.
 
+Note that re-running the experiment may overwrite any existing output files in the directory.
+Further, if the benchmark has restart capability, existing output may alter the experiments
+benchpark would run in the second run.  Generally, we would advise the user to remove the
+``$workspace/experiments`` directory before re-running the experiments using
+``ramble -P -D . on``.
diff --git a/docs/7-analyze-experiment.rst b/docs/7-analyze-experiment.rst
@@ -0,0 +1,11 @@
+==================================
+Analyzing Experiments in Benchpark
+==================================
+
+Once the experiments completed running, the command::
+
+  ramble -P -D . workspace analyze 
+
+can be used to analyze figures of merit and evaluate 
+`success/failure <https://googlecloudplatform.github.io/ramble/success_criteria.html>`_ 
+of the experiments. Ramble generates a file with summary of the results in ``$workspace``.
diff --git a/docs/FAQ-what-to-rerun.rst b/docs/FAQ-what-to-rerun.rst
@@ -0,0 +1,20 @@
+==========================================
+FAQ: I made changes.  What should I rerun?
+==========================================
+
+.. list-table:: I made changes.  What should I rerun?
+   :widths: 35 65
+   :header-rows: 1
+
+   * - What I changed
+     - Commands to rerun
+   * - configs
+     - ``ramble -P -D . workspace setup``
+   * - benchmark's package.py 
+     - ``ramble -P -D . workspace setup``
+   * - dependency of package.py
+     - ``ramble -P -D . workspace setup``
+   * - experiment parameters
+     - delete ``workspace/experiments``
+   * - wish to rerun experiments
+     - delete ``workspace/experiments``
diff --git a/docs/_static/.gitignore b/docs/_static/.gitignore
@@ -0,0 +1,4 @@
+# Ignore everything in this directory
+*
+# Except this file
+!.gitignore
diff --git a/docs/index.rst b/docs/index.rst
@@ -7,14 +7,17 @@
 
    1-getting-started
    2-benchpark-list
+   3-opt-edit-experiment
    4-benchpark-setup
    5-build-experiment
    6-run-experiment
+   7-analyze-experiment
+   FAQ-what-to-rerun
 
 .. toctree::
    :maxdepth: 1
    :caption: Contributing
 
-   add-a-system-config
-   add-an-experiment
    add-a-benchmark
+   add-an-experiment
+   add-a-system-config