Updated sim mode docs

src/user-guide/running-workflows/simulation-modes.rst

@@ -41,18 +41,56 @@ If :cylc:conf:`[runtime][<namespace>]execution time limit` and
 run length is computed by dividing the time limit by the speedup factor.
 
 
+Simulated Failure
+^^^^^^^^^^^^^^^^^
+
+By default all tasks succeed, sending all task outputs.
+
+But you may want to test how a workflow handles task failure.
+
+You can set some or all instances of a task to fail using
+:cylc.conf:`[runtime][<namespace>][simulation]fail cycle points`.
+`fail cycle points` takes either a list of cycle point strings or "all".
+
+Tasks set to fail will succeed on their second or following simulated
+submission. If you want all submissions to fail, set
+ :cylc.conf:`[runtime][<namespace>][simulation]fail try 1 only = False`.
+
+For example, to simulate a task you know to be flaky on the half
+hour but not on the hour:
+
+.. code-block:: flow.cylc
+
+   [[get_observations]]
+      execution retry delays = PT30S
+      [[[simulation]]]
+         fail cycle points = 2022-01-01T00:30Z,  2022-01-01T01:30Z
+
+In another case you might not expect the retry to work, and want to test
+whether your failure handling works correctly:
+
+.. code-block:: flow.cylc
+
+   [[get_data]]
+       execution retry delays = PT30S
+       [[[outputs]]]
+          server_broken = "data server unreachable"
+       [[[simulation]]]
+          fail try 1 only = false
+          fail cycle points = 2022-01-01T03:00Z
+
 Limitations
 ^^^^^^^^^^^
 
 Dummy tasks run locally, so dummy mode does not test communication with remote
 job platforms. However, it is easy to write a live-mode test workflow with
-simple ``sleep 10`` tasks that submit to a remote platform. 
+simple ``sleep 10`` tasks that submit to a remote platform.
 
 Alternate path branching is difficult to simulate effectively. You can
 configure certain tasks to fail via
 :cylc:conf:`[runtime][<namespace>][simulation]`, but all branches based
 on mutually exclusive custom outputs will run because all custom outputs get
-artificially completed in dummy mode and in simulation mode. 
+artificially completed in dummy mode and in simulation mode.
 
 .. note::