Merge pull request idaholab#26395 from GiudGiud/PR_multi_csv

Add support for multiple files when reading properties from CSV

framework/doc/content/source/dirackernels/ReporterPointSource.md

@@ -6,15 +6,15 @@ A `ReporterPointSource` reads in multiple point sources from a `Reporter`.  The
 Duplicated points, i.e. points with the same xyz coordinates, are dropped by [DiracKernels](/DiracKernels/index.md) and applied as a single point.  The input parameter [!param](/DiracKernels/ReporterPointSource/combine_duplicates) combines the values and weights of duplicated points when set to `True`.  Reporters containing duplicate points will produce an error when set to `False`.  The parameter `drop_duplicate_points` used by other DiracKernels to handle duplicate points is suppressed for the `ReporterPointSource` because it is expected that every duplicate point in a `ReporterPointSource` will have different value and weight and are not just multiples of the sames value.
 
 An example of a `ReporterPointSource` using a [ConstantReporter](/ConstantReporter.md)
-and a `VectorPostprocessor` of type [CSVReader](/CSVReader.md) is given by:
+and a `VectorPostprocessor` of type [CSVReaderVectorPostprocessor](/CSVReaderVectorPostprocessor.md) is given by:
 
 !listing test/tests/dirackernels/reporter_point_source/2d_vpp.i block=reporter_point_source
 
 The ConstantReporter provides the following data:
 
 !listing test/tests/dirackernels/reporter_point_source/2d_vpp.i block=Reporters/reporterData
 
-The `CSVReader` VectorPostprocessor is given by:
+The `CSVReaderVectorPostprocessor` is given by:
 
 !listing test/tests/dirackernels/reporter_point_source/2d_vpp.i block=VectorPostprocessors
 

framework/doc/content/source/functions/PiecewiseConstantFromCSV.md

@@ -11,8 +11,9 @@ information from it.
 It can assume CSV data
 
 - is sorted by element-id, in which case, when the function is evaluated at a point, it will locate the element containing it then return the value for that element in the CSV file
+- is sorted by node-id, in which case, when the function is evaluated at a point, it will locate the node at that point then return the value for that node in the CSV file
 - is sorted by blocks, in which case, when the function is evaluated at a point, it will locate the element containing it then return the value for that element's block in the CSV file
-- defines an interpolation grid, in which case the function will locate the closest point in that interpolation grid, then return the value for that point in the CSV file
+- defines an interpolation grid, with the voronoi [!param](/Functions/PiecewiseConstantFromCSV/read_type), in which case the function will locate the closest point in that interpolation grid, then return the value for that point in the CSV file
 
 
 For the latter case, the first columns of the CSV data must define the coordinates of each point forming the interpolation grid. The number of columns used to define these coordinates
@@ -22,13 +23,17 @@ must match the dimension of the mesh.
 When use data by block or by element, if there is multiple possibilities for the element to choose from, for example at a node,
 the element with the lowest ID will be used.
 
+!alert note
+The [!param](/Functions/PiecewiseConstantFromCSV/column_number) parameter assumes 0-based indexing of the columns in the CSV file. If you want the values from the leftmost column in the file, you must use a column number of `0`.
+
 ## Example Input Syntax
 
-In this example, we display three options for using CSV data to compute a function over an unstructured mesh:
+In this example, we display four options for using CSV data to compute a function over an unstructured mesh:
 
-- the `element` function, using the `reader_element` user object, assumes the CSV file is sorted by element ID, and returns the value of the element containing each point
-- the `nearest` function, using the `reader_nearest` user object, finds the closest point defined in the CSV file, and returns the corresponding value
-- the `block` function, using the `reader_block` user object, assumes the data in the CSV file is sorted by block, and returns the value corresponding to the block containing each point
+- the `element` Function, using the `reader_element` user object, assumes the CSV file is sorted by element ID, and returns the value of the element containing each point
+- the `node` Function, using the `reader_node` user object, assumes the CSV file is sorted by node ID, and returns the corresponding value at those nodes. Outside of these nodes, the function is currently set to error.
+- the `nearest` Function, using the `reader_nearest` user object, finds the closest point defined in the CSV file, and returns the corresponding value
+- the `block` Function, using the `reader_block` user object, assumes the data in the CSV file is sorted by block, and returns the value corresponding to the block containing each point
 
 
 !listing test/tests/functions/piecewise_constant_from_csv/piecewise_constant.i block=Functions UserObjects

framework/doc/content/source/userobjects/PropertyReadFile.md

@@ -29,6 +29,14 @@ to the element centroid.
 
 An example of a MOOSE object using the `PropertyReadFile` is the [PiecewiseConstantFromCSV.md] function.
 
+If specifying multiple files to the [!param](/UserObjects/PropertyReadFile/prop_file_name)
+parameter, a new file will be read every time the object is initialized, which happens right before
+user objects are executed. The [!param](/UserObjects/PropertyReadFile/execute_on) parameter should be used to control the frequency of these executions. The user object will always read the first file at construction time, and, as an exception, will not read a new file on its first execution.
+If all data files have been read, the last file specified is used.
+
+!alert note
+When using multiple files, please note the data from the files is not concatenated. Every time a new file is read, the old data is removed from consideration.
+
 ## Example input syntax
 
 In this example input file, the `PropertyReadFile` user object is used to load data from a CSV file

framework/doc/content/source/utils/MooseUtils.md

@@ -16,7 +16,7 @@ outside of the header, is numeric and can be converted to a C++ double. Addition
 assumed that the first row or column defines the number of columns for the entire file, if the
 number of columns differs from the first row an error will be produced.
 
-Within MOOSE this utility is utilized by the [CSVReader](/CSVReader.md), which is part of
+Within MOOSE this utility is utilized by the [CSVReaderVectorPostprocessor](/CSVReaderVectorPostprocessor.md), which is part of
 the [VectorPostprocessors] system. This object will be used to explain the use of the utility.
 
 Using the DelimitedFileReader is very simple and requires three steps. First, include the
@@ -30,7 +30,7 @@ is instantiated and read method is called, as shown in the unit test snippet bel
 This class is required to include the filename upon construction. Optionally, a second argument
 providing a pointer to a [libMesh] Communicator object may be provided. This argument should be
 used when the reader is used within a MooseObject. For example, as shown in [csv_reader_ctor],
-the CSVReader object passes a Communicator object to the reader. If not provided the reader will
+the `CSVReaderVectorPostprocessor` object passes a Communicator object to the reader. If not provided the reader will
 read the data on all processors. If provided it will only read on single processor and broadcast
 the data to the others.
 
@@ -63,8 +63,8 @@ The set methods must be called prior to the read method.
   character(s) will be ignored and all characters on a line that follow the character(s) will also
   be ignored.
 
-  !listing framework/src/vectorpostprocessors/CSVReader.C
-           start=CSVReader::
+  !listing framework/src/vectorpostprocessors/CSVReaderVectorPostprocessor.C
+           start=CSVReaderVectorPostprocessor::
            end=&_communicator
            include-end=True id=csv_reader_ctor
            caption=Construction of DelimitedFileReader object within a MooseObject initialization list.

framework/doc/content/source/vectorpostprocessors/CSVReader.md

@@ -0,0 +1 @@
+CSVReaderVectorPostprocessor.md

framework/doc/content/source/vectorpostprocessors/CSVReaderVectorPostprocessor.md

@@ -0,0 +1,22 @@
+# CSVReaderVectorPostprocessor
+
+The CSVReaderVectorPostprocessor reads [CSV](https://en.wikipedia.org/wiki/Comma-separated_values) data from a file and
+converts each column into a VectorPostprocessor vector. This object uses the
+[DelimitedFileReader](MooseUtils.md#delimitedfilereader) utility to perform the reading of the file.
+
+The names of the vectors declared by the `CSVReaderVectorPostprocessor` are the names of the columns in the CSV file.
+
+## Example Input Syntax
+
+In this example, the `example.csv` file containing data for year/month/day is being read by
+the `CSVReaderVectorPostprocessor`.
+
+!listing test/tests/vectorpostprocessors/csv_reader/read.i block=VectorPostprocessors
+
+!listing test/tests/vectorpostprocessors/csv_reader/example.csv
+
+!syntax parameters /VectorPostprocessors/CSVReaderVectorPostprocessor
+
+!syntax inputs /VectorPostprocessors/CSVReaderVectorPostprocessor
+
+!syntax children /VectorPostprocessors/CSVReaderVectorPostprocessor

framework/include/userobjects/PropertyReadFile.h

@@ -32,7 +32,7 @@ class PropertyReadFile : public GeneralUserObject
   PropertyReadFile(const InputParameters & parameters);
   virtual ~PropertyReadFile() {}
 
-  virtual void initialize() {}
+  virtual void initialize();
   virtual void execute() {}
   virtual void finalize() {}
 
@@ -119,7 +119,9 @@ class PropertyReadFile : public GeneralUserObject
 
 protected:
   /// Name of file containing property values
-  const std::string _prop_file_name;
+  const std::vector<FileName> _prop_file_names;
+  /// Index of the file we last read
+  unsigned int & _current_file_index;
   /// Use DelimitedFileReader to read and store data from file
   MooseUtils::DelimitedFileReader _reader;
 
@@ -153,4 +155,7 @@ class PropertyReadFile : public GeneralUserObject
   const unsigned int _nvoronoi;
   /// Number of blocks (for reading a CSV file with properties ordered by blocks)
   const unsigned int _nblock;
+
+  /// To keep track of initialization to avoid reading the files twice
+  bool & _initialize_called_once;
 };

framework/include/utils/DelimitedFileReader.h

@@ -92,6 +92,9 @@ class DelimitedFileReader
   const std::string & getComment() const { return _row_comment; }
   ///@}
 
+  /// Set the file name, used to change the file to read from
+  void setFileName(const std::string & new_file) { _filename = new_file; }
+
   /**
    * Return the column/row names.
    */
@@ -134,7 +137,7 @@ class DelimitedFileReader
 
 protected:
   /// The supplied filename.
-  const std::string _filename;
+  std::string _filename;
 
   /// Flag indicating if the file contains a header.
   HeaderFlag _header_flag;

framework/include/vectorpostprocessors/CSVReader.h → framework/include/vectorpostprocessors/CSVReaderVectorPostprocessor.h

@@ -13,11 +13,11 @@
 #include "GeneralVectorPostprocessor.h"
 #include "DelimitedFileReader.h"
 
-class CSVReader : public GeneralVectorPostprocessor
+class CSVReaderVectorPostprocessor : public GeneralVectorPostprocessor
 {
 public:
   static InputParameters validParams();
-  CSVReader(const InputParameters & parameters);
+  CSVReaderVectorPostprocessor(const InputParameters & parameters);
   virtual void initialize() override {}
   virtual void execute() override {}
 

framework/src/userobjects/PropertyReadFile.C

@@ -24,8 +24,11 @@ PropertyReadFile::validParams()
 {
   InputParameters params = GeneralUserObject::validParams();
   params.addClassDescription("User Object to read property data from an external file and assign "
-                             "to elements.");
-  params.addRequiredParam<FileName>("prop_file_name", "Name of the property file name");
+                             "it to elements / nodes / subdomains etc.");
+  params.addRequiredParam<std::vector<FileName>>(
+      "prop_file_name",
+      "Name(s) of the property file name. If specifying multiple files, the next file will be read "
+      "at initialization right before every execution");
   params.addRequiredParam<unsigned int>("nprop", "Number of tabulated property values");
   params.addParam<unsigned int>(
       "nvoronoi", 0, "Number of voronoi tesselations/grains/nearest neighbor regions");
@@ -50,14 +53,25 @@ PropertyReadFile::validParams()
       "Periodic or non-periodic grain distribution: Default is non-periodic");
   params.addParam<bool>(
       "use_zero_based_block_indexing", true, "Are the blocks numbered starting at zero?");
+
+  // Set an execution schedule to what makes sense currently
+  // We do not allow INITIAL because we read the file at construction
+  ExecFlagEnum & exec_enum = params.set<ExecFlagEnum>("execute_on", true);
+  exec_enum.removeAvailableFlags(EXEC_INITIAL, EXEC_FINAL, EXEC_LINEAR, EXEC_NONLINEAR);
+  params.setDocString("execute_on", exec_enum.getDocString());
+  // we must read the files as early as possible
+  params.set<bool>("force_preaux") = true;
+
   return params;
 }
 
 PropertyReadFile::PropertyReadFile(const InputParameters & parameters)
   : GeneralUserObject(parameters),
-    _prop_file_name(getParam<FileName>("prop_file_name")),
-    _reader(_prop_file_name),
-
+    _prop_file_names(getParam<std::vector<FileName>>("prop_file_name")),
+    _current_file_index(declareRestartableData<unsigned int>("file_index", 0)),
+    // index of files must be capped if restarting after having read all files
+    _reader(
+        _prop_file_names[std::min(_current_file_index, (unsigned int)_prop_file_names.size() - 1)]),
     _read_type(getParam<MooseEnum>("read_type").getEnum<ReadTypeEnum>()),
     _use_random_tesselation(getParam<bool>("use_random_voronoi")),
     _rand_seed(getParam<unsigned int>("rand_seed")),
@@ -69,7 +83,8 @@ PropertyReadFile::PropertyReadFile(const InputParameters & parameters)
     _nprop(getParam<unsigned int>("nprop")),
     _nvoronoi(isParamValid("ngrain") ? getParam<unsigned int>("ngrain")
                                      : getParam<unsigned int>("nvoronoi")),
-    _nblock(getParam<unsigned int>("nblock"))
+    _nblock(getParam<unsigned int>("nblock")),
+    _initialize_called_once(declareRestartableData<bool>("initialize_called", false))
 {
   if (!_use_random_tesselation && parameters.isParamSetByUser("rand_seed"))
     paramError("rand_seed",
@@ -86,11 +101,33 @@ PropertyReadFile::PropertyReadFile(const InputParameters & parameters)
   readData();
 }
 
+void
+PropertyReadFile::initialize()
+{
+  // Since we read at construction, no need to re-read the file on first initialize
+  if (!_initialize_called_once)
+  {
+    _initialize_called_once = true;
+    return;
+  }
+
+  // Set then read new file, only if we have not reached the last file
+  if (_current_file_index < _prop_file_names.size())
+  {
+    _reader.setFileName(_prop_file_names[_current_file_index]);
+    readData();
+  }
+  else if (_current_file_index == _prop_file_names.size())
+    mooseInfo("Last file specified has been read. The file will no longer be updated.");
+}
+
 void
 PropertyReadFile::readData()
 {
   if (_read_type == ReadTypeEnum::ELEMENT && _mesh.getMesh().allow_renumbering())
     mooseWarning("CSV data is sorted by element, but mesh element renumbering is on, be careful!");
+  if (_read_type == ReadTypeEnum::NODE && _mesh.getMesh().allow_renumbering())
+    mooseWarning("CSV data is sorted by node, but mesh node renumbering is on, be careful!");
 
   _reader.setFormatFlag(MooseUtils::DelimitedFileReader::FormatFlag::ROWS);
   _reader.read();
@@ -129,20 +166,30 @@ PropertyReadFile::readData()
 
   // make sure the data from file has enough rows and columns
   if (_reader.getData().size() < nobjects)
-    mooseError(
-        "Data in ", _prop_file_name, " does not have enough rows for ", nobjects, " objects.");
+    mooseError("Data in ",
+               _prop_file_names[_current_file_index],
+               " does not have enough rows for ",
+               nobjects,
+               " objects.");
   if (_reader.getData().size() > nobjects)
     mooseWarning("Data size in ",
-                 _prop_file_name,
+                 _prop_file_names[_current_file_index],
                  " is larger than ",
                  nobjects,
                  " objects, some data will not be used.");
   for (unsigned int i = 0; i < nobjects; i++)
     if (_reader.getData(i).size() < _nprop)
-      mooseError("Row ", i, " in ", _prop_file_name, " has number of data less than ", _nprop);
+      mooseError("Row ",
+                 i,
+                 " in ",
+                 _prop_file_names[_current_file_index],
+                 " has number of data less than ",
+                 _nprop);
 
   if (_read_type == ReadTypeEnum::VORONOI || _read_type == ReadTypeEnum::GRAIN)
     initVoronoiCenterPoints();
+
+  _current_file_index++;
 }
 
 void

framework/src/vectorpostprocessors/CSVReader.C → framework/src/vectorpostprocessors/CSVReaderVectorPostprocessor.C

@@ -11,13 +11,14 @@
 #include <fstream>
 
 // MOOSE includes
-#include "CSVReader.h"
+#include "CSVReaderVectorPostprocessor.h"
 #include "MooseUtils.h"
 
-registerMooseObject("MooseApp", CSVReader);
+registerMooseObject("MooseApp", CSVReaderVectorPostprocessor);
+registerMooseObjectRenamed("MooseApp", CSVReader, "06/30/2024 24:00", CSVReaderVectorPostprocessor);
 
 InputParameters
-CSVReader::validParams()
+CSVReaderVectorPostprocessor::validParams()
 {
   InputParameters params = GeneralVectorPostprocessor::validParams();
   params.addClassDescription(
@@ -51,7 +52,8 @@ CSVReader::validParams()
   return params;
 }
 
-CSVReader::CSVReader(const InputParameters & params) : GeneralVectorPostprocessor(params)
+CSVReaderVectorPostprocessor::CSVReaderVectorPostprocessor(const InputParameters & params)
+  : GeneralVectorPostprocessor(params)
 {
   /// The MOOSE delimited file reader.
   MooseUtils::DelimitedFileReader csv_reader(getParam<FileName>("csv_file"), &_communicator);

modules/optimization/doc/content/source/functions/ParameterMeshFunction.md

@@ -22,7 +22,7 @@ First step is to define a mesh for the parameters, which is most easily done by
 
 !listing parameter_mesh/create_mesh.i
 
-Running this input will create a `create_mesh_out.e` exodus file and `create_mesh_out_param_vec_0001.csv` CSV file. The CSV file is read by a [CSVReader.md] to create a vector-postprocessor of the parameter data. A ParameterMeshFunction then reads in the exodus file and retrieves the vector for the interpolation.
+Running this input will create a `create_mesh_out.e` exodus file and `create_mesh_out_param_vec_0001.csv` CSV file. The CSV file is read by a [CSVReaderVectorPostprocessor.md] to create a vector-postprocessor of the parameter data. A ParameterMeshFunction then reads in the exodus file and retrieves the vector for the interpolation.
 
 !listing parameter_mesh/parameter_mesh.i block=VectorPostprocessors Functions
 

modules/stochastic_tools/doc/content/source/samplers/VectorPostprocessorSampler.md

@@ -7,7 +7,7 @@ The sampler generates samples from [vector-postprocessor](VectorPostprocessors/i
 
 ## Example Input Syntax
 
-In the example below, a vectorpostprocessor (`[csv]`) are formed using [CSVReader](/CSVReader.md):
+In the example below, a vectorpostprocessor (`[csv]`) are formed using [CSVReaderVectorPostprocessor](/CSVReaderVectorPostprocessor.md):
 
 !listing modules/stochastic_tools/test/tests/samplers/vectorpostprocessor/vectorpostprocessor.i block=VectorPostprocessors
 

test/tests/functions/piecewise_constant_from_csv/data_element_t2.csv

@@ -0,0 +1,16 @@
+0,0,0.2
+0,0,1.6
+0,0,0.8
+0,0,1.8
+0,0,2
+1,0,2.2
+1,0,4.2
+1,0,6.2
+1,0,34.2
+1,0,48
+1,0,20
+1,0,0.2
+1,0,22
+1,0,34
+1,0,3.6
+1,0,-2