Skip to content

Commit

Permalink
Merge pull request #17 from ComputationalRadiationPhysics/doc-readmeU…
Browse files Browse the repository at this point in the history
…pdate

README: Update Motivation
  • Loading branch information
michaelsippel authored Dec 15, 2019
2 parents 0a43772 + e5acca1 commit 23d5d84
Showing 1 changed file with 42 additions and 22 deletions.
64 changes: 42 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,46 @@
[![License](https://img.shields.io/badge/license-MPL--2.0-blue.svg)](https://www.mozilla.org/en-US/MPL/2.0/)
<hr>

RedGrapes is a C++14 framework for declaratively creating and scheduling task-graphs, based on high-level resource descriptions.
RedGrapes is a C++14 framework for declaratively creating and scheduling task-graphs, based on a high-level resource description.

### Motivation
Writing scalable software using bare threads is hard and error-prone, especially if the workload depends on input parameters and asynchronous operations further complicating the program flow.
For this reason the decoupling of processing stages from their execution is useful because it allows to dynamically schedule them. This is typically done with task-graphs, which are directed acyclic graphs (DAGs), whose vertices are some sort of computation and the edges denote the execution precedence order.
This execution precedence results from the dataflow between the tasks, which gets
complex pretty fast and may also be dynamic which makes it nearly impossible to
manually write explicit task dependencies. So ideally these would be derived
from some sort of high-level description of the dataflow. The goal of this
project is to provide a task-based programming framework, where the task-graph
gets created declaratively.

Modern compute nodes concurrently perform computational tasks over various memory resource pools, cores, and accelerator devices.
In order to achieve high scalability in such a compute node, communication and computation tasks need to be overlapped extensively.

Up until now, software developers that took up to this challenge had to juggle data and in-node execution dependencies manually, which is a tedious and error-prone process.
Real-world workloads that depend on states at runtime and asynchronous communication models complicate the program flow even further.

For this reason, one should decouple aforementioned computational tasks from their execution model altogether.
A typical approach involves task-graphs, which are directed acyclic graphs (DAGs), whose vertices are some sort of computation (or communication) and the edges denote the execution precedence order.
The execution precedence arises from the order in which those tasks were declared by the programmer but also have to take into account the data dependencies between the tasks.

Consequently, **RedGrapes** provides you with a light-weight, application-level, task-based C++ programming framework.
Herein, a task-graph is generated declaratively from access to resources and order of your code, just as in serial programming.

### Concept
The Program is divided into *tasks*. Those are the smallest unit the scheduler operates with.
Because C++ is mainly a sequential language, the dataflow description is based on *resources* which represent some state shared between tasks. Each task has an annotation how the resources are accessed. Which *access modes* are possible is dependent on the type of the resource. A simple example would be read/write, but also array accesses or composite types could be used. A resource can be associated with a specific *acces mode* forming a *resource access*, of which two can then be used to check if they are conflicting. So each task carries in its *task-properties* a list of
resource-accesses.
If two tasks have conflicting resource-accesses, the first created task is executed
first. So the order of the tasks remains virtually.
When tasks are created, their resource-access-list is compared against the previous
enqueued tasks and corresponding dependencies are created in the task-graph.
The resulting task-graph can then be easily scheduled across parallel threads.

The program shall be divided into **tasks**.
A task is can be a sub-step in a computational solver, the exchange of data between two memory resource pools, or anything else.
Tasks are the smallest unit the RedGrapes scheduler operates with.
Data dependencies are described via **resources**, which are accessed and potentially manipulated by tasks.

Each task has an annotation how the resources are accessed.
Therein allowed **access modes** depend on the type of the resource.
A simple example would be read/write, but also more complex operations are possible, e.g. on arrays or just a sub view.
A *resource* can be associated with a specific *access mode* forming a **resource access**. These instances of a *resource access* can then be pairwise tested wheter they are conflicting and thereby creating a data-dependency (e.g. two writes to the same resource).
So each task carries a list of these resource-accesses in its so-called **task-properties**.
If two tasks have conflicting resource-accesses, the first created task is executed first.
This is exactly the behaviour that one would also achieve when programming serially, without hints given via resources.

When tasks are created, their resource-access list is compared against the previous enqueued tasks and corresponding dependencies are created in the task-graph.
The resulting task-graph is read by a scheduling algorithm that executes individual tasks, e.g. across parallel threads.

### Example

See [examples](examples) for examples covering more features.

```c++
```cpp
#include <cassert>
#include <redGrapes/manager.hpp>
#include <redGrapes/resource/ioresource.hpp>
Expand Down Expand Up @@ -68,16 +82,20 @@ int main()
```

## Documentation
RedGrapes is documented using in-code doxygen comments and reStructured-Text-files (in [docs/source](docs/source)), built with Sphinx.

RedGrapes is documented using in-code doxygen comments and reStructured-text files (in [docs/source](docs/source)), build with Sphinx.

* [Getting Started](docs/source/tutorial/index.rst)
* [Components](docs/source/components.rst)

## License

This Project is free software, licensed under the [Mozilla MPL 2.0 license](LICENSE).

## Authors
RedGrapes is based on the high-level ideas described in a [whitepaper by Huebl, Widera, Matthes](docs/2017_02_ResourceManagerDraft.pdf), members of the [Computational Radiation Physics Group](https://hzdr.de/crp) at [HZDR](https://www.hzdr.de/).
## Author Contributions

RedGrapes is developed by members of the [Computational Radiation Physics Group](https://hzdr.de/crp) at [HZDR](https://www.hzdr.de/).
Its conceptual design is based on a [whitepaper by A. Huebl, R. Widera, and M. Matthes (2017)](docs/2017_02_ResourceManagerDraft.pdf).

* [Michael Sippel](https://github.com/michaelsippel): library design & implementation
* [Dr. Sergei Bastrakov](https://github.com/sbastrakov): supervision
Expand All @@ -86,7 +104,9 @@ RedGrapes is based on the high-level ideas described in a [whitepaper by Huebl,
* [Alexander Matthes](https://github.com/theZiz): whitepaper

### Dependencies
Besides the C++14 standard, RedGrapes also depends on the following additional libraries:

RedGrapes requires a compiler supporting the C++14 standard.
RedGrapes further depends on the following libraries:

* [Boost Graph](https://www.boost.org/doc/libs/1_71_0/libs/graph/doc/)
* [optional for C++14](https://github.com/akrzemi1/Optional) by [Andrzej Krzemienski](https://github.com/akrzemi1)

0 comments on commit 23d5d84

Please sign in to comment.