Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skeleton refactoring #28

Merged
merged 77 commits into from
Dec 9, 2022
Merged
Changes from 1 commit
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
7ee19a1
DeviceTypeUtil: fixing name in string.
massimim Jul 14, 2022
1fe86f4
Adding a mutable version of getBackend in GridBase.
massimim Jul 14, 2022
ef1b18a
WIP: New container interface
massimim Jul 14, 2022
a07f632
Adding some extra information to report for a device report.
massimim Jul 14, 2022
cc64f77
WIP: New container interface
massimim Jul 14, 2022
f03bb2e
Documentation
massimim Jul 18, 2022
f4d851d
Merge remote-tracking branch 'origin/develop' into container
massimim Jul 18, 2022
1d06c32
Fixing merging issues.
massimim Jul 18, 2022
090e57f
Documentation
massimim Jul 18, 2022
182155b
Imposing coding standards on the Container class.
massimim Jul 18, 2022
0a944c3
WIP: new container graph mechanism.
massimim Jul 19, 2022
b2cefa1
Merge remote-tracking branch 'origin/develop' into container
massimim Aug 8, 2022
3c7a25b
WIP
massimim Aug 8, 2022
7942c0a
WIP
massimim Aug 9, 2022
23b2d34
WIP
massimim Aug 10, 2022
87ffb4c
WIP
massimim Aug 12, 2022
7d11d4d
WIP
massimim Aug 12, 2022
c05341e
WIP
massimim Aug 12, 2022
78bb946
WIP
massimim Aug 12, 2022
fe27bd2
WIP
massimim Aug 12, 2022
f709e8b
WIP
massimim Aug 12, 2022
cf2dee4
WIP
massimim Aug 12, 2022
0796ddb
WIP
massimim Aug 22, 2022
16f4e5d
WIP
massimim Aug 22, 2022
1583db7
WIP
massimim Aug 23, 2022
bd57943
WIP
massimim Aug 23, 2022
b62fb87
WIP
massimim Aug 24, 2022
150a045
WIP
massimim Aug 29, 2022
601915f
WIP
massimim Aug 30, 2022
1a67851
WIP
massimim Aug 31, 2022
cc478a8
WIP
massimim Sep 2, 2022
3ca3d30
Merge remote-tracking branch 'origin/develop' into container
massimim Sep 2, 2022
08a3a40
New container graph completed, including unit tests.
massimim Sep 12, 2022
04295df
WIP
massimim Sep 17, 2022
8059cf3
WIP
massimim Sep 17, 2022
b9c587b
New container graph completed, including unit tests.
massimim Sep 12, 2022
7d377c3
Merge branch 'container' into skeleton
massimim Sep 17, 2022
c3383ab
New container graph completed, including unit tests.
massimim Sep 12, 2022
61bad15
Merge branch 'container' into skeleton
massimim Sep 18, 2022
0b3aa25
WIP
massimim Sep 18, 2022
c696b27
Fixing compilation issue on windows.
massimim Sep 18, 2022
7401d94
WIP
massimim Sep 18, 2022
eec0094
Merge branch 'container' into skeleton
massimim Sep 18, 2022
0b3b74f
WIP
massimim Sep 24, 2022
df579ac
Merge remote-tracking branch 'origin/develop' into skeleton
massimim Oct 21, 2022
7e08c1c
Fixing merging issues after merging develop.
massimim Oct 21, 2022
2c4b9ca
Some clean up.
massimim Oct 21, 2022
e9f9f84
Merge remote-tracking branch 'origin/develop' into skeleton
massimim Oct 25, 2022
4f00eaa
Merge remote-tracking branch 'origin/develop' into skeleton
massimim Oct 25, 2022
f4072ff
WIP
massimim Oct 25, 2022
fe762b2
New Container for data transfers.
massimim Oct 25, 2022
a8b3a03
Managing communication nodes.
massimim Oct 30, 2022
e8f0218
Working on halo updates.
massimim Oct 31, 2022
30b428a
No OCC graph generator complete.
massimim Nov 3, 2022
d94769b
Fixing ioToDot text.
massimim Nov 3, 2022
c74bb2d
Fix for BFS.
massimim Nov 4, 2022
5696b5b
WIP - fixing execution order.
massimim Nov 4, 2022
aaa0a27
Standard, extended and two way OCC optimisations.
massimim Nov 4, 2022
c15335f
WIP
massimim Nov 8, 2022
7228886
WIP
massimim Nov 8, 2022
b7927d0
Fixing old managed container API.
massimim Nov 8, 2022
2a833b5
Mechanism to support eGrid in the new skeleton.
massimim Nov 8, 2022
db8a4fb
Mechanism to support eGrid in the new skeleton.
massimim Nov 8, 2022
78095e2
Merge remote-tracking branch 'origin/skeleton' into skeleton
massimim Nov 8, 2022
a924689
Cleaning up.
massimim Nov 8, 2022
5e874af
WIP
massimim Nov 9, 2022
ddaf844
Adding support for those cases where a stencil nodes follows the begi…
massimim Nov 10, 2022
58cd81c
Adding support for those cases where a stencil nodes follows the begi…
massimim Nov 10, 2022
4fd3454
Adding support for those cases where a stencil nodes follows the begi…
massimim Nov 10, 2022
347d9a3
Fixing compilation issue on Ubuntu.
massimim Nov 22, 2022
e49b486
Fixing compiler issues on Windows
massimim Nov 23, 2022
447d784
WIP
massimim Nov 25, 2022
c5fcc6a
Increasing domain side for 8-GPU computation.
massimim Nov 25, 2022
c85a387
Disabling domain-unit-test-staggered-grid NodeToVoxels.dGrid test.
massimim Nov 25, 2022
754ced0
Adding Ahmed's fix for the cmake-openmp issue on windows.
massimim Dec 8, 2022
108dd9a
Temporarily disabling staggered grid unit tests.
massimim Dec 8, 2022
4c9fc6b
Fixing compiler issue on Windows.
massimim Dec 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Working on halo updates.
massimim committed Oct 31, 2022
commit e8f0218e143ab7fada802e75bb1a6517be7c8219
14 changes: 7 additions & 7 deletions libNeonDomain/include/Neon/domain/internal/eGrid/eFieldDev.h
Original file line number Diff line number Diff line change
@@ -67,7 +67,7 @@ class eFieldDevice_t
std::array<
std::array<
std::vector<Neon::set::Transfer>,
Neon::set::TransferSemanticUtils::nOptions>,
Neon::set::StencilSemanticUtils::nOptions>,
Neon::set::TransferModeUtils::nOptions>
m_haloUpdateInfo;
};
@@ -125,7 +125,7 @@ class eFieldDevice_t
for (auto mode : {Neon::set::TransferMode::get, Neon::set::TransferMode::put}) {

{ // (GET,PUT), FORWARD, GRID
const auto structure = Neon::set::TransferSemantic::grid;
const auto structure = Neon::set::StencilSemantic::standard;
auto& transfers = h_haloUpdateInfo(mode, structure);
Neon::set::HuOptions huOptions(mode, transfers, structure);
this->haloUpdate__(m_data->grid->getBackend(), huOptions);
@@ -143,7 +143,7 @@ class eFieldDevice_t
}

auto h_haloUpdateInfo(Neon::set::TransferMode mode,
Neon::set::TransferSemantic structure)
Neon::set::StencilSemantic structure)
-> std::vector<Neon::set::Transfer>&
{
return m_data->m_haloUpdateInfo[static_cast<int>(mode)]
@@ -491,7 +491,7 @@ class eFieldDevice_t
#pragma omp parallel for num_threads(ompNDevs) default(shared)
for (int setIdx = 0; setIdx < nDevs; setIdx++) {
for (int cardIdx = 0; cardIdx < self().cardinality(); cardIdx++) {
constexpr auto structure = Neon::set::TransferSemantic::grid;
constexpr auto structure = Neon::set::StencilSemantic::standard;

auto& peerTransferOpt = opt.getPeerTransferOpt(bk);
h_huSoAByCardSingleDevFwd(peerTransferOpt,
@@ -656,7 +656,7 @@ class eFieldDevice_t

Neon::set::PeerTransferOption& peerTransferOpt = huOptions.getPeerTransferOpt(bk);
// LATTICE + FORWARD
constexpr auto structure = Neon::set::TransferSemantic::lattice;
constexpr auto structure = Neon::set::StencilSemantic::streaming;

h_huSoAByCardSingleDevFwd(peerTransferOpt,
gpuIdx, cardIdx,
@@ -808,7 +808,7 @@ class eFieldDevice_t
auto h_huSoAByCardSingleDevFwd(Neon::set::PeerTransferOption opt,
int devIdx,
const int cardIdx,
Neon::set::TransferSemantic structure) const
Neon::set::StencilSemantic structure) const
-> void
{
const Neon::Backend& bk = m_data->grid->getBackend();
@@ -846,7 +846,7 @@ class eFieldDevice_t
}
}

if (structure == Neon::set::TransferSemantic::lattice) {
if (structure == Neon::set::StencilSemantic::streaming) {
index_3d dir = grid().getStencil().neighbours().at(cardIdx);
switch (comDirection) {
case ComDirection_e::COM_DW: {
6 changes: 2 additions & 4 deletions libNeonSet/include/Neon/set/Containter.h
Original file line number Diff line number Diff line change
@@ -1,18 +1,16 @@
#pragma once

#include "Neon/set/DevSet.h"
#include "Neon/set/dependency/Token.h"
#include "functional"
#include "type_traits"

#include "Neon/set/container/ContainerAPI.h"
#include "Neon/set/container/types/HostManagedSyncType.h"
#include "Neon/set/container/types/SynchronizationContainerType.h"

#include "Neon/set/container/Loader.h"

namespace Neon::set {

struct Loader;

struct Container
{
@@ -111,7 +109,7 @@ struct Container
template <typename MultiXpuDataT>
static auto factoryDataTransfer(const MultiXpuDataT& multiXpuData,
Neon::set::TransferMode transferMode,
Neon::set::TransferSemantic transferSemantic)
Neon::set::StencilSemantic transferSemantic)
-> Neon::set::Container;

template <typename MxpuDataT>
4 changes: 3 additions & 1 deletion libNeonSet/include/Neon/set/Containter_imp.h
Original file line number Diff line number Diff line change
@@ -5,6 +5,8 @@
#include "functional"
#include "type_traits"

#include "Neon/set/container/Loader.h"

#include "Neon/set/container/DataTransferContainer.h"
#include "Neon/set/container/DeviceContainer.h"
#include "Neon/set/container/DeviceManagedContainer.h"
@@ -91,7 +93,7 @@ template <typename MultiXpuDataT>
auto Container::
factoryDataTransfer(const MultiXpuDataT& multiXpuData,
Neon::set::TransferMode transferMode,
Neon::set::TransferSemantic transferSemantic)
Neon::set::StencilSemantic transferSemantic)
-> Neon::set::Container
{
auto k = new Neon::set::internal::DataTransferContainer(multiXpuData,
8 changes: 4 additions & 4 deletions libNeonSet/include/Neon/set/HuOptions.h
Original file line number Diff line number Diff line change
@@ -11,17 +11,17 @@ struct HuOptions
bool m_startWithBarrier = true;
int m_streamSetIdx = 0;
Neon::set::PeerTransferOption m_peerTransferOpt;
Neon::set::TransferSemantic m_structure;
Neon::set::StencilSemantic m_structure;

public:
HuOptions(Neon::set::TransferMode transferMode /*< Mode of the transfer: put or get */,
bool startWithBarrier /*< If true a barrier is executed before initiating the halo update */,
int streamSetIdx = Neon::Backend::mainStreamIdx /*< Target stream for the halo update */,
Neon::set::TransferSemantic structure = Neon::set::TransferSemantic::grid /*< Structure on top of which the transfer is one: grid or lattice */);
Neon::set::StencilSemantic structure = Neon::set::StencilSemantic::standard /*< Structure on top of which the transfer is one: grid or lattice */);

HuOptions(Neon::set::TransferMode transferMode,
NEON_OUT std::vector<Neon::set::Transfer>& transfers,
Neon::set::TransferSemantic structure = Neon::set::TransferSemantic::grid);
Neon::set::StencilSemantic structure = Neon::set::StencilSemantic::standard);

auto getPeerTransferOpt(const Neon::Backend& bk) -> Neon::set::PeerTransferOption&;
auto startWithBarrier() const -> bool;
@@ -30,7 +30,7 @@ struct HuOptions
auto operationMode() const -> Neon::set::PeerTransferOption::operationMode_e;
auto transferMode() const -> Neon::set::TransferMode;
auto isExecuteMode() const -> bool;
auto getSemantic() const -> Neon::set::TransferSemantic;
auto getSemantic() const -> Neon::set::StencilSemantic;
};
} // namespace set
} // namespace Neon
41 changes: 41 additions & 0 deletions libNeonSet/include/Neon/set/StencilSemantic.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#pragma once
#include <string>
#include <vector>

#include "Neon/core/core.h"

namespace Neon::set {

enum struct StencilSemantic
{
standard = 0 /*< Transfer for halo update on grid structure */,
streaming = 1 /*< Transfer for halo update on lattice structure */
};


struct StencilSemanticUtils
{
static constexpr int nOptions = 2;

static auto toString(StencilSemantic opt) -> std::string;
static auto fromString(const std::string& opt) -> StencilSemantic;
static auto getOptions() -> std::array<StencilSemantic, nOptions>;

struct Cli
{
explicit Cli(std::string);
explicit Cli(StencilSemantic model);
Cli();

auto getOption() -> StencilSemantic;
auto set(const std::string& opt) -> void;
auto getStringOptions() -> std::string;

private:
bool mSet = false;
StencilSemantic mOption;
};
};


} // namespace Neon::set
14 changes: 7 additions & 7 deletions libNeonSet/include/Neon/set/Transfer.h
Original file line number Diff line number Diff line change
@@ -3,8 +3,8 @@
#include <vector>

#include "Neon/core/core.h"
#include "Neon/set/StencilSemantic.h"
#include "Neon/set/TransferMode.h"
#include "Neon/set/TransferSemantic.h"

namespace Neon {
namespace set {
@@ -31,14 +31,14 @@ struct Transfer
Endpoint_t m_dst;
Endpoint_t m_src;
size_t m_size{0};
TransferSemantic m_structure;
StencilSemantic m_structure;

public:
Transfer(TransferMode mode,
const Endpoint_t& dst,
const Endpoint_t& src,
size_t size,
TransferSemantic structure = TransferSemantic::grid)
StencilSemantic structure = StencilSemantic::standard)
: m_mode(mode), m_dst(dst), m_src(src), m_size(size), m_structure(structure)
{
}
@@ -96,14 +96,14 @@ struct PeerTransferOption
operationMode_e m_operationMode = Neon::set::PeerTransferOption::execute;
TransferMode m_transferMode = Neon::set::TransferMode::get;

Neon::set::TransferSemantic m_structure;
Neon::set::StencilSemantic m_structure;

public:
/**
* Constructor with stream parameter
*/
explicit PeerTransferOption(TransferMode tranferMode,
Neon::set::TransferSemantic structure)
Neon::set::StencilSemantic structure)
: m_transfers(nullptr),
m_streamSet(nullptr),
m_operationMode(operationMode_e::execute),
@@ -120,7 +120,7 @@ struct PeerTransferOption
*/
PeerTransferOption(TransferMode tranferMode,
std::vector<Transfer>& NEON_OUT transfers,
Neon::set::TransferSemantic structure)
Neon::set::StencilSemantic structure)
: m_transfers(&transfers),
m_streamSet(nullptr),
m_operationMode(operationMode_e::storeInfo),
@@ -180,7 +180,7 @@ struct PeerTransferOption
return *m_transfers;
}

auto structure() const -> const Neon::set::TransferSemantic&
auto structure() const -> const Neon::set::StencilSemantic&
{
return m_structure;
}
42 changes: 0 additions & 42 deletions libNeonSet/include/Neon/set/TransferSemantic.h

This file was deleted.

18 changes: 10 additions & 8 deletions libNeonSet/include/Neon/set/container/ContainerAPI.h
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
#pragma once

#include "Neon/set/container/types/ContainerOperationType.h"
#include "Neon/set/container/types/ContainerPatternType.h"

#include "Neon/set/DevSet.h"
#include "Neon/set/container/types/ContainerExecutionType.h"
#include "Neon/set/container/types/ContainerOperationType.h"
#include "Neon/set/container/types/ContainerPatternType.h"
//#include "Neon/set/dependency/Token.h"


#include "Neon/set/dependency/Token.h"
#include "functional"
#include "type_traits"

@@ -19,6 +17,10 @@ namespace Neon::set::container {
struct Graph;
}

namespace Neon::set::dataDependency {
struct Token;
}

namespace Neon::set::internal {

/**
@@ -71,7 +73,7 @@ struct ContainerAPI
/**
* Returns a handle to the internal graph of Containers.
*/
virtual auto getGraph()
virtual auto getGraph() const
-> const Neon::set::container::Graph&;

/**
@@ -81,8 +83,8 @@ struct ContainerAPI
virtual auto parse()
-> const std::vector<Neon::set::dataDependency::Token>&;

virtual auto getTransferMode()const
->Neon::set::TransferMode ;
virtual auto getTransferMode() const
-> Neon::set::TransferMode;

/**
* Returns a name associated to the container.
4 changes: 2 additions & 2 deletions libNeonSet/include/Neon/set/container/DataTransferContainer.h
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ struct DataTransferContainer

DataTransferContainer(const MxpuDataT& multiXpuData,
Neon::set::TransferMode transferMode,
Neon::set::TransferSemantic transferSemantic)
Neon::set::StencilSemantic transferSemantic)
: mMultiXpuData(multiXpuData),
mTransferMode(transferMode),
mTransferSemantic(transferSemantic)
@@ -64,7 +64,7 @@ struct DataTransferContainer
mDataTransferFun;
MxpuDataT mMultiXpuData;
Neon::set::TransferMode mTransferMode;
Neon::set::TransferSemantic mTransferSemantic;
Neon::set::StencilSemantic mTransferSemantic;
};

} // namespace Neon::set::internal
6 changes: 1 addition & 5 deletions libNeonSet/include/Neon/set/container/DeviceContainer.h
Original file line number Diff line number Diff line change
@@ -4,9 +4,7 @@
#include "Neon/set/container/ContainerAPI.h"
#include "Neon/set/container/Loader.h"

namespace Neon {
namespace set {
namespace internal {
namespace Neon::set::internal {

template <typename DataIteratorContainerT,
typename UserComputeLambdaT>
@@ -150,6 +148,4 @@ struct DeviceContainer : ContainerAPI
DataIteratorContainerT m_dataIteratorContainer;
};

} // namespace internal
} // namespace set
} // namespace Neon
Loading