Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize edge lookup and iteration without undue memory usage increase #534

Closed
milroy opened this issue Oct 9, 2019 · 1 comment
Closed

Comments

@milroy
Copy link
Member

milroy commented Oct 9, 2019

Flux-sched builds and mutates adjacency_list graphs, requiring many edge lookups and iterations. We need to consider how to optimize such operations without an excessive increase in memory footprint.

resource_graph_t currently uses vecS as its OutEdgeList type. While Boost documentation states that vecS uses the least memory "In the order of least space to most space, the selectors are vecS, slistS, listS, and setS."

it also mentions that edge lookups with vecS (Sequence) type edges are potentially costly:

The time complexity for this operation [edge lookup] is O(E/V) when the OutEdgeList type is a Sequence and it is O(log(E/V)) when the OutEdgeList type is an AssociativeContainer.

I have tested substituting hash_setS and setS for vecS as the adjacency_list OutEdgeList type to improve edge lookup performance in the forthcoming subgraph attachment (issue #511). Flux-sched builds successfully with both, but many (~60) Travis tests fail when using hash_setS. The failures are due, at least in part, to the unordered edge iteration that results from using the hash_setS type.

Flux-sched's DFU code explores out_edges in particular subsystems, e.g.:

if (!in_subsystem (*ei, subsystem) || stop_explore (*ei, subsystem))
which could benefit from restricting iteration "only though those outedges that belong to a particular subsystem" (quoting @dongahn ).

It may be possible to meet both of these objectives by choosing the optimal OutEdgeList type, and modifying edge descriptors, add_edge(), out_edges(), and out_edge_iterator.

@dongahn
Copy link
Member

dongahn commented Mar 10, 2022

This is good to have but not actionable for now. We can reopen this when we need this optimization.

@dongahn dongahn closed this as completed Mar 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants