Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate standardising distribute #1585

Open
oerc0122 opened this issue Mar 1, 2024 · 1 comment
Open

Investigate standardising distribute #1585

oerc0122 opened this issue Mar 1, 2024 · 1 comment
Labels
Low Priority Low Priority Tasks Parallel Task needing parallelisation Technical Debt code quality, unit tests, code duplicaton

Comments

@oerc0122
Copy link
Collaborator

oerc0122 commented Mar 1, 2024

Distribute currently uses its own implementation of split_vector_fixed_sum to account for the additional metadata, however, it may be possible to replace this, though it may require some re-working of the parallel merge procedures too.

Places which may be affected:
./horace_core/sqw/@DnDBase/distribute.m
./horace_core/sqw/@sqw/distribute.m
./horace_core/sqw/PixelData/@PixelDataMemory/distribute.m
./horace_core/sqw/PixelData/@PixelDataFileBacked/distribute.m
./herbert_core/utilities/classes/@IX_dataset/distribute.m
./herbert_core/utilities/misc/distribute.m
./herbert_core/classes/MPIFramework/parallel_call/ParallelSQWEval.m
./herbert_core/applications/multifit/MFParallel_Job.m

@oerc0122 oerc0122 added Technical Debt code quality, unit tests, code duplicaton Parallel Task needing parallelisation Low Priority Low Priority Tasks labels Mar 1, 2024
@abuts
Copy link
Member

abuts commented Mar 1, 2024

What I've seen in the code was reminiscent of selection between split_vector_fixed_sum and split_vector_max_sum || split_vector_max_sum_or_numel. I think thes covers all reasonable cases but may be something more is there too and another procedure should be introduced. If it is the case, it should be separately tested procedure with clear scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Low Priority Low Priority Tasks Parallel Task needing parallelisation Technical Debt code quality, unit tests, code duplicaton
Projects
None yet
Development

No branches or pull requests

2 participants