feat: multithreading support for erasure coding #1087

munna0908 · 2025-01-23T17:20:01Z

This PR introduces the following changes:

Utilizes TaskPool to spawn separate threads for the erasure encoding and decoding.
Adds a new num-threads flag, enabling users to specify the number of CPU threads available for TaskPool workers.

Docs PR

gmega

I think this is good, well done. I see one minor issue with the exception handling but otherwise looks great. We should wait for @dryajov's review on this one anyhow though.

gmega · 2025-01-24T13:02:19Z

codex/codex.nim

+      taskpool = Taskpool.new(numThreads = config.numThreads)
+    info "Threadpool started", numThreads = taskpool.numThreads
+  except Exception:
+    raise newException(Defect, "Failure in taskpool initialization.")


🟡 Not sure we want to be casting everything as Defect and bubbling it up this way? I think we'd use a result type for this instead?

@munna0908 Exception is a parent type for Defect and CatchableError, the latter of which all other "catchable" error types are derived from. Defect is not catchable, and therefore we should not be using except Expection where we can avoid it (despite its usage in other places in the codebase).

In this particular case, using except CatchableError and converting to a Defect is a valid pattern, and it also means that if a Defect was raised in the try block, it would not be caught in the except, and would be bubbled up the stack.

See https://status-im.github.io/nim-style-guide/errors.exceptions.html for more info on exception handling best practices (TL;DR using Result is recommended)

@gmega I agree that we should probably return a Result[void] type here, read the Result in the layer above, and then quit on error. However, the layer above currently quits on exception, and there are a fair few instances of raised Defects in the new proc, so converting everything to Result would require a decent amount of cascading changes. For the scope of this PR, I think that Defect is suitable since there is no recovery opportunity here.

So my suggestion would be something like:

try: # start taskpool except CatchableError as parent: raise newException(Defect, "Failure in taskpool initialisation: " & parent.msg, parent)

@emizzle Commit-link

codex/conf.nim

codex/erasure/erasure.nim

emizzle

Nice job Rahul! I left mostly suggestions, just one question around requiring threading to be turned on and the configuration requirements/validation.

emizzle · 2025-01-28T04:13:09Z

codex/codex.nim

+      taskpool = Taskpool.new(numThreads = config.numThreads)
+    info "Threadpool started", numThreads = taskpool.numThreads
+  except Exception:
+    raise newException(Defect, "Failure in taskpool initialization.")


@munna0908 Exception is a parent type for Defect and CatchableError, the latter of which all other "catchable" error types are derived from. Defect is not catchable, and therefore we should not be using except Expection where we can avoid it (despite its usage in other places in the codebase).

In this particular case, using except CatchableError and converting to a Defect is a valid pattern, and it also means that if a Defect was raised in the try block, it would not be caught in the except, and would be bubbled up the stack.

See https://status-im.github.io/nim-style-guide/errors.exceptions.html for more info on exception handling best practices (TL;DR using Result is recommended)

@gmega I agree that we should probably return a Result[void] type here, read the Result in the layer above, and then quit on error. However, the layer above currently quits on exception, and there are a fair few instances of raised Defects in the new proc, so converting everything to Result would require a decent amount of cascading changes. For the scope of this PR, I think that Defect is suitable since there is no recovery opportunity here.

So my suggestion would be something like:

try: # start taskpool except CatchableError as parent: raise newException(Defect, "Failure in taskpool initialisation: " & parent.msg, parent)

codex/erasure/erasure.nim

emizzle · 2025-01-28T04:26:36Z

codex/erasure/erasure.nim

+    blockSize, ecK, ecM: int,
+    data: ref seq[seq[byte]],
+    parity: ptr seq[seq[byte]],
+): Future[Result[void, string]] {.async.} =


🟡 This is completely correct, just highlighting that another option would be to continue the pattern in this module of returning ?!void, which is an alias forResult[void, CatchableError], from the questionable lib. Then, you can return success() or failure("something"). Again, just a suggestion.

Commit-link

emizzle · 2025-01-28T04:28:32Z

codex/erasure/erasure.nim

@@ -87,6 +90,21 @@ type
    # provided.
    minSize*: NBytes

+  EncodeTask = object
+    result: Atomic[bool]


result is generally a reserved word. Not sure if we should use something else? @dryajov?

@emizzle Good point, I will change this to success

Commit-link

It should be fine inside a struct/object, I don't think it will be a problem, but might be prudent to use a different name, res/ok?

emizzle · 2025-01-28T04:32:22Z

codex/erasure/erasure.nim

+    blockSize, ecK, ecM: int,
+    data, parity: ref seq[seq[byte]],
+    recovered: ptr seq[seq[byte]],
+): Future[Result[void, string]] {.async.} =


Probably a good idea to tell the compiler we don't want any exceptions leaking out of this routine, so we don't have any issues with crashing thread tasks

Suggested change

): Future[Result[void, string]] {.async.} =

): Future[Result[void, string]] {.async: (raises:[]).} =

Same for encodeAsync

@emizzle Commit-link

dryajov · 2025-01-28T16:38:16Z

codex/codex.nim

+    if config.numThreads == 0:
+      taskpool = Taskpool.new(numThreads = min(countProcessors(), 16))
+    elif config.numThreads < 2:
+      raise newException(


I would use a raiseAssert here instead.

Actually, I think this is better checked in the conf.nim so that the invalid value doesn't get passed the cmd validation.

dryajov · 2025-01-28T16:38:40Z

codex/codex.nim

+    info "Threadpool started", numThreads = taskpool.numThreads
+  except CatchableError as parent:
+    raise
+      newException(Defect, "Failure in taskpool initialization:" & parent.msg, parent)


Probably do a raiseAssert as well.

emizzle

Thanks for making the changes, Rahul. Apart from the minor suggestions (raiseAssert instead of raise newException(Defect...), this LGTM. So, I'll approve now.

emizzle

I just noticed the CancelledError changes, which brought to mind that maybe we need to consider gracefully terminating the TaskPool in the stop routine.

Sorry for the premature approval before!

emizzle · 2025-01-28T23:31:20Z

codex/erasure/erasure.nim

+    warn "Async thread error", err = exc.msg
+    return failure(exc.msg)
+  except CancelledError:
+    return failure("Thread cancelled")


Here, do we want to return a failure? I thought about this a bit further... this proc executes in the master thread, correct? If so, maybe we should propagate CancelledErrors. So the raises annotation would be:

{.async: (raises: [CacncelledError]).}

and the except block would re-raise:

except CancelledError as e: raise e

This brings up another question -- should the TaskPool be gracefully terminated in stop? Or is there no point to wait for worker threads to join master? IOW, would stopping the master thread kill the worker at the OS-level and possibly produce an out-of-band error?

Yes, we need to properly cleanup everything, propagate the cancelation, and then make sure we don't exit prematurely at the callsite of both asyncEncode/asyncDecode.

Btw, we can't simply exit on cancellation, we need to wait for the worker to finish, so wait threadPtr.wait() should still be called.

Hmm, couldn't that create a deadlock situation where the task is cancelled with cancelAndWait and but cancellation has wait on the thread to finish its operation before cancelling? Is there a way to forcefully kill the task and discard any errors or results?

emizzle · 2025-01-28T23:31:38Z

codex/erasure/erasure.nim

+    warn "Async thread error", err = exc.msg
+    return failure(exc.msg)
+  except CancelledError:
+    return failure("Thread cancelled")


Same comment as above.

dryajov

I'm still reviewing this changes, so lets not merge this just yet.

dryajov · 2025-01-30T01:03:46Z

codex/erasure/erasure.nim

+    parity: ptr seq[seq[byte]],
+): Future[?!void] {.async: (raises: []).} =
+  ## Create an encoder backend instance
+  let encoder = self.encoderProvider(blockSize, ecK, ecM)


The backend needs to be created in the worker thread, this creates the backend on the main thread, it should also be released once done.

dryajov · 2025-01-30T01:05:11Z

codex/erasure/erasure.nim

+    warn "Async thread error", err = exc.msg
+    return failure(exc.msg)
+  except CancelledError:
+    return failure("Thread cancelled")


Yes, we need to properly cleanup everything, propagate the cancelation, and then make sure we don't exit prematurely at the callsite of both asyncEncode/asyncDecode.

dryajov · 2025-01-30T01:08:21Z

codex/erasure/erasure.nim

+    return failure("Thread cancelled")
+  finally:
+    # release the encoder
+    t.encoder.release()


you also need to release the ThreadSignalPtr. This was actually a fun one, because it took me some time to figure out what was wrong, as I was seeing things fail arbitrarily. The issue is that, ThreadSignalPtr uses pipes undereat to communicate between threads and if you don't dispose of it properly, you'll eventually hit a limit on opened filehandles. So, you need to call threadPtr.close() here.

dryajov · 2025-01-30T01:11:07Z

codex/erasure/erasure.nim

+    warn "Async thread error", err = exc.msg
+    return failure(exc.msg)
+  except CancelledError:
+    return failure("Thread cancelled")


Btw, we can't simply exit on cancellation, we need to wait for the worker to finish, so wait threadPtr.wait() should still be called.

dryajov

Great job for a first pass @munna0908!

Let me summarize what needs to be changed:

Rework how buffers are passed arround from ptr seq[seq[]] to ptr UncheckedArray[ptr UncheckedArray[byte]]
Dispose of the thread signal
Handle cancelations
Add tests to explicitely test the multithreaded functionality in testerasure.nim

munna0908 · 2025-02-10T03:16:40Z

@dryajov I have accommodated all suggested changes including the test cases

dryajov · 2025-02-10T21:55:16Z

codex/erasure/erasure.nim

+      return failure(asyncExc.msg)
+    finally:
+      if exc of CancelledError:
+        raise (ref CancelledError) exc


no need to cast it here, just re-raise

@dryajov, this proc should only raise CancelledError, but exc is of type CatchableError.

dryajov · 2025-02-10T21:57:41Z

codex/erasure/erasure.nim

+      else:
+        return failure(exc.msg)
+  finally:
+    threadPtr.close().expect("closing once works")


maybe, this can be in a defer block instead?

Commit-link

gmega

Good job!

dryajov · 2025-02-11T19:04:11Z

codex/erasure/erasure.nim

+  except CatchableError as exc:
+    try:
+      await threadFut
+    except AsyncError as asyncExc:


Why AsyncError here and not CatchableError?

@dryajov AsyncError would be more appropriate as threadPtr.wait() raises an AsyncError, ref

dryajov

LGTM! Thanks!

munna0908 marked this pull request as draft January 23, 2025 17:20

munna0908 requested review from gmega and dryajov January 24, 2025 12:25

munna0908 self-assigned this Jan 24, 2025

munna0908 marked this pull request as ready for review January 24, 2025 12:26

munna0908 added the Client See https://miro.com/app/board/uXjVNZ03E-c=/ for details label Jan 24, 2025

gmega requested changes Jan 24, 2025

View reviewed changes

emizzle reviewed Jan 28, 2025

View reviewed changes

munna0908 requested review from gmega and emizzle January 28, 2025 14:54

munna0908 force-pushed the feature/async-erasure branch from 3d8d133 to f1b8dfc Compare January 28, 2025 14:55

dryajov reviewed Jan 28, 2025

View reviewed changes

emizzle approved these changes Jan 28, 2025

View reviewed changes

emizzle reviewed Jan 28, 2025

View reviewed changes

emizzle self-requested a review January 28, 2025 23:38

dryajov approved these changes Jan 29, 2025 •

edited

Loading

View reviewed changes

dryajov requested changes Jan 29, 2025

View reviewed changes

dryajov requested changes Jan 30, 2025

View reviewed changes

dryajov approved these changes Jan 30, 2025 •

edited

Loading

View reviewed changes

dryajov requested changes Jan 30, 2025

View reviewed changes

munna0908 force-pushed the feature/async-erasure branch from ce80d68 to 374a815 Compare February 7, 2025 11:45

munna0908 requested a review from dryajov February 10, 2025 03:17

dryajov reviewed Feb 10, 2025

View reviewed changes

munna0908 added 4 commits February 11, 2025 22:41

implement async encode

7df0efe

implement async decode

a686edc

cleanup code

c82d135

add num-threads flag

b101b58

munna0908 added 14 commits February 11, 2025 22:41

fix tests

d2f1131

code cleanup

a786292

improve return types and exception handling for async proc

615bff8

add validation check for numThreads flag

31a091e

modify encode method

8888b73

add new tests for aync encoding

61cb37a

modify decode method

72f22fb

cleanup test cases

03c5ec0

add new cli flag for threadCount

2d40e97

test cleanup

d1c7b1f

add new tests

024ea28

fix decodeAsync exception handling

2af6a12

code cleanup

e33c9fd

chore: cosmetic changes

1c859a8

munna0908 force-pushed the feature/async-erasure branch from de59003 to 1c859a8 Compare February 11, 2025 17:11

gmega approved these changes Feb 11, 2025

View reviewed changes

dryajov reviewed Feb 11, 2025

View reviewed changes

dryajov approved these changes Feb 12, 2025

View reviewed changes

gmega added this pull request to the merge queue Feb 12, 2025

Merged via the queue into master with commit c651488 Feb 12, 2025
19 checks passed

gmega deleted the feature/async-erasure branch February 12, 2025 19:14

	): Future[Result[void, string]] {.async.} =
	): Future[Result[void, string]] {.async: (raises:[]).} =

feat: multithreading support for erasure coding #1087

feat: multithreading support for erasure coding #1087

Conversation

munna0908 commented Jan 23, 2025 • edited Loading

gmega left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emizzle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emizzle left a comment

Choose a reason for hiding this comment

emizzle left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dryajov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dryajov left a comment

Choose a reason for hiding this comment

munna0908 commented Feb 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gmega left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dryajov left a comment

Choose a reason for hiding this comment

munna0908 commented Jan 23, 2025 •

edited

Loading

emizzle left a comment •

edited

Loading