Container Streaming/Retriever

[nvidianz]

Description

1. ContainerStreamer to stream containers.
2. ContainerRetriever to retrieve containers from a remote site.
3. Added examples for file and container streaming.
4. Merged the class loading functions in class_utils and FOBS.
5. Fixed a F3 bug that wipes out original exception.

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Quick tests passed locally by running ./runtest.sh.
• In-line docstrings updated.
• Documentation updated.

[nvidianz]

/build

[yanchengnv]

Looks good in general. See my comments for enhancement.

[ZiyueXu77]

A general comment: shall we do a comparison of memory footprint standard v.s. dict streaming v.s. file streaming? I will do some experiment on how to track the memory usage, I think using JobAPI should work

[ZiyueXu77]

I did some experiments but it seems the basic transmission without dict streaming has similar memory footprint as compared with streaming, not sure if it is expected (@nvidianz I created a PR on your branch, let me know if you find any mistakes in my code)

[ZiyueXu77]

Mis-interpreted the results last Friday, the result actually makes sense:

Performed a test with a 6 GB model:
Communicator - INFO - Received from simulator_server server. getTask: retrieve_dict size: 6GB (5994010153 Bytes)
Had system memory usage print out every 0.5 s, and the peak usage below:
Simulator w/o dict streaming: 57451 MB
Simulator w/ dict streaming: 53147 MB
Poc w/o dict streaming: 40910 MB
Poc w/ dict streaming: 35422 MB

Considering the largest layer is ~1GB, the difference should be ~5GB, so the above results on diff are reasonable, note that the recorded memory usage is for the general system, so only the diff is meaningful

[ZiyueXu77]

looks good and I can confirm the functionality part. This PR is ready for merge once Yan's comment is addressed, I will created a separate PR to refine the example.

[nvidianz]

/build

[nvidianz]

/build

[yanchengnv]

LGTM