Code

Lazy loading#220

Code

Draft

TosinSeg wants to merge 70 commits intodeepspeedai:maindeepspeedai/DeepSpeed-MII:mainfrom TosinSeg:lazy-loadingTosinSeg/DeepSpeed-MII:lazy-loadingCopy head branch name to clipboard

+1,517-683

Commits

Commits on Jun 19, 2023

• Removing load balancing config

  TosinSeg

  committed
  4eac006

  Copy full SHA for 4eac006

Commits on Jun 20, 2023

• Reformatting tests

  TosinSeg

  committed
  c68e999

  Copy full SHA for c68e999

• Fixed the formatting

  TosinSeg

  committed
  5ce1a92

  Copy full SHA for 5ce1a92

• Removed print statement

  TosinSeg

  committed
  fa10e19

  Copy full SHA for fa10e19

Commits on Jun 26, 2023

• Merging main

  TosinSeg

  committed
  f9cbd74

  Copy full SHA for f9cbd74

• Removing unused import

  TosinSeg

  committed
  8970f4e

  Copy full SHA for 8970f4e

• Fixing tests

  TosinSeg

  committed
  517bea8

  Copy full SHA for 517bea8

• Fixing merge issue

  TosinSeg

  committed
  58dd2b2

  Copy full SHA for 58dd2b2

• Creating hostfile when one is not provided

  TosinSeg

  committed
  bb0d551

  Copy full SHA for bb0d551

• Merge branch 'main' into Always_enable_load_balancing

  TosinSeg

  committed
  e2bb9d5

  Copy full SHA for e2bb9d5

• Fixing import statements removed by merge

  TosinSeg

  committed
  3823534

  Copy full SHA for 3823534

• Removing load_balancing check

  TosinSeg

  committed
  6f9b4ad

  Copy full SHA for 6f9b4ad

• Removing redudant definitions

  TosinSeg

  committed
  499b9ad

  Copy full SHA for 499b9ad

• Removing hostfile from test

  TosinSeg

  committed
  5419ef6

  Copy full SHA for 5419ef6

• Removing hostfile from non-persistent test

  TosinSeg

  committed
  a70b6de

  Copy full SHA for a70b6de

Commits on Jun 27, 2023

• initial changes

  TosinSeg

  committed
  eea658b

  Copy full SHA for eea658b

• Merge branch 'main' into multi-model-deployment

  TosinSeg

  committed
  20f0878

  Copy full SHA for 20f0878

Commits on Jun 28, 2023

• Maintaining current behavior

  TosinSeg

  committed
  c21c31b

  Copy full SHA for c21c31b

• Reading from score file

  TosinSeg

  committed
  f525329

  Copy full SHA for f525329

• fixing syntax errors

  TosinSeg

  committed
  3c0937f

  Copy full SHA for 3c0937f

• Fixing more syntax errors

  TosinSeg

  committed
  156ac83

  Copy full SHA for 156ac83

Commits on Jun 29, 2023

• Fixing more syntax issues

  TosinSeg

  committed
  38e270e

  Copy full SHA for 38e270e

• initial lb changes

  TosinSeg

  committed
  4d4e0d8

  Copy full SHA for 4d4e0d8

• Merge branch 'main' into multi-model-deployment

  TosinSeg

  committed
  01c8e59

  Copy full SHA for 01c8e59

• More load balancing changes

  TosinSeg

  committed
  f801b36

  Copy full SHA for f801b36

Commits on Jun 30, 2023

• LB changes and syntax

  TosinSeg

  committed
  fd4e2ed

  Copy full SHA for fd4e2ed

• Refactor client, and unpack request in load balancer

  TosinSeg

  committed
  0a3b7e5

  Copy full SHA for 0a3b7e5

Commits on Jul 3, 2023

• First working queries

  TosinSeg

  committed
  6523c04

  Copy full SHA for 6523c04

• Fixing conversational and q&a args

  TosinSeg

  committed
  06b40f5

  Copy full SHA for 06b40f5

Commits on Jul 5, 2023

• Updates to _allocate_processes and fixing example

  TosinSeg

  committed
  96d0dcb

  Copy full SHA for 96d0dcb

• Adding host map for allocating processes and formatting

  TosinSeg

  committed
  ab41d24

  Copy full SHA for ab41d24

• Fixing terminate functionality

  TosinSeg

  committed
  8673a9a

  Copy full SHA for 8673a9a

Commits on Jul 6, 2023

• Refactored client

  TosinSeg

  committed
  8d09b37

  Copy full SHA for 8d09b37

• More Refactoring and q/a example

  TosinSeg

  committed
  7a136d6

  Copy full SHA for 7a136d6

• Reformatting to maintain previous syntax

  TosinSeg

  committed
  2c6ec08

  Copy full SHA for 2c6ec08

• Removing print/debug statements

  TosinSeg

  committed
  0cb88a9

  Copy full SHA for 0cb88a9

• Fixing non-persistent deloyments

  TosinSeg

  committed
  7c0ee12

  Copy full SHA for 7c0ee12

Commits on Jul 7, 2023

• Refactoring Load balancer launch

  TosinSeg

  committed
  7a956d5

  Copy full SHA for 7a956d5

Commits on Jul 10, 2023

• Fixing restful gateway client

  TosinSeg

  committed
  f8cfe28

  Copy full SHA for f8cfe28

• Fixing replica issue

  TosinSeg

  committed
  079807d

  Copy full SHA for 079807d

• Fixing non persistent client

  TosinSeg

  committed
  ea1e47e

  Copy full SHA for ea1e47e

Commits on Jul 12, 2023

• Refactoring

  TosinSeg

  committed
  daab5e6

  Copy full SHA for daab5e6

• Update mii/models/score/generate.py

  Show description for 84073f9
  

  TosinSeg

  and

  mrwyattii

  authored
  84073f9

  Copy full SHA for 84073f9

Commits on Jul 13, 2023

• Merge branch 'multi-model-deployment' of github.com:TosinSeg/DeepSpeed-MII into multi-model-deployment

  

  Tosin Segun

  committed
  3ee3410

  Copy full SHA for 3ee3410

• Refactoring Load Balancer and request_proto

  

  Tosin Segun

  committed
  b4edc2b

  Copy full SHA for b4edc2b

• Formatting

  

  Tosin Segun

  committed
  6346194

  Copy full SHA for 6346194

Commits on Jul 14, 2023

• Fixing the client

  

  Tosin Segun

  committed
  94b6699

  Copy full SHA for 94b6699

Commits on Jul 21, 2023

• Initial partial deployment commit

  

  Tosin Segun

  committed
  710c20b

  Copy full SHA for 710c20b

• More partial deploy updates

  

  Tosin Segun

  committed
  c2636b7

  Copy full SHA for c2636b7

• Partial deploy started

  

  Tosin Segun

  committed
  189e75c

  Copy full SHA for 189e75c

Commits on Jul 24, 2023

• fixing add deploy api queries

  

  Tosin Segun

  committed
  adee843

  Copy full SHA for adee843

• Support for empty deployment 'group'

  

  Tosin Segun

  committed
  a145be5

  Copy full SHA for a145be5

• Support for empty deployment 'group'

  

  Tosin Segun

  committed
  082c05e

  Copy full SHA for 082c05e

Commits on Jul 25, 2023

• Partial Termination

  

  Tosin Segun

  committed
  3ce77d2

  Copy full SHA for 3ce77d2

• Refactoring

  

  Tosin Segun

  committed
  b40ecbd

  Copy full SHA for b40ecbd

• formatting

  

  Tosin Segun

  committed
  72dd95c

  Copy full SHA for 72dd95c

• fixing bug for partial termination

  

  Tosin Segun

  committed
  a4e3d56

  Copy full SHA for a4e3d56

• Removing comments

  

  Tosin Segun

  committed
  4b5bb47

  Copy full SHA for 4b5bb47

Commits on Jul 26, 2023

• Including GPU index map in score file

  

  Tosin Segun

  committed
  30d2b03

  Copy full SHA for 30d2b03

• Refactoring deployment

  

  Tosin Segun

  committed
  c5d5996

  Copy full SHA for c5d5996

• Refactoring and formatting

  

  Tosin Segun

  committed
  3ae1781

  Copy full SHA for 3ae1781

Commits on Jul 28, 2023

• Refactoring

  

  Tosin Segun

  committed
  4b8f02f

  Copy full SHA for 4b8f02f

• Fixing Readme

  

  Tosin Segun

  committed
  c51ce37

  Copy full SHA for c51ce37

• Refactoring GRPC

  

  Tosin Segun

  committed
  43479db

  Copy full SHA for 43479db

• Fixing LB process not terminating

  

  Tosin Segun

  committed
  e1b6d23

  Copy full SHA for e1b6d23

Commits on Jul 31, 2023

• Adding multi_deployment and partial deploy/terminate unit tests

  

  Tosin Segun

  committed
  1675bd8

  Copy full SHA for 1675bd8

• Removing comments

  

  Tosin Segun

  committed
  8684a61

  Copy full SHA for 8684a61

Commits on Aug 1, 2023

• Fixing spelling issues

  

  Tosin Segun

  committed
  56a7fce

  Copy full SHA for 56a7fce

Commits on Aug 4, 2023

• Initial lazy loading changes

  

  Tosin Segun

  committed
  71dd350

  Copy full SHA for 71dd350

Commits on Aug 10, 2023

• Adding print statements

  

  Tosin Segun

  committed
  66fd055

  Copy full SHA for 66fd055