Code

Improve overflow handling in ZeRO#6976

Code

Open

tjruwase wants to merge 67 commits intomasterdeepspeedai/DeepSpeed:masterfrom olruwase/ds_5241deepspeedai/DeepSpeed:olruwase/ds_5241Copy head branch name to clipboard

+691-169

Commits

Commits on Jan 28, 2025

• Improve overflow handling in ZeRO

  tjruwase

  committed
  a3a18f7

  Copy full SHA for a3a18f7

• Unit test and pydantic configuration

  tjruwase

  committed
  19431f8

  Copy full SHA for 19431f8

• Formatting fixes

  tjruwase

  committed
  406cf26

  Copy full SHA for 406cf26

Commits on Jan 29, 2025

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  35570f5

  Copy full SHA for 35570f5

• Remove unused symbol

  tjruwase

  committed
  cb78444

  Copy full SHA for cb78444

• Fix typo

  tjruwase

  committed
  ee1c1fd

  Copy full SHA for ee1c1fd

• Pydantic fp16 config

  tjruwase

  committed
  0b2cf73

  Copy full SHA for 0b2cf73

• Fix more typos

  tjruwase

  committed
  c7a90f9

  Copy full SHA for c7a90f9

• Address #4986

  tjruwase

  committed
  3694e07

  Copy full SHA for 3694e07

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  2bbcf00

  Copy full SHA for 2bbcf00

Commits on Jan 30, 2025

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  c1b87ea

  Copy full SHA for c1b87ea

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  5da6cd0

  Copy full SHA for 5da6cd0

• Merge branch 'master' into olruwase/ds_5241

  loadams

  authored
  a65d20c

  Copy full SHA for a65d20c

• Fix typo

  tjruwase

  committed
  ae039b2

  Copy full SHA for ae039b2

• Merge branch 'olruwase/ds_5241' of github.com:microsoft/DeepSpeed into olruwase/ds_5241

  tjruwase

  committed
  0446192

  Copy full SHA for 0446192

Commits on Jan 31, 2025

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  5d48745

  Copy full SHA for 5d48745

• Merge branch 'master' into olruwase/ds_5241

  loadams

  authored
  05c362d

  Copy full SHA for 05c362d

Commits on Feb 1, 2025

• Merge branch 'master' into olruwase/ds_5241

  loadams

  authored
  5e17ed6

  Copy full SHA for 5e17ed6

Commits on Feb 5, 2025

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  06bb3a6

  Copy full SHA for 06bb3a6

• Fix min loss scale

  tjruwase

  committed
  0d0ab3d

  Copy full SHA for 0d0ab3d

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  cccd5b1

  Copy full SHA for cccd5b1

Commits on Feb 6, 2025

• Fix UTs

  tjruwase

  committed
  2c6f630

  Copy full SHA for 2c6f630

• Merge branch 'olruwase/ds_5241' of github.com:microsoft/DeepSpeed into olruwase/ds_5241

  tjruwase

  committed
  21bfca0

  Copy full SHA for 21bfca0

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  5fe5810

  Copy full SHA for 5fe5810

• Using explicit GPU upcast for ZeRO-Offload (#6962)

  Show description for 732ceb7
  

  xylian86

  authored and

  tjruwase

  committed
  732ceb7

  Copy full SHA for 732ceb7

• Update version.txt after 0.16.3 release (#6965)

  Show description for db9aff9
  

  loadams

  authored and

  tjruwase

  committed
  db9aff9

  Copy full SHA for db9aff9

• Precisely track nvme optimizer offload (#6963)

  Show description for 4edeb03
  tjruwase

  committed
  4edeb03

  Copy full SHA for 4edeb03

• Update build_win.bat script to exclue GDS op as it lacks Windows support. (#6971)

  Show description for f00f4ea
  

  loadams

  authored and

  tjruwase

  committed
  f00f4ea

  Copy full SHA for f00f4ea

• Improve overflow handling in ZeRO

  Show description for c3846fa
  tjruwase

  committed
  c3846fa

  Copy full SHA for c3846fa

• Unit test and pydantic configuration

  Show description for 7d56ffa
  tjruwase

  committed
  7d56ffa

  Copy full SHA for 7d56ffa

• Formatting fixes

  Show description for 6ca11ef
  tjruwase

  committed
  6ca11ef

  Copy full SHA for 6ca11ef

• Add CUDA 12.8 support and comment on CUDA 12.7 (#6975)

  Show description for 49f3df8
  

  loadams

  authored and

  tjruwase

  committed
  49f3df8

  Copy full SHA for 49f3df8

• Update torch versions to support 2.6 (#6977)

  Show description for 8364b12
  

  loadams

  authored and

  tjruwase

  committed
  8364b12

  Copy full SHA for 8364b12

• Remove unused symbol

  Show description for ea9b473
  tjruwase

  committed
  ea9b473

  Copy full SHA for ea9b473

• Fix typo

  Show description for d2425a2
  tjruwase

  committed
  d2425a2

  Copy full SHA for d2425a2

• Pydantic fp16 config

  Show description for 7d5be07
  tjruwase

  committed
  7d5be07

  Copy full SHA for 7d5be07

• Fix more typos

  Show description for e8fc098
  tjruwase

  committed
  e8fc098

  Copy full SHA for e8fc098

• Address #4986

  Show description for 2bbb7b4
  tjruwase

  committed
  2bbb7b4

  Copy full SHA for 2bbb7b4

• generalize deepspeed linear and implement it for non cuda systems (#6932)

  Show description for 3ab5e88
  

  3 peoplecommitted
  3ab5e88

  Copy full SHA for 3ab5e88

• Fix typo

  Show description for 271db94
  tjruwase

  committed
  271db94

  Copy full SHA for 271db94

• Update recommended Windows whl building versions (#6983)

  Show description for b1900af
  

  loadams

  authored and

  tjruwase

  committed
  b1900af

  Copy full SHA for b1900af

• Title: Fix setup_env_ranks to Properly Set Environment Variables Instead of Raising Error (#6979)

  Show description for e3d10e5
  

  3 peoplecommitted
  e3d10e5

  Copy full SHA for e3d10e5

• Specify torchvision in nv-ds-chat workflow (prevents errors with torch 2.6) (#6982)

  Show description for b8d8e39
  

  loadams

  authored and

  tjruwase

  committed
  b8d8e39

  Copy full SHA for b8d8e39

• Remove assumption that padding only occurs on last rank (#6974)

  Show description for fde7df1
  

  4 peoplecommitted
  fde7df1

  Copy full SHA for fde7df1

• Use ds-specific module id to avoid conflicts (#6847)

  Show description for b0b0132
  

  tjruwase

  and

  loadams

  committed
  b0b0132

  Copy full SHA for b0b0132

• Update A6000 workflows to use newer docker container - 24.09 vs 24.03 (#6967)

  Show description for 353ab08
  

  loadams

  authored and

  tjruwase

  committed
  353ab08

  Copy full SHA for 353ab08

• Allow NVIDIA Blackwell (#6991)

  Show description for 14189a7
  

  fabiendupont

  authored and

  tjruwase

  committed
  14189a7

  Copy full SHA for 14189a7

• Update GH org references (#6998)

  Show description for 75996f8
  

  tjruwase

  and

  fabiendupont

  committed
  75996f8

  Copy full SHA for 75996f8

• Fix min loss scale

  Show description for b23c545
  tjruwase

  committed
  b23c545

  Copy full SHA for b23c545

• Fix UTs

  Show description for 7cd3a9f
  tjruwase

  committed
  7cd3a9f

  Copy full SHA for 7cd3a9f

• Update CNAME

  Show description for 2c5629e
  

  loadams

  authored and

  tjruwase

  committed
  2c5629e

  Copy full SHA for 2c5629e

• Update CNAME

  Show description for 6b15688
  

  loadams

  authored and

  tjruwase

  committed
  6b15688

  Copy full SHA for 6b15688

• [XPU] max1100 workflow update for docker and softwares (#7003)

  Show description for 3773d83
  

  Liangliang-Ma

  authored and

  tjruwase

  committed
  3773d83

  Copy full SHA for 3773d83

• autotp training(fix dco) (#7004)

  Show description for 64c4b04
  

  inkcherry

  authored and

  tjruwase

  committed
  64c4b04

  Copy full SHA for 64c4b04

• Merge branch 'olruwase/ds_5241' of github.com:microsoft/DeepSpeed into olruwase/ds_5241

  tjruwase

  committed
  5fa2910

  Copy full SHA for 5fa2910

Commits on Feb 7, 2025

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  1f5a672

  Copy full SHA for 1f5a672

• Fix ds-chat CI regression

  tjruwase

  committed
  9882116

  Copy full SHA for 9882116

• Merge branch 'olruwase/ds_7014' of github.com:microsoft/DeepSpeed into olruwase/ds_5241

  tjruwase

  committed
  97d7915

  Copy full SHA for 97d7915

• Fix bug

  tjruwase

  committed
  4a1dd0f

  Copy full SHA for 4a1dd0f

• Avoid naming collision on partition()

  tjruwase

  committed
  0ac4457

  Copy full SHA for 0ac4457

Commits on Feb 8, 2025

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  1597d48

  Copy full SHA for 1597d48

• Use new API

  tjruwase

  committed
  2ae2062

  Copy full SHA for 2ae2062

• Merge branch 'master' into olruwase/ds_7014

  tjruwase

  authored
  9fb73a4

  Copy full SHA for 9fb73a4

• Merge branch 'olruwase/ds_7014' of github.com:microsoft/DeepSpeed into olruwase/ds_5241

  tjruwase

  committed
  26fa8af

  Copy full SHA for 26fa8af

• Merge branch 'olruwase/ds_5241' of github.com:microsoft/DeepSpeed into olruwase/ds_5241

  tjruwase

  committed
  b565d77

  Copy full SHA for b565d77

Commits on Feb 10, 2025

• Merge branch 'master' into olruwase/ds_5241

  loadams

  committed
  d098c32

  Copy full SHA for d098c32

Commits on Feb 13, 2025

• Merge branch 'master' into olruwase/ds_5241

  tjruwase

  authored
  990a5ad

  Copy full SHA for 990a5ad