Code

[WIP] Hack my way to get OPT running#301

Code

Draft

thomasw21 wants to merge 25 commits intomainbigscience-workshop/Megatron-DeepSpeed:mainfrom thomas/optbigscience-workshop/Megatron-DeepSpeed:thomas/optCopy head branch name to clipboard

+497-55

Commits

Commits on Jul 4, 2022

• That's what I'm hoping is all I need

  thomasw21

  committed
  ce8d5e1

  Copy full SHA for ce8d5e1

• relu to work

  thomasw21

  committed
  b21d773

  Copy full SHA for b21d773

• Run inference for opt

  thomasw21

  committed
  43a9ab4

  Copy full SHA for 43a9ab4

Commits on Jul 5, 2022

• Test

  thomasw21

  committed
  8e095da

  Copy full SHA for 8e095da

• I think OPT doesn't have shared vocabulary embedding

  thomasw21

  committed
  e742a3f

  Copy full SHA for e742a3f

• Revert "I think OPT doesn't have shared vocabulary embedding"

  Show description for 55d1a4b
  thomasw21

  committed
  55d1a4b

  Copy full SHA for 55d1a4b

• Maybe this is what's wrong

  thomasw21

  committed
  b8090b7

  Copy full SHA for b8090b7

• Update model loading

  thomasw21

  committed
  dff9766

  Copy full SHA for dff9766

• WTF

  thomasw21

  committed
  01e269d

  Copy full SHA for 01e269d

• Woops

  thomasw21

  committed
  17a075d

  Copy full SHA for 17a075d

• Position offset

  thomasw21

  committed
  d45af14

  Copy full SHA for d45af14

• Add bs eval harness code

  thomasw21

  committed
  73a7947

  Copy full SHA for 73a7947

• Missing import

  thomasw21

  committed
  f34841c

  Copy full SHA for f34841c

• Update position ids

  thomasw21

  committed
  1423bf9

  Copy full SHA for 1423bf9

• That pathing breaks everything

  thomasw21

  committed
  fae49ca

  Copy full SHA for fae49ca

• add offloading possitility

  thomasw21

  committed
  6a326af

  Copy full SHA for 6a326af

• Black list all tasks that require greedy_until

  thomasw21

  committed
  b2caf4d

  Copy full SHA for b2caf4d

Commits on Jul 6, 2022

• Woops

  thomasw21

  committed
  48892b9

  Copy full SHA for 48892b9

• Disable CarbonEmissionTracker

  thomasw21

  committed
  1476691

  Copy full SHA for 1476691

• Enable CarbonEmissionTracker

  thomasw21

  committed
  5868d29

  Copy full SHA for 5868d29

• Woops

  thomasw21

  committed
  94ef27a

  Copy full SHA for 94ef27a

Commits on Jul 7, 2022

• Be very careful of random states

  thomasw21

  committed
  f713ff0

  Copy full SHA for f713ff0

• Woops

  thomasw21

  committed
  fab9fb9

  Copy full SHA for fab9fb9

• Woops

  thomasw21

  committed
  23d3f4c

  Copy full SHA for 23d3f4c

• Remove codecarbon

  thomasw21

  committed
  c6f9c02

  Copy full SHA for c6f9c02