Code

Preprocessing#39

Code

Closed

sangHa0411 wants to merge 119 commits intomainboostcampaitech2/final-project-level3-nlp-02:mainfrom preprocessingboostcampaitech2/final-project-level3-nlp-02:preprocessingCopy head branch name to clipboard

+3,959-457

Commits

Commits on Nov 30, 2021

• Making Preprocessing paper

  sangHa0411

  committed
  4699e24

  Copy full SHA for 4699e24

Commits on Dec 1, 2021

• upload preprocessor

  sangHa0411

  committed
  c37dd24

  Copy full SHA for c37dd24

• Making tokenizer optimization

  sangHa0411

  committed
  008cb27

  Copy full SHA for 008cb27

• remove filter

  sangHa0411

  committed
  9516a61

  Copy full SHA for 9516a61

• debug preprocessor

  sangHa0411

  committed
  b3a83af

  Copy full SHA for b3a83af

• Merge branch 'preprocessing_shp' into tokenizer_shp

  sangHa0411

  committed
  24962fa

  Copy full SHA for 24962fa

• Update Tokenizer optimization progress

  sangHa0411

  committed
  2796524

  Copy full SHA for 2796524

• Upload extra characters.csv

  sangHa0411

  committed
  86b3602

  Copy full SHA for 86b3602

• Update tokenizer optimization

  sangHa0411

  committed
  111682f

  Copy full SHA for 111682f

Commits on Dec 2, 2021

• Upload Bart tokenizer optimization

  sangHa0411

  committed
  d880092

  Copy full SHA for d880092

• Update gitignore

  sangHa0411

  committed
  1c0adc0

  Copy full SHA for 1c0adc0

• Debug BartTokenizer

  sangHa0411

  committed
  8d53465

  Copy full SHA for 8d53465

• Update tokenizer.py

  sangHa0411

  committed
  c031f86

  Copy full SHA for c031f86

• merge dataloader

  sangHa0411

  committed
  3c61265

  Copy full SHA for 3c61265

Commits on Dec 3, 2021

• update custom model

  sangHa0411

  committed
  f7e1ed4

  Copy full SHA for f7e1ed4

• longformerbart, Interim checkup

  changyong93

  committed
  b4d1c8a

  Copy full SHA for b4d1c8a

Commits on Dec 4, 2021

• ex1

  abbymark

  committed
  3eed6df

  Copy full SHA for 3eed6df

• Merge branch 'baseline_dataloader' into amc_private

  abbymark

  committed
  0d65645

  Copy full SHA for 0d65645

• backup for gpu problem

  abbymark

  committed
  28a3829

  Copy full SHA for 28a3829

• title_generator

  abbymark

  committed
  b3e8fee

  Copy full SHA for b3e8fee

Commits on Dec 5, 2021

• benchmark setup

  abbymark

  committed
  252737e

  Copy full SHA for 252737e

• dynamic quantization

  abbymark

  committed
  89196ea

  Copy full SHA for 89196ea

• quantization

  abbymark

  committed
  5bcac90

  Copy full SHA for 5bcac90

• quantization continue

  abbymark

  committed
  f411bc5

  Copy full SHA for f411bc5

• half quantization fix

  abbymark

  committed
  950fe8a

  Copy full SHA for 950fe8a

• feat: add R-drop

  Yebin46

  committed
  efb8b7d

  Copy full SHA for efb8b7d

Commits on Dec 6, 2021

• moving model python file to Model directory

  sangHa0411

  committed
  f247319

  Copy full SHA for f247319

• change name longformer bart

  sangHa0411

  committed
  1da4502

  Copy full SHA for 1da4502

• change folder name(Model->model)

  changyong93

  committed
  6f4ad29

  Copy full SHA for 6f4ad29

• fix invalid inputs

  Yebin46

  committed
  1b236e2

  Copy full SHA for 1b236e2

• add post_init()

  changyong93

  committed
  7646801

  Copy full SHA for 7646801

• Update modeling making for longformerbart model

  sangHa0411

  committed
  6423a96

  Copy full SHA for 6423a96

• chore: modify arguments for fast evaluation

  presto105

  committed
  69f8f8a

  Copy full SHA for 69f8f8a

• chore: modify rouge variables

  j961224

  authored
  1842bd4

  Copy full SHA for 1842bd4

• Implement longformerbart

  changyong93

  committed
  fdf8f4d

  Copy full SHA for fdf8f4d

• Implement longformerbart_f

  changyong93

  committed
  c90b4d8

  Copy full SHA for c90b4d8

• fix: nll loss for r-drop

  Yebin46

  committed
  f45b9cf

  Copy full SHA for f45b9cf

• merge baseline branch

  Yebin46

  committed
  ea5cc9a

  Copy full SHA for ea5cc9a

Commits on Dec 7, 2021

• chore: add annotation at rouge.py

  presto105

  committed
  a637c02

  Copy full SHA for a637c02

• chore: merge and add annotation at rouge.py

  presto105

  committed
  d3706b9

  Copy full SHA for d3706b9

• chore: modify start of token and default pretrained model

  presto105

  committed
  b40b99b

  Copy full SHA for b40b99b

• chore: modify args, dataset suffle and dataset extract relative ratio

  presto105

  committed
  08cf53d

  Copy full SHA for 08cf53d

• kd continue

  abbymark

  committed
  be94a63

  Copy full SHA for be94a63

• kd continued

  abbymark

  committed
  04ab2e4

  Copy full SHA for 04ab2e4

• basic kd

  abbymark

  committed
  947426c

  Copy full SHA for 947426c

• merge with main

  abbymark

  committed
  9df9d1d

  Copy full SHA for 9df9d1d

• cleaning test

  abbymark

  committed
  4564c48

  Copy full SHA for 4564c48

• cleaning

  abbymark

  committed
  0c7a3bc

  Copy full SHA for 0c7a3bc

• Merge branch 'baseline' into dev_r-drop

  Yebin46

  committed
  3da0bb9

  Copy full SHA for 3da0bb9

• fix: label concat

  Yebin46

  committed
  42d7402

  Copy full SHA for 42d7402

Commits on Dec 8, 2021

• test: reproducibility test

  presto105

  committed
  98b40ba

  Copy full SHA for 98b40ba

• feat: code refactoring, verify reproducibility

  presto105

  committed
  9ab3112

  Copy full SHA for 9ab3112

• chore: modify running code

  presto105

  committed
  aacffe4

  Copy full SHA for aacffe4

• 기타 오류 수정

  abbymark

  committed
  e774b8e

  Copy full SHA for e774b8e

Commits on Dec 9, 2021

• 오류 수정

  abbymark

  committed
  b04a9cf

  Copy full SHA for b04a9cf

• dynamic quantization test fixed

  abbymark

  committed
  b7447bc

  Copy full SHA for b7447bc

• add kobigbird_bart model

  gistarrr

  committed
  a6b1de6

  Copy full SHA for a6b1de6

Commits on Dec 10, 2021

• Confirm longformerbart model operating

  sangHa0411

  committed
  cda6666

  Copy full SHA for cda6666

• add bart decoder

  gistarrr

  committed
  cb13906

  Copy full SHA for cb13906

Commits on Dec 11, 2021

• tiny distillation

  abbymark

  committed
  ab8a928

  Copy full SHA for ab8a928

• Upload: Kobigbirdbart train v0

  j961224

  authored
  8d83dd3

  Copy full SHA for 8d83dd3

• Update: add EncoderDecoderModel class

  j961224

  authored
  cd9665f

  Copy full SHA for cd9665f

• Update: add doc_type_ids

  j961224

  authored
  f996091

  Copy full SHA for f996091

• Update: doc_type_ids args of EncoderDecoderModel

  j961224

  authored
  724deea

  Copy full SHA for 724deea

• Update: use_kobigbirdbart args

  j961224

  authored
  06a6d12

  Copy full SHA for 06a6d12

• fix gitignore

  gistarrr

  committed
  e9eb744

  Copy full SHA for e9eb744

• code refactoring and update precessor

  changyong93

  committed
  54bc21e

  Copy full SHA for 54bc21e

• complete train.py

  gistarrr

  committed
  d74c29a

  Copy full SHA for d74c29a

• Confirm Operating infilling bart model pretraining

  sangHa0411

  committed
  12cad41

  Copy full SHA for 12cad41

Commits on Dec 12, 2021

• Update: if no doc_type_ids

  j961224

  authored
  5908022

  Copy full SHA for 5908022

• modeling

  abbymark

  committed
  19d33ec

  Copy full SHA for 19d33ec

• Implementation of Infilling

  changyong93

  committed
  6a6c2ae

  Copy full SHA for 6a6c2ae

Commits on Dec 13, 2021

• Implementation of TAPT(text infilling)

  changyong93

  committed
  a0d48c6

  Copy full SHA for a0d48c6

• refine tiny distillation

  abbymark

  committed
  b83cc0e

  Copy full SHA for b83cc0e

• 기타

  abbymark

  committed
  fe53aa1

  Copy full SHA for fe53aa1

• Merge branch 'dev_optimizer' into optimizer_merge

  abbymark

  committed
  abc5cc1

  Copy full SHA for abc5cc1

• quantization 수정

  abbymark

  committed
  a259c6b

  Copy full SHA for a259c6b

• moved for merge

  abbymark

  committed
  4755278

  Copy full SHA for 4755278

• Merge branch 'dev_optimizer' into optimizer_merge

  abbymark

  committed
  f96d0b5

  Copy full SHA for f96d0b5

• test

  abbymark

  committed
  8ba74dc

  Copy full SHA for 8ba74dc

• Merge branch 'dev_optimizer' into optimizer_merge

  abbymark

  committed
  093cd06

  Copy full SHA for 093cd06

• Merge branch 'dev' into optimizer_merge

  abbymark

  committed
  71e0bb0

  Copy full SHA for 71e0bb0

• distillation method added to train

  abbymark

  committed
  ee4ae01

  Copy full SHA for ee4ae01

• argument fix

  abbymark

  committed
  3e91aef

  Copy full SHA for 3e91aef

• device fix

  abbymark

  committed
  0a1d39c

  Copy full SHA for 0a1d39c

• dataset change

  abbymark

  committed
  d6c3890

  Copy full SHA for d6c3890

• Update: implementation kobigbirdbart model

  gistarrr

  committed
  f26c378

  Copy full SHA for f26c378

• Merge branch 'model_bbb' of https://github.com/boostcampaitech2/final-project-level3-nlp-02 into model_bbb

  gistarrr

  committed
  521c1e4

  Copy full SHA for 521c1e4

• optimization dataset

  abbymark

  committed
  3944600

  Copy full SHA for 3944600

• Update: fix modeling_kobigbird_bart.py

  gistarrr

  committed
  a56ae37

  Copy full SHA for a56ae37

• update tapt

  changyong93

  committed
  49b5bb8

  Copy full SHA for 49b5bb8

• delete files

  changyong93

  committed
  3771d2d

  Copy full SHA for 3771d2d

• update tapt

  changyong93

  committed
  44a22a1

  Copy full SHA for 44a22a1

• Making using last checkpoint

  sangHa0411

  committed
  83886f6

  Copy full SHA for 83886f6

Commits on Dec 14, 2021

• Update:fix predict.py

  gistarrr

  committed
  e360f39

  Copy full SHA for e360f39

• Using data argument for pad_token_id & max_position_embeddings

  gistarrr

  committed
  d8bea23

  Copy full SHA for d8bea23

• Merge pull request #36 from boostcampaitech2/model_bbb

  Show description for 3322d70
  changyong93

  authored
  3322d70

  Copy full SHA for 3322d70

• relocate tokenizer.py

  changyong93

  committed
  659dc0c

  Copy full SHA for 659dc0c

• teacher forcing

  changyong93

  committed
  35c6ed7

  Copy full SHA for 35c6ed7

• merge dev and tapt and teacherforcing and bigbirdbart

  changyong93

  committed
  97c25e2

  Copy full SHA for 97c25e2

• temp

  Yebin46

  committed
  7e15660

  Copy full SHA for 7e15660

• 기타파일

  abbymark

  committed
  a455252

  Copy full SHA for a455252

• sh파일

  abbymark

  committed
  bb51402

  Copy full SHA for bb51402

• debug kobigbirdbart model

  gistarrr

  committed
  63d971a

  Copy full SHA for 63d971a

• fix error

  changyong93

  committed
  f1b58ce

  Copy full SHA for f1b58ce

• push pulled data

  changyong93

  committed
  23a97e1

  Copy full SHA for 23a97e1

Commits on Dec 15, 2021

• modify gitignore

  Yebin46

  committed
  1d471c3

  Copy full SHA for 1d471c3

• merge dev

  Yebin46

  committed
  1756fab

  Copy full SHA for 1756fab

• merge from dev

  gistarrr

  committed
  69e74cc

  Copy full SHA for 69e74cc

• merge from optimizer and serving

  gistarrr

  committed
  da6e270

  Copy full SHA for da6e270

• fix train.py and data_arg

  gistarrr

  committed
  07e7843

  Copy full SHA for 07e7843

• fix calculate relative eval steps

  gistarrr

  committed
  04a4bf5

  Copy full SHA for 04a4bf5

• fix trainer.py label input

  gistarrr

  committed
  5ff1b4b

  Copy full SHA for 5ff1b4b

• fix add_padding function

  gistarrr

  committed
  6c09855

  Copy full SHA for 6c09855

• code refactoring

  gistarrr

  committed
  8a536d2

  Copy full SHA for 8a536d2

• error fix bigbird encdoer model

  gistarrr

  committed
  0f2ae55

  Copy full SHA for 0f2ae55

Commits on Dec 16, 2021

• fix generation inputs

  Yebin46

  committed
  9a5aaa8

  Copy full SHA for 9a5aaa8

• Merge branch 'dev' of https://github.com/boostcampaitech2/final-project-level3-nlp-02 into dev

  Yebin46

  committed
  77cbdfb

  Copy full SHA for 77cbdfb

• Update preprocessing

  sangHa0411

  committed
  2333385

  Copy full SHA for 2333385