Code

Add CrowS-Pairs task#70

Code

Open

oskarvanderwal wants to merge 39 commits intobigscience-workshop:mainbigscience-workshop/evaluation:mainfrom oskarvanderwal:mainoskarvanderwal/evaluation:mainCopy head branch name to clipboard

+126

Commits

Commits on Oct 8, 2021

• Add template for CrowS-Pairs task

  oskarvanderwal

  committed
  98f0beb

  Copy full SHA for 98f0beb

• Adjust filename

  oskarvanderwal

  committed
  a251d23

  Copy full SHA for a251d23

• Loading CrowS-Pairs from url in right format

  oskarvanderwal

  committed
  5d1a11d

  Copy full SHA for 5d1a11d

• Tokenize dataset sentences+plan for scoring func.

  oskarvanderwal

  committed
  b8c3cd5

  Copy full SHA for b8c3cd5

• Add link to implementation scores crows-pairs

  oskarvanderwal

  committed
  8bbbd7e

  Copy full SHA for 8bbbd7e

• Implemented most of the eval logic (untested)

  oskarvanderwal

  committed
  03ee5ee

  Copy full SHA for 03ee5ee

• Add scoring sentence as separate function

  oskarvanderwal

  committed
  83d1cef

  Copy full SHA for 83d1cef

• Remove unnecessary todo item

  oskarvanderwal

  committed
  e0e7fbc

  Copy full SHA for e0e7fbc

• Add comment about dataset on huggingface datasets

  oskarvanderwal

  committed
  4096287

  Copy full SHA for 4096287

Commits on Oct 19, 2021

• Update crowspairs.py

  Show description for 1dce412
  oskarvanderwal

  authored
  1dce412

  Copy full SHA for 1dce412

• Update crowspairs.py

  Show description for aaf2bc1
  oskarvanderwal

  authored
  aaf2bc1

  Copy full SHA for aaf2bc1

Commits on Oct 27, 2021

• Simplified the evaluation logic

  ozzypossie

  committed
  eeabdc8

  Copy full SHA for eeabdc8

• Remove print statement

  ozzypossie

  committed
  d247e98

  Copy full SHA for d247e98

• Test

  ozzypossie

  committed
  b4bb270

  Copy full SHA for b4bb270

• Assume not start and stop token

  ozzypossie

  committed
  6a664bb

  Copy full SHA for 6a664bb

• Remove pair-score metric

  ozzypossie

  committed
  43d50ae

  Copy full SHA for 43d50ae

• Small bug fix

  ozzypossie

  committed
  801dd06

  Copy full SHA for 801dd06

• Remove print statement

  ozzypossie

  committed
  31c941c

  Copy full SHA for 31c941c

• Add confidence metric

  ozzypossie

  committed
  a05b622

  Copy full SHA for a05b622

• Add average model confidence to metric

  ozzypossie

  committed
  35a9e18

  Copy full SHA for 35a9e18

• Typo

  ozzypossie

  committed
  fcdd28e

  Copy full SHA for fcdd28e

• Comment out confidence metric

  ozzypossie

  committed
  3d8416f

  Copy full SHA for 3d8416f

Commits on Nov 3, 2021

• Change sentence score to perplexity

  oskarvanderwal

  authored
  919dc1f

  Copy full SHA for 919dc1f

• Update crowspairs.py

  oskarvanderwal

  authored
  7df4df2

  Copy full SHA for 7df4df2

• Update crowspairs.py

  oskarvanderwal

  authored
  01db11d

  Copy full SHA for 01db11d

• Update crowspairs.py

  oskarvanderwal

  authored
  c0a9fea

  Copy full SHA for c0a9fea

• Update crowspairs.py

  oskarvanderwal

  authored
  d1d50a4

  Copy full SHA for d1d50a4

• Update crowspairs.py

  oskarvanderwal

  authored
  ecfa84a

  Copy full SHA for ecfa84a

• Update crowspairs.py

  oskarvanderwal

  authored
  c979e93

  Copy full SHA for c979e93

• Update crowspairs.py

  oskarvanderwal

  authored
  0521b24

  Copy full SHA for 0521b24

• Update crowspairs.py

  oskarvanderwal

  authored
  9a83d92

  Copy full SHA for 9a83d92

• Update crowspairs.py

  oskarvanderwal

  authored
  904233e

  Copy full SHA for 904233e

• Update crowspairs.py

  oskarvanderwal

  authored
  7b01be8

  Copy full SHA for 7b01be8

• Update crowspairs.py

  oskarvanderwal

  authored
  f2b959a

  Copy full SHA for f2b959a

• Update crowspairs.py

  oskarvanderwal

  authored
  1805346

  Copy full SHA for 1805346

• Update crowspairs.py

  oskarvanderwal

  authored
  cd57052

  Copy full SHA for cd57052

Commits on Nov 10, 2021

• Cleaning code

  oskarvanderwal

  committed
  82ef242

  Copy full SHA for 82ef242

• Run make quality

  oskarvanderwal

  committed
  3e0e41c

  Copy full SHA for 3e0e41c

• Fix comment

  oskarvanderwal

  committed
  9f98a89

  Copy full SHA for 9f98a89