Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nächste Runde von Anpassungen #2

Draft
wants to merge 32 commits into
base: master
Choose a base branch
from
Draft

Nächste Runde von Anpassungen #2

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
a34629f
how to setup a venv
ulf1 Nov 14, 2020
ec746d5
relax version requirements
ulf1 Nov 14, 2020
68e8085
how to download data
ulf1 Nov 14, 2020
2babd51
dvc requires specific version range for networkx
ulf1 Nov 14, 2020
ec1bfaf
Unit tests with Github Actions
ulf1 Nov 14, 2020
a2d5905
install with extra_require
ulf1 Nov 14, 2020
94f23ea
split up stages
ulf1 Nov 14, 2020
d11455d
DVC configuration instructions
ulf1 Nov 16, 2020
5a2b28a
store model in a subfolder v1
ulf1 Nov 16, 2020
e2974e5
downgrade to 3.6
ulf1 Nov 17, 2020
06060f3
install python pkgs via requirements.txt to make use of the --use-fea…
ulf1 Nov 17, 2020
8fb449c
set odo.dwds.de as new SSH endpoint
ulf1 Nov 17, 2020
499bcfc
hash updated
ulf1 Nov 17, 2020
d30f624
comments about the packages' purpose
ulf1 Nov 17, 2020
9934160
Hinweis
ulf1 Nov 17, 2020
3597e19
DVC remote changed
ulf1 Nov 17, 2020
601b83e
version downgrade
ulf1 Nov 17, 2020
3935713
shebang was missing
ulf1 Nov 28, 2020
9ca61d8
path to package corrected
ulf1 Nov 28, 2020
a44049f
main.py as pkg script
ulf1 Nov 28, 2020
b06d98b
consult the requirements file for dependencies
ulf1 Nov 29, 2020
23909ee
readme updated
ulf1 Nov 29, 2020
111b24b
systests scripts refactored for a wider range of shells
ulf1 Nov 29, 2020
c2170f6
move global vars to systests folder
ulf1 Nov 29, 2020
8197352
python coding examples
ulf1 Nov 29, 2020
9b9c1b6
distributed examples evenly
ulf1 Nov 29, 2020
38de6cc
add missing deps to setup.py
ulf1 Nov 29, 2020
efc395e
avoid ray>=1
ulf1 Nov 29, 2020
8dcf03b
use ray 0.8
ulf1 Nov 29, 2020
0cc3210
set zip_safe=True
ulf1 Nov 29, 2020
b5b65ad
try ray==1.0.0
ulf1 Nov 29, 2020
fa0d008
install reqs files with resolver
ulf1 Nov 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,20 @@ Dependency Parsers](https://www.aclweb.org/anthology/P19-1012) (see [acl2019 bra

## Usage

### Install virtual env

```sh
python3.7 -m venv .venv
source .venv/bin/activate
python setup.py install
```

### Download training data and serialized model
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DVC benötigt SSH account.

  • Lösung 1: public www ordner auf odo
  • Lösung 2: persistent ID (Datenpublikation)


```sh
dvc pull
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dvc pull => ERROR: failed to pull data from the cloud - config file error: no remote specified. Create a default remote with dvc remote add -d <remote name> <remote url>

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wo liegen die Daten? Sind die Daten "public"? Wenn ein Password oder Token benötigt muss in der README.md stehen wer die Kontaktperson ist.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welche Daten meinst du genau? Der Parser ist im allgmeinen unabhängig von den Daten - müssen nur im CoNLL Format vorliegen. Für die aktuellen Modelle haben wir die Hamburger Treebank genutzt: https://corpora.uni-hamburg.de/hzsk/de/islandora/object/treebank:hdt

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warum gibt es den Ordner .dvc? Zum Beispiel in https://github.com/zentrum-lexikographie/IMSnPars/tree/master/data/hdt sind lauter DVC Referenzen auf conllu Dateien, die eigentlich auf einem Remote Server herumliegen müssten

```

### Transition-based parser

Training a new model:
Expand Down
20 changes: 11 additions & 9 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,14 @@
author_email='[email protected]',
packages=find_packages(exclude=['tests']),
install_requires=[
'dvc==1.6.6',
'dynet @ git+https://github.com/clab/dynet@7c533e#egg=dynet',
'networkx==2.4',
'conllu==3.1.1',
'Click==7.1.2',
'psutil==5.7.2',
'ray==0.8.7',
'boltons==20.2.1'
'dvc>=1.6.6',
'dynet>=2.0.0',
'networkx>=2.4',
'conllu>=3.1.1',
'Click>=7.1.2',
'psutil>=5.7.2',
'ray>=0.8.7',
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gibt es einen großen Unterschied (breaking change) zwischen ray=0.* und ray 1.*

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

da kennt sich @gremid besser aus

'boltons>=20.2.1'
],
extras_require={
'test': [
Expand All @@ -30,4 +30,6 @@
'console_scripts': [
'ims-nparser = imsnpars.cli:main',
],
})
},
python_requires=">=3.7"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Muss das wirklich 3.7 sein? Funktioniert auch 3.6?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ich glaube das hat sich aus dem original Repository ergeben und wurde übernommen. Wenn du Tests für ältere Versionen durchlaufen lässt, können wir das gerne abändern.

Copy link
Author

@ulf1 ulf1 Nov 15, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bisher steht im Fork und im Orginal Repo die Version 3.7 nur in der Readme.md

)