🚅⚡ Test Training with PyTorch Lightning #930

mberr · 2022-05-19T14:17:48Z

This PR adds tests for the PyTorch Lightning integration.

Issues

post_parameter_update is not applied, cf. failing pipeline for KG2E; fixed in 9052412
RuntimeError for CompGCN, RESCAL and TransH

src/pykeen/contrib/lightning.py:288: in lit_pipeline
    trainer.fit(model=lit)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:768: in fit
    self._call_and_handle_interrupt(
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:721: in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:809: in _fit_impl
    results = self._run(model, ckpt_path=self.ckpt_path)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1234: in _run
    results = self._run_stage()
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1321: in _run_stage
    return self._run_train()
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1351: in _run_train
    self.fit_loop.run()
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/base.py:204: in run
    self.advance(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/fit_loop.py:269: in advance
    self._outputs = self.epoch_loop.run(self._data_fetcher)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/base.py:204: in run
    self.advance(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py:208: in advance
    batch_output = self.batch_loop.run(batch, batch_idx)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/base.py:204: in run
    self.advance(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py:88: in advance
    outputs = self.optimizer_loop.run(split_batch, optimizers, batch_idx)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/base.py:204: in run
    self.advance(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py:203: in advance
    result = self._run_optimization(
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py:256: in _run_optimization
    self._optimizer_step(optimizer, opt_idx, batch_idx, closure)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py:369: in _optimizer_step
    self.trainer._call_lightning_module_hook(
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1593: in _call_lightning_module_hook
    output = fn(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py:1625: in optimizer_step
    optimizer.step(closure=optimizer_closure)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/core/optimizer.py:168: in step
    step_output = self._strategy.optimizer_step(self._optimizer, self._optimizer_idx, closure, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/strategies/strategy.py:193: in optimizer_step
    return self.precision_plugin.optimizer_step(model, optimizer, opt_idx, closure, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py:155: in optimizer_step
    return optimizer.step(closure=closure, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/torch/optim/optimizer.py:88: in wrapper
    return func(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/torch/autograd/grad_mode.py:27: in decorate_context
    return func(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/torch/optim/adam.py:100: in step
    loss = closure()
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py:140: in _wrap_closure
    closure_result = closure()
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py:148: in __call__
    self._result = self.closure(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py:143: in closure
    self._backward_fn(step_output.closure_loss)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py:311: in backward_fn
    self.trainer._call_strategy_hook("backward", loss, optimizer, opt_idx)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1763: in _call_strategy_hook
    output = fn(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/strategies/strategy.py:168: in backward
    self.precision_plugin.backward(self.lightning_module, closure_loss, *args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py:80: in backward
    model.backward(closure_loss, optimizer, *args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py:1370: in backward
    loss.backward(*args, **kwargs)
venv/venv-gpu/lib/python3.8/site-packages/torch/_tensor.py:363: in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)

trigger ci

cthoyt · 2022-05-24T14:51:32Z

src/pykeen/contrib/lightning.py

+    """
+    lit = lit_module_resolver.make(training_loop, pos_kwargs=training_loop_kwargs)
+    trainer = pytorch_lightning.Trainer(**(trainer_kwargs or {}))
+    trainer.fit(model=lit)


what's the point of this? it doesn't return anything

It modifies the model's weights in-place

maybe add a bit of extra description then? for poor fools like me ;)

cthoyt · 2022-05-24T14:52:38Z

tests/test_lightning.py

+        training_loop_kwargs=dict(
+            model=model,
+            # use a small configuration for testing
+            # TODO: this does not properly work for all models


which ones doesn't it work for?

1d00631 - this comment was outdated.

trigger ci

cthoyt · 2022-05-24T15:12:29Z

@PyKEEN-bot test

mberr added 3 commits May 19, 2022 16:17

add tests for training with PyTorch Lightning

2fe8478

trigger ci

fix imports

fca2153

trigger ci

parametrize test

8628f19

trigger ci

cthoyt reviewed May 20, 2022

View reviewed changes

tests/test_lightning.py Outdated Show resolved Hide resolved

cthoyt and others added 2 commits May 20, 2022 10:38

Make lightning always available in tests

926fc86

Trigger CI

move code from tests to module

3fb4d09

trigger ci

mberr added 4 commits May 20, 2022 12:07

fix some tests

2b24d6f

trigger ci

Add Lighting badge

c93219a

fix inductive models missing mode

fbe9cca

trigger ci

add todo comment

02a9174

trigger ci

fix imports

4c3a4b8

trigger ci

mberr mentioned this pull request May 21, 2022

Add PyKEEN to ecosystem-ci Lightning-AI/ecosystem-ci#50

Draft

5 tasks

mberr added 5 commits May 23, 2022 18:16

Merge branch 'master' into pyl-tests

fec9a6b

fix readme

31a937b

trigger ci

fix missing call to post_parameter_update

9052412

trigger ci

add docstr-coverage ignore for inherited method

2e2e724

trigger ci

skip tests if lightning is missing

ec36a93

trigger ci

mberr marked this pull request as ready for review May 24, 2022 14:45

mberr requested a review from cthoyt May 24, 2022 14:46

mberr changed the title ~~Test Training with PyTorch Lightning~~ 🚅⚡ Test Training with PyTorch Lightning May 24, 2022

cthoyt reviewed May 24, 2022

View reviewed changes

mberr added 2 commits May 24, 2022 16:55

add & remove comment

1d00631

trigger ci

update docstring

ca2f266

trigger ci

cthoyt approved these changes May 24, 2022

View reviewed changes

mberr enabled auto-merge (squash) May 24, 2022 15:02

Merge branch 'master' into pyl-tests

02274b7

Trigger CI

baa75fc

mberr merged commit 6d6e374 into master May 24, 2022

mberr deleted the pyl-tests branch May 24, 2022 15:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🚅⚡ Test Training with PyTorch Lightning #930

🚅⚡ Test Training with PyTorch Lightning #930

mberr commented May 19, 2022 •

edited

Loading

cthoyt commented May 20, 2022

mberr commented May 20, 2022

cthoyt May 24, 2022

mberr May 24, 2022

cthoyt May 24, 2022

mberr May 24, 2022

cthoyt May 24, 2022

mberr May 24, 2022

cthoyt commented May 24, 2022

🚅⚡ Test Training with PyTorch Lightning #930

🚅⚡ Test Training with PyTorch Lightning #930

Conversation

mberr commented May 19, 2022 • edited Loading

Issues

Related

cthoyt commented May 20, 2022

mberr commented May 20, 2022

cthoyt May 24, 2022

Choose a reason for hiding this comment

mberr May 24, 2022

Choose a reason for hiding this comment

cthoyt May 24, 2022

Choose a reason for hiding this comment

mberr May 24, 2022

Choose a reason for hiding this comment

cthoyt May 24, 2022

Choose a reason for hiding this comment

mberr May 24, 2022

Choose a reason for hiding this comment

cthoyt commented May 24, 2022

mberr commented May 19, 2022 •

edited

Loading