Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error while training Packnet-Sfm on EuRoC #36

Open
yhabib29 opened this issue Apr 3, 2023 · 0 comments
Open

Error while training Packnet-Sfm on EuRoC #36

yhabib29 opened this issue Apr 3, 2023 · 0 comments

Comments

@yhabib29
Copy link

yhabib29 commented Apr 3, 2023

Hi,

I could not train Packnet-Sfm network on this repo, I get an error when the network tries to infer pose:

File "/workspace/vidar/vidar/arch/models/depth/SelfSupervisedModel.py", line 76, in forward
    pose = self.compute_pose(rgb, self.networks['pose'], tgt=0, invert=True)
  File "/workspace/vidar/vidar/arch/models/BaseModel.py", line 39, in compute_pose
    for idx in ctx}
  File "/workspace/vidar/vidar/arch/models/BaseModel.py", line 39, in <dictcomp>
    for idx in ctx}
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'invert'

I used the following configuration:

wrapper:
    recipe: wrapper|default
    max_epochs: 1
arch:
    model:
        file: depth/SelfSupervisedModel
    networks:
        depth:
            recipe: networks/packnet|default
            depth_range: [0.1,20.0]
        pose:
            recipe: networks/conv_pose_net|default
    losses:
        reprojection:
            recipe: losses/reprojection|default
        smoothness:
            recipe: losses/smoothness|default
evaluation:
    depth:
        recipe: evaluation/depth|euroc_resize
optimizers:
    depth:
        recipe: optimizers|adam_20_05
    pose:
        recipe: optimizers|adam_20_05
datasets:
    train:
        recipe: datasets/euroc|train_selfsup
    validation:
        recipe: datasets/euroc|validation

Indeed, the compute_pose function in BaseModel.py is not meant for Packnet module.
In order to make it work, I had to change the following lines in SelfSupervisedModel.py based on the previous repo of packnet-sfm:

index 68c360b..2656cb1 100644
--- a/vidar/arch/models/depth/SelfSupervisedModel.py
+++ b/vidar/arch/models/depth/SelfSupervisedModel.py
@@ -7,6 +7,7 @@ from vidar.arch.models.BaseModel import BaseModel
 from vidar.arch.models.utils import make_rgb_scales, create_cameras
 from vidar.utils.data import get_from_dict
 from vidar.utils.config import cfg_has
+from vidar.geometry.pose import Pose
 
 
 class SelfSupervisedModel(BaseModel, ABC):
@@ -72,7 +73,11 @@ class SelfSupervisedModel(BaseModel, ABC):
             assert 'pose' in batch, 'You need input pose'
             pose = batch['pose']
         elif 'pose' in self.networks:
-            pose = self.compute_pose(rgb, self.networks['pose'], tgt=0, invert=True)
+            # pose = self.compute_pose(rgb, self.networks['pose'], tgt=0, invert=True)
+            # pose = self.networks['pose'](rgb[0], [rgb[key] for key in rgb.keys() if key != 0])
+            pose_vec = self.networks['pose'](rgb[0], [rgb[-1], rgb[1]])
+            pose = {-1: Pose.from_vec(pose_vec[0, 0].unsqueeze(0), 'euler'),
+                     1: Pose.from_vec(pose_vec[0, 1].unsqueeze(0), 'euler')}
             predictions['pose'] = pose
         else:
             pose = None

The inference worked so I tried to train the network on EuRoC/V1_01 easy scene.
I wanted to overfit the network just to check if the pipeline works.
However, the results are really bad.
How am I suppose to train Packnet-Sfm network on EuRoC using this repo ?

Thanks for your help !

@yhabib29 yhabib29 changed the title Error while training Packnet-Sfm Error while training Packnet-Sfm on EuRoC Apr 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant