HandHeld Object Modeling Using Neural Radiance Fields (NeRF)

We used the Nerfstudio opensource library to train the networks on the clouds. the network can be trained locally using cuda on Nvidia GPUs.

Using a video \ set of images of a handheld object, we use image processing to extract the hand from the images and remove it, then we train the result frames on the cloud again using the nerfstudio library.

steps

1. creating image frames out of an input video and resizing images.
2. running colmap on the images for preprocessing the data.
3. training the colmap output data using nerf.
4. creating a rendering result.

notes

our input data was 219 frames of size 720x1280, we resized our frames to 480x853 before doing data preprocessing so our images after hand extraction were of resolution 480x853.

Example of images Before Hand Extraction ( 219 Frames | 720 x 1280)

we exctracted the frames from the video and we got the following sequence.

Example of images After Hand Extraction ( 219 Frames | 480 x 853)

After removing the hands and the background noise, we got the following images.

Colmap Poses

After using Colmap on the sequence, we got the poses in this sequence around the object.

Rendering Result