Questions on building depth image into a point cloud

[stevensu1838]

Hi all,
Thank you so much for your great contribution. I've got several questions on building depth image from ROS into a point cloud in Unity.
First, do I have to use a kinect 2 with your package? I have only a kinect 1 and Intel real sense 435i. Do they do the same work with your package.
Second, I am clear how you send your depth image data from ROS. But can you please show me how you convert them into point cloud at the unity side? Can you please kindly show me which of these unity scripts you used to accomplish it.
Third, it shows that you've upgraded your ros reality project based on ROS#. How ever I don't find this package on github. The one you linked is not much different from ROS# package. Can you please show me where you updated project is on github. Looking forward to your reply. Thanks a million.

BR,
Steven

[dwhit]

Hi Steven,

Here are the answers to your questions:

Q. How do we convert the depth and color data to a point cloud?
A. Our package converts the depth and color images to a point cloud in a geometry shader, which can be found in Assets/Shaders/DepthGeometryPointSprites.shader

Q. Do you have to use a Kinect 2?
A. Our package is designed with the Kinect 2 in mind. The camera intrinsics of the Kinect 1 or the RealSense will be different, so parameters would need to be changed in the shader.

Q. Is there a new ROS# based package?
A. Yes there is, but it is not publically on GitHub yet. Our new package combines the communication functionality of ROS# with the control/visualization features of ROS Reality. If you would like to switch to ROS#, I keep a fork on my GitHub that works with the HoloLens (link)

[stevensu1838]

Hey buddy,
Thanks a lot for your reply. I do want to switch to ROS#. But I want to use your method converting depth image to point cloud. Can you please show me some directions on how to use your way of doing it together with ROS#? Thanks a lot

[dwhit]

Hi Steven,

I'm defending this week, so I don't have time right now, but in May I'll be updating/merging ros reality and ros-sharp so that the point cloud conversion works with the ros-sharp networking stack.

[stevensu1838]

Hi buddy,
How did you defending go?

[stevensu1838]

Hi buddy,
I hope your defending went well. Do you think you still could do updating/merging ros reality and ros-sharp so that the point cloud conversion works with the ros-sharp networking stack? In read me of ros-reality/ it says it is base on rossharp now but it is not yet. Do you think you could do use a favor? Thanks a million

[dwhit]

Hi Steven, my defending went great! I am now a doctor. I’m sorry but I didn’t have time to do this before I left school. I’m in Europe away from a computer until mid July. @NishanthJKumar has added the point cloud code into the ros-sharp stack before, he should be able to help. Sorry I can’t do more right now.

[stevensu1838]

Hey buddy,
It is definately good news and I am very happy to hear that. I am sure I can make it somehow. Enjoy your trip! Thanks anyway