Connect to a Specific Device using IP address as input #153

samialperen · 2022-10-14T02:38:19Z

…idual cameras with IP. In this way, we can connect to multiple cameras easier by specifying their IP adress. Device manager under depthai-python was used to assign static IP addresses to each camera

saching13

Thanks for the PR.

As of the IP address based access I can accept that. But can you also modify to include MxID based device selection for PoE too ?

And also would need the same changes for stereo_inertial_publisher.cpp in ros2.

As of the launch file. I would pass on that for now. It still limits to 3 devices. If you use the single device launch file and call it multiple times in an another launch file it should still do the same job. So the users can call it the number of devices they have. be it 3 or 10.

depthai_examples/launch/multi_stereo_inertial_node.launch

depthai_examples/ros1_src/stereo_inertial_publisher.cpp

…nnect one randomly. Also checked different possibilities to cover all conditional branches and added necessary error prints

samialperen · 2022-11-04T14:25:34Z

@saching13 how about the new commit? Now we have three way of connecting
1-) useWithIp
2-) useWithMxID
3-) useWithIP = false, useWithMxId = false --> current method in the file listing all available devices and connecting one randomly

If it looks good, I will bring changes to ROS2 code too.

zflat · 2022-11-07T18:24:55Z

@samialperen I'm interested in using this with ROS2. If you push an update to this branch I can test it out because I want to launch multiple OAK-D cameras and specify the IP address for each camera to launch. In my use-case I will associate specific IP addresses with specific top prefixes for further processing in a ROS node.

samialperen · 2022-11-07T19:10:27Z

@zflat heyo! I will try to bring the changes to ROS2 either tonight or tomorrow night. Stay tuned and thanks for offering testing it out.

…ult current method) options added to ROS2

samialperen · 2022-11-08T15:44:45Z

@zflat @saching13 ROS2 changes were added with the last commit and tested in ROS2 Foxy.

saching13 · 2022-11-09T04:28:58Z

@samialperen All looks good.
Can you also update the original stereo_inertial_node.launch file too and use it in the multiple ? feel free to add argument to disable the urdf if not needed.

…s and tested in noetic. Multi node now is calling stereo inertial node multiple times to give an idea to users how they can use multiple cameras

samialperen · 2022-11-09T16:56:34Z

@saching13 Done! Tested with a single USB in ROS noetic, and it is working fine. I did not have a chance to test multi-launch with multiple cameras since I no longer have the same setup.

zflat · 2022-11-10T14:49:42Z

@samialperen I did some initial testing and have some feedback

Need to check `DeviceInfo` after connecting to the camera

I found that if you check deviceInfo.status right after it is created with auto deviceInfo = dai::DeviceInfo(ipAddress); then the status has a value of 0. However, if you create the Device connection with device = std::make_shared<dai::Device>(pipeline, deviceInfo); and then check the associated device info the value I got was 1 using device.getDeviceInfo().state. Is this specific to the POE camera and different than the USB camera?

Need a try/catch and retry logic

I know this is just an example, but it would help if the application could gracefully handle re-trying if a connection can not be made or re-connecting if the connection is broken.

Please follow consistent capitalization for ros params

The new ros params like useWithIp should be use_with_ip to match the style of existing params like cam_pos_x.

samialperen · 2022-11-10T15:47:23Z

@zflat heyo! Thanks for the feedback.

I am not sure about deviceInfo.status. That is why I decided to go with deviceInfo.state which helps with better control.
I agree, but to keep this PR simple, I will suggest to open a feature request. We can add that as a new feature to fix connection failures to the camera.
They are already consistent. Check enableDotProjector and other ros params.

zflat · 2022-11-10T17:34:50Z

@samialperen Oops, I meant to write state and not status.

This is what I am logging out after I connect:

RCLCPP_INFO(rclcpp::get_logger("rclcpp"), "deviceInfo.state: %d", device.getDeviceInfo().state)

That logs "deviceInfo.state: 1" after dai::Device device(pipeline, deviceInfo) returns.

zflat · 2022-11-10T20:41:30Z

depthai_examples/ros1_src/stereo_inertial_publisher.cpp

+    if(useWithIP) // Connecting to a camera with specific IP, requires static IP config beforehand
+    {
+        auto deviceInfo = dai::DeviceInfo(ipAddress);
+        if(deviceInfo.state == X_LINK_UNBOOTED || deviceInfo.state == X_LINK_BOOTLOADER || deviceInfo.state == X_LINK_FLASH_BOOTED) {


The value of deviceInfo.state is always 0 here so the exception "Could NOT connect to device with IP Address" is always thrown. If we want to check if another process already has a connection, maybe we should do something like this:

if(useWithIP) { auto connectionInfo = dai::Device::getAllConnectedDevices(); for (auto const& info : connectionInfo) { if (info.name == ipAddress && info.state == X_LINK_BOOTED) { // Throw exception here.... } } auto deviceInfo = dai::DeviceInfo(ipAddress); if(poeMode) { device = std::make_shared<dai::Device>(pipeline, deviceInfo); } else { device = std::make_shared<dai::Device>(pipeline, deviceInfo, usb2Mode); } if(device.getDeviceInfo().state == X_LINK_BOOTED) { std::cout << "Device found with IP Address: " << ipAddress << std::endl; } else if(useWithMxId) { // ... }

@zflat Did you assign a static ip to your camera using depthai device manager? I tested it with my camera and it seems to be working. deviceInfo.state is actually an enum, so if it returns 1, it can take the following values

so 1 probably corresponds to X_LINK_BOOTED, which kinda tells me that you have not used device manager, because once you use device manager to assign static IP, deviceInfo.state becomes X_LINK_FLASH_BOOTED.

Yes, I did assign a static IP.
I tested this code on an OAK-1 (actually copy and pasted the logic to rgb_publisher.cpp because it can't use a pipeline that is set up for steroe) and also an OAK-D.

OAK-1

OAK-D

@zflat Could you try to flash the newest bootloader? I remember I had a problem connecting to my device with IP address before updating the bootloader. If it doesn't solve the issue, I should have OAK (Series 1) POE lying somewhere. I will test it with that guy.

I have tried flashing a new bootloader and I just get an error popup and then a message to the terminal that it was cancelled. Sorry I can't be of more help.

@zflat No problem! Lemme give it a try.

@zflat You are right. Somehow, deviceInfo.state returns 0 once it is not created by get all available devices. 0 means X_LINK_ANY_STATE, so I added that one to if condition and tested it with two cameras at the same time. It should be working fine now.

@zflat what was your execution command here ?

…eInfo.state = 0

saching13 · 2022-11-14T13:17:33Z

depthai_examples/ros1_src/stereo_inertial_publisher.cpp

+    if(useWithIP) // Connecting to a camera with specific IP, requires static IP config beforehand
+    {
+        auto deviceInfo = dai::DeviceInfo(ipAddress);
+        if(deviceInfo.state == X_LINK_ANY_STATE || deviceInfo.state == X_LINK_UNBOOTED || deviceInfo.state == X_LINK_BOOTLOADER || deviceInfo.state == X_LINK_FLASH_BOOTED) {


PoE device can be launched only if it is in Bootloader Mode I think. IIRC.

@saching13 I tested this with POE device too and sometimes it goes to X_LINK_ANY_STATE. Also, if you update the latest bootloader, it takes FLASH_BOOTED, so to be on the safe side, I included all the conditions, but if you think some of them is not necessary, then I will remove unnecessary ones.

is this the case ? @themarpe

XLink any state would be an issue querying the information from the device. We should fix that in core. Fo you have a repro @samialperen ?

Flash booted state happens when a pipeline is flashed onto device to run standalone on boot. That is a valid case to then stop that pipeline and start a new one

@themarpe @saching13 I have OAK-D version 1, OAK-D-PRO-W POE, OAK-D-PRO USB, OAK-D-PRO-W USB. If you can share more context about what is expected from each device (or device type USB vs poe), I can test this one and share the results here.

If queried you should see unbooted, bootloader, and booted if already running.
Check protocol for type of connection (usb/tcpip)

saching13 · 2022-11-14T13:18:39Z

depthai_examples/ros1_src/stereo_inertial_publisher.cpp

+            } else {
+                device = std::make_shared<dai::Device>(pipeline, deviceInfo, usb2Mode);
+            }


if(useWithIP)
section is already PoE region. you don't need to check for if PoE and else here. Do it only under MxID condition

Makes a lot of sense.

saching13 · 2022-11-14T13:22:31Z

depthai_examples/ros1_src/stereo_inertial_publisher.cpp

+    else if(useWithMxId) // Connecting to a camera with unique MxID
+    {
+        auto deviceInfo = dai::DeviceInfo(mxId);
+        if(deviceInfo.state == X_LINK_ANY_STATE || deviceInfo.state == X_LINK_UNBOOTED || deviceInfo.state == X_LINK_BOOTLOADER || deviceInfo.state == X_LINK_FLASH_BOOTED) {


No need for Flashed_Boot here. it means device is running on it's own using the NOR flash config already.

I will test this if it works fine, then I will remove flash booted, but as far as I remember, flashed booted comes up once you update the bootloader.

thoughts ? @themarpe

@samialperen
Just updating the bootloader should not cause a flash booted state as described in the other comment. I assume you have a pipeline flashed on the device, which is then booted.

Also this code isn't valid. DeviceInfo is just a hokder of information that other functions fill out. Setting mxid to constructor doesn't fill out the state or any other fields.

Use Device.getX device to search for devices and return deviceinfo objects

saching13 · 2022-11-14T13:23:05Z

depthai_examples/ros2_src/stereo_inertial_publisher.cpp

+    if(useWithIP) // Connecting to a camera with specific IP, requires static IP config beforehand
+    {
+        auto deviceInfo = dai::DeviceInfo(ipAddress);
+        if(deviceInfo.state == X_LINK_ANY_STATE || deviceInfo.state == X_LINK_UNBOOTED || deviceInfo.state == X_LINK_BOOTLOADER || deviceInfo.state == X_LINK_FLASH_BOOTED) {
+            std::cout << "Device found with IP Address: " << ipAddress <<  std::endl;
+            if(poeMode) {
+                device = std::make_shared<dai::Device>(pipeline, deviceInfo);
+            } else {
+                device = std::make_shared<dai::Device>(pipeline, deviceInfo, usb2Mode);


same as above

saching13 · 2022-11-14T13:23:18Z

depthai_examples/ros2_src/stereo_inertial_publisher.cpp

+        if(deviceInfo.state == X_LINK_ANY_STATE || deviceInfo.state == X_LINK_UNBOOTED || deviceInfo.state == X_LINK_BOOTLOADER || deviceInfo.state == X_LINK_FLASH_BOOTED) {
+            std::cout << "Connecting to device with MxID : " << mxId <<  std::endl;


same as above

saching13 · 2022-11-14T13:28:07Z

depthai_examples/ros2_src/stereo_inertial_publisher.cpp

-                throw std::runtime_error("\" DepthAI Device with MxId  \"" + mxId + "\" is already booted on different process.  \"");
+    if(useWithIP) // Connecting to a camera with specific IP, requires static IP config beforehand
+    {
+        auto deviceInfo = dai::DeviceInfo(ipAddress);


What will happen if useWithIp is set but ipAddress is not set ?
Same for useWithMxID

Good condition to check. will test and update it.

…mxID but do not specify the addresses.

themarpe

As discussed in other comments main issue this PR has is that it assumes that DeviceInfo searches for the device. Just creating DeviceInfo object leaves everything else empty (default values, which due to opt. could be any value) and is likely the reason for seeing states such as any state, etc...

To handle this, retrieve a device list of available device using Device::getAvailableDevices() and then check if a specified deviceId or ipAddress/usbPort was found. If so, pass that specific DeviceInfo to the constructor of a Device

themarpe · 2022-11-16T10:34:41Z

CC: @Serafadam

samialperen · 2022-11-16T16:12:40Z

As discussed in other comments main issue this PR has is that it assumes that DeviceInfo searches for the device. Just creating DeviceInfo object leaves everything else empty (default values, which due to opt. could be any value) and is likely the reason for seeing states such as any state, etc...

To handle this, retrieve a device list of available device using Device::getAvailableDevices() and then check if a specified deviceId or ipAddress/usbPort was found. If so, pass that specific DeviceInfo to the constructor of a Device

@themarpe this was my initial plan however, dai::DeviceInfo does not have a function to get IP address of the device. It only has deviceInfo.getMxId(). Would it be possible to update C++ API and add deviceInfo.getIPv4() function?

zflat · 2022-11-18T04:20:10Z

@themarpe this was my initial plan however, dai::DeviceInfo does not have a function to get IP address of the device. It only has deviceInfo.getMxId(). Would it be possible to update C++ API and add deviceInfo.getIPv4() function?

I found that deviceInfo.name can be compared to an IP address string. Maybe this is not documented and only works in some cases? Or is this a supported way to check if a device has a specific IP address?

themarpe · 2022-11-18T08:12:03Z

@samialperen
Name carries ip or usb path depending on protocol. Its the current to go means if retrieving it.

themarpe · 2022-11-18T08:13:01Z

Specific functions to retrieve ipv4 could be added. You may open a PR against core

samialperen · 2022-11-21T18:50:59Z

@themarpe I see. Lemme check deviceInfo.name, if it does not work out, I will open a PR for depthai-core. Thanks! I will keep updating here.

… of all devices then iterate through them.

themarpe · 2022-11-28T15:53:09Z

Note I think this PR could greatly be simplified.

Core already supports the following: dai::Device(pipeline, dai::DeviceInfo("mxid_or_IP_or_USB_path"));
So redundancy on carrying over whether its IP or ID, etc... can be removed. And if the fields is unpopulated, you may just fall back to "automatic search" and thats it

And Device constructor also handles the devices's STATE so that can also be removed.

samialperen · 2022-12-01T23:13:01Z

@themarpe I agree, the last commit was just to check whether everything is working or not.

saching13 · 2022-12-12T06:28:24Z

depthai_examples/launch/stereo_inertial_node.launch.py

+    ipAddress    = LaunchConfiguration('ipAddress',     default = 'x')
    mxId         = LaunchConfiguration('mxId',      default = 'x')


it is x here. you are checking for empty in the code.

@saching13 Good catch! I fixed it with the new commit

iliabaranov · 2022-12-29T19:42:17Z

+1 on this one, would make using multiple cameras (4 in our case :) ) much more reasonable.
Cheers!

samialperen · 2023-01-03T02:23:57Z

+1 on this one, would make using multiple cameras (4 in our case :) ) much more reasonable. Cheers!

@iliabaranov good to know :) Sorry for the late reply. I hope it will be merged soon.
@saching13 @themarpe Let me know if something else is missing.

Stereo inertial publisher node now supports a way to connect to indiv…

f73904f

…idual cameras with IP. In this way, we can connect to multiple cameras easier by specifying their IP adress. Device manager under depthai-python was used to assign static IP addresses to each camera

saching13 suggested changes Oct 30, 2022

View reviewed changes

depthai_examples/launch/multi_stereo_inertial_node.launch Outdated Show resolved Hide resolved

depthai_examples/ros1_src/stereo_inertial_publisher.cpp Outdated Show resolved Hide resolved

Added useWithMxId option as well an option to list all devices and co…

20a584e

…nnect one randomly. Also checked different possibilities to cover all conditional branches and added necessary error prints

useWithIP, useWithMxId, and none of them (connect automatically, defa…

e57960f

…ult current method) options added to ROS2

Stereo inertial node launch file updated to accommodate new parameter…

972091c

…s and tested in noetic. Multi node now is calling stereo inertial node multiple times to give an idea to users how they can use multiple cameras

zflat reviewed Nov 10, 2022

View reviewed changes

if condition is fixed to include any_state which corresponds to devic…

2cfcb1e

…eInfo.state = 0

saching13 suggested changes Nov 14, 2022

View reviewed changes

saakgun added 2 commits November 14, 2022 22:32

Unnecessary condition under useWithIP conditinal branch removed.

4e9eea5

Added conditions to ROS1 for the case when users select useWithIP or …

d31e32f

…mxID but do not specify the addresses.

themarpe requested changes Nov 16, 2022

View reviewed changes

saching13 mentioned this pull request Nov 28, 2022

[BUG] Enable to access Point cloud/image in ROS noetic with OAK-D Pro Poe camera (Ubuntu 20.04) #173

Open

saching13 changed the title ~~Stereo inertial publisher node now supports a way to connect to indiv…~~ Connect to a Specific Device using IP address as input Nov 28, 2022

now using standard get all available devices function to get a vector…

9d3f9d3

… of all devices then iterate through them.

simplifying the code.

c0b5e95

Clearing out unnecessary params from the previous commits.

0676041

saching13 reviewed Dec 12, 2022

View reviewed changes

Fixed non empty ipaddress and mxId check for ROS2 launch file.

d9b41bd

fixing ros2 pnh error

9514c6e

		if(deviceInfo.state == X_LINK_ANY_STATE \|\| deviceInfo.state == X_LINK_UNBOOTED \|\| deviceInfo.state == X_LINK_BOOTLOADER \|\| deviceInfo.state == X_LINK_FLASH_BOOTED) {
		std::cout << "Connecting to device with MxID : " << mxId << std::endl;

		ipAddress = LaunchConfiguration('ipAddress', default = 'x')
		mxId = LaunchConfiguration('mxId', default = 'x')

Connect to a Specific Device using IP address as input #153

Are you sure you want to change the base?

Connect to a Specific Device using IP address as input #153

Conversation

samialperen commented Oct 14, 2022

saching13 left a comment • edited Loading

Choose a reason for hiding this comment

samialperen commented Nov 4, 2022

zflat commented Nov 7, 2022

samialperen commented Nov 7, 2022

samialperen commented Nov 8, 2022

saching13 commented Nov 9, 2022

samialperen commented Nov 9, 2022

zflat commented Nov 10, 2022

Need to check DeviceInfo after connecting to the camera

Need a try/catch and retry logic

Please follow consistent capitalization for ros params

samialperen commented Nov 10, 2022

zflat commented Nov 10, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

themarpe left a comment

Choose a reason for hiding this comment

themarpe commented Nov 16, 2022

samialperen commented Nov 16, 2022

zflat commented Nov 18, 2022

themarpe commented Nov 18, 2022

themarpe commented Nov 18, 2022

samialperen commented Nov 21, 2022

themarpe commented Nov 28, 2022 • edited Loading

samialperen commented Dec 1, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iliabaranov commented Dec 29, 2022

samialperen commented Jan 3, 2023

saching13 left a comment •

edited

Loading

Need to check `DeviceInfo` after connecting to the camera

themarpe commented Nov 28, 2022 •

edited

Loading