Merge pull request #26 from NVIDIA-ISAAC-ROS/release-3.1

Isaac ROS 3.1

.gitattributes

@@ -0,0 +1,3 @@
+# Sphinx-generated large files
+public/**/*.pickle filter=lfs diff=lfs merge=lfs -text
+public/**/*.doctree filter=lfs diff=lfs merge=lfs -text

.github/workflows/static.yml

@@ -4,7 +4,9 @@ name: Deploy static content to Pages
 on:
   # Runs on pushes targeting the default branch
   push:
-    branches: ["main"]
+    branches:
+      - main
+      - release-*
 
   # Allows you to run this workflow manually from the Actions tab
   workflow_dispatch:

public/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: d86f066d449a1cad6fc01ca17aaf2d26
+config: c8b87ee34bfc5c9f79c9544ba65a06cd
 tags: 645f666f9bcd5a90fca523b33c5a78b7

public/_sources/about/index.rst.txt

@@ -6,4 +6,3 @@ About
 
    license.rst
    contact.rst
-   synthetic_data_attributions.rst

public/_sources/concepts/object_detection/detectnet/index.rst.txt

@@ -1,7 +1,7 @@
 DetectNet
 =========
 
-`DetectNetV2 <https://docs.nvidia.com/tao/tao-toolkit/text/object_detection/detectnet_v2.html>`__ is an NVIDIA-developed object-detection model that is included in the `TAO Toolkit <https://developer.nvidia.com/tao-toolkit>`__.
+`DetectNetV2 <https://docs.nvidia.com/tao/archive/5.3.0/text/object_detection/detectnet_v2.html>`__ is an NVIDIA-developed object-detection model that is included in the `TAO Toolkit <https://developer.nvidia.com/tao-toolkit>`__.
 To learn more about using DetectNet with TAO, see `here <https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_detectnet_v2>`__.
 
 Repositories and Packages

public/_sources/concepts/object_detection/index.rst.txt

@@ -7,7 +7,7 @@ Object detection finds bounding boxes in pixel coordinates of target objects in
 
 While classical computer vision algorithms have been somewhat successful in object detection,
 deep learned models trained on real examples with correct bounding boxes have changed the game.
-DNN architectures such as `DetectNetV2 <https://docs.nvidia.com/tao/tao-toolkit/text/object_detection/detectnet_v2.html>`__
+DNN architectures such as `DetectNetV2 <https://docs.nvidia.com/tao/archive/5.3.0/text/object_detection/detectnet_v2.html>`__
 and `YOLOv8 <https://yolov8.com/>`__ are world-class at detecting people, common objects, machine parts, or anything needed for the task at hand.
 
 .. figure:: :ir_lfs:`<resources/isaac_ros_docs/concepts/object_detection/predicted-bounding-boxes-consolidated-into-inference-image.png>`

public/_sources/concepts/pose_estimation/foundationpose/tutorial_tracking.rst.txt

@@ -20,7 +20,11 @@ Tutorial Walkthrough
 1.  Complete the
     :ref:`Isaac ROS FoundationPose Quickstart Guide <repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index:quickstart>`.
 
-2.  Open a new terminal and launch the Docker container using the
+2.  Download tracking rosbag from NGC:
+
+   :ir_assets:`<isaac_ros_foundationpose> <foundationpose_tracking.tar.gz>`
+
+3.  Open a new terminal and launch the Docker container using the
     ``run_dev.sh`` script:
 
     .. code:: bash
@@ -31,15 +35,15 @@ Tutorial Walkthrough
 .. tabs::
    .. group-tab:: Rosbag
 
-      3. Continuing inside the container, install the following dependencies:
+      4. Continuing inside the container, install the following dependencies:
 
          :ir_apt:
 
          .. code:: bash
 
             sudo apt-get install -y ros-humble-isaac-ros-examples
 
-      4. Run the following launch files to spin up a demo of this package:
+      5. Run the following launch files to spin up a demo of this package:
 
          Launch ``isaac_ros_foundationpose``:
 
@@ -58,43 +62,43 @@ Tutorial Walkthrough
 
          .. code:: bash
 
-            ros2 bag play -l ${ISAAC_ROS_WS}/src/isaac_ros_pose_estimation/resources/rosbags/foundationpose_tracking.bag/
+            ros2 bag play -l ${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/foundationpose_tracking.bag/
 
 
    .. group-tab:: RealSense Camera
 
-      3. Ensure that you have already set up your RealSense camera using the :doc:`RealSense setup tutorial </getting_started/hardware_setup/sensors/realsense_setup>`. If you have not, please set up the sensor and then restart this quickstart from the beginning.
+      4. Ensure that you have already set up your RealSense camera using the :doc:`RealSense setup tutorial </getting_started/hardware_setup/sensors/realsense_setup>`. If you have not, please set up the sensor and then restart this quickstart from the beginning.
 
-      4. Complete the :ref:`Isaac ROS RT-DETR tutorial <repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index:quickstart>`.
+      5. Complete the :ref:`Isaac ROS RT-DETR tutorial <repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index:quickstart>`.
 
-      5. Open a new terminal and launch the Docker container using the ``run_dev.sh`` script:
+      6. Open a new terminal and launch the Docker container using the ``run_dev.sh`` script:
 
          .. code:: bash
 
             cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
                ./scripts/run_dev.sh
 
-      6. Install the following dependencies:
+      7. Install the following dependencies:
 
          :ir_apt:
 
          .. code:: bash
 
             sudo apt-get install -y ros-humble-isaac-ros-examples ros-humble-isaac-ros-realsense
 
-      7. Place the object in front of the camera and run the launch file:
+      8. Place the object in front of the camera and run the launch file:
 
          .. code:: bash
 
             ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono_rect_depth,foundationpose_tracking mesh_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/Mac_and_cheese_0_1/Mac_and_cheese_0_1.obj texture_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/Mac_and_cheese_0_1/materials/textures/baked_mesh_tex0.png score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan rt_detr_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/synthetica_detr/sdetr_grasp.plan
 
    .. group-tab:: Hawk Camera
 
-      3. Ensure that you have already set up your Hawk camera using the :doc:`Hawk setup tutorial </getting_started/hardware_setup/sensors/hawk_setup>`. If you have not, please set up the sensor and then restart this quickstart from the beginning.
+      4. Ensure that you have already set up your Hawk camera using the :doc:`Hawk setup tutorial </getting_started/hardware_setup/sensors/hawk_setup>`. If you have not, please set up the sensor and then restart this quickstart from the beginning.
 
-      4.  Complete the :ref:`Isaac ROS RT-DETR tutorial <repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index:quickstart>`.
+      5. Complete the :ref:`Isaac ROS RT-DETR tutorial <repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index:quickstart>`.
 
-      5.  Open a new terminal and launch the Docker container using the ``run_dev.sh`` script:
+      6. Open a new terminal and launch the Docker container using the ``run_dev.sh`` script:
 
          .. code:: bash
 
@@ -104,21 +108,16 @@ Tutorial Walkthrough
          .. note::
 
             Due to high latency in the ``RT-DETR`` object detection pipeline, you need to manually modify ``src/isaac_ros_image_pipeline/isaac_ros_image_proc/nitros_resize_node.yaml`` and replace all ``capacity: 12`` with ``capacity: 50``.
-
-         .. todo::
-
-            Reduce latency in the pipeline and remove the above note.
-
 
-      6. Install the following dependencies:
+      7. Install the following dependencies:
 
          :ir_apt:
 
          .. code:: bash
 
             sudo apt-get install -y ros-humble-isaac-ros-examples ros-humble-isaac-ros-argus-camera
 
-      7. Place the object in front of the hawk camera and run the launch file:
+      8. Place the object in front of the hawk camera and run the launch file:
 
          .. code:: bash
 

public/_sources/concepts/scene_reconstruction/nvblox/index.rst.txt

@@ -58,6 +58,7 @@ To get started with nvblox, review the following examples:
 
    tutorials/tutorial_isaac_sim
    tutorials/tutorial_realsense
+   tutorials/tutorial_zed
    tutorials/tutorial_hawk
 
 

public/_sources/concepts/scene_reconstruction/nvblox/tutorials/tutorial_isaac_sim.rst.txt

@@ -97,9 +97,9 @@ The reconstruction is used for navigation with `Nav2 <https://nav2.org/>`__.
 Selecting the Sensors
 ---------------------
 
-Nvblox can integrate data from up to 3 cameras simultaneously.
+Nvblox can integrate data from 3d lidar and up to 3 cameras simultaneously.
 
-To specify the number of cameras you may use the ``num_cameras`` argument by 
+To enable the 3d lidar or more cameras you may use the ``lidar`` and ``num_cameras`` arguments by 
 modifying the following command from `Isaac Sim Example <#isaac-sim-example>`_:
 
 - **Terminal #2:**
@@ -109,9 +109,10 @@ modifying the following command from `Isaac Sim Example <#isaac-sim-example>`_:
       .. code:: bash
 
          ros2 launch nvblox_examples_bringup isaac_sim_example.launch.py \
-         num_cameras:=<"num_cameras">
+         lidar:=<"lidar"> num_cameras:=<"num_cameras">
 
-      where `"num_cameras"` is either 0, 1 or 3. 
+      where `"lidar"` can be set to ``True`` to enable 3d lidar and 
+      `"num_cameras"` is either 0, 1 or 3. 
       Setting `"num_cameras"` to 1 will enable the front stereo camera (default) 
       and setting it to 3 will enable the side stereo cameras additionally.
 
@@ -166,6 +167,7 @@ To run this example modify the following commands from `Isaac Sim Example <#isaa
 
       While nvblox can integrate data from up to 3 cameras into the 3d reconstruction,
       the people reconstruction is always running on the front camera only.
+      Lidar data integration is not supported in people mode.
 
 Reconstruction With Dynamic Scene Elements
 ------------------------------------------
@@ -206,6 +208,10 @@ To run this example modify the following commands from `Isaac Sim Example <#isaa
          ros2 launch nvblox_examples_bringup isaac_sim_example.launch.py \
          mode:=dynamic
 
+   .. note::
+
+      Lidar data integration is not supported in dynamic mode.
+
 
 Running on a Custom Scene
 ^^^^^^^^^^^^^^^^^^^^^^^^^

public/_sources/concepts/scene_reconstruction/nvblox/tutorials/tutorial_zed.rst.txt

@@ -0,0 +1,73 @@
+ZED Camera Examples
+===================
+
+.. figure:: :ir_lfs:`<resources/isaac_ros_docs/repositories_and_packages/isaac_ros_nvblox/zed_example.gif>`
+    :width: 600px
+    :align: center
+
+This page contains tutorials for running nvblox on the `ZED <https://www.stereolabs.com/docs/get-started-with-zed>`__ camera.
+
+.. note::
+
+   This tutorial requires a compatible ZED camera from
+   the list available  :doc:`here </getting_started/hardware_setup/sensors/zed_setup>`.
+
+Prerequisites
+-------------
+
+These are the steps common to running all examples on the zed.
+
+1. Complete the :doc:`ZED setup tutorial </getting_started/hardware_setup/sensors/zed_setup>`.
+
+2. Complete the :doc:`/getting_started/dev_env_setup`.
+
+.. note::
+
+    For best results we suggest increase maximum Linux kernel receive buffer size as
+    detailed `here <https://docs.ros.org/en/rolling/How-To-Guides/DDS-tuning.html#cyclone-dds-tuning>`__.
+
+Install
+-------
+
+1. Complete the :ref:`nvblox quickstart <repositories_and_packages/isaac_ros_nvblox/isaac_ros_nvblox/index:Quickstart>`.
+
+
+ZED Example
+-----------
+
+This example runs nvblox-based reconstruction from a single zed camera,
+either from live data coming directly off a camera, or from recorded
+data coming from a ROSbag.
+
+
+1. Navigate (inside the docker) to the workspace folder
+
+    .. code:: bash
+
+        cd /workspaces/isaac_ros-dev
+
+2. Run the zed example, either live from a sensor or from a recorded ROSbag.
+
+    .. tabs::
+
+        .. tab:: On a ROSbag
+
+            .. code:: bash
+
+                ros2 launch nvblox_examples_bringup zed_example.launch.py \
+                camera:=<ZED_CAMERA_MODEL> rosbag:=<YOUR_DATASET_PATH>
+        
+        .. tab:: Live from a sensor
+
+            .. code:: bash
+
+                ros2 launch nvblox_examples_bringup zed_example.launch.py \
+                camera:=<ZED_CAMERA_MODEL>
+
+    where ``<ZED_CAMERA_MODEL>`` is either ``zed2`` or ``zedx`` depending 
+    on the ZED camera model you are using.
+
+.. note::
+
+    To record your own ZED ROSbag, please refer to the official
+    `ROSbag recording tutorial <https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Recording-And-Playing-Back-Data/Recording-And-Playing-Back-Data.html>`__.

public/_sources/concepts/segmentation/segformer/index.rst.txt

@@ -1,7 +1,7 @@
 Segformer
 =========
 
-`Segformer <https://docs.nvidia.com/tao/tao-toolkit/text/semantic_segmentation/segformer.html>`__ is an NVIDIA-developed semantic-segmentation model that is included in the `TAO Toolkit <https://developer.nvidia.com/tao-toolkit>`__.
+`Segformer <https://docs.nvidia.com/tao/archive/5.3.0/text/semantic_segmentation/segformer.html>`__ is an NVIDIA-developed semantic-segmentation model that is included in the `TAO Toolkit <https://developer.nvidia.com/tao-toolkit>`__.
 To learn more about using PeopleSemSegformer with TAO, see `NGC PeopleSemSegformer <https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplesemsegformer>`__.
 
 Repositories and Packages

public/_sources/getting_started/hardware_setup/sensors/index.rst.txt

@@ -10,4 +10,5 @@ Isaac ROS officially supports the Hawk and RealSense Cameras.
 
    hawk_setup.rst
    realsense_setup.rst
+   zed_setup.rst
    camera_calibration.rst

public/_sources/getting_started/hardware_setup/sensors/zed_setup.rst.txt

@@ -0,0 +1,95 @@
+=========================
+Isaac ROS ZED Setup
+=========================
+
+.. figure:: :ir_lfs:`<resources/isaac_ros_docs/getting_started/zed_parts.png>`
+   :width: 200px
+   :align: center
+
+ZED cameras require the following to be able to publish data to ROS 2 topics:
+
+1. ZED SDK (Installed by ``Dockerfile.zed``)
+2. ``zed-ros2-wrapper`` (Cloned in Step 1 of Setup Instructions)
+3. ZED X driver (Installed by user on host machine in Step 3 of Setup Instructions)
+
+Camera Compatibility
+--------------------
+
+All cameras supported by the `zed ros2 wrapper <https://github.com/stereolabs/zed-ros2-wrapper>`_ work with Isaac ROS.
+
+================= ==============
+ZED Model         SQA Testing?
+================= ==============
+ZED 2i              ✓
+ZED X               ✓
+ZED 2               ✗
+ZED                 ✗
+ZED Mini            ✗
+ZED X Mini          ✗
+================= ==============
+
+
+Setup Instructions
+------------------
+
+.. note::
+
+  This tutorial assumes that you have set up your development environment by following the instructions :doc:`here </getting_started/dev_env_setup>`.
+
+1. Clone the ``zed-ros2-wrapper`` repository:
+
+  .. code:: bash
+
+      cd ${ISAAC_ROS_WS}/src && \
+      git clone --recurse-submodules https://github.com/stereolabs/zed-ros2-wrapper
+  .. note::
+    You must checkout humble-v4.0.8 branch of ``zed-ros2-wrapper`` repository for X86.
+
+
+2. If you are using ZED X or ZED X Mini refer to the appropriate `stereolabs setup guide <https://www.stereolabs.com/docs/get-started-with-zed-x/>`_.
+
+  .. note::
+    You do **not** need to install the ``ZED SDK`` because this is done by ``Dockerfile.zed``.
+
+  .. note::
+    You must install the ``ZED driver``.
+
+
+3. Plug in the USB cable of your ZED camera before launching the Docker container in the next step.
+
+4. Configure the container created by ``isaac_ros_common/scripts/run_dev.sh`` to include ``Dockerfile.zed``. Create the ``.isaac_ros_common-config`` file in the ``isaac_ros_common/scripts`` directory:
+
+  .. code:: bash
+
+    cd ${ISAAC_ROS_WS}/src/isaac_ros_common/scripts && \
+    touch .isaac_ros_common-config && \
+    echo CONFIG_IMAGE_KEY=ros2_humble.user.zed >> .isaac_ros_common-config
+
+5. Launch the Docker container.
+
+  .. code:: bash
+
+    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
+    ./scripts/run_dev.sh
+
+  This rebuilds the container image using ``Dockerfile.zed`` in one of its layered stage. It takes some time for rebuilding.
+
+6. Install the dependencies for the ``zed_wrapper`` package and build it:
+
+  .. code:: bash
+
+      cd ${ISAAC_ROS_WS} && \
+      sudo apt update && \
+      rosdep install --from-paths src/zed-ros2-wrapper --ignore-src -r -y && \
+      colcon build --symlink-install --packages-up-to zed_wrapper
+
+7. After the container image is rebuilt and you are inside the container, you can run ``/usr/local/zed/tools/ZED_Explorer`` to check that the ZED camera is connected.
+
+  .. code:: bash
+
+    /usr/local/zed/tools/ZED_Explorer
+
+  If everything is working as expected, you should see something similar to the following:
+
+  .. figure:: :ir_lfs:`<resources/isaac_ros_docs/getting_started/zed_explorer.png>`
+     :align: center