Skip to content

Commit

Permalink
Merge back OTX2.2RC12 to OTX2.3 (#4071)
Browse files Browse the repository at this point in the history 
* update for releases 2.2.0rc0

* Fix Classification explain forward issue (#3867)

Fix bug

* Fix e2e code error (#3871)

* Update test_cli.py

* Update tests/e2e/cli/test_cli.py

Co-authored-by: Eunwoo Shin <[email protected]>

* Update test_cli.py

* Update test_cli.py

---------

Co-authored-by: Eunwoo Shin <[email protected]>

* Add documentation about configurable input size (#3870)

* add docs about configurable input size

* update api usecase and fix bug

* Fix zero-shot e2e (#3876)

Fix

* Fix DeiT for multi-label classification (#3881)

Remove init_args

* Fix Semi-SL for ViT accuracy drop (#3883)

Remove init_args

* Update docs for 2.2 (#3884)

Update docs

* Fix mean and scale for segmentation task (#3885)

fix mean and scale

* Update MAPI in 2.2 (#3889)

* Bump MAPI

* Update exportable code requirements

* Improve Semi-SL for LiteHRNet (small-medium case) (#3891)

* change drop pixels value

* go safe, change only tested models

* minor

* Improve h-cls for eff models (#3893)

* Update step size for eff v2

* Update effb0 recipe

* Fix maskrcnn swin nncf acc drop (#3900)

update maskrcnn swimt model type to transformer

* Add keypoint detection recipe for single object cases (#3903)

* add rtmpose_tiny for single obj

* add rtmpose_tiny for single obj

* modify test subset name

* fix unit test

* update recipe with reset

* Improve acc drop of efficientnetv2 for h-label cls (#3907)

* Add warmup_iters for effv2

* Update max_epochs

* Fix pretrained weight cached dir for timm (#3909)

* Fix pretrained_weight for timm

* Fix unit-test

* Fix keypoint detection single obj recipe (#3915)

* add rtmpose_tiny for single obj

* modify test subset name

* fix unit test

* property for pck

* Fix cached dir for timm & hugging-face (#3914)

* Fix cached dir

* Pretrained weight download unit-test

* Fix pre-commit

* Fix wrong template id mapping for anomaly (#3916)

* Update script to allow setting otx version using env. variable (#3913)

* Fix Datamodule creation for OV in AutoConfigurator (#3920)

Fix datamodule for ov

* Update tpp file for 2.2.0 (#3921)

* Fix names for ignored scope [HOT-FIX, 2.2.0] (#3924)

fix names for ignored scope

* Fix classification rt_info (#3922)

* Restore output_raw_scores for classificaiton

* Add uts

* Fix linter

* Update label info (#3925)

add label info to init

Signed-off-by: Ashwin Vaidya <[email protected]>

* Fix binary classification metric task (#3928)

* Fix binary classification

* Add unit-tests

* Improve MaskRCNN SwinT NNCF (#3929)

* ignore heads and disable smooth quant

* add activations_range_estimator_params

* update changelog

* Fix get_item for Chained Tasks in Classification (#3931)

* Fix Task Chain

* Add multi-label case as well

* Add multi-label case as well2

* Add H-label case

* Correct Keyerror for h-label cls in label_groups for dm_label_categories using label's id/key (#3932)

Modify label_groups for dm_label_categories with id/key of label

* Remove datumaro attribute id from tiling, add subset names (#3933)

* remove datumaro attribute id from tiling

* add subset names

* Fix soft predictions for Semantic Segmentation (#3934)

fix soft preds

* Update STFPM config (#3935)

* Add missing pretrained weights when creating a docker image (#3938)

* Fix pre-trained weight downloader

* Remove if condition for pretrained wiehgt download

* Change default option 'full' to 'base' in otx install (#3937)

* Change option full to base for otx install

* Fix wrong code

* Fix issue

* Fix docs

* Fix auto adapt batch size in Converter (#3939)

* Enable auto adapt batch size into converter

* Fix wrong

* Fix hpo converter (#3940)

* save best hp after hpo

* add test

* Fix tiling XAI out of range (#3943)

- Fix tile merge XAI out of range

* enable model export (#3952)

Signed-off-by: Ashwin Vaidya <[email protected]>

* Move templates from OTX1.X to OTX2.X (#3951)

* add otx1.6 templates

* added new models

* delete entrypoints and nncf cfg

* updated some hyperparams

* fix for rtmdet_tiny

* updated converter

* Update classification templates

* Update det, r-det, vpm

* Update template.yaml

* changed warmaup value in train.yaml

---------

Co-authored-by: Kang, Harim <[email protected]>
Co-authored-by: Kim, Sungchul <[email protected]>

* Add missing tile recipes and various tile recipe changes  (#3942)

* add missing tile recipes

* Fix tiling XAI out of range (#3943)

- Fix tile merge XAI out of range

* update xai tile merge

* update rtdetr

* update tile recipes

* update rtdetr tile postprocess

* update rtdetr recipes and tile recipes

* update tile recipes

* fix rtdetr unittest

* update recipes

* refactor tile unit test

* address pr reviews

* remove unnecessary files

* update color channel

* fix image channel passing

* include tiling in cli integration test

* remove transform_bbox

---------

Co-authored-by: Vladislav Sovrasov <[email protected]>

* Support ImageFromBytes (#3948)

* add image_from_bytes

Signed-off-by: Ashwin Vaidya <[email protected]>

* refactor code

Signed-off-by: Ashwin Vaidya <[email protected]>

* allow empty anomalous masks

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* Change categories mapping logic (#3946)

* change pre-filtering logic

* Update src/otx/core/data/pre_filtering.py

Co-authored-by: Eunwoo Shin <[email protected]>

---------

Co-authored-by: Eunwoo Shin <[email protected]>

* Update for 2.2.0rc1 (#3956)

* Include Geti arrow dataset subset names (#3962)

* restrited number of output masks by tiling

* add geti subset name

* update num of max pred

* Include full image with anno in case there's no tile in tile dataset (#3964)

* include full image with anno incase there's no tile in dataset

* update test

* Add type checker in converter for callable functions (optimizer, scheduler) (#3968)

Fix converter callable functions (optimizer, scheduler)

* Update for 2.2.0rc2 (#3969)

update for 2.2.0rc2

* Fix config converter for tiling (#3973)

fix config converter for tiling

* Update for 2.2.0rc3 (#3975)

* Change sematic segmentation to consider bbox only annotations. (#3996)

* segmentation consider bbox only annotations

* add unit test

* add unit test

* update fixture

* use name attribute

* revert tox file

* update for 2.2.0rc4

---------

Co-authored-by: Yunchu Lee <[email protected]>

* Relieve memory usage criteria on batch size 2 during adaptive_bs (#4009)

* release memory usage cirteria on batch size 2 during adpative_bs

* update unit test

* update unit test

* Remove background label from RT Info for segmentation task (#4011)

* remove background from rt_info

* provide another solution

* fix unit test

* Fix num_trials calculation on dataset length less than num_class (#4014)

Fix balanced sampler

* Fix out_features in HierarchicalCBAMClsHead (#4016)

Fix out_features

* Fix empty anno (#4010)

* Refactor mask_target_single function to handle unsupported ground truth mask types and provide warnings for missing ground truth masks

* Refactor bbox_overlaps function to handle unsupported ground truth mask types and provide warnings for missing ground truth masks

* Refactor export script to export multiple directories

* Refactor test_bbox_overlaps_2d to handle mismatched batch dimensions of bboxes

* Refactor bbox_overlaps function error exception

* update changelog

---------

Co-authored-by: Harim Kang <[email protected]>

* Update for release 2.2.0rc5 (#4015)

* Prevent using too low confidence thresholds in detection (#4018)

Prevent writing too low confidence thresholds to MAPI configuration

* Update for release 2.2.0rc6 (#4027)

* Update pre-merge workflow (#4032)

* Update HPO interface (#4035)

* update hpo interface

* update unit test

* update CHANGELOG.md

* Enable keypoint detection training through config conversion (#4034)

enable keypoint det config converter

* Update for release 2.2.0rc7 (#4036)

update for release 2.2.0rc7

* Fix multilabel_accuracy of MixedHLabelAccuracy (#4042)

* Fix metric for multi-label

* Fix1

* Add CHANGELOG

* Update for release 2.2.0rc8 (#4043)

* Fix wrong indices setting in HLabelInfo (#4044)

* Fix wrong indices setting in label_info

* Add unit-test & update for releases

* Add legacy template LiteHRNet_18 template (#4049)

added legacy template

* Model templates: rename model_status value 'DISCONTINUED' to 'OBSOLETE' (#4051)

rename 'DISCONTINUED' to 'OBSOLETE' in model templates

* Enable export of feature vectors for semantic segmentation task (#4055)

* Upgrade MAPI in 2.2 (#4052)

* Update MRCNN model export to include feature vector and saliency map (#4056)

* Fix applying model's hparams when loading model from checkpoint (#4057)

* Update anomaly transforms (#4059)

* Update transforms

Signed-off-by: Ashwin Vaidya <[email protected]>

* Update transforms

Signed-off-by: Ashwin Vaidya <[email protected]>

* Update changelog

Signed-off-by: Ashwin Vaidya <[email protected]>

* Update __init__.py

---------

Signed-off-by: Ashwin Vaidya <[email protected]>
Co-authored-by: Emily Chun <[email protected]>

* Bump onnx to 1.17.0 to omit CVE-2024-5187 (#4063)

* Fix incorrect all_groups order configuration in HLabelInfo (#4067)

* Fix all_labels

* Update CHAGELOG

* label_groups change

* fix conflicts

---------

Signed-off-by: Ashwin Vaidya <[email protected]>
Co-authored-by: Yunchu Lee <[email protected]>
Co-authored-by: Harim Kang <[email protected]>
Co-authored-by: Emily Chun <[email protected]>
Co-authored-by: Eunwoo Shin <[email protected]>
Co-authored-by: Kim, Sungchul <[email protected]>
Co-authored-by: Vladislav Sovrasov <[email protected]>
Co-authored-by: Sooah Lee <[email protected]>
Co-authored-by: Eugene Liu <[email protected]>
Co-authored-by: Wonju Lee <[email protected]>
Co-authored-by: Ashwin Vaidya <[email protected]>
Co-authored-by: Leonardo Lai <[email protected]>
  • Loading branch information
@kprokofi @yunchu
@harimkang @chuneuny-emily @eunwoosh @sungchul2 @sovrasov @eugene123tw @wonjuleee @ashwinvaidya17 @leoll2
12 people authored Oct 28, 2024
1 parent 6672653 commit 41c1d38
Show file tree
Hide file tree
Showing 45 changed files with 671 additions and 159 deletions.
7 changes: 0 additions & 7 deletions .ci/docker/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,9 @@
FROM nvidia/cuda:12.1.0-runtime-ubuntu20.04@sha256:c1869c30f46fff478a37ed58d9dace7e08519541274f03424d0b78bd35b2c73a AS python_base_cuda
LABEL maintainer="OpenVINO Training Extensions Development Team"

ARG HTTP_PROXY
ARG HTTPS_PROXY
ARG NO_PROXY
ARG uid
ARG gid

# Setup proxies
ENV http_proxy=$HTTP_PROXY
ENV https_proxy=$HTTPS_PROXY
ENV no_proxy=$NO_PROXY
ENV DEBIAN_FRONTEND="noninteractive"

# hadolint ignore=DL3008
Expand Down
3 changes: 0 additions & 3 deletions .ci/docker/build.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,6 @@ fi
TAG=$1

docker build -f ./Dockerfile \
--build-arg HTTP_PROXY="${http_proxy:?}" \
--build-arg HTTPS_PROXY="${https_proxy:?}" \
--build-arg NO_PROXY="${no_proxy:?}" \
--build-arg ACTIONS_RUNNER_VER="$ACTIONS_RUNNER_VER" \
--build-arg gid="$(id -g)" \
--build-arg uid="$UID" \
Expand Down
8 changes: 1 addition & 7 deletions .ci/docker/start-runner.sh
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash

GPU_ID="all"
VER_CUDA="11.7.1"
VER_CUDA="12.1.0"
TAG_RUNNER="latest"
ADDITIONAL_LABELS=""
MOUNT_PATH=""
Expand Down Expand Up @@ -149,9 +149,6 @@ if [ "$DEBUG_CONTAINER" = true ]; then
--name "$CONTAINER_NAME" \
-e NVIDIA_VISIBLE_DEVICES="$GPU_ID" \
${ENV_FLAGS} \
-e http_proxy=http://proxy-chain.intel.com:911 \
-e https_proxy=http://proxy-chain.intel.com:912 \
-e no_proxy=intel.com,.intel.com,localhost,127.0.0.0/8 \
${MOUNT_FLAGS} \
${CACHE_MOUNT_FLAGS} \
"$DOCKER_REG_ADDR"/ote/ci/cu"$VER_CUDA"/runner:"$TAG_RUNNER"; RET=$?
Expand All @@ -172,9 +169,6 @@ else
--name "$CONTAINER_NAME" \
-e NVIDIA_VISIBLE_DEVICES="$GPU_ID" \
${ENV_FLAGS} \
-e http_proxy=http://proxy-chain.intel.com:911 \
-e https_proxy=http://proxy-chain.intel.com:912 \
-e no_proxy=intel.com,.intel.com,localhost,127.0.0.0/8 \
${MOUNT_FLAGS} \
${CACHE_MOUNT_FLAGS} \
"$DOCKER_REG_ADDR"/ote/ci/cu"$VER_CUDA"/runner:"$TAG_RUNNER"; RET=$?
Expand Down
32 changes: 29 additions & 3 deletions .github/workflows/pre_merge.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,6 @@ jobs:
- task: "multi_cls_classification"
- task: "multi_label_classification"
- task: "hlabel_classification"
- task: "detection"
- task: "instance_segmentation"
- task: "semantic_segmentation"
- task: "visual_prompting"
- task: "zero_shot_visual_prompting"
- task: "anomaly_classification"
Expand All @@ -127,3 +124,32 @@ jobs:
rm /tmp/requirements.txt
- name: Run Integration Test
run: tox -vv -e integration-test-${{ matrix.task }}
Integration-Test-Large:
if: |
github.event.pull_request.draft == false &&
!(startsWith(github.event.pull_request.title, '[WIP]'))
runs-on: [self-hosted, linux, x64, dev, dmount]
needs: Unit-Test
strategy:
fail-fast: false
matrix:
include:
- task: "detection"
- task: "instance_segmentation"
- task: "semantic_segmentation"
name: Integration-Test-Large-${{ matrix.task }}-py310
steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: Install Python
uses: actions/setup-python@39cd14951b08e74b54015e9e001cdefcf80e669f # v5.1.1
with:
python-version: "3.10"
- name: Install tox
run: |
python -m pip install --require-hashes --no-deps -r .ci/requirements.txt
pip-compile --generate-hashes --output-file=/tmp/requirements.txt --extra=ci_tox pyproject.toml
python -m pip install --require-hashes --no-deps -r /tmp/requirements.txt
rm /tmp/requirements.txt
- name: Run Integration Test
run: tox -vv -e integration-test-${{ matrix.task }}
60 changes: 48 additions & 12 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,29 +40,29 @@ All notable changes to this project will be documented in this file.
### New features

- Add RT-DETR model for Object Detection
(https://github.com/openvinotoolkit/training_extensions/pull/3741)
(<https://github.com/openvinotoolkit/training_extensions/pull/3741>)
- Add Multi-Label & H-label Classification with torchvision models
(https://github.com/openvinotoolkit/training_extensions/pull/3697)
(<https://github.com/openvinotoolkit/training_extensions/pull/3697>)
- Add Hugging-Face Model Wrapper for Classification
(https://github.com/openvinotoolkit/training_extensions/pull/3710)
(<https://github.com/openvinotoolkit/training_extensions/pull/3710>)
- Add LoRA finetuning capability for ViT Architectures
(https://github.com/openvinotoolkit/training_extensions/pull/3729)
(<https://github.com/openvinotoolkit/training_extensions/pull/3729>)
- Add Hugging-Face Model Wrapper for Object Detection
(https://github.com/openvinotoolkit/training_extensions/pull/3747)
(<https://github.com/openvinotoolkit/training_extensions/pull/3747>)
- Add Hugging-Face Model Wrapper for Semantic Segmentation
(https://github.com/openvinotoolkit/training_extensions/pull/3749)
(<https://github.com/openvinotoolkit/training_extensions/pull/3749>)
- Enable torch.compile to work with classification
(https://github.com/openvinotoolkit/training_extensions/pull/3758)
(<https://github.com/openvinotoolkit/training_extensions/pull/3758>)
- Add `otx benchmark` subcommand
(https://github.com/openvinotoolkit/training_extensions/pull/3762)
(<https://github.com/openvinotoolkit/training_extensions/pull/3762>)
- Add RTMPose for Keypoint Detection Task
(https://github.com/openvinotoolkit/training_extensions/pull/3781)
(<https://github.com/openvinotoolkit/training_extensions/pull/3781>, <https://github.com/openvinotoolkit/training_extensions/pull/4034>)
- Add Semi-SL MeanTeacher algorithm for Semantic Segmentation
(https://github.com/openvinotoolkit/training_extensions/pull/3801)
(<https://github.com/openvinotoolkit/training_extensions/pull/3801>)
- Update head and h-label format for hierarchical label classification
(https://github.com/openvinotoolkit/training_extensions/pull/3810)
(<https://github.com/openvinotoolkit/training_extensions/pull/3810>)
- Support configurable input size
(https://github.com/openvinotoolkit/training_extensions/pull/3788)
(<https://github.com/openvinotoolkit/training_extensions/pull/3788>)

### Enhancements

Expand Down Expand Up @@ -90,9 +90,21 @@ All notable changes to this project will be documented in this file.
(<https://github.com/openvinotoolkit/training_extensions/pull/3968>)
- Change sematic segmentation to consider bbox only annotations
(<https://github.com/openvinotoolkit/training_extensions/pull/3996>)
- Relieve memory usage criteria on batch size 2 during adaptive batch size
(<https://github.com/openvinotoolkit/training_extensions/pull/4009>)
- Remove background label from RT Info for segmentation task
(<https://github.com/openvinotoolkit/training_extensions/pull/4011>)
- Prevent using too low confidence thresholds in detection
(<https://github.com/openvinotoolkit/training_extensions/pull/4018>)
- Update HPO interface
(<https://github.com/openvinotoolkit/training_extensions/pull/4035>)
- Bump onnx to 1.17.0 to omit CVE-2024-5187
(<https://github.com/openvinotoolkit/training_extensions/pull/4063>)

### Bug fixes

- Update anomaly base transforms to use square resizing
(<https://github.com/openvinotoolkit/training_extensions/pull/4059>)
- Fix Combined Dataloader & unlabeled warmup loss in Semi-SL
(<https://github.com/openvinotoolkit/training_extensions/pull/3723>)
- Revert #3579 to fix issues with replacing coco_instance with a different format in some dataset
Expand All @@ -105,6 +117,30 @@ All notable changes to this project will be documented in this file.
(<https://github.com/openvinotoolkit/training_extensions/pull/3946>)
- Fix config converter for tiling
(<https://github.com/openvinotoolkit/training_extensions/pull/3973>)
- Fix `BboxOverlaps2D` handling of empty ground-truth annotations in datasets.
(<https://github.com/openvinotoolkit/training_extensions/pull/4010>)
- Fix num_trials calculation on dataset length less than num_class
(<https://github.com/openvinotoolkit/training_extensions/pull/4014>)
- Fix out_features in HierarchicalCBAMClsHead
(<https://github.com/openvinotoolkit/training_extensions/pull/4016>)
- Fix multilabel_accuracy of MixedHLabelAccuracy
(<https://github.com/openvinotoolkit/training_extensions/pull/4042>)
- Fix wrong indices setting in HLabelInfo
(<https://github.com/openvinotoolkit/training_extensions/pull/4044>)
- Add legacy template LiteHRNet_18 template
(<https://github.com/openvinotoolkit/training_extensions/pull/4049>)
- Model templates: rename model_status value 'DISCONTINUED' to 'OBSOLETE'
(<https://github.com/openvinotoolkit/training_extensions/pull/4051>)
- Enable export of feature vectors for semantic segmentation task
(<https://github.com/openvinotoolkit/training_extensions/pull/4055>)
- Update MRCNN model export to include feature vector and saliency map
(<https://github.com/openvinotoolkit/training_extensions/pull/4056>)
- Upgrade MAPI in 2.2
(<https://github.com/openvinotoolkit/training_extensions/pull/4052>)
- Fix applying model's hparams when loading model from checkpoint
(<https://github.com/openvinotoolkit/training_extensions/pull/4057>)
- Fix incorrect all_groups order configuration in HLabelInfo
(<https://github.com/openvinotoolkit/training_extensions/pull/4067>)

## \[v2.1.0\]

Expand Down
7 changes: 7 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,9 @@ In addition to the examples above, please refer to the documentation for tutoria
- Include full image with anno in case there's no tile in tile dataset
- Add type checker in converter for callable functions (optimizer, scheduler)
- Change sematic segmentation to consider bbox only annotations
- Relieve memory usage criteria on batch size 2 during adaptive batch size
- Remove background label from RT Info for segmentation task
- Prevent using too low confidence thresholds in detection

### Bug fixes

Expand All @@ -206,6 +209,10 @@ In addition to the examples above, please refer to the documentation for tutoria
- Add missing tile recipes and various tile recipe changes
- Change categories mapping logic
- Fix config converter for tiling
- Fix num_trials calculation on dataset length less than num_class
- Fix out_features in HierarchicalCBAMClsHead
- Fix multilabel_accuracy of MixedHLabelAccuracy
- Fix wrong indices setting in HLabelInfo

### Known issues

Expand Down
9 changes: 8 additions & 1 deletion docs/source/guide/release_notes/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Releases
.. toctree::
:maxdepth: 1

v2.2.0 (2024.09)
v2.2.0 (2024.10)
----------------

New features
Expand Down Expand Up @@ -38,6 +38,9 @@ Enhancements
- Include full image with anno in case there's no tile in tile dataset
- Add type checker in converter for callable functions (optimizer, scheduler)
- Change sematic segmentation to consider bbox only annotations
- Relieve memory usage criteria on batch size 2 during adaptive batch size
- Remove background label from RT Info for segmentation task
- Prevent using too low confidence thresholds in detection

Bug fixes
^^^^^^^^^
Expand All @@ -48,6 +51,10 @@ Bug fixes
- Add missing tile recipes and various tile recipe changes
- Change categories mapping logic
- Fix config converter for tiling
- Fix num_trials calculation on dataset length less than num_class
- Fix out_features in HierarchicalCBAMClsHead
- Fix multilabel_accuracy of MixedHLabelAccuracy
- Fix wrong indices setting in HLabelInfo

v2.1.0 (2024.07)
----------------
Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ xpu = [
"timm==1.0.3",
"openvino==2024.4",
"openvino-dev==2024.4",
"openvino-model-api==0.2.4",
"onnx==1.16.2",
"openvino-model-api==0.2.5",
"onnx==1.17.0",
"onnxconverter-common==1.14.0",
"nncf==2.13.0",
"anomalib[core]==1.1.0",
Expand All @@ -96,7 +96,7 @@ base = [
"openvino==2024.4",
"openvino-dev==2024.4",
"openvino-model-api==0.2.4",
"onnx==1.16.2",
"onnx==1.17.0",
"onnxconverter-common==1.14.0",
"nncf==2.13.0",
"anomalib[core]==1.1.0",
Expand Down
2 changes: 1 addition & 1 deletion src/otx/algo/classification/heads/hlabel_cls_head.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ def __init__(
self.fc_superclass = nn.Linear(in_channels * self.step_size[0] * self.step_size[1], num_multiclass_heads)
self.attention_fc = nn.Linear(num_multiclass_heads, in_channels * self.step_size[0] * self.step_size[1])
self.cbam = CBAM(in_channels)
self.fc_subclass = nn.Linear(in_channels * self.step_size[0] * self.step_size[1], num_single_label_classes)
self.fc_subclass = nn.Linear(in_channels * self.step_size[0] * self.step_size[1], num_classes)

self._init_layers()

Expand Down
19 changes: 17 additions & 2 deletions src/otx/algo/common/utils/bbox_overlaps.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@

from __future__ import annotations

import warnings

import torch
from torch import Tensor

Expand Down Expand Up @@ -142,15 +144,28 @@ def bbox_overlaps(
>>> assert tuple(bbox_overlaps(nonempty, empty).shape) == (1, 0)
>>> assert tuple(bbox_overlaps(empty, empty).shape) == (0, 0)
"""
if not (bboxes1.size(-1) == 4 or bboxes1.size(0) == 0):
msg = "bboxes1 must have a last dimension of size 4 or be an empty tensor."
raise ValueError(msg)

if not (bboxes2.size(-1) == 4 or bboxes2.size(0) == 0):
msg = "bboxes2 must have a last dimension of size 4 or be an empty tensor."
raise ValueError(msg)

if bboxes1.shape[:-2] != bboxes2.shape[:-2]:
msg = "The batch dimension of bboxes must be the same."
raise ValueError(msg)

batch_shape = bboxes1.shape[:-2]

rows = bboxes1.size(-2)
cols = bboxes2.size(-2)

if rows * cols == 0:
warnings.warn("No bboxes are provided! Returning empty boxes!", stacklevel=2)
if is_aligned:
return bboxes1.new((*batch_shape, rows))
return bboxes1.new((*batch_shape, rows, cols))
return bboxes1.new(batch_shape + (rows,)) # noqa: RUF005
return bboxes1.new(batch_shape + (rows, cols)) # noqa: RUF005

area1 = (bboxes1[..., 2] - bboxes1[..., 0]) * (bboxes1[..., 3] - bboxes1[..., 1])
area2 = (bboxes2[..., 2] - bboxes2[..., 0]) * (bboxes2[..., 3] - bboxes2[..., 1])
Expand Down
2 changes: 1 addition & 1 deletion src/otx/algo/instance_segmentation/maskrcnn.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,7 @@ def _exporter(self) -> OTXModelExporter:
"opset_version": 11,
"autograd_inlining": False,
},
output_names=["bboxes", "labels", "masks"],
output_names=["bboxes", "labels", "masks", "feature_vector", "saliency_map"] if self.explain_mode else None,
)

def load_from_otx_v1_ckpt(self, state_dict: dict, add_prefix: str = "model.") -> dict:
Expand Down
1 change: 1 addition & 0 deletions src/otx/algo/instance_segmentation/rtmdet_inst.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ def _exporter(self) -> OTXModelExporter:
"opset_version": 11,
"autograd_inlining": False,
},
# TODO(Eugene): Add XAI support for RTMDetInst
output_names=["bboxes", "labels", "masks", "feature_vector", "saliency_map"] if self.explain_mode else None,
)

Expand Down
35 changes: 30 additions & 5 deletions src/otx/algo/instance_segmentation/segmentors/two_stage.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -226,19 +226,44 @@ def export(
self,
batch_inputs: torch.Tensor,
batch_img_metas: list[dict],
) -> tuple[torch.Tensor, ...]:
"""Export for two stage detectors."""
x = self.extract_feat(batch_inputs)
explain_mode: bool,
) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor] | dict:
"""Export the model for ONNX/OpenVINO.
Args:
batch_inputs (torch.Tensor): image tensor with shape (N, C, H, W).
batch_img_metas (list[dict]): image information.
explain_mode (bool): whether to return feature vector.
Returns:
tuple[torch.Tensor, torch.Tensor, torch.Tensor] | dict:
- bboxes (torch.Tensor): bounding boxes.
- labels (torch.Tensor): labels.
- masks (torch.Tensor): masks.
- feature_vector (torch.Tensor, optional): feature vector.
- saliency_map (torch.Tensor, optional): saliency map.
"""
x = self.extract_feat(batch_inputs)
rpn_results_list = self.rpn_head.export(
x,
batch_img_metas,
rescale=False,
)

return self.roi_head.export(
bboxes, labels, masks = self.roi_head.export(
x,
rpn_results_list,
batch_img_metas,
rescale=False,
)

if explain_mode:
feature_vector = self.feature_vector_fn(x)
return {
"bboxes": bboxes,
"labels": labels,
"masks": masks,
"feature_vector": feature_vector,
# create dummy tensor as model API supports saliency_map
"saliency_map": torch.zeros(1),
}
return bboxes, labels, masks
Loading

0 comments on commit 41c1d38

Please sign in to comment.