v0.9.0

See https://github.com/quic/ai-hub-models/releases/v0.9.0 for changelog.

Signed-off-by: QAIHM Team <[email protected]>

.github/ISSUE_TEMPLATE/compute_issue_template

@@ -0,0 +1,33 @@
+---
+name: Compute Bring Your Own Model - Bug report
+about: Create a report to help us improve
+title: "[BUG] Compute BYOM Issue: <insert issue here>"
+labels: 'compute'
+assignees: ''
+
+---
+
+**Describe the issue**
+Please provide details relating to the issue you're hitting, if it is related to performance, accuracy or other model issues with bringing your own model to Qualcomm AI Hub, to deploy to compute devices.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Stack trace**
+If applicable, add screenshots to help explain your problem.
+
+**Host configuration:**
+ - OS and version: [e.g. Linux, Windows, macOS]
+ - Browser [e.g. chrome, safari]
+ - QAI-Hub-Models version:
+ - QAI-Hub client version:
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/iot_issue_template.md

@@ -0,0 +1,33 @@
+---
+name: IOT Bring Your Own Model - Model Issue
+about: Create a report to help us improve
+title: "[BUG] IOT BYOM Issue: <insert issue here>"
+labels: 'iot'
+assignees: ''
+
+---
+
+**Describe the issue**
+Please provide details relating to the issue you're hitting, if it is related to performance, accuracy or other model issues with bringing your own model to Qualcomm AI Hub, to deploy to iot devices.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Stack trace**
+If applicable, add screenshots to help explain your problem.
+
+**Host configuration:**
+ - OS and version: [e.g. Linux, Windows, macOS]
+ - Browser [e.g. chrome, safari]
+ - QAI-Hub-Models version:
+ - QAI-Hub client version:
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/mobile_issue_template.md

@@ -0,0 +1,33 @@
+---
+name: Mobile Bring Your Own Model - Model Issue
+about: Create a report to help us improve
+title: "[BUG] Mobile BYOM Issue: <insert issue here>"
+labels: 'mobile'
+assignees: ''
+
+---
+
+**Describe the issue**
+Please provide details relating to the issue you're hitting, if it is related to performance, accuracy or other model issues with bringing your own model to Qualcomm AI Hub, to deploy to mobile devices.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Stack trace**
+If applicable, add screenshots to help explain your problem.
+
+**Host configuration:**
+ - OS and version: [e.g. Linux, Windows, macOS]
+ - Browser [e.g. chrome, safari]
+ - QAI-Hub-Models version:
+ - QAI-Hub client version:
+
+**Additional context**
+Add any other context about the problem here.

README.md

@@ -286,6 +286,7 @@ Qualcomm® AI Hub Models is licensed under BSD-3. See the [LICENSE file](../LICE
 | [MobileNet-v3-Large-Quantized](https://aihub.qualcomm.com/models/mobilenet_v3_large_quantized) | [qai_hub_models.models.mobilenet_v3_large_quantized](qai_hub_models/models/mobilenet_v3_large_quantized/README.md) | ✔️ | ✔️ | ✔️
 | [MobileNet-v3-Small](https://aihub.qualcomm.com/models/mobilenet_v3_small) | [qai_hub_models.models.mobilenet_v3_small](qai_hub_models/models/mobilenet_v3_small/README.md) | ✔️ | ✔️ | ✔️
 | [RegNet](https://aihub.qualcomm.com/models/regnet) | [qai_hub_models.models.regnet](qai_hub_models/models/regnet/README.md) | ✔️ | ✔️ | ✔️
+| [RegNetQuantized](https://aihub.qualcomm.com/models/regnet_quantized) | [qai_hub_models.models.regnet_quantized](qai_hub_models/models/regnet_quantized/README.md) | ✔️ | ✔️ | ✔️
 | [ResNeXt101](https://aihub.qualcomm.com/models/resnext101) | [qai_hub_models.models.resnext101](qai_hub_models/models/resnext101/README.md) | ✔️ | ✔️ | ✔️
 | [ResNeXt101Quantized](https://aihub.qualcomm.com/models/resnext101_quantized) | [qai_hub_models.models.resnext101_quantized](qai_hub_models/models/resnext101_quantized/README.md) | ✔️ | ✔️ | ✔️
 | [ResNeXt50](https://aihub.qualcomm.com/models/resnext50) | [qai_hub_models.models.resnext50](qai_hub_models/models/resnext50/README.md) | ✔️ | ✔️ | ✔️
@@ -295,6 +296,7 @@ Qualcomm® AI Hub Models is licensed under BSD-3. See the [LICENSE file](../LICE
 | [ResNet18](https://aihub.qualcomm.com/models/resnet18) | [qai_hub_models.models.resnet18](qai_hub_models/models/resnet18/README.md) | ✔️ | ✔️ | ✔️
 | [ResNet18Quantized](https://aihub.qualcomm.com/models/resnet18_quantized) | [qai_hub_models.models.resnet18_quantized](qai_hub_models/models/resnet18_quantized/README.md) | ✔️ | ✔️ | ✔️
 | [ResNet50](https://aihub.qualcomm.com/models/resnet50) | [qai_hub_models.models.resnet50](qai_hub_models/models/resnet50/README.md) | ✔️ | ✔️ | ✔️
+| [ResNet50Quantized](https://aihub.qualcomm.com/models/resnet50_quantized) | [qai_hub_models.models.resnet50_quantized](qai_hub_models/models/resnet50_quantized/README.md) | ✔️ | ✔️ | ✔️
 | [Shufflenet-v2](https://aihub.qualcomm.com/models/shufflenet_v2) | [qai_hub_models.models.shufflenet_v2](qai_hub_models/models/shufflenet_v2/README.md) | ✔️ | ✔️ | ✔️
 | [Shufflenet-v2Quantized](https://aihub.qualcomm.com/models/shufflenet_v2_quantized) | [qai_hub_models.models.shufflenet_v2_quantized](qai_hub_models/models/shufflenet_v2_quantized/README.md) | ✔️ | ✔️ | ✔️
 | [SqueezeNet-1_1](https://aihub.qualcomm.com/models/squeezenet1_1) | [qai_hub_models.models.squeezenet1_1](qai_hub_models/models/squeezenet1_1/README.md) | ✔️ | ✔️ | ✔️
@@ -368,13 +370,16 @@ Qualcomm® AI Hub Models is licensed under BSD-3. See the [LICENSE file](../LICE
 | | | | |
 | **Pose Estimation**
 | [HRNetPose](https://aihub.qualcomm.com/models/hrnet_pose) | [qai_hub_models.models.hrnet_pose](qai_hub_models/models/hrnet_pose/README.md) | ✔️ | ✔️ | ✔️
+| [HRNetPoseQuantized](https://aihub.qualcomm.com/models/hrnet_pose_quantized) | [qai_hub_models.models.hrnet_pose_quantized](qai_hub_models/models/hrnet_pose_quantized/README.md) | ✔️ | ✔️ | ✔️
 | [LiteHRNet](https://aihub.qualcomm.com/models/litehrnet) | [qai_hub_models.models.litehrnet](qai_hub_models/models/litehrnet/README.md) | ✔️ | ✔️ | ✔️
 | [MediaPipe-Pose-Estimation](https://aihub.qualcomm.com/models/mediapipe_pose) | [qai_hub_models.models.mediapipe_pose](qai_hub_models/models/mediapipe_pose/README.md) | ✔️ | ✔️ | ✔️
 | [OpenPose](https://aihub.qualcomm.com/models/openpose) | [qai_hub_models.models.openpose](qai_hub_models/models/openpose/README.md) | ✔️ | ✔️ | ✔️
 | [Posenet-Mobilenet](https://aihub.qualcomm.com/models/posenet_mobilenet) | [qai_hub_models.models.posenet_mobilenet](qai_hub_models/models/posenet_mobilenet/README.md) | ✔️ | ✔️ | ✔️
+| [Posenet-Mobilenet-Quantized](https://aihub.qualcomm.com/models/posenet_mobilenet_quantized) | [qai_hub_models.models.posenet_mobilenet_quantized](qai_hub_models/models/posenet_mobilenet_quantized/README.md) | ✔️ | ✔️ | ✔️
 | | | | |
 | **Depth Estimation**
 | [Midas-V2](https://aihub.qualcomm.com/models/midas) | [qai_hub_models.models.midas](qai_hub_models/models/midas/README.md) | ✔️ | ✔️ | ✔️
+| [Midas-V2-Quantized](https://aihub.qualcomm.com/models/midas_quantized) | [qai_hub_models.models.midas_quantized](qai_hub_models/models/midas_quantized/README.md) | ✔️ | ✔️ | ✔️
 
 ### Audio
 

qai_hub_models/_version.py

@@ -2,4 +2,4 @@
 # Copyright (c) 2024 Qualcomm Innovation Center, Inc. All rights reserved.
 # SPDX-License-Identifier: BSD-3-Clause
 # ---------------------------------------------------------------------
-__version__ = "0.8.0"
+__version__ = "0.9.0"

qai_hub_models/evaluators/base_evaluators.py

@@ -135,7 +135,6 @@ def _for_each_batch(
             The input, output, and (if provided) ground_truth will be passed to this function after each inference.
     """
     torch_device = torch.device(device)
-    model.eval()
     model.to(torch_device)
     total_samples = 0
     num_samples = num_samples or len(data)

qai_hub_models/global_requirements.txt

@@ -9,6 +9,7 @@ Deprecated==1.2.11
 PySoundFile; sys_platform == 'win32'
 aimet-torch==1.31.2; sys_platform == "linux"
 albumentations==0.5.2
+audio2numpy==0.1.2
 basicsr==1.4.2
 boto3==1.34.119
 botocore==1.34.119
@@ -41,6 +42,7 @@ pytorch-lightning==1.6.0
 rapidfuzz==3.8.1
 regex==2023.10.3
 ruamel-yaml==0.18.6
+samplerate==0.2.1
 schema==0.7.5
 scikit-image==0.21.0
 scikit-learn==1.1.3

qai_hub_models/models/_shared/cityscapes_segmentation/app.py

@@ -119,8 +119,7 @@ def predict(self, image: Image, raw_output: bool = False) -> Image | np.ndarray:
         )
 
         input_tensor = preprocess_cityscapes_image(resized_image)
-        with torch.no_grad():
-            small_res_output = self.model(input_tensor)
+        small_res_output = self.model(input_tensor)
 
         output = F.interpolate(
             small_res_output,

qai_hub_models/models/_shared/convnext_tiny_quantized/model.py

@@ -122,5 +122,4 @@ def from_pretrained(
                 aimet_encodings = cls._default_aimet_encodings()
             load_encodings_to_sim(sim, aimet_encodings)
 
-        sim.model.eval()
         return cls(sim)

qai_hub_models/models/_shared/deeplab/app.py

@@ -64,9 +64,8 @@ def predict(self, image: Image, raw_output: bool = False) -> Image | np.ndarray:
         """
 
         input_tensor = preprocess_image(image)
-        with torch.no_grad():
-            output = self.model(input_tensor)
-            output = output[0]
+        output = self.model(input_tensor)
+        output = output[0]
         predictions = output.argmax(0).byte().cpu().numpy()
 
         if raw_output:

qai_hub_models/models/_shared/detr/app.py

@@ -81,8 +81,7 @@ def predict(
             )
         image_array = normalize_image_torchvision(preprocess_PIL_image(image))
 
-        with torch.no_grad():
-            outputs = self.model(image_array)
+        outputs = self.model(image_array)
         target_sizes = torch.tensor(image.size[::-1]).unsqueeze(0)
 
         out_logits, out_bbox = outputs[0], outputs[1]

qai_hub_models/models/_shared/detr/model.py

@@ -27,7 +27,6 @@ def __init__(self, model: nn.Module) -> None:
     @classmethod
     def from_pretrained(cls, ckpt_name: str):
         model = DetrForObjectDetection.from_pretrained(ckpt_name)
-        model.eval()
         return cls(model)
 
     def forward(self, image: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:

qai_hub_models/models/_shared/fastsam/model.py

@@ -22,7 +22,6 @@ def __init__(self, model: nn.Module) -> None:
     @classmethod
     def from_pretrained(cls, ckpt_name: str):
         model = FastSAM(ckpt_name).model
-        model.eval()
         return cls(model)
 
     def forward(self, image: torch.Tensor):

qai_hub_models/models/_shared/ffnet/model.py

@@ -69,7 +69,6 @@ class FFNet(CityscapesSegmentor):
     @classmethod
     def from_pretrained(cls: Type[FFNetType], variant_name: str) -> FFNetType:
         model = _load_ffnet_source_model(variant_name)
-        model.eval()
 
         return cls(model)
 
@@ -121,7 +120,7 @@ def _load_ffnet_source_model(variant_name) -> torch.nn.Module:
 
         from models.model_registry import model_entrypoint
 
-        model = model_entrypoint(variant_name)().eval()
+        model = model_entrypoint(variant_name)()
         return model
 
 

qai_hub_models/models/_shared/ffnet_quantized/model.py

@@ -78,5 +78,4 @@ def from_pretrained(
                 aimet_encodings = cls.default_aimet_encodings()
             load_encodings_to_sim(sim, aimet_encodings)
 
-        sim.model.eval()
         return cls(sim)

qai_hub_models/models/_shared/imagenet_classifier/app.py

@@ -75,6 +75,5 @@ def predict(self, image: Image) -> torch.Tensor:
             to a different Imagenet1K class.
         """
         input_tensor = preprocess_image(image, not self.normalization_in_network)
-        with torch.no_grad():
-            output = self.model(input_tensor)
+        output = self.model(input_tensor)
         return torch.softmax(output[0], dim=0)

qai_hub_models/models/_shared/imagenet_classifier/model.py

@@ -55,7 +55,6 @@ def __init__(
         self.normalize_input = normalize_input
         self.transform_input = transform_input
         self.net = net
-        self.eval()
 
     # Type annotation on image_tensor causes aimet onnx export failure
     def forward(self, image_tensor):

qai_hub_models/models/_shared/llama/__init__.py

@@ -0,0 +1,4 @@
+# ---------------------------------------------------------------------
+# Copyright (c) 2024 Qualcomm Innovation Center, Inc. All rights reserved.
+# SPDX-License-Identifier: BSD-3-Clause
+# ---------------------------------------------------------------------