GroupNormalizationPlugin not found

[hrishi121]

Description

I have a simple test model that has two conv layers and a group norm in pytorch, and I want to serialize it to TensorRT. I converted the pytorch model to ONNX opset20. The groupnorm layer is broken down into instancenorm+few other operations in the ONNX graph. I have been following the onnx_packnet example, where it suggests to replace the instancenorm nodes and replace it with "GroupNormalizationPlugin" node (as shown in the sample post_processing.py script)

I followed the build instructions for "Native build on Jetson (aarch64)" and was able to build TensorRT OSS repo on Jetson AGX Orin.

However, when I try to parse the ONNX model using trtexec, I keep getting that "GroupNormalizationPlugin" not found

[01/04/2025-01:25:44] [V] [TRT] Static check for parsing node: groupnorm1 [GroupNormalizationPlugin]                                                                          [22/1957]
[01/04/2025-01:25:44] [I] [TRT] No checker registered for op: GroupNormalizationPlugin. Attempting to check as plugin.                                                                 
[01/04/2025-01:25:44] [V] [TRT] Local registry did not find GroupNormalizationPlugin creator. Will try parent registry if enabled.                                                     
[01/04/2025-01:25:44] [E] [TRT] IPluginRegistry::getCreator: Error Code 4: API Usage Error (Cannot find plugin: GroupNormalizationPlugin, version: , namespace:.)                      
[01/04/2025-01:25:44] [V] [TRT] Static check for parsing node: /Relu [Relu]                                                                                                            
[01/04/2025-01:25:44] [V] [TRT] Static check for parsing node: /conv2/Conv [Conv]                                                                                                      
[01/04/2025-01:25:44] [V] [TRT] Static check for parsing node: /Relu_1 [Relu]                                                                                                          
[01/04/2025-01:25:44] [V] [TRT] Static check for parsing node: /global_pool/GlobalAveragePool [GlobalAveragePool]                                                                      
[01/04/2025-01:25:44] [V] [TRT] Static check for parsing node: /Flatten [Flatten]                                                                                                      
[01/04/2025-01:25:44] [V] [TRT] Static check for parsing node: /fc/Gemm [Gemm]                                                                                                         
[01/04/2025-01:25:44] [V] [TRT] Static check for parsing node: /softmax/Softmax [Softmax]                                                                                              
[01/04/2025-01:25:44] [E] [TRT] parsers/onnx/ModelImporter.cpp:961: While parsing node number 1 [GroupNormalizationPlugin -> "/group_norm/Add_output_0"]:                              
[01/04/2025-01:25:44] [E] [TRT] parsers/onnx/ModelImporter.cpp:964: --- Begin node ---                                                                                                 
input: "/conv1/Conv_output_0"                                                                                                                                                          
input: "scale"                                                                                                                                                                         
input: "bias"                                                                                                                                                                          
output: "/group_norm/Add_output_0"                                                                                                                                                     
name: "groupnorm1"                                                                                                                                                                     
op_type: "GroupNormalizationPlugin"                                                                                                                                                    
attribute {                                                                                                                                                                            
  name: "num_groups"                                                                                                                                                                   
  i: 4                                                                                                                                                                                 
  type: INT                                                                                                                                                                            
}
attribute {
  name: "eps"
  f: 1e-05
  type: FLOAT
}
attribute {
  name: "plugin_version"
  i: 1
  type: INT
}
attribute {
  name: "plugin_namespace"
  s: ""
  type: STRING
}

[01/04/2025-01:25:44] [E] [TRT] parsers/onnx/ModelImporter.cpp:965: --- End node ---
[01/04/2025-01:25:44] [E] [TRT] parsers/onnx/ModelImporter.cpp:967: ERROR: parsers/onnx/onnxOpCheckers.cpp:812 In function checkFallbackPluginImporter:
[6] creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[01/04/2025-01:25:44] [E] Failed to parse onnx file

These are the commands that I am running to parse the ONNX model using trtexec:

export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/home/nvidia/TensorRT/build/out:/home/nvidia/TensorRT-10.7.0.23/lib
/home/nvidia/TensorRT-10.7.0.23/bin/trtexec --onnx=/home/nvidia/replaced_plugin.onnx --verbose

where,

/home/nvidia/TensorRT/ path refers to TensorRT OSS path and the out directory contains the compiled .so files
/home/nvidia/TensorRT-10.7.0.23/lib is the path to the TensorRT-10.7 GA release downloaded from nvidia website

Environment

TensorRT Version: 10.7.0.23

NVIDIA GPU: Jetson AGX Orin

NVIDIA Driver Version:

CUDA Version: 12.6

CUDNN Version:

Operating System: Jetson native build

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable): 2.5.1

Baremetal or Container (if so, version):

Relevant Files

Model link: replaced_plugin.onnx

Steps To Reproduce

Commands or scripts:

Have you tried the latest release?: Yes

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

[lix19937]

If you has build the nvinfer_plugin.so, you can add some print infos in your code like https://github.com/NVIDIA/TensorRT/blob/release/10.7/plugin/groupNormalizationPlugin/groupNormalizationPlugin.cpp#L350

https://github.com/NVIDIA/TensorRT/blob/release/10.7/plugin/groupNormalizationPlugin/groupNormalizationPlugin.cpp#L394

https://github.com/NVIDIA/TensorRT/blob/release/10.7/plugin/groupNormalizationPlugin/groupNormalizationPlugin.cpp#L431

to check your plugin version is loaded right.

[hrishi121]

Hi @lix19937
Thank you for the pointers! With the print lines, I was able to find the problems:

1. I set the plugin_version in the ONNX model manually with type int, while the plugin registry needs it to be string. So all I had to do was set attrs['plugin_version'] = "1" instead of attrs['plugin_version'] = 1 in my ONNX model using onnx-graphsurgeon
2. The plugin needs libcudnn.so.8. I created a softlink with libcudnn.so.9 and added it to LD_LIBRARY_PATH, and it worked

I was able to bit-match the results with PyTorch and ONNX models as well.
Thank you!