Detectron2 - no detections below 832x832 in trt, onnxruntime versions detect correctly

[Huxwell]

Hey!

I've been trying square resolutions divisible by 64x64 (divisibility by 32 was not always enough for error-free conversion).
I'm doing this both for maskrcnn with coco weights, as well as custom keypoints with similar outcome.
The issue is, 1344x1344, 1280x1280, 960x960, 832x832 don't seem to loose accuracy, but 768x768, 704x704, 640x640 are fully blind (no detections or detections with confidence from activation function below 10%).
In onnxruntime the 640x640, 768x768 work perfectly too, so the problem is not in detectron2 export_model.py.

This leads to conclusion, that something in create_onnx.py or in one of the trt plugins (ROIAlign or Efficient NMS) uses width and height of the image as basis for some computation that gives different results 832x832.
Any tips on how to look for the solution @azhurkevich @letmejoing @RajUpadhyay ?

Related issues:
#2142
#3792 (comment)
facebookresearch/detectron2#5255

[RajUpadhyay]

@Huxwell
I have actually not touched detectron2 for quite a while but from what I remember, isn't detectron2's default configuration settings for min size = 800 & max size = 1300?

While I do understand that onnxruntime works for your onnx file generated from export_model.py but
how are you running your tensorrt?
Is it deepstream? If no then,
Is it the infer.py given to us by this repo? If yes then are you modifying the preprocessing where the model again tries to get the detectron2's default min_size (800) & max_size (1300).

Since the information is not enough, it is really difficult to narrow down your situation but from what I see, your models which have the dimensions less than 800 are not being supported.

Find a way to validate your onnx file generated from create_onnx.py. i.e., netron or maybe TensorRTEP (onnxruntime)
Validate your tensorrt engine using trtexec i.e., loadEngine.

I could be wrong but maybe you could try checking all this if you haven't already. Thanks.

[Huxwell]

Thanks for the tips.

I am running the model with infer.py
If I add print statement to Line 129 with scores:
https://github.com/NVIDIA/TensorRT/blob/release/9.0/samples/python/detectron2/infer.py#L127
I will get 100 scores, sorted.
For resolution >= 832, there will be some high values:

[[0.9966532  0.4419432  0.43740156 0.39249143 0.24443977 0.16181383
  0.15681778 0.11954353 0.11906159 0.07982704 0.06165875 0.06028476
  0.05638076 0.05417441 0.05292124 0.         0.         0. .......................

But for <832 (i.e. 768x768) it's:

[[0.00332  0.000143  0.0 0.0 0.0 0.0
  0.0 0.0 0.0 0.0 0.0 0.0
  0.0 0.0 0.0 0.         0.         0.

So I assume it's not a postprocesing issue and infer.py is correct.

TensorRTEP (onnxruntime)
Do you think it will work without ROIAlign and NMS as tensor plugins from create_onnx.py?
I will check, as well as try to validate with validators.

I also thought about 800 and 1300 being default in detectron2, but I had no problems with 1344x1344 in tensorrt, and no problems with 640x640 in onnx_runtime, so the problem is most likely in tensorrt conversion.

[edanpa-bw]

@Huxwell
Do the inputs have to be square inputs or if my width and height are divisible by 32 but aren't the same then its fine?

[Huxwell]

Sorry @edanpa-bw , I don't know.
1344x1344, 1280x1280, 960x960, 832x832 works for me
768x768, 704x704, 640x640 work in onnx, but 0.0% mAP after TensortRT conversion.
I think you need to test non-square for yourself, however making it divisible by 32 is probably not enough.
I moved away from detectron2 for now.