Convert conditional-detr and dn-detr pretrained weights (#210)

* add cond-detr r50&r101 dc5 models * refine readme * add DN-DETR converted weights
IDEA-Research · Mar 14, 2023 · b92b945 · b92b945
1 parent 80936c0
commit b92b945
Show file tree

Hide file tree

Showing 11 changed files with 136 additions and 2 deletions.
diff --git a/README.md b/README.md
@@ -83,6 +83,11 @@ Please refer to [Getting Started with detrex](https://detrex.readthedocs.io/en/l
 - [Analyze the model under detrex](https://detrex.readthedocs.io/en/latest/tutorials/Tools.html#model-analysis)
 - [Download and initialize with the pretrained backbone weights](https://detrex.readthedocs.io/en/latest/tutorials/Using_Pretrained_Backbone.html)
 - [Frequently asked questions](https://github.com/IDEA-Research/detrex/issues/109)
+- [A simple onnx convert tutorial provided by powermano](https://github.com/IDEA-Research/detrex/issues/192)
+- Simple training techniques: [Model-EMA](https://github.com/IDEA-Research/detrex/pull/201), [Mixed Precision Training](https://github.com/IDEA-Research/detrex/pull/198), [Activation Checkpoint](https://github.com/IDEA-Research/detrex/pull/200)
+- [Simple tutorial about custom dataset training](https://github.com/IDEA-Research/detrex/pull/187)
+
+Although some of the tutorials are currently presented with relatively simple content, we will constantly improve our documentation to help users achieve a better user experience.
 
 ## Documentation
 

diff --git a/docs/source/tutorials/Model_Zoo.md b/docs/source/tutorials/Model_Zoo.md
@@ -7,7 +7,7 @@
 
 
 ## COCO Object Detection Baselines
-Here we provides our pretrained baselines with **detrex**. And more pretrained weights will be released in the future version. We also provide our converted pretrained for the users which will be marked as `(converted)`.
+Here we provides our pretrained baselines with **detrex**. And more pretrained weights will be released in the future version. We also provide our converted pretrained weights for the users which will be marked as `(converted)`.
 
 ### DETR
 <table class="docutils"><tbody>
@@ -98,6 +98,28 @@ Here we provides our pretrained baselines with **detrex**. And more pretrained w
 <td align="center">50</td>
 <td align="center">41.6</td>
 <td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.1.0/conditional_detr_r50_50ep.pth">model</a></td>
+</tr>
+ <tr><td align="left"><a href="https://github.com/IDEA-Research/detrex/blob/main/projects/conditional_detr/configs/conditional_detr_r50_dc5_50ep.py">Conditional-DETR-R50-DC5 (converted)</a></td>
+<td align="center">R-50-DC5</td>
+<td align="center">IN1k</td>
+<td align="center">50</td>
+<td align="center">43.8</td>
+<td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.3.0/converted_conditional_detr_r50_dc5.pth">model</a></td>
+</tr>
+<!-- ROW: conditional_detr_r101_50ep -->
+ <tr><td align="left"><a href="https://github.com/IDEA-Research/detrex/blob/main/projects/conditional_detr/configs/conditional_detr_r101_50ep.py">Conditional-DETR-R101 (converted)</a></td>
+<td align="center">R-101</td>
+<td align="center">IN1k</td>
+<td align="center">50</td>
+<td align="center">43.0</td>
+<td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.1.0/converted_conditional_detr_r101_50ep.pth">model</a></td>
+</tr>
+ <tr><td align="left"><a href="https://github.com/IDEA-Research/detrex/blob/main/projects/conditional_detr/configs/conditional_detr_r101_dc5_50ep.py">Conditional-DETR-R101-DC5 (converted)</a></td>
+<td align="center">R-101-DC5</td>
+<td align="center">IN1k</td>
+<td align="center">50</td>
+<td align="center">45.1</td>
+<td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.3.0/converted_conditional_detr_r101_dc5.pth">model</a></td>
 </tr>
 </tbody></table>
 
@@ -195,6 +217,13 @@ Here we provides our pretrained baselines with **detrex**. And more pretrained w
 <td align="center">50</td>
 <td align="center">44.7</td>
 <td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.1.0/dn_detr_r50_50ep.pth"> model </a></td>
+</tr>
+ <tr><td align="left"> <a href="https://github.com/IDEA-Research/detrex/blob/main/projects/dn_detr/configs/dn_detr_r50_dc5_50ep.py"> DN-DETR-R50-DC5 (converted) </a> </td>
+<td align="center">R50</td>
+<td align="center">IN1k</td>
+<td align="center">50</td>
+<td align="center">46.3</td>
+<td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.3.0/converted_dn_detr_r50_dc5.pth"> model </a></td>
 </tr>
 </tbody></table>
 

diff --git a/projects/README.md b/projects/README.md
@@ -8,4 +8,5 @@ Here are projects that are built on detrex which show you use detrex as a librar
 - [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](./dn_detr/)
 - [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](./dino)
 - [Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment](./group_detr/)
-- [DETRs with Hybrid Matching](./h_deformable_detr/)
+- [DETRs with Hybrid Matching](./h_deformable_detr/)
+- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](./maskdino/)
diff --git a/projects/conditional_detr/README.md b/projects/conditional_detr/README.md
@@ -48,6 +48,13 @@ Here we provide the pretrained `Conditional-DETR` weights based on detrex.
 <td align="center">50</td>
 <td align="center">41.0</td>
 <td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.1.0/converted_conditional_detr_r50_50ep.pth">model</a></td>
+</tr>
+ <tr><td align="left"><a href="configs/conditional_detr_r50_dc5_50ep.py">Conditional-DETR-R50-DC5</a></td>
+<td align="center">R-50-DC5</td>
+<td align="center">IN1k</td>
+<td align="center">50</td>
+<td align="center">43.8</td>
+<td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.3.0/converted_conditional_detr_r50_dc5.pth">model</a></td>
 </tr>
 <!-- ROW: conditional_detr_r101_50ep -->
  <tr><td align="left"><a href="configs/conditional_detr_r101_50ep.py">Conditional-DETR-R101</a></td>
@@ -56,6 +63,13 @@ Here we provide the pretrained `Conditional-DETR` weights based on detrex.
 <td align="center">50</td>
 <td align="center">43.0</td>
 <td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.1.0/converted_conditional_detr_r101_50ep.pth">model</a></td>
+</tr>
+ <tr><td align="left"><a href="configs/conditional_detr_r101_dc5_50ep.py">Conditional-DETR-R101-DC5</a></td>
+<td align="center">R-101-DC5</td>
+<td align="center">IN1k</td>
+<td align="center">50</td>
+<td align="center">45.1</td>
+<td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.3.0/converted_conditional_detr_r101_dc5.pth">model</a></td>
 </tr>
 </tbody></table>
 

diff --git a/projects/conditional_detr/configs/conditional_detr_r101_dc5_50ep.py b/projects/conditional_detr/configs/conditional_detr_r101_dc5_50ep.py
@@ -0,0 +1,14 @@
+from .conditional_detr_r50_50ep import (
+    train,
+    dataloader,
+    optimizer,
+    lr_multiplier,
+)
+from .models.conditional_detr_r50_dc5 import model
+
+# modify training config
+train.init_checkpoint = "https://download.pytorch.org/models/resnet101-63fe2227.pth"
+train.output_dir = "./output/conditional_detr_r101_dc5_50ep"
+
+# modify model
+model.backbone.name = "resnet101"
diff --git a/projects/conditional_detr/configs/conditional_detr_r50_dc5_50ep.py b/projects/conditional_detr/configs/conditional_detr_r50_dc5_50ep.py
@@ -0,0 +1,11 @@
+from .conditional_detr_r50_50ep import (
+    train,
+    dataloader,
+    optimizer,
+    lr_multiplier,
+)
+from .models.conditional_detr_r50_dc5 import model
+
+# modify training config
+train.init_checkpoint = "https://download.pytorch.org/models/resnet50-0676ba61.pth"
+train.output_dir = "./output/conditional_detr_r50_dc5_50ep"
diff --git a/projects/conditional_detr/configs/models/conditional_detr_r50_dc5.py b/projects/conditional_detr/configs/models/conditional_detr_r50_dc5.py
@@ -0,0 +1,12 @@
+from detectron2.config import LazyCall as L
+from detrex.modeling.backbone.torchvision_resnet import TorchvisionResNet
+
+from .conditional_detr_r50 import model
+
+
+model.backbone=L(TorchvisionResNet)(
+    name="resnet50",
+    train_backbone=True,
+    dilation=True,
+    return_layers={"layer4": "res5"}
+)
diff --git a/projects/dab_detr/configs/dab_detr_r50_dc5_50ep.py b/projects/dab_detr/configs/dab_detr_r50_dc5_50ep.py
@@ -11,4 +11,5 @@
 train.output_dir = "./output/dab_detr_r50_dc5_50ep"
 
 # modify model
+# DAB-DETR using 10 temperature for DC5 model
 model.position_embedding.temperature = 10
diff --git a/projects/dn_detr/README.md b/projects/dn_detr/README.md
@@ -32,6 +32,29 @@ Here we provide the pretrained `DN-DETR` weights based on detrex.
 </tbody></table>
 
 
+## Converted Models
+<table><tbody>
+<!-- START TABLE -->
+<!-- TABLE HEADER -->
+<th valign="bottom">Name</th>
+<th valign="bottom">Backbone</th>
+<th valign="bottom">Pretrain</th>
+<th valign="bottom">Epochs</th>
+<th valign="bottom">box<br/>AP</th>
+<th valign="bottom">download</th>
+<!-- TABLE BODY -->
+<!-- ROW: dn_detr_r50_50ep -->
+ <tr><td align="left"><a href="configs/dn_detr_r50_dc5_50ep.py">DN-DETR-R50-DC5</a></td>
+<td align="center">R-50</td>
+<td align="center">IN1k</td>
+<td align="center">50</td>
+<td align="center">46.3</td>
+<td align="center"> <a href="https://github.com/IDEA-Research/detrex-storage/releases/download/v0.3.0/converted_dn_detr_r50_dc5.pth">model</a></td>
+</tr>
+</tr>
+</tbody></table>
+
+
 ## Training
 All configs can be trained with:
 ```bash

diff --git a/projects/dn_detr/configs/dn_detr_r50_dc5_50ep.py b/projects/dn_detr/configs/dn_detr_r50_dc5_50ep.py
@@ -0,0 +1,12 @@
+from .dn_detr_r50_50ep import (
+    train,
+    dataloader,
+    optimizer,
+    lr_multiplier,
+)
+from .models.dn_detr_r50_dc5 import model
+
+# modify training config
+train.init_checkpoint = "https://download.pytorch.org/models/resnet50-0676ba61.pth"
+train.output_dir = "./output/dab_detr_r50_dc5_50ep"
+
diff --git a/projects/dn_detr/configs/models/dn_detr_r50_dc5.py b/projects/dn_detr/configs/models/dn_detr_r50_dc5.py
@@ -0,0 +1,12 @@
+from detectron2.config import LazyCall as L
+from detrex.modeling.backbone.torchvision_resnet import TorchvisionResNet
+
+from .dn_detr_r50 import model
+
+
+model.backbone=L(TorchvisionResNet)(
+    name="resnet50",
+    train_backbone=True,
+    dilation=True,
+    return_layers={"layer4": "res5"}
+)