Skip to content

Latest commit

 

History

History
436 lines (379 loc) · 16.1 KB

README.md

File metadata and controls

436 lines (379 loc) · 16.1 KB

iNAS: Integral NAS for Device-Aware Salient Object Detection

Introduction

Integral search design (jointly consider backbone/head structures, design/deploy devices).

Covers mainstream handcraft saliency head design.

SOTA performance with large latency reduction on diverse hardware platforms.

Updates

0.2.0 was released in 15/11/2021:

  • Support training on semantic segmentation task.
  • Fix bugs in generating latency lookup table.
  • Provide pre-trained model for cityscape datasets.

Please refer to changelog.md for details and release history.

Dependencies and Installation

Dependencies

Install from a local clone

  1. Clone the repo

    git clone https://github.com/guyuchao/iNAS.git

  2. Install dependent packages

    conda create -n iNAS python=3.8
conda install -c pytorch pytorch=1.7 torchvision cudatoolkit=10.2
pip install -r requirements.txt

  3. Install iNAS
    Please run the following commands in the iNAS root path to install iNAS:

    python setup.py develop

Dataset Preparation

Folder Structure

iNAS
├── iNAS
├── experiment
├── scripts
├── options
├── datasets
│   ├── saliency
│   │   ├── DUTS-TR/            # Contains both images (.jpg) and labels (.png).
│   │   ├── DUTS-TR.lst         # Specify the image-label pair for training or testing.
│   │   ├── ECSSD/
│   │   ├── ECSSD.lst
│   │   ├── ...
│   ├── cityscape
│   │   ├── leftImg8bit/train
│   │   ├── leftImg8bit/val
│   │   ├── gtFine/
│   │   ├── train.txt
│   │   ├── val.txt
│   │   ├── ...

Common Image SOD Datasets

We provide a list of common salient object detection datasets.

Name Datasets Short Description Download
SOD Training DUTS-TR 10553 images for SOD training Google Drive / Baidu Drive (psd: w69q)
SOD Testing ECSSD 1000 images for SOD testing
DUT-OMRON 5168 images for SOD testing
DUTS-TE 5019 images for SOD testing
HKU-IS 4447 images for SOD testing
PASCAL-S 850 images for SOD testing

Common Image Semantic Segmentation Datasets

We provide a list of common image semantic segmentation datasets.

Name Datasets Short Description Download
Training Cityscape-Train 2975 images for semantic segmentation training Baidu Drive (psd: weiz)
Validation Cityscape-Val 500 images for semantic segmentation validation

How to Use

The iNAS integrates four main steps of one-shot neural architecture search:

  • Train supernet: Provide a fast performance evaluator for searching.
  • Search models: Find a pareto frontier based on performance evaluator and resource evaluator.
  • Convert weight/Retrain/Finetune: Promote searched model performance to its best. (We now support converting supernet weight to stand-alone models without retraining.)
  • Deploy: Test stand-alone models.

Please see Tutorial.md for the basic usage of those steps in iNAS.

Model Zoo

Task 1: Salient Object Detection

Trained supernet

iNAS Supernet training iteration horizontal flip ECSSD Fm@biggest ECSSD Fm@smallest download
paper 26400 (100 epoch) No 0.948 0.940
iNAS_SOD_100e_noaug 26400 (100 epoch) No 0.952 0.942 Google Drive / Baidu Drive (11tn)
iNAS_SOD_100e_aug 26400 (100 epoch) Yes 0.954 0.945 Google Drive / Baidu Drive (5ktg)
iNAS_SOD_200e_aug 52700 (200 epoch) Yes 0.955 0.948 Google Drive / Baidu Drive (vp8q)

Searched models on Intel Core CPU

iNAS Specialized Sub-nets ECSSD Fm/MAE #Params #Flops #Latency@CPU Download SalincyMaps
Handcraft Models We provide a speed benchmark of handcraft models and you can download it from Google Drive / Baidu Drive (b34c).
EGNet-R (ICCV19) 0.947/0.037 111.64M 120.85G 791.95ms
ITSD-R (CVPR20) 0.947/0.034 26.47M 9.65G 151.51ms
CSNet (ECCV20) 0.916/0.065 0.14M 0.72G 131.11ms
U2-Net (PR20) 0.943/0.041 1.13M 9.77G 186.53ms
Supernet: iNAS_SOD_100e_noaug Models and logs can be downloaded from Google Drive / Baidu Drive (jgg3).
[email protected][email protected] 0.943/0.036 5.09M 0.45G 29.97ms Google Drive
[email protected][email protected] 0.946/0.034 5.83M 0.58G 33.03ms Google Drive
[email protected][email protected] 0.949/0.034 8.15M 0.69G 38.74ms Google Drive
[email protected][email protected] 0.952/0.031 13.44M 0.85G 49.26ms Google Drive
Supernet: iNAS_SOD_200e_aug Models and logs can be download from Google Drive / Baidu Drive (x60e).
[email protected][email protected] 0.949/0.034 5.15M 0.48G 29.08ms Google Drive
[email protected][email protected] 0.952/0.032 7.48M 0.65G 37.35ms Google Drive
[email protected][email protected] 0.954/0.031 9.23M 0.75G 44.20ms Google Drive
[email protected][email protected] 0.956/0.030 16.29M 1.08G 61.88ms Google Drive

Searched models on Mobile Phone: coming soon

Searched models on GPU: coming soon

Searched models on Embedded Device: coming soon

Task 2: Semantic Segmentation

Trained supernet

iNAS Supernet training iteration Cityscape Val mIOU@biggest Cityscape Val mIOU@smallest download
iNAS_cityscape_8gpu_b3_120k_syncbn 120K 78.8 73.2 Google Drive / Baidu Drive (lr00)

Searched models on Intel Core CPU

iNAS Specialized Sub-nets Cityscape Val (mIOU) #Params #Flops #Latency@CPU Download
Handcraft Models Coming Soon.
ICNet (ECCV18) 67.7 - 28.3G -
SwiftNet (CVPR19) 75.4 - 104G -
NAS Methods Coming Soon.
DF1-Seg-d8 (CVPR19) 72.4 - - -
MobileNetV3 (ICCV19) 72.4 - 9.1G -
FasterSeg (ICLR20) 73.1 - 28.2G -
Supernet: iNAS_cityscape_8gpu_b3_120k_syncbn Coming Soon.

TODO List

  • Support multi-processing search (simply use data-parallel cannot increase search speed).
  • Complete documentations.
  • Add speed benchmark on semantic segmentation.

Citation

If you find this project useful in your research, please consider cite:

@inproceedings{gu2021inas,
  title={iNAS: Integral NAS for Device-Aware Salient Object Detection},
  author={Gu, Yu-Chao and Gao, Shang-Hua and Cao, Xu-Sheng and Du, Peng and Lu, Shao-Ping and Cheng, Ming-Ming},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={4934--4944},
  year={2021}
}

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (cc-by-nc-sa), where only non-commercial usage is allowed. For commercial usage, please contact us.

Acknowledgement

The project structure is borrowed from BasicSR, and parts of implementation and evaluation codes are borrowed from Once-For-All, BASNet and BiSeNet . Thanks for these excellent projects.

Contact

If you have any questions, please email [email protected].