Skip to content

Commit

Permalink
Update dints bundle (#353)
Browse files Browse the repository at this point in the history
### Description
Update dints bundle with re-trained checkpoints with new implementation
of dints in monai.

### Status
Ready

### Please ensure all the checkboxes:
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Codeformat tests passed locally by running `./runtests.sh
--codeformat`.
- [ ] In-line docstrings updated.
- [ ] Update `version` and `changelog` in `metadata.json` if changing an
existing bundle.
- [ ] Please ensure the naming rules in config files meet our
requirements (please refer to: `CONTRIBUTING.md`).
- [ ] Ensure versions of packages such as `monai`, `pytorch` and `numpy`
are correct in `metadata.json`.
- [ ] Descriptions should be consistent with the content, such as
`eval_metrics` of the provided weights and TorchScript modules.
- [ ] Files larger than 25MB are excluded and replaced by providing
download links in `large_file.yml`.
- [ ] Avoid using path that contains personal information within config
files (such as use `/home/your_name/` for `"bundle_root"`).

---------

Signed-off-by: dongyang0122 <[email protected]>
Signed-off-by: Yiheng Wang <[email protected]>
Co-authored-by: Yiheng Wang <[email protected]>
Co-authored-by: Yiheng Wang <[email protected]>
  • Loading branch information
3 people authored Apr 13, 2023
1 parent 8cd35da commit 69c5562
Show file tree
Hide file tree
Showing 8 changed files with 33 additions and 27 deletions.
3 changes: 2 additions & 1 deletion models/pancreas_ct_dints_segmentation/configs/evaluate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ validate#handlers:
- val_mean_dice
batch_transform: "$monai.handlers.from_engine(['image_meta_dict'])"
summary_ops: "*"
evaluating:
initialize:
- "$setattr(torch.backends.cudnn, 'benchmark', True)"
run:
- "$@validate#evaluator.run()"
5 changes: 3 additions & 2 deletions models/pancreas_ct_dints_segmentation/configs/inference.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ input_channels: 1
output_classes: 3
arch_ckpt_path: "$@bundle_root + '/models/search_code_18590.pt'"
arch_ckpt: "$torch.load(@arch_ckpt_path, map_location=torch.device('cuda'))"
bundle_root: "/workspace/MONAI/model-zoo/models/pancreas_ct_dints_segmentation"
bundle_root: "."
output_dir: "$@bundle_root + '/eval'"
dataset_dir: "/workspace/data/msd/Task07_Pancreas"
data_list_file_path: "$@bundle_root + '/configs/dataset_0.json'"
Expand Down Expand Up @@ -111,6 +111,7 @@ evaluator:
postprocessing: "@postprocessing"
val_handlers: "@handlers"
amp: true
evaluating:
initialize:
- "$setattr(torch.backends.cudnn, 'benchmark', True)"
run:
- "[email protected]()"
9 changes: 5 additions & 4 deletions models/pancreas_ct_dints_segmentation/configs/metadata.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{
"schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json",
"version": "0.3.6",
"version": "0.3.7",
"changelog": {
"0.3.7": "re-train model with updated dints implementation",
"0.3.6": "black autofix format and add name tag",
"0.3.5": "restructure readme to match updated template",
"0.3.4": "correct typos",
Expand All @@ -14,9 +15,9 @@
"0.1.0": "complete the model package",
"0.0.1": "initialize the model package structure"
},
"monai_version": "1.0.1",
"pytorch_version": "1.13.0",
"numpy_version": "1.21.2",
"monai_version": "1.2.0rc4",
"pytorch_version": "1.13.1",
"numpy_version": "1.22.2",
"optional_packages_version": {
"fire": "0.4.0",
"nibabel": "4.0.1",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,16 @@ train#handlers:
train#trainer#max_epochs: "$400*dist.get_world_size()"
train#trainer#train_handlers: "$@train#handlers[: -2 if dist.get_rank() > 0 else None]"
validate#evaluator#val_handlers: "$None if dist.get_rank() > 0 else @validate#handlers"
training:
initialize:
- "$import torch.distributed as dist"
- "$dist.init_process_group(backend='nccl')"
- "$dist.is_initialized() or dist.init_process_group(backend='nccl')"
- "$torch.cuda.set_device(@device)"
- "$monai.utils.set_determinism(seed=123)"
- "$setattr(torch.backends.cudnn, 'benchmark', True)"
run:
- "$@train#trainer.run()"
- "$dist.destroy_process_group()"
finalize:
- "$dist.is_initialized() and dist.destroy_process_group()"
train_data_partition: "$monai.data.partition_dataset(data=@train_datalist, num_partitions=dist.get_world_size(),
shuffle=True, even_divisible=True,)[dist.get_rank()]"
train#dataset:
Expand Down
6 changes: 3 additions & 3 deletions models/pancreas_ct_dints_segmentation/configs/train.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ input_channels: 1
output_classes: 3
arch_ckpt_path: "$@bundle_root + '/models/search_code_18590.pt'"
arch_ckpt: "$torch.load(@arch_ckpt_path, map_location=torch.device('cuda'))"
bundle_root: "/workspace/MONAI/model-zoo/models/pancreas_ct_dints_segmentation"
bundle_root: "."
ckpt_dir: "$@bundle_root + '/models'"
output_dir: "$@bundle_root + '/eval'"
dataset_dir: "/workspace/data/msd/Task07_Pancreas"
Expand Down Expand Up @@ -348,7 +348,7 @@ validate:
additional_metrics: "@validate#additional_metrics"
val_handlers: "@validate#handlers"
amp: true
training:
initialize:
- "$monai.utils.set_determinism(seed=123)"
- "$setattr(torch.backends.cudnn, 'benchmark', True)"
run:
- "$@train#trainer.run()"
16 changes: 8 additions & 8 deletions models/pancreas_ct_dints_segmentation/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,17 +59,17 @@ Dice score is used for evaluating the performance of the model. This model achie
#### Training Loss
The loss over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)

![Training loss over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_train_4-2.png)
![Training loss over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_train_4-3.png)

#### Validation Dice
The mean dice score over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)

![Validation mean dice score over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_validation_4-2.png)
![Validation mean dice score over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_validation_4-3.png)

### Searched Architecture Visualization
Users can install Graphviz for visualization of searched architectures (needed in custom/decode_plot.py). The edges between nodes indicate global structure, and numbers next to edges represent different operations in the cell searching space. An example of searched architecture is shown as follows:
Users can install Graphviz for visualization of searched architectures (needed in [decode_plot.py](https://github.com/Project-MONAI/tutorials/blob/main/automl/DiNTS/decode_plot.py)). The edges between nodes indicate global structure, and numbers next to edges represent different operations in the cell searching space. An example of searched architecture is shown as follows:

![Example of Searched Architecture](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_searched_arch_example.png)
![Example of Searched Architecture](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_searched_arch_example_1.png)

## MONAI Bundle Commands
In addition to the Pythonic APIs, a few command line interfaces (CLI) are provided to interact with the bundle. The CLI supports flexible use cases, such as overriding configs at runtime and predefining arguments in a file.
Expand All @@ -91,25 +91,25 @@ torchrun --nnodes=1 --nproc_per_node=8 -m scripts.search run --config_file confi
#### Execute training:

```
python -m monai.bundle run training --meta_file configs/metadata.json --config_file configs/train.yaml --logging_file configs/logging.conf
python -m monai.bundle run --config_file configs/train.yaml
```

#### Override the `train` config to execute multi-GPU training:

```
torchrun --nnodes=1 --nproc_per_node=2 -m monai.bundle run training --meta_file configs/metadata.json --config_file "['configs/train.yaml','configs/multi_gpu_train.yaml']" --logging_file configs/logging.conf
torchrun --nnodes=1 --nproc_per_node=8 -m monai.bundle run --config_file "['configs/train.yaml','configs/multi_gpu_train.yaml']"
```

#### Override the `train` config to execute evaluation with the trained model:

```
python -m monai.bundle run evaluating --meta_file configs/metadata.json --config_file "['configs/train.yaml','configs/evaluate.yaml']" --logging_file configs/logging.conf
python -m monai.bundle run --config_file "['configs/train.yaml','configs/evaluate.yaml']"
```

#### Execute inference:

```
python -m monai.bundle run evaluating --meta_file configs/metadata.json --config_file configs/inference.yaml --logging_file configs/logging.conf
python -m monai.bundle run --config_file configs/inference.yaml
```

#### Export checkpoint for TorchScript
Expand Down
12 changes: 6 additions & 6 deletions models/pancreas_ct_dints_segmentation/large_files.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
large_files:
- path: "models/search_code_18590.pt"
url: "https://drive.google.com/uc?export=download&id=1VjtVmT0_fMtY3HrNf3UMAoOyhgzjVmOv"
hash_val: "8b212e4a011d7db8d534cbdcac353ad8"
url: "https://drive.google.com/uc?export=download&id=1ZRpQl6NGa6_tSATO7uULIBqxNtOvsy5j"
hash_val: "2ff53724aa9d80d9025034d36785cdb8"
hash_type: "md5"
- path: "models/model.pt"
url: "https://drive.google.com/uc?export=download&id=1okM3vItihWWwwH_uti9UZbZFkft5A-OO"
hash_val: "c7fb16ccb2d4c86194b9b933be7a9d5f"
url: "https://drive.google.com/uc?export=download&id=1p0GXjgUMLvOj3_6seBCtByMOdgWW0Wue"
hash_val: "ce8531b5143b417f65d5b481f72bc911"
hash_type: "md5"
- path: "models/model.ts"
url: "https://drive.google.com/uc?export=download&id=1Swa1L8OV4P3IDlJAf--8WC08fZDPzBBo"
hash_val: "ecafa816fe364e0af86169f618a47980"
url: "https://drive.google.com/uc?export=download&id=1nZHEM81FlAp5-vyk1vVwoBEjbDC9fKd4"
hash_val: "b42221639099a5548c1a0e27772252ec"
hash_type: "md5"
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ ignore =
B905
B028
B907
C419
per_file_ignores = __init__.py: F401, __main__.py: F401
exclude = *.pyi,.git,.eggs,monai/_version.py,versioneer.py,venv,.venv,_version.py

Expand Down

0 comments on commit 69c5562

Please sign in to comment.