Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

swin-transformer quantization readme files changes #675

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

swin-transformer quantization readme files changes #675

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e788c43
Update swin_guide.md
Mhhhaster Jun 16, 2023
5bc031f
Update SwinTransformerWeightTransposeQKVWeight.py
Mhhhaster Jun 16, 2023
c877548
Update README.md
Mhhhaster Jun 16, 2023
f498c45
Merge branch 'NVIDIA:main' into main
Mhhhaster Jul 3, 2023
0633cca
Merge branch 'NVIDIA:main' into main
Mhhhaster Aug 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/swin_guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -219,10 +219,10 @@ pip install timm==0.4.12
pip install termcolor==1.1.0

bash -x run_test_v1_int8.sh <batch_size> ##profile of swin-v1 INT8 model
bash -x run_test_v1_int8_accuracy.sh <batch_size> ##test accuracy of swin-v1 INT8 model
bash -x run_test_v1_int8_accuracy.sh <imagenet-path> ##test accuracy of swin-v1 INT8 model

bash -x run_test_v2_int8.sh <batch_size> ##profile of swin-v2 INT8 model
bash -x run_test_v2_int8_accuracy.sh <batch_size> ##test accuracy of swin-v2 INT8 model
bash -x run_test_v2_int8_accuracy.sh <imagenet-path> ##test accuracy of swin-v2 INT8 model
```
Note: When testing PTQ accuracy for INT8 swin-v1-LARGE, we have to specify `--int8-mode 2` instead of `--int8-mode 1` in **run_test_int8.sh**.

Expand Down Expand Up @@ -517,4 +517,4 @@ On chips with Ampere architectures (like A30, A100), user can use `export NVIDIA
| BASE | 16 | 256 | 19.27 | 14.97 | 1.29 |
| BASE | 24 | 384 | - | 135.38 | - |
| LARGE | 16 | 256 | 31.37 | 24.20 | 1.30 |
| LARGE | 24 | 384 | - | - | - |
| LARGE | 24 | 384 | - | - | - |
4 changes: 2 additions & 2 deletions examples/pytorch/swin/Swin-Transformer-Quantization/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ For example, to evaluate the `Swin-T` with a single GPU. You can see **run.sh**
python -m torch.distributed.launch --nproc_per_node 1 \
--master_port 12345 main.py \
--eval \
--cfg SwinTransformer/configs/swin_tiny_patch4_window7_224.yaml \
--cfg SwinTransformer/configs/swin/swin_tiny_patch4_window7_224.yaml \
--resume ./calib-checkpoint/swin_tiny_patch4_window7_224_calib.pth \
--data-path <imagenet-path> \
--int8-mode 1\
Expand Down Expand Up @@ -214,4 +214,4 @@ python -m torch.distributed.launch --nproc_per_node 4 \
--batch-size 128 \
--num-epochs 5 \
--qat-lr 1e-5
```
```
4 changes: 3 additions & 1 deletion examples/pytorch/swin/SwinTransformerWeightTransposeQKVWeight.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,9 @@ def __init__(self, layer_num, window_size, depths, num_heads, ths_path, weights=
if version == 2:
logit_scale_name = 'layers.{}.blocks.{}.attn.logit_scale'.format(layer_idx, block_idx)
if logit_scale_name in weights:
self.weights.append(torch.clamp(weights[logit_scale_name], max=torch.log(torch.tensor(1. / 0.01))).exp())
device = weights[logit_scale_name].device
max_value = torch.log(torch.tensor(1. / 0.01)).to(device)
self.weights.append(torch.clamp(weights[logit_scale_name], max=max_value).exp())
else:
print("[ERROR][SwinTransformerWeights::__init__] missing weight {}.".format(logit_scale_name))
exit(-1)
Expand Down