Results on "imagenet-sketch" batch size and other hyperparameter settings

[arkel23]

https://github.com/rwightman/pytorch-image-models/blob/master/results/results-sketch.csv
Are all models pre-trained and fine-tuned using the same methodology (batch size, optimizer and LR and so on), or are they directly trained on that dataset? Are there any details regarding this process? I would like to use the results as a baseline reference but I need these details. Thanks.

[rwightman]

@arkel23 the models are not trained on any of the test or validation sets. Only the traditional ImageNet train set. They are trained on ImageNet with quite a variety of h-params and come from a wide variety of sources, incl a growing number trained by myself.

[rwightman]

The best training checkpoints are typically selected using top-1 accuracy score on the ImageNet validation set. The labels for the ImageNet test set aren't readily available (I've never found them), so I can't use that for test. But I do use the out-of-distribution test sets like Sketch, ImageNetV2, etc. Unlike validation, those test sets are not used to select from training checkpoints.

[rwightman]

And one more thing on this, a few of the models -- the ones that unsurprisingly do much better on test sets like Sketch -- were trained w/ access to larger datasets than ImageNet-1k. There are a few pretrained on ImageNet-21/22k and some trained in a semi-supervised or unsupervised fashion on even larger datasets like Instagram 1 billion hash tags, etc