NF Net training hparams

[shenfalong]

I am reproducing NF-ResNet50, This model has been released with Top1=80%. However, I have trained the model by myself with Top=72%, I think the training setting is differnet, such as pretrained-weights, dropout, droppath, lr scheduler... could you provide the training log along with the released model? Many thanks.

[bonlime]

You can get some idea for training configs here: https://github.com/rwightman/pytorch-image-models/blob/master/docs/training_hparam_examples.md

[rwightman]

There is a bit in #499 ... you need gradient clipping, either global norm clipping or AGC, you need fairy heavy augmentation and regularization. The NFNet paper describes hparams that will work quite well for NF-ResNet50.