Feature Request: Strategy for generating all of the nn.Module builtin layers

[qthequartermasterman]

There are many torch.nn.Module defined in the docs: https://pytorch.org/docs/stable/nn.html

We should support (or already support):

• Containers
• Convolution Layers
• Pooling layers
• Padding Layers
• Non-linear Activations (weighted sum, nonlinearity)
• Non-linear Activations (other)
• Normalization Layers
• Recurrent Layers
• Transformer Layers
  • Although we do not yet support native Transformer layers, we do support a strategy that generates arbitrary huggingface Transformers models by parsing their corresponding config objects.
• Linear Layers
• Dropout Layers
• Sparse Layers
• Distance Functions
• Loss Functions
• Vision Layers
• Shuffle Layers
• DataParallel Layers (multi-GPU, distributed)
• Utilities
• Quantized Functions
• Lazy Modules Initialization