String parser for CNNs construction

[pakozm]

Current syntax in all_all MLPs (ann.mlp.generate.all_all):

120 inputs 20 logistic 12 tanh 40 softmax

The new format should work with linear projection layers (as dense or kernels keywords?) and activation function layers. For instance:

120 inputs 20 dense logistic 12 dense tanh 40 dense softmax

would splitted by the parser into:

120 inputs | 20 dense | logistic | 12 dense | tanh | 40 dense | softmax

CNNs would need:

• data dimensionality structure
• kernels
• pooling layers

Something like this:

61x61 inputs 8 3x3 kernels logistic 3x2 maxpool 20 4x4 kernels logistic 256 dense logistic 61 dense softmax

The parser interprets strings like:

12

12x5

in general numberXnumberXnumber... assuming it is a matrix where number of dimension and its sizes is given by the previous format. Additionally, the stride of the kernel can be added as follows:

6x6+1+1

A greedy parser can traverse in left-to-right the string, storing the last dimension sizes and strides.

Example:

61x61 inputs builds a window/matrix with 2 dimensions

8 6x6 kernels builds 8 kernels with a receptive field of 6x6

We need two use cases:

• giving the parser a string, it returns an ANN to train (an ANN component).
• giving the parser a string and an ANN (usually trained), it returns another ANN which can be used to speed-up evaluation phase.

Syntax of the format:

CNN -> BEGIN LAYER+
BEGIN -> MATRIX inputs
LAYER -> NUM MATRIX kernels
        MATRIX maxpool
        NUM dense
        ACTF
ACTF -> logistic | tanh | softmax | ...
any of the activation functions in Lua table ann.components.actf

• A matrix is promoted to a vector when dense is applied.
• NUM MATRIX kernels builds NUM times a kernel with receptive field size given by MATRIX.
• ACTF applied to a matrix or vector keeps its shape.