Recurrent is gone for good

CMakeLists.txt

@@ -32,7 +32,6 @@ SET(luasrc
   NormStabilizer.lua
   Padding.lua
   Recurrence.lua
-  deprecated/Recurrent.lua
   RecurrentAttention.lua
   recursiveUtils.lua
   Recursor.lua

deprecated/README.md

@@ -1,87 +1,9 @@
 
 These modules are DEPRECATED:
   * [FastLSTM](#rnn.FastLSTM) : an LSTM with optional support for batch normalization;
-  * [Recurrent](#rnn.Recurrent) (DEPRECATED) : a generalized recurrent neural network container;
   * [LSTM](#rnn.LSTM) : a vanilla Long-Short Term Memory module (uses peephole connections);
   * [SeqLSTMP](#rnn.LSTM) : a vanilla Long-Short Term Memory module (uses peephole connections);
 
-<a name='rnn.Recurrent'></a>
-## Recurrent ##
-
-DEPRECATED : use [Recurrence](#rnn.Recurrence) instead.
-
-References :
- * A. [Sutsekever Thesis Sec. 2.5 and 2.8](http://www.cs.utoronto.ca/~ilya/pubs/ilya_sutskever_phd_thesis.pdf)
- * B. [Mikolov Thesis Sec. 3.2 and 3.3](http://www.fit.vutbr.cz/~imikolov/rnnlm/thesis.pdf)
- * C. [RNN and Backpropagation Guide](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.3.9311&rep=rep1&type=pdf)
-
-A [composite Module](https://github.com/torch/nn/blob/master/doc/containers.md#containers) for implementing Recurrent Neural Networks (RNN), excluding the output layer.
-
-The `nn.Recurrent(start, input, feedback, [transfer, rho, merge])` constructor takes 6 arguments:
- * `start` : the size of the output (excluding the batch dimension), or a Module that will be inserted between the `input` Module and `transfer` module during the first step of the propagation. When `start` is a size (a number or `torch.LongTensor`), then this *start* Module will be initialized as `nn.Add(start)` (see Ref. A).
- * `input` : a Module that processes input Tensors (or Tables). Output must be of same size as `start` (or its output in the case of a `start` Module), and same size as the output of the `feedback` Module.
- * `feedback` : a Module that feedbacks the previous output Tensor (or Tables) up to the `merge` module.
- * `merge` : a [table Module](https://github.com/torch/nn/blob/master/doc/table.md#table-layers) that merges the outputs of the `input` and `feedback` Module before being forwarded through the `transfer` Module.
- * `transfer` : a non-linear Module used to process the output of the `merge` module, or in the case of the first step, the output of the `start` Module.
- * `rho` : the maximum amount of backpropagation steps to take back in time. Limits the number of previous steps kept in memory. Due to the vanishing gradients effect, references A and B recommend `rho = 5` (or lower). Defaults to 99999.
-
-An RNN is used to process a sequence of inputs.
-Each step in the sequence should be propagated by its own `forward` (and `backward`),
-one `input` (and `gradOutput`) at a time.
-Each call to `forward` keeps a log of the intermediate states (the `input` and many `Module.outputs`)
-and increments the `step` attribute by 1.
-Method `backward` must be called in reverse order of the sequence of calls to `forward` in
-order to backpropgate through time (BPTT). This reverse order is necessary
-to return a `gradInput` for each call to `forward`.
-
-The `step` attribute is only reset to 1 when a call to the `forget` method is made.
-In which case, the Module is ready to process the next sequence (or batch thereof).
-Note that the longer the sequence, the more memory that will be required to store all the
-`output` and `gradInput` states (one for each time step).
-
-To use this module with batches, we suggest using different
-sequences of the same size within a batch and calling `updateParameters`
-every `rho` steps and `forget` at the end of the sequence.
-
-Note that calling the `evaluate` method turns off long-term memory;
-the RNN will only remember the previous output. This allows the RNN
-to handle long sequences without allocating any additional memory.
-
-
-For a simple concise example of how to make use of this module, please consult the
-[simple-recurrent-network.lua](examples/simple-recurrent-network.lua)
-training script.
-
-<a name='rnn.Recurrent.Sequencer'></a>
-### Decorate it with a Sequencer ###
-
-Note that any `AbstractRecurrent` instance can be decorated with a [Sequencer](#rnn.Sequencer)
-such that an entire sequence (a table) can be presented with a single `forward/backward` call.
-This is actually the recommended approach as it allows RNNs to be stacked and makes the
-rnn conform to the Module interface, i.e. each call to `forward` can be
-followed by its own immediate call to `backward` as each `input` to the
-model is an entire sequence, i.e. a table of tensors where each tensor represents
-a time-step.
-
-```lua
-seq = nn.Sequencer(module)
-```
-
-The [simple-sequencer-network.lua](examples/simple-sequencer-network.lua) training script
-is equivalent to the above mentionned [simple-recurrent-network.lua](examples/simple-recurrent-network.lua)
-script, except that it decorates the `rnn` with a `Sequencer` which takes
-a table of `inputs` and `gradOutputs` (the sequence for that batch).
-This lets the `Sequencer` handle the looping over the sequence.
-
-You should only think about using the `AbstractRecurrent` modules without
-a `Sequencer` if you intend to use it for real-time prediction.
-Actually, you can even use an `AbstractRecurrent` instance decorated by a `Sequencer`
-for real time prediction by calling `Sequencer:remember()` and presenting each
-time-step `input` as `{input}`.
-
-Other decorators can be used such as the [Repeater](#rnn.Repeater) or [RecurrentAttention](#rnn.RecurrentAttention).
-The `Sequencer` is only the most common one.
-
 <a name='rnn.LSTM'></a>
 ## LSTM ##
 References :

init.lua

@@ -4,7 +4,7 @@ require 'nn'
 
 -- create global rnn table:
 rnn = {}
-rnn.version = 2.4 -- merge dpnn
+rnn.version = 2.5 -- AbstractRecurrent(rho) -> AbstractRecurrent(module)
 
 -- lua 5.2 compat
 
@@ -147,7 +147,6 @@ require('rnn.MaskZeroCriterion')
 -- deprecated modules
 require('rnn.LSTM')
 require('rnn.FastLSTM')
-require('rnn.Recurrent')
 require('rnn.SeqLSTMP')
 
 -- prevent likely name conflicts