OpenNMT-tf 1.14.0

OpenNMT-tf 1.14.0

New features

• Multi source Transformer architecture with serial attention layers (see the example model configuration)
• Inference now accepts the parameter bucket_width: if set, the data will be sorted by length to increase the translation efficiency. The predictions will still be outputted in order as they are available. (Enabled by default when using automatic configuration.)

Fixes and improvements

• Improve greedy decoding speed (up to 50% faster)
• When using onmt-update-vocab with the merge directive, updated vocabulary files will be saved in the output directory alongside the updated checkpoint

OpenNMT-tf 1.13.1

OpenNMT-tf 1.13.1

Fixes and improvements

• Fix error when building an inference graph including a DenseBridge

OpenNMT-tf 1.13.0

OpenNMT-tf 1.13.0

New features

• RNMT+ decoder
• Parameter gradients_accum to accumulate gradients and delay parameters update
• Expose lower-level decoder APIs:
  • Decoder.step_fn: returns a callable and an initial state to run step by step decoding
  • Decoder.decode_from_inputs: decodes from full inputs (e.g. embeddings)

Fixes and improvements

• Make learning rate decay configuration more generic: parameters can be set via a decay_params map which allows using more meaningful parameters name (see this example configurations)
• By default, auto-configured Transformer models will accumulate gradients to simulate a training with 8 synchronous replicas (e.g. if you train with 4 GPUs, the gradients of 2 consecutive steps will be accumulated)

OpenNMT-tf 1.12.0

OpenNMT-tf 1.12.0

New features

• The command line argument --checkpoint_path can be used to load the weights of an existing checkpoint while starting from a fresh training state (i.e. with new learning rate schedule and optimizer variables)
• Parameter minimum_decoding_length to constrain the minimum length of decoded sequences

Fixes and improvements

• Major refactoring of dynamic decoding internals: decoding loops are now shared between all decoders that should only implement a step function
• Move event files of external evaluators to the eval/ subdirectory
• Report non normalized hypotheses score for clarity

OpenNMT-tf 1.11.0

OpenNMT-tf 1.11.0

New features

• onmt-convert-checkpoint script to convert checkpoints from one data type to another (e.g. train with FP16 but export in FP32)
• Additional output options for the score run type:
  • with_token_level to output the score of each token
  • with_alignments to output the source-target alignments
• Display the package version by running onmt-main -v

Fixes and improvements

• Fix error in SelfAttentionDecoder when memory is not defined (e.g. in LM tasks)
• Fix UnicodeDecodeError when printing predictions on the standard output in Docker containers
• Force onmt-update-vocab script to run on CPU
• Raise error if distributed training is configured but the train_and_eval run type is not used

OpenNMT-tf 1.10.1

OpenNMT-tf 1.10.1

Fixes and improvements

• Fix possible error when loading checkpoints without --model_type or --model after updating to a newer OpenNMT-tf version. The saved model description is now more future-proof regarding model class updates.
• Fix embedding visualization when the vocabulary file is stored in the model directory or when a joint vocabulary is used
• Improve encoder/decoder states compatibility check

OpenNMT-tf 1.10.0

OpenNMT-tf 1.10.0

New features

• --auto_config flag to use automatic training configuration values (e.g. optimizer, learning rate, batch size). For compatible models, the automatic values aim to deliver solid performance out of the box.
• Include all tokenization assets in exported models

Fixes and improvements

• Fix type error during evaluation and inference of FP16 Transformer models
• Update the model serving example to use a real pretrained model with the TensorFlow Serving 1.11 GPU Docker image
• Small training speed improvement when the optimizer implements sparse updates
• Revise some default configuration values:
  • change bucket_width default value to 1 (from 5)
  • change inference batch_size default value to 16 (from 1)

OpenNMT-tf 1.9.0

OpenNMT-tf 1.9.0

New features

• Mixed precision training of Transformer models
• Command line option --export_dir_base to configure the destination directory of manually exported models

Fixes and improvements

• Fix error when loading model configuration containing the OpenNMTTokenizer tokenizer
• Include OpenNMTTokenizer subword models in the graph assets

OpenNMT-tf 1.8.1

OpenNMT-tf 1.8.1

Fixes and improvements

• Fix backward incompatible change made to Model.__call__ output types

OpenNMT-tf 1.8.0

OpenNMT-tf 1.8.0

New features

• Guided alignment for models using SelfAttentionDecoder and AttentionalRNNDecoder
• with_scores inference option to also output the prediction score
• with_alignments inference option to also output the source-target alignments

Fixes and improvements

• SelfAttentionDecoder defines the first attention head of the last layer as its source-target attention vector