Skip to content

sony/mct_quantizers

Folders and files

NameName
Last commit message
Last commit date

Latest commit

721a86a · Jun 14, 2023

History

40 Commits
Jun 8, 2023
Jun 14, 2023
Jun 12, 2023
May 11, 2023
May 11, 2023
Jun 8, 2023
May 30, 2023
May 21, 2023
May 28, 2023
May 28, 2023

Repository files navigation

Model Compression Toolkit (MCT) Quantizers

The MCT Quantizers library is an open-source library developed by researchers and engineers working at Sony Semiconductor Israel.

It provides tools for easily representing a quantized neural network in both Keras and PyTorch. The library offers researchers, developers, and engineers a set of useful quantizers, along with a simple interface for implementing new custom quantizers.

High level description:

The library's quantizers interface consists of two main components:

  1. QuantizationWrapper: This object takes a layer with weights and a set of weight quantizers to infer a quantized layer.
  2. ActivationQuantizationHolder: An object that holds an activation quantizer to be used during inference.

Users can set the quantizers and all the quantization information for each layer by initializing the weights_quantizer and activation_quantizer API.

Please note that the quantization wrapper and the quantizers are framework-specific.

Quantizers:

The library provides the "Inferable Quantizer" interface for implementing new quantizers. This interface is based on the BaseInferableQuantizer class, which allows the definition of quantizers used for emulating inference-time quantization.

On top of BaseInferableQuantizer the library defines a set of framework-specific quantizers for both weights and activations:

  1. Keras Quantizers
  2. Pytorch Quantizers

The mark_quantizer Decorator

The @mark_quantizer decorator is used to assign each quantizer with static properties that define its task compatibility. Each quantizer class should be decorated with this decorator, which defines the following properties:

  • QuantizationTarget: An Enum that indicates whether the quantizer is intended for weights or activations quantization.
  • QuantizationMethod: A list of quantization methods (Uniform, Symmetric, etc.).
  • quantizer_type: An optional property that defines the type of the quantization technique. This is a helper property that allows the creation of advanced quantizers for specific tasks.

Getting Started

This section provides a quick guide to getting started. We begin with the installation process, either via source code or the pip server. Then, we provide a short example of usage.

Installation

Please refer to the MCT install guide for installing the pip package or building from the source.

From Source

git clone https://github.com/sony/mct_quantizers.git
python setup.py install

From PyPi - nightly package

Currently, only a nightly released package (unstable) is available via PyPi.

pip install mct-quantizers-nightly

Requirements

To use MCT Quantizers, you need to have one of the supported frameworks, Tensorflow or PyTorch, installed.

For use with Tensorflow, please install the following packages: tensorflow, tensorflow-model-optimization

For use with PyTorch, please install the following package: torch

You can also use the requirements file to set up your environment.

License

Apache License 2.0.