Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renaming TPC #1321

Merged
merged 20 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ auto:qat:
- model_compression_toolkit/qat/**

auto:target_platform_capabilities:
- model_compression_toolkit/target_platform_capabilities/**
- model_compression_toolkit/framework_quantization_capabilities/**

auto:trainable_infrastructure:
- model_compression_toolkit/trainable_infrastructure/**
Expand Down
6 changes: 3 additions & 3 deletions docsrc/source/api/api_docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,9 @@ keras_load_quantized_model
- :ref:`keras_load_quantized_model<ug-keras_load_quantized_model>`: A function to load a quantized keras model.


target_platform
================
- :ref:`target_platform<ug-target_platform>`: Module to create and model hardware-related settings to optimize the model according to, by the hardware the optimized model will use during inference.
target_platform_capabilities
==============================
- :ref:`target_platform_capabilities<ug-target_platform_capabilities>`: Module to create and model hardware-related settings to optimize the model according to, by the hardware the optimized model will use during inference.
- :ref:`get_target_platform_capabilities<ug-get_target_platform_capabilities>`: A function to get a target platform model for Tensorflow and Pytorch.
- :ref:`DefaultDict<ug-DefaultDict>`: Util class for creating a TargetPlatformCapabilities.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@


=======================================
Get TargetPlatformCapabilities
Get FrameworkQuantizationCapabilities
=======================================

.. autofunction:: model_compression_toolkit.get_target_platform_capabilities
Expand Down
14 changes: 7 additions & 7 deletions docsrc/source/api/api_docs/modules/layer_filters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,30 +15,30 @@ one may use the next filters to check if a layer configuration holds the created
Attribute Filters
==================

.. autoclass:: model_compression_toolkit.target_platform.AttributeFilter
.. autoclass:: model_compression_toolkit.target_platform_capabilities.AttributeFilter

|

.. autoclass:: model_compression_toolkit.target_platform.Eq
.. autoclass:: model_compression_toolkit.target_platform_capabilities.Eq

|

.. autoclass:: model_compression_toolkit.target_platform.NotEq
.. autoclass:: model_compression_toolkit.target_platform_capabilities.NotEq

|

.. autoclass:: model_compression_toolkit.target_platform.Greater
.. autoclass:: model_compression_toolkit.target_platform_capabilities.Greater

|


.. autoclass:: model_compression_toolkit.target_platform.GreaterEq
.. autoclass:: model_compression_toolkit.target_platform_capabilities.GreaterEq

|


.. autoclass:: model_compression_toolkit.target_platform.Smaller
.. autoclass:: model_compression_toolkit.target_platform_capabilities.Smaller

|

.. autoclass:: model_compression_toolkit.target_platform.SmallerEq
.. autoclass:: model_compression_toolkit.target_platform_capabilities.SmallerEq
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
:orphan:

.. _ug-target_platform:
.. _ug-target_platform_capabilities:


=================================
target_platform Module
=================================
=====================================
target_platform_capabilities Module
=====================================

MCT can be configured to quantize and optimize models for different hardware settings.
For example, when using qnnpack backend for Pytorch model inference, Pytorch `quantization
Expand All @@ -14,7 +14,7 @@ uses `per-tensor weights quantization <https://github.com/pytorch/pytorch/blob/m
for Conv2d, while when using tflite modeling, Tensorflow uses `per-channel weights quantization for
Conv2D <https://www.tensorflow.org/lite/performance/quantization_spec#per-axis_vs_per-tensor>`_.

This can be addressed in MCT by using the target_platform module, that can configure different
This can be addressed in MCT by using the target_platform_capabilities module, that can configure different
parameters that are hardware-related, and the optimization process will use this to optimize the model accordingly.
Models for IMX500, TFLite and qnnpack can be observed `here <https://github.com/sony/model_optimization/tree/main/model_compression_toolkit/target_platform_capabilities>`_, and can be used using :ref:`get_target_platform_capabilities function<ug-get_target_platform_capabilities>`.

Expand All @@ -36,68 +36,43 @@ Now, we will detail about the different components.

QuantizationMethod
==========================
.. autoclass:: model_compression_toolkit.target_platform.QuantizationMethod
.. autoclass:: model_compression_toolkit.target_platform_capabilities.QuantizationMethod



OpQuantizationConfig
======================
.. autoclass:: model_compression_toolkit.target_platform.OpQuantizationConfig
.. autoclass:: model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema.OpQuantizationConfig



AttributeQuantizationConfig
============================
.. autoclass:: model_compression_toolkit.target_platform.AttributeQuantizationConfig
.. autoclass:: model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema.AttributeQuantizationConfig


QuantizationConfigOptions
============================
.. autoclass:: model_compression_toolkit.target_platform.QuantizationConfigOptions
.. autoclass:: model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema.QuantizationConfigOptions


TargetPlatformModel
=======================
.. autoclass:: model_compression_toolkit.target_platform.TargetPlatformModel
TargetPlatformCapabilities
============================
.. autoclass:: model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema.TargetPlatformCapabilities


OperatorsSet
================
.. autoclass:: model_compression_toolkit.target_platform.OperatorsSet
.. autoclass:: model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema.OperatorsSet



Fusing
==============
.. autoclass:: model_compression_toolkit.target_platform.Fusing
.. autoclass:: model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema.Fusing



OperatorSetConcat
OperatorSetGroup
====================
.. autoclass:: model_compression_toolkit.target_platform.OperatorSetConcat


OperationsToLayers
=====================
.. autoclass:: model_compression_toolkit.target_platform.OperationsToLayers


OperationsSetToLayers
=========================
.. autoclass:: model_compression_toolkit.target_platform.OperationsSetToLayers


LayerFilterParams
=========================
.. autoclass:: model_compression_toolkit.target_platform.LayerFilterParams

More filters and usage examples are detailed :ref:`here<ug-layer_filters>`.


TargetPlatformCapabilities
=============================
.. autoclass:: model_compression_toolkit.target_platform.TargetPlatformCapabilities



.. autoclass:: model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema.OperatorSetGroup
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ For example, we can set a trainable weights quantizer with the following configu

.. code-block:: python

from model_compression_toolkit.target_platform_capabilities.target_platform import QuantizationMethod
from model_compression_toolkit.target_platform_capabilities.target_platform_capabilities import QuantizationMethod
from model_compression_toolkit.constants import THRESHOLD, MIN_THRESHOLD

TrainableQuantizerWeightsConfig(weights_quantization_method=QuantizationMethod.SYMMETRIC,
Expand All @@ -79,7 +79,7 @@ For example, we can set a trainable activation quantizer with the following conf

.. code-block:: python

from model_compression_toolkit.target_platform_capabilities.target_platform import QuantizationMethod
from model_compression_toolkit.target_platform_capabilities.target_platform_capabilities import QuantizationMethod
from model_compression_toolkit.constants import THRESHOLD, MIN_THRESHOLD

TrainableQuantizerActivationConfig(activation_quantization_method=QuantizationMethod.UNIFORM,
Expand Down
4 changes: 2 additions & 2 deletions docsrc/source/api/api_docs/notes/tpc_note.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

.. note::
For now, some fields of :class:`~model_compression_toolkit.target_platform.OpQuantizationConfig` are ignored during
For now, some fields of :class:`~model_compression_toolkit.target_platform_capabilities.OpQuantizationConfig` are ignored during
the optimization process such as quantization_preserving, fixed_scale, and fixed_zero_point.

- MCT will use more information from :class:`~model_compression_toolkit.target_platform.OpQuantizationConfig`, in the future.
- MCT will use more information from :class:`~model_compression_toolkit.target_platform_capabilities.OpQuantizationConfig`, in the future.

2 changes: 1 addition & 1 deletion model_compression_toolkit/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# ==============================================================================

from model_compression_toolkit.defaultdict import DefaultDict
from model_compression_toolkit.target_platform_capabilities import target_platform
from model_compression_toolkit import target_platform_capabilities
from model_compression_toolkit.target_platform_capabilities.tpc_models.get_target_platform_capabilities import get_target_platform_capabilities
from model_compression_toolkit import core
from model_compression_toolkit.logger import set_log_folder
Expand Down
4 changes: 1 addition & 3 deletions model_compression_toolkit/core/common/framework_info.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,8 @@
from enum import Enum
from typing import Dict, Any, List



from mct_quantizers import QuantizationMethod
from model_compression_toolkit.defaultdict import DefaultDict
from model_compression_toolkit.target_platform_capabilities.target_platform import QuantizationMethod


class ChannelAxis(Enum):
Expand Down
11 changes: 6 additions & 5 deletions model_compression_toolkit/core/common/fusion/layer_fusing.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@
from typing import Any, List
from model_compression_toolkit.core.common.graph.base_graph import Graph
from model_compression_toolkit.core.common.graph.base_node import BaseNode
from model_compression_toolkit.target_platform_capabilities.target_platform.targetplatform2framework import TargetPlatformCapabilities
from model_compression_toolkit.target_platform_capabilities.target_platform.targetplatform2framework.layer_filter_params import LayerFilterParams
from model_compression_toolkit.target_platform_capabilities.targetplatform2framework.framework_quantization_capabilities import \
FrameworkQuantizationCapabilities
from model_compression_toolkit.target_platform_capabilities.targetplatform2framework.layer_filter_params import LayerFilterParams


def filter_fusing_patterns(fusing_patterns: List[List[Any]], node: BaseNode, idx: int = 0) -> List[List[Any]]:
Expand Down Expand Up @@ -77,18 +78,18 @@ def disable_nodes_activation_quantization(nodes: List[BaseNode]):
qc.activation_quantization_cfg.enable_activation_quantization = False


def fusion(graph: Graph, tpc: TargetPlatformCapabilities) -> Graph:
def fusion(graph: Graph, fqc: FrameworkQuantizationCapabilities) -> Graph:
"""
Fusing defines a list of operators that should be combined and treated as a single operator,
hence no quantization is applied between them when they appear in the graph.
This function search and disable quantization for such patterns.
Args:
graph: Graph we apply the fusion on.
tpc: TargetPlatformCapabilities object that describes the desired inference target platform (includes fusing patterns MCT should handle).
fqc: FrameworkQuantizationCapabilities object that describes the desired inference target platform (includes fusing patterns MCT should handle).
Returns:
Graph after applying fusion activation marking.
"""
fusing_patterns = tpc.get_fusing_patterns()
fusing_patterns = fqc.get_fusing_patterns()
if len(fusing_patterns) == 0:
return graph

Expand Down
33 changes: 17 additions & 16 deletions model_compression_toolkit/core/common/graph/base_graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,9 @@
from model_compression_toolkit.core.common.pruning.pruning_section import PruningSection
from model_compression_toolkit.core.common.user_info import UserInformation
from model_compression_toolkit.logger import Logger
from model_compression_toolkit.target_platform_capabilities.target_platform.targetplatform2framework import \
TargetPlatformCapabilities, LayerFilterParams
from model_compression_toolkit.target_platform_capabilities.targetplatform2framework import LayerFilterParams
from model_compression_toolkit.target_platform_capabilities.targetplatform2framework.framework_quantization_capabilities import \
FrameworkQuantizationCapabilities

OutTensor = namedtuple('OutTensor', 'node node_out_index')

Expand Down Expand Up @@ -86,29 +87,29 @@ def set_fw_info(self,

self.fw_info = fw_info

def set_tpc(self,
tpc: TargetPlatformCapabilities):
def set_fqc(self,
fqc: FrameworkQuantizationCapabilities):
"""
Set the graph's TPC.
Set the graph's FQC.
Args:
tpc: TargetPlatformCapabilities object.
fqc: FrameworkQuantizationCapabilities object.
"""
# validate graph nodes are either from the framework or a custom layer defined in the TPC
# Validate graph nodes are either built-in layers from the framework or custom layers defined in the TPC
tpc_layers = tpc.op_sets_to_layers.get_layers()
tpc_filtered_layers = [layer for layer in tpc_layers if isinstance(layer, LayerFilterParams)]
# validate graph nodes are either from the framework or a custom layer defined in the FQC
# Validate graph nodes are either built-in layers from the framework or custom layers defined in the FQC
fqc_layers = fqc.op_sets_to_layers.get_layers()
fqc_filtered_layers = [layer for layer in fqc_layers if isinstance(layer, LayerFilterParams)]
for n in self.nodes:
is_node_in_tpc = any([n.is_match_type(_type) for _type in tpc_layers]) or \
any([n.is_match_filter_params(filtered_layer) for filtered_layer in tpc_filtered_layers])
is_node_in_fqc = any([n.is_match_type(_type) for _type in fqc_layers]) or \
any([n.is_match_filter_params(filtered_layer) for filtered_layer in fqc_filtered_layers])
if n.is_custom:
if not is_node_in_tpc:
if not is_node_in_fqc:
Logger.critical(f'MCT does not support optimizing Keras custom layers. Found a layer of type {n.type}. '
' Please add the custom layer to Target Platform Capabilities (TPC), or file a feature '
' Please add the custom layer to Framework Quantization Capabilities (FQC), or file a feature '
'request or an issue if you believe this should be supported.') # pragma: no cover
if any([qc.default_weight_attr_config.enable_weights_quantization for qc in n.get_qco(tpc).quantization_configurations]):
if any([qc.default_weight_attr_config.enable_weights_quantization for qc in n.get_qco(fqc).quantization_configurations]):
Logger.critical(f'Layer identified: {n.type}. MCT does not support weight quantization for Keras custom layers.') # pragma: no cover

self.tpc = tpc
self.fqc = fqc

def get_topo_sorted_nodes(self):
"""
Expand Down
Loading
Loading