Completely transition of MO to Interpolate-4 (#970)
* Refactored infer function and function supported_attrs for the layer Interpolate. * Small change. * Deleted unneeded checks in transformations ResizeToInterpolate2D and ResizeToInterpolate3D. * Small fix in the extractor of ONNX Resize. * Now the extractor of TF ResizeBilinear generates Interpolate-1 again, because 'axis' in final version of Interpolate-4 specification is an input but is not attribute. * Now the extractor of TF ResizeNearest generates Interpolate-1 again, because 'axis' in final version of Interpolate-4 specification is an input but is not attribute. * Added static method get_axis into class Interpolate. * Refactored class CanBeFused in the transformation InterpolateSequenceToInterpolate. * Fixed transformation InterpolateSequenceToInterpolate according to the last version of the specification of Interpolate-4. * Started to write support of Interpolate-4 in the transformation InterpolateWithConcat. * Added support for Interpolate-4 into the transformation InterpolateWithConcat. * Added support for Interpolate-4 into the transformation InterpolateConcat. * Added support for Interpolate-4 into the transformation InterpolateReshapeWA. * Added support for Interpolate-4 into the transformation InterpolateTranspose. * Started to add test for opset4 case of the transformation InterpolateSequenceToInterpolate. * Added test for InterpolateSequenceToInterpolate (test_2d_interpolate_sequence_1_opset4_case). * Added test for InterpolateSequenceToInterpolate (test_2d_interpolate_sequence_4_opset4_case). * Added another test for InterpolateSequenceToInterpolate (test_2d_interpolate_sequence_5_opset4_case). * Added another test for InterpolateSequenceToInterpolate (test_3d_interpolate_sequence_1_opset4_case). * Finished addition of tests for opset4 case of InterpolateSequenceToInterpolate. * Small change. * Now opset is only opset1 or opset4 in the transformation InterpolateTranspose. * Small fixes in transformations ResizeToInterpolate2D and ResizeToInterpolate3D. * Deleted reading of unused ONNX attributes. * Fixed docstring of the transformation InterpolateV1ToInterpolateV4. * Added node name in assert about axes input. * Fixes in the definition of the operation ONNXResize11. * Now Interpolate-4 cannot have 'extension' as opset. * Now the transformation InterpolateV1ToInterpolateV4 uses find_and_replace_pattern but not replace_sub_graph. * Fixed tests for transformations InterpolateReshapeWA and InterpolateConcat. * Fixed some tests. * Rewritten operation Interpolate-4 class according to new variant of documentation. * Some fixes in ONNXResize11 operation class. * Now the transformation ONNXResize11ToInterpolate generates Interpolate-4 with 4 inputs. * Now the transformation UpsampleToResample generates Interpolate-4 with 4 inputs. * Now the transformation NearestNeighborUpsampling generates Interpolate-4 with 4 inputs. * Now transformations ResizeToInterpolate2D and ResizeToInterpolate3D generate Interpolate-4 with 4 inputs. * Now the transformation SplitConcatPairToInterpolate generates Interpolate-4 with 4 inputs. * Now the transformation UnsqueezeTileReshapeBlockToInterpolate generates Interpolate-4 with 4 inputs. * Now the transformation InterpolateV1ToInterpolateV4 generates Interpolate-4 with 4 inputs. * Some fixes. * Fixed the transformation InterpolateSequenceToInterpolate according to new variant of Interpolate-4 specification. * Fixed typos. * Added shape_calculation_mode to supported_attrs. * Small fixes. * Added operation ONNXResize10 and the transformation ONNXResize10ToInterpolate4. * Fixed function correct_scales_using_dst_shape. * Some fixes in InterpolateSequenceToInterpolate. * Fixed bug in the method __call__ of the class CanBeFused: now self.accumulated_axes is correctly cleared in all cases. * Small change. * Fixed tests for the transformation SplitConcatPairToInterpolate. * Now transformations InterpolateWithConcat, InterpolateReshapeWA, InterpolateConcat support Interpolate-4. * Fixed the transformation InterpolateTranspose for the case of Interpolate-4. * Written the back transformation InterpolateV4AxesCorrection to convert 'axes' input of Interpolate-4 from NHWC to NCHW layout. * Added PermuteInput in Interpolate-4 infer. * Fixed typos. * Deleted the transformation InterpolateAxesCorrection. * Now Interpolate-4 permutes axis, not shape in input port 3. * Small fix. * Some fix. * Fixed bug in the transformation UpsampleToResample. * Added some debug prints. * Added more debug prints. * Now ONNX Upsample-9 operation is read as ONNXResize10. * Small fix. * Small fixes. * Fixed tests for the transformation SplitConcatPairToInterpolate. * Deleted debug prints. * Deleted some debug prints. * Fixes in the transformation UnsqueezeTileReshapeBlockToInterpolate and its tests. * Small fix in the transformation InterpolateSequenceToInterpolate. * Started to write nGraph transformation to convert Interpolate-1 to Interpolate-4. * Deleted redundant files. * Small fixes. * Small fix. * Written draft of the transformation Interpolate-1 -> Interpolate-4. * Small fix. * Now ONNX Importer reads Resize-10 as Interpolate-4. * Fixes in the test onnx_model_resize10_import_only. * Small fix in the test for the conversion Interpolate-1 -> Interpolate-4. * Small fixes. * Fixed NGraphReaderTests for Interpolate. * Some fixes. * Deleted class for Resample operation. * Fix in the transformation NearestNeighborUpsampling: fixed precision of the input 'scales' of generated Interpolate-4. * Fixed typo. * Now the TF operations ResizeBilinear is readed as internal MO operation TFResizeBilinear. This internal operation is converted into Interpolate-4. * Small fix in BOM-file. * Added checks of existence of attributes of TF ResizeBilinear operation. * Small fixes in the conversion of the internal MO operation TFResizeBilinear to Interpolate-4. * Small fixes. * Small fixes. * Now the transformation ONNXResize10ToInterpolateV4 calculates sizes input as input_shape * (scales + epsilon). * Added the internal MO operation TFResizeNearestNeighbor. * Fixes in the transformation SplitConcatPairToInterpolate and its tests. * Fixes in the transformation UnsqueezeTileReshapeBlockToInterpolate and its tests. * Written the transformation that converts the internal operation TFResizeNearestNeighbor into Interpolate-4. * Now MO reads the TF operation ResizeNearestNeighbor as the internal MO operation TFResizeNearestNeighbor. * Small fix. * Now the specification of Interpolate-4 clarifies that the mode linear_onnx supports only 2D or 4D input tensors. * Small fix. * Some fixes. * Moved the transformation ONNXResize10ToInterpolateV4 to the front stage. * Deleted infer function and function supported_attrs for ONNXResize10 operation. * Deleted supported_attrs() for TFResizeBilinear and TFResizeNearestNeighbor. * Some fixes. * Fixes in the shape infer function of the nGraph operation Interpolate-4. Now 'axes' input can be non-constant. In the such case, all elements of the output shape are Dimension::dynamic(). * Deleted corner cases processing in transformations TFResizeBilinearToInterpolateV4 and TFResizeNearestNeighborToInterpolateV4. * Rewritten the function replace_resize_bilinear. * Written inner MO operation TFResize that covers TF operations ResizeBilinear and ResizeNearestNeighbor. * Now TF operations ResizeBilinear and ResizeNearestNeighbor are read as an internal operation TFResize in MO. Transformations TFResizeNearestNeighborToInterpolateV4 and TFResizeBilinearToInterpolateV4 are fused into one transformation TFResizeToInterpolateV4. * Some changes in the shape infer function of nGraph op Interpolate-4. * Small fix. * Some changes. * The transformation TFResizeToInterpolateV4 is moved to the front stage. * Deleted redundant assert. * Deleted transformations ResizeToInterpolate2D and ResizeToInterpolate3D. * Some renaming. * Small change. * Deleted .copy() in the shape infer function of the internal operation TFResize. * Small fix. * Small fixes. * Added comment about the case when the input 'axes' of Interpolate-4 is non-constant. * Written test for Interpolate-4 shape infer, for the case when the input 'axes' is non-constant and shape_calculation_mode = scales. * Some fixes. * Small fixes. * Small fix. * Added yet another test for the case of non-constant 'axes' input of Interpolate-4 (when shape_calculation_mode = sizes). * Added some comment. * Small fix. * Reverted changes for InterpolateWithConcat. * Added type checks for all inputs of nGraph operation Interpolate-4. * Added u32 and u64 to supported element types of sizes and axes inputs of nGraph operation Interpolate-4. * Fixed some functional tests. * Some changes. * Added helper function float32_array. * Now the MO transformation InterpolateV1ToInterpolate preserves names of layers. * Small fix. * Small fix. * Reverted some change. * Small fixes. * Small fix. * Small fix. * Small fix. * Small fix. * Reverted changes in the nGraph reader tests for Interpolate-1. * Some revert. * Fixed some copyright year.
This commit is contained in:
Vladimir Gavrilov authored and 
GitHub committed
1 parent
bd3884b602
commit
c1136cd7b0
34 files changed
+1156
-564
No files matched your search
@@ -0,0 +1,31 @@
|
||||
"""
|
||||
Copyright (C) 2018-2020 Intel Corporation
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
"""
|
||||
|
||||
|
||||
from mo.graph.graph import Node, Graph
|
||||
from mo.ops.op import Op
|
||||
|
||||
|
||||
class ONNXResize10(Op):
|
||||
op = 'ONNXResize10'
|
||||
|
||||
def __init__(self, graph: Graph, attrs: dict):
|
||||
mandatory_props = {
|
||||
'op': self.op,
|
||||
'in_ports_count': 2,
|
||||
'out_ports_count': 1,
|
||||
}
|
||||
super().__init__(graph, mandatory_props, attrs)
|
||||
@@ -0,0 +1,67 @@
|
||||
"""
|
||||
Copyright (C) 2018-2020 Intel Corporation
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
"""
|
||||
|
||||
from mo.front.common.layout import get_height_dim, get_width_dim
|
||||
from mo.front.common.partial_infer.utils import int64_array
|
||||
from mo.graph.graph import Node, Graph
|
||||
from mo.ops.op import Op
|
||||
|
||||
|
||||
class TFResize(Op):
|
||||
op = 'TFResize'
|
||||
|
||||
def __init__(self, graph: Graph, attrs: dict):
|
||||
mandatory_props = {
|
||||
'op': self.op,
|
||||
'out_ports_count': 1,
|
||||
'in_ports_count': 2,
|
||||
'infer': TFResize.tf_resize_infer
|
||||
}
|
||||
super().__init__(graph, mandatory_props, attrs)
|
||||
|
||||
@staticmethod
|
||||
def tf_resize_infer(node: Node):
|
||||
input_shape = node.in_port(0).data.get_shape()
|
||||
if input_shape is None:
|
||||
return
|
||||
|
||||
attrs_msg = "If half_pixel_centers attribute of the node {} with op {} is True, " \
|
||||
"the attribute align_corners must be False"
|
||||
node_name = node.soft_get('name', node.id)
|
||||
assert not node.half_pixel_centers or (node.half_pixel_centers and not node.align_corners), \
|
||||
attrs_msg.format(node_name, node.op)
|
||||
|
||||
connected_in_ports = [port for port in node.in_ports().values() if not port.disconnected()]
|
||||
assert len(connected_in_ports) == 2, \
|
||||
"Node {} with op {} number of inputs must be equal to 2.".format(node_name, node.op)
|
||||
|
||||
new_sizes_value = node.in_port(1).data.get_value()
|
||||
assert new_sizes_value is not None, "Node {} with op {} has no value in input port 1".format(node_name, node.op)
|
||||
|
||||
input_rank = len(input_shape)
|
||||
assert input_rank == 4, \
|
||||
"Resized input data of the node {} with op {} must be 4D tensor".format(node_name, node.op)
|
||||
|
||||
len_msg = "Op {} with name {} supports only resize with respect to height and width dimension simultaneously"
|
||||
assert len(new_sizes_value) == 2, len_msg.format(node_name, node.op)
|
||||
|
||||
output_shape = int64_array(input_shape.copy())
|
||||
|
||||
layout = node.graph.graph['layout']
|
||||
output_shape[get_height_dim(layout, input_rank)] = new_sizes_value[0]
|
||||
output_shape[get_width_dim(layout, input_rank)] = new_sizes_value[1]
|
||||
|
||||
node.out_port(0).data.set_shape(output_shape)
|
||||
@@ -22,6 +22,7 @@ import numpy as np
|
||||
from mo.front.common.partial_infer.utils import int64_array
|
||||
from mo.front.extractor import bool_to_str
|
||||
from mo.graph.graph import Node, Graph
|
||||
from mo.graph.perm_inputs import PermuteInputs
|
||||
from mo.ops.op import Op, PermuteAttrs
|
||||
|
||||
|
||||
@@ -61,6 +62,9 @@ def infer_for_opset4(node: Node):
|
||||
for i, axis in enumerate(axes):
|
||||
output_shape[axis] = math.floor(scales[i] * output_shape[axis] + 1.0e-5)
|
||||
|
||||
if node.is_in_port_connected(3):
|
||||
PermuteInputs().set_input_permutation(node.in_node(3), node, 'input:0', 'axis')
|
||||
|
||||
node.out_port(0).data.set_shape(output_shape)
|
||||
|
||||
|
||||
@@ -103,7 +107,8 @@ def correct_scales_using_dst_shape(node, dst_shape, src_shape, axes):
|
||||
if scales_value is None or len(scales_value) != len(dst_shape):
|
||||
corrected_scales = np.zeros(len(dst_shape))
|
||||
for i, axis in enumerate(list(axes)):
|
||||
corrected_scales[i] = math.floor((dst_shape[i] / src_shape[axis]) + 1.0e-5)
|
||||
corrected_scales[i] = dst_shape[i] / src_shape[axis]
|
||||
node.in_port(2).data.set_value(corrected_scales)
|
||||
|
||||
|
||||
class Interpolate(Op):
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
"""
|
||||
Copyright (C) 2020 Intel Corporation
|
||||
Copyright (C) 2018-2020 Intel Corporation
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
|
||||
@@ -1,102 +0,0 @@
|
||||
"""
|
||||
Copyright (C) 2018-2020 Intel Corporation
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
"""
|
||||
|
||||
import logging as log
|
||||
|
||||
from extensions.ops.resize_factor_utils import factor_update
|
||||
from mo.front.common.layout import get_batch_dim, get_features_dim, get_height_dim, get_width_dim, shape_for_layout
|
||||
from mo.graph.graph import Node, Graph
|
||||
from mo.ops.op import Op
|
||||
|
||||
|
||||
class ResampleOp(Op):
|
||||
enabled = False
|
||||
op = 'Resample'
|
||||
|
||||
def __init__(self, graph: Graph, attrs: dict):
|
||||
mandatory_props = {
|
||||
'type': __class__.op,
|
||||
'op': __class__.op,
|
||||
'factor': None,
|
||||
'in_ports_count': 2,
|
||||
'out_ports_count': 1,
|
||||
'infer': None
|
||||
}
|
||||
super().__init__(graph, mandatory_props, attrs)
|
||||
|
||||
def supported_attrs(self):
|
||||
return [
|
||||
'antialias',
|
||||
'height',
|
||||
'width',
|
||||
'resample_type',
|
||||
'factor',
|
||||
]
|
||||
|
||||
def backend_attrs(self):
|
||||
return [
|
||||
'antialias',
|
||||
'height',
|
||||
'width',
|
||||
('type', 'resample_type'),
|
||||
'factor'
|
||||
]
|
||||
|
||||
@staticmethod
|
||||
def resample_infer(node: Node):
|
||||
layout = node.graph.graph['layout']
|
||||
assert len(layout) == 4
|
||||
|
||||
input_shape = node.in_node(0).shape
|
||||
if input_shape is None:
|
||||
return
|
||||
in_height = input_shape[get_height_dim(layout, 4)]
|
||||
in_width = input_shape[get_width_dim(layout, 4)]
|
||||
|
||||
if node.has('fw') and node.fw == 'tf':
|
||||
dst_shape = node.in_node(1).value
|
||||
if dst_shape is None or len(input_shape) != 4 or len(dst_shape) != 2:
|
||||
log.error(
|
||||
'Node {} with op {} cannot be converted to Resample layer because there is no enough info about '
|
||||
'src/dst shapes: src_shape = {}, dst_shape = {}'.format(node.name, node.op, input_shape, dst_shape))
|
||||
node.type = None # prevent translation to a valid IE layer
|
||||
return
|
||||
out_height = dst_shape[0]
|
||||
out_width = dst_shape[1]
|
||||
else:
|
||||
if len(node.in_nodes()) == 1:
|
||||
if node.has('width') and node.has('height'):
|
||||
out_height = node.height
|
||||
out_width = node.width
|
||||
else:
|
||||
out_height = node.factor * in_height
|
||||
out_width = node.factor * in_width
|
||||
else:
|
||||
out_height = node.in_node(1).shape[get_height_dim(layout, 4)]
|
||||
out_width = node.in_node(1).shape[get_width_dim(layout, 4)]
|
||||
|
||||
node.factor = factor_update(
|
||||
node.factor,
|
||||
[float(out_height) / in_height, float(out_width) / in_width],
|
||||
[in_height, in_width],
|
||||
[out_height, out_width],
|
||||
node.soft_get('name'))
|
||||
|
||||
node.out_node().shape = shape_for_layout(layout,
|
||||
batch=input_shape[get_batch_dim(layout, 4)],
|
||||
features=input_shape[get_features_dim(layout, 4)],
|
||||
height=out_height,
|
||||
width=out_width)
|
||||
Reference in new issue
Block a user