IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Enable conversions to LSTM/GRU/RNN Sequence in MOC (#13470)

Browse Source 
* Enable SequenceFusion, TensorIteratorToSequence, GRUCellFusion transformation in MOC, the transformations have been deleted from plugin pipelines

* clean-up: delete augru transformation header

* Temporary disabled TensorIterator IR reader tests

* temporary disable tests, it's not TI issue, usqueeze doesn't have reference impl for fp64 and bf16

* Update ConvertTensorItearatorToSequence transformations: add Unsqueeze to pattern

* Codestyle

* Specify negative values for LSTM/GRU/RNN Sequence ops

* fix conflicts with master branch

* Insert ShapeOf subgraph in ConvertSequencesToTensorIterator transformation in case of dynamic input shape

* codestyle

* fix conflict with master branch

* fix onednn version

* Update low latency v2 to support GRU/RNN/LSTM Sequence ops, fix accuracy issue

* fix tensor names

* fix tensor name issue on gna

* fix transformation tests, fix copying rt_info in the transformation

* fix warning, enable bf16, f64 ti tests

* codestyle

* fix functional tests

* EliminateDuplicateTIInputs transformation, fix copy_with_new_inputs method of TI op

* move EliminateDuplicateTIInputs transformation

* codestyle
This commit is contained in:
Ivan Tikhonov
2022-11-17 13:27:20 +04:00
committed by GitHub
parent 0cab188059
commit 5c0225d358
14 changed files with 509 additions and 195 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ops/sequence/GRUSequence_5.md
+1 -1
View File
@@ -65,7 +65,7 @@ A single cell in the sequence is implemented in the same way as in <a href="#GRU
* **2**: `initial_hidden_state` - 3D tensor of type *T1* `[batch_size, num_directions, hidden_size]`, input hidden state data. **Required.**
* **3**: `sequence_lengths` - 1D tensor of type *T2* `[batch_size]`, specifies real sequence lengths for each batch element. **Required.**
* **3**: `sequence_lengths` - 1D tensor of type *T2* `[batch_size]`, specifies real sequence lengths for each batch element. In case of negative values in this input, the operation behavior is undefined. **Required.**
* **4**: `W` - 3D tensor of type *T1* `[num_directions, 3 * hidden_size, input_size]`, the weights for matrix multiplication, gate order: zrh. **Required.**
docs/ops/sequence/LSTMSequence_1.md
+1 -1
View File
@@ -59,7 +59,7 @@ A single cell in the sequence is implemented in the same way as in <a href="#LST
* **3**: `initial_cell_state` - 3D tensor of type *T1* `[batch_size, num_directions, hidden_size]`, input cell state data. **Required.**
* **4**: `sequence_lengths` - 1D tensor of type *T2* `[batch_size]`, specifies real sequence lengths for each batch element. **Required.**
* **4**: `sequence_lengths` - 1D tensor of type *T2* `[batch_size]`, specifies real sequence lengths for each batch element. In case of negative values in this input, the operation behavior is undefined. **Required.**
* **5**: `W` - 3D tensor of type *T1* `[num_directions, 4 * hidden_size, input_size]`, the weights for matrix multiplication, gate order: fico. **Required.**
docs/ops/sequence/RNNSequence_5.md
+1 -1
View File
@@ -57,7 +57,7 @@ A single cell in the sequence is implemented in the same way as in <a href="#RNN
* **2**: `H` - 3D tensor of type *T1* `[batch_size, num_directions, hidden_size]`, input hidden state data. **Required.**
* **3**: `sequence_lengths` - 1D tensor of type *T2* `[batch_size]`, specifies real sequence lengths for each batch element. **Required.**
* **3**: `sequence_lengths` - 1D tensor of type *T2* `[batch_size]`, specifies real sequence lengths for each batch element. In case of negative values in this input, the operation behavior is undefined. **Required.**
* **4**: `W` - 3D tensor of type *T1* `[num_directions, hidden_size, input_size]`, the weights for matrix multiplication. **Required.**
src/common/transformations/include/transformations/common_optimizations/eliminate_duplicate_ti_inputs.hpp
+30
View File
@@ -0,0 +1,30 @@
// Copyright (C) 2018-2022 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include <memory>
#include <openvino/pass/graph_rewrite.hpp>
#include <transformations_visibility.hpp>
#include <vector>
namespace ov {
namespace pass {
class TRANSFORMATIONS_API EliminateDuplicateTIInputs;
} // namespace pass
} // namespace ov
/*
* @ingroup ie_transformation_common_api
* @brief EliminateDuplicateTIInputs transformation
* removes duplicated inputs of SubgraphOps.
*/
class ov::pass::EliminateDuplicateTIInputs : public ov::pass::MatcherPass {
public:
OPENVINO_RTTI("EliminateDuplicateTIInputs", "0");
EliminateDuplicateTIInputs();
};
src/common/transformations/src/transformations/common_optimizations/eliminate_duplicate_ti_inputs.cpp
+118
View File
@@ -0,0 +1,118 @@
// Copyright (C) 2018-2022 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include "transformations/common_optimizations/eliminate_duplicate_ti_inputs.hpp"
#include <memory>
#include <ngraph/rt_info.hpp>
#include <vector>
#include "itt.hpp"
#include "openvino/opsets/opset10.hpp"
#include "openvino/pass/pattern/op/wrap_type.hpp"
using namespace ov::op::util;
ov::pass::EliminateDuplicateTIInputs::EliminateDuplicateTIInputs() {
MATCHER_SCOPE(EliminateDuplicateTIInputs);
auto ti = pattern::wrap_type<ov::opset10::TensorIterator>();
ov::matcher_pass_callback callback = [=](pattern::Matcher& m) {
auto ti = std::dynamic_pointer_cast<ov::opset10::TensorIterator>(m.get_match_root());
if (ti == nullptr) {
return false;
}
std::vector<std::shared_ptr<SubGraphOp::InputDescription>> should_stay;
std::map<std::shared_ptr<SubGraphOp::InputDescription>,
std::vector<std::shared_ptr<SubGraphOp::InputDescription>>>
need_to_eliminate;
auto input_descs = ti->get_input_descriptions();
for (auto& key : input_descs) {
auto is_equal = [&](const std::shared_ptr<SubGraphOp::InputDescription>& input) -> bool {
if (ti->input_value(input->m_input_index) == ti->input_value(key->m_input_index)) {
auto invariant_l = std::dynamic_pointer_cast<SubGraphOp::InvariantInputDescription>(input);
auto invariant_r = std::dynamic_pointer_cast<SubGraphOp::InvariantInputDescription>(key);
if (invariant_l && invariant_r) {
return true;
}
auto slice_l = std::dynamic_pointer_cast<SubGraphOp::SliceInputDescription>(input);
auto slice_r = std::dynamic_pointer_cast<SubGraphOp::SliceInputDescription>(key);
if (slice_l && slice_r) {
return slice_l->m_axis == slice_r->m_axis && slice_l->m_start == slice_r->m_start &&
slice_l->m_end == slice_r->m_end && slice_l->m_part_size == slice_r->m_part_size &&
slice_l->m_stride == slice_r->m_stride;
}
auto merged_l = std::dynamic_pointer_cast<SubGraphOp::MergedInputDescription>(input);
auto merged_r = std::dynamic_pointer_cast<SubGraphOp::MergedInputDescription>(key);
if (merged_l && merged_r) {
return merged_l->m_body_value_index == merged_r->m_body_value_index;
}
}
return false;
};
auto it = std::find_if(should_stay.begin(), should_stay.end(), is_equal);
if (it == should_stay.end()) {
should_stay.push_back(key);
} else {
need_to_eliminate[*it].push_back(key);
}
}
if (need_to_eliminate.empty()) {
return false;
}
const auto& body = ti->get_function();
// re-connect outputs of duplicate Parameters
for (const auto& it : need_to_eliminate) {
for (const auto& redundant : it.second) {
auto parameters = body->get_parameters();
parameters[redundant->m_body_parameter_index]->output(0).replace(
parameters[it.first->m_body_parameter_index]);
}
}
// Create new TI
auto new_ti = std::make_shared<opset10::TensorIterator>();
new_ti->set_output_descriptions(0, ti->get_output_descriptions());
ov::ParameterVector new_params;
for (const auto& remain : should_stay) {
auto par = body->get_parameters()[remain->m_body_parameter_index];
new_params.push_back(par);
}
auto new_body = std::make_shared<ov::Model>(body->get_results(), new_params);
new_ti->set_body(new_body);
for (const auto& remain : should_stay) {
auto par = body->get_parameters()[remain->m_body_parameter_index];
auto in = ti->input_value(remain->m_input_index);
if (auto invariant = std::dynamic_pointer_cast<SubGraphOp::InvariantInputDescription>(remain)) {
new_ti->set_invariant_input(par, in);
} else if (auto merged = std::dynamic_pointer_cast<SubGraphOp::MergedInputDescription>(remain)) {
auto results = body->get_results();
new_ti->set_merged_input(par, in, results[merged->m_body_value_index]);
} else if (auto slice = std::dynamic_pointer_cast<SubGraphOp::SliceInputDescription>(remain)) {
new_ti->set_sliced_input(par,
in,
slice->m_start,
slice->m_stride,
slice->m_part_size,
slice->m_end,
slice->m_axis);
}
}
new_ti->validate_and_infer_types();
copy_runtime_info(ti, new_ti);
replace_node(ti, new_ti);
new_ti->set_friendly_name(ti->get_friendly_name());
return true;
};
auto m = std::make_shared<pattern::Matcher>(ti, matcher_name);
this->register_matcher(m, callback);
}
src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp
+8 -1
View File
@@ -20,6 +20,7 @@
#include <transformations/common_optimizations/disable_random_uniform_constant_folding.hpp>
#include <transformations/common_optimizations/disable_shapeof_constant_folding.hpp>
#include <transformations/common_optimizations/divide_fusion.hpp>
#include <transformations/common_optimizations/eliminate_duplicate_ti_inputs.hpp>
#include <transformations/common_optimizations/eliminate_unsqueeze_gather.hpp>
#include <transformations/common_optimizations/fold_subgraph_empty_inputs.hpp>
#include <transformations/common_optimizations/fq_mul_fusion.hpp>
@@ -52,6 +53,7 @@
#include <transformations/common_optimizations/remove_multi_subgraph_op_dangling_params.hpp>
#include <transformations/common_optimizations/reshape_sequence_fusion.hpp>
#include <transformations/common_optimizations/ric_fusion.hpp>
#include <transformations/common_optimizations/sequence_fusion.hpp>
#include <transformations/common_optimizations/shuffle_channels_fusion.hpp>
#include <transformations/common_optimizations/simplify_shape_of_sub_graph.hpp>
#include <transformations/common_optimizations/softmax_fusion.hpp>
@@ -70,6 +72,7 @@
#include <transformations/op_conversions/convert_divide.hpp>
#include <transformations/op_conversions/convert_negative.hpp>
#include <transformations/op_conversions/convert_scatter_elements_to_scatter.hpp>
#include <transformations/op_conversions/convert_ti_to_sequences.hpp>
#include <transformations/smart_reshape/lstm_states_broadcast.hpp>
#include <transformations/smart_reshape/reshape_sinking.hpp>
@@ -169,7 +172,11 @@ bool ov::pass::MOCTransformations::run_on_model(const std::shared_ptr<ngraph::Fu
common_fusions->add_matcher<ov::pass::GeluFusion>();
common_fusions->add_matcher<ov::pass::LeakyReluFusion>();
common_fusions->add_matcher<ov::pass::RandomUniformFusion>();
common_fusions->add_matcher<ov::pass::SplitConcatPairToInterpolateFusion>(m_use_shapes);
common_fusions->add_matcher<ov::pass::EliminateDuplicateTIInputs>();
common_fusions->add_matcher<ov::pass::GRUCellFusion>();
common_fusions->add_matcher<ov::pass::SequenceFusion>();
common_fusions->add_matcher<ngraph::pass::ConvertTensorIteratorToSequence>();
common_fusions->add_matcher<ngraph::pass::SplitConcatPairToInterpolateFusion>(m_use_shapes);
if (m_use_shapes) {
common_fusions->add_matcher<ov::pass::NearestNeighborUpsamplingFusion>();
}
src/common/transformations/src/transformations/op_conversions/convert_ti_to_sequences.cpp
+22 -25
View File
@@ -100,15 +100,17 @@ bool convertTensorIteratorToSequence(const std::shared_ptr<ov::opset5::TensorIte
? nullptr
: std::make_shared<ov::opset5::Unsqueeze>(ti_inputs[ordered_in_descs[2]->m_input_index], axis_1);
const size_t batch_dim = slice_axis == 0 ? 1 : 0;
auto batch_dimension = ngraph::op::util::node_to_get_shape_value_of_indices_from_shape_source(
ti_inputs[ordered_in_descs[0]->m_input_index],
{batch_dim});
auto seq_lengths_scalar = ov::opset5::Constant::create(ngraph::element::i32, {}, {ti->get_num_iterations()});
auto seq_lengths = ov::op::util::make_try_fold<ov::opset5::Broadcast>(seq_lengths_scalar, batch_dimension);
auto axis_0 = ov::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {0});
auto shape_of = std::make_shared<ov::opset5::ShapeOf>(X);
auto batch_dimension =
std::make_shared<ov::opset5::Gather>(shape_of,
ov::opset5::Constant::create(ngraph::element::i64, {1}, {0}),
ov::opset5::Constant::create(ngraph::element::i64, {}, {0}));
auto seq_len_dim =
std::make_shared<ov::opset5::Gather>(shape_of,
ov::opset5::Constant::create(ngraph::element::i64, {1}, {1}),
ov::opset5::Constant::create(ngraph::element::i64, {}, {0}));
auto seq_lengths = std::make_shared<ov::opset5::Broadcast>(seq_len_dim, batch_dimension);
auto axis_0 = ov::opset5::Constant::create(ov::element::i64, ngraph::Shape{1}, {0});
auto W = ov::op::util::make_try_fold<ov::opset5::Unsqueeze>(w_pattern, axis_0);
auto R = ov::op::util::make_try_fold<ov::opset5::Unsqueeze>(r_pattern, axis_0);
auto B = ov::op::util::make_try_fold<ov::opset5::Unsqueeze>(b_pattern, axis_0);
@@ -190,13 +192,7 @@ bool convertTensorIteratorToSequence(const std::shared_ptr<ov::opset5::TensorIte
for (size_t i = 0; i < ordered_out_descs.size(); ++i) {
if (ordered_out_descs[i]) {
for (const auto& input : ti->output(ordered_out_descs[i]->m_output_index).get_target_inputs()) {
input.replace_source_output(outputs[i]->output(0));
}
NGRAPH_SUPPRESS_DEPRECATED_START
outputs[i]->get_output_tensor(0).set_name(
ngraph::op::util::create_ie_output_name(ti->output(ordered_out_descs[i]->m_output_index)));
NGRAPH_SUPPRESS_DEPRECATED_END
ti->output(ordered_out_descs[i]->m_output_index).replace(outputs[i]->output(0));
}
}
@@ -210,12 +206,12 @@ bool convertTensorIteratorToSequence(const std::shared_ptr<ov::opset5::TensorIte
if (c_pattern.get_node_shared_ptr()) {
new_nodes.emplace_back(initial_cell_state);
}
if (!std::dynamic_pointer_cast<ov::opset5::Constant>(seq_lengths)) {
new_nodes.emplace_back(batch_dimension);
new_nodes.emplace_back(batch_dimension->get_input_node_shared_ptr(0));
new_nodes.emplace_back(seq_lengths_scalar);
new_nodes.emplace_back(seq_lengths);
}
new_nodes.emplace_back(batch_dimension);
new_nodes.emplace_back(shape_of);
new_nodes.emplace_back(seq_len_dim);
new_nodes.emplace_back(seq_lengths);
if (slice_axis == 0) {
new_nodes.emplace_back(out.get_node_shared_ptr());
new_nodes.emplace_back(X.get_node_shared_ptr());
@@ -250,7 +246,7 @@ ov::pass::ConvertTensorIteratorToLSTMSequence::ConvertTensorIteratorToLSTMSequen
auto cell = ngraph::pattern::wrap_type<ov::opset1::LSTMCell, ov::opset5::LSTMCell>(cell_inputs);
auto pattern_2 = ngraph::pattern::wrap_type<ov::opset5::Constant>(ngraph::pattern::rank_equals(1));
auto unsqueeze = ngraph::pattern::wrap_type<ov::opset5::Reshape>({cell, pattern_2});
auto unsqueeze = ngraph::pattern::wrap_type<ov::opset5::Reshape, ov::opset5::Unsqueeze>({cell, pattern_2});
ngraph::pattern::Matcher matcher(unsqueeze);
bool match = false;
@@ -309,7 +305,7 @@ ov::pass::ConvertTensorIteratorToRNNSequence::ConvertTensorIteratorToRNNSequence
auto cell = ngraph::pattern::wrap_type<ov::opset5::RNNCell>(cell_inputs);
auto pattern_2 = ngraph::pattern::wrap_type<ov::opset5::Constant>(ngraph::pattern::rank_equals(1));
auto unsqueeze = ngraph::pattern::wrap_type<ov::opset5::Reshape>({cell, pattern_2});
auto unsqueeze = ngraph::pattern::wrap_type<ov::opset5::Reshape, ov::opset5::Unsqueeze>({cell, pattern_2});
ngraph::pattern::Matcher matcher(unsqueeze);
bool match = false;
@@ -368,7 +364,8 @@ ov::pass::ConvertTensorIteratorToGRUSequence::ConvertTensorIteratorToGRUSequence
auto cell = ngraph::pattern::wrap_type<ov::opset5::GRUCell>(cell_inputs);
auto pattern_2 = ngraph::pattern::wrap_type<ov::opset5::Constant>(ngraph::pattern::rank_equals(1));
auto unsqueeze = ngraph::pattern::wrap_type<ov::opset5::Reshape>({cell, pattern_2});
auto unsqueeze = ngraph::pattern::wrap_type<ov::opset5::Reshape, ov::opset5::Unsqueeze>({cell, pattern_2});
ngraph::pattern::Matcher matcher(unsqueeze);
bool match = false;
src/core/src/op/tensor_iterator.cpp
+4 -29
View File
@@ -191,37 +191,12 @@ void op::v0::TensorIterator::try_to_set_num_iterations_if_no_slice_inputs() {
std::shared_ptr<Node> op::v0::TensorIterator::clone_with_new_inputs(const OutputVector& new_args) const {
OV_OP_SCOPE(v0_TensorIterator_clone_with_new_inputs);
auto op = make_shared<op::v0::TensorIterator>(new_args);
NGRAPH_CHECK(op.get(), op != nullptr, "Cannot clone ", description(), " operation with name ", get_friendly_name());
op->set_output_size(m_output_descriptions[0].size());
std::vector<::ngraph::element::Type> types(m_bodies[0]->get_parameters().size());
std::vector<ov::PartialShape> new_shapes(m_bodies[0]->get_parameters().size());
for (size_t input_index = 0; input_index < new_args.size(); ++input_index) {
for (auto& input_description : m_input_descriptions[0]) {
if (input_description->m_input_index == input_index) {
types[input_description->m_body_parameter_index] = new_args[input_index].get_element_type();
new_shapes[input_description->m_body_parameter_index] = new_args[input_index].get_partial_shape();
if (new_shapes[input_description->m_body_parameter_index].is_static()) {
if (auto slice_in = ::ngraph::as_type_ptr<ngraph::op::v0::TensorIterator::SliceInputDescription>(
input_description)) {
new_shapes[slice_in->m_body_parameter_index][slice_in->m_axis] = slice_in->m_part_size;
}
}
}
}
}
auto op = make_shared<op::v0::TensorIterator>();
op->set_arguments(new_args);
op->set_output_size(m_output_descriptions.size());
op->m_num_iterations = m_num_iterations;
auto func =
std::make_shared<Model>(m_bodies[0]->get_results(), m_bodies[0]->get_sinks(), m_bodies[0]->get_parameters());
NGRAPH_SUPPRESS_DEPRECATED_START;
auto spec_func = specialize_function(func, types, new_shapes, std::vector<void*>(new_args.size(), nullptr));
NGRAPH_SUPPRESS_DEPRECATED_END;
op->m_bodies[0] =
std::make_shared<Model>(spec_func->get_results(), spec_func->get_sinks(), spec_func->get_parameters());
op->m_bodies[0] = clone_function(*get_function());
for (auto& input_description : m_input_descriptions[0]) {
op->m_input_descriptions[0].push_back(input_description->copy());
src/core/src/op/unsqueeze.cpp
+2
View File
@@ -92,6 +92,8 @@ bool evaluate_unsqueeze(const Node* node,
NGRAPH_TYPE_CASE(evaluate_unsqueeze, u64, arg0, out);
NGRAPH_TYPE_CASE(evaluate_unsqueeze, f16, arg0, out);
NGRAPH_TYPE_CASE(evaluate_unsqueeze, f32, arg0, out);
NGRAPH_TYPE_CASE(evaluate_unsqueeze, f64, arg0, out);
NGRAPH_TYPE_CASE(evaluate_unsqueeze, bf16, arg0, out);
default:
rc = false;
break;
src/core/src/pass/low_latency.cpp
+213 -59
View File
@@ -6,12 +6,17 @@
#include <memory>
#include <ngraph/log.hpp>
#include <ngraph/opsets/opset1.hpp>
#include <ngraph/opsets/opset6.hpp>
#include <ngraph/opsets/opset7.hpp>
#include <ngraph/pattern/op/wrap_type.hpp>
#include <ngraph/rt_info.hpp>
#include <ngraph/variant.hpp>
#include <openvino/cc/pass/itt.hpp>
#include <openvino/op/util/variable.hpp>
#include <openvino/opsets/opset1.hpp>
#include <openvino/opsets/opset9.hpp>
#include <openvino/util/log.hpp>
NGRAPH_SUPPRESS_DEPRECATED_START
NGRAPH_RTTI_DEFINITION(ngraph::pass::LowLatency, "LowLatency", 0);
@@ -93,13 +98,21 @@ NGRAPH_SUPPRESS_DEPRECATED_END
namespace {
void UnrollSingleIteration(const shared_ptr<ngraph::op::util::SubGraphOp>& sub_graph_op,
const shared_ptr<ov::Model>& outer_f) {
using namespace ngraph::opset7;
const string msg_low_latency_2_already_applied = "LowLatency2 transformation cannot be applied because the "
"ReadValue node is already an input to the TensorIterator."
"LowLatency2 transformation may have already been applied, please"
"do not call it more then once.";
const string msg_low_latency_already_applied = "LowLatency2 transformation cannot be applied because the "
"ReadValue node is already inside the TensorIterator. "
"LowLatency transformation may have been applied, please do "
"not call LowLatency2 after LowLatency.";
void unroll_single_iteration(const shared_ptr<ov::op::util::SubGraphOp>& sub_graph_op,
const shared_ptr<ov::Model>& outer_f) {
using namespace ov::opset9;
const auto& params = sub_graph_op->get_function()->get_parameters();
const auto& results = sub_graph_op->get_function()->get_results();
// before: Layer1 -> TI [input -> bodyParameter -> Layer2 -> ...]
// after: Layer1 -> Layer2 ->...
for (const auto& in : sub_graph_op->get_input_descriptions()) {
@@ -116,9 +129,9 @@ void UnrollSingleIteration(const shared_ptr<ngraph::op::util::SubGraphOp>& sub_g
const auto& connect_to = results.at(out->m_body_value_index)->get_input_source_output(0);
for (auto& input_to : sub_graph_op->output(out->m_output_index).get_target_inputs()) {
// create IE output name
std::string out_name = sub_graph_op->get_friendly_name();
string out_name = sub_graph_op->get_friendly_name();
if (sub_graph_op->get_output_size() != 1)
out_name += "." + std::to_string(out->m_output_index);
out_name += "." + to_string(out->m_output_index);
// IECompatibility: insert identity (Unsqueeze + Squeeze) to store the TensorIterator
// output names
@@ -129,115 +142,256 @@ void UnrollSingleIteration(const shared_ptr<ngraph::op::util::SubGraphOp>& sub_g
new_ops.push_back(identity_1);
new_ops.push_back(identity_2);
identity_2->output(0).get_tensor().add_names(input_to.get_source_output().get_names());
input_to.replace_source_output(identity_2);
}
}
outer_f->add_sinks(sub_graph_op->get_function()->get_sinks());
ngraph::copy_runtime_info(sub_graph_op, sub_graph_op->get_function()->get_ops());
ngraph::copy_runtime_info(sub_graph_op, new_ops);
ov::copy_runtime_info(sub_graph_op, sub_graph_op->get_function()->get_ops());
ov::copy_runtime_info(sub_graph_op, new_ops);
}
ngraph::Output<ngraph::Node> create_init_subgraph(const shared_ptr<ngraph::op::util::SubGraphOp>& sub_graph_op,
const ngraph::Output<ngraph::Node>& in_node) {
using namespace ngraph::opset7;
ov::Output<ov::Node> create_init_subgraph(const ov::Output<ov::Node>& in_node, ov::pass::NodeRegistry& to) {
using namespace ov::opset9;
auto const_zero = make_shared<Constant>(in_node.get_element_type(), ov::Shape{1}, 0);
auto shape_of = make_shared<ShapeOf>(in_node);
auto broadcast = make_shared<Broadcast>(const_zero, shape_of);
copy_runtime_info(sub_graph_op, {const_zero, shape_of, broadcast});
auto const_zero = to.make<Constant>(in_node.get_element_type(), ov::Shape{1}, 0);
auto shape_of = to.make<ShapeOf>(in_node);
auto broadcast = to.make<Broadcast>(const_zero, shape_of);
return broadcast->output(0);
}
shared_ptr<ov::opset9::Assign> replace_with_memory(const ov::Input<ov::Node>& input,
const ov::Output<ov::Node>& output,
const string& variable_name,
bool use_const_initializer,
ov::pass::NodeRegistry& to) {
using namespace ov::opset9;
using namespace ov::op::util;
VariableInfo var_info{ov::PartialShape::dynamic(), ov::element::dynamic, variable_name};
auto variable = make_shared<Variable>(var_info);
ov::Output<ov::Node> read_value_in = input.get_source_output();
if (use_const_initializer) {
read_value_in = create_init_subgraph(read_value_in, to);
}
auto read_value = to.make<ReadValue>(read_value_in, variable);
input.replace_source_output(read_value->output(0));
read_value->set_friendly_name(variable_name);
auto assign = to.make<Assign>(output, variable);
// control dependency so that ReadValue is processed before Assign
assign->add_control_dependency(read_value);
return assign;
}
std::vector<shared_ptr<ov::opset9::Assign>> replace_with_memory(const shared_ptr<ov::Node>& node,
const vector<size_t>& indexes,
bool use_const_initializer,
ov::pass::NodeRegistry& to) {
std::vector<shared_ptr<ov::opset9::Assign>> new_assigns;
for (const auto& idx : indexes) {
auto in = node->input(idx);
auto out = node->output(idx);
new_assigns.push_back(replace_with_memory(in, out, node->get_friendly_name(), use_const_initializer, to));
}
return new_assigns;
}
bool need_unroll(const shared_ptr<ov::Node>& op) {
const auto p_shape = op->get_input_partial_shape(0);
if (p_shape.rank().is_dynamic() || p_shape[1].is_dynamic() || p_shape[1].get_length() != 1) {
return false;
}
return true;
}
ov::OutputVector prepare_inputs(const shared_ptr<ov::Node>& op, size_t seq_len_idx, ov::pass::NodeRegistry& to) {
using namespace ov::opset9;
ov::OutputVector inputs;
auto axis_0 = to.make<Constant>(ov::element::i32, ov::Shape{1}, 0);
auto axis_1 = to.make<Constant>(ov::element::i32, ov::Shape{1}, 1);
size_t num_lstm_inputs_without_peepholes = 7;
for (size_t i = 0; i < max(op->get_input_size(), num_lstm_inputs_without_peepholes); ++i) {
if (i < seq_len_idx) {
inputs.push_back(to.make<Squeeze>(op->get_input_source_output(i), axis_1));
} else if (i > seq_len_idx) {
inputs.push_back(to.make<Squeeze>(op->get_input_source_output(i), axis_0));
}
}
return inputs;
}
std::vector<shared_ptr<ov::opset9::Assign>> process_sequence(const shared_ptr<ov::Node>& op,
bool m_use_const_initializer,
ov::pass::NodeRegistry& to) {
using namespace ov::opset9;
shared_ptr<ov::Node> cell;
std::vector<shared_ptr<ov::opset9::Assign>> new_assigns;
bool unroll = false;
if (auto lstm_seq_v0 = dynamic_pointer_cast<ov::opset1::LSTMSequence>(op)) {
unroll = need_unroll(op);
new_assigns = replace_with_memory(op, {1, 2}, m_use_const_initializer, to);
if (unroll) {
auto inputs = prepare_inputs(op, 3, to);
cell = to.make<LSTMCell>(inputs[0],
inputs[1],
inputs[2],
inputs[3],
inputs[4],
inputs[5],
lstm_seq_v0->get_hidden_size(),
lstm_seq_v0->get_activations(),
lstm_seq_v0->get_activations_alpha(),
lstm_seq_v0->get_activations_beta(),
lstm_seq_v0->get_clip_threshold());
}
} else if (auto lstm_seq_v5 = dynamic_pointer_cast<LSTMSequence>(op)) {
unroll = need_unroll(op);
new_assigns = replace_with_memory(op, {1, 2}, m_use_const_initializer, to);
if (unroll) {
auto inputs = prepare_inputs(op, 3, to);
cell = to.make<LSTMCell>(inputs[0],
inputs[1],
inputs[2],
inputs[3],
inputs[4],
inputs[5],
lstm_seq_v5->get_hidden_size(),
lstm_seq_v5->get_activations(),
lstm_seq_v5->get_activations_alpha(),
lstm_seq_v5->get_activations_beta(),
lstm_seq_v5->get_clip());
}
} else if (auto gru_seq = dynamic_pointer_cast<GRUSequence>(op)) {
unroll = need_unroll(op);
new_assigns = replace_with_memory(op, {1}, m_use_const_initializer, to);
if (unroll) {
auto inputs = prepare_inputs(op, 2, to);
cell = to.make<GRUCell>(inputs[0],
inputs[1],
inputs[2],
inputs[3],
inputs[4],
gru_seq->get_hidden_size(),
gru_seq->get_activations(),
gru_seq->get_activations_alpha(),
gru_seq->get_activations_beta(),
gru_seq->get_clip(),
gru_seq->get_linear_before_reset());
}
} else if (auto rnn_seq = dynamic_pointer_cast<RNNSequence>(op)) {
unroll = need_unroll(op);
new_assigns = replace_with_memory(op, {1}, m_use_const_initializer, to);
if (unroll) {
auto inputs = prepare_inputs(op, 2, to);
cell = to.make<RNNCell>(inputs[0],
inputs[1],
inputs[2],
inputs[3],
inputs[4],
rnn_seq->get_hidden_size(),
rnn_seq->get_activations(),
rnn_seq->get_activations_alpha(),
rnn_seq->get_activations_beta(),
rnn_seq->get_clip());
}
} else {
// unsupported sequence or not sequence
return {};
}
if (unroll && cell) {
auto axis_1_2 = to.make<Constant>(ov::element::i32, ov::Shape{2}, vector<int>{1, 2});
auto axis_1 = to.make<Constant>(ov::element::i32, ov::Shape{1}, 1);
ov::OutputVector outputs;
outputs.push_back(to.make<Unsqueeze>(cell->output(0), axis_1_2));
for (const auto& out : cell->outputs()) {
outputs.push_back(to.make<Unsqueeze>(out, axis_1));
}
replace_node(op, outputs);
copy_runtime_info(op, to.get());
}
return new_assigns;
}
} // namespace
bool ov::pass::LowLatency2::run_on_model(const shared_ptr<Model>& f) {
using namespace ngraph::opset7;
RUN_ON_MODEL_SCOPE(LowLatency2);
ngraph::SinkVector assigns;
using namespace ov::opset9;
using namespace ov::op::util;
NodeRegistry to;
ov::SinkVector assigns;
for (const auto& op : f->get_ordered_ops()) {
if (const auto& sub_graph_op = dynamic_pointer_cast<ngraph::op::util::SubGraphOp>(op)) {
if (const auto& sub_graph_op = dynamic_pointer_cast<SubGraphOp>(op)) {
int64_t variable_id = 0;
const auto& func = sub_graph_op->get_function();
const auto& params = func->get_parameters();
for (const auto& in : sub_graph_op->get_input_descriptions()) {
// Process all back edges
if (const auto& merged_in =
dynamic_pointer_cast<ngraph::op::util::SubGraphOp::MergedInputDescription>(in)) {
if (const auto& merged_in = dynamic_pointer_cast<SubGraphOp::MergedInputDescription>(in)) {
// create new Variable
const string& param_name = params.at(merged_in->m_body_parameter_index)->get_friendly_name();
const string& var_name =
generate_variable_name(sub_graph_op->get_friendly_name(), param_name, variable_id);
const auto& input = sub_graph_op->input(merged_in->m_input_index);
if (std::dynamic_pointer_cast<ngraph::op::ReadValueBase>(
input.get_source_output().get_node_shared_ptr()) != nullptr) {
NGRAPH_DEBUG << "LowLatency2 transformation cannot be applied because the "
<< "ReadValue node is already an input to the TensorIterator."
<< "LowLatency2 transformation may have already been applied, please "
<< "do not call it more then once.";
if (dynamic_pointer_cast<ReadValueBase>(input.get_source_output().get_node_shared_ptr()) !=
nullptr) {
NGRAPH_DEBUG << msg_low_latency_2_already_applied;
return false;
}
const auto& param =
sub_graph_op->get_function()->get_parameters().at(merged_in->m_body_parameter_index);
for (const auto& in_to : param->output(0).get_target_inputs()) {
if (dynamic_cast<ngraph::op::ReadValueBase*>(in_to.get_node()) != nullptr) {
NGRAPH_DEBUG << "LowLatency2 transformation cannot be applied because the "
<< "ReadValue node is already inside the TensorIterator. "
<< "LowLatency transformation may have been applied, please do "
<< "not call LowLatency2 after LowLatency.";
if (dynamic_cast<ReadValueBase*>(in_to.get_node()) != nullptr) {
NGRAPH_DEBUG << msg_low_latency_already_applied;
return false;
}
}
ngraph::VariableInfo var_info{PartialShape::dynamic(), element::dynamic, var_name};
auto variable = make_shared<ngraph::Variable>(var_info);
// insert ReadValue
// insert ReadValue and Assign ops:
//
// Layers -> [new op: ReadValue] -> Subgraph operation
Output<Node> read_value_in = input.get_source_output();
if (m_use_const_initializer) {
read_value_in = create_init_subgraph(sub_graph_op, read_value_in);
}
auto read_value = make_shared<ReadValue>(read_value_in, variable);
input.replace_source_output(read_value->output(0));
read_value->set_friendly_name(var_name);
ngraph::copy_runtime_info(sub_graph_op, read_value);
/* insert Assign
//
// Subgraph operation -> [new op: Assign]
// \
// ---> Layers -> ...
*/
//
const auto& out_desc = sub_graph_op->get_output_descriptions();
bool is_output_exist = std::any_of(
out_desc.begin(),
out_desc.end(),
[&merged_in](const std::shared_ptr<ngraph::op::util::SubGraphOp::OutputDescription>& out) {
return out->m_body_value_index == merged_in->m_body_value_index;
});
bool is_output_exist = any_of(out_desc.begin(),
out_desc.end(),
[&merged_in](const shared_ptr<SubGraphOp::OutputDescription>& out) {
return out->m_body_value_index == merged_in->m_body_value_index;
});
// Create new output if it doesn't exist.
if (!is_output_exist) {
sub_graph_op->get_iter_value(func->get_results().at(merged_in->m_body_value_index));
}
Output<Node> output;
for (const auto& out : sub_graph_op->get_output_descriptions()) {
if (out->m_body_value_index == merged_in->m_body_value_index) {
auto assign = make_shared<Assign>(sub_graph_op->output(out->m_output_index), variable);
copy_runtime_info(sub_graph_op, assign);
// control dependency so that ReadValue is processed before Assign
assign->add_control_dependency(read_value);
assigns.emplace_back(assign);
output = sub_graph_op->output(out->m_output_index);
break;
}
}
auto assign = replace_with_memory(input, output, var_name, m_use_const_initializer, to);
assigns.emplace_back(assign);
copy_runtime_info(sub_graph_op, to.get());
}
variable_id++;
}
if (sub_graph_op->get_num_iterations() == 1) {
UnrollSingleIteration(sub_graph_op, f);
unroll_single_iteration(sub_graph_op, f);
}
} else {
auto new_assigns = process_sequence(op, m_use_const_initializer, to);
if (!new_assigns.empty()) {
assigns.insert(assigns.end(), new_assigns.begin(), new_assigns.end());
}
}
}
src/plugins/intel_cpu/src/plugin.cpp
+2 -14
View File
@@ -134,6 +134,7 @@
#include "nodes/normalize.h"
#include "nodes/mha.h"
#include "utils/denormals.hpp"
#include "transformations/common_optimizations/augru_cell_fusion.hpp"
#if !defined(__arm__) && !defined(_M_ARM) && !defined(__aarch64__) && !defined(_M_ARM64)
#ifndef __GNUC_PREREQ
@@ -304,13 +305,13 @@ static void TransformationUpToCPUSpecificOpSet(std::shared_ptr<ngraph::Function>
static const auto precisions = get_convert_precisions();
manager.register_pass<ov::pass::AUGRUCellFusion>();
manager.register_pass<ngraph::pass::CommonOptimizations>();
manager.register_pass<ngraph::pass::WrapInterpolateIntoTransposes>();
manager.register_pass<ngraph::pass::TransposeSinking>();
manager.register_pass<ngraph::pass::ConvertSequenceToTensorIterator>();
manager.register_pass<ngraph::pass::ConvertOpSet3ToOpSet2>();
manager.register_pass<ngraph::pass::ConvertOpSet2ToOpSet1>();
manager.register_pass<ngraph::pass::ConvertTensorIteratorToSequence>();
manager.register_pass<ngraph::pass::LSTMCellDecomposition>();
manager.register_pass<ngraph::pass::GRUCellDecomposition>();
manager.register_pass<ngraph::pass::RNNCellDecomposition>();
@@ -430,19 +431,6 @@ static void TransformationUpToCPUSpecificOpSet(std::shared_ptr<ngraph::Function>
return isCellPrimitiveSupported(node);
});
pass_config->set_callback<ngraph::pass::ConvertTensorIteratorToRNNSequence,
ngraph::pass::ConvertTensorIteratorToLSTMSequence,
ngraph::pass::ConvertTensorIteratorToGRUSequence>(
[isCellPrimitiveSupported](const_node_ptr &node) -> bool {
if (const auto& ti_op = std::dynamic_pointer_cast<const ngraph::op::TensorIterator>(node)) {
size_t count_rnn = 0;
for (const auto &op : ti_op->get_body()->get_ops())
count_rnn += isCellPrimitiveSupported(op);
return count_rnn != 1;
}
return true;
});
pass_config->set_callback<ngraph::pass::MVN6Decomposition>(
[](const_node_ptr &node) -> bool {
std::string errorMessage;
src/plugins/intel_gpu/src/plugin/transformations_pipeline.cpp
-18
View File
@@ -152,7 +152,6 @@ void TransformationsPipeline::apply(std::shared_ptr<ov::Model> func) {
manager.register_pass<ngraph::pass::ConvertOpSet3ToOpSet2>();
manager.register_pass<ngraph::pass::ConvertOpSet2ToOpSet1>();
manager.register_pass<ngraph::pass::ConvertTensorIteratorToSequence>();
manager.register_pass<ngraph::pass::LSTMCellDecomposition>();
manager.register_pass<ngraph::pass::GRUCellDecomposition>();
manager.register_pass<ngraph::pass::RNNCellDecomposition>();
@@ -331,19 +330,6 @@ void TransformationsPipeline::apply(std::shared_ptr<ov::Model> func) {
return isSequencePrimitiveSupported(node);
});
pass_config->set_callback<ngraph::pass::ConvertTensorIteratorToRNNSequence,
ngraph::pass::ConvertTensorIteratorToLSTMSequence,
ngraph::pass::ConvertTensorIteratorToGRUSequence>(
[isCellPrimitiveSupported](const_node_ptr &node) -> bool {
if (const auto& ti_op = std::dynamic_pointer_cast<const ngraph::op::TensorIterator>(node)) {
size_t count_rnn = 0;
for (const auto &op : ti_op->get_body()->get_ops())
count_rnn += isCellPrimitiveSupported(op);
return count_rnn != 1;
}
return true;
});
pass_config->set_callback<ngraph::pass::MVN6Decomposition>(
[](const_node_ptr &node) -> bool {
const auto mvn = std::dynamic_pointer_cast<const ngraph::op::v6::MVN>(node);
@@ -421,10 +407,6 @@ void TransformationsPipeline::apply(std::shared_ptr<ov::Model> func) {
pass_config->disable<ngraph::pass::ConvertSoftMax8ToSoftMax1>();
pass_config->enable<ngraph::pass::ConvertGather8ToGather7>();
if (!config.enable_loop_unrolling) {
pass_config->disable<ngraph::pass::ConvertTensorIteratorToSequence>();
}
pass_config->enable<ngraph::pass::ConvertInterpolate1ToInterpolate4>();
if (enableInt8) {
src/tests/functional/inference_engine/transformations/common_optimizations/eliminate_duplicate_ti_inputs.cpp
+79
View File
@@ -0,0 +1,79 @@
// Copyright (C) 2018-2022 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include <gtest/gtest.h>
#include <openvino/opsets/opset10.hpp>
#include <transformations/common_optimizations/eliminate_duplicate_ti_inputs.hpp>
#include "common_test_utils/ngraph_test_utils.hpp"
using namespace testing;
using namespace std;
using namespace ov::opset10;
TEST(TransformationTests, EliminateDuplicateTIInputs) {
shared_ptr<ov::Model> model;
auto invariant = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto split = make_shared<Parameter>(ov::element::f32, ov::Shape{5});
auto merged = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto ti = make_shared<TensorIterator>();
auto inv_A = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto inv_B = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto inv_C = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto split_A = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto split_B = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto split_C = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto merged_A = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto merged_B = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto merged_C = make_shared<Parameter>(ov::element::f32, ov::Shape{1});
auto relu = make_shared<Relu>(merged_A);
auto res_A = make_shared<Result>(relu);
auto concat = make_shared<Concat>(ov::OutputVector{inv_A, inv_B, inv_C, split_A, split_B, split_C,
merged_A, merged_B, merged_C}, 0);
auto ti_res = make_shared<Result>(concat);
auto body = make_shared<ov::Model>(ov::ResultVector{ti_res, res_A},
ov::ParameterVector{inv_A, inv_B, inv_C, split_A, split_B, split_C,
merged_A, merged_B, merged_C});
ti->set_body(body);
ti->set_invariant_input(inv_A, invariant);
ti->set_invariant_input(inv_B, invariant);
ti->set_invariant_input(inv_C, invariant);
ti->set_sliced_input(split_A, split, 0, 1, 1, -1, 0);
ti->set_sliced_input(split_B, split, 0, 1, 1, -1, 0);
ti->set_sliced_input(split_C, split, 0, 1, 1, -1, 0);
ti->set_merged_input(merged_A, merged, res_A);
ti->set_merged_input(merged_B, merged, res_A);
ti->set_merged_input(merged_C, merged, res_A);
ti->get_iter_value(ti_res);
auto res = make_shared<Result>(ti->output(0));
model = std::make_shared<ov::Model>(ov::ResultVector{res}, ov::ParameterVector{invariant, split, merged});
ov::pass::Manager manager;
manager.register_pass<ov::pass::EliminateDuplicateTIInputs>();
manager.run_passes(model);
shared_ptr<TensorIterator> ti_after_transformation;
for (const auto& op : model->get_ordered_ops()) {
if (ti_after_transformation = dynamic_pointer_cast<TensorIterator>(op)) {
break;
}
}
EXPECT_NE(ti_after_transformation, nullptr);
EXPECT_EQ(ti_after_transformation->get_body()->get_parameters().size(), 3);
EXPECT_EQ(ti_after_transformation->inputs().size(), 3);
}
src/tests/functional/inference_engine/transformations/op_conversions/convert_ti_to_sequences_test.cpp
+28 -46
View File
@@ -21,6 +21,24 @@
using namespace testing;
using namespace ngraph;
namespace {
std::shared_ptr<ov::Node> create_seq_len(const std::shared_ptr<ov::Node>& X) {
auto shape_of = std::make_shared<opset5::ShapeOf>(X);
auto batch_dimension = std::make_shared<ngraph::opset5::Gather>(
shape_of,
ngraph::opset5::Constant::create(ngraph::element::i64, {1}, {0}),
ngraph::opset5::Constant::create(ngraph::element::i64, {}, {0}));
auto seq_len_dim = std::make_shared<ngraph::opset5::Gather>(
shape_of,
ngraph::opset5::Constant::create(ngraph::element::i64, {1}, {1}),
ngraph::opset5::Constant::create(ngraph::element::i64, {}, {0}));
auto seq_lengths = std::make_shared<opset5::Broadcast>(seq_len_dim, batch_dimension);
return seq_lengths;
}
} // namespace
TEST(TransformationTests, ConvertTensorIteratorToLSTMSequence) {
std::shared_ptr<ngraph::Function> f(nullptr), f_ref(nullptr);
{
@@ -95,7 +113,7 @@ TEST(TransformationTests, ConvertTensorIteratorToLSTMSequence) {
auto in_1 = std::make_shared<ngraph::opset5::Unsqueeze>(Y, axis_1);
auto in_2 = std::make_shared<ngraph::opset5::Unsqueeze>(Z, axis_1);
auto seq_lengths = ngraph::opset5::Constant::create(element::i32, Shape{1}, {2});
auto seq_lengths = create_seq_len(X);
auto lstm_seq = std::make_shared<opset5::LSTMSequence>(X, in_1, in_2, seq_lengths, W, R, B, 128, op::RecurrentSequenceDirection::FORWARD);
auto axis_out = ngraph::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {1});
auto out_0 = std::make_shared<ngraph::opset5::Squeeze>(lstm_seq->output(0), axis_out);
@@ -176,12 +194,8 @@ TEST(TransformationTests, ConvertTensorIteratorToLSTMSequenceDynamicReshapeCase)
auto in_1 = std::make_shared<ngraph::opset5::Unsqueeze>(Y, axis_1);
auto in_2 = std::make_shared<ngraph::opset5::Unsqueeze>(Z, axis_1);
auto shape_of = std::make_shared<opset5::ShapeOf>(X);
auto batch_dimension = ngraph::op::util::make_try_fold<ngraph::opset7::Gather>(
shape_of,
ngraph::opset5::Constant::create(ngraph::element::i64, { 1 }, { 0 }),
ngraph::opset5::Constant::create(ngraph::element::i64, {}, { 0 }));
auto seq_lengths = std::make_shared<opset5::Broadcast>(ngraph::opset5::Constant::create(element::i32, Shape{}, { 2 }), batch_dimension);
auto seq_lengths = create_seq_len(X);
auto w_val = std::vector<float>(512 * 16, 0);
auto r_val = std::vector<float>(512 * 128, 0);
@@ -263,14 +277,7 @@ TEST(TransformationTests, ConvertTensorIteratorToLSTMSequenceDynamicSqueezeCase)
auto in_1 = std::make_shared<ngraph::opset5::Unsqueeze>(Y, axis_1);
auto in_2 = std::make_shared<ngraph::opset5::Unsqueeze>(Z, axis_1);
auto shape_of = std::make_shared<opset5::ShapeOf>(X);
auto batch_dimension = ngraph::op::util::make_try_fold<ngraph::opset7::Gather>(
shape_of,
ngraph::opset5::Constant::create(ngraph::element::i64, {1}, {0}),
ngraph::opset5::Constant::create(ngraph::element::i64, {}, {0}));
auto seq_lengths =
std::make_shared<opset5::Broadcast>(ngraph::opset5::Constant::create(element::i32, Shape{}, {2}),
batch_dimension);
auto seq_lengths = create_seq_len(X);
auto w_val = std::vector<float>(512 * 16, 0);
auto r_val = std::vector<float>(512 * 128, 0);
@@ -361,7 +368,7 @@ TEST(TransformationTests, ConvertTensorIteratorToRNNSequence) {
auto axis_1 = ngraph::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {1});
auto in_1 = std::make_shared<ngraph::opset5::Unsqueeze>(Y, axis_1);
auto seq_lengths = ngraph::opset5::Constant::create(element::i32, Shape{1}, {2});
auto seq_lengths = create_seq_len(X);
auto rnn_sequence = std::make_shared<opset5::RNNSequence>(X, in_1, seq_lengths, W, R, B, 128, op::RecurrentSequenceDirection::FORWARD);
auto axis_out = ngraph::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {1});
auto out_0 = std::make_shared<ngraph::opset5::Squeeze>(rnn_sequence->output(0), axis_out);
@@ -434,12 +441,7 @@ TEST(TransformationTests, ConvertTensorIteratorToRNNSequenceDynamicReshapeCase)
auto axis_1 = ngraph::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{ 1 }, { 1 });
auto in_1 = std::make_shared<ngraph::opset5::Unsqueeze>(Y, axis_1);
auto shape_of = std::make_shared<opset5::ShapeOf>(X);
auto batch_dimension = ngraph::op::util::make_try_fold<ngraph::opset7::Gather>(
shape_of,
ngraph::opset5::Constant::create(ngraph::element::i64, { 1 }, { 0 }),
ngraph::opset5::Constant::create(ngraph::element::i64, {}, { 0 }));
auto seq_lengths = std::make_shared<opset5::Broadcast>(ngraph::opset5::Constant::create(element::i32, Shape{}, { 2 }), batch_dimension);
auto seq_lengths = create_seq_len(X);
auto rnn_sequence = std::make_shared<opset5::RNNSequence>(X, in_1, seq_lengths, W, R, B, 128, op::RecurrentSequenceDirection::FORWARD);
auto axis_out = ngraph::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{ 1 }, { 1 });
@@ -513,15 +515,7 @@ TEST(TransformationTests, ConvertTensorIteratorToRNNSequenceDynamicSqueezeCase)
auto axis_1 = ngraph::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {1});
auto in_1 = std::make_shared<ngraph::opset5::Unsqueeze>(Y, axis_1);
auto shape_of = std::make_shared<opset5::ShapeOf>(X);
auto batch_dimension = ngraph::op::util::make_try_fold<ngraph::opset7::Gather>(
shape_of,
ngraph::opset5::Constant::create(ngraph::element::i64, {1}, {0}),
ngraph::opset5::Constant::create(ngraph::element::i64, {}, {0}));
auto seq_lengths =
std::make_shared<opset5::Broadcast>(ngraph::opset5::Constant::create(element::i32, Shape{}, {2}),
batch_dimension);
auto seq_lengths = create_seq_len(X);
auto rnn_sequence = std::make_shared<opset5::RNNSequence>(X,
in_1,
seq_lengths,
@@ -601,7 +595,7 @@ TEST(TransformationTests, ConvertTensorIteratorToGRUSequence) {
auto R = ngraph::opset5::Constant::create(ngraph::element::f32, ngraph::Shape{ 1, 384, 128 }, r_val);
auto B = ngraph::opset5::Constant::create(ngraph::element::f32, ngraph::Shape{ 1, 384 }, b_val);
auto seq_lengths = ngraph::opset5::Constant::create(element::i32, Shape{1}, {2});
auto seq_lengths = create_seq_len(X);
auto gru_sequence = std::make_shared<opset5::GRUSequence>(X, in_1, seq_lengths, W, R, B, 128, op::RecurrentSequenceDirection::FORWARD);
auto axis_out = ngraph::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {1});
auto out_0 = std::make_shared<ngraph::opset5::Squeeze>(gru_sequence->output(0), axis_out);
@@ -674,12 +668,7 @@ TEST(TransformationTests, ConvertTensorIteratorToGRUSequenceDynamicReshapeCase)
auto R = ngraph::opset5::Constant::create(ngraph::element::f32, ngraph::Shape{ 1, 384, 128 }, r_val);
auto B = ngraph::opset5::Constant::create(ngraph::element::f32, ngraph::Shape{ 1, 384 }, b_val);
auto shape_of = std::make_shared<opset5::ShapeOf>(X);
auto batch_dimension = ngraph::op::util::make_try_fold<ngraph::opset7::Gather>(
shape_of,
ngraph::opset5::Constant::create(ngraph::element::i64, { 1 }, { 0 }),
ngraph::opset5::Constant::create(ngraph::element::i64, {}, { 0 }));
auto seq_lengths = std::make_shared<opset5::Broadcast>(ngraph::opset5::Constant::create(element::i32, Shape{}, { 2 }), batch_dimension);
auto seq_lengths = create_seq_len(X);
auto gru_sequence = std::make_shared<opset5::GRUSequence>(X, in_1, seq_lengths, W, R, B, 128, op::RecurrentSequenceDirection::FORWARD);
auto axis_out = ngraph::opset5::Constant::create(ngraph::element::i64, ngraph::Shape{ 1 }, { 1 });
@@ -753,14 +742,7 @@ TEST(TransformationTests, ConvertTensorIteratorToGRUSequenceDynamicSqueezeCase)
auto R = ngraph::opset5::Constant::create(ngraph::element::f32, ngraph::Shape{1, 384, 128}, r_val);
auto B = ngraph::opset5::Constant::create(ngraph::element::f32, ngraph::Shape{1, 384}, b_val);
auto shape_of = std::make_shared<opset5::ShapeOf>(X);
auto batch_dimension = ngraph::op::util::make_try_fold<ngraph::opset7::Gather>(
shape_of,
ngraph::opset5::Constant::create(ngraph::element::i64, {1}, {0}),
ngraph::opset5::Constant::create(ngraph::element::i64, {}, {0}));
auto seq_lengths =
std::make_shared<opset5::Broadcast>(ngraph::opset5::Constant::create(element::i32, Shape{}, {2}),
batch_dimension);
auto seq_lengths = create_seq_len(X);
auto gru_sequence = std::make_shared<opset5::GRUSequence>(X,
in_1,