[CPU] migrate cpu plugin api 2.0 (#18124)

* [CPU] CPU plugin migrates to plugin API 2.0

* Fix legacy config/metric issue

* Fix some issue of ov_cpu_func_tests

1. set_tensors_impl segment fault
2. ov::loaded_from_cache unsupported issue

* Resolve some comments

1. ov::loaded_from_cache issue
2. throw_if_cancelled issue
3. import_model issue
4. set_tensor_impl issue
5. batched_inference issue

* Fix dynamic shape inference issue

* Fix build error

* keep original model info in infer_request

* Fix minor error

* cache internal tensors for input/output precision change

* Disable import model test cases with precision changes

* fix precision issue

* Fix issue for import model

* Fix InferRequestCancellationTests exception issue

* Skip InferRequestIOBBlobTest.*secondCallGetInputDoNotReAllocateData due to new plugin api have different behavior

* Fix graph name issue

* Fix ROI issues

* Fix Transpose shape issue

* Skip vie::Version test due to change to ov::Version

* Solve input port name changes issue

* Solve preprocess layout issue

* Fix minor issue

* tidy up code

* Fix conflict after rebase

* Fix Windows build warning

* Add aux tensors for precision change issue

* Fix import/export model issue

* WA single layer name changed by preprocess

* Revert "WA single layer name changed by preprocess"

This reverts commit bc8fcdd43c.

* Skip some legacy tests due to plugin api 2.0 is enabled

1. skip some python legacy tests for plugin api 2.0 some different behaviors
2. skip some smoke tests due to output port name was changed

* Fix 2 build warnings

* Skip some AUTO plugin tests

* Fix property issue caused by AUTO plugin

* Skip PSROIPooling issues

* Follow header files reference policy

* Split out transformation fixing for nop_elimination

* Fix AUTO plugin mismatch issue for get_tensor function

* Fix aux tensor shape issue

* Fix tensor shape issue

* WA python sync inference sample's segmentfault issue

* Fix reshape issue for dynamic inference

* Fixed incorrect tensor name in e2e test

Fixe issue: e2e ONNX_Customized_Cascade_Rcnn_api_2_True_batch_1_device_CPU_precision_FP325den8cnk

* Fix python segmentfault issue of plugin api 2.0

* Fix python segmentfault issue of plugin api 2.0

* Revert "Fix python segmentfault issue of plugin api 2.0"

This reverts commit 6f502e5d86.

* Fix onnx_duplicated_output_name due to empty tensor

Co-authored-by: Bell, Song <bell.song@intel.com>

* Remove redundant code

* Remove python segment fault WA

* Keep rt_info to fix test failure in case of legacy public api

* Fix output port names missing issue

* Adress some reviewers' comments

* Restore OnnxBackendNodeModelTest::test_maxpool_with_argmax_2d_precomputed_pads_cpu after fixing has been merged

* Resolve tensor sharing issue when there are same name output port name

In some case, model has 2 or more same name input/output ports, they aslo have the same
precision and partial_shape. Compiled_model will share the same ov::Descriptor::Tensor pointer
and ov::Tensor between multiple such ports.
Considered solving python segment fault issue to create seperated input/output ports, which also
need handle such tensor shared case, this patch will do it.

* Resolve tensor sharing issue when there are same name output port name

In some case, model has 2 or more same name input/output ports, they aslo have the same
precision and partial_shape. Compiled_model will share the same ov::Descriptor::Tensor pointer
and ov::Tensor between multiple such ports.
Considered solving python segment fault issue to create seperated input/output ports, which also
need handle such tensor shared case, this patch will do it.

* Better method to find shrared tensor desc

* rename with snake_case style

* Remove ngraph header files

* Keep external_ptr naming

* Add OPENVINO_SUPPRESS_DEPRECATED for some legacy code

* Use port's tensor_ptr to replace creating new tensor_ptr

* Resolve some reviewer comments

* Implement ov::IInferRequestInternalWrapper::GetPreProcess to recover python GetPrepProcess tests

* Remove unnecessary header files reference

* Assert the risk of precision change and reorder at the same time

* Modify legacy python test to fit plugin api 2.0 behavior

* Recover smoke_Transpose(2|4|5|6)D/TransposeLayerTest.CompareWithRefs due to fixing is merged

* Fix typo issue

* Address reviewer's comments

* Disable precision coversion

* Fix error when CpuBlockedMemoryDesc

* Remove precision mismatch WA

* WA precision issue for query_model

* Solve precision mismatch between compiled model and graph

* Fixe failure of query_model

* Rebase to new plugin api update

* Recover the test cases of precision mismatch

* Try to fix name changing for graph model

* Remove tets code

* Remove fp64

* Rebase to new plugin api update

* Update for some failure cases

* Fix bert_benchmark failure issue

* Avoid segment fault in arm acl

Legacy public api + cpu plugin api will add convert op by preprocess by default for unsupported precision,
but ACLConvertExecutor cannot support dimension > 6, so this test will be segment fault due to dimension > 6

smoke_TestNumpyBroadcastNgraphEvaluate/BroadcastLayerTest.CompareWithRefs/targetShape=(1.2.3.4.5.6.7.8.9.10)_axesMapping=()_mode=numpy_inShape=(1.2.1.4.1.6.1.8.1.10)_inNPrec=I8_trgDev=CPU
smoke_TestNumpyBroadcastNgraphEvaluate/BroadcastLayerTest.CompareWithRefs/targetShape=(1.2.3.4.5.6.7.8.9.10)_axesMapping=()_mode=numpy_inShape=(1.2.1.4.1.6.1.8.1.10)_inNPrec=U8_trgDev=CPU

* Remove precision change from preprocess to avoid ACL unsupport convert dim > 6

* ACLConvertExecutor cannot support dimension > 6, don't let preprocess to add Convert

* Revert "ACLConvertExecutor cannot support dimension > 6, don't let preprocess to add Convert"

This reverts commit fd7a8b35af.

* Revert "Remove precision change from preprocess to avoid ACL unsupport convert dim > 6"

This reverts commit 3c2d9a5f17.

* Debug

* Debug incorrect precision checking issue

* Debug Eltwise FP64 unsupported issue

* Add logs for precision

* debug log

* Update for new dependent PRs merged

* Fix failure caused by preprocess

Fix below failures due to cannot find ops by name
     smoke_LPT/ReduceMaxTransformation.CompareWithRefImpl/f32_[1,3,10,10]_CPU_f32__256*

* Fix build error

* Fix failure caused by missing code during rebase

* Add debug

* Fix precision unsupport issue

* U16/I16/U64 precision support

* Resolve the issue of f64 reorder

Fix below issue:
Cannot create reorder primitive: unsupported reorder case

* Fix convert multiple child edge issue

* Solve ROI tensor failure issues

* Temporarily disable num_nodes comparison

* Only change convert precision for fp64

* Put convert precision change before reorder to avoid confusion

* Add debug log for transformation

* Fix rebase confilict

* Fix clang issue

* Temporarily disable test_infer_mixed_values python test of bf16

* Solve issue of smoke_ConvertCPULayerTest_BOOL_Dynamic_inputPRC=BF16 choose FP32 primType rather than BP16 primType

* Fix issue of pytorch_tests/test_outer.py

There are 2 output ports, but with the same port name, they should share the same tensor.

* Fix arm cannot find Eltwise executor issue

smoke_SetBlobCPU/SetBlobTest.CompareWithRefs/Type=INPUT_Device=CPU_PrecisionInNet=FP16_PrecisionInNgraph=BOOL
will report below error:
	 [ GENERAL_ERROR ] Supported Eltwise executor is not found
It need change convert precision to avoid such problem.

* Fix memory overwritten issue

* Temporarily skip arm fp16 SetBlobTest

* Fix compile error after rebase

* Restore smoke_IsOp test due to fixing pr merged

* Fix float to bf16 issue in avx2 isa

* solve onnx test xfail issue

* Skip test cases that ARM Eltwise executor FP16 is not supported

smoke_SetBlobCPU/SetBlobTest.CompareWithRefs/Type=INPUT_Device=CPU_PrecisionInNet=FP16_PrecisionInNgraph=BOOL
smoke_SetBlobCPU/SetBlobTest.CompareWithRefs/Type=BOTH_Device=CPU_PrecisionInNet=FP16_PrecisionInNgraph=BOOL

      [ GENERAL_ERROR ] Supported Eltwise executor is not found

* [CPU] improve reorder to support any precision

* Implement ReorderExecutor

* Fix builld error

* Not cache executor due to its primitive has been cached

* Keep convert one time at most

At most insert one convert if needed, if still cannot do reorder it will throw exception rather than insert the second convert
For example, below reorder will not be supported:
   FP64<->I64/U64/U32
   U32<->I64/U64
   U32<->I16/U16
   FP64<->FP64
   BIN<->BIN

* Only do conversion if layout is same

* update for only convert case

* Update for reviewer comments

* update for failure cases

* Address reviewer comments

* Update rebase issue

* minor update

* Solve unsupported precision issue in tranfromation rather than init_edge

* Remove unnecessary convert in init_edge

* Minor changes

* Update Reorder::reorderData

* Solve issue if only coversion without reorder

* Address reviewer comments

* Address reviewer comments

* Keep exception for unsuported precision

* update

* Revert reorder executor implement

* Solve float->bool issue on transformation pipeline

* Solve I64 is not supported issues

* Solve reviewer's comments

* Fixed dynamic top_k node issue

* Skip nhwc and nChw16c test cases for ConvertLayer

* Update for reviewers' comments

* Fix some failures

* Update for several failure cases

* Update for apiConformanceTests failures

* Fix incorrect node name after import model

* update

* update comments

* Solve issue of smoke_MatMul_NoTranspose and smoke_MatMul_BothTranspose

* Fixed AlignMatMulInputRanks scalar issue

* Address reviewers' comments, remove redundant path in graph.cpp

* Remove test_div_uint8_cpu from xfail_issue_58676

* Solve invalid number of nodes for smoke_Snippets_BroadcastSelect

* ConstantResultSubgraphTest of u16/i16/u32/i64/u64

* restore smoke_SetBlobCPU BOOL tests for arm

* [CPU] Fix ARM precision issue

ARM64 ACL prefers fp16 than fp32, API 2.0 requires input/output precision not changes,
then fp32 input will trigger convert node is added to convert fp32 to fp16.

* Solve some ARM64 failures

* Fix arm64 InferRequestVariableStateTest tests out of memory issue

ARM64 will force fp16 precision, which cause states memory can be fp16, so memcpy to state_memory
cannot use float * element_size, else it will be out of memory bound.

* Skip 2 arm64 tests caused by forcing fp16 precision

* Revert "Fix arm64 InferRequestVariableStateTest tests out of memory issue"

This reverts commit 3e12bd48c2.

* Fix python test_get_profiling_info failure issue

---------

Co-authored-by: Bell, Song <bell.song@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>

src/bindings/python/tests/test_runtime/test_core.py

@@ -65,7 +65,8 @@ def test_core_class(device):
 
     input_tensor = Tensor(input_data)
     results = request.infer({"data": input_tensor})
-    assert np.allclose(results[list(results)[0]], expected_output)
+    # convert node may be introduced by API 2.0, which brings some deviation
+    assert np.allclose(results[list(results)[0]], expected_output, 1e-4, 1e-4)
 
 
 # request - https://docs.pytest.org/en/7.1.x/reference/reference.html#request

src/bindings/python/tests/test_runtime/test_infer_request.py

@@ -109,8 +109,8 @@ def test_get_profiling_info(device):
     request.infer({tensor_name: img})
     assert request.latency > 0
     prof_info = request.get_profiling_info()
-    soft_max_node = next(node for node in prof_info if node.node_name == "fc_out")
-    assert "Softmax" in soft_max_node.node_type
+    soft_max_node = next(node for node in prof_info if node.node_type == "Softmax")
+    assert soft_max_node
     assert soft_max_node.status == ProfilingInfo.Status.EXECUTED
     assert isinstance(soft_max_node.real_time, datetime.timedelta)
     assert isinstance(soft_max_node.cpu_time, datetime.timedelta)

src/bindings/python/tests_compatibility/__init__.py

@@ -135,7 +135,6 @@ xfail_issue_49754 = xfail_test(reason="RuntimeError: Unsupported dynamic ops: v1
 xfail_issue_52463 = xfail_test(reason="test_operator_add_size1_singleton_broadcast_cpu - "
                                       "Not equal to tolerance")
 xfail_issue_58033 = xfail_test(reason="Einsum operation misses support for complex ellipsis equations")
-xfail_issue_58676 = xfail_test(reason="AssertionError: Not equal to tolerance rtol=0.001, atol=1e-07")
 xfail_issue_onnx_models_140 = xfail_test(reason="https://github.com/onnx/models/issues/140")
 
 xfail_issue_63033 = xfail_test(reason="BatchNormalization: Training mode is not supported")

src/bindings/python/tests_compatibility/test_inference_engine/test_ExecutableNetwork.py

@@ -131,7 +131,8 @@ def test_wait_before_start(device):
   requests = exec_net.requests
   for id in range(num_requests):
       status = requests[id].wait()
-      assert status == ie.StatusCode.INFER_NOT_STARTED
+      # Plugin API 2.0 has the different behavior will not return this status
+      # assert status == ie.StatusCode.INFER_NOT_STARTED
       request_handler = exec_net.start_async(request_id=id, inputs={'parameter': img})
       status = requests[id].wait()
       assert status == ie.StatusCode.OK

src/bindings/python/tests_compatibility/test_inference_engine/test_InferRequest.py

@@ -302,7 +302,8 @@ def test_async_infer_callback_wait_before_start(device):
     request = exec_net.requests[0]
     request.set_completion_callback(callback)
     status = request.wait()
-    assert status == ie.StatusCode.INFER_NOT_STARTED
+    # Plugin API 2.0 has the different behavior will not return this status
+    # assert status == ie.StatusCode.INFER_NOT_STARTED
     request.async_infer({'parameter': img})
     status = request.wait()
     assert status == ie.StatusCode.OK
@@ -320,7 +321,8 @@ def test_async_infer_callback_wait_in_callback(device):
             self.cv = threading.Condition()
             self.request.set_completion_callback(self.callback)
             self.status_code = self.request.wait(ie.WaitMode.STATUS_ONLY)
-            assert self.status_code == ie.StatusCode.INFER_NOT_STARTED
+            # Plugin API 2.0 has the different behavior will not return this status
+            # assert self.status_code == ie.StatusCode.INFER_NOT_STARTED
 
         def callback(self, statusCode, userdata):
             self.status_code = self.request.wait(ie.WaitMode.STATUS_ONLY)
@@ -508,7 +510,7 @@ def test_set_blob_with_incorrect_size(device):
     blob = ie.Blob(tensor_desc)
     with pytest.raises(RuntimeError) as e:
         exec_net.requests[0].set_blob("data", blob)
-    assert f"Input blob size is not equal network input size" in str(e.value)
+    assert f"Can't set the input tensor" in str(e.value)
     with pytest.raises(RuntimeError) as e:
         exec_net.requests[0].set_blob("out", blob)
-    assert f"Output blob size is not equal network output size" in str(e.value)
+    assert f"Can't set the output tensor" in str(e.value)

src/bindings/python/tests_compatibility/test_onnx/test_zoo_models.py

@@ -21,7 +21,6 @@ from tests_compatibility import (
     xfail_issue_47495,
     xfail_issue_48145,
     xfail_issue_48190,
-    xfail_issue_58676,
     xfail_issue_78843,
     xfail_issue_onnx_models_140)
 

src/frontends/onnx/tests/__init__.py

@@ -45,7 +45,6 @@ xfail_issue_90649 = xfail_test(reason="RuntimeError: OV does not support the fol
                                       "BlackmanWindow, DFT, HammingWindow, HannWindow, LayerNormalization, "
                                       "MelWeightMatrix, SequenceMap, STFT")
 xfail_issue_35923 = xfail_test(reason="RuntimeError: PReLU without weights is not supported")
-xfail_issue_35927 = xfail_test(reason="RuntimeError: B has zero dimension that is not allowable")
 xfail_issue_38091 = xfail_test(reason="AssertionError: Mismatched elements")
 xfail_issue_38699 = xfail_test(reason="RuntimeError: OV does not support the following ONNX operations: "
                                       "ai.onnx.preview.training.Gradient")
@@ -148,7 +147,6 @@ xfail_issue_82039 = xfail_test(reason="Unsupported data type Optional, RuntimeEr
 
 xfail_issue_86911 = xfail_test(reason="LSTM_Seq_len_unpacked - AssertionError: zoo models results mismatch")
 xfail_issue_91151 = xfail_test(reason="RuntimeError: model input (shape={3,4}) and blob (shape=(1)) are incompatible")
-xfail_issue_91490 = xfail_test(reason="y has zero dimension which is not allowed")
 xfail_issue_101965 = xfail_test(reason="Mismatch with numpy-based expected results.")
 xfail_issue_113506 = xfail_test(reason="Unsupported operation of type: LSTMSequence Node expects 7 inputs. Actual: 8")
 

src/frontends/onnx/tests/tests_python/test_backend.py

@@ -46,7 +46,6 @@ from tests import (
     xfail_issue_82039,
     xfail_issue_90649,
     xfail_issue_91151,
-    xfail_issue_91490,
     xfail_issue_99949,
     xfail_issue_99950,
     xfail_issue_99952,
@@ -290,11 +289,6 @@ tests_expected_to_fail = [
         xfail_issue_38710,
         "OnnxBackendNodeModelTest.test_reshape_allowzero_reordered_cpu",
     ),
-    (
-        xfail_issue_91490,
-        "OnnxBackendNodeModelTest.test_tril_zero_cpu",
-        "OnnxBackendNodeModelTest.test_triu_zero_cpu",
-    ),
     (
         skip_dynamic_model,
         "OnnxBackendNodeModelTest.test_triu_one_row_cpu",
@@ -575,8 +569,6 @@ tests_expected_to_fail = [
         xfail_issue_99973,
         "OnnxBackendNodeModelTest.test_split_1d_uneven_split_opset18_cpu",
         "OnnxBackendNodeModelTest.test_split_2d_uneven_split_opset18_cpu",
-        "OnnxBackendNodeModelTest.test_split_zero_size_splits_opset13_cpu",
-        "OnnxBackendNodeModelTest.test_split_zero_size_splits_opset18_cpu",
     ),
     (
         xfail_issue_101965,

src/frontends/onnx/tests/tests_python/test_ops_reshape.py

@@ -15,8 +15,7 @@ from tests.tests_python.utils import (
     run_model,
     run_node,
 )
-from tests import (xfail_issue_35927,
-                   xfail_issue_44858,
+from tests import (xfail_issue_44858,
                    xfail_dynamic_rank)
 
 
@@ -126,7 +125,6 @@ def test_transpose():
     assert np.array_equal(graph_results, [expected_output])
 
 
-@xfail_issue_35927
 def test_slice_opset1():
     data = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
 

src/frontends/tensorflow/tests/compilation.cpp

@@ -20,7 +20,8 @@ TEST_F(CompileModelsTests, NgramCompilation) {
     ov::CompiledModel compiled_model = core.compile_model(model, "CPU");
     const auto runtime_model = compiled_model.get_runtime_model();
 
-    EXPECT_EQ(runtime_model->get_ordered_ops().size(), 4);
+    // A convert node will be inserted for CPU plugin API 2.0
+    EXPECT_EQ(runtime_model->get_ordered_ops().size(), 5);
     EXPECT_EQ(runtime_model->get_parameters().size(), 2);
     EXPECT_EQ(runtime_model->get_results().size(), 1);
 }

src/inference/src/blob_transform.cpp

@@ -363,7 +363,7 @@ void blob_copy(Blob::Ptr src, Blob::Ptr dst) {
 
     if (src->getTensorDesc().getPrecision() != dst->getTensorDesc().getPrecision())
         IE_THROW() << "Unimplemented blob transformation from precision " << src->getTensorDesc().getPrecision()
-                   << " to " << src->getTensorDesc().getPrecision();
+                   << " to " << dst->getTensorDesc().getPrecision();
 
     if (src->getTensorDesc().getDims() != dst->getTensorDesc().getDims())
         IE_THROW() << "Unimplemented blob transformation from different shapes ";

src/inference/src/cpp_interfaces/interface/ie_iinfer_request_internal.cpp

@@ -155,7 +155,7 @@ void IInferRequestInternal::SetBlob(const std::string& name, const Blob::Ptr& us
                                    ? InferenceEngine::details::product(foundInput->getTensorDesc().getDims())
                                    : 1;
             if (!isInputDynamic && dataSize != inputSize) {
-                IE_THROW() << "Input blob size is not equal network input size (" << dataSize << "!=" << inputSize
+                IE_THROW() << "Input tensor size is not equal network input size (" << dataSize << "!=" << inputSize
                            << ").";
             }
             _inputs[name] = userBlob;

src/inference/src/dev/converter_utils.cpp

@@ -495,7 +495,7 @@ class IInferRequestInternalWrapper : public InferenceEngine::IInferRequestIntern
             if (get_legacy_name_from_port(port) == legacy_name)
                 return port;
         }
-        OPENVINO_ASSERT(false, "Cannot find port with name: ", legacy_name);
+        OPENVINO_THROW("Failed to find input or output with name: \'", legacy_name, "\'");
     }
 
 public:

src/inference/src/dev/preprocessing/preprocessing.cpp

@@ -129,11 +129,12 @@ bool ov::pass::AddPreprocessing::run_on_model(const std::shared_ptr<ov::Model>&
             output_info->getLayout() != InferenceEngine::Layout::SCALAR) {
             std::stringstream stream;
             stream << output_info->getLayout();
-            preproc.output(i).tensor().set_layout(ov::Layout{stream.str()});
+            if (stream.str() == "NHWC") {
+                if (const_output.get_partial_shape().is_static() && const_output.get_shape().size() == 4)
+                    preproc.output(i).model().set_layout("NCHW");
+                preproc.output(i).postprocess().convert_layout(ov::Layout{stream.str()});
+            }
         }
-
-        if (const_output.get_partial_shape().is_static() && const_output.get_shape().size() == 4)
-            preproc.output(i).model().set_layout("NCHW");
     }
 
     ov::pass::Manager manager(get_pass_config());

src/inference/src/ie_common.cpp

@@ -18,6 +18,7 @@
 #include "ie_parameter.hpp"
 #include "ngraph/opsets/opset.hpp"
 #include "openvino/core/except.hpp"
+#include "openvino/runtime/exception.hpp"
 
 namespace InferenceEngine {
 IE_SUPPRESS_DEPRECATED_START
@@ -64,6 +65,8 @@ void Rethrow() {
         throw e;
     } catch (const InferenceEngine::InferCancelled& e) {
         throw e;
+    } catch (const ov::Cancelled& e) {
+        IE_THROW(InferCancelled) << e.what();
     } catch (const std::exception& e) {
         IE_THROW() << e.what();
     } catch (...) {

src/plugins/intel_cpu/src/async_infer_request.cpp

@@ -3,15 +3,19 @@
 //
 
 #include "async_infer_request.h"
-#include <memory>
 
-ov::intel_cpu::AsyncInferRequest::AsyncInferRequest(const InferenceEngine::IInferRequestInternal::Ptr& inferRequest,
-                                                    const InferenceEngine::ITaskExecutor::Ptr& taskExecutor,
-                                                    const InferenceEngine::ITaskExecutor::Ptr& callbackExecutor)
-    : InferenceEngine::AsyncInferRequestThreadSafeDefault(inferRequest, taskExecutor, callbackExecutor) {
-    static_cast<InferRequestBase*>(inferRequest.get())->SetAsyncRequest(this);
+ov::intel_cpu::AsyncInferRequest::AsyncInferRequest(
+    const std::shared_ptr<IInferRequest>& request,
+    const std::shared_ptr<ov::threading::ITaskExecutor>& task_executor,
+    const std::shared_ptr<ov::threading::ITaskExecutor>& callback_executor)
+    : ov::IAsyncInferRequest(request, task_executor, callback_executor) {
+    static_cast<SyncInferRequest*>(request.get())->set_async_request(this);
 }
 
 ov::intel_cpu::AsyncInferRequest::~AsyncInferRequest() {
-    StopAndWait();
+    stop_and_wait();
+}
+
+void ov::intel_cpu::AsyncInferRequest::throw_if_canceled() const {
+    check_cancelled_state();
 }

src/plugins/intel_cpu/src/async_infer_request.h

@@ -4,22 +4,21 @@
 
 #pragma once
 
-#include <string>
-#include <map>
-#include <cpp_interfaces/impl/ie_infer_async_request_thread_safe_default.hpp>
 #include "infer_request.h"
+#include "openvino/runtime/iasync_infer_request.hpp"
 
 namespace ov {
 namespace intel_cpu {
 
-class AsyncInferRequest : public InferenceEngine::AsyncInferRequestThreadSafeDefault {
+class AsyncInferRequest : public ov::IAsyncInferRequest {
 public:
-    AsyncInferRequest(const InferenceEngine::IInferRequestInternal::Ptr &inferRequest,
-                      const InferenceEngine::ITaskExecutor::Ptr &taskExecutor,
-                      const InferenceEngine::ITaskExecutor::Ptr &callbackExecutor);
+    AsyncInferRequest(const std::shared_ptr<IInferRequest>& request,
+                      const std::shared_ptr<ov::threading::ITaskExecutor>& task_executor,
+                      const std::shared_ptr<ov::threading::ITaskExecutor>& callback_executor);
     ~AsyncInferRequest();
+
+    void throw_if_canceled() const;
 };
 
-}   // namespace intel_cpu
-}   // namespace ov
-
+}  // namespace intel_cpu
+}  // namespace ov

src/plugins/intel_cpu/src/compiled_model.cpp

@@ -1,149 +1,131 @@
 // Copyright (C) 2018-2023 Intel Corporation
 // SPDX-License-Identifier: Apache-2.0
 //
+#include "ie_metric_helpers.hpp"  // must be included first
 
-#include <ie_metric_helpers.hpp>
-#include <precision_utils.h>
-#include "exec_network.h"
-#include <low_precision/low_precision.hpp>
-
+#include "compiled_model.h"
 #include "async_infer_request.h"
 #include "infer_request.h"
-#include "memory_state.h"
 #include "itt.h"
+#include "low_precision/low_precision.hpp"
+#include "memory_state.h"
+#include "nodes/memory.hpp"
+#include "openvino/core/type/element_type.hpp"
 #include "openvino/runtime/intel_cpu/properties.hpp"
 #include "serialize.h"
-#include "ngraph/type/element_type.hpp"
-#include "nodes/memory.hpp"
-#include <threading/ie_executor_manager.hpp>
+#include "threading/ie_executor_manager.hpp"
+#include "transformations/transformation_pipeline.h"
 #define FIX_62820 0
 #if FIX_62820 && ((IE_THREAD == IE_THREAD_TBB) || (IE_THREAD == IE_THREAD_TBB_AUTO))
-#include <threading/ie_tbb_streams_executor.hpp>
+#    include <threading/ie_tbb_streams_executor.hpp>
 #endif
-#include <threading/ie_cpu_streams_executor.hpp>
-#include <ie_system_conf.h>
-#include <ngraph/opsets/opset1.hpp>
-#include <transformations/utils/utils.hpp>
-#include <ie_ngraph_utils.hpp>
-#include "cpp_interfaces/interface/ie_iplugin_internal.hpp"
-#include "ie_icore.hpp"
+
 #include "openvino/runtime/properties.hpp"
 #include "openvino/util/common_util.hpp"
+#include "threading/ie_cpu_streams_executor.hpp"
+#include "transformations/utils/utils.hpp"
 
-#include <algorithm>
-#include <unordered_set>
-#include <utility>
+#include <cpu/x64/cpu_isa_traits.hpp>
 #include <cstring>
+#include <utility>
 
-using namespace InferenceEngine;
-using namespace InferenceEngine::details;
+using namespace ov::threading;
 
 namespace ov {
 namespace intel_cpu {
 
-InferenceEngine::IInferRequestInternal::Ptr
-ExecNetwork::CreateInferRequestImpl(const std::vector<std::shared_ptr<const ov::Node>>& inputs,
-                                    const std::vector<std::shared_ptr<const ov::Node>>& outputs) {
-    if (!this->_plugin || !_plugin->IsNewAPI())
-        return nullptr;
-    return std::make_shared<InferRequest>(inputs, outputs, std::static_pointer_cast<ExecNetwork>(shared_from_this()));
-}
-
-InferenceEngine::IInferRequestInternal::Ptr
-ExecNetwork::CreateInferRequestImpl(InferenceEngine::InputsDataMap networkInputs,
-                                    InferenceEngine::OutputsDataMap networkOutputs) {
-    return std::make_shared<LegacyInferRequest>(networkInputs, networkOutputs, std::static_pointer_cast<ExecNetwork>(shared_from_this()));
-}
-
-struct ImmediateSerialExecutor : public ITaskExecutor {
-    void run(InferenceEngine::Task task) override {
+struct ImmediateSerialExecutor : public ov::threading::ITaskExecutor {
+    void run(ov::threading::Task task) override {
         std::lock_guard<std::mutex> l{_mutex};
         task();
     }
     std::mutex _mutex;
 };
 
-ExecNetwork::ExecNetwork(const InferenceEngine::CNNNetwork &network,
-                         const Config &cfg,
-                         const ExtensionManager::Ptr& extMgr,
-                         const std::shared_ptr<InferenceEngine::IInferencePlugin>& plugin) :
-    InferenceEngine::ExecutableNetworkThreadSafeDefault{nullptr, nullptr},
-    extensionManager(extMgr),
-    _network(network),
-    _cfg{cfg},
-    _name{network.getName()} {
-    SetPointerToPlugin(plugin);
-    auto function = network.getFunction();
-    if (function == nullptr) {
-        IE_THROW() << "CPU plug-in doesn't support not ngraph-based model!";
-    }
-    bool isFloatModel = !ov::op::util::has_op_with_type<ngraph::op::FakeQuantize>(function);
+CompiledModel::CompiledModel(const std::shared_ptr<ov::Model>& model,
+                             const std::shared_ptr<const ov::IPlugin>& plugin,
+                             const Config& cfg,
+                             const ExtensionManager::Ptr& extMgr,
+                             const bool loaded_from_cache)
+    : ov::ICompiledModel::ICompiledModel(model, plugin),
+      m_model(model),
+      m_plugin(plugin),
+      m_cfg{cfg},
+      extensionManager(extMgr),
+      m_name{model->get_name()},
+      m_loaded_from_cache(loaded_from_cache) {
+    bool isFloatModel = !ov::op::util::has_op_with_type<ngraph::op::FakeQuantize>(m_model);
 
-    _mutex = std::make_shared<std::mutex>();
-    const auto& core = _plugin->GetCore();
+    m_mutex = std::make_shared<std::mutex>();
+    const auto& core = m_plugin->get_core();
     if (!core)
-        IE_THROW() << "Unable to get API version. Core is unavailable";
-    _cfg.isLegacyApi = !core->isNewAPI();
-
+        OPENVINO_THROW("Unable to get API version. Core is unavailable");
+    m_cfg.isLegacyApi = !core->is_new_api();
 
     if (cfg.exclusiveAsyncRequests) {
         // special case when all InferRequests are muxed into a single queue
-        _taskExecutor = _plugin->executorManager()->getExecutor("CPU");
+        m_task_executor = m_plugin->get_executor_manager()->get_executor("CPU");
     } else {
         auto streamsExecutorConfig =
             is_cpu_map_available()
-                ? _cfg.streamExecutorConfig
-                : InferenceEngine::IStreamsExecutor::Config::MakeDefaultMultiThreaded(_cfg.streamExecutorConfig,
-                                                                                      isFloatModel);
+                ? m_cfg.streamExecutorConfig
+                : IStreamsExecutor::Config::make_default_multi_threaded(m_cfg.streamExecutorConfig, isFloatModel);
         streamsExecutorConfig._name = "CPUStreamsExecutor";
-        _cfg.streamExecutorConfig._threads = streamsExecutorConfig._threads;
+        m_cfg.streamExecutorConfig._threads = streamsExecutorConfig._threads;
 #if FIX_62820 && (IE_THREAD == IE_THREAD_TBB || IE_THREAD == IE_THREAD_TBB_AUTO)
-        _taskExecutor = std::make_shared<TBBStreamsExecutor>(streamsExecutorConfig);
+        m_task_executor = std::make_shared<TBBStreamsExecutor>(streamsExecutorConfig);
 #else
-        _taskExecutor = _plugin->executorManager()->getIdleCPUStreamsExecutor(streamsExecutorConfig);
+        m_task_executor = m_plugin->get_executor_manager()->get_idle_cpu_streams_executor(streamsExecutorConfig);
 #endif
     }
     if (0 != cfg.streamExecutorConfig._streams) {
 #if FIX_62820 && (IE_THREAD == IE_THREAD_TBB || IE_THREAD == IE_THREAD_TBB_AUTO)
         // There is no additional threads but we still need serialize callback execution to preserve legacy behaviour
-        _callbackExecutor = std::make_shared<ImmediateSerialExecutor>();
+        m_callback_executor = std::make_shared<ImmediateSerialExecutor>();
 #else
-        _callbackExecutor = _plugin->executorManager()->getIdleCPUStreamsExecutor(
-                                IStreamsExecutor::Config{"CPUCallbackExecutor", 1, 0, IStreamsExecutor::ThreadBindingType::NONE});
+        m_callback_executor = m_plugin->get_executor_manager()->get_idle_cpu_streams_executor(
+            IStreamsExecutor::Config{"CPUCallbackExecutor", 1, 0, IStreamsExecutor::ThreadBindingType::NONE});
 #endif
     } else {
-        _callbackExecutor = _taskExecutor;
+        m_callback_executor = m_task_executor;
     }
-    int streams = std::max(1, _cfg.streamExecutorConfig._streams);
-    std::vector<Task> tasks; tasks.resize(streams);
-    _graphs.resize(streams);
-    if (_cfg.streamExecutorConfig._streams != 0) {
+
+    if (m_task_executor)
+        set_task_executor(m_task_executor);
+    if (m_callback_executor)
+        set_callback_executor(m_callback_executor);
+
+    int streams = std::max(1, m_cfg.streamExecutorConfig._streams);
+    std::vector<Task> tasks;
+    tasks.resize(streams);
+    m_graphs.resize(streams);
+    if (m_cfg.streamExecutorConfig._streams != 0) {
         auto all_graphs_ready = [&] {
-            return std::all_of(_graphs.begin(), _graphs.end(), [&] (Graph& graph) {
+            return std::all_of(m_graphs.begin(), m_graphs.end(), [&](Graph& graph) {
                 return graph.IsReady();
             });
         };
         do {
             for (auto&& task : tasks) {
                 task = [this] {
-                    ExecNetwork::GetGraph();
+                    CompiledModel::get_graph();
                 };
             }
-            _taskExecutor->runAndWait(tasks);
+            m_task_executor->run_and_wait(tasks);
         } while (!all_graphs_ready());
     } else {
-        ExecNetwork::GetGraph();
+        CompiledModel::get_graph();
     }
 
     // Save all MemoryLayer data tensors. Will use insight about mechanics
     // of MemoryLayer implementation. It uses output edge of MemoryLayer
     // producer as storage for tensor to keep it between infer calls.
-    if (_graphs.size() == 1) {
-        for (auto &node : GetGraph()._graph.GetNodes()) {
+    if (m_graphs.size() == 1) {
+        for (auto& node : get_graph()._graph.GetNodes()) {
             if (node->getType() == Type::MemoryInput) {
                 auto memoryNode = dynamic_cast<node::MemoryInput*>(node.get());
                 if (!memoryNode) {
-                    IE_THROW() << "Cannot cast " << node->getName() << " to MemoryInput";
+                    OPENVINO_THROW("Cannot cast ", node->getName(), " to MemoryInput");
                 }
                 auto state_store = memoryNode->getStore();
                 auto state_name = memoryNode->getId();
@@ -153,44 +135,44 @@ ExecNetwork::ExecNetwork(const InferenceEngine::CNNNetwork &network,
                 if (suffix_idx != std::string::npos)
                     state_name = state_name.substr(0, suffix_idx);
 
-                memoryStates.emplace_back(new VariableState(state_name, state_store));
+                m_memory_states.emplace_back(new VariableState(state_name, state_store));
             }
         }
     }
 }
 
-ExecNetwork::GraphGuard::Lock ExecNetwork::GetGraph() const {
+CompiledModel::GraphGuard::Lock CompiledModel::get_graph() const {
     int streamId = 0;
     int socketId = 0;
-    auto streamsExecutor = dynamic_cast<InferenceEngine::IStreamsExecutor*>(_taskExecutor.get());
+    auto streamsExecutor = std::dynamic_pointer_cast<IStreamsExecutor>(m_task_executor);
     if (nullptr != streamsExecutor) {
-        streamId = streamsExecutor->GetStreamId();
-        socketId = streamsExecutor->GetSocketId();
+        streamId = streamsExecutor->get_stream_id();
+        socketId = streamsExecutor->get_socket_id();
     }
-    auto graphLock = GraphGuard::Lock(_graphs[streamId % _graphs.size()]);
+    auto graphLock = GraphGuard::Lock(m_graphs[streamId % m_graphs.size()]);
     if (!graphLock._graph.IsReady()) {
         std::exception_ptr exception;
         auto makeGraph = [&] {
             try {
                 GraphContext::Ptr ctx;
                 {
-                    std::lock_guard<std::mutex> lock{*_mutex.get()};
+                    std::lock_guard<std::mutex> lock{*m_mutex.get()};
                     // disable weights caching if graph was created only once
-                    auto weightsCache = _cfg.streamExecutorConfig._streams != 1 ? _socketWeights[socketId] : nullptr;
-
+                    auto weightsCache = m_cfg.streamExecutorConfig._streams != 1 ? m_socketWeights[socketId] : nullptr;
                     auto isQuantizedFlag =
-                        (_cfg.lpTransformsMode == Config::On) &&
-                        ov::pass::low_precision::LowPrecision::isFunctionQuantized(_network.getFunction());
+                        (m_cfg.lpTransformsMode == Config::On) &&
+                        ov::pass::low_precision::LowPrecision::isFunctionQuantized(m_model);
 
-                    ctx = std::make_shared<GraphContext>(_cfg, extensionManager, weightsCache, isQuantizedFlag);
+                    ctx = std::make_shared<GraphContext>(m_cfg, extensionManager, weightsCache, isQuantizedFlag);
                 }
-                graphLock._graph.CreateGraph(_network, ctx);
+                const std::shared_ptr<const ov::Model> model = m_model;
+                graphLock._graph.CreateGraph(model, ctx);
             } catch (...) {
                 exception = std::current_exception();
             }
         };
         if (nullptr != streamsExecutor) {
-            streamsExecutor->Execute(makeGraph);
+            streamsExecutor->execute(makeGraph);
         } else {
             makeGraph();
         }
@@ -201,44 +183,29 @@ ExecNetwork::GraphGuard::Lock ExecNetwork::GetGraph() const {
     return graphLock;
 }
 
-InferenceEngine::IInferRequestInternal::Ptr ExecNetwork::CreateInferRequest() {
-    return CreateAsyncInferRequestFromSync<AsyncInferRequest>();
+std::shared_ptr<ov::ISyncInferRequest> CompiledModel::create_sync_infer_request() const {
+    m_numRequests++;
+    return std::make_shared<SyncInferRequest>(std::static_pointer_cast<const CompiledModel>(shared_from_this()));
 }
 
-std::shared_ptr<ngraph::Function> ExecNetwork::GetExecGraphInfo() {
-    if (_graphs.empty())
-        IE_THROW() << "No graph was found";
-
-    return GetGraph()._graph.dump();
+std::shared_ptr<ov::IAsyncInferRequest> CompiledModel::create_infer_request() const {
+    auto internal_request = create_sync_infer_request();
+    auto async_infer_request =
+        std::make_shared<AsyncInferRequest>(std::static_pointer_cast<SyncInferRequest>(internal_request),
+                                            get_task_executor(),
+                                            get_callback_executor());
+    return async_infer_request;
 }
 
-Parameter ExecNetwork::GetConfigLegacy(const std::string &name) const {
-    if (_graphs.empty())
-        IE_THROW() << "No graph was found";
-    /* legacy implementation return all the parameters which is actually not correct
-     * since they are not reconfigurable. Fixed for new API */
-    Config engConfig = GetGraph()._graph.getConfig();
-    auto option = engConfig._config.find(name);
-    if (option != engConfig._config.end()) {
-        return option->second;
-    } else {
-        IE_THROW() << "Unsupported ExecutableNetwork config key: " << name;
-    }
+std::shared_ptr<const ov::Model> CompiledModel::get_runtime_model() const {
+    if (m_graphs.empty())
+        OPENVINO_THROW("No graph was found");
+
+    return get_graph()._graph.dump();
 }
 
-/**
- * Only legacy parameters are supported.
- * No RW peroperties supported for new API.
- * All the RO properties are covered with GetMetric() method and
- * GetConfig() is not expected to be called by new API with params from new configuration API.
- */
-Parameter ExecNetwork::GetConfig(const std::string &name) const {
-    /* Internally legacy parameters are used with new API as part of migration procedure.
-     * This fallback can be removed as soon as migration completed */
-    return GetConfigLegacy(name);
-}
-
-InferenceEngine::Parameter ExecNetwork::GetMetricLegacy(const std::string &name, const GraphGuard& graph) const {
+ov::Any CompiledModel::get_metric_legacy(const std::string& name, const GraphGuard& graph) const {
+    OPENVINO_SUPPRESS_DEPRECATED_START
     if (name == METRIC_KEY(NETWORK_NAME)) {
         IE_SET_METRIC_RETURN(NETWORK_NAME, graph.dump()->get_friendly_name());
     } else if (name == METRIC_KEY(SUPPORTED_METRICS)) {
@@ -250,7 +217,7 @@ InferenceEngine::Parameter ExecNetwork::GetMetricLegacy(const std::string &name,
         IE_SET_METRIC_RETURN(SUPPORTED_METRICS, metrics);
     } else if (name == METRIC_KEY(SUPPORTED_CONFIG_KEYS)) {
         std::vector<std::string> configKeys;
-        for (auto && key : graph.getConfig()._config) {
+        for (auto&& key : graph.getConfig()._config) {
             configKeys.push_back(key.first);
         }
         IE_SET_METRIC_RETURN(SUPPORTED_CONFIG_KEYS, configKeys);
@@ -259,31 +226,38 @@ InferenceEngine::Parameter ExecNetwork::GetMetricLegacy(const std::string &name,
         auto option = engConfig._config.find(CONFIG_KEY(CPU_THROUGHPUT_STREAMS));
         IE_ASSERT(option != engConfig._config.end());
         auto streams = std::stoi(option->second);
-        IE_SET_METRIC_RETURN(OPTIMAL_NUMBER_OF_INFER_REQUESTS, static_cast<unsigned int>(
-            streams ? streams : 1));
+        IE_SET_METRIC_RETURN(OPTIMAL_NUMBER_OF_INFER_REQUESTS, static_cast<unsigned int>(streams ? streams : 1));
     } else {
-        IE_THROW() << "Unsupported ExecutableNetwork metric: " << name;
+        OPENVINO_THROW("Unsupported property: ", name);
     }
+    OPENVINO_SUPPRESS_DEPRECATED_END
 }
 
-InferenceEngine::Parameter ExecNetwork::GetMetric(const std::string &name) const {
-    if (_graphs.empty())
-        IE_THROW() << "No graph was found";
+ov::Any CompiledModel::get_property(const std::string& name) const {
+    if (m_graphs.empty())
+        OPENVINO_THROW("No graph was found");
+
+    if (name == ov::loaded_from_cache) {
+        return m_loaded_from_cache;
+    }
+
+    Config engConfig = get_graph()._graph.getConfig();
+    auto option = engConfig._config.find(name);
+    if (option != engConfig._config.end()) {
+        return option->second;
+    }
+
     // @todo Can't we just use local copy (_cfg) instead?
-    auto graphLock = GetGraph();
+    auto graphLock = get_graph();
     const auto& graph = graphLock._graph;
     const auto& config = graph.getConfig();
 
-    if (_cfg.isLegacyApi) {
-        return GetMetricLegacy(name, graph);
-    }
-
     auto RO_property = [](const std::string& propertyName) {
         return ov::PropertyName(propertyName, ov::PropertyMutability::RO);
     };
 
     if (name == ov::supported_properties) {
-        return std::vector<ov::PropertyName> {
+        return std::vector<ov::PropertyName>{
             RO_property(ov::supported_properties.name()),
             RO_property(ov::model_name.name()),
             RO_property(ov::optimal_number_of_infer_requests.name()),
@@ -310,20 +284,22 @@ InferenceEngine::Parameter ExecNetwork::GetMetric(const std::string &name) const
         return decltype(ov::model_name)::value_type(modelName);
     } else if (name == ov::optimal_number_of_infer_requests) {
         const auto streams = config.streamExecutorConfig._streams;
-        return decltype(ov::optimal_number_of_infer_requests)::value_type(streams); // ov::optimal_number_of_infer_requests has no negative values
+        return decltype(ov::optimal_number_of_infer_requests)::value_type(
+            streams);  // ov::optimal_number_of_infer_requests has no negative values
     } else if (name == ov::num_streams) {
         const auto streams = config.streamExecutorConfig._streams;
-        return decltype(ov::num_streams)::value_type(streams); // ov::num_streams has special negative values (AUTO = -1, NUMA = -2)
+        return decltype(ov::num_streams)::value_type(
+            streams);  // ov::num_streams has special negative values (AUTO = -1, NUMA = -2)
     } else if (name == ov::affinity) {
         const auto affinity = config.streamExecutorConfig._threadBindingType;
         switch (affinity) {
-        case InferenceEngine::IStreamsExecutor::ThreadBindingType::NONE:
+        case IStreamsExecutor::ThreadBindingType::NONE:
             return ov::Affinity::NONE;
-        case InferenceEngine::IStreamsExecutor::ThreadBindingType::CORES:
+        case IStreamsExecutor::ThreadBindingType::CORES:
             return ov::Affinity::CORE;
-        case InferenceEngine::IStreamsExecutor::ThreadBindingType::NUMA:
+        case IStreamsExecutor::ThreadBindingType::NUMA:
             return ov::Affinity::NUMA;
-        case InferenceEngine::IStreamsExecutor::ThreadBindingType::HYBRID_AWARE:
+        case IStreamsExecutor::ThreadBindingType::HYBRID_AWARE:
             return ov::Affinity::HYBRID_AWARE;
         }
         return ov::Affinity::NONE;
@@ -353,21 +329,23 @@ InferenceEngine::Parameter ExecNetwork::GetMetric(const std::string &name) const
         const auto perfHintNumRequests = config.perfHintsConfig.ovPerfHintNumRequests;
         return decltype(ov::hint::num_requests)::value_type(perfHintNumRequests);
     } else if (name == ov::execution_devices) {
-        return decltype(ov::execution_devices)::value_type{_plugin->GetName()};
+        return decltype(ov::execution_devices)::value_type{m_plugin->get_device_name()};
     } else if (name == ov::intel_cpu::denormals_optimization) {
-        return decltype(ov::intel_cpu::denormals_optimization)::value_type(config.denormalsOptMode == Config::DenormalsOptMode::DO_On);
+        return decltype(ov::intel_cpu::denormals_optimization)::value_type(config.denormalsOptMode ==
+                                                                           Config::DenormalsOptMode::DO_On);
     } else if (name == ov::intel_cpu::sparse_weights_decompression_rate) {
-        return decltype(ov::intel_cpu::sparse_weights_decompression_rate)::value_type(config.fcSparseWeiDecompressionRate);
+        return decltype(ov::intel_cpu::sparse_weights_decompression_rate)::value_type(
+            config.fcSparseWeiDecompressionRate);
     }
     /* Internally legacy parameters are used with new API as part of migration procedure.
      * This fallback can be removed as soon as migration completed */
-    return GetMetricLegacy(name, graph);
+    return get_metric_legacy(name, graph);
 }
 
-void ExecNetwork::Export(std::ostream& modelStream) {
-    CNNNetworkSerializer serializer(modelStream, extensionManager);
-    serializer <<_network;
+void CompiledModel::export_model(std::ostream& modelStream) const {
+    ModelSerializer serializer(modelStream, extensionManager);
+    serializer << m_model;
 }
 
-}   // namespace intel_cpu
-}   // namespace ov
+}  // namespace intel_cpu
+}  // namespace ov

src/plugins/intel_cpu/src/compiled_model.h

@@ -0,0 +1,89 @@
+// Copyright (C) 2018-2023 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#pragma once
+
+#include <string>
+#include <vector>
+
+#include "extension_mngr.h"
+#include "graph.h"
+#include "graph_context.h"
+#include "openvino/runtime/icompiled_model.hpp"
+#include "openvino/runtime/iinfer_request.hpp"
+#include "openvino/runtime/iplugin.hpp"
+#include "openvino/runtime/isync_infer_request.hpp"
+#include "openvino/runtime/threading/thread_local.hpp"
+
+namespace ov {
+namespace intel_cpu {
+
+class CompiledModel : public ov::ICompiledModel {
+public:
+    typedef std::shared_ptr<CompiledModel> Ptr;
+
+    CompiledModel(const std::shared_ptr<ov::Model>& model,
+                  const std::shared_ptr<const ov::IPlugin>& plugin,
+                  const Config& cfg,
+                  const ExtensionManager::Ptr& extMgr,
+                  const bool loaded_from_cache = false);
+
+    std::shared_ptr<ov::IAsyncInferRequest> create_infer_request() const override;
+
+    void export_model(std::ostream& model) const override;
+
+    std::shared_ptr<const ov::Model> get_runtime_model() const override;
+
+    ov::Any get_property(const std::string& name) const override;
+
+    void set_property(const ov::AnyMap& properties) override {
+        OPENVINO_ASSERT_HELPER(::ov::NotImplemented,
+                               "",
+                               false,
+                               "Not Implemented",
+                               "CompiledModel::set_property is not supported by CPU plugin!");
+    };
+
+private:
+    std::shared_ptr<ov::ISyncInferRequest> create_sync_infer_request() const override;
+    friend class SyncInferRequest;
+
+    const std::shared_ptr<ov::Model> m_model;
+    std::vector<std::shared_ptr<ov::IVariableState>> m_memory_states;
+    const std::shared_ptr<const ov::IPlugin> m_plugin;
+    std::shared_ptr<ov::threading::ITaskExecutor> m_task_executor = nullptr;      //!< Holds a task executor
+    std::shared_ptr<ov::threading::ITaskExecutor> m_callback_executor = nullptr;  //!< Holds a callback executor
+
+    // Generic synchronization primitive on CompiledModel level.
+    // Usage example: helps to avoid data races during CPU Graph initialization in multi-streams scenario
+    std::shared_ptr<std::mutex> m_mutex;
+    Config m_cfg;
+    ExtensionManager::Ptr extensionManager;
+    mutable std::atomic_int m_numRequests = {0};
+    std::string m_name;
+    struct GraphGuard : public Graph {
+        std::mutex _mutex;
+        struct Lock : public std::unique_lock<std::mutex> {
+            explicit Lock(GraphGuard& graph) : std::unique_lock<std::mutex>(graph._mutex), _graph(graph) {}
+            GraphGuard& _graph;
+        };
+    };
+
+    const bool m_loaded_from_cache;
+    // WARNING: Do not use m_graphs directly.
+    mutable std::deque<GraphGuard> m_graphs;
+    mutable SocketsWeights m_socketWeights;
+
+    /* WARNING: Use get_graph() function to get access to graph in current stream.
+     * NOTE: Main thread is interpreted as master thread of external stream so use this function to get access to graphs
+     *       even from main thread
+     */
+    GraphGuard::Lock get_graph() const;
+
+    ov::Any get_metric_legacy(const std::string& name, const GraphGuard& graph) const;
+};
+
+}   // namespace intel_cpu
+}   // namespace ov
+

src/plugins/intel_cpu/src/config.cpp

@@ -16,6 +16,7 @@
 
 #include "cpp_interfaces/interface/ie_internal_plugin_config.hpp"
 #include "openvino/core/type/element_type_traits.hpp"
+#include "openvino/runtime/intel_cpu/properties.hpp"
 #include "openvino/runtime/properties.hpp"
 #include "utils/debug_capabilities.h"
 #include "cpu/x64/cpu_isa_traits.hpp"
@@ -23,31 +24,31 @@
 namespace ov {
 namespace intel_cpu {
 
-using namespace InferenceEngine;
+using namespace ov::threading;
 using namespace dnnl::impl::cpu::x64;
 
 Config::Config() {
     // this is default mode
 #if defined(__APPLE__) || defined(_WIN32)
-    streamExecutorConfig._threadBindingType = InferenceEngine::IStreamsExecutor::NONE;
+    streamExecutorConfig._threadBindingType = IStreamsExecutor::NONE;
 #else
-    streamExecutorConfig._threadBindingType = InferenceEngine::IStreamsExecutor::CORES;
+    streamExecutorConfig._threadBindingType = IStreamsExecutor::CORES;
 #endif
 
 // for the TBB code-path, additional configuration depending on the OS and CPU types
 #if (IE_THREAD == IE_THREAD_TBB || IE_THREAD == IE_THREAD_TBB_AUTO)
 #    if defined(__APPLE__) || defined(_WIN32)
     // 'CORES' is not implemented for Win/MacOS; so the 'NONE' or 'NUMA' is default
-    auto numaNodes = getAvailableNUMANodes();
+    auto numaNodes = get_available_numa_nodes();
     if (numaNodes.size() > 1) {
-        streamExecutorConfig._threadBindingType = InferenceEngine::IStreamsExecutor::NUMA;
+        streamExecutorConfig._threadBindingType = IStreamsExecutor::NUMA;
     } else {
-        streamExecutorConfig._threadBindingType = InferenceEngine::IStreamsExecutor::NONE;
+        streamExecutorConfig._threadBindingType = IStreamsExecutor::NONE;
     }
 #    endif
 
-    if (getAvailableCoresTypes().size() > 1 /*Hybrid CPU*/) {
-        streamExecutorConfig._threadBindingType = InferenceEngine::IStreamsExecutor::HYBRID_AWARE;
+    if (get_available_cores_types().size() > 1 /*Hybrid CPU*/) {
+        streamExecutorConfig._threadBindingType = IStreamsExecutor::HYBRID_AWARE;
     }
 #endif
 
@@ -69,16 +70,17 @@ void Config::applyDebugCapsProperties() {
 }
 #endif
 
-void Config::readProperties(const std::map<std::string, std::string> &prop, const ModelType modelType) {
-    const auto streamExecutorConfigKeys = streamExecutorConfig.SupportedKeys();
+void Config::readProperties(const ov::AnyMap& prop, const ModelType modelType) {
+    const auto streamExecutorConfigKeys =
+        streamExecutorConfig.get_property(ov::supported_properties.name()).as<std::vector<std::string>>();
     const auto hintsConfigKeys = perfHintsConfig.SupportedKeys();
     for (const auto& kvp : prop) {
         const auto& key = kvp.first;
-        const auto& val = kvp.second;
+        const auto& val = kvp.second.as<std::string>();
         IE_SUPPRESS_DEPRECATED_START
         if (streamExecutorConfigKeys.end() !=
             std::find(std::begin(streamExecutorConfigKeys), std::end(streamExecutorConfigKeys), key)) {
-            streamExecutorConfig.SetConfig(key, val);
+            streamExecutorConfig.set_property(key, val);
             if (key == ov::affinity.name()) {
                 const auto affinity_val = ov::util::from_string(val, ov::affinity);
                 if (affinity_val == ov::Affinity::CORE || affinity_val == ov::Affinity::HYBRID_AWARE) {
@@ -92,10 +94,10 @@ void Config::readProperties(const std::map<std::string, std::string> &prop, cons
         } else if (hintsConfigKeys.end() != std::find(hintsConfigKeys.begin(), hintsConfigKeys.end(), key)) {
             perfHintsConfig.SetConfig(key, val);
         } else if (key == ov::hint::enable_cpu_pinning.name()) {
-            if (val == PluginConfigParams::YES) {
+            if (val == InferenceEngine::PluginConfigParams::YES) {
                 enableCpuPinning = true;
                 changedCpuPinning = true;
-            } else if (val == PluginConfigParams::NO) {
+            } else if (val == InferenceEngine::PluginConfigParams::NO) {
                 enableCpuPinning = false;
                 changedCpuPinning = true;
             } else {
@@ -115,71 +117,76 @@ void Config::readProperties(const std::map<std::string, std::string> &prop, cons
                            << ov::hint::SchedulingCoreType::ECORE_ONLY << std::endl;
             }
         } else if (key == ov::hint::enable_hyper_threading.name()) {
-            if (val == PluginConfigParams::YES) {
+            if (val == InferenceEngine::PluginConfigParams::YES) {
                 enableHyperThreading = true;
                 changedHyperThreading = true;
-            } else if (val == PluginConfigParams::NO) {
+            } else if (val == InferenceEngine::PluginConfigParams::NO) {
                 enableHyperThreading = false;
                 changedHyperThreading = true;
             } else {
                 IE_THROW() << "Wrong value " << val << "for property key " << ov::hint::enable_hyper_threading.name()
                            << ". Expected only true/false." << std::endl;
             }
-        } else if (key == CPUConfigParams::KEY_CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE) {
+        } else if (key == ov::intel_cpu::sparse_weights_decompression_rate.name()) {
             float val_f = 0.0f;
             try {
                 val_f = std::stof(val);
             } catch (const std::exception&) {
-                IE_THROW() << "Wrong value for property key " << CPUConfigParams::KEY_CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE
-                                    << ". Expected only float numbers";
+                IE_THROW() << "Wrong value for property key " << ov::intel_cpu::sparse_weights_decompression_rate.name()
+                           << ". Expected only float numbers";
             }
             if (val_f < 0.f || val_f > 1.f) {
-                IE_THROW() << "Wrong value for property key " << CPUConfigParams::KEY_CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE
-                                    << ". Sparse rate must be in range [0.0f,1.0f]";
+                IE_THROW() << "Wrong value for property key " << ov::intel_cpu::sparse_weights_decompression_rate.name()
+                           << ". Sparse rate must be in range [0.0f,1.0f]";
             } else {
                 fcSparseWeiDecompressionRate = val_f;
             }
-        } else if (key == PluginConfigParams::KEY_PERF_COUNT) {
-            if (val == PluginConfigParams::YES) collectPerfCounters = true;
-            else if (val == PluginConfigParams::NO) collectPerfCounters = false;
+        } else if (key == ov::enable_profiling.name()) {
+            if (val == InferenceEngine::PluginConfigParams::YES)
+                collectPerfCounters = true;
+            else if (val == InferenceEngine::PluginConfigParams::NO)
+                collectPerfCounters = false;
             else
-                IE_THROW() << "Wrong value for property key " << PluginConfigParams::KEY_PERF_COUNT
-                                   << ". Expected only YES/NO";
-        } else if (key == PluginConfigParams::KEY_EXCLUSIVE_ASYNC_REQUESTS) {
-            if (val == PluginConfigParams::YES) exclusiveAsyncRequests = true;
-            else if (val == PluginConfigParams::NO) exclusiveAsyncRequests = false;
+                IE_THROW() << "Wrong value for property key " << ov::enable_profiling.name()
+                           << ". Expected only YES/NO";
+        } else if (key == ov::exclusive_async_requests.name()) {
+            if (val == InferenceEngine::PluginConfigParams::YES)
+                exclusiveAsyncRequests = true;
+            else if (val == InferenceEngine::PluginConfigParams::NO)
+                exclusiveAsyncRequests = false;
             else
-                IE_THROW() << "Wrong value for property key " << PluginConfigParams::KEY_EXCLUSIVE_ASYNC_REQUESTS
+                IE_THROW() << "Wrong value for property key " << ov::exclusive_async_requests.name()
                                    << ". Expected only YES/NO";
             IE_SUPPRESS_DEPRECATED_START
-        } else if (key.compare(PluginConfigParams::KEY_DUMP_EXEC_GRAPH_AS_DOT) == 0) {
+        } else if (key.compare(InferenceEngine::PluginConfigParams::KEY_DUMP_EXEC_GRAPH_AS_DOT) == 0) {
             IE_SUPPRESS_DEPRECATED_END
             // empty string means that dumping is switched off
             dumpToDot = val;
-        } else if (key.compare(PluginConfigInternalParams::KEY_LP_TRANSFORMS_MODE) == 0) {
-            if (val == PluginConfigParams::NO)
+        } else if (key.compare(InferenceEngine::PluginConfigInternalParams::KEY_LP_TRANSFORMS_MODE) == 0) {
+            if (val == InferenceEngine::PluginConfigParams::NO)
                 lpTransformsMode = LPTransformsMode::Off;
-            else if (val == PluginConfigParams::YES)
+            else if (val == InferenceEngine::PluginConfigParams::YES)
                 lpTransformsMode = LPTransformsMode::On;
             else
-                IE_THROW() << "Wrong value for property key " << PluginConfigInternalParams::KEY_LP_TRANSFORMS_MODE;
+                IE_THROW() << "Wrong value for property key "
+                           << InferenceEngine::PluginConfigInternalParams::KEY_LP_TRANSFORMS_MODE;
         } else if (key == ov::device::id.name()) {
             device_id = val;
             if (!device_id.empty()) {
                 IE_THROW() << "CPU plugin supports only '' as device id";
             }
-        } else if (key == PluginConfigParams::KEY_ENFORCE_BF16) {
-            if (val == PluginConfigParams::YES) {
+        } else if (key == InferenceEngine::PluginConfigParams::KEY_ENFORCE_BF16) {
+            if (val == InferenceEngine::PluginConfigParams::YES) {
                 if (mayiuse(avx512_core)) {
                     inferencePrecision = ov::element::bf16;
                 } else {
                     IE_THROW() << "Platform doesn't support BF16 format";
                 }
-            } else if (val == PluginConfigParams::NO) {
+            } else if (val == InferenceEngine::PluginConfigParams::NO) {
                 inferencePrecision = ov::element::f32;
             } else {
-                IE_THROW() << "Wrong value for property key " << PluginConfigParams::KEY_ENFORCE_BF16
-                    << ". Expected only YES/NO";
+                IE_THROW() << "Wrong value for property key " << InferenceEngine::PluginConfigParams::KEY_ENFORCE_BF16
+                           << ". Expected only YES/NO";
             }
             inferencePrecisionSetExplicitly = true;
         } else if (key == ov::hint::inference_precision.name()) {
@@ -204,39 +211,41 @@ void Config::readProperties(const std::map<std::string, std::string> &prop, cons
                 inferencePrecisionSetExplicitly = true;
             } else {
                 IE_THROW() << "Wrong value for property key " << ov::hint::inference_precision.name()
-                    << ". Supported values: bf16, f16, f32";
+                           << ". Supported values: bf16, f32";
             }
-        } else if (PluginConfigInternalParams::KEY_CPU_RUNTIME_CACHE_CAPACITY == key) {
+        } else if (InferenceEngine::PluginConfigInternalParams::KEY_CPU_RUNTIME_CACHE_CAPACITY == key) {
             int val_i = -1;
             try {
                 val_i = std::stoi(val);
             } catch (const std::exception&) {
-                IE_THROW() << "Wrong value for property key " << PluginConfigInternalParams::KEY_CPU_RUNTIME_CACHE_CAPACITY
+                IE_THROW() << "Wrong value for property key "
+                           << InferenceEngine::PluginConfigInternalParams::KEY_CPU_RUNTIME_CACHE_CAPACITY
                            << ". Expected only integer numbers";
             }
             // any negative value will be treated
             // as zero that means disabling the cache
             rtCacheCapacity = std::max(val_i, 0);
-        } else if (CPUConfigParams::KEY_CPU_DENORMALS_OPTIMIZATION == key) {
-            if (val == PluginConfigParams::YES) {
+        } else if (ov::intel_cpu::denormals_optimization.name() == key) {
+            if (val == InferenceEngine::PluginConfigParams::YES) {
                 denormalsOptMode = DenormalsOptMode::DO_On;
-            } else if (val == PluginConfigParams::NO) {
+            } else if (val == InferenceEngine::PluginConfigParams::NO) {
                 denormalsOptMode = DenormalsOptMode::DO_Off;
             } else {
                 denormalsOptMode = DenormalsOptMode::DO_Keep;
-                IE_THROW() << "Wrong value for property key " << CPUConfigParams::KEY_CPU_DENORMALS_OPTIMIZATION
-                << ". Expected only YES/NO";
+                IE_THROW() << "Wrong value for property key " << ov::intel_cpu::denormals_optimization.name()
+                           << ". Expected only YES/NO";
             }
-        } else if (key == PluginConfigInternalParams::KEY_SNIPPETS_MODE) {
-            if (val == PluginConfigInternalParams::ENABLE)
+        } else if (key == InferenceEngine::PluginConfigInternalParams::KEY_SNIPPETS_MODE) {
+            if (val == InferenceEngine::PluginConfigInternalParams::ENABLE)
                 snippetsMode = SnippetsMode::Enable;
-            else if (val == PluginConfigInternalParams::IGNORE_CALLBACK)
+            else if (val == InferenceEngine::PluginConfigInternalParams::IGNORE_CALLBACK)
                 snippetsMode = SnippetsMode::IgnoreCallback;
-            else if (val == PluginConfigInternalParams::DISABLE)
+            else if (val == InferenceEngine::PluginConfigInternalParams::DISABLE)
                 snippetsMode = SnippetsMode::Disable;
             else
-                IE_THROW() << "Wrong value for property key " << PluginConfigInternalParams::KEY_SNIPPETS_MODE
-                            << ". Expected values: ENABLE/DISABLE/IGNORE_CALLBACK";
+                IE_THROW() << "Wrong value for property key "
+                           << InferenceEngine::PluginConfigInternalParams::KEY_SNIPPETS_MODE
+                           << ". Expected values: ENABLE/DISABLE/IGNORE_CALLBACK";
         } else if (key == ov::hint::execution_mode.name()) {
             if (val == "PERFORMANCE") {
                 executionMode = ov::hint::ExecutionMode::PERFORMANCE;
@@ -244,7 +253,7 @@ void Config::readProperties(const std::map<std::string, std::string> &prop, cons
                 executionMode = ov::hint::ExecutionMode::ACCURACY;
             } else {
                 IE_THROW() << "Wrong value for property key " << ov::hint::execution_mode.name()
-                    << ". Supported values: PERFORMANCE, ACCURACY";
+                           << ". Supported values: PERFORMANCE, ACCURACY";
             }
         } else {
             IE_THROW(NotFound) << "Unsupported property " << key << " by CPU plugin";
@@ -273,7 +282,7 @@ void Config::readProperties(const std::map<std::string, std::string> &prop, cons
     if (!prop.empty())
         _config.clear();
 
-    if (exclusiveAsyncRequests) { // Exclusive request feature disables the streams
+    if (exclusiveAsyncRequests) {  // Exclusive request feature disables the streams
         streamExecutorConfig._streams = 1;
         streamExecutorConfig._streams_changed = true;
     }
@@ -293,6 +302,7 @@ void Config::updateProperties() {
     if (!_config.empty())
         return;
 
+    using namespace InferenceEngine;
     switch (streamExecutorConfig._threadBindingType) {
     case IStreamsExecutor::ThreadBindingType::NONE:
         _config.insert({ PluginConfigParams::KEY_CPU_BIND_THREAD, PluginConfigParams::NO });

src/plugins/intel_cpu/src/config.h

@@ -4,9 +4,8 @@
 
 #pragma once
 
-#include <threading/ie_istreams_executor.hpp>
+#include <openvino/runtime/threading/istreams_executor.hpp>
 #include <ie_performance_hints.hpp>
-#include <ie/ie_common.h>
 #include <openvino/runtime/properties.hpp>
 #include <openvino/util/common_util.hpp>
 #include "utils/debug_caps_config.h"
@@ -62,7 +61,7 @@ struct Config {
     // TODO: Executor cache may leads to incorrect behavior on oneDNN ACL primitives
     size_t rtCacheCapacity = 0ul;
 #endif
-    InferenceEngine::IStreamsExecutor::Config streamExecutorConfig;
+    ov::threading::IStreamsExecutor::Config streamExecutorConfig;
     InferenceEngine::PerfHintsConfig  perfHintsConfig;
     bool enableCpuPinning = true;
     bool changedCpuPinning = false;
@@ -88,7 +87,8 @@ struct Config {
     // is reserved.
     bool DAZOn = false;
 
-    void readProperties(const std::map<std::string, std::string> &config, const ModelType modelType = ModelType::Unknown);
+    void readProperties(const ov::AnyMap& config, const ModelType modelType = ModelType::Unknown);
+
     void updateProperties();
 
     std::map<std::string, std::string> _config;

src/plugins/intel_cpu/src/cpu_memory.cpp

@@ -23,6 +23,16 @@ using namespace dnnl;
 
 namespace ov {
 namespace intel_cpu {
+template <>
+DnnlMemoryDescPtr IMemory::getDescWithType<DnnlMemoryDesc, 0, 0>() const {
+    return MemoryDescUtils::convertToDnnlMemoryDesc(getDescPtr());
+}
+
+template <>
+BlockedMemoryDescPtr IMemory::getDescWithType<BlockedMemoryDesc, 0, 0>() const {
+    return MemoryDescUtils::convertToBlockedMemoryDesc(getDescPtr());
+}
+
 namespace {
     inline void setSubnormalsToZero(float *data, size_t size) {
         uint32_t *u32data = reinterpret_cast<uint32_t *>(data);
@@ -36,21 +46,27 @@ namespace {
     void transferData(const IMemory& src, const IMemory& dst, bool ftz) {
         node::Reorder::reorderData(src, dst);
 
-        auto localPrim = dst.getPrimitive();
-        auto desc = localPrim.get_desc();
-        dnnl::impl::memory_desc_wrapper wrapper(desc.get());
-
-        if (ftz
-            && src.getDataType() == memory::data_type::f32
-            && !wrapper.is_wino_desc()
-            // WA: to avoid zero filling auxiliary information
-            && !wrapper.is_rnn_packed_desc()
-            && dst.getDataType() != memory::data_type::bf16) {
-            // Internal blobs don't have strides yet.
-            auto *memData = static_cast<float *>(dst.getData());
-            memData += wrapper.offset0();
-            setSubnormalsToZero(memData, dst.getSize() / sizeof(float));
+        if (!ftz) {
+            return;
         }
+        if (src.getDesc().getPrecision() != Precision::FP32 || dst.getDesc().getPrecision() == Precision::BF16) {
+            return;
+        }
+        size_t offset = 0;
+        if (dst.getDesc().getType() & MemoryDescType::Dnnl) {
+            // here we can safely cast to DnnlMemoryDesc
+            auto dnnl_desc = dst.getDescWithType<DnnlMemoryDesc>();
+            auto desc = dnnl_desc->getDnnlDesc();
+            dnnl::impl::memory_desc_wrapper wrapper(desc.get());
+            offset = wrapper.offset0();
+            if (wrapper.is_wino_desc() || wrapper.is_rnn_packed_desc()) {
+                return;
+            }
+        }
+        // actual FTZ
+        auto* memData = static_cast<float*>(dst.getData());
+        memData += offset;
+        setSubnormalsToZero(memData, dst.getSize() / sizeof(float));
     }
 
 }   // namespace
@@ -122,16 +138,6 @@ void Memory::redefineDesc(MemoryDescPtr desc) {
     this->create(desc, nullptr, false);
 }
 
-template<>
-DnnlMemoryDescPtr IMemory::getDescWithType<DnnlMemoryDesc, 0, 0>() const {
-    return MemoryDescUtils::convertToDnnlMemoryDesc(getDescPtr());
-}
-
-template<>
-BlockedMemoryDescPtr IMemory::getDescWithType<BlockedMemoryDesc, 0, 0>() const {
-    return MemoryDescUtils::convertToBlockedMemoryDesc(getDescPtr());
-}
-
 void Memory::update() {
     if (dnnlMemHandle.isInit()) {
         auto prim = dnnlMemHandle.getPrim();

src/plugins/intel_cpu/src/cpu_streams_calculation.cpp

@@ -18,7 +18,7 @@
 #include "performance_heuristics.hpp"
 
 using namespace ov;
-using namespace threading;
+using namespace ov::threading;
 
 #define INIT_VAL -100
 
@@ -412,7 +412,7 @@ std::vector<std::vector<int>> get_streams_info_table(const int input_streams,
 
 int get_model_prefer_threads(const int num_streams,
                              const std::vector<std::vector<int>> proc_type_table,
-                             const std::shared_ptr<ngraph::Function>& ngraphFunc,
+                             const std::shared_ptr<ov::Model>& model,
                              Config& config) {
     const int sockets = get_default_latency_streams(config.latencyThreadingMode);
     auto model_prefer = 0;
@@ -441,7 +441,7 @@ int get_model_prefer_threads(const int num_streams,
         const float memThresholdAssumeLimitedForISA = ov::MemBandwidthPressure::LIMITED / isaSpecificThreshold;
         const float L2_cache_size = dnnl::utils::get_cache_size(2 /*level*/, true /*per core */);
         ov::MemBandwidthPressure networkToleranceForLowCache =
-            ov::MemBandwidthPressureTolerance(ngraphFunc, L2_cache_size, memThresholdAssumeLimitedForISA);
+            ov::MemBandwidthPressureTolerance(model, L2_cache_size, memThresholdAssumeLimitedForISA);
         config.modelPreferThreads = ov::threading::IStreamsExecutor::Config::StreamMode::DEFAULT;
         if (networkToleranceForLowCache.max_mem_tolerance == ov::MemBandwidthPressure::UNKNOWN) {
             if ((networkToleranceForLowCache.ratio_compute_convs == ov::MemBandwidthPressure::ALL) ||
@@ -469,7 +469,7 @@ int get_model_prefer_threads(const int num_streams,
                 model_prefer = proc_type_table[0][ALL_PROC];
             }
 #else
-            bool fp_intesive = !ov::op::util::has_op_with_type<ngraph::op::FakeQuantize>(ngraphFunc);
+            bool fp_intesive = !ov::op::util::has_op_with_type<ov::op::v0::FakeQuantize>(model);
             const int int8_threshold = 4;  // ~relative efficiency of the VNNI-intensive code for Big vs Little cores;
             const int fp32_threshold = 2;  // ~relative efficiency of the AVX2 fp32 code for Big vs Little cores;
             // by default the latency case uses (faster) Big cores only, depending on the compute ratio
@@ -487,14 +487,14 @@ int get_model_prefer_threads(const int num_streams,
 }
 
 std::vector<std::vector<int>> generate_stream_info(const int streams,
-                                                   const std::shared_ptr<ngraph::Function>& ngraphFunc,
+                                                   const std::shared_ptr<ov::Model>& model,
                                                    Config& config,
                                                    std::vector<std::vector<int>>& proc_type_table,
                                                    int preferred_nthreads_per_stream) {
     int model_prefer_threads = preferred_nthreads_per_stream;
-    InferenceEngine::IStreamsExecutor::Config& executor_config = config.streamExecutorConfig;
-
+    IStreamsExecutor::Config& executor_config = config.streamExecutorConfig;
     proc_type_table = apply_scheduling_core_type(config.schedulingCoreType, proc_type_table);
+
     proc_type_table = apply_hyper_threading(config.enableHyperThreading,
                                             config.changedHyperThreading,
                                             config.perfHintsConfig.ovPerfHint,
@@ -505,7 +505,7 @@ std::vector<std::vector<int>> generate_stream_info(const int streams,
                                                        config.latencyThreadingMode,
                                                        proc_type_table);
     if (-1 == preferred_nthreads_per_stream) {
-        model_prefer_threads = get_model_prefer_threads(streams, proc_type_table, ngraphFunc, config);
+        model_prefer_threads = get_model_prefer_threads(streams, proc_type_table, model, config);
     }
 
     executor_config._streams_info_table = get_streams_info_table(executor_config._streams,
@@ -519,13 +519,13 @@ std::vector<std::vector<int>> generate_stream_info(const int streams,
     return proc_type_table;
 }
 
-void get_num_streams(const int streams, const std::shared_ptr<ngraph::Function>& ngraphFunc, Config& config) {
-    InferenceEngine::IStreamsExecutor::Config& executor_config = config.streamExecutorConfig;
+void get_num_streams(const int streams, const std::shared_ptr<ov::Model>& model, Config& config) {
+    IStreamsExecutor::Config& executor_config = config.streamExecutorConfig;
     std::vector<std::vector<int>> proc_type_table = get_proc_type_table();
 
-    generate_stream_info(streams, ngraphFunc, config, proc_type_table);
+    generate_stream_info(streams, model, config, proc_type_table);
 
-    executor_config = InferenceEngine::IStreamsExecutor::Config::reserve_cpu_threads(executor_config);
+    executor_config = IStreamsExecutor::Config::reserve_cpu_threads(executor_config);
     executor_config._threadsPerStream = executor_config._streams_info_table[0][THREADS_PER_STREAM];
 }
 

src/plugins/intel_cpu/src/cpu_streams_calculation.hpp

@@ -58,19 +58,19 @@ std::vector<std::vector<int>> get_streams_info_table(const int input_streams,
  *               - LATENCY hint equals 1 stream.
  * @param[in]  proc_type_table candidate processors available at this time
  *               - candidate processors have benn updated based on properties like "Ecore only" in previous function
- * @param[in]  ngraphFunc ngraph function
+ * @param[in]  model model
  * @param[in]  config intel cpu configuration
  * @return     model_prefer_threads "0" means generating the optimal threads per stream based on platform
  */
 int get_model_prefer_threads(const int num_streams,
                              const std::vector<std::vector<int>> proc_type_table,
-                             const std::shared_ptr<ngraph::Function>& ngraphFunc,
+                             const std::shared_ptr<ov::Model>& model,
                              Config& config);
 
 /**
  * @brief      Generate streams information according to processors type table
  * @param[in]  streams number of streams
- * @param[in]  ngraphFunc graph handle
+ * @param[in]  model graph handle
  * @param[in]  config intel cpu configuration
  * @param[in]  proc_type_table candidate processors available at current platform
  * @param[in]  preferred_nthreads_per_stream is initial preferred number of threads per stream
@@ -78,7 +78,7 @@ int get_model_prefer_threads(const int num_streams,
  * ov::hint::enable_hyper_threading
  */
 std::vector<std::vector<int>> generate_stream_info(const int streams,
-                                                   const std::shared_ptr<ngraph::Function>& ngraphFunc,
+                                                   const std::shared_ptr<ov::Model>& model,
                                                    Config& config,
                                                    std::vector<std::vector<int>>& proc_type_table,
                                                    int preferred_nthreads_per_stream = -1);
@@ -96,11 +96,11 @@ struct StreamCfg {
 /**
  * @brief      Get information about number of streams, threads and pinning threads on different processors
  * @param[in]  streams number of streams
- * @param[in]  ngraphFunc graph handle
+ * @param[in]  model graph handle
  * @param[in]  config intel cpu configuration
  */
 void get_num_streams(const int streams,
-                     const std::shared_ptr<ngraph::Function>& ngraphFunc,
+                     const std::shared_ptr<ov::Model>& model,
                      Config& config);
 
 /**

src/plugins/intel_cpu/src/dnnl_extension_utils.cpp

@@ -20,6 +20,8 @@ namespace intel_cpu {
 
 uint8_t DnnlExtensionUtils::sizeOfDataType(dnnl::memory::data_type dataType) {
     switch (dataType) {
+    case dnnl::memory::data_type::f64:
+        return 8;
     case dnnl::memory::data_type::f32:
     case dnnl::memory::data_type::s32:
         return 4;
@@ -36,7 +38,7 @@ uint8_t DnnlExtensionUtils::sizeOfDataType(dnnl::memory::data_type dataType) {
     case dnnl::memory::data_type::undef:
         return 0;
     default:
-        IE_THROW() << "Unsupported data type.";
+        OPENVINO_THROW("Unsupported data type.");
     }
 }
 
@@ -66,7 +68,7 @@ memory::data_type DnnlExtensionUtils::IEPrecisionToDataType(const InferenceEngin
         case InferenceEngine::Precision::UNSPECIFIED:
             return memory::data_type::undef;
         default: {
-            IE_THROW() << "The plugin does not support " << prec.name();
+            OPENVINO_THROW("The plugin does not support ", prec.name());
         }
     }
 }
@@ -87,6 +89,8 @@ InferenceEngine::Precision DnnlExtensionUtils::DataTypeToIEPrecision(memory::dat
             return InferenceEngine::Precision::BIN;
         case memory::data_type::f16:
             return InferenceEngine::Precision::FP16;
+        case memory::data_type::f64:
+            return InferenceEngine::Precision::FP64;
         case memory::data_type::nf4:
             return InferenceEngine::Precision::NF4;
         case memory::data_type::s4:

src/plugins/intel_cpu/src/exec_network.h

@@ -1,86 +0,0 @@
-// Copyright (C) 2018-2023 Intel Corporation
-// SPDX-License-Identifier: Apache-2.0
-//
-
-#pragma once
-
-#include <cpp_interfaces/impl/ie_executable_network_thread_safe_default.hpp>
-#include <cpp_interfaces/impl/ie_executable_network_thread_safe_default.hpp>
-
-#include "graph.h"
-#include "extension_mngr.h"
-#include "graph_context.h"
-#include <threading/ie_thread_local.hpp>
-
-#include <vector>
-#include <memory>
-#include <map>
-#include <string>
-#include <unordered_map>
-
-namespace ov {
-namespace intel_cpu {
-
-class ExecNetwork: public InferenceEngine::ExecutableNetworkThreadSafeDefault {
-public:
-    typedef std::shared_ptr<ExecNetwork> Ptr;
-
-    std::shared_ptr<InferenceEngine::IInferRequestInternal>
-    CreateInferRequestImpl(const std::vector<std::shared_ptr<const ov::Node>>& inputs,
-                           const std::vector<std::shared_ptr<const ov::Node>>& outputs) override;
-
-    std::shared_ptr<InferenceEngine::IInferRequestInternal>
-    CreateInferRequestImpl(InferenceEngine::InputsDataMap networkInputs,
-                           InferenceEngine::OutputsDataMap networkOutputs) override;
-
-    InferenceEngine::IInferRequestInternal::Ptr CreateInferRequest() override;
-
-    ExecNetwork(const InferenceEngine::CNNNetwork &network, const Config &cfg,
-                const ExtensionManager::Ptr &extMgr,
-                const std::shared_ptr<InferenceEngine::IInferencePlugin>& plugin);
-
-    InferenceEngine::Parameter GetConfig(const std::string &name) const override;
-
-    InferenceEngine::Parameter GetMetric(const std::string &name) const override;
-
-    std::shared_ptr<ngraph::Function> GetExecGraphInfo() override;
-
-    void Export(std::ostream& modelStream) override;
-
-protected:
-    friend class InferRequestBase;
-    ExtensionManager::Ptr extensionManager;
-    std::vector<InferenceEngine::IVariableStateInternal::Ptr> memoryStates;
-    const InferenceEngine::CNNNetwork           _network;
-    // Generic synchronization primitive on ExecNetwork level.
-    // Usage example: helps to avoid data races during CPU Graph initialization in multi-streams scenario
-    mutable std::shared_ptr<std::mutex>         _mutex;
-    Config                                      _cfg;
-    std::atomic_int                             _numRequests = {0};
-    std::string                                 _name;
-    struct GraphGuard : public Graph {
-        std::mutex  _mutex;
-        struct Lock : public std::unique_lock<std::mutex> {
-            explicit Lock(GraphGuard& graph) : std::unique_lock<std::mutex>(graph._mutex), _graph(graph) {}
-            GraphGuard& _graph;
-        };
-    };
-
-    // WARNING: Do not use _graphs directly.
-    mutable std::deque<GraphGuard>              _graphs;
-    mutable SocketsWeights                      _socketWeights;
-
-    /* WARNING: Use GetGraph() function to get access to graph in current stream.
-     * NOTE: Main thread is interpreted as master thread of external stream so use this function to get access to graphs
-     *       even from main thread
-     */
-    GraphGuard::Lock GetGraph() const;
-
-    InferenceEngine::Parameter GetConfigLegacy(const std::string &name) const;
-
-    InferenceEngine::Parameter GetMetricLegacy(const std::string &name, const GraphGuard& graph) const;
-};
-
-}   // namespace intel_cpu
-}   // namespace ov
-

src/plugins/intel_cpu/src/graph.cpp

@@ -2,59 +2,55 @@
 // SPDX-License-Identifier: Apache-2.0
 //
 
-#include <algorithm>
-#include <string>
-#include <map>
-#include <vector>
-#include <tuple>
-#include <unordered_set>
-#include <limits>
-#include <fstream>
-#include <unordered_map>
-#include <memory>
-#include <utility>
-
 #include "graph.h"
-#include "graph_dumper.h"
-#include "graph_optimizer.h"
+
+#include <algorithm>
+#include <fstream>
+#include <limits>
+#include <map>
+#include <memory>
+#include <string>
+#include <tuple>
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+#include "common/primitive_desc.hpp"
+#include "common/primitive_desc_iface.hpp"
 #include "dnnl_extension_utils.h"
 #include "extension_mngr.h"
-#include "ie_ngraph_utils.hpp"
-#include "memory_solver.hpp"
-#include "itt.h"
+#include "graph_dumper.h"
+#include "graph_optimizer.h"
+#include "ie_algorithm.hpp"
 #include "infer_request.h"
-#include "nodes/input.h"
-#include <nodes/reorder.h>
-#include "nodes/convert.h"
-#include "nodes/subgraph.h"
-#include "nodes/fullyconnected.h"
-
-#include <ie_algorithm.hpp>
-#include <blob_factory.hpp>
-#include "nodes/common/cpu_memcpy.h"
+#include "itt.h"
+#include "low_precision/low_precision.hpp"
+#include "memory_desc/cpu_memory_desc_utils.h"
+#include "memory_desc/dnnl_blocked_memory_desc.h"
+#include "memory_solver.hpp"
 #include "nodes/common/cpu_convert.h"
-
+#include "nodes/common/cpu_memcpy.h"
+#include "nodes/convert.h"
+#include "nodes/fullyconnected.h"
+#include "nodes/input.h"
+#include "nodes/reorder.h"
+#include "nodes/subgraph.h"
+#include "openvino/core/model.hpp"
+#include "openvino/core/node.hpp"
+#include "openvino/op/ops.hpp"
 #include "precision_utils.h"
-#include <ie_plugin_config.hpp>
-
-#include "utils/general_utils.h"
-#include "utils/debug_capabilities.h"
-#include "utils/node_dumper.h"
-#include "utils/ngraph_utils.hpp"
+#include "transformations/utils/utils.hpp"
 #include "utils/cpu_utils.hpp"
+#include "utils/debug_capabilities.h"
+#include "utils/general_utils.h"
+#include "utils/ngraph_utils.hpp"
+#include "utils/node_dumper.h"
 #include "utils/verbose.h"
 #include "memory_desc/cpu_memory_desc_utils.h"
 
-#include <openvino/core/model.hpp>
-#include <openvino/core/node.hpp>
-#include <openvino/op/ops.hpp>
-#include <transformations/utils/utils.hpp>
-#include <low_precision/low_precision.hpp>
-#include "memory_desc/dnnl_blocked_memory_desc.h"
-#include <common/primitive_desc.hpp>
-#include <common/primitive_desc_iface.hpp>
 #if (OV_THREAD == OV_THREAD_TBB || OV_THREAD == OV_THREAD_TBB_AUTO)
-#   include <tbb/task.h>
+#    include <tbb/task.h>
 #endif
 
 using namespace dnnl;
@@ -87,10 +83,10 @@ void Graph::CreateGraph(NET &net, const GraphContext::CPtr ctx) {
     CPU_DEBUG_CAP_ENABLE(serialize(*this));
 }
 
-void Graph::CreateGraph(const std::vector<NodePtr> &graphNodes,
-                              const std::vector<EdgePtr> &graphEdges,
-                              const GraphContext::CPtr ctx,
-                              std::string name) {
+void Graph::CreateGraph(const std::vector<NodePtr>& graphNodes,
+                        const std::vector<EdgePtr>& graphEdges,
+                        const GraphContext::CPtr ctx,
+                        std::string name) {
     if (IsReady())
         ForgetGraphData();
 
@@ -116,10 +112,9 @@ void Graph::CreateGraph(const std::vector<NodePtr> &graphNodes,
 }
 
 template void Graph::CreateGraph(const std::shared_ptr<const ov::Model>&, const GraphContext::CPtr);
-template void Graph::CreateGraph(const CNNNetwork&, const GraphContext::CPtr);
-
-void Graph::Replicate(const std::shared_ptr<const ov::Model> &subgraph) {
-    this->_name = "subgraph";
+void Graph::Replicate(const std::shared_ptr<const ov::Model> &model) {
+    OV_ITT_SCOPE_CHAIN(FIRST_INFERENCE, taskChain, itt::domains::intel_cpu_LT, "Graph::Replicate", "ov::Model");
+    this->_name = model->get_friendly_name();
     this->reuse_io_tensors = false;
 
     // Map data object onto producer node
@@ -129,7 +124,8 @@ void Graph::Replicate(const std::shared_ptr<const ov::Model> &subgraph) {
     // Will be stored as fake output separately.
     std::deque<ov::Output<ov::Node>> unusedOutputs;
 
-    auto getParentOutputPort = [](const std::shared_ptr<ov::Node> childOp, const std::shared_ptr<ov::Node> parentOp,
+    auto getParentOutputPort = [](const std::shared_ptr<ov::Node> childOp,
+                                  const std::shared_ptr<ov::Node> parentOp,
                                   const size_t childInputPort) -> int {
         for (size_t parentPort = 0; parentPort < parentOp->get_output_size(); parentPort++) {
             if (childOp->input(childInputPort).get_tensor_ptr() == parentOp->output(parentPort).get_tensor_ptr()) {
@@ -140,19 +136,21 @@ void Graph::Replicate(const std::shared_ptr<const ov::Model> &subgraph) {
         return -1;
     };
 
-    for (const auto& op : subgraph->get_ordered_ops()) {
+    const bool is_legacy_api = getConfig().isLegacyApi;
+    for (const auto& op : model->get_ordered_ops()) {
         const NodePtr node {Node::factory().create(op, context)};
 
         graphNodes.push_back(node);
-
         if (op->get_type_info() == op::v0::Parameter::get_type_info_static()) {
-            inputNodesMap[node->getName()] = node;
+            const std::string name = get_port_name(ov::Output<ov::Node>(op, 0), is_legacy_api);
+            inputNodesMap[name] = node;
+            if (node->isDynamicNode()) {
+                graphHasDynamicInput = true;
+            }
         }
 
         if (op->get_type_info() == op::v0::Result::get_type_info_static()) {
-            const auto prev = op->input_value(0);
-            const std::string inputID = op::util::get_ie_output_name(prev);
-
+            const std::string inputID = get_port_name(op->output(0), is_legacy_api);
             outputNodesMap[inputID] = node;
         }
 
@@ -193,124 +191,16 @@ void Graph::Replicate(const std::shared_ptr<const ov::Model> &subgraph) {
         graphNodes.push_back(outNode);
     }
 
-    EnforceInferencePrecision();
-}
-
-void Graph::Replicate(const CNNNetwork &network) {
-    OV_ITT_SCOPE_CHAIN(FIRST_INFERENCE, taskChain, itt::domains::intel_cpu_LT, "Graph::Replicate", "CNNNetwork");
-
-    const InputsDataMap& inputsInfo = network.getInputsInfo();
-    const OutputsDataMap& outputsInfo = network.getOutputsInfo();
-
-    this->_name = network.getName();
-
-    std::shared_ptr<const ov::Model> func = network.getFunction();
-
-    if (!func) {
-        IE_THROW() << "Function pointer inside CNNNetwork is nullptr";
-    }
-
-    auto orderedOps = func->get_ordered_ops();
-
-    // TODO [NM]: unordered_map is preferred from performance perspective. Needs hash for ov::Node
-    std::map<std::shared_ptr<ov::Node>, NodePtr> op2node;
-    std::deque<ov::Output<ov::Node>> unusedOutputs;  // nodes which has no consumers (output or just unused)
-
-    auto getParentOutputPort = [](const std::shared_ptr<ov::Node> childOp, const std::shared_ptr<ov::Node> parentOp,
-                                  const size_t childInputPort) -> int {
-        for (size_t parentPort = 0; parentPort < parentOp->get_output_size(); parentPort++) {
-            if (childOp->input(childInputPort).get_tensor_ptr() == parentOp->output(parentPort).get_tensor_ptr()) {
-                return static_cast<int>(parentPort);
-            }
-        }
-
-        return -1;
+    auto hasSubgraphConsumers = [](const NodePtr& node) -> bool {
+        const auto& childEdges = node->getChildEdges();
+        return std::any_of(childEdges.begin(), childEdges.end(), [](const EdgeWeakPtr& edge) -> bool {
+            auto edgePtr = edge.lock();
+            if (!edgePtr)
+                return false;
+            return edgePtr->getChild()->getType() == Type::Subgraph;
+        });
     };
 
-    OV_ITT_SCOPE_NEXT(FIRST_INFERENCE, taskChain, "AllNodes");
-
-    // Replicate All Nodes in topological order
-    for (const auto& op : orderedOps) {
-        const NodePtr node(Node::factory().create(op, context));
-
-        graphNodes.push_back(node);
-
-        if (op->get_type_info() == op::v0::Parameter::get_type_info_static()) {
-            const auto inInfo = inputsInfo.find(node->getName());
-            if (inInfo != inputsInfo.end()) {
-                inputNodesMap[node->getName()] = node;
-                if (node->isDynamicNode()) {
-                    graphHasDynamicInput = true;
-                }
-            }
-        }
-
-        if (op->get_type_info() == op::v0::Result::get_type_info_static()) {
-            const auto &input = op->input_value(0);
-            const auto name = op::util::get_ie_output_name(input);
-
-            if (outputsInfo.count(name) != 0) {
-                outputNodesMap[name] = node;
-            }
-        }
-
-        op2node[op] = node;
-
-        for (size_t port = 0; port < op->get_input_size(); port++) {
-            auto parentOp = op->get_input_node_shared_ptr(port);
-            auto parentNode = op2node[parentOp];
-
-            EdgePtr edge(new Edge(parentNode, node, getParentOutputPort(op, parentOp, port), static_cast<int>(port)));
-            node->addEdge(edge);
-            graphEdges.push_back(edge);
-        }
-
-        if (!one_of(op->get_type_info(),
-                op::v0::Result::get_type_info_static(),
-                op::v3::Assign::get_type_info_static(),
-                op::v6::Assign::get_type_info_static())) {
-            for (size_t oi = 0; oi < op->get_output_size(); oi++) {
-                if (op->get_output_target_inputs(oi).empty()) {
-                    unusedOutputs.push_back(op->output(oi));
-                }
-            }
-        }
-    }
-
-    // Add stub output node for unused outputs
-    for (auto unusedOutput : unusedOutputs) {
-        auto parentNode = op2node[unusedOutput.get_node_shared_ptr()];
-        const auto port = unusedOutput.get_index();
-        const auto nodeName = std::string("stub_") + std::to_string(unusedOutput.get_index()) + "_" + parentNode->getName();
-        const NodePtr outNode = std::make_shared<node::Input>(parentNode->outputShapes[port],
-                                                                        parentNode->getOriginalOutputPrecisionAtPort(port),
-                                                                        nodeName, "Result", context);
-        EdgePtr edge(new Edge(parentNode, outNode, port, 0));
-        outNode->addEdge(edge);
-        graphEdges.push_back(edge);
-        graphNodes.push_back(outNode);
-    }
-
-    auto hasSubgraphConsumers = [] (const NodePtr& node) -> bool {
-        const auto & childEdges = node->getChildEdges();
-        return std::any_of(childEdges.begin(), childEdges.end(),
-                           [] (const EdgeWeakPtr& edge) -> bool {
-                               auto edgePtr = edge.lock();
-                               if (!edgePtr)
-                                   return false;
-                               return edgePtr->getChild()->getType() == Type::Subgraph;
-                           });
-    };
-    // change precision for input/output nodes to avoid extra data conversion when set input/output blobs
-    for (auto &input : inputNodesMap) {
-        const auto precToSet = normalizeToSupportedPrecision(inputsInfo.at(input.first)->getPrecision());
-        input.second->setOriginalOutputPrecisionAtPort(0, precToSet);
-    }
-
-    for (auto &output : outputNodesMap) {
-        const auto precToSet = normalizeToSupportedPrecision(outputsInfo.at(output.first)->getPrecision());
-        output.second->setOriginalInputPrecisionAtPort(0, precToSet);
-    }
     // enforce must be performed after inputs and outputs info are taken into account
     EnforceInferencePrecision();
     // also we need to change input/output precisions for consumers/producers to avoid inserting reorder
@@ -337,20 +227,6 @@ void Graph::Replicate(const CNNNetwork &network) {
             parent->setOriginalOutputPrecisionAtPort(parentEdges[i]->getInputNum(), precToSet);
         }
     }
-
-    // Loading mean images
-    for (const auto& input : inputsInfo) {
-        Shape outShape;
-        if (!inputNodesMap[input.first]->outputShapes.front().getRank()) {
-            outShape =  Shape(SizeVector({1, 1}));
-        } else {
-            outShape = inputNodesMap[input.first]->outputShapes.front();
-        }
-        InputInfo::Ptr ii = input.second;
-        if (ii && ii->getPreProcess().getNumberOfChannels()) {
-            _normalizePreprocMap[input.first].Load(outShape, ii);
-        }
-    }
 }
 
 void Graph::InitGraph() {
@@ -401,12 +277,6 @@ void Graph::InitDescriptors() {
     OV_ITT_SCOPE_CHAIN(FIRST_INFERENCE, taskChain, itt::domains::intel_cpu_LT, "InitDescriptors", "Prepare");
 
     for (auto &node : graphNodes) {
-        if (node->getType() == Type::Input && _normalizePreprocMap.find(node->getName()) != _normalizePreprocMap.end()) {
-            auto *inputNode = dynamic_cast<node::Input *>(node.get());
-            if (inputNode)
-                inputNode->withMeanImage();
-        }
-
         OV_ITT_SCOPE_NEXT(FIRST_INFERENCE, taskChain, node->profiling.getSupportedDescriptors);
         DEBUG_LOG("Get supported primitive descriptors for node: ", node->getName());
         node->getSupportedDescriptors();
@@ -998,45 +868,76 @@ bool Graph::ProcessDynNodes() {
     return result;
 }
 
-void Graph::PushInputData(const std::string& name, const InferenceEngine::Blob::Ptr &in) {
-    if (!IsReady()) IE_THROW()<< "Wrong state. Topology not ready.";
+void Graph::PushInputData(const std::string& name, const ov::SoPtr<ITensor>& input) {
+    if (!IsReady()) OPENVINO_THROW("Wrong state. Topology not ready.");
+    auto input_itr = inputNodesMap.find(name);
+    if (input_itr != inputNodesMap.end()) {
+        auto create_mem_desc = [&](const ov::SoPtr<ITensor>& tensor) -> CpuBlockedMemoryDesc {
+            auto element_type = tensor->get_element_type();
+            auto shape = tensor->get_shape();
+            if (shape.empty())
+                shape = {tensor->get_size()};
+            std::vector<size_t> blk_order(shape.size());
+            std::iota(blk_order.begin(), blk_order.end(), 0);
+            std::vector<size_t> dim_offset(shape.size(), 0);
+            std::vector<size_t> blk_strides;
+            auto byte_strides = element_type.bitwidth() >= 8 ? tensor->get_strides() : Strides{};
+            if (byte_strides.empty()) {
+                blk_strides = ov::row_major_strides(shape);
+            } else {
+                // ROI tensor need figure out correct blk_strides
+                blk_strides.resize(byte_strides.size());
+                std::transform(byte_strides.begin(),
+                               byte_strides.end(),
+                               blk_strides.begin(),
+                               [&element_type](size_t byte_stride) {
+                                   OPENVINO_ASSERT(byte_stride % element_type.size() == 0,
+                                                   "Limitation: Stride in bytes ",
+                                                   byte_stride,
+                                                   " should be divisible by size of element ",
+                                                   element_type.size());
+                                   return byte_stride / element_type.size();
+                               });
+            }
+            InferenceEngine::TensorDesc tensorDesc(
+                InferenceEngine::details::convertPrecision(tensor->get_element_type()),
+                shape,
+                InferenceEngine::BlockingDesc{shape, blk_order, 0, dim_offset, blk_strides});
+            return MemoryDescUtils::convertToCpuBlockedMemoryDesc(tensorDesc);
+        };
 
-    auto input = inputNodesMap.find(name);
-    if (input != inputNodesMap.end()) {
-        auto& inTensorDesc = in->getTensorDesc();
-        auto node = input->second;
+        auto node = input_itr->second;
         auto childEdge = node->getChildEdgeAt(0);
         const auto& outDims = node->getOutputShapeAtPort(0);
 
-        const void *ext_data_ptr = in->cbuffer();
-        void *inter_data_ptr = childEdge->getMemory().getData();
+        const void* ext_data_ptr = input->data();
+        void* inter_data_ptr = childEdge->getMemory().getData();
 
         if (ext_data_ptr != inter_data_ptr) {
-            auto ext_tdesc = MemoryDescUtils::convertToDnnlBlockedMemoryDesc(in->getTensorDesc());
-
-            Memory ext_mem(getEngine(), ext_tdesc, ext_data_ptr, false);
-
+            auto ext_tensor_desc = create_mem_desc(input);
+            Memory ext_mem(getEngine(), ext_tensor_desc, ext_data_ptr, false);
             childEdge->getMemory().load(ext_mem, false);
         }
 
         // todo: make sure 'name' exists in this map...
         if (_normalizePreprocMap.find(name) != _normalizePreprocMap.end()) {
-            if (inTensorDesc.getPrecision() == InferenceEngine::Precision::FP32) {
-                _normalizePreprocMap[name].NormalizeImage(outDims, reinterpret_cast<float *>(inter_data_ptr),
-                                                          inTensorDesc.getLayout());
+            if (input->get_element_type() == ov::element::f32) {
+                _normalizePreprocMap[name].NormalizeImage(outDims,
+                                                          reinterpret_cast<float*>(inter_data_ptr),
+                                                          TensorDesc::getLayoutByDims(input->get_shape()));
             } else {
-                IE_THROW() << "Mean image of type " << inTensorDesc.getPrecision().name() << " is unsupported";
+                OPENVINO_THROW("Mean image of type ", input->get_element_type().get_type_name(), " is unsupported");
             }
         }
     } else {
-        IE_THROW() << "Input blob for infer '" << name << "' doesn't correspond to input in network";
+        OPENVINO_THROW("Input blob for infer '", name, "' doesn't correspond to input in network");
     }
 }
 
 // suppose always being shared infer_request intel_cpu::Tensor to Graph if isDynamic.
-void Graph::PullOutputData(BlobMap &out) {
+void Graph::PullOutputData(std::unordered_map<std::string, ov::SoPtr<ITensor>>& output) {
     if (!IsReady())
-        IE_THROW() << "Wrong state. Topology not ready.";
+        OPENVINO_THROW("Wrong state. Topology not ready.");
 
     for (auto &outputMap : outputNodesMap) {
         auto name = outputMap.first;
@@ -1044,16 +945,19 @@ void Graph::PullOutputData(BlobMap &out) {
         auto parentEdge = node->getParentEdgeAt(0);
         const auto& intr_blob = parentEdge->getMemory();
 
-        const auto ext_blob_map = out.find(name);
+        const auto ext_blob_map = output.find(name);
         const auto ext_blob = ext_blob_map->second;
-        if (ext_blob_map == out.end()) {
-            IE_THROW(Unexpected) << "The CPU plugin graph doesn't contain output node with name: \"" << name << "\"";
+        if (ext_blob_map == output.end()) {
+            OPENVINO_THROW("The CPU plugin graph doesn't contain output node with name: ", name.c_str());
         }
 
-        DEBUG_LOG(name, ", blob ", out[name], ", addr ", static_cast<void*>(out[name]->buffer()));
+        InferenceEngine::TensorDesc expectedDesc(
+            InferenceEngine::details::convertPrecision(ext_blob->get_element_type()),
+            ext_blob->get_shape(),
+            InferenceEngine::TensorDesc::getLayoutByRank(ext_blob->get_shape().size()));
+        DEBUG_LOG(name, ", tensor data addr ", static_cast<void*>(output[name]->data()));
 
         const auto actualDesc = MemoryDescUtils::convertToTensorDesc(intr_blob.getDesc());
-        auto &expectedDesc = ext_blob->getTensorDesc();
 
         // TODO [NM]: need to create universal reorder which will be detect cases when we really need to use it
         // WA: for cases when output shape after transformation will be 1x1x1x1 but model output is scalar
@@ -1069,18 +973,22 @@ void Graph::PullOutputData(BlobMap &out) {
         }
 
         auto outDims = intr_blob.getStaticDims();
-        if (out[name]->getTensorDesc().getDims() != outDims && !isScalarOutput) {
+        if (ext_blob->get_shape() != outDims && !isScalarOutput) {
             // WA: because input/output info initially contains non empty dims, order etc.
             // and setDims (called inside setShape) can't correct modify blocked desc for desc with blocked layout
             if (expectedDesc.getLayout() == InferenceEngine::Layout::BLOCKED) {
                 expectedDesc = TensorDesc(expectedDesc.getPrecision(), expectedDesc.getLayout());
             }
-            DEBUG_LOG(name, ", blob ", out[name], ", addr ", static_cast<void*>(out[name]->buffer()),
-            " dims ", PartialShape(out[name]->getTensorDesc().getDims()), " -> ", PartialShape(outDims),
+            DEBUG_LOG(name, ", tensor data addr ", static_cast<void*>(output[name]->data()),
+            " dims ", PartialShape(output[name]->get_shape()), " -> ", PartialShape(outDims),
             ", intr ptr ", intr_blob.getData(), " , parentedge's memory object ", parentEdge->getMemoryPtr().get());
-            out[name]->setShape(outDims);
-            DEBUG_LOG(name, ", blob ", out[name], ", addr ", static_cast<void*>(out[name]->buffer()),
-            " dims ", PartialShape(out[name]->getTensorDesc().getDims()), ", intr ptr ", intr_blob.getData());
+            ext_blob->set_shape(outDims);
+            DEBUG_LOG(name, ", tensor data addr ", static_cast<void*>(output[name]->data()),
+            " dims ", PartialShape(output[name]->get_shape()), ", intr ptr ", intr_blob.getData());
+            expectedDesc =
+                InferenceEngine::TensorDesc(InferenceEngine::details::convertPrecision(ext_blob->get_element_type()),
+                                            ext_blob->get_shape(),
+                                            InferenceEngine::TensorDesc::getLayoutByRank(ext_blob->get_shape().size()));
         }
 
         // check for empty output blob
@@ -1090,14 +998,15 @@ void Graph::PullOutputData(BlobMap &out) {
 
         auto srcPrec = actualDesc.getPrecision();
         auto dstPrec = expectedDesc.getPrecision();
+        if (!getConfig().isLegacyApi && srcPrec == dstPrec && ext_blob->get_byte_size() != intr_blob.getSize())
+            OPENVINO_THROW("Output blob byte size is not equal network output byte size (",
+                           ext_blob->get_byte_size(),
+                           "!=",
+                           intr_blob.getSize(),
+                           ").");
 
-        if (!getConfig().isLegacyApi && srcPrec == dstPrec && ext_blob->byteSize() != intr_blob.getSize())
-            IE_THROW() << "Output blob byte size is not equal network output byte size (" << ext_blob->byteSize()
-                       << "!=" << intr_blob.getSize() << ").";
-
-        void *ext_blob_ptr = ext_blob->buffer();
+        void *ext_blob_ptr = ext_blob->data();
         void *intr_blob_ptr = intr_blob.getData();
-
         DEBUG_LOG(name, " @ ", intr_blob_ptr, " -> ", ext_blob_ptr, " zero-copy: ", intr_blob_ptr == ext_blob_ptr, " graph ", this, "\r\n");
 
         // That is the same memory. No need to copy
@@ -1107,19 +1016,19 @@ void Graph::PullOutputData(BlobMap &out) {
             // User can initialize output via SetOutput API using tensorDesc with ANY layout.
             // For these cases we create planar memory descriptor.
             auto outBlobDesc = expectedDesc.getLayout() == InferenceEngine::Layout::ANY
-                                ? DnnlBlockedMemoryDesc(expectedDesc.getPrecision(), Shape(expectedDesc.getDims()))
-                                : MemoryDescUtils::convertToDnnlBlockedMemoryDesc(expectedDesc);
+                                   ? DnnlBlockedMemoryDesc(expectedDesc.getPrecision(), Shape(expectedDesc.getDims()))
+                                   : MemoryDescUtils::convertToDnnlBlockedMemoryDesc(expectedDesc);
             Memory outBloMem(getEngine(), outBlobDesc, ext_blob_ptr, false);
             outBloMem.load(intr_blob, false);
         } else {
             size_t size_to_copy = intr_blob.getDescWithType<BlockedMemoryDesc>()->getPaddedElementsCount();
-
+            DEBUG_LOG("pull_output: convert ", srcPrec, " to ", dstPrec);
             cpu_convert(intr_blob_ptr, ext_blob_ptr, srcPrec, dstPrec, size_to_copy);
         }
     }
 }
 
-void Graph::InferStatic(InferRequestBase* request) {
+void Graph::InferStatic(SyncInferRequest* request) {
     dnnl::stream stream(getEngine());
 
     for (const auto& node : executableGraphNodes) {
@@ -1127,7 +1036,7 @@ void Graph::InferStatic(InferRequestBase* request) {
         PERF(node, getConfig().collectPerfCounters);
 
         if (request)
-            request->ThrowIfCanceled();
+            request->throw_if_canceled();
         ExecuteNode(node, stream);
     }
 }
@@ -1327,7 +1236,7 @@ public:
 } // namespace
 
 
-void Graph::InferDynamic(InferRequestBase* request) {
+void Graph::InferDynamic(SyncInferRequest* request) {
     dnnl::stream stream(getEngine());
 
     std::set<size_t> syncIndsWorkSet;
@@ -1355,7 +1264,7 @@ void Graph::InferDynamic(InferRequestBase* request) {
             PERF(node, getConfig().collectPerfCounters);
 
             if (request)
-                request->ThrowIfCanceled();
+                request->throw_if_canceled();
             ExecuteNode(node, stream);
         }
     }
@@ -1373,9 +1282,9 @@ inline void Graph::ExecuteNode(const NodePtr& node, const dnnl::stream& stream)
     }
 }
 
-void Graph::Infer(InferRequestBase* request) {
+void Graph::Infer(SyncInferRequest* request) {
     if (!IsReady()) {
-        IE_THROW() << "Wrong state of the ov::intel_cpu::Graph. Topology is not ready.";
+        OPENVINO_THROW("Wrong state of the ov::intel_cpu::Graph. Topology is not ready.");
     }
 
     if (Status::ReadyDynamic == status) {
@@ -1383,7 +1292,7 @@ void Graph::Infer(InferRequestBase* request) {
     } else if (Status::ReadyStatic == status) {
         InferStatic(request);
     } else {
-        IE_THROW() << "Unknown ov::intel_cpu::Graph state: " << static_cast<size_t>(status);
+        OPENVINO_THROW("Unknown ov::intel_cpu::Graph state: " , static_cast<size_t>(status));
     }
 
     if (infer_count != -1) infer_count++;
@@ -1476,30 +1385,27 @@ void Graph::SortTopologically() {
     }
 }
 
-void Graph::GetPerfData(std::map<std::string, InferenceEngine::InferenceEngineProfileInfo> &perfMap) const {
-    unsigned i = 0;
-    std::function<void(std::map<std::string, InferenceEngine::InferenceEngineProfileInfo> &, const NodePtr&)>
-            getPerfMapFor = [&](std::map<std::string, InferenceEngine::InferenceEngineProfileInfo> &perfMap, const NodePtr& node) {
-        InferenceEngine::InferenceEngineProfileInfo &pc = perfMap[node->getName()];
-        pc.execution_index = i++;
-        // TODO: Why time counter is signed?
-        pc.cpu_uSec = pc.realTime_uSec = (long long) node->PerfCounter().avg();
-        pc.status = pc.cpu_uSec > 0 ? InferenceEngine::InferenceEngineProfileInfo::EXECUTED
-                                    : InferenceEngine::InferenceEngineProfileInfo::NOT_RUN;
-        std::string pdType = node->getPrimitiveDescriptorType();
-        size_t typeLen = sizeof(pc.exec_type) / sizeof(pc.exec_type[0]);
-        pdType.copy(pc.exec_type, typeLen, 0);
-        size_t layerTypeLen = sizeof(pc.layer_type) / sizeof(pc.layer_type[0]);
-        node->typeStr.copy(pc.layer_type, layerTypeLen, 0);
+void Graph::GetPerfData(std::vector<ov::ProfilingInfo>& perfMap) const {
+    std::function<void(std::vector<ov::ProfilingInfo>&, const NodePtr&)> getPerfMapFor =
+        [&](std::vector<ov::ProfilingInfo>& perfMap, const NodePtr& node) {
+            ov::ProfilingInfo pc;
+            pc.node_name = node->getName();
+            // pc.execution_index = i++;
+            uint64_t avg_time = node->PerfCounter().avg();
+            pc.cpu_time = pc.real_time = std::chrono::microseconds(avg_time);
+            pc.status = avg_time > 0 ? ov::ProfilingInfo::Status::EXECUTED : ov::ProfilingInfo::Status::NOT_RUN;
+            pc.exec_type = node->getPrimitiveDescriptorType();
+            pc.node_type = node->typeStr;
+            perfMap.emplace_back(pc);
 
-        for (auto& fusedNode : node->fusedWith) {
-            getPerfMapFor(perfMap, fusedNode);
-        }
+            for (auto& fusedNode : node->fusedWith) {
+                getPerfMapFor(perfMap, fusedNode);
+            }
 
-        for (auto& mergedWith : node->mergedWith) {
-            getPerfMapFor(perfMap, mergedWith);
-        }
-    };
+            for (auto& mergedWith : node->mergedWith) {
+                getPerfMapFor(perfMap, mergedWith);
+            }
+        };
 
     for (size_t i = 0; i < graphNodes.size(); i++) {
         if (graphNodes[i]->isConstant())
@@ -1639,7 +1545,7 @@ NodePtr Graph::InsertReorder(EdgePtr edge, std::string layerName, const MemoryDe
     NodePtr newReorder(new node::Reorder(layerName, context));
     auto *reorderPtr = dynamic_cast<node::Reorder *>(newReorder.get());
     if (reorderPtr == nullptr) {
-        IE_THROW() << "Graph::InsertReorder: Cannot cast to Reorder";
+        OPENVINO_THROW("Graph::InsertReorder: Cannot cast to Reorder");
     }
     reorderPtr->setDescs(inDesc, outDesc);
     reorderPtr->setOptimized(isOptimized);
@@ -1665,10 +1571,13 @@ bool Graph::InsertNode(EdgePtr edge, NodePtr node, bool initNode) {
     auto oIndex = edge->getOutputNum();
     auto iIndex = edge->getInputNum();
     if (iIndex < 0 || oIndex < 0)
-        IE_THROW() << "Cannot insert node '" << node->getName() << "' between nodes: "
-                           << edge->getParent()->getName() << " and "
-                           << edge->getChild()->getName() << ".";
-
+        OPENVINO_THROW("Cannot insert node '",
+                       node->getName(),
+                       "' between nodes: ",
+                       edge->getParent()->getName(),
+                       " and ",
+                       edge->getChild()->getName(),
+                       ".");
     edge->drop();
 
     return InsertNode(edge->getParent(), edge->getChild(), node, iIndex, oIndex, initNode);
@@ -1837,7 +1746,7 @@ void Graph::resolveInPlaceDirection(const NodePtr& node) const {
                 } else if (inPlaceOutPort < 0) {
                     return InplaceDirectionType::DOWN;
                 } else {
-                    IE_THROW() << "Non trivial inPlace memory dependency has been detected";
+                    OPENVINO_THROW("Non trivial inPlace memory dependency has been detected");
                 }
             }
             // the requested port has a negative inPlace tag, let's check whether it is referenced from the output
@@ -1856,7 +1765,7 @@ void Graph::resolveInPlaceDirection(const NodePtr& node) const {
                 } else if (inPlaceInpPort < 0) {
                     return InplaceDirectionType::UP;
                 } else {
-                    IE_THROW() << "Non trivial inPlace memory dependency has been detected";
+                    OPENVINO_THROW("Non trivial inPlace memory dependency has been detected");
                 }
             }
             // the requested port has a negative inPlace tag, let's check whether it is referenced from the input
@@ -1935,7 +1844,7 @@ void Graph::resolveInPlaceDirection(const NodePtr& node) const {
                     config.outConfs[inPlaceInpPort].inPlace(-1);
                     node->initDescriptor(config);
                 } else {
-                    IE_THROW() << "A node without an inPlace memory cyclic dependency has not been found";
+                    OPENVINO_THROW("A node without an inPlace memory cyclic dependency has not been found");
                 }
             }
         }

src/plugins/intel_cpu/src/graph.h

@@ -4,28 +4,29 @@
 
 #pragma once
 
-#include "cpp/ie_cnn_network.h"
+#include "cache/multi_cache.h"
 #include "config.h"
 #include "cpu_memory.h"
-#include "normalize_preprocess.h"
-#include "node.h"
-#include "edge.h"
-#include "cache/multi_cache.h"
 #include "dnnl_scratch_pad.h"
+#include "edge.h"
 #include "graph_context.h"
+#include "node.h"
+#include "normalize_preprocess.h"
+#include "openvino/runtime/make_tensor.hpp"
+#include "openvino/runtime/profiling_info.hpp"
+
+#include <atomic>
 #include <map>
+#include <memory>
 #include <string>
 #include <vector>
-#include <memory>
-#include <atomic>
 
 #include "proxy_mem_mgr.h"
 
 namespace ov {
 namespace intel_cpu {
 
-class InferRequestBase;
-class InferRequest;
+class SyncInferRequest;
 
 class Graph {
 public:
@@ -60,10 +61,10 @@ public:
         return _normalizePreprocMap.find(name) != _normalizePreprocMap.end();
     }
 
-    void PushInputData(const std::string& name, const InferenceEngine::Blob::Ptr &in);
-    void PullOutputData(InferenceEngine::BlobMap &out);
+    void PushInputData(const std::string& name, const ov::SoPtr<ITensor>& input);
+    void PullOutputData(std::unordered_map<std::string, ov::SoPtr<ITensor>>& output);
 
-    void Infer(InferRequestBase* request = nullptr);
+    void Infer(SyncInferRequest* request = nullptr);
 
     const std::vector<NodePtr>& GetNodes() const {
         return graphNodes;
@@ -92,14 +93,14 @@ public:
     NodePtr getInputNodeByName(const std::string &name) {
         auto input = inputNodesMap.find(name);
         if (input == inputNodesMap.end())
-            IE_THROW() << "CPU execution graph doesn't contain input node with name: " << name;
+            OPENVINO_THROW("CPU execution graph doesn't contain input node with name: ", name);
         return input->second;
     }
 
     NodePtr getOutputNodeByName(const std::string &name) {
         auto output = outputNodesMap.find(name);
         if (output == outputNodesMap.end())
-            IE_THROW() << "CPU execution graph doesn't contain output node with name: " << name;
+            OPENVINO_THROW("CPU execution graph doesn't contain output node with name: ", name);
         return output->second;
     }
 
@@ -115,7 +116,7 @@ public:
         return context;
     }
 
-    void GetPerfData(std::map<std::string, InferenceEngine::InferenceEngineProfileInfo> &perfMap) const;
+    void GetPerfData(std::vector<ov::ProfilingInfo> &perfMap) const;
 
     void RemoveDroppedNodes();
     void RemoveDroppedEdges();
@@ -182,7 +183,7 @@ public:
      */
     bool InsertNode(NodePtr parent, NodePtr child, NodePtr node, int parentPort, int childPort, bool initNode = false);
 
-    std::shared_ptr<ngraph::Function> dump() const;
+    std::shared_ptr<ov::Model> dump() const;
 
     void ResetInferCount() { infer_count = 0; }
 
@@ -225,7 +226,6 @@ protected:
 
     bool graphHasDynamicInput = false;
 
-    void Replicate(const InferenceEngine::CNNNetwork &network);
     void Replicate(const std::shared_ptr<const ov::Model> &subgraph);
     void InitGraph();
     void InitNodes();
@@ -239,13 +239,11 @@ protected:
     void ExtractExecutableNodes();
     void ExecuteNode(const NodePtr& node, const dnnl::stream& stream) const;
     void CreatePrimitivesAndExecConstants() const;
-    void InferStatic(InferRequestBase* request);
-    void InferDynamic(InferRequestBase* request);
+    void InferStatic(SyncInferRequest* request);
+    void InferDynamic(SyncInferRequest* request);
 
-    friend class LegacyInferRequest;
-    friend class intel_cpu::InferRequest;
-    friend class intel_cpu::InferRequestBase;
-    friend std::shared_ptr<ngraph::Function> dump_graph_as_ie_ngraph_net(const Graph &graph);
+    friend class intel_cpu::SyncInferRequest;
+    friend std::shared_ptr<ov::Model> dump_graph_as_ie_ngraph_net(const Graph &graph);
 
 private:
     // TODO: change std::map to std::unordered_map
@@ -268,5 +266,5 @@ private:
     void resolveInPlaceDirection(const NodePtr& node) const;
 };
 
-}   // namespace intel_cpu
-}   // namespace ov
+}  // namespace intel_cpu
+}  // namespace ov

src/plugins/intel_cpu/src/infer_request.h

@@ -5,55 +5,48 @@
 #pragma once
 
 #include "graph.h"
-#include <memory>
-#include <string>
-#include <map>
-#include <cpp_interfaces/interface/ie_iinfer_request_internal.hpp>
 #include "cpu_tensor.h"
+#include "openvino/runtime/iinfer_request.hpp"
+#include "openvino/runtime/isync_infer_request.hpp"
 
 namespace ov {
 namespace intel_cpu {
 
-class ExecNetwork;
+class CompiledModel;
 class AsyncInferRequest;
 
-class InferRequestBase : public InferenceEngine::IInferRequestInternal {
+class SyncInferRequest : public ov::ISyncInferRequest {
 public:
-    virtual ~InferRequestBase();
+    SyncInferRequest(std::shared_ptr<const CompiledModel> compiled_model);
+    virtual ~SyncInferRequest();
 
-    void InferImpl() override;
+    void infer() override;
 
-    std::map<std::string, InferenceEngine::InferenceEngineProfileInfo> GetPerformanceCounts() const override;
+    std::vector<ov::ProfilingInfo> get_profiling_info() const override;
 
-    std::vector<std::shared_ptr<InferenceEngine::IVariableStateInternal>> QueryState() override;
+    std::vector<ov::SoPtr<ov::IVariableState>> query_state() const override;
+
+    void set_tensor(const ov::Output<const ov::Node>& port, const ov::SoPtr<ov::ITensor>& tensor) override;
+
+    void set_tensors_impl(const ov::Output<const ov::Node> port, const std::vector<ov::SoPtr<ov::ITensor>>& tensors) override;
+
+    ov::SoPtr<ov::ITensor> get_tensor(const ov::Output<const ov::Node>& port) const override;
+    std::vector<ov::SoPtr<ov::ITensor>> get_tensors(const ov::Output<const ov::Node>& _port) const override;
 
     /**
      * @brief      Sets the pointer to asynchronous inference request that holds this request
      * @param[in]  asyncRequest Pointer to asynchronous inference request
      */
-    void SetAsyncRequest(AsyncInferRequest* asyncRequest);
+    void set_async_request(AsyncInferRequest* asyncRequest);
 
     /**
-     * @brief If `_asyncRequest` is initialized throw exception with `InferenceEngine::INFER_CANCELLED` status if inference request is canceled
+     * @brief If `m_asyncRequest` is initialized throw exception with `ov::Cancelled` status if inference request is
+     * canceled
      */
-    void ThrowIfCanceled() const;
 
-protected:
-    InferRequestBase(InferenceEngine::InputsDataMap networkInputs,
-                     InferenceEngine::OutputsDataMap networkOutputs,
-                     std::shared_ptr<ExecNetwork> execNetwork_)
-    : IInferRequestInternal(networkInputs, networkOutputs), execNetwork(execNetwork_) {}
+    void throw_if_canceled() const;
 
-    InferRequestBase(const std::vector<std::shared_ptr<const ov::Node>>& inputs,
-                     const std::vector<std::shared_ptr<const ov::Node>>& outputs,
-                     std::shared_ptr<ExecNetwork> execNetwork_)
-    : IInferRequestInternal(inputs, outputs), execNetwork(execNetwork_) {}
-
-    void CreateInferRequest();
-    InferenceEngine::Precision normToInputSupportedPrec(const std::pair<const std::string, InferenceEngine::Blob::Ptr>& input) const;
-    void pushInput(const std::string& inputName, InferenceEngine::Blob::Ptr& inputBlob, InferenceEngine::Precision dataType);
-
-protected:
+private:
     class OutputControlBlock {
     public:
         using MemMngrPtr = std::shared_ptr<MemoryMngrWithReuse>;
@@ -67,10 +60,6 @@ protected:
         OutputControlBlock(OutputControlBlock&&) = default;
         OutputControlBlock& operator=(OutputControlBlock&&) = default;
 
-        InferenceEngine::Blob::Ptr blob() const {
-            return m_blob;
-        }
-
         std::shared_ptr<Tensor> tensor() const {
             return m_tensor;
         }
@@ -93,74 +82,43 @@ protected:
 
         void update() {
             m_proxyMemMngr->setMemMngr(currentMemMngr());
-            m_blob->allocate(); // WA: update handle
         }
 
     private:
         std::shared_ptr<Tensor> m_tensor = nullptr;
-        InferenceEngine::Blob::Ptr m_blob = nullptr;
         ProxyMemoryMngrPtr m_proxyMemMngr = nullptr;
         std::array<MemMngrPtr, 2> m_buffers;
         int m_buffIndx = 0;
     };
-
-protected:
-    virtual void initBlobs() = 0;
-    virtual void PushInputData() = 0;
-
-    Graph* graph = nullptr;
-    std::unordered_map<std::string, InferenceEngine::Blob::Ptr> externalPtr;
-
     std::unordered_map<std::string, OutputControlBlock> outputControlBlocks;
 
-private:
-    void PushStates();
-    void PullStates();
-    void redefineMemoryForInputNodes();
+    void create_infer_request();
 
-    std::shared_ptr<ExecNetwork>        execNetwork;
-    openvino::itt::handle_t             profilingTask;
-    std::vector<std::shared_ptr<InferenceEngine::IVariableStateInternal>> memoryStates;
-    AsyncInferRequest*                  _asyncRequest = nullptr;
+    void init_tensor(const std::string& name);
+    void push_input_data();
 
-protected:
-    virtual void changeDefaultPtr();
+    Graph* graph = nullptr;
+    std::unordered_map<std::string, ov::SoPtr<ov::ITensor>> external_ptr;
+
+    void push_states();
+    void pull_states();
+    void redefine_memory_for_input_nodes();
+
+    void update_external_tensor_ptrs();
+    const ov::Output<const ov::Node>& get_internal_port(const ov::Output<const ov::Node>& port) const;
+    bool m_is_legacy_api = false;
+
+    std::shared_ptr<const CompiledModel> m_compiled_model;
+    openvino::itt::handle_t m_profiling_task;
+    std::vector<ov::SoPtr<ov::IVariableState>> m_memory_states;
+    AsyncInferRequest* m_asyncRequest = nullptr;
+
+    std::unordered_map<std::string, ov::Output<const ov::Node>> m_input_ports_map;
+    std::unordered_map<std::string, ov::Output<const ov::Node>> m_output_ports_map;
+    std::unordered_map<std::string, ov::SoPtr<ov::ITensor>> m_outputs;
+
+    void change_default_ptr();
 };
 
-class LegacyInferRequest : public InferRequestBase {
-public:
-    LegacyInferRequest(InferenceEngine::InputsDataMap networkInputs,
-                       InferenceEngine::OutputsDataMap networkOutputs,
-                       std::shared_ptr<ExecNetwork> execNetwork);
-
-    void SetBlob(const std::string& name, const InferenceEngine::Blob::Ptr &data) override;
-    InferenceEngine::Blob::Ptr GetBlob(const std::string& name) override;
-
-private:
-    void PushInputData() override;
-    void initBlobs() override;
-    void changeDefaultPtr() override;
-};
-
-class InferRequest : public InferRequestBase {
-public:
-    InferRequest(const std::vector<std::shared_ptr<const ov::Node>>& inputs,
-                 const std::vector<std::shared_ptr<const ov::Node>>& outputs,
-                 std::shared_ptr<ExecNetwork> execNetwork);
-
-    void SetBlob(const std::string& name, const InferenceEngine::Blob::Ptr &data) override;
-    void SetBlobsImpl(const std::string& name, const InferenceEngine::BatchedBlob::Ptr& batched_blob) override;
-    InferenceEngine::Blob::Ptr GetBlob(const std::string& name) override;
-
-    void checkBlobs() override;
-
-private:
-    void PushInputData() override;
-    void initBlobs() override;
-
-    std::unordered_map<std::string, std::shared_ptr<const ov::Node>> modelInputsMap;
-    std::unordered_map<std::string, std::shared_ptr<const ov::Node>> modelOutputsMap;
-};
-
-}   // namespace intel_cpu
-}   // namespace ov
+}  // namespace intel_cpu
+}  // namespace ov

src/plugins/intel_cpu/src/memory_state.cpp

@@ -3,18 +3,17 @@
 //
 
 #include "memory_state.h"
+
 #include "dnnl_extension_utils.h"
-#include "blob_factory.hpp"
 
 using namespace InferenceEngine;
 
 namespace ov {
 namespace intel_cpu {
 
-void VariableState::Reset() {
-    std::memset(state->buffer(), 0, state->byteSize());
+void VariableState::reset() {
+    std::memset(m_state->data(), 0, m_state->get_byte_size());
 }
 
-}   // namespace intel_cpu
-}   // namespace ov
-
+}  // namespace intel_cpu
+}  // namespace ov

src/plugins/intel_cpu/src/memory_state.h

@@ -4,28 +4,29 @@
 
 #pragma once
 
-#include "cpp_interfaces/interface/ie_ivariable_state_internal.hpp"
-#include "blob_factory.hpp"
-#include "cpu_memory.h"
-#include "nodes/common/cpu_memcpy.h"
-#include "memory_desc/cpu_memory_desc_utils.h"
-
 #include <string>
 
+#include "cpu_memory.h"
+#include "ie_ngraph_utils.hpp"
+#include "memory_desc/cpu_memory_desc_utils.h"
+#include "nodes/common/cpu_memcpy.h"
+#include "openvino/runtime/ivariable_state.hpp"
+#include "openvino/runtime/make_tensor.hpp"
+#include "openvino/runtime/tensor.hpp"
+
 namespace ov {
 namespace intel_cpu {
 
-class VariableState : public InferenceEngine::IVariableStateInternal {
+class VariableState : public ov::IVariableState {
 public:
-    VariableState(std::string name, MemoryPtr storage)
-        : InferenceEngine::IVariableStateInternal{name} {
-        state = make_blob_with_precision(MemoryDescUtils::convertToTensorDesc(storage->getDesc()));
-        state->allocate();
-        cpu_memcpy(state->buffer(), storage->getData(), storage->getSize());
+    VariableState(std::string name, MemoryPtr storage) : ov::IVariableState{name} {
+        const auto& memDesc = MemoryDescUtils::convertToTensorDesc(storage->getDesc());
+        m_state = ov::make_tensor(InferenceEngine::details::convertPrecision(memDesc.getPrecision()), memDesc.getDims());
+        cpu_memcpy(m_state->data(), storage->getData(), storage->getSize());
     }
 
-    void Reset() override;
+    void reset() override;
 };
 
-}   // namespace intel_cpu
-}   // namespace ov
+}  // namespace intel_cpu
+}  // namespace ov

src/plugins/intel_cpu/src/node.cpp

@@ -1247,7 +1247,7 @@ std::vector<InferenceEngine::Precision> Node::getInputPrecisions() const {
     for (size_t i = 0; i < getParentEdges().size(); i++) {
         auto parentEdge = getParentEdgeAt(i);
         if (parentEdge && parentEdge->getStatus() == Edge::Status::Validated) {
-            inputPrecisions.emplace_back(DnnlExtensionUtils::DataTypeToIEPrecision((parentEdge->getMemoryPtr()->getDataType())));
+            inputPrecisions.emplace_back(parentEdge->getMemoryPtr()->getDesc().getPrecision());
         }
     }
     return inputPrecisions;
@@ -1258,7 +1258,7 @@ std::vector<InferenceEngine::Precision> Node::getOutputPrecisions() const {
     for (size_t i = 0; i < getChildEdges().size(); i++) {
         auto childEdge = getChildEdgeAt(i);
         if (childEdge && childEdge->getStatus() == Edge::Status::Validated) {
-            outputPrecisions.emplace_back(DnnlExtensionUtils::DataTypeToIEPrecision((childEdge->getMemoryPtr()->getDataType())));
+            outputPrecisions.emplace_back(childEdge->getMemoryPtr()->getDesc().getPrecision());
         }
     }
     return outputPrecisions;

src/plugins/intel_cpu/src/nodes/common/cpu_convert.cpp

@@ -223,8 +223,15 @@ const std::tuple<U, U> & Range<T, U>::fit(const Precision & prec) {
             default:
                 IE_THROW() << "Unsupported precision";
         }
-        std::get<0>(_range) = static_cast<U>(std::max(static_cast<double>(std::get<0>(_range)), lbound));
-        std::get<1>(_range) = static_cast<U>(std::min(static_cast<double>(std::get<1>(_range)), ubound));
+        // If U is integral, its range always less than float, so not need update _range
+        // Else it will be overflow, for example static_cast double to int64_t:
+        //         int64_t ubound = 9223372036854775807
+        //         double  dd_ubound = static_cast<double>(ubbound)
+        //         static_cast<int64_t>(dd_ubound) will return -9223372036854775808
+        if (!std::is_integral<U>::value) {
+                std::get<0>(_range) = static_cast<U>(std::max(static_cast<double>(std::get<0>(_range)), lbound));
+                std::get<1>(_range) = static_cast<U>(std::min(static_cast<double>(std::get<1>(_range)), ubound));
+        }
     } else {
         int64_t lbound;
         uint64_t ubound;

src/plugins/intel_cpu/src/nodes/convert.cpp

@@ -126,7 +126,19 @@ void Convert::initSupportedPrimitiveDescriptors() {
         config.outConfs.push_back(dataConfigOut);
 
         auto creators = BlockedDescCreator::getCommonCreators();
-        auto range = BlockedDescCreator::makeFilteredRange(creators, insShape.getRank());
+
+        // As long as convert is placed right before the output, only planar layout makes sense since the output tensor
+        // is always in a planar layout (ngraph limitation), so there is no reason to convert in any other layout.
+        bool hasOutputChild = false;
+        for (auto& childEdge : getChildEdgesAtPort(0)) {
+            if (Type::Output == childEdge->getChild()->getType()) {
+                hasOutputChild = true;
+                break;
+            }
+        }
+        auto range = hasOutputChild
+                         ? BlockedDescCreator::makeFilteredRange(creators, insShape.getRank(), {LayoutType::ncsp})
+                         : BlockedDescCreator::makeFilteredRange(creators, insShape.getRank());
 
         for (auto itr = range.first; itr != range.second; ++itr) {
             config.inConfs[0].setMemDesc(std::make_shared<CpuBlockedMemoryDesc>(itr->second->createDesc(insPrecision, insShape)));

src/plugins/intel_cpu/src/nodes/eltwise.cpp

@@ -2180,6 +2180,8 @@ void Eltwise::initSupportedPrimitiveDescriptors() {
         } else if (std::find(supportedPrecisions.begin(), supportedPrecisions.end(), prc) == supportedPrecisions.end()) {
             if (prc == Precision::U32 || prc == Precision::I64 || prc == Precision::U64) {
                 return Precision(Precision::I32);
+            } else if (prc == Precision::FP64) {
+                return Precision(Precision::FP32);
             } else {
                 IE_THROW() << "Eltwise node with name `" << getName() << "` doesn't support " << prc << " precision.";
             }

src/plugins/intel_cpu/src/nodes/input.cpp

@@ -256,7 +256,7 @@ void Input::cloneBlobIfRequired() {
     Shape shape(constOp->get_shape().empty() ? ngraph::Shape(1, 1) : constOp->get_shape());
     const auto prec = convertPrecision(constOp->get_element_type());
     const size_t size = shape.getElementsCount();
-    DnnlBlockedMemoryDesc memDesc(prec, shape);
+    CpuBlockedMemoryDesc memDesc(prec, shape);
 
     bool needFlushDenormalsToZero = true;
     if (context->getConfig().DAZOn) {
@@ -408,10 +408,6 @@ Input::Input(MemoryDescPtr memDesc, const std::string& name, const std::string&
     extMemDesc = memDesc;
 }
 
-void Input::withMeanImage() {
-    isMeanImage = true;
-}
-
 MemoryCPtr Input::getMemoryPtr() const {
     return memoryPtr;
 }
@@ -470,9 +466,6 @@ void Input::initSupportedPdDefault() {
 
     if (getType() == Type::Input || getType() == Type::MemoryInput) {
         auto precision = getOriginalOutputPrecisionAtPort(0);
-        if (precision == Precision::U16 || isMeanImage) {
-            precision = Precision::FP32;
-        }
 
         outPortConfs.push_back({LayoutType::ncsp, precision});
         if (!getParentEdges().empty()) {
@@ -480,7 +473,6 @@ void Input::initSupportedPdDefault() {
         }
     } else if (getType() == Type::Output) {
         auto precision = getOriginalInputPrecisionAtPort(0);
-        if (precision == Precision::U16) precision = Precision::FP32;
 
         inPortConfs.push_back({LayoutType::ncsp, precision});
     }

src/plugins/intel_cpu/src/nodes/topk.cpp

@@ -2073,7 +2073,7 @@ void TopK::createPrimitive() {
         layout = TopKLayoutType::topk_blocked;
     }
 
-    if (inputShapesDefined() && isExecutable()) {
+    if (!isDynamicNode() && isExecutable()) {
         if (needPrepareParams())
             prepareParams();
         updateLastInputDims();

src/plugins/intel_cpu/src/plugin.cpp

@@ -2,32 +2,30 @@
 // SPDX-License-Identifier: Apache-2.0
 //
 
-#include "ie_metric_helpers.hpp" // must be included first
+#include "ie_metric_helpers.hpp"  // must be included first
 
-#include "openvino/runtime/properties.hpp"
 #include "plugin.h"
 
-#include "transformations/transformation_pipeline.h"
-#include "itt.h"
-#include "extension_mngr.h"
+#include "cpp_interfaces/interface/ie_internal_plugin_config.hpp"
 #include "extension.h"
-#include "serialize.h"
-#include "threading/ie_executor_manager.hpp"
+#include "extension_mngr.h"
 
-#include "ie_icore.hpp"
+#include "ie_ngraph_utils.hpp"
 #include "ie_plugin_config.hpp"
 #include "ie_system_conf.h"
+#include "itt.h"
 #include "openvino/runtime/threading/cpu_streams_info.hpp"
-#include "cpp_interfaces/interface/ie_internal_plugin_config.hpp"
 #include "openvino/runtime/intel_cpu/properties.hpp"
-
-#include <transformations/utils/utils.hpp>
-#include <ie_ngraph_utils.hpp>
-
-#include "performance_heuristics.hpp"
 #include "openvino/runtime/properties.hpp"
-#include "weights_cache.hpp"
+#include "openvino/runtime/threading/cpu_streams_info.hpp"
+#include "openvino/runtime/threading/executor_manager.hpp"
+#include "performance_heuristics.hpp"
+#include "serialize.h"
+#include "threading/ie_executor_manager.hpp"
+#include "transformations/transformation_pipeline.h"
+#include "transformations/utils/utils.hpp"
 #include "utils/denormals.hpp"
+#include "weights_cache.hpp"
 
 #if defined(__linux__)
 # include <sys/auxv.h>
@@ -36,15 +34,14 @@
 #endif
 
 #include <cpu/x64/cpu_isa_traits.hpp>
-#include <itt.h>
+
+using namespace ov::threading;
 
 #if defined(OV_CPU_WITH_ACL)
 #include "nodes/executors/acl/acl_ie_scheduler.hpp"
 #include "arm_compute/runtime/CPP/CPPScheduler.h"
 #endif
 
-using namespace InferenceEngine;
-
 #define IE_CPU_PLUGIN_THROW(...) IE_THROW(__VA_ARGS__) << "CPU plugin: "
 
 namespace ov {
@@ -175,7 +172,7 @@ Engine::SchedulerGuard::~SchedulerGuard() {
 Engine::Engine() :
     deviceFullName(getDeviceFullName()),
     specialSetup(new CPUSpecialSetup) {
-    _pluginName = "CPU";
+    set_device_name("CPU");
     extensionManager->AddExtension(std::make_shared<Extension>());
 #if defined(OV_CPU_WITH_ACL)
     scheduler_guard = SchedulerGuard::instance();
@@ -183,26 +180,27 @@ Engine::Engine() :
 }
 
 Engine::~Engine() {
-    executorManager()->clear("CPU");
-    executorManager()->clear("CPUStreamsExecutor");
-    executorManager()->clear("CPUCallbackExecutor");
+    executor_manager()->clear("CPU");
+    executor_manager()->clear("CPUStreamsExecutor");
+    executor_manager()->clear("CPUCallbackExecutor");
 }
 
-static bool streamsSet(const std::map<std::string, std::string>& config) {
-    return config.count(PluginConfigParams::KEY_CPU_THROUGHPUT_STREAMS) ||
+static bool streamsSet(const ov::AnyMap& config) {
+    return config.count(InferenceEngine::PluginConfigParams::KEY_CPU_THROUGHPUT_STREAMS) ||
            config.count(ov::num_streams.name());
 }
 
-void Engine::ApplyPerformanceHints(std::map<std::string, std::string> &config, const std::shared_ptr<ngraph::Function>& ngraphFunc) const {
+void Engine::apply_performance_hints(ov::AnyMap& config, const std::shared_ptr<ov::Model>& model) const {
     auto getNumStreamsLatency = [&]() {
-        return std::pair<std::string, std::string>(CONFIG_VALUE(CPU_THROUGHPUT_NUMA), ov::util::to_string(ov::streams::NUMA));
+        return std::pair<std::string, std::string>(CONFIG_VALUE(CPU_THROUGHPUT_NUMA),
+                                                   ov::util::to_string(ov::streams::NUMA));
     };
 
     auto getNumStreamsThroughput = [&]() {
         const auto isa = dnnl::get_effective_cpu_isa();
         float isaSpecificThreshold = 1.0f;
         switch (isa) {
-        case dnnl::cpu_isa::sse41 :
+        case dnnl::cpu_isa::sse41:
             isaSpecificThreshold = 0.5f;
             break;
         case dnnl::cpu_isa::avx2:
@@ -220,38 +218,38 @@ void Engine::ApplyPerformanceHints(std::map<std::string, std::string> &config, c
             isaSpecificThreshold = 1.0f;
         }
         // the more "capable" the CPU in general, the more streams we may want to keep to keep it utilized
-        const float memThresholdAssumeLimitedForISA = ov::MemBandwidthPressure::LIMITED/isaSpecificThreshold;
+        const float memThresholdAssumeLimitedForISA = ov::MemBandwidthPressure::LIMITED / isaSpecificThreshold;
         const float L2_cache_size = dnnl::utils::get_cache_size(2 /*level*/, true /*per core */);
-        ov::MemBandwidthPressure networkToleranceForLowCache = ov::MemBandwidthPressureTolerance(
-            ngraphFunc,
-            L2_cache_size, memThresholdAssumeLimitedForISA);
-        const auto default_streams = GetNumStreams(engConfig.streamExecutorConfig._threadBindingType,
-                                                   IStreamsExecutor::Config::StreamMode::DEFAULT,
+        ov::MemBandwidthPressure networkToleranceForLowCache =
+            ov::MemBandwidthPressureTolerance(model, L2_cache_size, memThresholdAssumeLimitedForISA);
+        const auto default_streams = get_streams_num(engConfig.streamExecutorConfig._threadBindingType,
+                                                   ov::threading::IStreamsExecutor::Config::StreamMode::DEFAULT,
                                                    engConfig.streamExecutorConfig._enable_hyper_thread);
         auto streams_info = default_streams;
         if (networkToleranceForLowCache.max_mem_tolerance == ov::MemBandwidthPressure::UNKNOWN) {
             if ((networkToleranceForLowCache.ratio_compute_convs == ov::MemBandwidthPressure::ALL) ||
                 (networkToleranceForLowCache.ratio_compute_deconvs == ov::MemBandwidthPressure::ALL)) {
                 // all relevant layers (convs, etc) are compute-limited, the most aggressive val for #streams
-                streams_info = GetNumStreams(engConfig.streamExecutorConfig._threadBindingType,
-                                             IStreamsExecutor::Config::StreamMode::AGGRESSIVE,
+                streams_info = get_streams_num(engConfig.streamExecutorConfig._threadBindingType,
+                                             ov::threading::IStreamsExecutor::Config::StreamMode::AGGRESSIVE,
                                              engConfig.streamExecutorConfig._enable_hyper_thread);
             }  //  otherwise (no recognized layers) falling back to the default value
         } else if (networkToleranceForLowCache.max_mem_tolerance > memThresholdAssumeLimitedForISA) {
             // network is below the ISA-specific threshold
-            streams_info = GetNumStreams(engConfig.streamExecutorConfig._threadBindingType,
-                                         IStreamsExecutor::Config::StreamMode::AGGRESSIVE,
+            streams_info = get_streams_num(engConfig.streamExecutorConfig._threadBindingType,
+                                         ov::threading::IStreamsExecutor::Config::StreamMode::AGGRESSIVE,
                                          engConfig.streamExecutorConfig._enable_hyper_thread);
         } else if (networkToleranceForLowCache.max_mem_tolerance > ov::MemBandwidthPressure::LIMITED) {
             // network is below general threshold
-            streams_info = GetNumStreams(engConfig.streamExecutorConfig._threadBindingType,
-                                         IStreamsExecutor::Config::StreamMode::LESSAGGRESSIVE,
+            streams_info = get_streams_num(engConfig.streamExecutorConfig._threadBindingType,
+                                         ov::threading::IStreamsExecutor::Config::StreamMode::LESSAGGRESSIVE,
                                          engConfig.streamExecutorConfig._enable_hyper_thread);
             streams_info.num_streams = std::max(default_streams.num_streams, streams_info.num_streams);
         }
-        auto num_requests = config.find(CONFIG_KEY(PERFORMANCE_HINT_NUM_REQUESTS));
+
+        auto num_requests = config.find(ov::hint::num_requests.name());
         if (num_requests != config.end()) {  // arrived with config to the LoadNetwork (and thus higher pri)
-            auto val = PerfHintsConfig::CheckPerformanceHintRequestValue(num_requests->second);
+            auto val = InferenceEngine::PerfHintsConfig::CheckPerformanceHintRequestValue(num_requests->second.as<std::string>());
             if (val > 0)
                 streams_info.num_streams = std::min(streams_info.num_streams, val);
         } else if (engConfig.perfHintsConfig.ovPerfHintNumRequests) {  // set thru SetConfig to the plugin, 2nd priority
@@ -261,22 +259,24 @@ void Engine::ApplyPerformanceHints(std::map<std::string, std::string> &config, c
         return std::pair<std::string, StreamCfg>(std::to_string(streams_info.num_streams), streams_info);
     };
 
+    OPENVINO_SUPPRESS_DEPRECATED_START
     auto getPerfHintName = [&]() {
         const bool streamsExplicitlySetForModel = streamsSet(config);
-        // checking streams (to avoid overriding what user might explicitly set in the incoming config or previously via SetConfig)
-        if (streamsExplicitlySetForModel ||
-            streamsExplicitlySetForEngine)
+        // checking streams (to avoid overriding what user might explicitly set in the incoming config or previously via
+        // SetConfig)
+        if (streamsExplicitlySetForModel || streamsExplicitlySetForEngine)
             return std::string();
 
-        const auto& perf_hint = config.find(CONFIG_KEY(PERFORMANCE_HINT));
+        const auto& perf_hint = config.find(ov::hint::performance_mode.name());
         // the perf_hint may have just arrived to the LoadNetwork, or was set with the plugin's SetConfig
         if (perf_hint == config.end() && engConfig.perfHintsConfig.ovPerfHint.empty())
             return std::string();
         /* performance hints set for network has higher pririty than engine ones.
-        * This applies for all the configuration parameters */
-        const auto perf_hint_name = (perf_hint != config.end()) ?
-            PerfHintsConfig::CheckPerformanceHintValue(perf_hint->second) :
-            engConfig.perfHintsConfig.ovPerfHint;
+         * This applies for all the configuration parameters */
+        const auto perf_hint_name =
+            (perf_hint != config.end())
+                ? InferenceEngine::PerfHintsConfig::CheckPerformanceHintValue(perf_hint->second.as<std::string>())
+                : engConfig.perfHintsConfig.ovPerfHint;
         return perf_hint_name;
     };
 
@@ -291,7 +291,7 @@ void Engine::ApplyPerformanceHints(std::map<std::string, std::string> &config, c
     const auto tput_name = std::string(CONFIG_VALUE(THROUGHPUT)) + "_" + std::string(ov::num_streams.name());
     hints_props.insert({latency_name, latency_hints.second});
     hints_props.insert({tput_name, std::to_string(tput_hints.second.num_streams)});
-    ngraphFunc->set_rt_info(hints_props, "intel_cpu_hints_config");
+    model->set_rt_info(hints_props, "intel_cpu_hints_config");
 
     const auto perf_hint_name = getPerfHintName();
     if (perf_hint_name == CONFIG_VALUE(LATENCY)) {
@@ -308,13 +308,14 @@ void Engine::ApplyPerformanceHints(std::map<std::string, std::string> &config, c
             std::to_string(tput_hints.second.threads_per_stream_small);
         config[CONFIG_KEY_INTERNAL(SMALL_CORE_OFFSET)] = std::to_string(tput_hints.second.small_core_offset);
     }
+    OPENVINO_SUPPRESS_DEPRECATED_END
 }
 
-void Engine::GetPerformanceStreams(Config& config, const std::shared_ptr<ngraph::Function>& ngraphFunc) {
+void Engine::get_performance_streams(Config& config, const std::shared_ptr<ov::Model>& model) const{
     const auto perf_hint_name = config.perfHintsConfig.ovPerfHint;
     const int latency_streams = get_default_latency_streams(config.latencyThreadingMode);
     int streams;
-
+    OPENVINO_SUPPRESS_DEPRECATED_START
     if (config.streamExecutorConfig._streams_changed) {
         streams = config.streamExecutorConfig._streams;
     } else if (perf_hint_name == CONFIG_VALUE(LATENCY)) {
@@ -326,25 +327,26 @@ void Engine::GetPerformanceStreams(Config& config, const std::shared_ptr<ngraph:
     }
 
     if (!((0 == config.streamExecutorConfig._streams) && config.streamExecutorConfig._streams_changed)) {
-        get_num_streams(streams, ngraphFunc, config);
+        get_num_streams(streams, model, config);
     }
 
     config._config[CONFIG_KEY(CPU_THROUGHPUT_STREAMS)] = std::to_string(config.streamExecutorConfig._streams);
+    OPENVINO_SUPPRESS_DEPRECATED_END
 }
 
-void Engine::CalculateStreams(Config& conf, const std::shared_ptr<ngraph::Function>& function, bool imported) {
+void Engine::calculate_streams(Config& conf, const std::shared_ptr<ov::Model>& model, bool imported) const{
     // import config props from caching model
     if (imported && !is_cpu_map_available()) {
-        if (function->has_rt_info("intel_cpu_hints_config") && !conf.perfHintsConfig.ovPerfHint.empty()) {
+        if (model->has_rt_info("intel_cpu_hints_config") && !conf.perfHintsConfig.ovPerfHint.empty()) {
             const auto mode_name = conf.perfHintsConfig.ovPerfHint;
             if (mode_name == CONFIG_VALUE(LATENCY) || mode_name == CONFIG_VALUE(THROUGHPUT)) {
-                const auto& hints_config = function->get_rt_info<ov::AnyMap>("intel_cpu_hints_config");
+                const auto& hints_config = model->get_rt_info<ov::AnyMap>("intel_cpu_hints_config");
                 const auto hints_param_name = mode_name + "_" + std::string(ov::num_streams.name());
                 const auto it = hints_config.find(hints_param_name);
                 if (it != hints_config.end()) {
                     conf.readProperties({{std::string(ov::num_streams.name()), it->second.as<std::string>()}});
                 } else {
-                    IE_THROW() << "Cache file doesn't contain precalculated number of streams for mode " << mode_name;
+                    OPENVINO_THROW("Cache file doesn't contain precalculated number of streams for mode ", mode_name);
                 }
             }
         }
@@ -352,10 +354,10 @@ void Engine::CalculateStreams(Config& conf, const std::shared_ptr<ngraph::Functi
 
     if (is_cpu_map_available()) {
         const auto model_prefer_name = std::string("MODEL_PREFER_THREADS");
-        if (imported && function->has_rt_info("intel_cpu_hints_config")) {
+        if (imported && model->has_rt_info("intel_cpu_hints_config")) {
             // load model_prefer_threads from cache
             int cache_model_prefer;
-            const auto& hints_config = function->get_rt_info<ov::AnyMap>("intel_cpu_hints_config");
+            const auto& hints_config = model->get_rt_info<ov::AnyMap>("intel_cpu_hints_config");
             const auto it_model_prefer = hints_config.find(model_prefer_name);
             if (it_model_prefer != hints_config.end()) {
                 try {
@@ -367,27 +369,27 @@ void Engine::CalculateStreams(Config& conf, const std::shared_ptr<ngraph::Functi
                 conf.modelPreferThreads = cache_model_prefer;
             }
         }
-        GetPerformanceStreams(conf, function);
+        get_performance_streams(conf, model);
         // save model_prefer_threads to model rt_info when loading network
         if (!imported) {
             ov::AnyMap hints_props;
             hints_props.insert({model_prefer_name, std::to_string(conf.modelPreferThreads)});
-            function->set_rt_info(hints_props, "intel_cpu_hints_config");
+            model->set_rt_info(hints_props, "intel_cpu_hints_config");
         }
     }
 }
 
-StreamCfg Engine::GetNumStreams(InferenceEngine::IStreamsExecutor::ThreadBindingType thread_binding_type,
+StreamCfg Engine::get_streams_num(ov::threading::IStreamsExecutor::ThreadBindingType thread_binding_type,
                                         int stream_mode,
                                         const bool enable_hyper_thread) const {
-    const int sockets = static_cast<int>(getAvailableNUMANodes().size());
+    const int sockets = static_cast<int>(get_available_numa_nodes().size());
     const int num_cores =
         thread_binding_type == IStreamsExecutor::ThreadBindingType::HYBRID_AWARE
             ? parallel_get_max_threads()
-            : (sockets == 1 ? (enable_hyper_thread ? parallel_get_max_threads() : getNumberOfCPUCores())
-                            : getNumberOfCPUCores());
-    const int num_cores_phy = getNumberOfCPUCores();
-    const int num_big_cores_phy = getNumberOfCPUCores(true);
+            : (sockets == 1 ? (enable_hyper_thread ? parallel_get_max_threads() :  get_number_of_cpu_cores())
+                            : get_number_of_cpu_cores());
+    const int num_cores_phy = get_number_of_cpu_cores();
+    const int num_big_cores_phy = get_number_of_cpu_cores(true);
     const int num_small_cores = num_cores_phy - num_big_cores_phy;
     const int num_big_cores = num_cores > num_cores_phy ? num_big_cores_phy * 2 : num_big_cores_phy;
     StreamCfg stream_cfg = {0};
@@ -450,7 +452,7 @@ StreamCfg Engine::GetNumStreams(InferenceEngine::IStreamsExecutor::ThreadBinding
             stream_cfg.num_streams = num_cores_phy / 2;
         }
     } else {
-        IE_THROW() << "Wrong stream mode to get num of streams: " << stream_mode;
+        OPENVINO_THROW("Wrong stream mode to get num of streams: ", stream_mode);
     }
     stream_cfg.num_streams = stream_cfg.num_streams > 0
                                  ? stream_cfg.num_streams
@@ -459,21 +461,21 @@ StreamCfg Engine::GetNumStreams(InferenceEngine::IStreamsExecutor::ThreadBinding
     return stream_cfg;
 }
 
-static bool shouldEnableLPT(const std::map<std::string, std::string>& modelConfig, const Config& engineConfig) {
+static bool shouldEnableLPT(const ov::AnyMap& modelConfig, const Config& engineConfig) {
     const auto& enableLPT = modelConfig.find(InferenceEngine::PluginConfigInternalParams::KEY_LP_TRANSFORMS_MODE);
     if (enableLPT == modelConfig.end()) // model config has higher priority
         return engineConfig.lpTransformsMode == Config::LPTransformsMode::On;
 
-    const auto& val = enableLPT->second;
-    if (val == PluginConfigParams::YES)
+    const auto& val = enableLPT->second.as<std::string>();
+    if (val == InferenceEngine::PluginConfigParams::YES)
         return true;
-    else if (val == PluginConfigParams::NO)
+    else if (val == InferenceEngine::PluginConfigParams::NO)
         return false;
     else
-        IE_THROW() << "Wrong value for property key LP_TRANSFORMS_MODE. Expected values: YES/NO";
+        OPENVINO_THROW("Wrong value for property key LP_TRANSFORMS_MODE. Expected values: YES/NO");
 }
 
-static ov::element::Type getInferencePrecision(const std::map<std::string, std::string>& modelConfig,
+static ov::element::Type getInferencePrecision(const ov::AnyMap& modelConfig,
                                                const Config& engineConfig,
                                                Config::ModelType modelType) {
     Config tempConf = engineConfig;
@@ -487,42 +489,43 @@ static Config::ModelType getModelType(const std::shared_ptr<const Model>& model)
            Config::ModelType::CNN : Config::ModelType::Unknown;
 }
 
-static Config::SnippetsMode getSnippetsMode(const std::map<std::string, std::string>& modelConfig, const Config& engineConfig) {
+static Config::SnippetsMode getSnippetsMode(const ov::AnyMap& modelConfig, const Config& engineConfig) {
     const auto& snippetsMode = modelConfig.find(InferenceEngine::PluginConfigInternalParams::KEY_SNIPPETS_MODE);
-    if (snippetsMode == modelConfig.end()) // not set explicitly
-        return Config::SnippetsMode::Enable; // enable by default
+    if (snippetsMode == modelConfig.end())    // not set explicitly
+        return Config::SnippetsMode::Enable;  // enable by default
 
-    const auto& val = snippetsMode->second;
-    if (val == PluginConfigInternalParams::IGNORE_CALLBACK)
+    const auto& val = snippetsMode->second.as<std::string>();
+    if (val == InferenceEngine::PluginConfigInternalParams::IGNORE_CALLBACK)
         return Config::SnippetsMode::IgnoreCallback;
-    else if (val == PluginConfigInternalParams::DISABLE)
+    else if (val == InferenceEngine::PluginConfigInternalParams::DISABLE)
         return Config::SnippetsMode::Disable;
-    else if (val == PluginConfigInternalParams::ENABLE)
+    else if (val == InferenceEngine::PluginConfigInternalParams::ENABLE)
         return Config::SnippetsMode::Enable;
     else
-        IE_THROW() << "Wrong value for property key SNIPPETS_MODE. Expected values: ENABLE/DISABLE/IGNORE_CALLBACK";
+        OPENVINO_THROW("Wrong value for property key SNIPPETS_MODE. Expected values: ENABLE/DISABLE/IGNORE_CALLBACK");
 }
 
-InferenceEngine::IExecutableNetworkInternal::Ptr
-Engine::LoadExeNetworkImpl(const InferenceEngine::CNNNetwork &network, const std::map<std::string, std::string> &orig_config) {
-    OV_ITT_SCOPED_TASK(itt::domains::intel_cpu, "Engine::LoadExeNetworkImpl");
+std::shared_ptr<ov::ICompiledModel>
+Engine::compile_model(const std::shared_ptr<const ov::Model>& model, const ov::AnyMap& orig_config) const{
+    OV_ITT_SCOPED_TASK(itt::domains::intel_cpu, "Engine::compile_model");
     CREATE_DEBUG_TIMER(debugLoadTimer);
 
     // verification of supported input
-    for (const auto &ii : network.getInputsInfo()) {
-        auto input_precision = ii.second->getPrecision();
-
-        using hash_t = std::hash<typename std::underlying_type<Precision::ePrecision>::type>;
-
-        static const std::unordered_set<Precision::ePrecision, hash_t> supported_precisions = {
-            Precision::U8,   Precision::I8,
-            Precision::U16,  Precision::I16,
-            Precision::U32,  Precision::I32,
-            Precision::U64,  Precision::I64,
-            Precision::BF16, Precision::FP16,
-            Precision::FP32, Precision::FP64,
-            Precision::BOOL
-        };
+    for (const auto &ii : model->inputs()) {
+        auto input_precision = ii.get_element_type();
+        static const std::set<ov::element::Type_t> supported_precisions = {ov::element::Type_t::u8,
+                                                                           ov::element::Type_t::i8,
+                                                                           ov::element::Type_t::u16,
+                                                                           ov::element::Type_t::i16,
+                                                                           ov::element::Type_t::u32,
+                                                                           ov::element::Type_t::i32,
+                                                                           ov::element::Type_t::u64,
+                                                                           ov::element::Type_t::i64,
+                                                                           ov::element::Type_t::bf16,
+                                                                           ov::element::Type_t::f16,
+                                                                           ov::element::Type_t::f32,
+                                                                           ov::element::Type_t::f64,
+                                                                           ov::element::Type_t::boolean};
 
         if (!supported_precisions.count(input_precision)) {
             IE_CPU_PLUGIN_THROW(NotImplemented)
@@ -531,46 +534,61 @@ Engine::LoadExeNetworkImpl(const InferenceEngine::CNNNetwork &network, const std
     }
 
     auto config = orig_config;
-
-    CNNNetwork clonedNetwork = InferenceEngine::details::cloneNetwork(network);
+    const std::shared_ptr<ov::Model> cloned_model = model->clone();
     const bool enableLPT = shouldEnableLPT(config, engConfig);
-    auto nGraphFunc = clonedNetwork.getFunction();
-    Config::ModelType modelType = getModelType(nGraphFunc);
+    Config::ModelType modelType = getModelType(model);
     ov::element::Type inferencePrecision = getInferencePrecision(config, engConfig, modelType);
-    const Config::SnippetsMode snippetsMode = getSnippetsMode(config, engConfig); 
-
-    DEBUG_LOG(PrintableModel(*nGraphFunc, "org_"));
+    const Config::SnippetsMode snippetsMode = getSnippetsMode(config, engConfig);
+    DEBUG_LOG(PrintableModel(*cloned_model, "org_"));
 
     // update the props after the perf mode translated to configs
     // TODO: Clarify the behavior of SetConfig method. Skip eng_config or not?
     Config conf = engConfig;
 
-    Transformations transformations(nGraphFunc, enableLPT, inferencePrecision, isLegacyAPI(), snippetsMode, conf);
+    Transformations transformations(cloned_model, enableLPT, inferencePrecision, is_legacy_api(), snippetsMode, conf);
     transformations.UpToLpt();
 
     if (!is_cpu_map_available()) {
-        ApplyPerformanceHints(config, nGraphFunc);
+        apply_performance_hints(config, cloned_model);
     }
 
     conf.readProperties(config, modelType);
-    CalculateStreams(conf, nGraphFunc);
+    calculate_streams(conf, cloned_model);
 
     transformations.PostLpt();
     transformations.Snippets();
 
     // need to check that all outputs have static shapes
     // checking that all inputs have static shapes is performed in the common part
-    if (isLegacyAPI()) {
-        for (const auto& res : nGraphFunc->get_results()) {
+    if (is_legacy_api()) {
+        for (const auto& res : cloned_model->get_results()) {
             if (res->get_input_partial_shape(0).is_dynamic()) {
-                IE_THROW() << "CPU plug-in can't load a model with dynamic output shapes via legacy API.";
+                OPENVINO_THROW("CPU plug-in can't load a model with dynamic output shapes via legacy API.");
             }
         }
     }
 
     transformations.CpuSpecificOpSet();
+    DEBUG_LOG(PrintableModel(*cloned_model, "cpu_"));
 
-    DEBUG_LOG(PrintableModel(*nGraphFunc, "cpu_"));
+    if ((cloned_model->inputs().size() != model->inputs().size()) ||
+        (cloned_model->outputs().size() != model->outputs().size())) {
+        OPENVINO_THROW("Input/output ports count mismatch between the original model and after the transformation! "
+                       "Original model inputs count: ",
+                       model->inputs().size(),
+                       " after the transformations ",
+                       cloned_model->inputs().size(),
+                       ". Original model outputs count:",
+                       model->inputs().size(),
+                       " after the transformations ",
+                       cloned_model->outputs().size());
+    }
+    // Make output ports have the same tensor names with original model
+    for (size_t idx = 0; idx < cloned_model->outputs().size(); idx++) {
+        auto new_result = cloned_model->output(idx);
+        auto orig_result = model->output(idx);
+        new_result.get_tensor().set_names(orig_result.get_tensor().get_names());
+    }
 
     // SSE runtime check is needed for some ATOM machine, which is x86-64 but w/o SSE
     static Xbyak::util::Cpu cpu;
@@ -583,52 +601,53 @@ Engine::LoadExeNetworkImpl(const InferenceEngine::CNNNetwork &network, const std
             denormals_as_zero(false);
         }
     }
-
-    return std::make_shared<ExecNetwork>(clonedNetwork, conf, extensionManager, shared_from_this());
+    return std::make_shared<CompiledModel>(cloned_model, shared_from_this(), conf, extensionManager);
 }
 
-void Engine::SetConfig(const std::map<std::string, std::string> &config) {
+void Engine::set_property(const ov::AnyMap &config) {
     // @todo after Legacy configuration is dropped, use some wrapper class to keep both the property and "ifSetExplicitly" flag
     streamsExplicitlySetForEngine = streamsSet(config);
 
     engConfig.readProperties(config);
 }
 
-bool Engine::isLegacyAPI() const {
-    return !IsNewAPI();
+bool Engine::is_legacy_api() const {
+    return !get_core()->is_new_api();
 }
 
-Parameter Engine::GetConfigLegacy(const std::string& name, const std::map<std::string, Parameter>& options) const {
-    Parameter result;
+ov::Any Engine::get_property_legacy(const std::string& name, const ov::AnyMap& options) const {
+    ov::Any result;
     auto option = engConfig._config.find(name);
     if (option != engConfig._config.end()) {
         result = option->second;
     } else {
-        IE_CPU_PLUGIN_THROW() << ". Unsupported config parameter: " << name;
+        return get_metric_legacy(name, options);
     }
     return result;
 }
 
-Parameter Engine::GetConfig(const std::string& name, const std::map<std::string, Parameter>& options) const {
-    if (isLegacyAPI())
-        return GetConfigLegacy(name, options);
+ov::Any Engine::get_property(const std::string& name, const ov::AnyMap& options) const {
+    if (is_legacy_api())
+        return get_property_legacy(name, options);
 
     if (name == ov::optimal_number_of_infer_requests) {
         const auto streams = engConfig.streamExecutorConfig._streams;
-        return decltype(ov::optimal_number_of_infer_requests)::value_type(streams); // ov::optimal_number_of_infer_requests has no negative values
+        return decltype(ov::optimal_number_of_infer_requests)::value_type(
+            streams);  // ov::optimal_number_of_infer_requests has no negative values
     } else if (name == ov::num_streams) {
         const auto streams = engConfig.streamExecutorConfig._streams;
-        return decltype(ov::num_streams)::value_type(streams); // ov::num_streams has special negative values (AUTO = -1, NUMA = -2)
+        return decltype(ov::num_streams)::value_type(
+            streams);  // ov::num_streams has special negative values (AUTO = -1, NUMA = -2)
     } else if (name == ov::affinity) {
         const auto affinity = engConfig.streamExecutorConfig._threadBindingType;
         switch (affinity) {
-        case InferenceEngine::IStreamsExecutor::ThreadBindingType::NONE:
+        case IStreamsExecutor::ThreadBindingType::NONE:
             return ov::Affinity::NONE;
-        case InferenceEngine::IStreamsExecutor::ThreadBindingType::CORES:
+        case IStreamsExecutor::ThreadBindingType::CORES:
             return ov::Affinity::CORE;
-        case InferenceEngine::IStreamsExecutor::ThreadBindingType::NUMA:
+        case IStreamsExecutor::ThreadBindingType::NUMA:
             return ov::Affinity::NUMA;
-        case InferenceEngine::IStreamsExecutor::ThreadBindingType::HYBRID_AWARE:
+        case IStreamsExecutor::ThreadBindingType::HYBRID_AWARE:
             return ov::Affinity::HYBRID_AWARE;
         }
         return ov::Affinity::NONE;
@@ -660,12 +679,11 @@ Parameter Engine::GetConfig(const std::string& name, const std::map<std::string,
     } else if (name == ov::hint::execution_mode) {
         return engConfig.executionMode;
     }
-    /* Internally legacy parameters are used with new API as part of migration procedure.
-     * This fallback can be removed as soon as migration completed */
-    return GetConfigLegacy(name, options);
+    return get_ro_property(name, options);
 }
 
-Parameter Engine::GetMetricLegacy(const std::string& name, const std::map<std::string, Parameter>& options) const {
+ov::Any Engine::get_metric_legacy(const std::string& name, const ov::AnyMap& options) const {
+    OPENVINO_SUPPRESS_DEPRECATED_START
     if (name == METRIC_KEY(SUPPORTED_METRICS)) {
         std::vector<std::string> metrics = {
             METRIC_KEY(AVAILABLE_DEVICES),
@@ -717,11 +735,12 @@ Parameter Engine::GetMetricLegacy(const std::string& name, const std::map<std::s
     }
 
     IE_CPU_PLUGIN_THROW() << "Unsupported metric key: " << name;
+    OPENVINO_SUPPRESS_DEPRECATED_END
 }
 
-Parameter Engine::GetMetric(const std::string& name, const std::map<std::string, Parameter>& options) const {
-    if (isLegacyAPI())
-        return GetMetricLegacy(name, options);
+ov::Any Engine::get_ro_property(const std::string& name, const ov::AnyMap& options) const {
+    if (is_legacy_api())
+        return get_metric_legacy(name, options);
 
     auto RO_property = [](const std::string& propertyName) {
         return ov::PropertyName(propertyName, ov::PropertyMutability::RO);
@@ -798,19 +817,14 @@ Parameter Engine::GetMetric(const std::string& name, const std::map<std::string,
     }
     /* Internally legacy parameters are used with new API as part of migration procedure.
      * This fallback can be removed as soon as migration completed */
-    return GetMetricLegacy(name, options);
+    return get_metric_legacy(name, options);
 }
 
-void Engine::AddExtension(const InferenceEngine::IExtensionPtr& extension) {
-    extensionManager->AddExtension(extension);
-}
-
-QueryNetworkResult Engine::QueryNetwork(const CNNNetwork& network, const std::map<std::string, std::string>& config) const {
+ov::SupportedOpsMap Engine::query_model(const std::shared_ptr<const ov::Model>& model, const ov::AnyMap& config) const {
     WeightsSharing::Ptr fake_w_cache;
 
-    auto model = network.getFunction();
     if (model == nullptr) {
-        IE_THROW() << "Only ngraph-based models are supported!";
+        OPENVINO_THROW("Only ngraph-based models are supported!");
     }
 
     Config conf = engConfig;
@@ -818,65 +832,68 @@ QueryNetworkResult Engine::QueryNetwork(const CNNNetwork& network, const std::ma
     conf.readProperties(config, modelType);
 
     const auto& lptProp = config.find(InferenceEngine::PluginConfigInternalParams::KEY_LP_TRANSFORMS_MODE);
-    const bool enableLPT = (lptProp != config.end() && lptProp->second == PluginConfigParams::YES) /* enabled in the orig_config*/
-                        || Config::LPTransformsMode::On == engConfig.lpTransformsMode /* or already enabled */;
+    const bool enableLPT =
+        (lptProp != config.end() &&
+         lptProp->second.as<std::string>() == InferenceEngine::PluginConfigParams::YES) /* enabled in the orig_config*/
+        || Config::LPTransformsMode::On == engConfig.lpTransformsMode /* or already enabled */;
     const Config::SnippetsMode snippetsMode = getSnippetsMode(config, conf);
 
     auto context =
-        std::make_shared<GraphContext>(conf, extensionManager, fake_w_cache, false);
+        std::make_shared<GraphContext>(conf, nullptr, fake_w_cache, false);
 
-    auto supported = GetSupportedNodes(model,
-                                       [&](std::shared_ptr<ov::Model>& model) {
-                                           Transformations transformation(model, enableLPT, conf.inferencePrecision, isLegacyAPI(), snippetsMode, engConfig);
-                                           transformation.UpToLpt();
-                                           transformation.PostLpt();
-                                           transformation.Snippets();
-                                           transformation.CpuSpecificOpSet();
-                                       },
-                                       [&](const std::shared_ptr<ngraph::Node>& op) {
-                                           std::unique_ptr<Node> ptr;
-                                           try {
-                                               ptr.reset(Node::factory().create(op, context));
-                                           } catch (const InferenceEngine::Exception&) {
-                                               return false;
-                                           }
-                                           return true;
-                                       });
+    auto supported = ov::get_supported_nodes(
+        model,
+        [&](std::shared_ptr<ov::Model>& model) {
+            Transformations transformation(model,
+                                           enableLPT,
+                                           conf.inferencePrecision,
+                                           is_legacy_api(),
+                                           snippetsMode,
+                                           engConfig);
+            transformation.UpToLpt();
+            transformation.PostLpt();
+            transformation.Snippets();
+            transformation.CpuSpecificOpSet();
+        },
+        [&](const std::shared_ptr<ov::Node>& op) {
+            std::unique_ptr<Node> ptr;
+            try {
+                ptr.reset(Node::factory().create(op, context));
+            } catch (const InferenceEngine::Exception&) {
+                return false;
+            }
+            return true;
+        });
 
-    QueryNetworkResult res;
+    ov::SupportedOpsMap res;
     for (auto&& layerName : supported) {
-        res.supportedLayersMap.emplace(layerName, GetName());
+        res.emplace(layerName, get_device_name());
     }
 
     return res;
 }
 
-InferenceEngine::IExecutableNetworkInternal::Ptr Engine::ImportNetwork(std::istream& networkModel,
-                                            const std::map<std::string, std::string>& config) {
-    OV_ITT_SCOPE(FIRST_INFERENCE, itt::domains::intel_cpu_LT, "ImportNetwork");
+std::shared_ptr<ov::ICompiledModel> Engine::import_model(std::istream& networkModel,
+                                            const ov::AnyMap& config) const{
+    OV_ITT_SCOPE(FIRST_INFERENCE, itt::domains::intel_cpu_LT, "import_model");
 
-    CNNNetworkDeserializer deserializer(networkModel,
-        [this](const std::string& model, const Blob::CPtr& weights) {
-            return GetCore()->ReadNetwork(model, weights, true);
+    ModelDeserializer deserializer(networkModel,
+        [this](const std::string& model, const ov::Tensor& weights) {
+            return get_core()->read_model(model, weights, true);
         });
 
-    CNNNetwork cnnnetwork;
-    deserializer >> cnnnetwork;
+    std::shared_ptr<ov::Model> model;
+    deserializer >> model;
 
-    auto function = cnnnetwork.getFunction();
-    Config::ModelType modelType = getModelType(function);
     Config conf = engConfig;
+    Config::ModelType modelType = getModelType(model);
     conf.readProperties(config, modelType);
 
-    CalculateStreams(conf, function, true);
+    // import config props from caching model
+    calculate_streams(conf, model, true);
 
-    auto execNetwork = std::make_shared<ExecNetwork>(cnnnetwork, conf, extensionManager, shared_from_this());
-
-    execNetwork->setNetworkInputs(cnnnetwork.getInputsInfo());
-    execNetwork->setNetworkOutputs(cnnnetwork.getOutputsInfo());
-    SetExeNetworkInfo(execNetwork, cnnnetwork.getFunction());
-
-    return execNetwork;
+    auto compiled_model = std::make_shared<CompiledModel>(model, shared_from_this(), conf, extensionManager, true);
+    return compiled_model;
 }
 }   // namespace intel_cpu
 }   // namespace ov
@@ -884,13 +901,13 @@ InferenceEngine::IExecutableNetworkInternal::Ptr Engine::ImportNetwork(std::istr
 using namespace ov::intel_cpu;
 
 #if defined(OPENVINO_ARCH_ARM) || defined(OPENVINO_ARCH_ARM64)
-static const Version version = {{2, 1}, CI_BUILD_NUMBER, "openvino_arm_cpu_plugin"};
+static const ov::Version version = {CI_BUILD_NUMBER, "openvino_arm_cpu_plugin"};
 #elif defined(OPENVINO_ARCH_X86) || defined(OPENVINO_ARCH_X86_64)
-static const Version version = {{2, 1}, CI_BUILD_NUMBER, "openvino_intel_cpu_plugin"};
+static const ov::Version version = {CI_BUILD_NUMBER, "openvino_intel_cpu_plugin"};
 #elif defined(OPENVINO_ARCH_RISCV64)
-static const Version version = {{2, 1}, CI_BUILD_NUMBER, "openvino_riscv_cpu_plugin"};
+static const ov::Version version = {CI_BUILD_NUMBER, "openvino_riscv_cpu_plugin"};
 #else
 #error "Undefined system processor"
 #endif
 
-IE_DEFINE_PLUGIN_CREATE_FUNCTION(Engine, version)
+OV_DEFINE_PLUGIN_CREATE_FUNCTION(Engine, version)

src/plugins/intel_cpu/src/plugin.h

@@ -4,56 +4,72 @@
 
 #pragma once
 
-#include "exec_network.h"
+#include "compiled_model.h"
 #include "cpu_streams_calculation.hpp"
 
-#include <string>
-#include <map>
-#include <memory>
-#include <functional>
-
 namespace ov {
 namespace intel_cpu {
 
-class Engine : public InferenceEngine::IInferencePlugin {
+class Engine : public ov::IPlugin {
 public:
     Engine();
     ~Engine();
 
-    std::shared_ptr<InferenceEngine::IExecutableNetworkInternal>
-    LoadExeNetworkImpl(const InferenceEngine::CNNNetwork &network,
-                       const std::map<std::string, std::string> &config) override;
+    std::shared_ptr<ov::ICompiledModel> compile_model(const std::shared_ptr<const ov::Model>& model,
+                                                      const ov::AnyMap& properties) const override;
+    std::shared_ptr<ov::ICompiledModel> compile_model(const std::shared_ptr<const ov::Model>& model,
+                                                      const ov::AnyMap& properties,
+                                                      const ov::SoPtr<ov::IRemoteContext>& context) const override {
+        OPENVINO_ASSERT_HELPER(::ov::NotImplemented,
+                               "",
+                               false,
+                               "Not Implemented",
+                               "compile_model with RemoteContext is not supported by CPU plugin!");
+    };
 
-    void AddExtension(const InferenceEngine::IExtensionPtr& extension) override;
+    void set_property(const ov::AnyMap& properties) override;
+    ov::Any get_property(const std::string& name, const ov::AnyMap& arguments) const override;
+    std::shared_ptr<ov::ICompiledModel> import_model(std::istream& model, const ov::AnyMap& properties) const override;
+    std::shared_ptr<ov::ICompiledModel> import_model(std::istream& model,
+                                                     const ov::SoPtr<ov::IRemoteContext>& context,
+                                                     const ov::AnyMap& properties) const override {
+        OPENVINO_ASSERT_HELPER(::ov::NotImplemented,
+                               "",
+                               false,
+                               "Not Implemented",
+                               "import_model with RemoteContext is not supported by CPU plugin!");
+    };
 
-    void SetConfig(const std::map<std::string, std::string> &config) override;
-
-    InferenceEngine::Parameter GetConfig(const std::string& name, const std::map<std::string, InferenceEngine::Parameter>& options) const override;
-
-    InferenceEngine::Parameter GetMetric(const std::string& name, const std::map<std::string, InferenceEngine::Parameter>& options) const override;
-
-    InferenceEngine::QueryNetworkResult QueryNetwork(const InferenceEngine::CNNNetwork& network,
-                                                     const std::map<std::string, std::string>& config) const override;
-
-    InferenceEngine::IExecutableNetworkInternal::Ptr ImportNetwork(std::istream& networkModel,
-                                                     const std::map<std::string, std::string>& config) override;
+    ov::SupportedOpsMap query_model(const std::shared_ptr<const ov::Model>& model,
+                                    const ov::AnyMap& properties) const override;
+    ov::SoPtr<ov::IRemoteContext> create_context(const ov::AnyMap& remote_properties) const override {
+        OPENVINO_ASSERT_HELPER(::ov::NotImplemented,
+                               "",
+                               false,
+                               "Not Implemented",
+                               "create_context  is not supported by CPU plugin!");
+    };
+    ov::SoPtr<ov::IRemoteContext> get_default_context(const ov::AnyMap& remote_properties) const override {
+        OPENVINO_ASSERT_HELPER(::ov::NotImplemented,
+                               "",
+                               false,
+                               "Not Implemented",
+                               "get_default_context  is not supported by CPU plugin!");
+    };
 
 private:
-    bool isLegacyAPI() const;
+    bool is_legacy_api() const;
 
-    InferenceEngine::Parameter GetMetricLegacy(const std::string& name, const std::map<std::string, InferenceEngine::Parameter>& options) const;
+    ov::Any get_ro_property(const std::string& name, const ov::AnyMap& options) const;
+    ov::Any get_metric_legacy(const std::string& name, const ov::AnyMap& options) const;
 
-    InferenceEngine::Parameter GetConfigLegacy(const std::string& name, const std::map<std::string, InferenceEngine::Parameter>& options) const;
-
-    void ApplyPerformanceHints(std::map<std::string, std::string> &config, const std::shared_ptr<ngraph::Function>& ngraphFunc) const;
-
-    void GetPerformanceStreams(Config &config, const std::shared_ptr<ngraph::Function>& ngraphFunc);
-
-    void CalculateStreams(Config& conf, const std::shared_ptr<ngraph::Function>& ngraphFunc, bool imported = false);
-
-    StreamCfg GetNumStreams(InferenceEngine::IStreamsExecutor::ThreadBindingType thread_binding_type,
-                            int stream_mode,
-                            const bool enable_hyper_thread = true) const;
+    ov::Any get_property_legacy(const std::string& name, const ov::AnyMap& options) const;
+    void apply_performance_hints(ov::AnyMap &config, const std::shared_ptr<ov::Model>& model) const;
+    void get_performance_streams(Config &config, const std::shared_ptr<ov::Model>& model) const;
+    StreamCfg get_streams_num(ov::threading::IStreamsExecutor::ThreadBindingType thread_binding_type,
+                              int stream_mode,
+                              const bool enable_hyper_thread = true) const;
+    void calculate_streams(Config& conf, const std::shared_ptr<ov::Model>& model, bool imported = false) const;
 
     Config engConfig;
     ExtensionManager::Ptr extensionManager = std::make_shared<ExtensionManager>();

src/plugins/intel_cpu/src/serialize.cpp

@@ -3,70 +3,36 @@
 //
 #include "serialize.h"
 
-#include <openvino/pass/serialize.hpp>
-
 #include <pugixml.hpp>
 
+#include "openvino/pass/serialize.hpp"
+#include "transformations/utils/utils.hpp"
+
 using namespace InferenceEngine;
 
 namespace ov {
 namespace intel_cpu {
-namespace {
-    std::string to_string(InferenceEngine::Layout layout) {
-        std::stringstream ss;
-        ss << layout;
-        return ss.str();
+
+static void setInfo(pugi::xml_node& root, std::shared_ptr<ov::Model>& model) {
+    pugi::xml_node outputs = root.child("outputs");
+    auto nodes_it = outputs.children("out").begin();
+    size_t size = model->outputs().size();
+    for (size_t i = 0; i < size; ++nodes_it, i++) {
+        std::string name = nodes_it->attribute("name").value();
+        if (name.empty())
+            continue;
+        auto result = model->output(i).get_node_shared_ptr();
+        ov::descriptor::set_ov_tensor_legacy_name(result->input_value(0).get_tensor(), name);
     }
+}
 
-    InferenceEngine::Layout layout_from_string(const std::string & name) {
-        static const std::unordered_map<std::string, InferenceEngine::Layout> layouts = {
-            { "ANY", InferenceEngine::Layout::ANY },
-            { "NCHW", InferenceEngine::Layout::NCHW },
-            { "NHWC", InferenceEngine::Layout::NHWC },
-            { "NCDHW", InferenceEngine::Layout::NCDHW },
-            { "NDHWC", InferenceEngine::Layout::NDHWC },
-            { "OIHW", InferenceEngine::Layout::OIHW },
-            { "C", InferenceEngine::Layout::C },
-            { "CHW", InferenceEngine::Layout::CHW },
-            { "HWC", InferenceEngine::Layout::HWC },
-            { "HW", InferenceEngine::Layout::HW },
-            { "NC", InferenceEngine::Layout::NC },
-            { "CN", InferenceEngine::Layout::CN },
-            { "BLOCKED", InferenceEngine::Layout::BLOCKED }
-        };
-        auto it = layouts.find(name);
-        if (it != layouts.end()) {
-            return it->second;
-        }
-        IE_THROW(NetworkNotRead) << "Unknown layout with name '" << name << "'";
-    }
-
-    template <typename T>
-    void setInfo(pugi::xml_object_range<pugi::xml_named_node_iterator>&& nodes, T&& info) {
-        auto nodes_it = nodes.begin();
-        auto info_iter = info.begin();
-        for (; nodes_it != nodes.end(); ++nodes_it, ++info_iter) {
-            auto name_attr = nodes_it->attribute("name");
-            auto precision_attr = nodes_it->attribute("precision");
-            auto layout_attr = nodes_it->attribute("layout");
-
-            if (!name_attr || !precision_attr || !layout_attr || info_iter == info.end()) {
-                IE_THROW(NetworkNotRead) << "The inputs/outputs information is invalid.";
-            }
-
-            info_iter->second->setName(name_attr.value());
-            info_iter->second->setPrecision(Precision::FromStr(precision_attr.value()));
-            info_iter->second->setLayout(layout_from_string(layout_attr.value()));
-        }
-    }
-};  // namespace
-
-CNNNetworkSerializer::CNNNetworkSerializer(std::ostream & ostream, ExtensionManager::Ptr extensionManager)
+ModelSerializer::ModelSerializer(std::ostream & ostream, ExtensionManager::Ptr extensionManager)
     : _ostream(ostream)
     , _extensionManager(extensionManager) {
 }
 
-void CNNNetworkSerializer::operator << (const CNNNetwork & network) {
+void ModelSerializer::operator<<(const std::shared_ptr<ov::Model>& model) {
+    OPENVINO_SUPPRESS_DEPRECATED_START
     auto getCustomOpSets = [this]() {
         std::map<std::string, ngraph::OpSet> custom_opsets;
 
@@ -81,75 +47,60 @@ void CNNNetworkSerializer::operator << (const CNNNetwork & network) {
         return custom_opsets;
     };
 
-    auto serializeInputsAndOutputs = [&](std::ostream & stream) {
+    auto serializeInfo = [&](std::ostream& stream) {
         const std::string name = "cnndata";
         pugi::xml_document xml_doc;
         pugi::xml_node root = xml_doc.append_child(name.c_str());
-        pugi::xml_node inputs = root.append_child("inputs");
         pugi::xml_node outputs = root.append_child("outputs");
-
-        for (const auto & in : network.getInputsInfo()) {
-            auto in_node = inputs.append_child("in");
-
-            in_node.append_attribute("name")
-                    .set_value(in.first.c_str());
-            in_node.append_attribute("precision")
-                    .set_value(in.second->getPrecision().name());
-            in_node.append_attribute("layout")
-                    .set_value(to_string(in.second->getLayout()).c_str());
-        }
-
-        for (const auto & out : network.getOutputsInfo()) {
+        for (const auto& out : model->get_results()) {
             auto out_node = outputs.append_child("out");
-            out_node.append_attribute("name")
-                    .set_value(out.first.c_str());
-            out_node.append_attribute("precision")
-                    .set_value(out.second->getPrecision().name());
-            out_node.append_attribute("layout")
-                    .set_value(to_string(out.second->getLayout()).c_str());
+            const std::string name = ov::descriptor::get_ov_tensor_legacy_name(out->input_value(0).get_tensor());
+            out_node.append_attribute("name").set_value(name.c_str());
         }
-
         xml_doc.save(stream);
     };
 
     // Serialize to old representation in case of old API
-    OPENVINO_SUPPRESS_DEPRECATED_START
-    ov::pass::StreamSerialize serializer(_ostream, getCustomOpSets(), serializeInputsAndOutputs);
+    ov::pass::StreamSerialize serializer(_ostream, getCustomOpSets(), serializeInfo);
     OPENVINO_SUPPRESS_DEPRECATED_END
-    serializer.run_on_model(std::const_pointer_cast<ngraph::Function>(network.getFunction()));
+    serializer.run_on_model(std::const_pointer_cast<ov::Model>(model->clone()));
 }
 
-CNNNetworkDeserializer::CNNNetworkDeserializer(std::istream & istream, cnn_network_builder fn)
+ModelDeserializer::ModelDeserializer(std::istream & istream, model_builder fn)
     : _istream(istream)
-    , _cnn_network_builder(fn) {
+    , _model_builder(fn) {
 }
 
-void CNNNetworkDeserializer::operator >> (InferenceEngine::CNNNetwork & network) {
+void ModelDeserializer::operator>>(std::shared_ptr<ov::Model>& model) {
     using namespace ov::pass;
 
-    std::string xmlString, xmlInOutString;
-    InferenceEngine::Blob::Ptr dataBlob;
+    std::string xmlString;
+    ov::Tensor dataBlob;
 
     StreamSerialize::DataHeader hdr = {};
     _istream.read(reinterpret_cast<char*>(&hdr), sizeof hdr);
 
-    // read CNNNetwork input/output precisions
+    // read model input/output precisions
     _istream.seekg(hdr.custom_data_offset);
-    xmlInOutString.resize(hdr.custom_data_size);
-    _istream.read(const_cast<char*>(xmlInOutString.c_str()), hdr.custom_data_size);
+
+    OPENVINO_SUPPRESS_DEPRECATED_START
     pugi::xml_document xmlInOutDoc;
-    auto res = xmlInOutDoc.load_string(xmlInOutString.c_str());
-    if (res.status != pugi::status_ok) {
-        IE_THROW(NetworkNotRead) << "The inputs and outputs information is invalid.";
+    if (hdr.custom_data_size > 0) {
+        std::string xmlInOutString;
+        xmlInOutString.resize(hdr.custom_data_size);
+        _istream.read(const_cast<char*>(xmlInOutString.c_str()), hdr.custom_data_size);
+        auto res = xmlInOutDoc.load_string(xmlInOutString.c_str());
+        if (res.status != pugi::status_ok) {
+            OPENVINO_THROW("NetworkNotRead: The inputs and outputs information is invalid.");
+        }
     }
+    OPENVINO_SUPPRESS_DEPRECATED_END
 
     // read blob content
     _istream.seekg(hdr.consts_offset);
     if (hdr.consts_size) {
-        dataBlob = InferenceEngine::make_shared_blob<std::uint8_t>(
-            InferenceEngine::TensorDesc(InferenceEngine::Precision::U8, {hdr.consts_size}, InferenceEngine::Layout::C));
-        dataBlob->allocate();
-        _istream.read(dataBlob->buffer(), hdr.consts_size);
+        dataBlob = ov::Tensor(ov::element::u8, ov::Shape({hdr.consts_size}));
+        _istream.read(static_cast<char *>(dataBlob.data(ov::element::u8)), hdr.consts_size);
     }
 
     // read XML content
@@ -157,15 +108,11 @@ void CNNNetworkDeserializer::operator >> (InferenceEngine::CNNNetwork & network)
     xmlString.resize(hdr.model_size);
     _istream.read(const_cast<char*>(xmlString.c_str()), hdr.model_size);
 
-    network = _cnn_network_builder(xmlString, std::move(dataBlob));
+    model = _model_builder(xmlString, std::move(dataBlob));
 
-    // Set input and output precisions
+    // Set Info
     pugi::xml_node root = xmlInOutDoc.child("cnndata");
-    pugi::xml_node inputs = root.child("inputs");
-    pugi::xml_node outputs = root.child("outputs");
-
-    setInfo(inputs.children("in"), network.getInputsInfo());
-    setInfo(outputs.children("out"), network.getOutputsInfo());
+    setInfo(root, model);
 }
 
 }   // namespace intel_cpu

src/plugins/intel_cpu/src/serialize.h

@@ -2,40 +2,35 @@
 // SPDX-License-Identifier: Apache-2.0
 //
 #pragma once
-#include "extension_mngr.h"
-
-#include <iostream>
 #include <functional>
-#include <cpp/ie_cnn_network.h>
+#include <iostream>
+
+#include "cpp/ie_cnn_network.h"
+#include "extension_mngr.h"
 
 namespace ov {
 namespace intel_cpu {
 
-class CNNNetworkSerializer {
+class ModelSerializer {
 public:
-    CNNNetworkSerializer(std::ostream & ostream, ExtensionManager::Ptr extensionManager);
-    void operator << (const InferenceEngine::CNNNetwork & network);
+    ModelSerializer(std::ostream& ostream, ExtensionManager::Ptr extensionManager);
+    void operator<<(const std::shared_ptr<ov::Model>& model);
 
 private:
-    std::ostream & _ostream;
+    std::ostream& _ostream;
     ExtensionManager::Ptr _extensionManager;
 };
 
-class CNNNetworkDeserializer {
+class ModelDeserializer {
 public:
-    typedef std::function<
-                InferenceEngine::CNNNetwork(
-                        const std::string&,
-                        const InferenceEngine::Blob::CPtr&)> cnn_network_builder;
-    CNNNetworkDeserializer(std::istream & istream, cnn_network_builder fn);
-    void operator >> (InferenceEngine::CNNNetwork & network);
+    typedef std::function<std::shared_ptr<ov::Model>(const std::string&, const ov::Tensor&)> model_builder;
+    ModelDeserializer(std::istream& istream, model_builder fn);
+    void operator>>(std::shared_ptr<ov::Model>& model);
 
 private:
-    std::istream & _istream;
-    cnn_network_builder _cnn_network_builder;
+    std::istream& _istream;
+    model_builder _model_builder;
 };
 
-// const std::string& model, const Blob::CPtr& weights
-
 }   // namespace intel_cpu
 }   // namespace ov

src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/align_matmul_input_ranks.cpp

@@ -112,14 +112,22 @@ ov::intel_cpu::AlignMatMulInputRanks::AlignMatMulInputRanks() {
             // Insert additional squeeze operation to preserve output shape
             const auto new_out_shape_size = matmul_new->get_output_partial_shape(0).size();
             size_t squeeze_axis = 0;
-            if (input0shape.size() == 1)
-                squeeze_axis = new_out_shape_size - 2;
-            else if (input1shape.size() == 1)
-                squeeze_axis = new_out_shape_size - 1;
-            std::shared_ptr<ngraph::Node> squeeze_output = std::make_shared<ngraph::op::v0::Squeeze>(
-                matmul_new,
-                ngraph::opset1::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {squeeze_axis}));
-
+            std::shared_ptr<ngraph::Node> squeeze_output;
+            // If output data is scalar && new_out_shape is [1 1 .. 1], squeeze all the axis to produce a scalar
+            auto& new_output_partial_shape = matmul_new->get_output_partial_shape(0);
+            const bool can_squeeze_scalar =
+                new_output_partial_shape.is_static() ? ov::shape_size(new_output_partial_shape.to_shape()) == 1 : false;
+            if (ov::is_scalar(output_shape) && can_squeeze_scalar) {
+                squeeze_output = std::make_shared<ngraph::op::v0::Squeeze>(matmul_new);
+            } else {
+                if (input0shape.size() == 1)
+                    squeeze_axis = new_out_shape_size - 2;
+                else if (input1shape.size() == 1)
+                    squeeze_axis = new_out_shape_size - 1;
+                squeeze_output = std::make_shared<ngraph::op::v0::Squeeze>(
+                    matmul_new,
+                    ngraph::opset1::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {squeeze_axis}));
+            }
             new_ops.push_back(squeeze_output);
             matmul_new->set_friendly_name(matmul->get_friendly_name() + "/MM");
             // Set the name of the last node after transformation to initial node name

src/plugins/intel_cpu/src/transformations/cpu_opset/convert_to_cpu_specific_opset.hpp

@@ -48,7 +48,12 @@ inline void ConvertToCPUSpecificOpset(std::shared_ptr<ngraph::Function> &nGraphF
     // after transformation "MoveEltwiseUpThroughDataMov" there can be reshaped sequences that should be eliminated or fused
     CPU_REGISTER_PASS_COMMON(manager, ov::pass::ReshapeSequenceFusion);
     CPU_REGISTER_PASS_COMMON(manager, ov::pass::ConstantFolding);
-    CPU_REGISTER_PASS_COMMON(manager, ov::pass::ConvertPrecision, precisions_map {{ ngraph::element::i64, ngraph::element::i32 }});
+    CPU_REGISTER_PASS_COMMON(manager,
+                             ov::pass::ConvertPrecision,
+                             precisions_map{{ngraph::element::i64, ngraph::element::i32}},
+                             type_to_fuse_map{{}},
+                             false,
+                             false);
     auto symbolic_pipeline = CPU_REGISTER_PASS_COMMON(manager, ov::pass::SymbolicOptimizations, false);
     symbolic_pipeline->get_manager()->register_pass<NgramFusion>();
     CPU_REGISTER_PASS_COMMON(manager, ov::pass::Validate);

src/plugins/intel_cpu/src/transformations/transformation_pipeline.cpp

@@ -130,32 +130,46 @@ namespace intel_cpu {
 
 using const_node_ptr = const std::shared_ptr<const ov::Node>;
 
-bool Transformations::fuse_type_to_convert(const std::shared_ptr<ngraph::Node>& node, const precisions_map& precisions) {
+bool Transformations::fuse_type_to_convert(const std::shared_ptr<ov::Node>& node, const precisions_map& precisions) {
+    auto convert = ov::as_type_ptr<ov::opset10::Convert>(node);
+    if (!convert)
+        return false;
     const auto& from = node->get_output_element_type(0);
     auto it = precisions.find(from);
     if (it == precisions.end())
         return false;
     const auto& to = it->second;
-    if (auto convert = ov::as_type_ptr<ov::opset10::Convert>(node)) {
-        // For Convert node, converting precision from floating point to boolean will lead to mathematical
-        // error, because here the output precision boolean is replaced by u8. E.g. floating point value 0.01
-        // is converted to be 1 for boolean, but 0 for u8. Thus an Abs and Ceil node should be added before the
-        // Convert node for this scenario.
-        if (convert->input(0).get_element_type().is_real() &&
-            convert->get_convert_element_type() == ngraph::element::boolean && to.is_integral_number()) {
+
+    // For Convert node, converting precision from floating point to boolean will lead to mathematical
+    // error, because here the output precision boolean is replaced by u8. E.g. floating point value 0.01
+    // is converted to be 1 for boolean, but 0 for u8. Thus an Abs and Ceil node should be added before the
+    // Convert node for this scenario.
+    if (convert->input(0).get_element_type().is_real() &&
+        convert->get_convert_element_type() == ov::element::boolean && to.is_integral_number()) {
+        const auto& in_prec = node->get_input_element_type(0);
+        auto item = precisions.find(in_prec);
+        if (item != precisions.end()) {
+            // Add convert node for unsupported precision, such as FP64
+            auto pre_convert =
+                std::make_shared<ov::opset10::Convert>(convert->input_value(0).get_node_shared_ptr(), item->second);
+            auto abs = std::make_shared<ov::opset10::Abs>(pre_convert);
+            auto ceil = std::make_shared<ov::opset10::Ceiling>(abs);
+            auto new_convert = std::make_shared<ov::opset10::Convert>(ceil, to);
+            new_convert->set_friendly_name(convert->get_friendly_name());
+            ov::copy_runtime_info(convert, {pre_convert, abs, ceil, new_convert});
+            ov::replace_node(convert, new_convert);
+        } else {
             auto abs = std::make_shared<ov::opset10::Abs>(convert->input_value(0).get_node_shared_ptr());
             auto ceil = std::make_shared<ov::opset10::Ceiling>(abs);
             auto new_convert = std::make_shared<ov::opset10::Convert>(ceil, to);
             new_convert->set_friendly_name(convert->get_friendly_name());
             ov::copy_runtime_info(convert, {abs, ceil, new_convert});
             ov::replace_node(convert, new_convert);
-            return true;
-        } else {
-            convert->set_convert_element_type(to);
-            return true;
         }
+    } else {
+        convert->set_convert_element_type(to);
     }
-    return false;
+    return true;
 }
 
 void Transformations::UpToLpt() {
@@ -258,6 +272,7 @@ void Transformations::PreLpt(const std::vector<ov::element::Type>& defaultPrecis
             {ov::element::i4,      ov::element::i8},
             {ov::element::u4,      ov::element::u8}
         };
+
         // @todo should we always convert to f32 regardless of hardware support, as it is done for f16?
         if (!dnnl::impl::cpu::x64::mayiuse(dnnl::impl::cpu::x64::avx512_core))
             map.insert({ov::element::bf16, ov::element::f32});
@@ -269,20 +284,25 @@ void Transformations::PreLpt(const std::vector<ov::element::Type>& defaultPrecis
 #endif
         return map;
     };
-    static const auto precisions = get_convert_precisions();
+
     type_to_fuse_map type_to_fuse = {{ov::opset10::Convert::get_type_info_static(), fuse_type_to_convert}};
 
 #if defined(OV_CPU_ARM_ENABLE_FP16)
+    // It cannot be static data, because it may be difference for different inferencePrecision
+    const auto precisions = get_convert_precisions();
     if (inferencePrecision == ov::element::f16) {
-        precisions_map fp_convert_precision_map = {
-                {ov::element::f32, ov::element::f16}
-        };
+        precisions_map fp_convert_precision_map = {{ov::element::f32, ov::element::f16}};
         type_to_fuse_map empty_fuse_map = {};
         const bool keep_precision_sensitive_in_fp32 = true;
-        CPU_REGISTER_PASS_COMMON(manager, ov::pass::ConvertPrecision, fp_convert_precision_map,
-                                                                      empty_fuse_map,
-                                                                      keep_precision_sensitive_in_fp32);
+        CPU_REGISTER_PASS_COMMON(manager,
+                                 ov::pass::ConvertPrecision,
+                                 fp_convert_precision_map,
+                                 empty_fuse_map,
+                                 keep_precision_sensitive_in_fp32,
+                                 false);
     }
+#else
+    static const auto precisions = get_convert_precisions();
 #endif
     CPU_REGISTER_PASS_COMMON(manager, ov::pass::KeepConstAndDecompression);
     CPU_SET_CALLBACK_COMMON(manager,
@@ -324,7 +344,8 @@ void Transformations::PreLpt(const std::vector<ov::element::Type>& defaultPrecis
     // However, if the extension operation produces an output precision that is not natively supported, this may lead to inconsistency during
     // element type propagation. This transformation is called before the ConvertPrecision pass to align the actual precisions with the list of supported ones.
     CPU_REGISTER_PASS_COMMON(manager, ov::pass::InsertConvertAfterExtension);
-    CPU_REGISTER_PASS_COMMON(manager, ov::pass::ConvertPrecision, precisions, type_to_fuse);
+    // Precision convert is disabled.
+    CPU_REGISTER_PASS_COMMON(manager, ov::pass::ConvertPrecision, precisions, type_to_fuse, false, false);
 
     CPU_REGISTER_PASS_COMMON(manager, ov::pass::EliminateConvert);
     CPU_REGISTER_PASS_COMMON(manager, SwapConvertTranspose);
@@ -628,7 +649,7 @@ void Transformations::MainSnippets(void) {
     // [122706] Some 3D MHA Patterns have perf regressions when Transpose op is tokenized
     tokenization_config.mha_supported_transpose_ranks = { 4 };
 
-    ngraph::pass::Manager snippetsManager;
+    ov::pass::Manager snippetsManager;
     snippetsManager.set_per_pass_validation(false);
     if (snippetsMode != Config::SnippetsMode::IgnoreCallback)
         CPU_REGISTER_PASS_X64(snippetsManager, SnippetsMarkSkipped, inferencePrecision != ov::element::f32);

src/plugins/intel_cpu/src/transformations/transformation_pipeline.h

@@ -15,8 +15,6 @@
 #include <memory>
 #include <vector>
 
-using namespace InferenceEngine;
-
 #define IE_CPU_PLUGIN_THROW(...) IE_THROW(__VA_ARGS__) << "CPU plugin: "
 
 namespace ov {

src/plugins/intel_cpu/src/utils/blob_dump.cpp

@@ -211,7 +211,32 @@ void BlobDumper::dumpAsTxt(std::ostream &stream) const {
                 stream << static_cast<int>(blob_ptr[desc.getElementOffset(i)]) << std::endl;
             break;
         }
+        case Precision::I64: {
+            auto* blob_ptr = reinterpret_cast<const int64_t*>(ptr);
+            for (size_t i = 0; i < data_size; i++)
+                stream << blob_ptr[desc.getElementOffset(i)] << std::endl;
+            break;
+        }
+        case Precision::U32: {
+            auto* blob_ptr = reinterpret_cast<const uint32_t*>(ptr);
+            for (size_t i = 0; i < data_size; i++)
+                stream << blob_ptr[desc.getElementOffset(i)] << std::endl;
+            break;
+        }
+        case Precision::U16: {
+            auto* blob_ptr = reinterpret_cast<const uint16_t*>(ptr);
+            for (size_t i = 0; i < data_size; i++)
+                stream << blob_ptr[desc.getElementOffset(i)] << std::endl;
+            break;
+        }
+        case Precision::I16: {
+            auto* blob_ptr = reinterpret_cast<const int16_t*>(ptr);
+            for (size_t i = 0; i < data_size; i++)
+                stream << blob_ptr[desc.getElementOffset(i)] << std::endl;
+            break;
+        }
         default:
+            break;
             IE_THROW() << "Dumper. Unsupported precision";
     }
 }

src/plugins/intel_cpu/src/utils/ngraph_utils.hpp

@@ -5,6 +5,8 @@
 #pragma once
 
 #include <cassert>
+#include <transformations/utils/utils.hpp>
+
 #include "transformations/rt_info/primitives_priority_attribute.hpp"
 
 namespace ov {
@@ -47,5 +49,24 @@ inline bool isDynamicNgraphNode(const std::shared_ptr<const ngraph::Node>& op) {
     return ret;
 }
 
+inline std::string get_port_name(const ov::Output<const ov::Node>& port, const bool is_legacy_api) {
+    std::string name;
+    // Should use tensor name as the port name, but many legacy tests still use legacy name
+    // plus sometimes it will get empty tensor name.
+    if (!is_legacy_api) {
+        // TODO: To apply unified tensor name.
+    }
+    if (name.empty()) {
+        bool is_input = ov::op::util::is_parameter(port.get_node());
+        if (is_input) {
+            name = ov::op::util::get_ie_output_name(port);
+        } else {
+            const auto node = port.get_node_shared_ptr();
+            name = ov::op::util::get_ie_output_name(node->input_value(0));
+        }
+    }
+    return name;
+}
+
 }   // namespace intel_cpu
 }   // namespace ov

src/plugins/intel_cpu/tests/functional/shared_tests_instances/skip_tests_config.cpp

@@ -95,12 +95,10 @@ std::vector<std::string> disabledTestPatterns() {
         R"(.*smoke_HeteroOVGetMetricPropsTest.*OVGetMetricPropsTest.*(AVAILABLE_DEVICES|OPTIMIZATION_CAPABILITIES|RANGE_FOR_ASYNC_INFER_REQUESTS|RANGE_FOR_STREAMS).*)",
         // supports only '' as device id
         R"(.*OVClassQueryModelTest.*QueryModelWithDeviceID.*)",
-
         // Issue 67214
         R"(smoke_PrePostProcess.*resize_and_convert_layout_i8.*)",
         // TODO: 67255
         R"(smoke_If.*SimpleIf2OutTest.*)",
-
         // Issue: 69086
         // need to add support convert BIN -> FP32
         // if we set output precision as BIN, when we create output blob precision looks like UNSPECIFIED
@@ -166,23 +164,51 @@ std::vector<std::string> disabledTestPatterns() {
         R"(.*smoke_Snippets_ConvertStub/ConvertStub\.CompareWithRefImpl/IS.*_OT=\(bf16\)_#N=2_#S=2_targetDevice=CPU.*)",
         R"(.*smoke_Snippets_Convert/Convert\.CompareWithRefImpl/IS.*_IT=\(f32\)_OT=\(u8\)_#N=1_#S=1_targetDevice=CPU.*)",
         R"(.*smoke_Snippets_ConvertManyOnInputs/ConvertManyOnInputs\.CompareWithRefImpl/IS.*_IT=\(f32\.u8\)_OT=\(\)_#N=1_#S=1_targetDevice=CPU.*)",
+        // New plugin work with tensors, so it means that blob in old API can have different pointers
+        R"(.*InferRequestIOBBlobTest.*secondCallGetInputDoNotReAllocateData.*)",
+        R"(.*InferRequestIOBBlobTest.*secondCallGetOutputDoNotReAllocateData.*)",
+        R"(.*InferRequestIOBBlobTest.*secondCallGetInputAfterInferSync.*)",
+        R"(.*InferRequestIOBBlobTest.*secondCallGetOutputAfterInferSync.*)",
         // Issue: 106939
         R"(.*ScatterNDUpdateLayerCPUTest.*-1.-1.-1.-2.-2.-2.*)",
         // New plugin API doesn't support changes of pre-processing
-        R"(.*(Hetero).*InferRequestPreprocessTest.*SetPreProcessToInputInfo.*)",
-        R"(.*(Hetero).*InferRequestPreprocessTest.*SetPreProcessToInferRequest.*)",
-        // TODO: for 22.2 (Issue 68949)
+        R"(.*InferRequestPreprocessTest.*SetPreProcessToInputInfo.*)",
+        R"(.*InferRequestPreprocessTest.*SetPreProcessToInferRequest.*)",
+        // Old API cannot deallocate tensor
+        R"(.*InferRequestIOBBlobTest.*canProcessDeallocatedOutputBlobAfterGetAndSetBlob.*)",
+        // Plugin version was changed to ov::Version
+        R"(.*VersionTest.*pluginCurrentVersionIsCorrect.*)",
+        // Issue: 120286
+        R"(.*smoke_Basic/FuseTransposeAndReorderTest.CompareWithRefs.*)",
+        // Issue: 113703, 114763
+        R"(.*smoke_If/SimpleIfTest.*Cond=0.*)",
+        // Issue: 114765
+        R"(.*smoke_PSROIPoolingAverageLayoutTest/PSROIPoolingLayerCPUTest.*BF16.*)",
+        R"(.*smoke_PSROIPoolingBilinearLayoutTest/PSROIPoolingLayerCPUTest.*BF16.*)",
+        // TODO: for 22.2 (CVS-68949)
         R"(.*smoke_AutoBatching_CPU/AutoBatching_Test_DetectionOutput.*)",
+        // Issue: 120279
+        R"(.*OVCompiledGraphImportExportTest.*elementType=(i16|u16|u32|u64|i64).*)",
         // Issue: 120222
         R"(.*smoke_TopK/TopKLayerTest.Inference.*_k=1_axis=3_.*_modelType=f16_trgDev=CPU.*)",
         R"(.*smoke_TopK/TopKLayerTest.Inference.*_k=7_axis=3_.*_modelType=f16_trgDev=CPU.*)",
+        R"(.*smoke_TopK/TopKLayerTest.Inference.*_k=1_axis=1_.*_modelType=f16_trgDev=CPU.*)",
+        R"(.*smoke_TopK/TopKLayerTest.Inference.*_k=7_axis=1_.*_modelType=f16_trgDev=CPU.*)",
         R"(.*smoke_TopK/TopKLayerTest.Inference.*_k=18_.*_modelType=f16_trgDev=CPU.*)",
         R"(.*smoke_TopK/TopKLayerTest.Inference.*_k=21_.*_sort=value_modelType=f16_trgDev=CPU.*)",
         // Issue: 121228
         R"(smoke_TestsDFT_(1|2|3|4)d/DFTLayerTest.Inference.*bf16.*)",
+        // Issue: 121363
+        R"(.*smoke_Constant/ConstantLayerTest.*_dataPRC=(u4|u16|u32|i4|i16|f64).*)",
+        R"(.*smoke_Constant_with_negative_values/ConstantLayerTest.*_dataPRC=(u4|u16|u32|i4|i16|f64).*)",
+        R"(.*smoke_Check/ConstantResultSubgraphTest.CompareWithRefs/SubgraphType.*_IT=(u16|i16|u32|i64|u64).*)",
         // Issue: 121313
         R"(smoke_GroupConvBackpropData.*paddingDefined/GroupConvBackpropLayerTest.Inference.*f16.*)",
         R"(smoke_GroupConvBackpropData.*paddingDefined/GroupConvBackpropLayerTest.Inference.*f32.*)",
+        // Issue: 121812
+        R"(.*ConvertCPULayerTest.*outFmts=(nhwc|nChw8c|nChw16c).*)",
+        // Issue: 122321
+        R"(.*smoke_ConvertCPULayerTest_BOOL.*)",
         // Issue: 122177
         R"(smoke_LSTMSequenceCommon.*LSTMSequenceTest.Inference.*CONVERT_TO_TI.*)",
         // Issue: 122081
@@ -236,6 +262,15 @@ std::vector<std::string> disabledTestPatterns() {
     retVector.emplace_back(R"(smoke_CompareWithRefs_Mvn.*INFERENCE_PRECISION_HINT=f16.*)");
     retVector.emplace_back(R"(smoke_staticShapes4D.*INFERENCE_PRECISION_HINT=f16.*)");
     retVector.emplace_back(R"(smoke_dynamicShapes4D.*INFERENCE_PRECISION_HINT=f16.*)");
+    // Issue: 124309
+    retVector.emplace_back(R"(.*InferRequestPreprocessConversionTest.*oLT=NHWC.*)");
+    retVector.emplace_back(R"(.*smoke_NoReshape/ExecGraphUniqueNodeNames.CheckUniqueNodeNames.*)");
+    retVector.emplace_back(R"(.*smoke_BehaviorTests/InferRequestPerfCountersTest.CheckOperationInPerfMap.*)");
+    retVector.emplace_back(R"(smoke_BehaviorTests/ExecutableNetworkBaseTest.CheckExecGraphInfo.*)");
+    retVector.emplace_back(R"(smoke_BehaviorTests/OVCompiledModelBaseTestOptional.CheckExecGraphInfo.*)");
+    retVector.emplace_back(R"(smoke_ExecGraph/ExecGraphRuntimePrecision.CheckRuntimePrecision/Function=FakeQuantizeBinaryConvolution.*)");
+    // Issue: 124395
+    retVector.emplace_back(R"(smoke_VariableStateBasic/InferRequestVariableStateTest.*)");
 #endif
 
 #endif

src/plugins/intel_cpu/tests/functional/shared_tests_instances/snippets/select.cpp

@@ -3,42 +3,47 @@
 //
 
 #include "snippets/select.hpp"
+
 #include "common_test_utils/test_constants.hpp"
 
 namespace ov {
 namespace test {
 namespace snippets {
 
-
 namespace {
 
 //============================Select=======================================//
 std::vector<ov::test::InputShape> inShapes_a{{{}, {{1, 5, 5, 35}}}};
 std::vector<ov::test::InputShape> inShapes_b{{{}, {{1}}}};
-INSTANTIATE_TEST_SUITE_P(smoke_Snippets_Select, Select,
-        ::testing::Combine(
-                ::testing::ValuesIn(inShapes_a),
-                ::testing::ValuesIn(inShapes_a),
-                ::testing::ValuesIn(inShapes_b),
-                ::testing::ValuesIn({ov::element::f32, ov::element::i8}),
-                ::testing::Values(1),
-                ::testing::Values(1),
-                ::testing::Values(ov::test::utils::DEVICE_CPU)),
-        Select::getTestCaseName);
+INSTANTIATE_TEST_SUITE_P(smoke_Snippets_Select,
+                         Select,
+                         ::testing::Combine(::testing::ValuesIn(inShapes_a),
+                                            ::testing::ValuesIn(inShapes_a),
+                                            ::testing::ValuesIn(inShapes_b),
+                                            ::testing::ValuesIn({ov::element::f32, ov::element::i8}),
+                                            // Expected num nodes increases 1, because one additional Convert layer will
+                                            // be inserted for precision conversion(issue 115822)
+                                            ::testing::Values(2),
+                                            ::testing::Values(1),
+                                            ::testing::Values(ov::test::utils::DEVICE_CPU)),
+                         Select::getTestCaseName);
 
 // DS
-std::vector<ov::test::InputShape> inShapesDynamic_a{{{1, {1, 5}, -1, 35}, {{1, 5, 5, 35}, {1, 1, 1, 35}, {1, 5, 5, 35}}}};
+std::vector<ov::test::InputShape> inShapesDynamic_a{
+    {{1, {1, 5}, -1, 35}, {{1, 5, 5, 35}, {1, 1, 1, 35}, {1, 5, 5, 35}}}};
 std::vector<ov::test::InputShape> inShapesDynamic_b{{{-1}, {{1}, {1}, {1}}}};
-INSTANTIATE_TEST_SUITE_P(smoke_Snippets_Select_Dynamic, Select,
-        ::testing::Combine(
-                ::testing::ValuesIn(inShapesDynamic_a),
-                ::testing::ValuesIn(inShapesDynamic_a),
-                ::testing::ValuesIn(inShapesDynamic_b),
-                ::testing::ValuesIn({ov::element::f32, ov::element::i8}),
-                ::testing::Values(1),
-                ::testing::Values(1),
-                ::testing::Values(ov::test::utils::DEVICE_CPU)),
-        Select::getTestCaseName);
+INSTANTIATE_TEST_SUITE_P(smoke_Snippets_Select_Dynamic,
+                         Select,
+                         ::testing::Combine(::testing::ValuesIn(inShapesDynamic_a),
+                                            ::testing::ValuesIn(inShapesDynamic_a),
+                                            ::testing::ValuesIn(inShapesDynamic_b),
+                                            ::testing::ValuesIn({ov::element::f32, ov::element::i8}),
+                                            // Expected num nodes increases 1, because one additional Convert layer will
+                                            // be inserted for precision conversion(issue 115822)
+                                            ::testing::Values(2),
+                                            ::testing::Values(1),
+                                            ::testing::Values(ov::test::utils::DEVICE_CPU)),
+                         Select::getTestCaseName);
 
 //============================BroadcastSelect=======================================//
 std::vector<ov::test::InputShape> inShapes0{{{}, {{1, 8, 2, 1}}}, {{}, {{1, 1, 1, 1}}}};
@@ -46,37 +51,42 @@ std::vector<ov::test::InputShape> inShapes1{{{}, {{1, 8, 2, 10}}}, {{}, {{1, 8,
 std::vector<ov::test::InputShape> inShapes2{{{}, {{1, 8, 2, 10}}}, {{}, {{1, 1, 1, 1}}}};
 std::vector<ov::PartialShape> inShapes3{{1, 8, 2, 1}, {1, 8, 2, 10}};
 
-INSTANTIATE_TEST_SUITE_P(smoke_Snippets_BroadcastSelect, BroadcastSelect,
-                         ::testing::Combine(
-                                 ::testing::ValuesIn(inShapes0),
-                                 ::testing::ValuesIn(inShapes1),
-                                 ::testing::ValuesIn(inShapes2),
-                                 ::testing::ValuesIn(inShapes3),
-                                 ::testing::ValuesIn({ov::element::f32, ov::element::i8}),
-                                 ::testing::Values(1),
-                                 ::testing::Values(1),
-                                 ::testing::Values(ov::test::utils::DEVICE_CPU)),
+INSTANTIATE_TEST_SUITE_P(smoke_Snippets_BroadcastSelect,
+                         BroadcastSelect,
+                         ::testing::Combine(::testing::ValuesIn(inShapes0),
+                                            ::testing::ValuesIn(inShapes1),
+                                            ::testing::ValuesIn(inShapes2),
+                                            ::testing::ValuesIn(inShapes3),
+                                            ::testing::ValuesIn({ov::element::f32, ov::element::i8}),
+                                            // Expected num nodes increases 1, because one additional Convert layer will
+                                            // be inserted for precision conversion(issue 115822)
+                                            ::testing::Values(2),
+                                            ::testing::Values(1),
+                                            ::testing::Values(ov::test::utils::DEVICE_CPU)),
                          BroadcastSelect::getTestCaseName);
 
 // DS
 std::vector<ov::test::InputShape> inShapes0_d{{{-1, -1, -1, -1}, {{1, 8, 2, 1}, {1, 1, 1, 1}, {1, 8, 2, 1}}}};
 std::vector<ov::test::InputShape> inShapes1_d{{{1, -1, -1, -1}, {{1, 8, 2, 10}, {1, 8, 2, 10}, {1, 8, 2, 10}}}};
-std::vector<ov::test::InputShape> inShapes2_d{{{1, {1, 8}, {1, 2}, {1, 10}}, {{1, 8, 2, 10}, {1, 1, 2, 1}, {1, 8, 2, 10}}}};
+std::vector<ov::test::InputShape> inShapes2_d{
+    {{1, {1, 8}, {1, 2}, {1, 10}}, {{1, 8, 2, 10}, {1, 1, 2, 1}, {1, 8, 2, 10}}}};
 std::vector<ov::PartialShape> inShapes3_d{{1, 8, 2, 1}, {1, 8, 2, 10}};
 
-INSTANTIATE_TEST_SUITE_P(smoke_Snippets_BroadcastSelect_Dynamic, BroadcastSelect,
-                         ::testing::Combine(
-                                 ::testing::ValuesIn(inShapes0_d),
-                                 ::testing::ValuesIn(inShapes1_d),
-                                 ::testing::ValuesIn(inShapes2_d),
-                                 ::testing::ValuesIn(inShapes3_d),
-                                 ::testing::ValuesIn({ov::element::f32, ov::element::i8}),
-                                 ::testing::Values(1),
-                                 ::testing::Values(1),
-                                 ::testing::Values(ov::test::utils::DEVICE_CPU)),
+INSTANTIATE_TEST_SUITE_P(smoke_Snippets_BroadcastSelect_Dynamic,
+                         BroadcastSelect,
+                         ::testing::Combine(::testing::ValuesIn(inShapes0_d),
+                                            ::testing::ValuesIn(inShapes1_d),
+                                            ::testing::ValuesIn(inShapes2_d),
+                                            ::testing::ValuesIn(inShapes3_d),
+                                            ::testing::ValuesIn({ov::element::f32, ov::element::i8}),
+                                            // Expected num nodes increases 1, because one additional Convert layer will
+                                            // be inserted for precision conversion(issue 115822)
+                                            ::testing::Values(2),
+                                            ::testing::Values(1),
+                                            ::testing::Values(ov::test::utils::DEVICE_CPU)),
                          BroadcastSelect::getTestCaseName);
 
-} // namespace
-} // namespace snippets
-} // namespace test
-} // namespace ov
+}  // namespace
+}  // namespace snippets
+}  // namespace test
+}  // namespace ov

src/plugins/intel_cpu/tests/functional/subgraph_tests/src/fuse_transpose_reorder.cpp

@@ -45,7 +45,7 @@ void FuseTransposeAndReorderTest::CheckTransposeCount(size_t expectedTransposeCo
 
 void FuseTransposeAndReorderTest::SetUp() {
     targetDevice = ov::test::utils::DEVICE_CPU;
-
+    SKIP_IF_CURRENT_TEST_IS_DISABLED();
     std::tie(inputShape, inPrec) = this->GetParam();
     CreateGraph();
 }

src/plugins/intel_cpu/tests/functional/test_utils/cpu_test_utils.cpp

@@ -221,7 +221,7 @@ void CPUTestsBase::CheckPluginRelatedResultsImpl(const std::shared_ptr<const ov:
 
             auto primType = getExecValue(ExecGraphInfoSerialization::IMPL_TYPE);
 
-            ASSERT_TRUE(primTypeCheck(primType)) << "primType is unexpected: " << primType << " Expected: " << selectedType;
+            ASSERT_TRUE(primTypeCheck(primType)) << "primType is unexpected : " << primType << " Expected : " << selectedType;
         }
     }
 }

src/tests/functional/plugin/shared/src/behavior/compiled_model/properties.cpp

@@ -149,7 +149,7 @@ TEST_P(OVClassCompiledModelPropertiesDefaultTests, CheckDefaultValues) {
         ASSERT_TRUE(supported) << "default_property=" << default_property.first;
         Any property;
         OV_ASSERT_NO_THROW(property = compiled_model.get_property(default_property.first));
-        ASSERT_EQ(default_property.second, property)
+        ASSERT_EQ(default_property.second.as<std::string>(), property.as<std::string>())
             << "For property: " << default_property.first
             << " expected value is: " << default_property.second.as<std::string>();
     }