DOCS: ported changes from 2022.1 release branch (#11206)
* Extensibility guide with FE extensions and remove OV_FRAMEWORK_MAP from docs * Rework of Extensibility Intro, adopted examples to missing OPENVINO_FRAMEWORK_MAP * Removed OPENVINO_FRAMEWORK_MAP reference * Frontend extension detailed documentation * Fixed distributed snippets * Fixed snippet inclusion in FE extension document and chapter headers * Fixed wrong name in a snippet reference * Fixed test for template extension due to changed number of loaded extensions * Update docs/Extensibility_UG/frontend_extensions.md Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> * Minor fixes in extension snippets * Small grammar fix Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> * DOCS: transition banner (#10973) * transition banner * minor fix * update transition banner * updates * update custom.js * updates * updates * Documentation fixes (#11044) * Benchmark app usage * Fixed link to the devices * More fixes * Update docs/OV_Runtime_UG/multi_device.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Removed several hardcoded links Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Updated documentation for compile_tool (#11049) * Added deployment guide (#11060) * Added deployment guide * Added local distribution * Updates * Fixed more indentations * Removed obsolete code snippets (#11061) * Removed obsolete code snippets * NCC style * Fixed NCC for BA * Add a troubleshooting issue for PRC installation (#11074) * updates * adding gna to linux * add missing reference * update * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * update * minor updates * add gna item to yum and apt * add gna to get started page * update reference formatting * merge commit * add a troubleshooting issue * update * update * fix CVS-71846 Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * DOCS: fixed hardcoded links (#11100) * Fixes * Use links * applying reviewers comments to the Opt Guide (#11093) * applying reviewrs comments * fixed refs, more structuring (bold, bullets, etc) * refactoring tput/latency sections * next iteration (mostly latency), also brushed the auto-batching and other sections * updates sync/async images * common opts brushed * WIP tput redesigned * minor brushing of common and auto-batching * Tput fully refactored * fixed doc name in the link * moved int8 perf counters to the right section * fixed links * fixed broken quotes * fixed more links * add ref to the internals to the TOC * Added a note on the batch size Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com> * [80085] New images for docs (#11114) * change doc structure * fix manager tools * fix manager tools 3 step * fix manager tools 3 step * new img * new img for OV Runtime * fix steps * steps * fix intendents * change list * fix space * fix space * code snippets fix * change display * Benchmarks 2022 1 (#11130) * Minor fixes * Updates for 2022.1 * Edits according to the review * Edits according to review comments * Edits according to review comments * Edits according to review comments * Fixed table * Edits according to review comments * Removed config for Intel® Core™ i7-11850HE * Removed forward-tacotron-duration-prediction-241 graph * Added resnet-18-pytorch * Add info about Docker images in Deployment guide (#11136) * Renamed user guides (#11137) * fix screenshot (#11140) * More conservative recommendations on dynamic shapes usage in docs (#11161) * More conservative recommendations about using dynamic shapes * Duplicated statement from C++ part to Python part of reshape doc (no semantical changes) * Update ShapeInference.md (#11168) * Benchmarks 2022 1 updates (#11180) * Updated graphs * Quick fix for TODO in Dynamic Shapes article * Anchor link fixes * Fixed DM config (#11199) * DOCS: doxy sphinxtabs (#11027) * initial implementation of doxy sphinxtabs * fixes * fixes * fixes * fixes * fixes * WA for ignored visibility attribute * Fixes Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com> Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Ilya Naumov <ilya.naumov@intel.com> Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
This commit is contained in:
161 files changed
+3254
-3078
No files matched your search
@@ -4,6 +4,13 @@
|
||||
|
||||
set(TARGET_NAME ie_docs_snippets)
|
||||
|
||||
if(CMAKE_COMPILER_IS_GNUCXX OR OV_COMPILER_IS_CLANG)
|
||||
ie_add_compiler_flags(-Wno-unused-variable)
|
||||
if(CMAKE_COMPILER_IS_GNUCXX)
|
||||
ie_add_compiler_flags(-Wno-unused-variable -Wno-unused-but-set-variable)
|
||||
endif()
|
||||
endif()
|
||||
|
||||
file(GLOB SOURCES "${CMAKE_CURRENT_SOURCE_DIR}/*.cpp"
|
||||
"${CMAKE_CURRENT_SOURCE_DIR}/gpu/*.cpp")
|
||||
|
||||
@@ -57,9 +64,9 @@ endif()
|
||||
|
||||
# remove OpenCV related sources
|
||||
if (ENABLE_OPENCV)
|
||||
find_package(OpenCV QUIET)
|
||||
find_package(OpenCV QUIET)
|
||||
else()
|
||||
set(OpenCV_FOUND FALSE)
|
||||
set(OpenCV_FOUND OFF)
|
||||
endif()
|
||||
|
||||
if(NOT OpenCV_FOUND)
|
||||
@@ -102,30 +109,25 @@ if(ENABLE_OV_ONNX_FRONTEND)
|
||||
target_link_libraries(${TARGET_NAME} PRIVATE openvino_onnx_frontend)
|
||||
endif()
|
||||
|
||||
if(NOT MSVC)
|
||||
target_compile_options(${TARGET_NAME} PRIVATE -Wno-unused-variable)
|
||||
if(CMAKE_COMPILER_IS_GNUCXX)
|
||||
target_compile_options(${TARGET_NAME} PRIVATE -Wno-unused-but-set-variable)
|
||||
endif()
|
||||
endif()
|
||||
|
||||
target_link_libraries(${TARGET_NAME} PRIVATE openvino::runtime openvino::runtime::dev)
|
||||
|
||||
# ov_ncc_naming_style(FOR_TARGET "${TARGET_NAME}"
|
||||
# SOURCE_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
|
||||
# ADDITIONAL_INCLUDE_DIRECTORIES
|
||||
# $<TARGET_PROPERTY:openvino::runtime,INTERFACE_INCLUDE_DIRECTORIES>)
|
||||
|
||||
#
|
||||
# Example
|
||||
#
|
||||
|
||||
set(TARGET_NAME "ov_integration_snippet")
|
||||
# [cmake:integration_example]
|
||||
cmake_minimum_required(VERSION 3.10)
|
||||
set(CMAKE_CXX_STANDARD 11)
|
||||
|
||||
|
||||
find_package(OpenVINO REQUIRED)
|
||||
|
||||
add_executable(${TARGET_NAME} src/main.cpp)
|
||||
|
||||
target_link_libraries(${TARGET_NAME} PRIVATE openvino::runtime)
|
||||
# [cmake:integration_example]
|
||||
if(NOT MSVC)
|
||||
target_compile_options(${TARGET_NAME} PRIVATE -Wno-unused-variable)
|
||||
if(CMAKE_COMPILER_IS_GNUCXX)
|
||||
target_compile_options(${TARGET_NAME} PRIVATE -Wno-unused-but-set-variable)
|
||||
endif()
|
||||
endif()
|
||||
@@ -1,13 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
#include <openvino/core/model.hpp>
|
||||
#include <openvino/pass/visualize_tree.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part0]
|
||||
std::shared_ptr<ov::Model> model;
|
||||
// ...
|
||||
ov::pass::VisualizeTree("after.png").run_on_model(model); // Visualize the nGraph function to an image
|
||||
//! [part0]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,13 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
#include <ngraph/function.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part1]
|
||||
std::shared_ptr<ngraph::Function> nGraph;
|
||||
// ...
|
||||
CNNNetwork network(nGraph);
|
||||
network.serialize("test_ir.xml", "test_ir.bin");
|
||||
//! [part1]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part0]
|
||||
InferenceEngine::Core core;
|
||||
std::vector<std::string> availableDevices = core.GetAvailableDevices();
|
||||
//! [part0]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part1]
|
||||
InferenceEngine::Core core;
|
||||
bool dumpDotFile = core.GetConfig("HETERO", HETERO_CONFIG_KEY(DUMP_GRAPH_DOT)).as<bool>();
|
||||
//! [part1]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part2]
|
||||
InferenceEngine::Core core;
|
||||
std::string cpuDeviceName = core.GetMetric("GPU", METRIC_KEY(FULL_DEVICE_NAME)).as<std::string>();
|
||||
//! [part2]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part3]
|
||||
InferenceEngine::Core core;
|
||||
auto network = core.ReadNetwork("sample.xml");
|
||||
auto exeNetwork = core.LoadNetwork(network, "CPU");
|
||||
auto nireq = exeNetwork.GetMetric(METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)).as<unsigned int>();
|
||||
//! [part3]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part4]
|
||||
InferenceEngine::Core core;
|
||||
auto network = core.ReadNetwork("sample.xml");
|
||||
auto exeNetwork = core.LoadNetwork(network, "MYRIAD");
|
||||
float temperature = exeNetwork.GetMetric(METRIC_KEY(DEVICE_THERMAL)).as<float>();
|
||||
//! [part4]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part5]
|
||||
InferenceEngine::Core core;
|
||||
auto network = core.ReadNetwork("sample.xml");
|
||||
auto exeNetwork = core.LoadNetwork(network, "CPU");
|
||||
auto ncores = exeNetwork.GetConfig(PluginConfigParams::KEY_CPU_THREADS_NUM).as<std::string>();
|
||||
//! [part5]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,16 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part1]
|
||||
Core ie;
|
||||
auto netReader = ie.ReadNetwork("sample.xml");
|
||||
InferenceEngine::InputsDataMap info(netReader.getInputsInfo());
|
||||
auto& inputInfoFirst = info.begin()->second;
|
||||
for (auto& it : info) {
|
||||
it.second->setPrecision(Precision::U8);
|
||||
}
|
||||
//! [part1]
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -1,14 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part2]
|
||||
//Lock Intel MSS surface
|
||||
mfxFrameSurface1 *frame_in; //Input MSS surface.
|
||||
mfxFrameAllocator* pAlloc = &m_mfxCore.FrameAllocator();
|
||||
pAlloc->Lock(pAlloc->pthis, frame_in->Data.MemId, &frame_in->Data);
|
||||
//Inference Engine code
|
||||
//! [part2]
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part3]
|
||||
InferenceEngine::SizeVector dims_src = {
|
||||
1 /* batch, N*/,
|
||||
(size_t) frame_in->Info.Height /* Height */,
|
||||
(size_t) frame_in->Info.Width /* Width */,
|
||||
3 /*Channels,*/,
|
||||
};
|
||||
InferenceEngine::TensorDesc desc(InferenceEngine::Precision::U8, dims_src, InferenceEngine::NHWC);
|
||||
/* wrapping the surface data, as RGB is interleaved, need to pass only ptr to the R, notice that this wouldn’t work with planar formats as these are 3 separate planes/pointers*/
|
||||
InferenceEngine::TBlob<uint8_t>::Ptr p = InferenceEngine::make_shared_blob<uint8_t>( desc, (uint8_t*) frame_in->Data.R);
|
||||
inferRequest.SetBlob("input", p);
|
||||
inferRequest.Infer();
|
||||
//Make sure to unlock the surface upon inference completion, to return the ownership back to the Intel MSS
|
||||
pAlloc->Unlock(pAlloc->pthis, frame_in->Data.MemId, &frame_in->Data);
|
||||
//! [part3]
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part4]
|
||||
InferenceEngine::SizeVector dims_src = {
|
||||
1 /* batch, N*/,
|
||||
3 /*Channels,*/,
|
||||
(size_t) frame_in->Info.Height /* Height */,
|
||||
(size_t) frame_in->Info.Width /* Width */,
|
||||
};
|
||||
TensorDesc desc(InferenceEngine::Precision::U8, dims_src, InferenceEngine::NCHW);
|
||||
/* wrapping the RGBP surface data*/
|
||||
InferenceEngine::TBlob<uint8_t>::Ptr p = InferenceEngine::make_shared_blob<uint8_t>( desc, (uint8_t*) frame_in->Data.R);
|
||||
inferRequest.SetBlob("input", p);
|
||||
// …
|
||||
//! [part4]
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
#include <opencv2/core/core.hpp>
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
InferenceEngine::InferRequest inferRequest;
|
||||
//! [part5]
|
||||
cv::Mat frame(cv::Size(100, 100), CV_8UC3); // regular CV_8UC3 image, interleaved
|
||||
// creating blob that wraps the OpenCV’s Mat
|
||||
// (the data it points should persists until the blob is released):
|
||||
InferenceEngine::SizeVector dims_src = {
|
||||
1 /* batch, N*/,
|
||||
(size_t)frame.rows /* Height */,
|
||||
(size_t)frame.cols /* Width */,
|
||||
(size_t)frame.channels() /*Channels,*/,
|
||||
};
|
||||
InferenceEngine::TensorDesc desc(InferenceEngine::Precision::U8, dims_src, InferenceEngine::NHWC);
|
||||
InferenceEngine::TBlob<uint8_t>::Ptr p = InferenceEngine::make_shared_blob<uint8_t>( desc, (uint8_t*)frame.data, frame.step[0] * frame.rows);
|
||||
inferRequest.SetBlob("input", p);
|
||||
inferRequest.Infer();
|
||||
// …
|
||||
// similarly, you can wrap the output tensor (let’s assume it is FP32)
|
||||
// notice that the output should be also explicitly stated as NHWC with setLayout
|
||||
auto output_blob = inferRequest.GetBlob("output");
|
||||
const float* output_data = output_blob->buffer().as<float*>();
|
||||
auto dims = output_blob->getTensorDesc().getDims();
|
||||
cv::Mat res (dims[2], dims[3], CV_32FC3, (void *)output_data);
|
||||
//! [part5]
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
using namespace InferenceEngine;
|
||||
//! [part6]
|
||||
InferenceEngine::Core ie;
|
||||
auto network = ie.ReadNetwork("Model.xml", "Model.bin");
|
||||
InferenceEngine::InputsDataMap input_info(network.getInputsInfo());
|
||||
|
||||
auto executable_network = ie.LoadNetwork(network, "GPU");
|
||||
auto infer_request = executable_network.CreateInferRequest();
|
||||
|
||||
for (auto & item : input_info) {
|
||||
std::string input_name = item.first;
|
||||
auto input = infer_request.GetBlob(input_name);
|
||||
/** Lock/Fill input tensor with data **/
|
||||
unsigned char* data = input->buffer().as<PrecisionTrait<Precision::U8>::value_type*>();
|
||||
// ...
|
||||
}
|
||||
|
||||
infer_request.Infer();
|
||||
//! [part6]
|
||||
return 0;
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
InferenceEngine::Core core;
|
||||
auto network0 = core.ReadNetwork("sample.xml");
|
||||
auto network1 = core.ReadNetwork("sample.xml");
|
||||
//! [part7]
|
||||
//these two networks go thru same plugin (aka device) and their requests will not overlap.
|
||||
auto executable_network0 = core.LoadNetwork(network0, "CPU",
|
||||
{{InferenceEngine::PluginConfigParams::KEY_EXCLUSIVE_ASYNC_REQUESTS, InferenceEngine::PluginConfigParams::YES}});
|
||||
auto executable_network1 = core.LoadNetwork(network1, "GPU",
|
||||
{{InferenceEngine::PluginConfigParams::KEY_EXCLUSIVE_ASYNC_REQUESTS, InferenceEngine::PluginConfigParams::YES}});
|
||||
//! [part7]
|
||||
return 0;
|
||||
}
|
||||
@@ -6,7 +6,8 @@ while(true) {
|
||||
// capture frame
|
||||
// populate NEXT InferRequest
|
||||
// start NEXT InferRequest //this call is async and returns immediately
|
||||
// wait for the CURRENT InferRequest //processed in a dedicated thread
|
||||
|
||||
// wait for the CURRENT InferRequest
|
||||
// display CURRENT result
|
||||
// swap CURRENT and NEXT InferRequests
|
||||
}
|
||||
|
||||
@@ -12,11 +12,11 @@ class AcceleratorSyncRequest : public IInferRequestInternal {
|
||||
public:
|
||||
using Ptr = std::shared_ptr<AcceleratorSyncRequest>;
|
||||
|
||||
void Preprocess();
|
||||
void WriteToDevice();
|
||||
void RunOnDevice();
|
||||
void ReadFromDevice();
|
||||
void PostProcess();
|
||||
void preprocess();
|
||||
void write_to_device();
|
||||
void run_on_device();
|
||||
void read_from_device();
|
||||
void post_process();
|
||||
};
|
||||
|
||||
// ! [async_infer_request:define_pipeline]
|
||||
@@ -40,19 +40,19 @@ class AcceleratorAsyncInferRequest : public AsyncInferRequestThreadSafeDefault {
|
||||
// Five pipeline stages of synchronous infer request are run by different executors
|
||||
_pipeline = {
|
||||
{ _preprocessExecutor , [this] {
|
||||
_accSyncRequest->Preprocess();
|
||||
_accSyncRequest->preprocess();
|
||||
}},
|
||||
{ _writeToDeviceExecutor , [this] {
|
||||
_accSyncRequest->WriteToDevice();
|
||||
_accSyncRequest->write_to_device();
|
||||
}},
|
||||
{ _runOnDeviceExecutor , [this] {
|
||||
_accSyncRequest->RunOnDevice();
|
||||
_accSyncRequest->run_on_device();
|
||||
}},
|
||||
{ _readFromDeviceExecutor , [this] {
|
||||
_accSyncRequest->ReadFromDevice();
|
||||
_accSyncRequest->read_from_device();
|
||||
}},
|
||||
{ _postProcessExecutor , [this] {
|
||||
_accSyncRequest->PostProcess();
|
||||
_accSyncRequest->post_process();
|
||||
}},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
|
||||
int main() {
|
||||
InferenceEngine::Core core;
|
||||
int numRequests = 42;
|
||||
int i = 1;
|
||||
auto network = core.ReadNetwork("sample.xml");
|
||||
auto executable_network = core.LoadNetwork(network, "CPU");
|
||||
//! [part0]
|
||||
struct Request {
|
||||
InferenceEngine::InferRequest inferRequest;
|
||||
int frameidx;
|
||||
};
|
||||
//! [part0]
|
||||
|
||||
//! [part1]
|
||||
// numRequests is the number of frames (max size, equal to the number of VPUs in use)
|
||||
std::vector<Request> request(numRequests);
|
||||
//! [part1]
|
||||
|
||||
//! [part2]
|
||||
// initialize infer request pointer – Consult IE API for more detail.
|
||||
request[i].inferRequest = executable_network.CreateInferRequest();
|
||||
//! [part2]
|
||||
|
||||
//! [part3]
|
||||
// Run inference
|
||||
request[i].inferRequest.StartAsync();
|
||||
//! [part3]
|
||||
|
||||
//! [part4]
|
||||
request[i].inferRequest.SetCompletionCallback([] () {});
|
||||
//! [part4]
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
#include <ie_core.hpp>
|
||||
#include "ngraph/opsets/opset.hpp"
|
||||
#include "ngraph/opsets/opset3.hpp"
|
||||
|
||||
|
||||
int main() {
|
||||
//! [part0]
|
||||
|
||||
using namespace std;
|
||||
using namespace ngraph;
|
||||
|
||||
auto arg0 = make_shared<opset3::Parameter>(element::f32, Shape{7});
|
||||
auto arg1 = make_shared<opset3::Parameter>(element::f32, Shape{7});
|
||||
// Create an 'Add' operation with two inputs 'arg0' and 'arg1'
|
||||
auto add0 = make_shared<opset3::Add>(arg0, arg1);
|
||||
auto abs0 = make_shared<opset3::Abs>(add0);
|
||||
// Create a node whose inputs/attributes will be specified later
|
||||
auto acos0 = make_shared<opset3::Acos>();
|
||||
// Create a node using opset factories
|
||||
auto add1 = shared_ptr<Node>(get_opset3().create("Add"));
|
||||
// Set inputs to nodes explicitly
|
||||
acos0->set_argument(0, add0);
|
||||
add1->set_argument(0, acos0);
|
||||
add1->set_argument(1, abs0);
|
||||
|
||||
// Create a graph with one output (add1) and four inputs (arg0, arg1)
|
||||
auto ng_function = make_shared<Function>(OutputVector{add1}, ParameterVector{arg0, arg1});
|
||||
// Run shape inference on the nodes
|
||||
ng_function->validate_nodes_and_infer_types();
|
||||
|
||||
//! [part0]
|
||||
|
||||
//! [part1]
|
||||
InferenceEngine::CNNNetwork net (ng_function);
|
||||
//! [part1]
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -2,20 +2,123 @@
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
//
|
||||
#include <openvino/openvino.hpp>
|
||||
#include <openvino/core/op_extension.hpp>
|
||||
//! [add_extension_header]
|
||||
//#include <openvino/core/op_extension.hpp>
|
||||
//! [add_extension_header]
|
||||
//! [add_frontend_extension_header]
|
||||
#include <openvino/frontend/extension.hpp>
|
||||
//! [add_frontend_extension_header]
|
||||
|
||||
//! [frontend_extension_Identity_header]
|
||||
#include <openvino/frontend/extension.hpp>
|
||||
//! [frontend_extension_Identity_header]
|
||||
|
||||
//! [frontend_extension_ThresholdedReLU_header]
|
||||
#include <openvino/opsets/opset8.hpp>
|
||||
//! [frontend_extension_ThresholdedReLU_header]
|
||||
|
||||
#include <identity.hpp>
|
||||
|
||||
//! [frontend_extension_CustomOperation]
|
||||
class CustomOperation : public ov::op::Op {
|
||||
|
||||
std::string attr1;
|
||||
int attr2;
|
||||
|
||||
public:
|
||||
|
||||
OPENVINO_OP("CustomOperation");
|
||||
|
||||
bool visit_attributes(ov::AttributeVisitor& visitor) override {
|
||||
visitor.on_attribute("attr1", attr1);
|
||||
visitor.on_attribute("attr2", attr2);
|
||||
return true;
|
||||
}
|
||||
|
||||
// ... implement other required methods
|
||||
//! [frontend_extension_CustomOperation]
|
||||
std::shared_ptr<ov::Node> clone_with_new_inputs(const ov::OutputVector&) const override { return nullptr; }
|
||||
};
|
||||
|
||||
int main() {
|
||||
{
|
||||
//! [add_extension]
|
||||
ov::Core core;
|
||||
// Use operation type to add operation extension
|
||||
|
||||
// Use operation type to add operation extension
|
||||
core.add_extension<TemplateExtension::Identity>();
|
||||
// or you can add operation extension to this method
|
||||
|
||||
// or you can add operation extension object which is equivalent form
|
||||
core.add_extension(ov::OpExtension<TemplateExtension::Identity>());
|
||||
//! [add_extension]
|
||||
}
|
||||
{
|
||||
ov::Core core;
|
||||
|
||||
//! [add_frontend_extension]
|
||||
// Register mapping for new frontends: FW's "TemplateIdentity" operation to TemplateExtension::Identity
|
||||
core.add_extension(ov::frontend::OpExtension<TemplateExtension::Identity>("Identity"));
|
||||
|
||||
// Register more sophisticated mapping with decomposition
|
||||
core.add_extension(ov::frontend::ConversionExtension(
|
||||
"Identity",
|
||||
[](const ov::frontend::NodeContext& context) {
|
||||
// Arbitrary decomposition code here
|
||||
// Return a vector of operation outputs
|
||||
return ov::OutputVector{ std::make_shared<TemplateExtension::Identity>(context.get_input(0)) };
|
||||
}));
|
||||
//! [add_frontend_extension]
|
||||
}
|
||||
{
|
||||
//! [frontend_extension_Identity]
|
||||
auto extension1 = ov::frontend::OpExtension<TemplateExtension::Identity>("Identity");
|
||||
|
||||
// or even simpler if original FW type and OV type of operations match, that is "Identity"
|
||||
auto extension2 = ov::frontend::OpExtension<TemplateExtension::Identity>();
|
||||
//! [frontend_extension_Identity]
|
||||
|
||||
//! [frontend_extension_read_model]
|
||||
ov::Core core;
|
||||
// Add arbitrary number of extensions before calling read_model method
|
||||
core.add_extension(ov::frontend::OpExtension<TemplateExtension::Identity>());
|
||||
core.read_model("/path/to/model.onnx");
|
||||
//! [frontend_extension_read_model]
|
||||
|
||||
//! [frontend_extension_MyRelu]
|
||||
core.add_extension(ov::frontend::OpExtension<>("Relu", "MyRelu"));
|
||||
//! [frontend_extension_MyRelu]
|
||||
|
||||
//! [frontend_extension_CustomOperation_as_is]
|
||||
core.add_extension(ov::frontend::OpExtension<CustomOperation>());
|
||||
//! [frontend_extension_CustomOperation_as_is]
|
||||
|
||||
//! [frontend_extension_CustomOperation_rename]
|
||||
core.add_extension(ov::frontend::OpExtension<CustomOperation>(
|
||||
{ {"attr1", "fw_attr1"}, {"attr2", "fw_attr2"} },
|
||||
{}
|
||||
));
|
||||
//! [frontend_extension_CustomOperation_rename]
|
||||
|
||||
//! [frontend_extension_CustomOperation_rename_set]
|
||||
core.add_extension(ov::frontend::OpExtension<CustomOperation>(
|
||||
{ {"attr1", "fw_attr1"} },
|
||||
{ {"attr2", 5} }
|
||||
));
|
||||
//! [frontend_extension_CustomOperation_rename_set]
|
||||
|
||||
//! [frontend_extension_ThresholdedReLU]
|
||||
core.add_extension(ov::frontend::ConversionExtension(
|
||||
"ThresholdedReLU",
|
||||
[](const ov::frontend::NodeContext& node) {
|
||||
auto greater = std::make_shared<ov::opset8::Greater>(
|
||||
node.get_input(0),
|
||||
ov::opset8::Constant::create(ov::element::f32, {}, {node.get_attribute<float>("alpha")}));
|
||||
auto casted = std::make_shared<ov::opset8::Convert>(greater, ov::element::f32);
|
||||
return ov::OutputVector{ std::make_shared<ov::opset8::Multiply>(node.get_input(0), casted) };
|
||||
}));
|
||||
//! [frontend_extension_ThresholdedReLU]
|
||||
}
|
||||
{
|
||||
//! [add_extension_lib]
|
||||
ov::Core core;
|
||||
// Load extensions library to ov::Core
|
||||
|
||||
@@ -8,8 +8,13 @@ import openvino.runtime as ov
|
||||
# Not implemented
|
||||
#! [add_extension]
|
||||
|
||||
#! [add_frontend_extension]
|
||||
# Not implemented
|
||||
#! [add_frontend_extension]
|
||||
|
||||
#! [add_extension_lib]
|
||||
core = ov.Core()
|
||||
# Load extensions library to ov::Core
|
||||
# Load extensions library to ov.Core
|
||||
core.add_extension("openvino_template_extension.so")
|
||||
#! [add_extension_lib]
|
||||
|
||||
@@ -9,6 +9,10 @@ core = Core()
|
||||
available_devices = core.available_devices
|
||||
# [get_available_devices]
|
||||
|
||||
# [hetero_priorities]
|
||||
device_priorites = core.get_property("HETERO", "MULTI_DEVICE_PRIORITIES")
|
||||
# [hetero_priorities]
|
||||
|
||||
# [cpu_device_name]
|
||||
cpu_device_name = core.get_property("CPU", "FULL_DEVICE_NAME")
|
||||
# [cpu_device_name]
|
||||
@@ -22,7 +26,7 @@ compiled_model = core.compile_model(model, "CPU", config)
|
||||
|
||||
# [optimal_number_of_infer_requests]
|
||||
compiled_model = core.compile_model(model, "CPU")
|
||||
nireq = compiled_model.get_property("OPTIMAL_NUMBER_OF_INFER_REQUESTS");
|
||||
nireq = compiled_model.get_property("OPTIMAL_NUMBER_OF_INFER_REQUESTS")
|
||||
# [optimal_number_of_infer_requests]
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user