Files
openvino/samples/cpp/benchmark_app/shared_blob_allocator.hpp
Fedor Zharinov e9874ec1d4 Dynamic reshapes (#7788)
* Merged and compiling

* Fix for dynamic shape type

* review fixes

* renamed blob shape to tensor shape, small improvements

* fix code style

* added parsing of multiple shapes

* store latency per group, add isIdleRequestAvailable() to Infer Queue

* added cached random inputs

* redesign pipeline, added new metrics(avg, max, min), added metrics per groups

* fixed code style

* small improvements

* modified tensor parameters parsing

* modified -i parameter parsing: added possibility to specify input names

* implemented image cashing

* added cashed blobs creating

* added -pcseq flag, modified batch filling, changes fps formula

* improvements

* code formatting

* code formatting2

* apply suggestions from review

* replaced Buffer class with InferenceEngine Blobs

* use batch size in blobs filling

* added shared blob allocator to handle blob's data

* fixed warnings & code style

* allocate blobs

* fix for networks with image info input

* added comments & fixed codestyle

* clear data in free() in SharedBlobAllocator

* remove unnecessary check

* Delimeter is changed to ::

* stylefix

* added layout from string function, small improvements

* modified parsing to enable : in input parameters

* small fixes

* small fixes

* added missed blob allocation, fixes

* [TEST]added support for remote blobs

* fix remote blobs

* new inputs/files output format

* removed vectors resize which caused bugs

* made cl::Buffer type under ifdef, fix inputs filling

* changed batch() function to not throwing exceptions

* removed unused var

* fix code style

* replace empty name in input files with name from net input

* restored old behaviour for static models

* fix code style

* fix warning - made const iterator

* fix warning - remove reference in loop variable

* added random and image_info input types to -i, fix problem with layout

* replaced batch() with getBatchSize() in main

* fix layout, shape, tensor shape parameters parsing

* upd help messages for input, tensor shape and pcseq command

* added buffer for cl output blobs, small fixes

Signed-off-by: ivikhrev <ivan.vikhrev@intel.com>

* added legacy mode

* restore setBlob

* code style formatting

* move collecting latency for groups under flag

* removed not applicable layouts

* added hint to error message when wrong input name in -tensor_shape was specified

* added new metrics to statistics report

* Apply suggestions from code review

* fix binary blobs filling when layout is CN

* apply suggestions

* moved file in the right place after rebase

* improved -pcseq output

* updated args and readme

* removed TEMPLATE plugin registration

* fix -shape arg  decsription

* enable providing several -i args as input

* renamed legacy_mode to inference_only and made it default for static models, renamed tensor_shape to data_shape

* upd readme

* use getBlob() in inference only mode

* fix old input type for static case

* fix typo

* upd readme

* move log about benchmark mode to the measuring perfomance step

* added class for latency metrics

* upd readme, fix typos, renamed funcs

* fix warning and upd parsing to avoid error with : in file paths

* fix error on centos : error: use of deleted function ‘std::basic_stringstream<char>::basic_stringstream(const std::basic_stringstream<char>&)

* added check for key in inputs

* renamed input to inputs

* adjust batch size for binary blobs

* replaced warning with exception in bench mode defining

* align measurement cycle with master

Co-authored-by: ivikhrev <ivan.vikhrev@intel.com>
2021-12-17 12:20:43 +03:00

44 lines
996 B
C++

// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include "ie_allocator.hpp"
template <class T>
class SharedBlobAllocator : public InferenceEngine::IAllocator {
public:
SharedBlobAllocator(const T* data, size_t size) : data(data), size(size){};
~SharedBlobAllocator() {
free((void*)data);
};
void* lock(void* handle, InferenceEngine::LockOp op = InferenceEngine::LOCK_FOR_WRITE) noexcept override {
if (handle == data) {
return (void*)data;
}
return nullptr;
}
void unlock(void* handle) noexcept override{};
void* alloc(size_t size) noexcept override {
return size <= this->size ? (void*)data : nullptr;
};
bool free(void* handle) noexcept override {
if (handle == data) {
delete[] data;
data = nullptr;
return true;
}
return false;
};
private:
const T* data;
size_t size;
};