[docs] python snippet for dynamic shapes (#10762)
* Create snipp * link python snipp with doc * fix docs * Apply suggestions from code review Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com> * Fix cpp comments Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
This commit is contained in:
@@ -42,7 +42,21 @@ To avoid the tricks mentioned in the previous section there is a way to directly
|
||||
This is achieved with the same reshape method that is used for alternating static shape of inputs.
|
||||
Dynamic dimensions are specified as `-1` or `ov::Dimension()` instead of a positive number used for static dimensions:
|
||||
|
||||
@snippet snippets/ov_dynamic_shapes.cpp ov_dynamic_shapes:reshape_undefined
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_dynamic_shapes:reshape_undefined]
|
||||
|
||||
.. tab:: Python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.py
|
||||
:language: python
|
||||
:fragment: [reshape_undefined]
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
To simplify the code, the examples assume that the model has a single input and single output.
|
||||
However, there are no limitations on the number of inputs and outputs to apply dynamic shapes.
|
||||
@@ -66,7 +80,21 @@ Use this capability to save time on calling `reshape` method in the end applicat
|
||||
Besides marking a dimension just dynamic, you can also specify lower and/or upper bounds that define a range of allowed values for the dimension.
|
||||
Bounds are coded as arguments for `ov::Dimension`:
|
||||
|
||||
@snippet snippets/ov_dynamic_shapes.cpp ov_dynamic_shapes:reshape_bounds
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_dynamic_shapes:reshape_bounds]
|
||||
|
||||
.. tab:: Python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.py
|
||||
:language: python
|
||||
:fragment: [reshape_bounds]
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
Information about bounds gives opportunity for the inference plugin to apply additional optimizations.
|
||||
Using dynamic shapes assumes the plugins apply more loose optimization technique during model compilation
|
||||
@@ -86,7 +114,21 @@ Preparing model with the reshape method was the first step.
|
||||
The second step is passing a tensor with an appropriate shape to infer request.
|
||||
This is similar to [regular steps](integrate_with_your_application.md), but now we can pass tensors with different shapes for the same executable model and even for the same inference request:
|
||||
|
||||
@snippet snippets/ov_dynamic_shapes.cpp ov_dynamic_shapes:set_input_tensor
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_dynamic_shapes:set_input_tensor]
|
||||
|
||||
.. tab:: Python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.py
|
||||
:language: python
|
||||
:fragment: [set_input_tensor]
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
In the example above `set_input_tensor` is used to specify input tensors.
|
||||
The real dimensions of the tensor is always static, because it is a concrete tensor and it doesn't have any dimension variations in contrast to model inputs.
|
||||
@@ -97,7 +139,21 @@ Without doing that, the tensor returned by `get_input_tensor` is an empty tensor
|
||||
Setting shape for input tensor is required when the corresponding input has at least one dynamic dimension regardless of bounds information.
|
||||
The following example makes the same sequence of two infer request as the previous example but using `get_input_tensor` instead of `set_input_tensor`:
|
||||
|
||||
@snippet snippets/ov_dynamic_shapes.cpp ov_dynamic_shapes:get_input_tensor
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_dynamic_shapes:get_input_tensor]
|
||||
|
||||
.. tab:: Python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.py
|
||||
:language: python
|
||||
:fragment: [get_input_tensor]
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
### Dynamic Shapes in Outputs
|
||||
|
||||
@@ -108,13 +164,41 @@ The same is true for other dimensions, like sequence length for NLP models or sp
|
||||
Whether or not output has dynamic dimensions can be examined by querying output partial shape after model read or reshape.
|
||||
The same is applicable for inputs. For example:
|
||||
|
||||
@snippet snippets/ov_dynamic_shapes.cpp ov_dynamic_shapes:print_dynamic
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_dynamic_shapes:print_dynamic]
|
||||
|
||||
.. tab:: Python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.py
|
||||
:language: python
|
||||
:fragment: [print_dynamic]
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
Appearing `?` or ranges like `1..10` means there are dynamic dimensions in corresponding inputs or outputs.
|
||||
|
||||
Or more programmatically:
|
||||
|
||||
@snippet snippets/ov_dynamic_shapes.cpp ov_dynamic_shapes:detect_dynamic
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_dynamic_shapes:detect_dynamic]
|
||||
|
||||
.. tab:: Python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.py
|
||||
:language: python
|
||||
:fragment: [detect_dynamic]
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
If at least one dynamic dimension exists in output of the model, shape of the corresponding output tensor will be set as the result of inference call.
|
||||
Before the first inference, memory for such a tensor is not allocated and has shape `[0]`.
|
||||
|
||||
Reference in New Issue
Block a user