diff --git a/docs/MO_DG/prepare_model/convert_model/Convert_Model_Tutorials.md b/docs/MO_DG/prepare_model/convert_model/Convert_Model_Tutorials.md index a46d1cbcb76..c647709d218 100644 --- a/docs/MO_DG/prepare_model/convert_model/Convert_Model_Tutorials.md +++ b/docs/MO_DG/prepare_model/convert_model/Convert_Model_Tutorials.md @@ -35,9 +35,10 @@ openvino_docs_MO_DG_prepare_model_convert_model_mxnet_specific_Convert_Style_Transfer_From_MXNet openvino_docs_MO_DG_prepare_model_convert_model_kaldi_specific_Aspire_Tdnn_Model -@endsphinxdirective This section provides a set of tutorials that demonstrate conversion methods for specific TensorFlow, ONNX, PyTorch, MXNet, and Kaldi models, that unnecessarily cover your case. -Before studying the tutorials, try to convert the model out-of-the-box by specifying only the `--input_model` parameter in the command line. +Before studying the tutorials, try to convert the model out-of-the-box by specifying only the ``--input_model`` parameter in the command line. -You will find a collection of [Python tutorials](../../../tutorials.md) written for running on Jupyter notebooks that provide an introduction to the OpenVINO™ toolkit and explain how to use the Python API and tools for optimized deep learning inference. +You will find a collection of :doc:`Python tutorials ` written for running on Jupyter notebooks that provide an introduction to the OpenVINO™ toolkit and explain how to use the Python API and tools for optimized deep learning inference. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Faster_RCNN.md b/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Faster_RCNN.md index e16c86c8ed2..ccebf3b7178 100644 --- a/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Faster_RCNN.md +++ b/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Faster_RCNN.md @@ -1,23 +1,29 @@ # Converting an ONNX Faster R-CNN Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Faster_RCNN} -The instructions below are applicable **only** to the Faster R-CNN model converted to the ONNX file format from the [maskrcnn-benchmark model](https://github.com/facebookresearch/maskrcnn-benchmark): +@sphinxdirective -1. Download the pretrained model file from [onnx/models](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn): - * (commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117). +The instructions below are applicable **only** to the Faster R-CNN model converted to the ONNX file format from the `maskrcnn-benchmark model `__: + +1. Download the pretrained model file from `onnx/models `__ (commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117). 2. Generate the Intermediate Representation of the model, by changing your current working directory to the Model Optimizer installation directory, and running Model Optimizer with the following parameters: -```sh - mo \ ---input_model FasterRCNN-10.onnx \ ---input_shape [1,3,800,800] \ ---input 0:2 \ ---mean_values [102.9801,115.9465,122.7717] \ ---transformations_config front/onnx/faster_rcnn.json -``` -Be aware that the height and width specified with the `input_shape` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the [Faster R-CNN article](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn). +.. code-block:: sh -3. Interpret the outputs of the generated IR: class indices, probabilities and box coordinates. Below are the outputs from the "DetectionOutput" layer: - * class indices. - * probabilities. - * box coordinates. + mo \ + --input_model FasterRCNN-10.onnx \ + --input_shape [1,3,800,800] \ + --input 0:2 \ + --mean_values [102.9801,115.9465,122.7717] \ + --transformations_config front/onnx/faster_rcnn.json + + +Be aware that the height and width specified with the ``input_shape`` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the `Faster R-CNN article `__. + +3. Interpret the outputs of the generated IR: class indices, probabilities and box coordinates. Below are the outputs from the ``DetectionOutput`` layer: + + * class indices + * probabilities + * box coordinates + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_GPT2.md b/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_GPT2.md index cdd2b41b7de..c694ad431a4 100644 --- a/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_GPT2.md +++ b/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_GPT2.md @@ -1,17 +1,25 @@ # Converting an ONNX GPT-2 Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_GPT2} -[Public pretrained GPT-2 model](https://github.com/onnx/models/tree/master/text/machine_comprehension/gpt-2) is a large +@sphinxdirective + +`Public pre-trained GPT-2 model `__ is a large transformer-based language model with a simple objective: predict the next word, given all of the previous words within some text. -## Downloading the Pre-Trained Base GPT-2 Model +Downloading the Pre-Trained Base GPT-2 Model +############################################ -To download the model, go to [this model](https://github.com/onnx/models/blob/master/text/machine_comprehension/gpt-2/model/gpt2-10.onnx), and press **Download**. +To download the model, go to `this model `__, and press **Download**. -To download the model and sample test data, go to [this model](https://github.com/onnx/models/blob/master/text/machine_comprehension/gpt-2/model/gpt2-10.tar.gz), and press **Download**. +To download the model and sample test data, go to `this model `__, and press **Download**. -## Converting an ONNX GPT-2 Model to IR +Converting an ONNX GPT-2 Model to IR +#################################### Generate the Intermediate Representation of the model GPT-2 by running Model Optimizer with the following parameters: -```sh -mo --input_model gpt2-10.onnx --input_shape [X,Y,Z] --output_dir -``` + +.. code-block:: sh + + mo --input_model gpt2-10.onnx --input_shape [X,Y,Z] --output_dir + + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Mask_RCNN.md b/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Mask_RCNN.md index fed92fabe20..63f64bb3982 100644 --- a/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Mask_RCNN.md +++ b/docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Mask_RCNN.md @@ -1,26 +1,32 @@ # Converting an ONNX Mask R-CNN Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Mask_RCNN} -The instructions below are applicable **only** to the Mask R-CNN model converted to the ONNX file format from the [maskrcnn-benchmark model](https://github.com/facebookresearch/maskrcnn-benchmark). +@sphinxdirective -1. Download the pretrained model file from [onnx/models](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn): - * commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117. +The instructions below are applicable **only** to the Mask R-CNN model converted to the ONNX file format from the `maskrcnn-benchmark model `__. + +1. Download the pretrained model file from `onnx/models `__ (commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117). 2. Generate the Intermediate Representation of the model by changing your current working directory to the Model Optimizer installation directory and running Model Optimizer with the following parameters: -```sh - mo \ ---input_model mask_rcnn_R_50_FPN_1x.onnx \ ---input "0:2" \ ---input_shape [1,3,800,800] \ ---mean_values [102.9801,115.9465,122.7717] \ ---transformations_config front/onnx/mask_rcnn.json -``` -Be aware that the height and width specified with the `input_shape` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the [documentation](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn). +.. code-block:: sh -3. Interpret the outputs of the generated IR file: masks, class indices, probabilities and box coordinates. - * masks. - * class indices. - * probabilities. - * box coordinates. + mo \ + --input_model mask_rcnn_R_50_FPN_1x.onnx \ + --input "0:2" \ + --input_shape [1,3,800,800] \ + --mean_values [102.9801,115.9465,122.7717] \ + --transformations_config front/onnx/mask_rcnn.json -The first one is a layer with the name `6849/sink_port_0`, and rest are outputs from the `DetectionOutput` layer. + +Be aware that the height and width specified with the ``input_shape`` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the `documentation `__. + +3. Interpret the outputs of the generated IR file: masks, class indices, probabilities and box coordinates: + + * masks + * class indices + * probabilities + * box coordinates + +The first one is a layer with the name ``6849/sink_port_0``, and rest are outputs from the ``DetectionOutput`` layer. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_AttentionOCR_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_AttentionOCR_From_Tensorflow.md index ced208fe3f3..6465ddc2abb 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_AttentionOCR_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_AttentionOCR_From_Tensorflow.md @@ -1,35 +1,49 @@ # Converting a TensorFlow Attention OCR Model {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_AttentionOCR_From_Tensorflow} -This tutorial explains how to convert the Attention OCR (AOCR) model from the [TensorFlow Attention OCR repository](https://github.com/emedvedev/attention-ocr) to the Intermediate Representation (IR). +@sphinxdirective -## Extracting a Model from `aocr` Library +This tutorial explains how to convert the Attention OCR (AOCR) model from the `TensorFlow Attention OCR repository `__ to the Intermediate Representation (IR). + +Extracting a Model from ``aocr`` Library +######################################## + +To get an AOCR model, download ``aocr`` Python library: + +.. code-block:: sh + + pip install git+https://github.com/emedvedev/attention-ocr.git@master#egg=aocr -To get an AOCR model, download `aocr` Python library: -``` -pip install git+https://github.com/emedvedev/attention-ocr.git@master#egg=aocr -``` This library contains a pretrained model and allows training and running AOCR, using the command line. After installation of `aocr`, extract the model: -``` -aocr export --format=frozengraph model/path/ -``` -Once extracted, the model can be found in `model/path/` folder. -## Converting the TensorFlow AOCR Model to IR +.. code-block:: sh + + aocr export --format=frozengraph model/path/ + +Once extracted, the model can be found in ``model/path/`` folder. + +Converting the TensorFlow AOCR Model to IR +########################################## The original AOCR model includes the preprocessing data, which contains: + * Decoding input data to binary format where input data is an image represented as a string. * Resizing binary image to working resolution. -The resized image is sent to the convolution neural network (CNN). Because Model Optimizer does not support image decoding, the preprocessing part of the model should be cut off, using the `--input` command-line parameter. -```sh -mo \ ---input_model=model/path/frozen_graph.pb \ ---input="map/TensorArrayStack/TensorArrayGatherV3:0[1,32,86,1]" \ ---output "transpose_1,transpose_2" \ ---output_dir path/to/ir/ -``` +The resized image is sent to the convolution neural network (CNN). Because Model Optimizer does not support image decoding, the preprocessing part of the model should be cut off, using the ``--input`` command-line parameter. + +.. code-block:: sh + + mo \ + --input_model=model/path/frozen_graph.pb \ + --input="map/TensorArrayStack/TensorArrayGatherV3:0[1,32,86,1]" \ + --output "transpose_1,transpose_2" \ + --output_dir path/to/ir/ + Where: -* `map/TensorArrayStack/TensorArrayGatherV3:0[1 32 86 1]` - name of node producing tensor after preprocessing. -* `transpose_1` - name of the node producing tensor with predicted characters. -* `transpose_2` - name of the node producing tensor with predicted characters probabilities. + +* ``map/TensorArrayStack/TensorArrayGatherV3:0[1 32 86 1]`` - name of node producing tensor after preprocessing. +* ``transpose_1`` - name of the node producing tensor with predicted characters. +* ``transpose_2`` - name of the node producing tensor with predicted characters probabilities. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md index b0ef95f1dd5..6d56a3c1d18 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md @@ -1,79 +1,109 @@ # Converting a TensorFlow BERT Model {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_BERT_From_Tensorflow} +@sphinxdirective + Pretrained models for BERT (Bidirectional Encoder Representations from Transformers) are -[publicly available](https://github.com/google-research/bert). +`publicly available `__. -## Supported Models +.. _supported_models: -The following models from the pretrained [BERT model list](https://github.com/google-research/bert#pre-trained-models) are currently supported: +Supported Models +################ -* `BERT-Base, Cased` -* `BERT-Base, Uncased` -* `BERT-Base, Multilingual Cased` -* `BERT-Base, Multilingual Uncased` -* `BERT-Base, Chinese` -* `BERT-Large, Cased` -* `BERT-Large, Uncased` +The following models from the pretrained `BERT model list `__ are currently supported: -## Downloading the Pretrained BERT Model +* ``BERT-Base, Cased`` +* ``BERT-Base, Uncased`` +* ``BERT-Base, Multilingual Cased`` +* ``BERT-Base, Multilingual Uncased`` +* ``BERT-Base, Chinese`` +* ``BERT-Large, Cased`` +* ``BERT-Large, Uncased`` -Download and unzip an archive with the [BERT-Base, Multilingual Uncased Model](https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip). +Downloading the Pretrained BERT Model +##################################### -After the archive is unzipped, the directory `uncased_L-12_H-768_A-12` is created and contains the following files: -* `bert_config.json` -* `bert_model.ckpt.data-00000-of-00001` -* `bert_model.ckpt.index` -* `bert_model.ckpt.meta` -* `vocab.txt` +Download and unzip an archive with the `BERT-Base, Multilingual Uncased Model `__. -Pretrained model meta-graph files are `bert_model.ckpt.*`. +After the archive is unzipped, the directory ``uncased_L-12_H-768_A-12`` is created and contains the following files: -## Converting a TensorFlow BERT Model to IR +* ``bert_config.json`` +* ``bert_model.ckpt.data-00000-of-00001`` +* ``bert_model.ckpt.index`` +* ``bert_model.ckpt.meta`` +* ``vocab.txt`` + +Pretrained model meta-graph files are ``bert_model.ckpt.*``. + +Converting a TensorFlow BERT Model to IR +######################################### To generate the BERT Intermediate Representation (IR) of the model, run Model Optimizer with the following parameters: -```sh - mo \ ---input_meta_graph uncased_L-12_H-768_A-12/bert_model.ckpt.meta \ ---output bert/pooler/dense/Tanh \ ---input Placeholder{i32},Placeholder_1{i32},Placeholder_2{i32} -``` + +.. code-block:: sh + + mo \ + --input_meta_graph uncased_L-12_H-768_A-12/bert_model.ckpt.meta \ + --output bert/pooler/dense/Tanh \ + --input Placeholder{i32},Placeholder_1{i32},Placeholder_2{i32} + Pretrained models are not suitable for batch reshaping out-of-the-box because of multiple hardcoded shapes in the model. -# Converting a Reshapable TensorFlow BERT Model to OpenVINO IR +Converting a Reshapable TensorFlow BERT Model to OpenVINO IR +============================================================= Follow these steps to make a pretrained TensorFlow BERT model reshapable over batch dimension: -1. Download a pretrained BERT model you want to use from the Supported Models list + +1. Download a pretrained BERT model you want to use from the `Supported Models list <#supported_models>`__. + 2. Clone google-research/bert git repository: -```sh -https://github.com/google-research/bert.git -``` -3. Go to the root directory of the cloned repository:
-```sh -cd bert -``` -4. (Optional) Checkout to the commit that the conversion was tested on:
-```sh -git checkout eedf5716c -``` + +.. code-block:: sh + + https://github.com/google-research/bert.git + +3. Go to the root directory of the cloned repository: + +.. code-block:: sh + + cd bert + +4. (Optional) Checkout to the commit that the conversion was tested on: + +.. code-block:: sh + + git checkout eedf5716c + 5. Download script to load GLUE data: - * For UNIX-like systems, run the following command: -```sh -wget https://gist.githubusercontent.com/W4ngatang/60c2bdb54d156a41194446737ce03e2e/raw/17b8dd0d724281ed7c3b2aeeda662b92809aadd5/download_glue_data.py -``` - * For Windows systems:
- Download the [Python script](https://gist.githubusercontent.com/W4ngatang/60c2bdb54d156a41194446737ce03e2e/raw/17b8dd0d724281ed7c3b2aeeda662b92809aadd5/download_glue_data.py) to the current working directory. + + * For UNIX-like systems, run the following command: + + .. code-block:: sh + + wget https://gist.githubusercontent.com/W4ngatang/60c2bdb54d156a41194446737ce03e2e/raw/17b8dd0d724281ed7c3b2aeeda662b92809aadd5/download_glue_data.py + + * For Windows systems: + + Download the `Python script `__ to the current working directory. + 6. Download GLUE data by running: -```sh -python3 download_glue_data.py --tasks MRPC -``` -7. Open the file `modeling.py` in the text editor and delete lines 923-924. They should look like this: -```python + +.. code-block:: sh + + python3 download_glue_data.py --tasks MRPC + +7. Open the file ``modeling.py`` in the text editor and delete lines 923-924. They should look like this: + +.. code-block:: python + if not non_static_indexes: return shape -``` -8. Open the file `run_classifier.py` and insert the following code after the line 645: -```python + +8. Open the file ``run_classifier.py`` and insert the following code after the line 645: + +.. code-block:: python + import os, sys import tensorflow as tf from tensorflow.python.framework import graph_io @@ -86,34 +116,42 @@ python3 download_glue_data.py --tasks MRPC graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False) print('BERT frozen model path {}'.format(os.path.join(os.path.dirname(__file__), 'inference_graph.pb'))) sys.exit(0) -``` + Lines before the inserted code should look like this: -```python + +.. code-block:: python + (total_loss, per_example_loss, logits, probabilities) = create_model( bert_config, is_training, input_ids, input_mask, segment_ids, label_ids, num_labels, use_one_hot_embeddings) -``` -9. Set environment variables `BERT_BASE_DIR`, `BERT_REPO_DIR` and run the script `run_classifier.py` to create `inference_graph.pb` file in the root of the cloned BERT repository. -```sh -export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12 -export BERT_REPO_DIR=/current/working/directory -python3 run_classifier.py \ - --task_name=MRPC \ - --do_eval=true \ - --data_dir=$BERT_REPO_DIR/glue_data/MRPC \ - --vocab_file=$BERT_BASE_DIR/vocab.txt \ - --bert_config_file=$BERT_BASE_DIR/bert_config.json \ - --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \ - --output_dir=./ -``` + +9. Set environment variables ``BERT_BASE_DIR``, ``BERT_REPO_DIR`` and run the script ``run_classifier.py`` to create ``inference_graph.pb`` file in the root of the cloned BERT repository. + +.. code-block:: sh + + export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12 + export BERT_REPO_DIR=/current/working/directory + + python3 run_classifier.py \ + --task_name=MRPC \ + --do_eval=true \ + --data_dir=$BERT_REPO_DIR/glue_data/MRPC \ + --vocab_file=$BERT_BASE_DIR/vocab.txt \ + --bert_config_file=$BERT_BASE_DIR/bert_config.json \ + --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \ + --output_dir=./ Run Model Optimizer with the following command line parameters to generate reshape-able BERT Intermediate Representation (IR): -```sh - mo \ ---input_model inference_graph.pb \ ---input "IteratorGetNext:0{i32}[1,128],IteratorGetNext:1{i32}[1,128],IteratorGetNext:4{i32}[1,128]" -``` -For other applicable parameters, refer to the [Convert Model from TensorFlow](../Convert_Model_From_TensorFlow.md) guide. -For more information about reshape abilities, refer to the [Using Shape Inference](../../../../OV_Runtime_UG/ShapeInference.md) guide. +.. code-block:: sh + + mo \ + --input_model inference_graph.pb \ + --input "IteratorGetNext:0{i32}[1,128],IteratorGetNext:1{i32}[1,128],IteratorGetNext:4{i32}[1,128]" + +For other applicable parameters, refer to the :doc:`Convert Model from TensorFlow ` guide. + +For more information about reshape abilities, refer to the :doc:`Using Shape Inference ` guide. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md index 68a399c6792..4eee022ef71 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md @@ -1,52 +1,77 @@ # Converting a TensorFlow CRNN Model {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_CRNN_From_Tensorflow} +@sphinxdirective + This tutorial explains how to convert a CRNN model to OpenVINO™ Intermediate Representation (IR). There are several public versions of TensorFlow CRNN model implementation available on GitHub. This tutorial explains how to convert the model from -the [CRNN Tensorflow](https://github.com/MaybeShewill-CV/CRNN_Tensorflow) repository to IR, and is validated with Python 3.7, TensorFlow 1.15.0, and protobuf 3.19.0. +the `CRNN Tensorflow `__ repository to IR, and is validated with Python 3.7, TensorFlow 1.15.0, and protobuf 3.19.0. If you have another implementation of CRNN model, it can be converted to OpenVINO IR in a similar way. You need to get inference graph and run Model Optimizer on it. **To convert the model to IR:** **Step 1.** Clone this GitHub repository and check out the commit: - 1. Clone the repository: -```sh -git clone https://github.com/MaybeShewill-CV/CRNN_Tensorflow.git -``` - 2. Go to the `CRNN_Tensorflow` directory of the cloned repository: -```sh -cd path/to/CRNN_Tensorflow -``` - 3. Check out the necessary commit: -```sh -git checkout 64f1f1867bffaacfeacc7a80eebf5834a5726122 -``` + +1. Clone the repository: + +.. code-block:: sh + + git clone https://github.com/MaybeShewill-CV/CRNN_Tensorflow.git + +2. Go to the ``CRNN_Tensorflow`` directory of the cloned repository: + +.. code-block:: sh + + cd path/to/CRNN_Tensorflow + +3. Check out the necessary commit: + +.. code-block:: sh + + git checkout 64f1f1867bffaacfeacc7a80eebf5834a5726122 + **Step 2.** Train the model using the framework or the pretrained checkpoint provided in this repository. + **Step 3.** Create an inference graph: - 1. Add the `CRNN_Tensorflow` folder to `PYTHONPATH`. - * For Linux: -```sh -export PYTHONPATH="${PYTHONPATH}:/path/to/CRNN_Tensorflow/" -``` - * For Windows, add `/path/to/CRNN_Tensorflow/` to the `PYTHONPATH` environment variable in settings. - 2. Edit the `tools/demo_shadownet.py` script. After `saver.restore(sess=sess, save_path=weights_path)` line, add the following code: -```python -from tensorflow.python.framework import graph_io -frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['shadow/LSTMLayers/transpose_time_major']) -graph_io.write_graph(frozen, '.', 'frozen_graph.pb', as_text=False) -``` - 3. Run the demo with the following command: -```sh -python tools/demo_shadownet.py --image_path data/test_images/test_01.jpg --weights_path model/shadownet/shadownet_2017-10-17-11-47-46.ckpt-199999 -``` - If you want to use your checkpoint, replace the path in the `--weights_path` parameter with a path to your checkpoint. - 4. In the `CRNN_Tensorflow` directory, you will find the inference CRNN graph `frozen_graph.pb`. You can use this graph with OpenVINO - to convert the model to IR and then run inference. + +1. Add the ``CRNN_Tensorflow`` folder to ``PYTHONPATH``. + + * For Linux: + + .. code-block:: sh + + export PYTHONPATH="${PYTHONPATH}:/path/to/CRNN_Tensorflow/" + + + * For Windows, add ``/path/to/CRNN_Tensorflow/`` to the ``PYTHONPATH`` environment variable in settings. + + + +2. Edit the ``tools/demo_shadownet.py`` script. After ``saver.restore(sess=sess, save_path=weights_path)`` line, add the following code: + +.. code-block:: python + + from tensorflow.python.framework import graph_io + frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['shadow/LSTMLayers/transpose_time_major']) + graph_io.write_graph(frozen, '.', 'frozen_graph.pb', as_text=False) + +3. Run the demo with the following command: + +.. code-block:: sh + + python tools/demo_shadownet.py --image_path data/test_images/test_01.jpg --weights_path model/shadownet/shadownet_2017-10-17-11-47-46.ckpt-199999 + + +If you want to use your checkpoint, replace the path in the ``--weights_path`` parameter with a path to your checkpoint. + +4. In the ``CRNN_Tensorflow`` directory, you will find the inference CRNN graph ``frozen_graph.pb``. You can use this graph with OpenVINO to convert the model to IR and then run inference. **Step 4.** Convert the model to IR: -```sh -mo --input_model path/to/your/CRNN_Tensorflow/frozen_graph.pb -``` +.. code-block:: sh + + mo --input_model path/to/your/CRNN_Tensorflow/frozen_graph.pb + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md index 24bc0b35d34..f024ec5465e 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md @@ -1,81 +1,99 @@ # Converting a TensorFlow DeepSpeech Model {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_DeepSpeech_From_Tensorflow} -[DeepSpeech project](https://github.com/mozilla/DeepSpeech) provides an engine to train speech-to-text models. +@sphinxdirective -## Downloading the Pretrained DeepSpeech Model +`DeepSpeech project `__ provides an engine to train speech-to-text models. + +Downloading the Pretrained DeepSpeech Model +########################################### Create a directory where model and metagraph with pretrained weights will be stored: -``` -mkdir deepspeech -cd deepspeech -``` -[Pretrained English speech-to-text model](https://github.com/mozilla/DeepSpeech/releases/tag/v0.8.2) is publicly available. + +.. code-block:: sh + + mkdir deepspeech + cd deepspeech + +`Pre-trained English speech-to-text model `__ is publicly available. To download the model, follow the instruction below: * For UNIX-like systems, run the following command: -``` -wget -O - https://github.com/mozilla/DeepSpeech/archive/v0.8.2.tar.gz | tar xvfz - -wget -O - https://github.com/mozilla/DeepSpeech/releases/download/v0.8.2/deepspeech-0.8.2-checkpoint.tar.gz | tar xvfz - -``` + +.. code-block:: sh + + wget -O - https://github.com/mozilla/DeepSpeech/archive/v0.8.2.tar.gz | tar xvfz - + wget -O - https://github.com/mozilla/DeepSpeech/releases/download/v0.8.2/deepspeech-0.8.2-checkpoint.tar.gz | tar xvfz - + * For Windows systems: - 1. Download [the archive with the model](https://github.com/mozilla/DeepSpeech/archive/v0.8.2.tar.gz). - 2. Download the [TensorFlow MetaGraph with pretrained weights](https://github.com/mozilla/DeepSpeech/releases/download/v0.8.2/deepspeech-0.8.2-checkpoint.tar.gz). + + 1. Download `the archive with the model `__. + 2. Download the `TensorFlow MetaGraph with pre-trained weights `__. 3. Unpack it with a file archiver application. -## Freezing the Model into a *.pb File +Freezing the Model into a *.pb File +################################### After unpacking the archives above, you have to freeze the model. This requires TensorFlow version 1, which is not available under Python 3.8, so you need Python 3.7 or lower. Before freezing, deploy a virtual environment and install the required packages: -``` -virtualenv --python=python3.7 venv-deep-speech -source venv-deep-speech/bin/activate -cd DeepSpeech-0.8.2 -pip3 install -e . -``` + +.. code-block:: sh + + virtualenv --python=python3.7 venv-deep-speech + source venv-deep-speech/bin/activate + cd DeepSpeech-0.8.2 + pip3 install -e . + Freeze the model with the following command: -``` -python3 DeepSpeech.py --checkpoint_dir ../deepspeech-0.8.2-checkpoint --export_dir ../ -``` -After that, you will get the pretrained frozen model file `output_graph.pb` in the directory `deepspeech` created at + +.. code-block:: sh + + python3 DeepSpeech.py --checkpoint_dir ../deepspeech-0.8.2-checkpoint --export_dir ../ + +After that, you will get the pretrained frozen model file ``output_graph.pb`` in the directory ``deepspeech`` created at the beginning. The model contains the preprocessing and main parts. The first preprocessing part performs conversion of input spectrogram into a form useful for speech recognition (mel). This part of the model is not convertible into -the IR because it contains unsupported operations `AudioSpectrogram` and `Mfcc`. +the IR because it contains unsupported operations ``AudioSpectrogram`` and ``Mfcc``. The main and most computationally expensive part of the model converts the preprocessed audio into text. There are two specificities with the supported part of the model. The first is that the model contains an input with sequence length. So the model can be converted with a fixed input length shape, thus the model is not reshapable. -Refer to the [Using Shape Inference](../../../../OV_Runtime_UG/ShapeInference.md) guide. +Refer to the :doc:`Using Shape Inference ` guide. -The second is that the frozen model still has two variables: `previous_state_c` and `previous_state_h`, figure +The second is that the frozen model still has two variables: ``previous_state_c`` and ``previous_state_h``, figure with the frozen *.pb model is below. It means that the model keeps training these variables at each inference. -![DeepSpeech model view](../../../img/DeepSpeech-0.8.2.png) +.. image:: ./_static/images/DeepSpeech-0.8.2.png -At the first inference, the variables are initialized with zero tensors. After execution, the results of the `BlockLSTM' +At the first inference, the variables are initialized with zero tensors. After execution, the results of the ``BlockLSTM`` are assigned to cell state and hidden state, which are these two variables. -## Converting the Main Part of DeepSpeech Model into OpenVINO IR +Converting the Main Part of DeepSpeech Model into OpenVINO IR +############################################################# -Model Optimizer assumes that the output model is for inference only. That is why you should cut `previous_state_c` -and `previous_state_h` variables off and resolve keeping cell and hidden states on the application level. +Model Optimizer assumes that the output model is for inference only. That is why you should cut ``previous_state_c`` and ``previous_state_h`` variables off and resolve keeping cell and hidden states on the application level. There are certain limitations for the model conversion: -- Time length (`time_len`) and sequence length (`seq_len`) are equal. -- Original model cannot be reshaped, so you should keep original shapes. + +* Time length (``time_len``) and sequence length (``seq_len``) are equal. +* Original model cannot be reshaped, so you should keep original shapes. To generate the IR, run Model Optimizer with the following parameters: -```sh -mo \ ---input_model output_graph.pb \ ---input "input_lengths->[16],input_node[1,16,19,26],previous_state_h[1,2048],previous_state_c[1,2048]" \ ---output "cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd_1,cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd,logits" -``` + +.. code-block:: sh + + mo \ + --input_model output_graph.pb \ + --input "input_lengths->[16],input_node[1,16,19,26],previous_state_h[1,2048],previous_state_c[1,2048]" \ + --output "cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd_1,cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd,logits" + Where: -* `input_lengths->[16]` Replaces the input node with name "input_lengths" with a constant tensor of shape [1] with a - single integer value of 16. This means that the model now can consume input sequences of length 16 only. -* `input_node[1 16 19 26],previous_state_h[1 2048],previous_state_c[1 2048]` replaces the variables with a placeholder. -* `--output ".../GatherNd_1,.../GatherNd,logits" ` output node names. + +* ``input_lengths->[16]`` Replaces the input node with name "input_lengths" with a constant tensor of shape [1] with a single integer value of 16. This means that the model now can consume input sequences of length 16 only. +* ``input_node[1 16 19 26],previous_state_h[1 2048],previous_state_c[1 2048]`` replaces the variables with a placeholder. +* ``--output ".../GatherNd_1,.../GatherNd,logits"`` output node names. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_EfficientDet_Models.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_EfficientDet_Models.md index 7c28307fa4e..be7df6181e3 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_EfficientDet_Models.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_EfficientDet_Models.md @@ -1,93 +1,121 @@ # Converting TensorFlow EfficientDet Models {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_EfficientDet_Models} +@sphinxdirective + This tutorial explains how to convert EfficientDet public object detection models to the Intermediate Representation (IR). -## Converting EfficientDet Model to the IR +.. _efficientdet-to-ir: + +Converting EfficientDet Model to the IR +####################################### There are several public versions of EfficientDet model implementation available on GitHub. This tutorial explains how to -convert models from the [repository](https://github.com/google/automl/tree/master/efficientdet) - (commit 96e1fee) to the OpenVINO format. +convert models from the `repository `__ (commit 96e1fee) to the OpenVINO format. -### Getting a Frozen TensorFlow Model +Getting a Frozen TensorFlow Model ++++++++++++++++++++++++++++++++++ Follow the instructions below to get frozen TensorFlow EfficientDet model. EfficientDet-D4 model is an example: -1. Clone the repository:
-```sh -git clone https://github.com/google/automl -cd automl/efficientdet -``` -2. (Optional) Checkout to the commit that the conversion was tested on:
-```sh -git checkout 96e1fee -``` -3. Install required dependencies:
-```sh -python3 -m pip install --upgrade pip -python3 -m pip install -r requirements.txt -python3 -m pip install --upgrade tensorflow-model-optimization -``` -4. Download and extract the model checkpoint [efficientdet-d4.tar.gz](https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco2/efficientdet-d4.tar.gz) -referenced in the **"Pretrained EfficientDet Checkpoints"** section of the model repository:
-```sh -wget https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco2/efficientdet-d4.tar.gz -tar zxvf efficientdet-d4.tar.gz -``` -5. Freeze the model:
-```sh - mo --runmode=saved_model --model_name=efficientdet-d4 --ckpt_path=efficientdet-d4 --saved_model_dir=savedmodeldir -``` -As a result, the frozen model file `savedmodeldir/efficientdet-d4_frozen.pb` will be generated. +1. Clone the repository: -> **NOTE**: For custom trained models, specify `--hparams` flag to `config.yaml` which was used during training. +.. code-block:: sh -> **NOTE**: If you see an error *AttributeError: module 'tensorflow_core.python.keras.api._v2.keras.initializers' has no attribute 'variance_scaling'*, apply the fix from the [patch](https://github.com/google/automl/pull/846). + git clone https://github.com/google/automl + cd automl/efficientdet -### Converting an EfficientDet TensorFlow Model to the IR +2. (Optional) Checkout to the commit that the conversion was tested on: -To generate the IR of the EfficientDet TensorFlow model, run:
-```sh -mo \ ---input_model savedmodeldir/efficientdet-d4_frozen.pb \ ---transformations_config front/tf/automl_efficientdet.json \ ---input_shape [1,$IMAGE_SIZE,$IMAGE_SIZE,3] \ ---reverse_input_channels -``` +.. code-block:: sh -Where `$IMAGE_SIZE` is the size that the input image of the original TensorFlow model will be resized to. Different -EfficientDet models were trained with different input image sizes. To determine the right one, refer to the `efficientdet_model_param_dict` -dictionary in the [hparams_config.py](https://github.com/google/automl/blob/96e1fee/efficientdet/hparams_config.py#L304) file. -The attribute `image_size` specifies the shape to be defined for the model conversion. + git checkout 96e1fee -The `transformations_config` command line parameter specifies the configuration json file containing hints -for the Model Optimizer on how to convert the model and trigger transformations implemented in the -`/openvino/tools/mo/front/tf/AutomlEfficientDet.py`. The json file contains some parameters which must be changed if you -train the model yourself and modified the `hparams_config` file or the parameters are different from the ones used for EfficientDet-D4. -The attribute names are self-explanatory or match the name in the `hparams_config` file. +3. Install required dependencies: -> **NOTE**: The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the `RGB<->BGR` conversion specifying the command-line parameter: `--reverse_input_channels`. Otherwise, inference results may be incorrect. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the [Converting a Model to Intermediate Representation (IR)](@ref openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model) guide. +.. code-block:: sh + + python3 -m pip install --upgrade pip + python3 -m pip install -r requirements.txt + python3 -m pip install --upgrade tensorflow-model-optimization + +4. Download and extract the model checkpoint `efficientdet-d4.tar.gz `__ +referenced in the **"Pretrained EfficientDet Checkpoints"** section of the model repository: + +.. code-block:: sh + + wget https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco2/efficientdet-d4.tar.gz + tar zxvf efficientdet-d4.tar.gz + +5. Freeze the model: + +.. code-block:: sh + + mo --runmode=saved_model --model_name=efficientdet-d4 --ckpt_path=efficientdet-d4 --saved_model_dir=savedmodeldir + +As a result, the frozen model file ``savedmodeldir/efficientdet-d4_frozen.pb`` will be generated. + +.. note:: + + For custom trained models, specify ``--hparams`` flag to ``config.yaml`` which was used during training. + +.. note:: + + If you see an error *AttributeError: module 'tensorflow_core.python.keras.api._v2.keras.initializers' has no attribute 'variance_scaling'*, apply the fix from the `patch `__. + +Converting an EfficientDet TensorFlow Model to the IR ++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +To generate the IR of the EfficientDet TensorFlow model, run: + +.. code-block:: sh + + mo \ + --input_model savedmodeldir/efficientdet-d4_frozen.pb \ + --transformations_config front/tf/automl_efficientdet.json \ + --input_shape [1,$IMAGE_SIZE,$IMAGE_SIZE,3] \ + --reverse_input_channels + + +Where ``$IMAGE_SIZE`` is the size that the input image of the original TensorFlow model will be resized to. Different +EfficientDet models were trained with different input image sizes. To determine the right one, refer to the ``efficientdet_model_param_dict`` +dictionary in the `hparams_config.py `__ file. +The attribute ``image_size`` specifies the shape to be defined for the model conversion. + +The ``transformations_config`` command line parameter specifies the configuration json file containing hints for the Model Optimizer on how to convert the model and trigger transformations implemented in the ``/openvino/tools/mo/front/tf/AutomlEfficientDet.py``. The json file contains some parameters which must be changed if you +train the model yourself and modified the ``hparams_config`` file or the parameters are different from the ones used for EfficientDet-D4. +The attribute names are self-explanatory or match the name in the ``hparams_config`` file. + +.. note:: + + The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the ``RGB<->BGR`` conversion specifying the command-line parameter: ``--reverse_input_channels``. Otherwise, inference results may be incorrect. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the :doc:`Converting a Model to Intermediate Representation (IR) ` guide. OpenVINO toolkit provides samples that can be used to infer EfficientDet model. -For more information, refer to the [Open Model Zoo Demos](@ref omz_demos). +For more information, refer to the :doc:`Open Model Zoo Demos `. -## Interpreting Results of the TensorFlow Model and the IR +Interpreting Results of the TensorFlow Model and the IR +####################################################### -The TensorFlow model produces as output a list of 7-element tuples: `[image_id, y_min, x_min, y_max, x_max, confidence, class_id]`, where: -* `image_id` -- image batch index. -* `y_min` -- absolute `y` coordinate of the lower left corner of the detected object. -* `x_min` -- absolute `x` coordinate of the lower left corner of the detected object. -* `y_max` -- absolute `y` coordinate of the upper right corner of the detected object. -* `x_max` -- absolute `x` coordinate of the upper right corner of the detected object. -* `confidence` -- is the confidence of the detected object. -* `class_id` -- is the id of the detected object class counted from 1. +The TensorFlow model produces as output a list of 7-element tuples: ``[image_id, y_min, x_min, y_max, x_max, confidence, class_id]``, where: -The output of the IR is a list of 7-element tuples: `[image_id, class_id, confidence, x_min, y_min, x_max, y_max]`, where: -* `image_id` -- image batch index. -* `class_id` -- is the id of the detected object class counted from 0. -* `confidence` -- is the confidence of the detected object. -* `x_min` -- normalized `x` coordinate of the lower left corner of the detected object. -* `y_min` -- normalized `y` coordinate of the lower left corner of the detected object. -* `x_max` -- normalized `x` coordinate of the upper right corner of the detected object. -* `y_max` -- normalized `y` coordinate of the upper right corner of the detected object. +* ``image_id`` -- image batch index. +* ``y_min`` -- absolute ``y`` coordinate of the lower left corner of the detected object. +* ``x_min`` -- absolute ``x`` coordinate of the lower left corner of the detected object. +* ``y_max`` -- absolute ``y`` coordinate of the upper right corner of the detected object. +* ``x_max`` -- absolute ``x`` coordinate of the upper right corner of the detected object. +* ``confidence`` -- the confidence of the detected object. +* ``class_id`` -- the id of the detected object class counted from 1. -The first element with `image_id = -1` means end of data. +The output of the IR is a list of 7-element tuples: ``[image_id, class_id, confidence, x_min, y_min, x_max, y_max]``, where: + +* ``image_id`` -- image batch index. +* ``class_id`` -- the id of the detected object class counted from 0. +* ``confidence`` -- the confidence of the detected object. +* ``x_min`` -- normalized ``x`` coordinate of the lower left corner of the detected object. +* ``y_min`` -- normalized ``y`` coordinate of the lower left corner of the detected object. +* ``x_max`` -- normalized ``x`` coordinate of the upper right corner of the detected object. +* ``y_max`` -- normalized ``y`` coordinate of the upper right corner of the detected object. + +The first element with ``image_id = -1`` means end of data. + + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_FaceNet_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_FaceNet_From_Tensorflow.md index 5e987b1bcd1..aa7b3056c8b 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_FaceNet_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_FaceNet_From_Tensorflow.md @@ -1,28 +1,33 @@ # Converting TensorFlow FaceNet Models {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_FaceNet_From_Tensorflow} -[Public pretrained FaceNet models](https://github.com/davidsandberg/facenet#pre-trained-models) contain both training +@sphinxdirective + +`Public pre-trained FaceNet models `__ contain both training and inference part of graph. Switch between this two states is manageable with placeholder value. Intermediate Representation (IR) models are intended for inference, which means that train part is redundant. -There are two inputs in this network: boolean `phase_train` which manages state of the graph (train/infer) and -`batch_size` which is a part of batch joining pattern. +There are two inputs in this network: boolean ``phase_train`` which manages state of the graph (train/infer) and +``batch_size`` which is a part of batch joining pattern. +.. image:: ./_static/images/FaceNet.svg -![FaceNet model view](../../../img/FaceNet.svg) - -## Converting a TensorFlow FaceNet Model to the IR +Converting a TensorFlow FaceNet Model to the IR +############################################### To generate a FaceNet OpenVINO model, feed a TensorFlow FaceNet model to Model Optimizer with the following parameters: -```sh - mo ---input_model path_to_model/model_name.pb \ ---freeze_placeholder_with_value "phase_train->False" -``` -The batch joining pattern transforms to a placeholder with the model default shape if `--input_shape` or `--batch`*/*`-b` are not -provided. Otherwise, the placeholder shape has custom parameters. +.. code-block:: sh -* `--freeze_placeholder_with_value "phase_train->False"` to switch graph to inference mode -* `--batch`*/*`-b` is applicable to override original network batch -* `--input_shape` is applicable with or without `--input` + mo + --input_model path_to_model/model_name.pb \ + --freeze_placeholder_with_value "phase_train->False" + + +The batch joining pattern transforms to a placeholder with the model default shape if ``--input_shape`` or ``--batch`*/*`-b`` are not provided. Otherwise, the placeholder shape has custom parameters. + +* ``--freeze_placeholder_with_value "phase_train->False"`` to switch graph to inference mode +* ``--batch`*/*`-b`` is applicable to override original network batch +* ``--input_shape`` is applicable with or without ``--input`` * other options are applicable + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_GNMT_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_GNMT_From_Tensorflow.md index 6f984f53b1d..0cb950cf94a 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_GNMT_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_GNMT_From_Tensorflow.md @@ -1,276 +1,304 @@ # Converting a TensorFlow GNMT Model {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_GNMT_From_Tensorflow} +@sphinxdirective + This tutorial explains how to convert Google Neural Machine Translation (GNMT) model to the Intermediate Representation (IR). -There are several public versions of TensorFlow GNMT model implementation available on GitHub. This tutorial explains how to convert the GNMT model from the [TensorFlow Neural Machine Translation (NMT) repository](https://github.com/tensorflow/nmt) to the IR. +There are several public versions of TensorFlow GNMT model implementation available on GitHub. This tutorial explains how to convert the GNMT model from the `TensorFlow Neural Machine Translation (NMT) repository `__ to the IR. -## Creating a Patch File +Creating a Patch File +##################### Before converting the model, you need to create a patch file for the repository. The patch modifies the framework code by adding a special command-line argument to the framework options that enables inference graph dumping: -1. Go to a writable directory and create a `GNMT_inference.patch` file. +1. Go to a writable directory and create a ``GNMT_inference.patch`` file. 2. Copy the following diff code to the file: -```git -diff --git a/nmt/inference.py b/nmt/inference.py -index 2cbef07..e185490 100644 ---- a/nmt/inference.py -+++ b/nmt/inference.py -@@ -17,9 +17,11 @@ - from __future__ import print_function - import codecs -+import os - import time +.. code-block:: cpp - import tensorflow as tf -+from tensorflow.python.framework import graph_io + diff --git a/nmt/inference.py b/nmt/inference.py + index 2cbef07..e185490 100644 + --- a/nmt/inference.py + +++ b/nmt/inference.py + @@ -17,9 +17,11 @@ + from __future__ import print_function - from . import attention_model - from . import gnmt_model -@@ -105,6 +107,29 @@ def start_sess_and_load_model(infer_model, ckpt_path): - return sess, loaded_infer_model + import codecs + +import os + import time + + import tensorflow as tf + +from tensorflow.python.framework import graph_io + + from . import attention_model + from . import gnmt_model + @@ -105,6 +107,29 @@ def start_sess_and_load_model(infer_model, ckpt_path): + return sess, loaded_infer_model -+def inference_dump_graph(ckpt_path, path_to_dump, hparams, scope=None): -+ model_creator = get_model_creator(hparams) -+ infer_model = model_helper.create_infer_model(model_creator, hparams, scope) -+ sess = tf.Session( -+ graph=infer_model.graph, config=utils.get_config_proto()) -+ with infer_model.graph.as_default(): -+ loaded_infer_model = model_helper.load_model( -+ infer_model.model, ckpt_path, sess, "infer") -+ utils.print_out("Dumping inference graph to {}".format(path_to_dump)) -+ loaded_infer_model.saver.save( -+ sess, -+ os.path.join(path_to_dump + 'inference_GNMT_graph') -+ ) -+ utils.print_out("Dumping done!") -+ -+ output_node_name = 'index_to_string_Lookup' -+ utils.print_out("Freezing GNMT graph with output node {}...".format(output_node_name)) -+ frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, -+ [output_node_name]) -+ graph_io.write_graph(frozen, '.', os.path.join(path_to_dump, 'frozen_GNMT_inference_graph.pb'), as_text=False) -+ utils.print_out("Freezing done. Freezed model frozen_GNMT_inference_graph.pb saved to {}".format(path_to_dump)) -+ -+ - def inference(ckpt_path, - inference_input_file, - inference_output_file, -diff --git a/nmt/nmt.py b/nmt/nmt.py -index f5823d8..a733748 100644 ---- a/nmt/nmt.py -+++ b/nmt/nmt.py -@@ -310,6 +310,13 @@ def add_arguments(parser): - parser.add_argument("--num_intra_threads", type=int, default=0, - help="number of intra_op_parallelism_threads") + +def inference_dump_graph(ckpt_path, path_to_dump, hparams, scope=None): + + model_creator = get_model_creator(hparams) + + infer_model = model_helper.create_infer_model(model_creator, hparams, scope) + + sess = tf.Session( + + graph=infer_model.graph, config=utils.get_config_proto()) + + with infer_model.graph.as_default(): + + loaded_infer_model = model_helper.load_model( + + infer_model.model, ckpt_path, sess, "infer") + + utils.print_out("Dumping inference graph to {}".format(path_to_dump)) + + loaded_infer_model.saver.save( + + sess, + + os.path.join(path_to_dump + 'inference_GNMT_graph') + + ) + + utils.print_out("Dumping done!") + + + + output_node_name = 'index_to_string_Lookup' + + utils.print_out("Freezing GNMT graph with output node {}...".format(output_node_name)) + + frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, + + [output_node_name]) + + graph_io.write_graph(frozen, '.', os.path.join(path_to_dump, 'frozen_GNMT_inference_graph.pb'), as_text=False) + + utils.print_out("Freezing done. Freezed model frozen_GNMT_inference_graph.pb saved to {}".format(path_to_dump)) + + + + + def inference(ckpt_path, + inference_input_file, + inference_output_file, + diff --git a/nmt/nmt.py b/nmt/nmt.py + index f5823d8..a733748 100644 + --- a/nmt/nmt.py + +++ b/nmt/nmt.py + @@ -310,6 +310,13 @@ def add_arguments(parser): + parser.add_argument("--num_intra_threads", type=int, default=0, + help="number of intra_op_parallelism_threads") -+ # Special argument for inference model dumping without inference -+ parser.add_argument("--dump_inference_model", type="bool", nargs="?", -+ const=True, default=False, -+ help="Argument for dump inference graph for specified trained ckpt") -+ -+ parser.add_argument("--path_to_dump", type=str, default="", -+ help="Path to dump inference graph.") + + # Special argument for inference model dumping without inference + + parser.add_argument("--dump_inference_model", type="bool", nargs="?", + + const=True, default=False, + + help="Argument for dump inference graph for specified trained ckpt") + + + + parser.add_argument("--path_to_dump", type=str, default="", + + help="Path to dump inference graph.") - def create_hparams(flags): - """Create training hparams.""" -@@ -396,6 +403,9 @@ def create_hparams(flags): - language_model=flags.language_model, - num_intra_threads=flags.num_intra_threads, - num_inter_threads=flags.num_inter_threads, -+ -+ dump_inference_model=flags.dump_inference_model, -+ path_to_dump=flags.path_to_dump, - ) + def create_hparams(flags): + """Create training hparams.""" + @@ -396,6 +403,9 @@ def create_hparams(flags): + language_model=flags.language_model, + num_intra_threads=flags.num_intra_threads, + num_inter_threads=flags.num_inter_threads, + + + + dump_inference_model=flags.dump_inference_model, + + path_to_dump=flags.path_to_dump, + ) -@@ -613,7 +623,7 @@ def create_or_load_hparams( - return hparams + @@ -613,7 +623,7 @@ def create_or_load_hparams( + return hparams --def run_main(flags, default_hparams, train_fn, inference_fn, target_session=""): -+def run_main(flags, default_hparams, train_fn, inference_fn, inference_dump, target_session=""): - """Run main.""" - # Job - jobid = flags.jobid -@@ -653,8 +663,26 @@ def run_main(flags, default_hparams, train_fn, inference_fn, target_session=""): - out_dir, default_hparams, flags.hparams_path, - save_hparams=(jobid == 0)) + -def run_main(flags, default_hparams, train_fn, inference_fn, target_session=""): + +def run_main(flags, default_hparams, train_fn, inference_fn, inference_dump, target_session=""): + """Run main.""" + # Job + jobid = flags.jobid + @@ -653,8 +663,26 @@ def run_main(flags, default_hparams, train_fn, inference_fn, target_session=""): + out_dir, default_hparams, flags.hparams_path, + save_hparams=(jobid == 0)) -- ## Train / Decode -- if flags.inference_input_file: -+ # Dumping inference model -+ if flags.dump_inference_model: -+ # Inference indices -+ hparams.inference_indices = None -+ if flags.inference_list: -+ (hparams.inference_indices) = ( -+ [int(token) for token in flags.inference_list.split(",")]) -+ -+ # Ckpt -+ ckpt = flags.ckpt -+ if not ckpt: -+ ckpt = tf.train.latest_checkpoint(out_dir) -+ -+ # Path to dump graph -+ assert flags.path_to_dump != "", "Please, specify path_to_dump model." -+ path_to_dump = flags.path_to_dump -+ if not tf.gfile.Exists(path_to_dump): tf.gfile.MakeDirs(path_to_dump) -+ -+ inference_dump(ckpt, path_to_dump, hparams) -+ elif flags.inference_input_file: - # Inference output directory - trans_file = flags.inference_output_file - assert trans_file -@@ -693,7 +721,8 @@ def main(unused_argv): - default_hparams = create_hparams(FLAGS) - train_fn = train.train - inference_fn = inference.inference -- run_main(FLAGS, default_hparams, train_fn, inference_fn) -+ inference_dump = inference.inference_dump_graph -+ run_main(FLAGS, default_hparams, train_fn, inference_fn, inference_dump) + - ## Train / Decode + - if flags.inference_input_file: + + # Dumping inference model + + if flags.dump_inference_model: + + # Inference indices + + hparams.inference_indices = None + + if flags.inference_list: + + (hparams.inference_indices) = ( + + [int(token) for token in flags.inference_list.split(",")]) + + + + # Ckpt + + ckpt = flags.ckpt + + if not ckpt: + + ckpt = tf.train.latest_checkpoint(out_dir) + + + + # Path to dump graph + + assert flags.path_to_dump != "", "Please, specify path_to_dump model." + + path_to_dump = flags.path_to_dump + + if not tf.gfile.Exists(path_to_dump): tf.gfile.MakeDirs(path_to_dump) + + + + inference_dump(ckpt, path_to_dump, hparams) + + elif flags.inference_input_file: + # Inference output directory + trans_file = flags.inference_output_file + assert trans_file + @@ -693,7 +721,8 @@ def main(unused_argv): + default_hparams = create_hparams(FLAGS) + train_fn = train.train + inference_fn = inference.inference + - run_main(FLAGS, default_hparams, train_fn, inference_fn) + + inference_dump = inference.inference_dump_graph + + run_main(FLAGS, default_hparams, train_fn, inference_fn, inference_dump) - if __name__ == "__main__": + if __name__ == "__main__": + -``` 3. Save and close the file. -## Converting a GNMT Model to the IR +Converting a GNMT Model to the IR +################################# -> **NOTE**: Use TensorFlow version 1.13 or lower. +.. note:: Use TensorFlow version 1.13 or lower. **Step 1**. Clone the GitHub repository and check out the commit: 1. Clone the NMT reposirory: -```sh -git clone https://github.com/tensorflow/nmt.git -``` + +.. code-block:: sh + + git clone https://github.com/tensorflow/nmt.git + 2. Check out the necessary commit: -```sh -git checkout b278487980832417ad8ac701c672b5c3dc7fa553 -``` + +.. code-block:: sh + + git checkout b278487980832417ad8ac701c672b5c3dc7fa553 + **Step 2**. Get a trained model. You have two options: -* Train the model with the GNMT `wmt16_gnmt_4_layer.json` or `wmt16_gnmt_8_layer.json` configuration file using the NMT framework. +* Train the model with the GNMT ``wmt16_gnmt_4_layer.json`` or ``wmt16_gnmt_8_layer.json`` configuration file using the NMT framework. * *Do not use the pre-trained checkpoints provided in the NMT repository, as they are outdated and can be incompatible with the current repository version.* -This tutorial assumes the use of the trained GNMT model from `wmt16_gnmt_4_layer.json` config, German to English translation. +This tutorial assumes the use of the trained GNMT model from ``wmt16_gnmt_4_layer.json`` config, German to English translation. **Step 3**. Create an inference graph: The OpenVINO assumes that a model is used for inference only. Hence, before converting the model into the IR, you need to transform the training graph into the inference graph. For the GNMT model, the training graph and the inference graph have different decoders: the training graph uses a greedy search decoding algorithm, while the inference graph uses a beam search decoding algorithm. -1. Apply the `GNMT_inference.patch` patch to the repository. Refer to the Create a Patch File instructions if you do not have it: -```sh - git apply /path/to/patch/GNMT_inference.patch -``` +1. Apply the ``GNMT_inference.patch`` patch to the repository. `Create a Patch File <#Creating-a-Patch-File>`__ instructions if you do not have it: + +.. code-block:: sh + + git apply /path/to/patch/GNMT_inference.patch + 2. Run the NMT framework to dump the inference model: -```sh -python -m nmt.nmt - --src=de - --tgt=en - --ckpt=/path/to/ckpt/translate.ckpt - --hparams_path=/path/to/repository/nmt/nmt/standard_hparams/wmt16_gnmt_4_layer.json - --vocab_prefix=/path/to/vocab/vocab.bpe.32000 - --out_dir="" - --dump_inference_model - --infer_mode beam_search - --path_to_dump /path/to/dump/model/ -``` +.. code-block:: sh -If you use different checkpoints, use the corresponding values for the `src`,`tgt`,`ckpt`,`hparams_path`, and `vocab_prefix` parameters. -Inference checkpoint `inference_GNMT_graph` and frozen inference graph `frozen_GNMT_inference_graph.pb` will appear in the `/path/to/dump/model/` folder. + python -m nmt.nmt + --src=de + --tgt=en + --ckpt=/path/to/ckpt/translate.ckpt + --hparams_path=/path/to/repository/nmt/nmt/standard_hparams/wmt16_gnmt_4_layer.json + --vocab_prefix=/path/to/vocab/vocab.bpe.32000 + --out_dir="" + --dump_inference_model + --infer_mode beam_search + --path_to_dump /path/to/dump/model/ + + +If you use different checkpoints, use the corresponding values for the ``src``, ``tgt``, ``ckpt``, ``hparams_path``, and ``vocab_prefix`` parameters. +Inference checkpoint ``inference_GNMT_graph`` and frozen inference graph ``frozen_GNMT_inference_graph.pb`` will appear in the ``/path/to/dump/model/`` folder. + +To generate ``vocab.bpe.32000``, execute the ``nmt/scripts/wmt16_en_de.sh`` script. If you face an issue of a size mismatch between the checkpoint graph's embedding layer and vocabulary (both src and target), make sure you add the following code to the ``nmt.py`` file to the ``extend_hparams`` function after the line 508 (after initialization of the ``src_vocab_size`` and ``tgt_vocab_size`` variables): + +.. code-block:: py + + src_vocab_size -= 1 + tgt_vocab_size -= 1 -To generate `vocab.bpe.32000`, execute the `nmt/scripts/wmt16_en_de.sh` script. If you face an issue of a size mismatch between the checkpoint graph's embedding layer and vocabulary (both src and target), make sure you add the following code to the `nmt.py` file to the `extend_hparams` function after the line 508 (after initialization of the `src_vocab_size` and `tgt_vocab_size` variables): -```py -src_vocab_size -= 1 -tgt_vocab_size -= 1 -``` **Step 4**. Convert the model to the IR: -```sh -mo ---input_model /path/to/dump/model/frozen_GNMT_inference_graph.pb ---input "IteratorGetNext:1{i32}[1],IteratorGetNext:0{i32}[1,50],dynamic_seq2seq/hash_table_Lookup_1:0[1]->[2],dynamic_seq2seq/hash_table_Lookup:0[1]->[1]" ---output dynamic_seq2seq/decoder/decoder/GatherTree ---output_dir /path/to/output/IR/ -``` +.. code-block:: sh -Input and output cutting with the `--input` and `--output` options is required since OpenVINO™ does not support `IteratorGetNext` and `LookupTableFindV2` operations. + mo + --input_model /path/to/dump/model/frozen_GNMT_inference_graph.pb + --input "IteratorGetNext:1{i32}[1],IteratorGetNext:0{i32}[1,50],dynamic_seq2seq/hash_table_Lookup_1:0[1]->[2],dynamic_seq2seq/hash_table_Lookup:0[1]->[1]" + --output dynamic_seq2seq/decoder/decoder/GatherTree + --output_dir /path/to/output/IR/ + + +Input and output cutting with the ``--input`` and ``--output`` options is required since OpenVINO™ does not support ``IteratorGetNext`` and ``LookupTableFindV2`` operations. Input cutting: -* `IteratorGetNext` operation iterates over a dataset. It is cut by output ports: port 0 contains data tensor with shape `[batch_size, max_sequence_length]`, port 1 contains `sequence_length` for every batch with shape `[batch_size]`. +* ``IteratorGetNext`` operation iterates over a dataset. It is cut by output ports: port 0 contains data tensor with shape ``[batch_size, max_sequence_length]``, port 1 contains ``sequence_length`` for every batch with shape ``[batch_size]``. -* `LookupTableFindV2` operations (`dynamic_seq2seq/hash_table_Lookup_1` and `dynamic_seq2seq/hash_table_Lookup` nodes in the graph) are cut with constant values). +* ``LookupTableFindV2`` operations (``dynamic_seq2seq/hash_table_Lookup_1`` and ``dynamic_seq2seq/hash_table_Lookup`` nodes in the graph) are cut with constant values). Output cutting: -* `LookupTableFindV2` operation is cut from the output and the `dynamic_seq2seq/decoder/decoder/GatherTree` node is treated as a new exit point. +* ``LookupTableFindV2`` operation is cut from the output and the ``dynamic_seq2seq/decoder/decoder/GatherTree`` node is treated as a new exit point. -For more information about model cutting, refer to the [Cutting Off Parts of a Model](@ref openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model) guide. +For more information about model cutting, refer to the :doc:`Cutting Off Parts of a Model ` guide. -## Using a GNMT Model +Using a GNMT Model +################## -> **NOTE**: This step assumes you have converted a model to the Intermediate Representation. +.. note:: + + This step assumes you have converted a model to the Intermediate Representation. Inputs of the model: -* `IteratorGetNext/placeholder_out_port_0` input with shape `[batch_size, max_sequence_length]` contains `batch_size` decoded input sentences. - Every sentence is decoded the same way as indices of sentence elements in vocabulary and padded with index of `eos` (end of sentence symbol). If the length of the sentence is less than `max_sequence_length`, remaining elements are filled with index of `eos` token. -* `IteratorGetNext/placeholder_out_port_1` input with shape `[batch_size]` contains sequence lengths for every sentence from the first input. - For example, if `max_sequence_length = 50`, `batch_size = 1` and the sentence has only 30 elements, then the input tensor for `IteratorGetNext/placeholder_out_port_1` should be `[30]`. +* ``IteratorGetNext/placeholder_out_port_0`` input with shape ``[batch_size, max_sequence_length]`` contains ``batch_size`` decoded input sentences. Every sentence is decoded the same way as indices of sentence elements in vocabulary and padded with index of ``eos`` (end of sentence symbol). If the length of the sentence is less than ``max_sequence_length``, remaining elements are filled with index of ``eos`` token. + +* ``IteratorGetNext/placeholder_out_port_1`` input with shape ``[batch_size]`` contains sequence lengths for every sentence from the first input. For example, if ``max_sequence_length = 50``, ``batch_size = 1`` and the sentence has only 30 elements, then the input tensor for ``IteratorGetNext/placeholder_out_port_1`` should be ``[30]``. Outputs of the model: -* `dynamic_seq2seq/decoder/decoder/GatherTree` tensor with shape `[max_sequence_length * 2, batch, beam_size]`, - that contains `beam_size` best translations for every sentence from input (also decoded as indices of words in +* ``dynamic_seq2seq/decoder/decoder/GatherTree`` tensor with shape ``[max_sequence_length * 2, batch, beam_size]``, + that contains ``beam_size`` best translations for every sentence from input (also decoded as indices of words in vocabulary). -> **NOTE**: The shape of this tensor in TensorFlow can be different: instead of `max_sequence_length * 2`, it can be any value less than that, because OpenVINO does not support dynamic shapes of outputs, while TensorFlow can stop decoding iterations when `eos` symbol is generated. -#### Running GNMT IR +.. note:: + The shape of this tensor in TensorFlow can be different: instead of ``max_sequence_length * 2``, it can be any value less than that, because OpenVINO does not support dynamic shapes of outputs, while TensorFlow can stop decoding iterations when ``eos`` symbol is generated. + +Running GNMT IR +--------------- 1. With benchmark app: -```sh -benchmark_app -m -d CPU -``` + +.. code-block:: sh + + benchmark_app -m -d CPU + 2. With OpenVINO Runtime Python API: -> **NOTE**: Before running the example, insert a path to your GNMT `.xml` and `.bin` files into `MODEL_PATH` and `WEIGHTS_PATH`, and fill `input_data_tensor` and `seq_lengths` tensors according to your input data. +.. note:: -```python -from openvino.inference_engine import IENetwork, IECore + Before running the example, insert a path to your GNMT ``.xml`` and ``.bin`` files into ``MODEL_PATH`` and ``WEIGHTS_PATH``, and fill ``input_data_tensor`` and ``seq_lengths`` tensors according to your input data. -MODEL_PATH = '/path/to/IR/frozen_GNMT_inference_graph.xml' -WEIGHTS_PATH = '/path/to/IR/frozen_GNMT_inference_graph.bin' +.. code-block:: py -# Creating network -net = IENetwork( - model=MODEL_PATH, - weights=WEIGHTS_PATH) + from openvino.inference_engine import IENetwork, IECore -# Creating input data -input_data = {'IteratorGetNext/placeholder_out_port_0': input_data_tensor, - 'IteratorGetNext/placeholder_out_port_1': seq_lengths} + MODEL_PATH = '/path/to/IR/frozen_GNMT_inference_graph.xml' + WEIGHTS_PATH = '/path/to/IR/frozen_GNMT_inference_graph.bin' -# Creating plugin and loading extensions -ie = IECore() -ie.add_extension(extension_path="libcpu_extension.so", device_name="CPU") + # Creating network + net = IENetwork( + model=MODEL_PATH, + weights=WEIGHTS_PATH) -# Loading network -exec_net = ie.load_network(network=net, device_name="CPU") + # Creating input data + input_data = {'IteratorGetNext/placeholder_out_port_0': input_data_tensor, + 'IteratorGetNext/placeholder_out_port_1': seq_lengths} -# Run inference -result_ie = exec_net.infer(input_data) -``` + # Creating plugin and loading extensions + ie = IECore() + ie.add_extension(extension_path="libcpu_extension.so", device_name="CPU") -For more information about Python API, refer to the [OpenVINO Runtime Python API](https://docs.openvino.ai/latest/api/api_reference.html) guide. + # Loading network + exec_net = ie.load_network(network=net, device_name="CPU") + + # Run inference + result_ie = exec_net.infer(input_data) + + +For more information about Python API, refer to the `OpenVINO Runtime Python API `__ guide. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md index 5c7820f4cba..fb9951ddfad 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md @@ -1,49 +1,58 @@ # Converting a TensorFlow Neural Collaborative Filtering Model {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_NCF_From_Tensorflow} +@sphinxdirective + This tutorial explains how to convert Neural Collaborative Filtering (NCF) model to the OpenVINO Intermediate Representation. -[Public TensorFlow NCF model](https://github.com/tensorflow/models/tree/master/official/recommendation) does not contain pre-trained weights. To convert this model to the IR: - 1. Use [the instructions](https://github.com/tensorflow/models/tree/master/official/recommendation#train-and-evaluate-model) from this repository to train the model. - 2. Freeze the inference graph you get in the previous step in `model_dir`, following -the instructions from the **Freezing Custom Models in Python** section of the -[Converting a TensorFlow Model](../Convert_Model_From_TensorFlow.md) guide. +`Public TensorFlow NCF model `__ does not contain pre-trained weights. To convert this model to the IR: + +1. Use `the instructions `__ from this repository to train the model. + +2. Freeze the inference graph you get in the previous step in ``model_dir``, following the instructions from the **Freezing Custom Models in Python** section of the :doc:`Converting a TensorFlow Model ` guide. + Run the following commands: -```python -import tensorflow as tf -from tensorflow.python.framework import graph_io -sess = tf.compat.v1.Session() -saver = tf.compat.v1.train.import_meta_graph("/path/to/model/model.meta") -saver.restore(sess, tf.train.latest_checkpoint('/path/to/model/')) +.. code-block:: py -frozen = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph_def, \ - ["rating/BiasAdd"]) -graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False) -``` -where `rating/BiasAdd` is an output node. + import tensorflow as tf + from tensorflow.python.framework import graph_io - 3. Convert the model to the OpenVINO format. If you look at your frozen model, you can see that -it has one input that is split into four `ResourceGather` layers. (Click image to zoom in.) + sess = tf.compat.v1.Session() + saver = tf.compat.v1.train.import_meta_graph("/path/to/model/model.meta") + saver.restore(sess, tf.train.latest_checkpoint('/path/to/model/')) -![NCF model beginning](../../../img/NCF_start.svg) + frozen = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph_def, \ + ["rating/BiasAdd"]) + graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False) - However, as the Model Optimizer does not support such data feeding, you should skip it. Cut -the edges incoming in `ResourceGather` port 1: -```sh - mo --input_model inference_graph.pb \ ---input 1:embedding/embedding_lookup,1:embedding_1/embedding_lookup, \ -1:embedding_2/embedding_lookup,1:embedding_3/embedding_lookup \ ---input_shape [256],[256],[256],[256] \ ---output_dir -``` -In the `input_shape` parameter, 256 specifies the `batch_size` for your model. +where ``rating/BiasAdd`` is an output node. + +3. Convert the model to the OpenVINO format. If you look at your frozen model, you can see that +it has one input that is split into four ``ResourceGather`` layers. (Click image to zoom in.) + +.. image:: ./_static/images/NCF_start.svg + +However, as the Model Optimizer does not support such data feeding, you should skip it. Cut +the edges incoming in ``ResourceGather`` port 1: + +.. code-block:: shell + + mo --input_model inference_graph.pb \ + --input 1:embedding/embedding_lookup,1:embedding_1/embedding_lookup, \ + 1:embedding_2/embedding_lookup,1:embedding_3/embedding_lookup \ + --input_shape [256],[256],[256],[256] \ + --output_dir + +In the ``input_shape`` parameter, 256 specifies the ``batch_size`` for your model. Alternatively, you can do steps 2 and 3 in one command line: -```sh - mo --input_meta_graph /path/to/model/model.meta \ ---input 1:embedding/embedding_lookup,1:embedding_1/embedding_lookup, \ -1:embedding_2/embedding_lookup,1:embedding_3/embedding_lookup \ ---input_shape [256],[256],[256],[256] --output rating/BiasAdd \ ---output_dir -``` +.. code-block:: shell + + mo --input_meta_graph /path/to/model/model.meta \ + --input 1:embedding/embedding_lookup,1:embedding_1/embedding_lookup, \ + 1:embedding_2/embedding_lookup,1:embedding_3/embedding_lookup \ + --input_shape [256],[256],[256],[256] --output rating/BiasAdd \ + --output_dir + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Object_Detection_API_Models.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Object_Detection_API_Models.md index 87b2dd159eb..0a40001afeb 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Object_Detection_API_Models.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Object_Detection_API_Models.md @@ -1,19 +1,20 @@ # Converting TensorFlow Object Detection API Models {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Object_Detection_API_Models} -**NOTES**: - -* Starting with the 2022.1 release, Model Optimizer can convert the TensorFlow Object Detection API Faster and Mask RCNNs topologies differently. By default, Model Optimizer adds operation "Proposal" to the generated IR. This operation needs an additional input to the model with name "image_info" which should be fed with several values describing the preprocessing applied to the input image (refer to the [Proposal](@ref openvino_docs_ops_detection_Proposal_4) operation specification for more information). However, this input is redundant for the models trained and inferred with equal size images. Model Optimizer can generate IR for such models and insert operation [DetectionOutput](@ref openvino_docs_ops_detection_DetectionOutput_1) instead of `Proposal`. The `DetectionOutput` operation does not require additional model input "image_info". Moreover, for some models the produced inference results are closer to the original TensorFlow model. In order to trigger new behavior, the attribute "operation_to_add" in the corresponding JSON transformation configuration file should be set to value "DetectionOutput" instead of default one "Proposal". -* Starting with the 2021.1 release, Model Optimizer converts the TensorFlow Object Detection API SSDs, Faster and Mask RCNNs topologies keeping shape-calculating sub-graphs by default, so topologies can be re-shaped in the OpenVINO Runtime using dedicated reshape API. Refer to the [Using Shape Inference](@ref openvino_docs_OV_UG_ShapeInference) guide for more information on how to use this feature. It is possible to change the both spatial dimensions of the input image and batch size. -* To generate IRs for TF 1 SSD topologies, Model Optimizer creates a number of `PriorBoxClustered` operations instead of a constant node with prior boxes calculated for the particular input image size. This change allows you to reshape the topology in the OpenVINO Runtime using dedicated API. The reshaping is supported for all SSD topologies except FPNs, which contain hardcoded shapes for some operations preventing from changing topology input shape. - -## Converting a Model - -You can download TensorFlow Object Detection API models from the TensorFlow 1 Detection Model Zoo or TensorFlow 2 Detection Model Zoo. - -> **NOTE**: Before converting, make sure you have configured Model Optimizer. For configuration steps, refer to the [Configuring Model Optimizer](../../../Deep_Learning_Model_Optimizer_DevGuide.md). - @sphinxdirective +* Starting with the 2022.1 release, Model Optimizer can convert the TensorFlow Object Detection API Faster and Mask RCNNs topologies differently. By default, Model Optimizer adds operation "Proposal" to the generated IR. This operation needs an additional input to the model with name "image_info" which should be fed with several values describing the preprocessing applied to the input image (refer to the :doc:`Proposal ` operation specification for more information). However, this input is redundant for the models trained and inferred with equal size images. Model Optimizer can generate IR for such models and insert operation :doc:`DetectionOutput ` instead of ``Proposal``. The `DetectionOutput` operation does not require additional model input "image_info". Moreover, for some models the produced inference results are closer to the original TensorFlow model. In order to trigger new behavior, the attribute "operation_to_add" in the corresponding JSON transformation configuration file should be set to value "DetectionOutput" instead of default one "Proposal". +* Starting with the 2021.1 release, Model Optimizer converts the TensorFlow Object Detection API SSDs, Faster and Mask RCNNs topologies keeping shape-calculating sub-graphs by default, so topologies can be re-shaped in the OpenVINO Runtime using dedicated reshape API. Refer to the :doc:`Using Shape Inference ` guide for more information on how to use this feature. It is possible to change the both spatial dimensions of the input image and batch size. +* To generate IRs for TF 1 SSD topologies, Model Optimizer creates a number of ``PriorBoxClustered`` operations instead of a constant node with prior boxes calculated for the particular input image size. This change allows you to reshape the topology in the OpenVINO Runtime using dedicated API. The reshaping is supported for all SSD topologies except FPNs, which contain hardcoded shapes for some operations preventing from changing topology input shape. + +Converting a Model +################## + +You can download TensorFlow Object Detection API models from the `TensorFlow 1 Detection Model Zoo `__ or `TensorFlow 2 Detection Model Zoo `__. + +.. note:: + + Before converting, make sure you have configured Model Optimizer. For configuration steps, refer to the :doc:`Configuring Model Optimizer `. + To convert a TensorFlow Object Detection API model, run the ``mo`` command with the following required parameters: * ``--input_model `` - File with a pretrained model (binary or text .pb file after freezing) OR ``--saved_model_dir `` for the TensorFlow 2 models @@ -48,105 +49,119 @@ To convert a TensorFlow Object Detection API model, run the ``mo`` command with * ``rfcn_support_api_v1.14.json`` - for RFCN topology from the models zoo frozen with TensorFlow version 1.14.0 or higher * ``--tensorflow_object_detection_api_pipeline_config `` - A special configuration file that describes the topology hyper-parameters and structure of the TensorFlow Object Detection API model. For the models downloaded from the TensorFlow Object Detection API zoo, the configuration file is named ``pipeline.config``. If you plan to train a model yourself, you can find templates for these files in the `models repository `__. -* ``--input_shape`` (optional) - A custom input image shape. For more information how the ``--input_shape`` parameter is handled for the TensorFlow Object Detection API models, refer to the :ref:`Custom Input Shape ` guide. +* ``--input_shape`` (optional) - A custom input image shape. For more information how the ``--input_shape`` parameter is handled for the TensorFlow Object Detection API models, refer to the `Custom Input Shape <#Custom-Input-Shape>`__ guide. -@endsphinxdirective +.. note:: + + The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the ``RGB<->BGR`` conversion specifying the command-line parameter: ``--reverse_input_channels``. Otherwise, inference results may be incorrect. If you convert a TensorFlow Object Detection API model to use with the OpenVINO sample applications, you must specify the ``--reverse_input_channels`` parameter. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the :doc:`Converting a Model to Intermediate Representation (IR) ` guide. + +Additionally to the mandatory parameters listed above you can use optional conversion parameters if needed. A full list of parameters is available in the :doc:`Converting a TensorFlow Model ` guide. + +For example, if you downloaded the pre-trained `SSD InceptionV2 topology `__ and extracted archive to the directory ``/tmp/ssd_inception_v2_coco_2018_01_28``, the sample command line to convert the model looks as follows: + +.. code-block:: sh + + mo --input_model=/tmp/ssd_inception_v2_coco_2018_01_28/frozen_inference_graph.pb --transformations_config front/tf/ssd_v2_support.json --tensorflow_object_detection_api_pipeline_config /tmp/ssd_inception_v2_coco_2018_01_28/pipeline.config --reverse_input_channels -> **NOTE**: The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the `RGB<->BGR` conversion specifying the command-line parameter: `--reverse_input_channels`. Otherwise, inference results may be incorrect. If you convert a TensorFlow Object Detection API model to use with the OpenVINO sample applications, you must specify the `--reverse_input_channels` parameter. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the [Converting a Model to Intermediate Representation (IR)](../Converting_Model.md) guide. - -Additionally to the mandatory parameters listed above you can use optional conversion parameters if needed. A full list of parameters is available in the [Converting a TensorFlow Model](../Convert_Model_From_TensorFlow.md) guide. - -For example, if you downloaded the pre-trained [SSD InceptionV2 topology](http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2018_01_28.tar.gz) and extracted archive to the directory `/tmp/ssd_inception_v2_coco_2018_01_28`, the sample command line to convert the model looks as follows: - -``` -mo --input_model=/tmp/ssd_inception_v2_coco_2018_01_28/frozen_inference_graph.pb --transformations_config front/tf/ssd_v2_support.json --tensorflow_object_detection_api_pipeline_config /tmp/ssd_inception_v2_coco_2018_01_28/pipeline.config --reverse_input_channels -``` - -## OpenVINO™ Toolkit Samples and Open Model Zoo Demos +OpenVINO™ Toolkit Samples and Open Model Zoo Demos +################################################## OpenVINO comes with a number of samples to demonstrate use of OpenVINO Runtime API. Additionally, Open Model Zoo provides set of demo applications to show implementation of close to real life applications, based on deep learning in various tasks, including Image Classification, Visual Object Detection, Text Recognition, Speech Recognition, Natural Language Processing and others. Refer to the links below for more details. -* [OpenVINO Samples](@ref openvino_docs_OV_UG_Samples_Overview) -* [Open Model Zoo Demos](@ref omz_demos) +* :doc:`OpenVINO Samples ` +* :doc:`Open Model Zoo Demos ` -## Feeding Input Images to the Samples +Feeding Input Images to the Samples +################################### There are several important notes about feeding input images to the samples: -1. OpenVINO samples stretch input image to the size of the input operation without preserving aspect ratio. This behavior is usually correct for most topologies (including SSDs), but incorrect for other models like Faster R-CNN, Mask R-CNN and R-FCN. These models usually use keeps aspect ratio resizer. The type of preprocessing is defined in the pipeline configuration file in the section `image_resizer`. If keeping aspect ratio is used, then it is necessary to resize image before passing it to the sample and optionally pad the resized image with 0s (if the attribute "pad_to_max_dimension" in the pipeline.config is equal to "true"). +1. OpenVINO samples stretch input image to the size of the input operation without preserving aspect ratio. This behavior is usually correct for most topologies (including SSDs), but incorrect for other models like Faster R-CNN, Mask R-CNN and R-FCN. These models usually use keeps aspect ratio resizer. The type of preprocessing is defined in the pipeline configuration file in the section ``image_resizer``. If keeping aspect ratio is used, then it is necessary to resize image before passing it to the sample and optionally pad the resized image with 0s (if the attribute "pad_to_max_dimension" in the pipeline.config is equal to "true"). -2. TensorFlow implementation of image resize may be different from the one implemented in the sample. Even reading input image from compressed format (like `.jpg`) could give different results in the sample and TensorFlow. If it is necessary to compare accuracy between the TensorFlow and the OpenVINO, it is recommended to pass pre-resized input image in a non-compressed format (like `.bmp`). +2. TensorFlow implementation of image resize may be different from the one implemented in the sample. Even reading input image from compressed format (like ``.jpg``) could give different results in the sample and TensorFlow. If it is necessary to compare accuracy between the TensorFlow and the OpenVINO, it is recommended to pass pre-resized input image in a non-compressed format (like ``.bmp``). -3. If you want to infer the model with the OpenVINO samples, convert the model specifying the `--reverse_input_channels` command line parameter. The samples load images in BGR channels order, while TensorFlow models were trained with images in RGB order. When the `--reverse_input_channels` command line parameter is specified, Model Optimizer performs first convolution or other channel dependent operation weights modification so the output will be like the image is passed with RGB channels order. +3. If you want to infer the model with the OpenVINO samples, convert the model specifying the ``--reverse_input_channels`` command line parameter. The samples load images in BGR channels order, while TensorFlow models were trained with images in RGB order. When the ``--reverse_input_channels`` command line parameter is specified, Model Optimizer performs first convolution or other channel dependent operation weights modification so the output will be like the image is passed with RGB channels order. 4. Read carefully the messages printed by Model Optimizer during a model conversion. They contain important instructions on how to prepare input data before running the inference and how to interpret the output. -@sphinxdirective +Custom Input Shape +################## -.. _custom-input-shape: +Model Optimizer handles the command line parameter ``--input_shape`` for TensorFlow Object Detection API models in a special way depending on the image resizer type defined in the ``pipeline.config`` file. TensorFlow Object Detection API generates different ``Preprocessor`` sub-graph based on the image resizer type. Model Optimizer supports two types of image resizer: -@endsphinxdirective +* ``fixed_shape_resizer`` --- *Stretches* input image to the specific height and width. The ``pipeline.config`` snippet below shows a ``fixed_shape_resizer`` sample definition: +.. code-block:: sh -## Custom Input Shape -Model Optimizer handles the command line parameter `--input_shape` for TensorFlow Object Detection API models in a special way depending on the image resizer type defined in the `pipeline.config` file. TensorFlow Object Detection API generates different `Preprocessor` sub-graph based on the image resizer type. Model Optimizer supports two types of image resizer: -* `fixed_shape_resizer` --- *Stretches* input image to the specific height and width. The `pipeline.config` snippet below shows a `fixed_shape_resizer` sample definition: -``` -image_resizer { - fixed_shape_resizer { - height: 300 - width: 300 + image_resizer { + fixed_shape_resizer { + height: 300 + width: 300 + } } -} -``` -* `keep_aspect_ratio_resizer` --- Resizes the input image *keeping aspect ratio* to satisfy the minimum and maximum size constraints. The `pipeline.config` snippet below shows a `keep_aspect_ratio_resizer` sample definition: -``` -image_resizer { - keep_aspect_ratio_resizer { - min_dimension: 600 - max_dimension: 1024 + +* ``keep_aspect_ratio_resizer`` --- Resizes the input image *keeping aspect ratio* to satisfy the minimum and maximum size constraints. The ``pipeline.config`` snippet below shows a ``keep_aspect_ratio_resizer`` sample definition: + +.. code-block:: sh + + image_resizer { + keep_aspect_ratio_resizer { + min_dimension: 600 + max_dimension: 1024 + } } -} -``` + If an additional parameter "pad_to_max_dimension" is equal to "true", then the resized image will be padded with 0s to the square image of size "max_dimension". -### Fixed Shape Resizer Replacement -* If the `--input_shape` command line parameter is not specified, Model Optimizer generates an input operation with the height and width as defined in the `pipeline.config`. +Fixed Shape Resizer Replacement ++++++++++++++++++++++++++++++++ -* If the `--input_shape [1, H, W, 3]` command line parameter is specified, Model Optimizer sets the input operation height to `H` and width to `W` and convert the model. However, the conversion may fail because of the following reasons: - * The model is not reshape-able, meaning that it's not possible to change the size of the model input image. For example, SSD FPN models have `Reshape` operations with hard-coded output shapes, but the input size to these `Reshape` instances depends on the input image size. In this case, Model Optimizer shows an error during the shape inference phase. Run Model Optimizer with `--log_level DEBUG` to see the inferred operations output shapes to see the mismatch. - * Custom input shape is too small. For example, if you specify `--input_shape [1,100,100,3]` to convert a SSD Inception V2 model, one of convolution or pooling nodes decreases input tensor spatial dimensions to non-positive values. In this case, Model Optimizer shows error message like this: '[ ERROR ] Shape [ 1 -1 -1 256] is not fully defined for output X of "node_name".' +* If the ``--input_shape`` command line parameter is not specified, Model Optimizer generates an input operation with the height and width as defined in the ``pipeline.config``. + +* If the ``--input_shape [1, H, W, 3]`` command line parameter is specified, Model Optimizer sets the input operation height to ``H`` and width to ``W`` and convert the model. However, the conversion may fail because of the following reasons: + + * The model is not reshape-able, meaning that it's not possible to change the size of the model input image. For example, SSD FPN models have ``Reshape`` operations with hard-coded output shapes, but the input size to these ``Reshape`` instances depends on the input image size. In this case, Model Optimizer shows an error during the shape inference phase. Run Model Optimizer with ``--log_level DEBUG`` to see the inferred operations output shapes to see the mismatch. + * Custom input shape is too small. For example, if you specify ``--input_shape [1,100,100,3]`` to convert a SSD Inception V2 model, one of convolution or pooling nodes decreases input tensor spatial dimensions to non-positive values. In this case, Model Optimizer shows error message like this: '[ ERROR ] Shape [ 1 -1 -1 256] is not fully defined for output X of "node_name".' -### Keeping Aspect Ratio Resizer Replacement -* If the `--input_shape` command line parameter is not specified, Model Optimizer generates an input operation with both height and width equal to the value of parameter `min_dimension` in the `keep_aspect_ratio_resizer`. +Keeping Aspect Ratio Resizer Replacement +++++++++++++++++++++++++++++++++++++++++ -* If the `--input_shape [1, H, W, 3]` command line parameter is specified, Model Optimizer scales the specified input image height `H` and width `W` to satisfy the `min_dimension` and `max_dimension` constraints defined in the `keep_aspect_ratio_resizer`. The following function calculates the input operation height and width: +* If the ``--input_shape`` command line parameter is not specified, Model Optimizer generates an input operation with both height and width equal to the value of parameter ``min_dimension`` in the ``keep_aspect_ratio_resizer``. -```python -def calculate_shape_keeping_aspect_ratio(H: int, W: int, min_dimension: int, max_dimension: int): - ratio_min = min_dimension / min(H, W) - ratio_max = max_dimension / max(H, W) - ratio = min(ratio_min, ratio_max) - return int(round(H * ratio)), int(round(W * ratio)) -``` -The `--input_shape` command line parameter should be specified only if the "pad_to_max_dimension" does not exist of is set to "false" in the `keep_aspect_ratio_resizer`. +* If the ``--input_shape [1, H, W, 3]`` command line parameter is specified, Model Optimizer scales the specified input image height ``H`` and width ``W`` to satisfy the ``min_dimension`` and ``max_dimension`` constraints defined in the ``keep_aspect_ratio_resizer``. The following function calculates the input operation height and width: -Models with `keep_aspect_ratio_resizer` were trained to recognize object in real aspect ratio, in contrast with most of the classification topologies trained to recognize objects stretched vertically and horizontally as well. By default, Model Optimizer converts topologies with `keep_aspect_ratio_resizer` to consume a square input image. If the non-square image is provided as input, it is stretched without keeping aspect ratio that results to object detection quality decrease. +.. code-block:: py -> **NOTE**: It is highly recommended to specify the `--input_shape` command line parameter for the models with `keep_aspect_ratio_resizer`, if the input image dimensions are known in advance. + def calculate_shape_keeping_aspect_ratio(H: int, W: int, min_dimension: int, max_dimension: int): + ratio_min = min_dimension / min(H, W) + ratio_max = max_dimension / max(H, W) + ratio = min(ratio_min, ratio_max) + return int(round(H * ratio)), int(round(W * ratio)) -## Model Conversion Process in Detail +The ``--input_shape`` command line parameter should be specified only if the "pad_to_max_dimension" does not exist of is set to "false" in the ``keep_aspect_ratio_resizer``. -This section is intended for users who want to understand how Model Optimizer performs Object Detection API models conversion in details. The information in this section is also useful for users having complex models that are not converted with Model Optimizer out of the box. It is highly recommended to read the **Graph Transformation Extensions** section in the [Model Optimizer Extensibility](../../customize_model_optimizer/Customize_Model_Optimizer.md) documentation first to understand sub-graph replacement concepts which are used here. +Models with ``keep_aspect_ratio_resizer`` were trained to recognize object in real aspect ratio, in contrast with most of the classification topologies trained to recognize objects stretched vertically and horizontally as well. By default, Model Optimizer converts topologies with ``keep_aspect_ratio_resizer`` to consume a square input image. If the non-square image is provided as input, it is stretched without keeping aspect ratio that results to object detection quality decrease. -It is also important to open the model in the [TensorBoard](https://www.tensorflow.org/guide/summaries_and_tensorboard) to see the topology structure. Model Optimizer can create an event file that can be then fed to the TensorBoard tool. Run Model Optimizer, providing two command line parameters: -* `--input_model ` --- Path to the frozen model. -* `--tensorboard_logdir` --- Path to the directory where TensorBoard looks for the event files. +.. note:: -Implementation of the transformations for Object Detection API models is located in the [file](https://github.com/openvinotoolkit/openvino/blob/releases/2022/1/tools/mo/openvino/tools/mo/front/tf/ObjectDetectionAPI.py). Refer to the code in this file to understand the details of the conversion process. + It is highly recommended to specify the ``--input_shape`` command line parameter for the models with ``keep_aspect_ratio_resizer``, if the input image dimensions are known in advance. +Model Conversion Process in Detail +################################## + +This section is intended for users who want to understand how Model Optimizer performs Object Detection API models conversion in details. The information in this section is also useful for users having complex models that are not converted with Model Optimizer out of the box. It is highly recommended to read the **Graph Transformation Extensions** section in the :doc:`Model Optimizer Extensibility ` documentation first to understand sub-graph replacement concepts which are used here. + +It is also important to open the model in the `TensorBoard `__ to see the topology structure. Model Optimizer can create an event file that can be then fed to the TensorBoard tool. Run Model Optimizer, providing two command line parameters: + +* ``--input_model `` --- Path to the frozen model. +* ``--tensorboard_logdir`` --- Path to the directory where TensorBoard looks for the event files. + +Implementation of the transformations for Object Detection API models is located in the `file `__. Refer to the code in this file to understand the details of the conversion process. + + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_RetinaNet_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_RetinaNet_From_Tensorflow.md index 16a77341c5f..6278f614415 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_RetinaNet_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_RetinaNet_From_Tensorflow.md @@ -1,15 +1,21 @@ # Converting a TensorFlow RetinaNet Model {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_RetinaNet_From_Tensorflow} +@sphinxdirective + This tutorial explains how to convert a RetinaNet model to the Intermediate Representation (IR). -[Public RetinaNet model](https://github.com/fizyr/keras-retinanet) does not contain pretrained TensorFlow weights. -To convert this model to the TensorFlow format, follow the [Reproduce Keras to TensorFlow Conversion tutorial](@ref omz_models_model_retinanet_tf). +`Public RetinaNet model `__ does not contain pretrained TensorFlow weights. +To convert this model to the TensorFlow format, follow the `Reproduce Keras to TensorFlow Conversion tutorial `__. After converting the model to TensorFlow format, run the Model Optimizer command below: -```sh -mo --input "input_1[1,1333,1333,3]" --input_model retinanet_resnet50_coco_best_v2.1.0.pb --transformations_config front/tf/retinanet.json -``` -Where `transformations_config` command-line parameter specifies the configuration json file containing model conversion hints for the Model Optimizer. +.. code-block:: sh + + mo --input "input_1[1,1333,1333,3]" --input_model retinanet_resnet50_coco_best_v2.1.0.pb --transformations_config front/tf/retinanet.json + + +Where ``transformations_config`` command-line parameter specifies the configuration json file containing model conversion hints for the Model Optimizer. The json file contains some parameters that need to be changed if you train the model yourself. It also contains information on how to match endpoints to replace the subgraph nodes. After the model is converted to the OpenVINO IR format, the output nodes will be replaced with DetectionOutput layer. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Slim_Library_Models.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Slim_Library_Models.md index fb8bd420b44..ffecf5a24d3 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Slim_Library_Models.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Slim_Library_Models.md @@ -1,76 +1,90 @@ # Converting TensorFlow Slim Image Classification Model Library Models {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Slim_Library_Models} -TensorFlow-Slim Image Classification Model Library is a library to define, train and evaluate classification models in TensorFlow. The library contains Python scripts defining the classification topologies together with checkpoint files for several pre-trained classification topologies. To convert a TensorFlow-Slim library model, complete the following steps: +@sphinxdirective -1. Download the TensorFlow-Slim models [git repository](https://github.com/tensorflow/models). -2. Download the pre-trained model [checkpoint](https://github.com/tensorflow/models/tree/master/research/slim#pre-trained-models). +`TensorFlow-Slim Image Classification Model Library `__ is a library to define, train and evaluate classification models in TensorFlow. The library contains Python scripts defining the classification topologies together with checkpoint files for several pre-trained classification topologies. To convert a TensorFlow-Slim library model, complete the following steps: + +1. Download the TensorFlow-Slim models `git repository `__. +2. Download the pre-trained model `checkpoint `__. 3. Export the inference graph. 4. Convert the model using the Model Optimizer. -The [Example of an Inception V1 Model Conversion](#example_of_an_inception_v1_model_conversion) below illustrates the process of converting an Inception V1 Model. +The `Example of an Inception V1 Model Conversion <#example_of_an_inception_v1_model_conversion>`__ below illustrates the process of converting an Inception V1 Model. + +Example of an Inception V1 Model Conversion +########################################### -## Example of an Inception V1 Model Conversion This example demonstrates how to convert the model on Linux OSes, but it could be easily adopted for the Windows OSes. **Step 1**. Create a new directory to clone the TensorFlow-Slim git repository to: -```sh -mkdir tf_models -``` -```sh -git clone https://github.com/tensorflow/models.git tf_models -``` +.. code-block:: sh -**Step 2**. Download and unpack the [Inception V1 model checkpoint file](http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz): + mkdir tf_models -```sh -wget http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz -``` -```sh -tar xzvf inception_v1_2016_08_28.tar.gz -``` +.. code-block:: sh -**Step 3**. Export the inference graph --- the protobuf file (`.pb`) containing the architecture of the topology. This file *does not* contain the neural network weights and cannot be used for inference. + git clone https://github.com/tensorflow/models.git tf_models + + +**Step 2**. Download and unpack the `Inception V1 model checkpoint file `__: + +.. code-block:: sh + + wget http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz + +.. code-block:: sh + + tar xzvf inception_v1_2016_08_28.tar.gz + +**Step 3**. Export the inference graph --- the protobuf file (``.pb``) containing the architecture of the topology. This file *does not* contain the neural network weights and cannot be used for inference. + +.. code-block:: sh + + python3 tf_models/research/slim/export_inference_graph.py \ + --model_name inception_v1 \ + --output_file inception_v1_inference_graph.pb -```sh -python3 tf_models/research/slim/export_inference_graph.py \ - --model_name inception_v1 \ - --output_file inception_v1_inference_graph.pb -``` Model Optimizer comes with the summarize graph utility, which identifies graph input and output nodes. Run the utility to determine input/output nodes of the Inception V1 model: -```sh -python3 /openvino/tools/mo/utils/summarize_graph.py --input_model ./inception_v1_inference_graph.pb -``` +.. code-block:: sh -The output looks as follows:
-```sh -1 input(s) detected: -Name: input, type: float32, shape: (-1,224,224,3) -1 output(s) detected: -InceptionV1/Logits/Predictions/Reshape_1 -``` -The tool finds one input node with name `input`, type `float32`, fixed image size `(224,224,3)` and undefined batch size `-1`. The output node name is `InceptionV1/Logits/Predictions/Reshape_1`.
+ python3 /openvino/tools/mo/utils/summarize_graph.py --input_model ./inception_v1_inference_graph.pb + +The output looks as follows: + +.. code-block:: sh + + 1 input(s) detected: + Name: input, type: float32, shape: (-1,224,224,3) + 1 output(s) detected: + InceptionV1/Logits/Predictions/Reshape_1 + +The tool finds one input node with name ``input``, type ``float32``, fixed image size ``(224,224,3)`` and undefined batch size ``-1``. The output node name is ``InceptionV1/Logits/Predictions/Reshape_1``. **Step 4**. Convert the model with the Model Optimizer: -```sh -mo --input_model ./inception_v1_inference_graph.pb --input_checkpoint ./inception_v1.ckpt -b 1 --mean_value [127.5,127.5,127.5] --scale 127.5 -``` +.. code-block:: sh -The `-b` command line parameter is required because the Model Optimizer cannot convert a model with undefined input size. + mo --input_model ./inception_v1_inference_graph.pb --input_checkpoint ./inception_v1.ckpt -b 1 --mean_value [127.5,127.5,127.5] --scale 127.5 -For the information on why `--mean_values` and `--scale` command-line parameters are used, refer to the [Mean and Scale Values for TensorFlow-Slim Models](#tf_slim_mean_scale_values). -## Mean and Scale Values for TensorFlow-Slim Models +The ``-b`` command line parameter is required because the Model Optimizer cannot convert a model with undefined input size. + +For the information on why ``--mean_values`` and ``--scale`` command-line parameters are used, refer to the `Mean and Scale Values for TensorFlow-Slim Models <#Mean-and-Scale-Values-for-TensorFlow-Slim-Models>`__. + +Mean and Scale Values for TensorFlow-Slim Models +################################################# + The TensorFlow-Slim Models were trained with normalized input data. There are several different normalization algorithms used in the Slim library. OpenVINO classification sample does not perform image pre-processing except resizing to the input layer size. It is necessary to pass mean and scale values to the Model Optimizer so they are embedded into the generated IR in order to get correct classification results. -The file [preprocessing_factory.py](https://github.com/tensorflow/models/blob/master/research/slim/preprocessing/preprocessing_factory.py) contains a dictionary variable `preprocessing_fn_map` defining mapping between the model type and pre-processing function to be used. The function code should be analyzed to figure out the mean/scale values. +The file `preprocessing_factory.py `__ contains a dictionary variable ``preprocessing_fn_map`` defining mapping between the model type and pre-processing function to be used. The function code should be analyzed to figure out the mean/scale values. -The [inception_preprocessing.py](https://github.com/tensorflow/models/blob/master/research/slim/preprocessing/inception_preprocessing.py) file defines the pre-processing function for the Inception models. The `preprocess_for_eval` function contains the following code: +The `inception_preprocessing.py `__ file defines the pre-processing function for the Inception models. The ``preprocess_for_eval`` function contains the following code: + +.. code-block:: python -```python3 ... import tensorflow as tf if image.dtype != tf.float32: @@ -79,12 +93,14 @@ The [inception_preprocessing.py](https://github.com/tensorflow/models/blob/maste image = tf.subtract(image, 0.5) image = tf.multiply(image, 2.0) return image -``` -Firstly, the `image` is converted to data type `tf.float32` and the values in the tensor are scaled to the `[0, 1]` range using the [tf.image.convert_image_dtype](https://www.tensorflow.org/api_docs/python/tf/image/convert_image_dtype) function. Then the `0.5` is subtracted from the image values and values multiplied by `2.0`. The final image range of values is `[-1, 1]`. -OpenVINO classification sample reads an input image as a three-dimensional array of integer values from the range `[0, 255]`. In order to scale them to `[-1, 1]` range, the mean value `127.5` for each image channel should be specified as well as a scale factor `127.5`. +Firstly, the ``image`` is converted to data type `tf.float32` and the values in the tensor are scaled to the ``[0, 1]`` range using the `tf.image.convert_image_dtype `__ function. Then the ``0.5`` is subtracted from the image values and values multiplied by ``2.0``. The final image range of values is ``[-1, 1]``. + +OpenVINO classification sample reads an input image as a three-dimensional array of integer values from the range ``[0, 255]``. In order to scale them to ``[-1, 1]`` range, the mean value ``127.5`` for each image channel should be specified as well as a scale factor ``127.5``. Similarly, the mean/scale values can be determined for other Slim models. -The exact mean/scale values are defined in the table with list of supported TensorFlow-Slim models at the [Converting a TensorFlow Model](../Convert_Model_From_TensorFlow.md) guide. +The exact mean/scale values are defined in the table with list of supported TensorFlow-Slim models at the :doc:`Converting a TensorFlow Model ` guide. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_WideAndDeep_Family_Models.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_WideAndDeep_Family_Models.md index b3fd681df24..c3583380d2e 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_WideAndDeep_Family_Models.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_WideAndDeep_Family_Models.md @@ -1,23 +1,40 @@ # Converting TensorFlow Wide and Deep Family Models {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_WideAndDeep_Family_Models} +@sphinxdirective + The Wide and Deep models is a combination of wide and deep parts for memorization and generalization of object features respectively. These models can contain different types of object features such as numerical, categorical, sparse and sequential features. These feature types are specified through Tensorflow tf.feature_column API. Table below presents what feature types are supported by the OpenVINO toolkit. -| numeric | (weighted) categorical | categorical with hash | bucketized | sequential | crossed | -|:-------:|:----------------------:|:---------------------:|:----------:|:----------:|:-------:| -| yes | yes | no | yes | yes | no | +.. list-table:: + :header-rows: 1 -> **NOTE**: The categorical with hash and crossed features are currently unsupported since OpenVINO does not cover tensors of the `string` type and operations with them. + * - numeric + - (weighted) categorical + - categorical with hash + - bucketized + - sequential + - crossed + * - yes + - yes + - no + - yes + - yes + - no -## Preparing an Example of Wide and Deep Model + +.. note:: The categorical with hash and crossed features are currently unsupported since OpenVINO does not cover tensors of the `string` type and operations with them. + +Preparing an Example of Wide and Deep Model +########################################### **Step 1**. Clone the GitHub repository with TensorFlow models and move to the directory with an example of Wide and Deep model: -```sh -git clone https://github.com/tensorflow/models.git --branch r2.2.0; -cd official/r1/wide_deep -``` +.. code-block:: sh + + git clone https://github.com/tensorflow/models.git --branch r2.2.0; + cd official/r1/wide_deep + The Wide and Deep model is no longer in the master branch of the repository but is still available in the r2.2.0 branch. @@ -25,109 +42,114 @@ The Wide and Deep model is no longer in the master branch of the repository but **Step 2**. Train the model As the OpenVINO™ toolkit does not support the categorical with hash and crossed features, such feature types must be switched off in the model -by changing the `build_model_columns()` function in `census_dataset.py` as follows: +by changing the ``build_model_columns()`` function in `census_dataset.py` as follows: -```python -def build_model_columns(): - """Builds a set of wide and deep feature columns.""" - # Continuous variable columns - age = tf.feature_column.numeric_column('age') - education_num = tf.feature_column.numeric_column('education_num') - capital_gain = tf.feature_column.numeric_column('capital_gain') - capital_loss = tf.feature_column.numeric_column('capital_loss') - hours_per_week = tf.feature_column.numeric_column('hours_per_week') - education = tf.feature_column.categorical_column_with_vocabulary_list( - 'education', [ - 'Bachelors', 'HS-grad', '11th', 'Masters', '9th', 'Some-college', - 'Assoc-acdm', 'Assoc-voc', '7th-8th', 'Doctorate', 'Prof-school', - '5th-6th', '10th', '1st-4th', 'Preschool', '12th']) - marital_status = tf.feature_column.categorical_column_with_vocabulary_list( - 'marital_status', [ - 'Married-civ-spouse', 'Divorced', 'Married-spouse-absent', - 'Never-married', 'Separated', 'Married-AF-spouse', 'Widowed']) - relationship = tf.feature_column.categorical_column_with_vocabulary_list( - 'relationship', [ - 'Husband', 'Not-in-family', 'Wife', 'Own-child', 'Unmarried', - 'Other-relative']) - workclass = tf.feature_column.categorical_column_with_vocabulary_list( - 'workclass', [ - 'Self-emp-not-inc', 'Private', 'State-gov', 'Federal-gov', - 'Local-gov', '?', 'Self-emp-inc', 'Without-pay', 'Never-worked']) - # To show an example of hashing: - #occupation = tf.feature_column.categorical_column_with_hash_bucket( - # 'occupation', hash_bucket_size=_HASH_BUCKET_SIZE) - # Transformations. - age_buckets = tf.feature_column.bucketized_column( - age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65]) - # Wide columns and deep columns. - base_columns = [ - education, marital_status, relationship, workclass, - age_buckets, - ] - crossed_columns = [] - wide_columns = base_columns + crossed_columns - deep_columns = [ - age, - education_num, - capital_gain, - capital_loss, - hours_per_week, - tf.feature_column.indicator_column(workclass), - tf.feature_column.indicator_column(education), - tf.feature_column.indicator_column(marital_status), - tf.feature_column.indicator_column(relationship), - # To show an example of embedding - ] - return wide_columns, deep_columns -``` +.. code-block:: python + + def build_model_columns(): + """Builds a set of wide and deep feature columns.""" + # Continuous variable columns + age = tf.feature_column.numeric_column('age') + education_num = tf.feature_column.numeric_column('education_num') + capital_gain = tf.feature_column.numeric_column('capital_gain') + capital_loss = tf.feature_column.numeric_column('capital_loss') + hours_per_week = tf.feature_column.numeric_column('hours_per_week') + education = tf.feature_column.categorical_column_with_vocabulary_list( + 'education', [ + 'Bachelors', 'HS-grad', '11th', 'Masters', '9th', 'Some-college', + 'Assoc-acdm', 'Assoc-voc', '7th-8th', 'Doctorate', 'Prof-school', + '5th-6th', '10th', '1st-4th', 'Preschool', '12th']) + marital_status = tf.feature_column.categorical_column_with_vocabulary_list( + 'marital_status', [ + 'Married-civ-spouse', 'Divorced', 'Married-spouse-absent', + 'Never-married', 'Separated', 'Married-AF-spouse', 'Widowed']) + relationship = tf.feature_column.categorical_column_with_vocabulary_list( + 'relationship', [ + 'Husband', 'Not-in-family', 'Wife', 'Own-child', 'Unmarried', + 'Other-relative']) + workclass = tf.feature_column.categorical_column_with_vocabulary_list( + 'workclass', [ + 'Self-emp-not-inc', 'Private', 'State-gov', 'Federal-gov', + 'Local-gov', '?', 'Self-emp-inc', 'Without-pay', 'Never-worked']) + # To show an example of hashing: + #occupation = tf.feature_column.categorical_column_with_hash_bucket( + # 'occupation', hash_bucket_size=_HASH_BUCKET_SIZE) + # Transformations. + age_buckets = tf.feature_column.bucketized_column( + age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65]) + # Wide columns and deep columns. + base_columns = [ + education, marital_status, relationship, workclass, + age_buckets, + ] + crossed_columns = [] + wide_columns = base_columns + crossed_columns + deep_columns = [ + age, + education_num, + capital_gain, + capital_loss, + hours_per_week, + tf.feature_column.indicator_column(workclass), + tf.feature_column.indicator_column(education), + tf.feature_column.indicator_column(marital_status), + tf.feature_column.indicator_column(relationship), + # To show an example of embedding + ] + return wide_columns, deep_columns After that, start training with the following command: -```sh -python census_main.py -``` +.. code-block:: sh -## Converting the Wide and Deep Model to IR + python census_main.py + + +Converting the Wide and Deep Model to IR +######################################## Use the following command line to convert the saved model file with the checkpoint: -```sh - mo ---input_checkpoint checkpoint --input_meta_graph model.ckpt.meta ---input "IteratorGetNext:0[2], - IteratorGetNext:1[2], - IteratorGetNext:2[2], - IteratorGetNext:4[2], - IteratorGetNext:7[2], - linear/linear_model/linear_model/linear_model/education/to_sparse_input/indices:0[10,2]{i64}, - linear/linear_model/linear_model/linear_model/education/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, - linear/linear_model/linear_model/linear_model/education/to_sparse_input/dense_shape:0[2]{i64}->[2,50], - linear/linear_model/linear_model/linear_model/marital_status/to_sparse_input/indices:0[10,2]{i64}, - linear/linear_model/linear_model/linear_model/marital_status/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, - linear/linear_model/linear_model/linear_model/marital_status/to_sparse_input/dense_shape:0[2]{i64}->[2,50], - linear/linear_model/linear_model/linear_model/relationship/to_sparse_input/indices:0[10,2]{i64}, - linear/linear_model/linear_model/linear_model/relationship/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, - linear/linear_model/linear_model/linear_model/relationship/to_sparse_input/dense_shape:0[2]{i64}->[2,50], - linear/linear_model/linear_model/linear_model/workclass/to_sparse_input/indices:0[10,2]{i64}, - linear/linear_model/linear_model/linear_model/workclass/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, - linear/linear_model/linear_model/linear_model/workclass/to_sparse_input/dense_shape:0[2]{i64}->[2,50], - dnn/input_from_feature_columns/input_layer/education_indicator/to_sparse_input/indices:0[10,2]{i64}, - dnn/input_from_feature_columns/input_layer/education_indicator/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, - dnn/input_from_feature_columns/input_layer/education_indicator/to_sparse_input/dense_shape:0[2]{i64}->[2,50], - dnn/input_from_feature_columns/input_layer/marital_status_indicator/to_sparse_input/indices:0[10,2]{i64}, - dnn/input_from_feature_columns/input_layer/marital_status_indicator/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, - dnn/input_from_feature_columns/input_layer/marital_status_indicator/to_sparse_input/dense_shape:0[2]{i64}->[2,50], - dnn/input_from_feature_columns/input_layer/relationship_indicator/to_sparse_input/indices:0[10,2]{i64}, - dnn/input_from_feature_columns/input_layer/relationship_indicator/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, - dnn/input_from_feature_columns/input_layer/relationship_indicator/to_sparse_input/dense_shape:0[2]{i64}->[2,50], - dnn/input_from_feature_columns/input_layer/workclass_indicator/to_sparse_input/indices:0[10,2]{i64}, - dnn/input_from_feature_columns/input_layer/workclass_indicator/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, - dnn/input_from_feature_columns/input_layer/workclass_indicator/to_sparse_input/dense_shape:0[2]{i64}->[2,50]" ---output head/predictions/probabilities -``` +.. code-block:: sh -The model contains operations unsupported by the OpenVINO™ toolkit such as `IteratorGetNext` and `LookupTableFindV2`, so the Model Optimizer must prune these nodes. -The pruning is specified through `--input` option. The prunings for `IteratorGetNext:*` nodes correspond to numeric features. -The pruning for each categorical feature consists of three prunings for the following nodes: `*/to_sparse_input/indices:0`, `*/hash_table_Lookup/LookupTableFindV2:0`, and `*/to_sparse_input/dense_shape:0`. + mo + --input_checkpoint checkpoint --input_meta_graph model.ckpt.meta + --input "IteratorGetNext:0[2], + IteratorGetNext:1[2], + IteratorGetNext:2[2], + IteratorGetNext:4[2], + IteratorGetNext:7[2], + linear/linear_model/linear_model/linear_model/education/to_sparse_input/indices:0[10,2]{i64}, + linear/linear_model/linear_model/linear_model/education/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, + linear/linear_model/linear_model/linear_model/education/to_sparse_input/dense_shape:0[2]{i64}->[2,50], + linear/linear_model/linear_model/linear_model/marital_status/to_sparse_input/indices:0[10,2]{i64}, + linear/linear_model/linear_model/linear_model/marital_status/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, + linear/linear_model/linear_model/linear_model/marital_status/to_sparse_input/dense_shape:0[2]{i64}->[2,50], + linear/linear_model/linear_model/linear_model/relationship/to_sparse_input/indices:0[10,2]{i64}, + linear/linear_model/linear_model/linear_model/relationship/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, + linear/linear_model/linear_model/linear_model/relationship/to_sparse_input/dense_shape:0[2]{i64}->[2,50], + linear/linear_model/linear_model/linear_model/workclass/to_sparse_input/indices:0[10,2]{i64}, + linear/linear_model/linear_model/linear_model/workclass/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, + linear/linear_model/linear_model/linear_model/workclass/to_sparse_input/dense_shape:0[2]{i64}->[2,50], + dnn/input_from_feature_columns/input_layer/education_indicator/to_sparse_input/indices:0[10,2]{i64}, + dnn/input_from_feature_columns/input_layer/education_indicator/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, + dnn/input_from_feature_columns/input_layer/education_indicator/to_sparse_input/dense_shape:0[2]{i64}->[2,50], + dnn/input_from_feature_columns/input_layer/marital_status_indicator/to_sparse_input/indices:0[10,2]{i64}, + dnn/input_from_feature_columns/input_layer/marital_status_indicator/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, + dnn/input_from_feature_columns/input_layer/marital_status_indicator/to_sparse_input/dense_shape:0[2]{i64}->[2,50], + dnn/input_from_feature_columns/input_layer/relationship_indicator/to_sparse_input/indices:0[10,2]{i64}, + dnn/input_from_feature_columns/input_layer/relationship_indicator/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, + dnn/input_from_feature_columns/input_layer/relationship_indicator/to_sparse_input/dense_shape:0[2]{i64}->[2,50], + dnn/input_from_feature_columns/input_layer/workclass_indicator/to_sparse_input/indices:0[10,2]{i64}, + dnn/input_from_feature_columns/input_layer/workclass_indicator/hash_table_Lookup/LookupTableFindV2:0[10]{i64}, + dnn/input_from_feature_columns/input_layer/workclass_indicator/to_sparse_input/dense_shape:0[2]{i64}->[2,50]" + --output head/predictions/probabilities + + +The model contains operations unsupported by the OpenVINO™ toolkit such as ``IteratorGetNext`` and ``LookupTableFindV2``, so the Model Optimizer must prune these nodes. +The pruning is specified through `--input` option. The prunings for ``IteratorGetNext:*`` nodes correspond to numeric features. +The pruning for each categorical feature consists of three prunings for the following nodes: ``*/to_sparse_input/indices:0``, ``*/hash_table_Lookup/LookupTableFindV2:0``, and ``*/to_sparse_input/dense_shape:0``. The above command line generates an OpenVINO model for a batch of two objects, with the total number of actual categorical feature values equal to 10 and maximum size of a sparse categorical feature for one object equal to 50. + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_XLNet_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_XLNet_From_Tensorflow.md index 1f0cf82bb83..e85d952770b 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_XLNet_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_XLNet_From_Tensorflow.md @@ -1,29 +1,35 @@ # Converting a TensorFlow XLNet Model {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_XLNet_From_Tensorflow} +@sphinxdirective + Pretrained models for XLNet (Bidirectional Encoder Representations from Transformers) are -[publicly available](https://github.com/zihangdai/xlnet). +`publicly available `__. -## Supported Models +Supported Models +################ -The following models from the pretrained [XLNet model list](https://github.com/zihangdai/xlnet#pre-trained-models) are currently supported: +The following models from the pretrained `XLNet model list `__ are currently supported: -* [XLNet-Large, Cased](https://storage.googleapis.com/xlnet/released_models/cased_L-24_H-1024_A-16.zip) -* [XLNet-Base, Cased](https://storage.googleapis.com/xlnet/released_models/cased_L-12_H-768_A-12.zip) +* `XLNet-Large, Cased `__ +* `XLNet-Base, Cased `__ -## Downloading the Pretrained Base XLNet Model +Downloading the Pretrained Base XLNet Model +########################################### -Download and unzip an archive with the [XLNet-Base, Cased](https://storage.googleapis.com/xlnet/released_models/cased_L-12_H-768_A-12.zip). +Download and unzip an archive with the `XLNet-Base, Cased `__. -After the archive is unzipped, the directory `cased_L-12_H-768_A-12` is created and contains the following files: -* TensorFlow checkpoint (`xlnet_model.ckpt`), containing the pretrained weights (which is actually 3 files) -* sentence piece model (`spiece.model`) used for (de)tokenization -* config file (`xlnet_config.json`), which specifies the hyperparameters of the model +After the archive is unzipped, the directory ``cased_L-12_H-768_A-12`` is created and contains the following files: + +* TensorFlow checkpoint (``xlnet_model.ckpt``), containing the pretrained weights (which is actually 3 files) +* sentence piece model (``spiece.model``) used for (de)tokenization +* config file (``xlnet_config.json``), which specifies the hyperparameters of the model To get pb-file from the archive contents, you need to do the following. 1. Run commands - ```sh + .. code-block:: sh + cd ~ mkdir XLNet-Base cd XLNet-Base @@ -31,84 +37,84 @@ To get pb-file from the archive contents, you need to do the following. wget https://storage.googleapis.com/xlnet/released_models/cased_L-12_H-768_A-12.zip unzip cased_L-12_H-768_A-12.zip mkdir try_save - ``` - - + 2. Save and run the following Python script in `~/XLNet-Base/xlnet`: - > **NOTE**: The original model repository has been tested with TensorFlow 1.13.1 under Python2. + .. note:: The original model repository has been tested with TensorFlow 1.13.1 under Python2. - ```python - from collections import namedtuple - - import tensorflow as tf - from tensorflow.python.framework import graph_io - - import model_utils - import xlnet + .. code-block:: python - LENGTHS = 50 - BATCH = 1 - OUTPUT_DIR = '~/XLNet-Base/try_save/' - INIT_CKPT_PATH = '~/XLNet-Base/xlnet_cased_L-12_H-768_A-12/xlnet_model.ckpt' - XLNET_CONFIG_PATH = '~/XLNet-Base/xlnet_cased_L-12_H-768_A-12/xlnet_config.json' - - FLags = namedtuple('FLags', 'use_tpu init_checkpoint') - FLAGS = FLags(use_tpu=False, init_checkpoint=INIT_CKPT_PATH) - - xlnet_config = xlnet.XLNetConfig(json_path=XLNET_CONFIG_PATH) - run_config = xlnet.RunConfig(is_training=False, use_tpu=False, use_bfloat16=False, dropout=0.1, dropatt=0.1,) - - - sentence_features_input_idx = tf.compat.v1.placeholder(tf.int32, shape=[LENGTHS, BATCH], name='input_ids') - sentence_features_segment_ids = tf.compat.v1.placeholder(tf.int32, shape=[LENGTHS, BATCH], name='seg_ids') - sentence_features_input_mask = tf.compat.v1.placeholder(tf.float32, shape=[LENGTHS, BATCH], name='input_mask') - - with tf.compat.v1.Session() as sess: - xlnet_model = xlnet.XLNetModel(xlnet_config=xlnet_config, run_config=run_config, - input_ids=sentence_features_input_idx, - seg_ids=sentence_features_segment_ids, - input_mask=sentence_features_input_mask) - - sess.run(tf.compat.v1.global_variables_initializer()) - model_utils.init_from_checkpoint(FLAGS, True) - - # Save the variables to disk. - saver = tf.compat.v1.train.Saver() - - # Saving checkpoint - save_path = saver.save(sess, OUTPUT_DIR + "model.ckpt") - - # Freezing model - outputs = ['model/transformer/dropout_2/Identity'] - graph_def_freezed = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), outputs) - - # Saving non-frozen and frozen model to pb - graph_io.write_graph(sess.graph.as_graph_def(), OUTPUT_DIR, 'model.pb', as_text=False) - graph_io.write_graph(graph_def_freezed,OUTPUT_DIR, 'model_frozen.pb', - as_text=False) - - # Write to tensorboard - with tf.compat.v1.summary.FileWriter(logdir=OUTPUT_DIR, graph_def=graph_def_freezed) as writer: - writer.flush() - ``` + from collections import namedtuple + + import tensorflow as tf + from tensorflow.python.framework import graph_io + + import model_utils + import xlnet -## Downloading the Pretrained Large XLNet Model + LENGTHS = 50 + BATCH = 1 + OUTPUT_DIR = '~/XLNet-Base/try_save/' + INIT_CKPT_PATH = '~/XLNet-Base/xlnet_cased_L-12_H-768_A-12/xlnet_model.ckpt' + XLNET_CONFIG_PATH = '~/XLNet-Base/xlnet_cased_L-12_H-768_A-12/xlnet_config.json' + + FLags = namedtuple('FLags', 'use_tpu init_checkpoint') + FLAGS = FLags(use_tpu=False, init_checkpoint=INIT_CKPT_PATH) + + xlnet_config = xlnet.XLNetConfig(json_path=XLNET_CONFIG_PATH) + run_config = xlnet.RunConfig(is_training=False, use_tpu=False, use_bfloat16=False, dropout=0.1, dropatt=0.1,) + + + sentence_features_input_idx = tf.compat.v1.placeholder(tf.int32, shape=[LENGTHS, BATCH], name='input_ids') + sentence_features_segment_ids = tf.compat.v1.placeholder(tf.int32, shape=[LENGTHS, BATCH], name='seg_ids') + sentence_features_input_mask = tf.compat.v1.placeholder(tf.float32, shape=[LENGTHS, BATCH], name='input_mask') + + with tf.compat.v1.Session() as sess: + xlnet_model = xlnet.XLNetModel(xlnet_config=xlnet_config, run_config=run_config, + input_ids=sentence_features_input_idx, + seg_ids=sentence_features_segment_ids, + input_mask=sentence_features_input_mask) + + sess.run(tf.compat.v1.global_variables_initializer()) + model_utils.init_from_checkpoint(FLAGS, True) + + # Save the variables to disk. + saver = tf.compat.v1.train.Saver() + + # Saving checkpoint + save_path = saver.save(sess, OUTPUT_DIR + "model.ckpt") + + # Freezing model + outputs = ['model/transformer/dropout_2/Identity'] + graph_def_freezed = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), outputs) + + # Saving non-frozen and frozen model to pb + graph_io.write_graph(sess.graph.as_graph_def(), OUTPUT_DIR, 'model.pb', as_text=False) + graph_io.write_graph(graph_def_freezed,OUTPUT_DIR, 'model_frozen.pb', + as_text=False) + + # Write to tensorboard + with tf.compat.v1.summary.FileWriter(logdir=OUTPUT_DIR, graph_def=graph_def_freezed) as writer: + writer.flush() + +Downloading the Pretrained Large XLNet Model +############################################ -Download and unzip an archive with the [XLNet-Large, Cased](https://storage.googleapis.com/xlnet/released_models/cased_L-24_H-1024_A-16.zip). +Download and unzip an archive with the `XLNet-Base, Cased `__. -After unzipping the archive, the directory `cased_L-12_H-1024_A-16` is created and contains the following files: +After unzipping the archive, the directory ``cased_L-12_H-1024_A-16`` is created and contains the following files: -* TensorFlow checkpoint (`xlnet_model.ckpt`) containing the pretrained weights (which is actually 3 files) -* sentence piece model (`spiece.model`) used for (de)tokenization -* config file (`xlnet_config.json`) which specifies the hyperparameters of the model +* TensorFlow checkpoint (``xlnet_model.ckpt``) containing the pretrained weights (which is actually 3 files) +* sentence piece model (``spiece.model``) used for (de)tokenization +* config file (``xlnet_config.json``) which specifies the hyperparameters of the model -To get `pb-file` from the archive contents, follow the instructions below: +To get ``pb-file`` from the archive contents, follow the instructions below: 1. Run commands - ```sh + .. code-block:: sh + cd ~ mkdir XLNet-Large cd XLNet-Large @@ -116,74 +122,76 @@ To get `pb-file` from the archive contents, follow the instructions below: wget https://storage.googleapis.com/xlnet/released_models/cased_L-24_H-1024_A-16.zip unzip cased_L-24_H-1024_A-16.zip mkdir try_save - ``` + -2. Save and run the following Python script in `~/XLNet-Large/xlnet`: +2. Save and run the following Python script in ``~/XLNet-Large/xlnet``: - ```python - from collections import namedtuple - - import tensorflow as tf - from tensorflow.python.framework import graph_io - - import model_utils - import xlnet - - LENGTHS = 50 - BATCH = 1 - OUTPUT_DIR = '~/XLNet-Large/try_save' - INIT_CKPT_PATH = '~/XLNet-Large/cased_L-24_H-1024_A-16/xlnet_model.ckpt' - XLNET_CONFIG_PATH = '~/XLNet-Large/cased_L-24_H-1024_A-16/xlnet_config.json' - - FLags = namedtuple('FLags', 'use_tpu init_checkpoint') - FLAGS = FLags(use_tpu=False, init_checkpoint=INIT_CKPT_PATH) - - xlnet_config = xlnet.XLNetConfig(json_path=XLNET_CONFIG_PATH) - run_config = xlnet.RunConfig(is_training=False, use_tpu=False, use_bfloat16=False, dropout=0.1, dropatt=0.1,) - - - sentence_features_input_idx = tf.compat.v1.placeholder(tf.int32, shape=[LENGTHS, BATCH], name='input_ids') - sentence_features_segment_ids = tf.compat.v1.placeholder(tf.int32, shape=[LENGTHS, BATCH], name='seg_ids') - sentence_features_input_mask = tf.compat.v1.placeholder(tf.float32, shape=[LENGTHS, BATCH], name='input_mask') - - with tf.compat.v1.Session() as sess: - xlnet_model = xlnet.XLNetModel(xlnet_config=xlnet_config, run_config=run_config, - input_ids=sentence_features_input_idx, - seg_ids=sentence_features_segment_ids, - input_mask=sentence_features_input_mask) - - sess.run(tf.compat.v1.global_variables_initializer()) - model_utils.init_from_checkpoint(FLAGS, True) - - # Save the variables to disk. - saver = tf.compat.v1.train.Saver() - - # Saving checkpoint - save_path = saver.save(sess, OUTPUT_DIR + "model.ckpt") - - # Freezing model - outputs = ['model/transformer/dropout_2/Identity'] - graph_def_freezed = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), outputs) - - # Saving non-frozen and frozen model to pb - graph_io.write_graph(sess.graph.as_graph_def(), OUTPUT_DIR, 'model.pb', as_text=False) - graph_io.write_graph(graph_def_freezed,OUTPUT_DIR, 'model_frozen.pb', - as_text=False) - - # Write to tensorboard - with tf.compat.v1.summary.FileWriter(logdir=OUTPUT_DIR, graph_def=graph_def_freezed) as writer: - writer.flush() - ``` + .. code-block:: python + + from collections import namedtuple + + import tensorflow as tf + from tensorflow.python.framework import graph_io + + import model_utils + import xlnet + + LENGTHS = 50 + BATCH = 1 + OUTPUT_DIR = '~/XLNet-Large/try_save' + INIT_CKPT_PATH = '~/XLNet-Large/cased_L-24_H-1024_A-16/xlnet_model.ckpt' + XLNET_CONFIG_PATH = '~/XLNet-Large/cased_L-24_H-1024_A-16/xlnet_config.json' + + FLags = namedtuple('FLags', 'use_tpu init_checkpoint') + FLAGS = FLags(use_tpu=False, init_checkpoint=INIT_CKPT_PATH) + + xlnet_config = xlnet.XLNetConfig(json_path=XLNET_CONFIG_PATH) + run_config = xlnet.RunConfig(is_training=False, use_tpu=False, use_bfloat16=False, dropout=0.1, dropatt=0.1,) + + + sentence_features_input_idx = tf.compat.v1.placeholder(tf.int32, shape=[LENGTHS, BATCH], name='input_ids') + sentence_features_segment_ids = tf.compat.v1.placeholder(tf.int32, shape=[LENGTHS, BATCH], name='seg_ids') + sentence_features_input_mask = tf.compat.v1.placeholder(tf.float32, shape=[LENGTHS, BATCH], name='input_mask') + + with tf.compat.v1.Session() as sess: + xlnet_model = xlnet.XLNetModel(xlnet_config=xlnet_config, run_config=run_config, + input_ids=sentence_features_input_idx, + seg_ids=sentence_features_segment_ids, + input_mask=sentence_features_input_mask) + + sess.run(tf.compat.v1.global_variables_initializer()) + model_utils.init_from_checkpoint(FLAGS, True) + + # Save the variables to disk. + saver = tf.compat.v1.train.Saver() + + # Saving checkpoint + save_path = saver.save(sess, OUTPUT_DIR + "model.ckpt") + + # Freezing model + outputs = ['model/transformer/dropout_2/Identity'] + graph_def_freezed = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), outputs) + + # Saving non-frozen and frozen model to pb + graph_io.write_graph(sess.graph.as_graph_def(), OUTPUT_DIR, 'model.pb', as_text=False) + graph_io.write_graph(graph_def_freezed,OUTPUT_DIR, 'model_frozen.pb', + as_text=False) + + # Write to tensorboard + with tf.compat.v1.summary.FileWriter(logdir=OUTPUT_DIR, graph_def=graph_def_freezed) as writer: + writer.flush() -The script should save into `~/XLNet-Large/xlnet`. +The script should save into ``~/XLNet-Large/xlnet``. -## Converting a frozen TensorFlow XLNet Model to IR +Converting a frozen TensorFlow XLNet Model to IR +################################################# To generate the XLNet Intermediate Representation (IR) of the model, run Model Optimizer with the following parameters: -```sh -mo --input_model path-to-model/model_frozen.pb \ - --input "input_mask[50,1],input_ids[50,1],seg_ids[50,1]" -``` +.. code-block:: sh + mo --input_model path-to-model/model_frozen.pb \ + --input "input_mask[50,1],input_ids[50,1],seg_ids[50,1]" + +@endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_lm_1b_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_lm_1b_From_Tensorflow.md index 5c441717b02..7cd7ccf23c4 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_lm_1b_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_lm_1b_From_Tensorflow.md @@ -1,100 +1,121 @@ # Converting a TensorFlow Language Model on One Billion Word Benchmark {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_lm_1b_From_Tensorflow} -## Downloading a Pre-trained Language Model on One Billion Word Benchmark +@sphinxdirective -TensorFlow provides a pretrained [Language Model on One Billion Word Benchmark](https://github.com/tensorflow/models/tree/r2.3.0/research/lm_1b). +Downloading a Pre-trained Language Model on One Billion Word Benchmark +###################################################################### + +TensorFlow provides a pretrained `Language Model on One Billion Word Benchmark `__. To download the model for IR conversion, follow the instructions: + 1. Create new directory to store the model: -```shell -mkdir lm_1b -``` -2. Go to the `lm_1b` directory: -```shell -cd lm_1b -``` + +.. code-block:: shell + + mkdir lm_1b + +2. Go to the ``lm_1b`` directory: + +.. code-block:: shell + + cd lm_1b + 3. Download the model GraphDef file: -``` -wget http://download.tensorflow.org/models/LM_LSTM_CNN/graph-2016-09-10.pbtxt -``` + +.. code-block:: shell + + wget http://download.tensorflow.org/models/LM_LSTM_CNN/graph-2016-09-10.pbtxt + 4. Create new directory to store 12 checkpoint shared files: -```shell -mkdir ckpt -``` -5. Go to the `ckpt` directory: -```shell -cd ckpt -``` + +.. code-block:: shell + + mkdir ckpt + +5. Go to the ``ckpt`` directory: + +.. code-block:: shell + + cd ckpt + 6. Download 12 checkpoint shared files: -``` -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-base -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-char-embedding -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-lstm -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax0 -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax1 -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax2 -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax3 -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax4 -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax5 -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax6 -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax7 -wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax8 -``` -Once you have downloaded the pretrained model files, you will have the `lm_1b` directory with the following hierarchy: - -``` -lm_1b/ - graph-2016-09-10.pbtxt - ckpt/ - ckpt-base - ckpt-char-embedding - ckpt-lstm - ckpt-softmax0 - ckpt-softmax1 - ckpt-softmax2 - ckpt-softmax3 - ckpt-softmax4 - ckpt-softmax5 - ckpt-softmax6 - ckpt-softmax7 - ckpt-softmax8 -``` +.. code-block:: shell + + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-base + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-char-embedding + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-lstm + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax0 + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax1 + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax2 + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax3 + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax4 + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax5 + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax6 + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax7 + wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax8 -![lm_1b model view](../../../img/lm_1b.svg) +Once you have downloaded the pretrained model files, you will have the ``lm_1b`` directory with the following hierarchy: -The frozen model still has two variables: `Variable` and `Variable_1`. +.. code-block:: shell + + lm_1b/ + graph-2016-09-10.pbtxt + ckpt/ + ckpt-base + ckpt-char-embedding + ckpt-lstm + ckpt-softmax0 + ckpt-softmax1 + ckpt-softmax2 + ckpt-softmax3 + ckpt-softmax4 + ckpt-softmax5 + ckpt-softmax6 + ckpt-softmax7 + ckpt-softmax8 + + + +.. image:: ./_static/images/lm_1b.svg + +The frozen model still has two variables: ``Variable`` and ``Variable_1``. It means that the model keeps training those variables at each inference. At the first inference of this graph, the variables are initialized by initial values. -After executing the `lstm` nodes, results of execution are assigned to these two variables. +After executing the ``lstm`` nodes, results of execution are assigned to these two variables. -With each inference of the `lm_1b` graph, `lstm` initial states data is taken from previous inference -from variables, and states of current inference of `lstm` is reassigned to the same variables. +With each inference of the ``lm_1b`` graph, ``lstm`` initial states data is taken from previous inference +from variables, and states of current inference of ``lstm`` is reassigned to the same variables. It helps the model to remember the context of the words that it takes as input. -## Converting a TensorFlow Language Model on One Billion Word Benchmark to IR +Converting a TensorFlow Language Model on One Billion Word Benchmark to IR +########################################################################## Model Optimizer assumes that output model is for inference only. Therefore, you should cut those variables off and resolve keeping cell and hidden states on application level. There is a certain limitation for the model conversion: the original model cannot be reshaped, so you should keep original shapes. -To generate the `lm_1b` Intermediate Representation (IR), provide TensorFlow `lm_1b` model to the +To generate the ``lm_1b`` Intermediate Representation (IR), provide TensorFlow ``lm_1b`` model to the Model Optimizer with parameters: -```sh - mo ---input_model lm_1b/graph-2016-09-10.pbtxt \ ---input_checkpoint lm_1b/ckpt \ ---input_model_is_text \ ---input_shape [50],[50],[1,9216],[1,9216] \ ---output softmax_out,lstm/lstm_0/concat_2,lstm/lstm_1/concat_2 \ ---input char_embedding/EmbeddingLookupUnique/Unique:0,char_embedding/EmbeddingLookupUnique/Unique:1,Variable/read,Variable_1/read -``` + +.. code-block:: shell + + mo + --input_model lm_1b/graph-2016-09-10.pbtxt \ + --input_checkpoint lm_1b/ckpt \ + --input_model_is_text \ + --input_shape [50],[50],[1,9216],[1,9216] \ + --output softmax_out,lstm/lstm_0/concat_2,lstm/lstm_1/concat_2 \ + --input char_embedding/EmbeddingLookupUnique/Unique:0,char_embedding/EmbeddingLookupUnique/Unique:1,Variable/read,Variable_1/read Where: -* `--input char_embedding/EmbeddingLookupUnique/Unique:0,char_embedding/EmbeddingLookupUnique/Unique:1,Variable/read,Variable_1/read` - and `--input_shape [50],[50],[1,9216],[1,9216]` replace the variables with a placeholder. -* `--output softmax_out,lstm/lstm_0/concat_2,lstm/lstm_1/concat_2` specifies output node name and names of LSTM cell states. + +* ``--input char_embedding/EmbeddingLookupUnique/Unique:0,char_embedding/EmbeddingLookupUnique/Unique:1,Variable/read,Variable_1/read`` and ``--input_shape [50],[50],[1,9216],[1,9216]`` replace the variables with a placeholder. +* ``--output softmax_out,lstm/lstm_0/concat_2,lstm/lstm_1/concat_2`` specifies output node name and names of LSTM cell states. + +@endsphinxdirective diff --git a/docs/_static/images/DeepSpeech-0.8.2.png b/docs/_static/images/DeepSpeech-0.8.2.png new file mode 100644 index 00000000000..ddab04ac34a --- /dev/null +++ b/docs/_static/images/DeepSpeech-0.8.2.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fdff3768930f683b81ca466be4f947af3172933a702cd38201a254df27a68556 +size 62498 diff --git a/docs/_static/images/FaceNet.svg b/docs/_static/images/FaceNet.svg new file mode 100644 index 00000000000..89927473829 --- /dev/null +++ b/docs/_static/images/FaceNet.svg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f2720b6d3b5e680978a91379c8c37366285299aab31aa139ad9abea8334aae34 +size 57687 diff --git a/docs/_static/images/NCF_start.svg b/docs/_static/images/NCF_start.svg new file mode 100644 index 00000000000..345d9561721 --- /dev/null +++ b/docs/_static/images/NCF_start.svg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5c0389fe34562993b1285f1994dbc878e9547a841c903bf204074ed2219b6bc7 +size 323210 diff --git a/docs/_static/images/lm_1b.svg b/docs/_static/images/lm_1b.svg new file mode 100644 index 00000000000..df8075c0416 --- /dev/null +++ b/docs/_static/images/lm_1b.svg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:062fa64afa0cc43c4a2c2c0442e499b6176c837857222af30bad2fa7c9515420 +size 95508