Doc Migration (master) (#1377)
* Doc Migration from Gitlab (#1289) * doc migration * fix * Update FakeQuantize_1.md * Update performance_benchmarks.md * Updates graphs for FPGA * Update performance_benchmarks.md * Change DL Workbench structure (#1) * Changed DL Workbench structure * Fixed tags * fixes * Update ie_docs.xml * Update performance_benchmarks_faq.md * Fixes in DL Workbench layout * Fixes for CVS-31290 * [DL Workbench] Minor correction * Fix for CVS-30955 * Added nGraph deprecation notice as requested by Zoe * fix broken links in api doxy layouts * CVS-31131 fixes * Additional fixes * Fixed POT TOC * Update PAC_Configure.md PAC DCP 1.2.1 install guide. * Update inference_engine_intro.md * fix broken link * Update opset.md * fix * added opset4 to layout * added new opsets to layout, set labels for them * Update VisionAcceleratorFPGA_Configure.md Updated from 2020.3 to 2020.4 Co-authored-by: domi2000 <domi2000@users.noreply.github.com>
This commit is contained in:
Nikolay Tyukaevanddomi2000 authored and 
GitHub committed
1 parent
4037613db3
commit
ef45b5da8d
532 files changed
+30445
-313
No files matched your search
@@ -0,0 +1,212 @@
|
||||
# Custom Layers Guide {#openvino_docs_HOWTO_Custom_Layers_Guide}
|
||||
|
||||
The Intel® Distribution of OpenVINO™ toolkit supports neural network model layers in multiple frameworks including TensorFlow*, Caffe*, MXNet*, Kaldi* and ONYX*. The list of known layers is different for each of the supported frameworks. To see the layers supported by your framework, refer to [supported frameworks](../MO_DG/prepare_model/Supported_Frameworks_Layers.md).
|
||||
|
||||
Custom layers are layers that are not included in the list of known layers. If your topology contains any layers that are not in the list of known layers, the Model Optimizer classifies them as custom.
|
||||
|
||||
This guide illustrates the workflow for running inference on topologies featuring custom layers, allowing you to plug in your own implementation for existing or completely new layers.
|
||||
For a step-by-step example of creating and executing a custom layer, see the [Custom Layer Implementation Tutorials for Linux and Windows.](https://github.com/david-drew/OpenVINO-Custom-Layers/tree/master/2019.r2.0)
|
||||
|
||||
## Terms used in this guide
|
||||
|
||||
- *Layer* — The abstract concept of a math function that is selected for a specific purpose (relu, sigmoid, tanh, convolutional). This is one of a sequential series of building blocks within the neural network.
|
||||
- *Kernel* — The implementation of a layer function, in this case, the math programmed (in C++ and Python) to perform the layer operation for target hardware (CPU or GPU).
|
||||
- *Intermediate Representation (IR)* — Neural Network used only by the Inference Engine in OpenVINO abstracting the different frameworks and describing topology, layer parameters and weights.
|
||||
The original format will be a supported framework such as TensorFlow, Caffe, or MXNet.
|
||||
|
||||
- *Model Extension Generator* — Generates template source code files for each of the extensions needed by the Model Optimizer and the Inference Engine.
|
||||
|
||||
- *Inference Engine Extension* — Device-specific module implementing custom layers (a set of kernels).
|
||||
|
||||
|
||||
## Custom Layer Overview
|
||||
|
||||
The [Model Optimizer](https://docs.openvinotoolkit.org/2019_R1.1/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) searches the list of known layers for each layer contained in the input model topology before building the model's internal representation, optimizing the model, and producing the Intermediate Representation files.
|
||||
|
||||
The [Inference Engine](https://docs.openvinotoolkit.org/2019_R1.1/_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide.html) loads the layers from the input model IR files into the specified device plugin, which will search a list of known layer implementations for the device. If your topology contains layers that are not in the list of known layers for the device, the Inference Engine considers the layer to be unsupported and reports an error. To see the layers that are supported by each device plugin for the Inference Engine, refer to the [Supported Devices](https://docs.openvinotoolkit.org/2019_R1.1/_docs_IE_DG_supported_plugins_Supported_Devices.html) documentation.
|
||||
<br>
|
||||
**Note:** If a device doesn't support a particular layer, an alternative to creating a new custom layer is to target an additional device using the HETERO plugin. The [Heterogeneous Plugin](https://docs.openvinotoolkit.org/2019_R1.1/_docs_IE_DG_supported_plugins_HETERO.html) may be used to run an inference model on multiple devices allowing the unsupported layers on one device to "fallback" to run on another device (e.g., CPU) that does support those layers.
|
||||
|
||||
## Custom Layer Implementation Workflow
|
||||
|
||||
When implementing a custom layer for your pre-trained model in the Intel® Distribution of OpenVINO™ toolkit, you will need to add extensions to both the Model Optimizer and the Inference Engine.
|
||||
|
||||
## Custom Layer Extensions for the Model Optimizer
|
||||
|
||||
The following figure shows the basic processing steps for the Model Optimizer highlighting the two necessary custom layer extensions, the Custom Layer Extractor and the Custom Layer Operation.
|
||||
|
||||
|
||||
|
||||
|
||||
The Model Optimizer first extracts information from the input model which includes the topology of the model layers along with parameters, input and output format, etc., for each layer. The model is then optimized from the various known characteristics of the layers, interconnects, and data flow which partly comes from the layer operation providing details including the shape of the output for each layer. Finally, the optimized model is output to the model IR files needed by the Inference Engine to run the model.
|
||||
|
||||
The Model Optimizer starts with a library of known extractors and operations for each [supported model framework](https://docs.openvinotoolkit.org/2019_R1.1/_docs_MO_DG_prepare_model_Supported_Frameworks_Layers.html) which must be extended to use each unknown custom layer. The custom layer extensions needed by the Model Optimizer are:
|
||||
|
||||
- Custom Layer Extractor
|
||||
- Responsible for identifying the custom layer operation and extracting the parameters for each instance of the custom layer. The layer parameters are stored per instance and used by the layer operation before finally appearing in the output IR. Typically the input layer parameters are unchanged, which is the case covered by this tutorial.
|
||||
- Custom Layer Operation
|
||||
- Responsible for specifying the attributes that are supported by the custom layer and computing the output shape for each instance of the custom layer from its parameters. <br> The `--mo-op` command-line argument shown in the examples below generates a custom layer operation for the Model Optimizer.
|
||||
|
||||
## Custom Layer Extensions for the Inference Engine
|
||||
|
||||
The following figure shows the basic flow for the Inference Engine highlighting two custom layer extensions for the CPU and GPU Plugins, the Custom Layer CPU extension and the Custom Layer GPU Extension.
|
||||
|
||||
|
||||
|
||||
Each device plugin includes a library of optimized implementations to execute known layer operations which must be extended to execute a custom layer. The custom layer extension is implemented according to the target device:
|
||||
|
||||
- Custom Layer CPU Extension
|
||||
- A compiled shared library (.so or .dll binary) needed by the CPU Plugin for executing the custom layer on the CPU.
|
||||
- Custom Layer GPU Extension
|
||||
- OpenCL source code (.cl) for the custom layer kernel that will be compiled to execute on the GPU along with a layer description file (.xml) needed by the GPU Plugin for the custom layer kernel.
|
||||
|
||||
## Model Extension Generator
|
||||
|
||||
Using answers to interactive questions or a *.json* configuration file, the Model Extension Generator tool generates template source code files for each of the extensions needed by the Model Optimizer and the Inference Engine. To complete the implementation of each extension, the template functions may need to be edited to fill-in details specific to the custom layer or the actual custom layer functionality itself.
|
||||
|
||||
### Command-line
|
||||
|
||||
The Model Extension Generator is included in the Intel® Distribution of OpenVINO™ toolkit installation and is run using the command (here with the "--help" option):
|
||||
|
||||
```bash
|
||||
python3 /opt/intel/openvino/deployment_tools/tools/extension_generator/extgen.py new --help
|
||||
```
|
||||
|
||||
where the output will appear similar to:
|
||||
|
||||
```
|
||||
usage: You can use any combination of the following arguments:
|
||||
|
||||
Arguments to configure extension generation in the interactive mode:
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
--mo-caffe-ext generate a Model Optimizer Caffe* extractor
|
||||
--mo-mxnet-ext generate a Model Optimizer MXNet* extractor
|
||||
--mo-tf-ext generate a Model Optimizer TensorFlow* extractor
|
||||
--mo-op generate a Model Optimizer operation
|
||||
--ie-cpu-ext generate an Inference Engine CPU extension
|
||||
--ie-gpu-ext generate an Inference Engine GPU extension
|
||||
--output_dir OUTPUT_DIR
|
||||
set an output directory. If not specified, the current
|
||||
directory is used by default.
|
||||
```
|
||||
|
||||
The available command-line arguments are used to specify which extension(s) to generate templates for the Model Optimizer or Inference Engine. The generated extension files for each argument will appear starting from the top of the output directory as follows:
|
||||
|
||||
Command-line Argument | Output Directory Location |
|
||||
--------------------- | ------------------------------ |
|
||||
`--mo-caffe-ext` | user_mo_extensions/front/caffe |
|
||||
`--mo-mxnet-ext` | user_mo_extensions/front/mxnet |
|
||||
`--mo-tf-ext` | user_mo_extensions/front/tf |
|
||||
`--mo-op` | user_mo_extensions/ops |
|
||||
`--ie-cpu-ext` | user_ie_extensions/cpu |
|
||||
`--ie-gpu-ext` | user_ie_extensions/gpu |
|
||||
|
||||
### Extension Workflow
|
||||
|
||||
The workflow for each generated extension follows the same basic steps:
|
||||
|
||||
|
||||
|
||||
**Step 1: Generate:** Use the Model Extension Generator to generate the Custom Layer Template Files.
|
||||
|
||||
**Step 2: Edit:** Edit the Custom Layer Template Files as necessary to create the specialized Custom Layer Extension Source Code.
|
||||
|
||||
**Step 3: Specify:** Specify the custom layer extension locations to be used by the Model Optimizer or Inference Engine.
|
||||
|
||||
## Caffe\* Models with Custom Layers <a name="caffe-models-with-custom-layers"></a>
|
||||
|
||||
If your Caffe\* model has custom layers:
|
||||
|
||||
**Register the custom layers as extensions to the Model Optimizer**. For instructions, see [Extending Model Optimizer with New Primitives](../MO_DG/prepare_model/customize_model_optimizer/Extending_Model_Optimizer_with_New_Primitives.md). When your custom layers are registered as extensions, the Model Optimizer generates a valid and optimized Intermediate Representation. You will need a bit of Python\* code that lets the Model Optimizer;
|
||||
|
||||
- Generate a valid Intermediate Representation according to the rules you specified.
|
||||
- Be independent from the availability of Caffe on your computer.
|
||||
|
||||
If your model contains Custom Layers, it is important to understand the internal workflow of the Model Optimizer. Consider the following example.
|
||||
|
||||
**Example**:
|
||||
|
||||
The network has:
|
||||
|
||||
* One input layer (#1)
|
||||
* One output Layer (#5)
|
||||
* Three internal layers (#2, 3, 4)
|
||||
|
||||
The custom and standard layer types are:
|
||||
|
||||
* Layers #2 and #5 are implemented as Model Optimizer extensions.
|
||||
* Layers #1 and #4 are supported in Model Optimizer out-of-the box.
|
||||
* Layer #3 is neither in the list of supported layers nor in extensions, but is specified in CustomLayersMapping.xml.
|
||||
|
||||
> **NOTE**: If any of the layers are not in one of three categories described above, the Model Optimizer fails with an appropriate message and a link to the corresponding question in [Model Optimizer FAQ](../MO_DG/prepare_model/Model_Optimizer_FAQ.md).
|
||||
|
||||
The general process is as shown:
|
||||
|
||||
|
||||
<br>
|
||||
|
||||
**Step 1:** The example model is fed to the Model Optimizer that **loads the model** with the special parser built on top of the `caffe.proto` file. In case of failure, the Model Optimizer asks you to prepare the parser that can read the model. For more information, refer to the Model Optimizer, <a href="MO_FAQ.html#FAQ1">FAQ #1</a>.
|
||||
|
||||
**Step 2:** The Model Optimizer **extracts the attributes of all layers** by going through the list of layers and attempting to find the appropriate extractor. In order of priority, the Model Optimizer checks if the layer is:
|
||||
|
||||
* A. Registered as a Model Optimizer extension
|
||||
* B. Registered as a standard Model Optimizer layer
|
||||
|
||||
When the Model Optimizer finds a satisfying condition from the list above, it extracts the attributes according to the following rules:
|
||||
|
||||
* For A. - takes only the parameters specified in the extension
|
||||
* For B. - takes only the parameters specified in the standard extractor
|
||||
<br>
|
||||
|
||||
**Step 3:** The Model Optimizer **calculates the output shape of all layers**. The logic is the same as it is for the priorities. **Important:** the Model Optimizer always takes the first available option.
|
||||
|
||||
**Step 4:** The Model Optimizer **optimizes the original model and produces the two Intermediate Representation (IR) files in .xml and .bin**.
|
||||
<br>
|
||||
|
||||
## TensorFlow\* Models with Custom Layers <a name="Tensorflow-models-with-custom-layers"></a>
|
||||
|
||||
You have two options for TensorFlow\* models with custom layers:
|
||||
<br>
|
||||
|
||||
* **Register those layers as extensions to the Model Optimizer.** In this case, the Model Optimizer generates a valid and optimized Intermediate Representation.
|
||||
* **If you have sub-graphs that should not be expressed with the analogous sub-graph in the Intermediate Representation, but another sub-graph should appear in the model, the Model Optimizer provides such an option.** This feature is helpful for many TensorFlow models. To read more, see [Sub-graph Replacement in the Model Optimizer](../MO_DG/prepare_model/customize_model_optimizer/Subgraph_Replacement_Model_Optimizer.md).
|
||||
|
||||
## MXNet\* Models with Custom Layers <a name="mxnet-models-with-custom-layers"></a>
|
||||
|
||||
There are two options to convert your MXNet* model that contains custom layers:
|
||||
|
||||
1. Register the custom layers as extensions to the Model Optimizer. For instructions, see [Extending MXNet Model Optimizer with New Primitives](../MO_DG/prepare_model/customize_model_optimizer/Extending_MXNet_Model_Optimizer_with_New_Primitives.md). When your custom layers are registered as extensions, the Model Optimizer generates a valid and optimized Intermediate Representation. You can create Model Optimizer extensions for both MXNet layers with op `Custom` and layers which are not standard MXNet layers.
|
||||
|
||||
2. If you have sub-graphs that should not be expressed with the analogous sub-graph in the Intermediate Representation, but another sub-graph should appear in the model, the Model Optimizer provides such an option. In MXNet the function is actively used for ssd models provides an opportunity to for the necessary subgraph sequences and replace them. To read more, see [Sub-graph Replacement in the Model Optimizer](../MO_DG/prepare_model/customize_model_optimizer/Subgraph_Replacement_Model_Optimizer.md).
|
||||
|
||||
## Kaldi\* Models with Custom Layers <a name="Kaldi-models-with-custom-layers"></a>
|
||||
For information on converting your Kaldi* model containing custom layers see [Converting a Kaldi Model in the Model Optimizer Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi.html).
|
||||
|
||||
## ONNX\* Models with Custom Layers <a name="ONNX-models-with-custom-layers"></a>
|
||||
For information on converting your ONNX* model containing custom layers see [Converting an ONNX Model in the Model Optimizer Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX.html).
|
||||
|
||||
## Step-by-Step Custom Layers Tutorial
|
||||
For a step-by-step walk-through creating and executing a custom layer, see [Custom Layer Implementation Tutorial for Linux and Windows.](https://github.com/david-drew/OpenVINO-Custom-Layers/tree/master/2019.r2.0)
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- Intel® Distribution of OpenVINO™ toolkit home page: [https://software.intel.com/en-us/openvino-toolkit](https://software.intel.com/en-us/openvino-toolkit)
|
||||
- OpenVINO™ toolkit online documentation: [https://docs.openvinotoolkit.org](https://docs.openvinotoolkit.org)
|
||||
- [Model Optimizer Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
|
||||
- [Kernel Extensivility in the Inference Engine Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Integrate_your_kernels_into_IE.html)
|
||||
- [Inference Engine Samples Overview](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Samples_Overview.html)
|
||||
- [Overview of OpenVINO™ Toolkit Pre-Trained Models](https://docs.openvinotoolkit.org/latest/_intel_models_index.html)
|
||||
- [Inference Engine Tutorials](https://github.com/intel-iot-devkit/inference-tutorials-generic)
|
||||
- For IoT Libraries and Code Samples see the [Intel® IoT Developer Kit](https://github.com/intel-iot-devkit).
|
||||
|
||||
## Converting Models:
|
||||
|
||||
- [Convert Your Caffe* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_Caffe.md)
|
||||
- [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
|
||||
- [Convert Your MXNet* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_MxNet.md)
|
||||
- [Convert Your ONNX* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_ONNX.md)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
# Regression tests howto {#openvino_docs_HOWTO_add_regression_test_vpu}
|
||||
|
||||
## Purpose
|
||||
|
||||
This document contains instructions for correctly modifying a set of regression tests.
|
||||
|
||||
## Common
|
||||
|
||||
Regression tests for Myriad and HDDL plugins are on the path:
|
||||
`inference-engine/tests/functional/vpu/regression_tests/`
|
||||
|
||||
The tests are divided into 4 groups:
|
||||
* Classification
|
||||
* Detection
|
||||
* Raw-results
|
||||
* Compilation
|
||||
* VPU hetero
|
||||
|
||||
Testing framework – [Google Test](https://github.com/google/googletest/).
|
||||
Each group contains [parameterized](https://github.com/google/googletest/blob/master/googletest/docs/advanced.md) tests. The main idea is that to add a new test, you only need to add a new parameter. Except for scenarios different from the generalized case.
|
||||
|
||||
## Classsification and Detection tests
|
||||
|
||||
These groups contains two cases:
|
||||
|
||||
* For generalized scenario (` VpuNoClassificationRegression, VpuNoDetectionRegression`)
|
||||
* For specific scenario (` VpuNoClassificationRegressionSpecific, VpuNoDetectionRegressionSpecific`)
|
||||
|
||||
### Generalized scenario
|
||||
|
||||
If You want test new parameter(batch, precision, model and etc.) then You need to edit the existing initialization of parameterized tests or create a new one.
|
||||
Example of initialization of parameterized tests:
|
||||
|
||||
``` c++
|
||||
INSTANTIATE_TEST_CASE_P(
|
||||
VPURegTestWithResources_nightly,
|
||||
VpuNoClassificationRegression,
|
||||
Combine(ValuesIn(VpuTestParamsContainer::testingPlugin()),
|
||||
Values(Precision::FP16),
|
||||
Values(1), // batches
|
||||
Values(true), //IsHwAdaptiveMode
|
||||
Values(false), //DoReshape
|
||||
Values(3, 5, 7), //Resources
|
||||
Values(false), //IsIgnoreStatistic
|
||||
Values(ClassificationSrcParam{ModelName::GoogleNetV1, SourceImages::kCat3, 0.01, Regression::EMean::eValues})),
|
||||
VpuNoClassificationRegression::getTestCaseName);
|
||||
```
|
||||
|
||||
### Specific scenario
|
||||
|
||||
If You need a test to perform some actions that are not provided in the generalized scenario, then add a specific test case. As with the generalized scenario You can change parameters for these tests.
|
||||
Example of specific test case:
|
||||
|
||||
``` c++
|
||||
TEST_P(VpuNoClassificationRegressionSpecific, onAlexNetWithNetworkConfig) {
|
||||
DISABLE_ON_WINDOWS_IF(HDDL_PLUGIN);
|
||||
DISABLE_IF(do_reshape_);
|
||||
|
||||
if (!hw_adaptive_mode_) {
|
||||
config_[VPU_CONFIG_KEY(NETWORK_CONFIG)] = "data=data,scale=1";
|
||||
}
|
||||
|
||||
assertThat().classificationResultsForInferRequestAPI()
|
||||
.on(SourceImages::kDog2)
|
||||
.withInputPrecision(in_precision_)
|
||||
.times(batch_)
|
||||
.withBatch(batch_)
|
||||
.onModel(ModelName::AlexNet)
|
||||
.setMean(Regression::EMean::eImage)
|
||||
.onFP16()
|
||||
.withTopK(1)
|
||||
.withPluginConfig(config_)
|
||||
.equalToReferenceWithDelta(0.04);
|
||||
}
|
||||
```
|
||||
|
||||
## Raw-results tests
|
||||
|
||||
There is no generalized scenario and recommendations are the same as for specific test cases for Classification/Detection groups.
|
||||
|
||||
## Compilation tests
|
||||
|
||||
The tests are in the `vpu_classification_regression.cpp` file and contains only one scenario ` VpuNoRegressionWithCompilation `. To add a new test just update parameters just as in generalized scenarion of Classification/Detection test groups.
|
||||
@@ -0,0 +1,94 @@
|
||||
# Fuzzing howto {#openvino_docs_HOWTO_fuzzing_HOWTO}
|
||||
|
||||
## Intended Audience
|
||||
|
||||
This document is for a developer who wants to contribute fuzz tests.
|
||||
|
||||
## Purpose
|
||||
|
||||
This document walks you through creating your first fuzzer, running it and evaluating its quality.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Linux OS or Mac OS.
|
||||
|
||||
- [American Fuzzy Loop](http://lcamtuf.coredump.cx/afl/) if building with GCC.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Create a fuzz test in the existing project at `./tests/fuzz`. Fuzz test must
|
||||
follow `<test name>-fuzzer.cc` naming scheme and implement a
|
||||
`LLVMFuzzerTestOneInput` entry point.
|
||||
|
||||
``` bash
|
||||
cat << EOF > ./tests/fuzz/test_name-fuzzer.cc
|
||||
#include <stdint.h>
|
||||
#include <cstdlib>
|
||||
|
||||
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
|
||||
// put your fuzzing code here and use data+size as input.
|
||||
return 0; // always return 0
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
2. Implement test logic under `LLVMFuzzerTestOneInput`.
|
||||
|
||||
See example fuzz test at `tests/fuzz/read_network-fuzzer.cc`.
|
||||
|
||||
3. Build fuzz tests with `-DENABLE_FUZZING=ON` flag for cmake.
|
||||
|
||||
``` bash
|
||||
mkdir -p build && \
|
||||
(cd build && \
|
||||
CXX=afl-g++ CC=afl-gcc cmake -DCMAKE_BUILD_TYPE=Debug -DENABLE_FUZZING=ON -DENABLE_TESTS=ON .. && \
|
||||
make fuzz --jobs=$(getconf _NPROCESSORS_ONLN))
|
||||
```
|
||||
|
||||
4. Prepare sample inputs for your fuzz test to teach fuzzer engine on input
|
||||
structure
|
||||
|
||||
``` bash
|
||||
(cd bin/intel64/Debug && \
|
||||
mkdir test_name-corpus && \
|
||||
echo sample input > test_name-corpus/in1.txt)
|
||||
```
|
||||
|
||||
5. Evaluate fuzz test with `afl-fuzz` fuzzing engine
|
||||
|
||||
Run fuzz test:
|
||||
|
||||
``` bash
|
||||
(cd bin/intel64/Debug && \
|
||||
afl-fuzz -i test_name-corpus -o test_name-out -- ./test_name-fuzzer @@
|
||||
```
|
||||
|
||||
While fuzz test is running it prints out statistics. Besides just crashes `uniq
|
||||
crashes` and hangs `uniq hangs` you should care about fuzz test quality:
|
||||
|
||||
- Fuzz test should be fast - speed of execution `exec speed` should be at least
|
||||
100 exec/s. Speed less than 20 exec/s is not acceptable.
|
||||
|
||||
- Fuzz test should be able to explore new code paths `map coverage` and
|
||||
`findings in depth`. Confirm it is increasing while fuzz test is running.
|
||||
|
||||
6. Reproduce fuzz test findings
|
||||
|
||||
All issues found by fuzz test are stored as a file in output folder specified
|
||||
earlier via `-o` afl-fuzz option. To reproduce an issue run fuzz test executable
|
||||
with an issue file as an argument.
|
||||
|
||||
## Summary
|
||||
|
||||
We have created a simple fuzz test, run it and asses its results.
|
||||
|
||||
## Extension
|
||||
|
||||
Try run parallel fuzzing with the help of
|
||||
[afl-utils](https://gitlab.com/rc0r/afl-utils).
|
||||
|
||||
## Tips or FAQs
|
||||
|
||||
GCC 7 in Ubuntu 18.04 LTS has a
|
||||
[defect](https://bugs.launchpad.net/ubuntu/+source/afl/+bug/1774816). Upgrade
|
||||
GCC 7 for AFL to work. GCC version `Ubuntu 7.3.0-27ubuntu1~18.04` works OK.
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c2f362a39ae6c2af080e4f055b6fdba4954f918f85731545d1df3d687d9213d5
|
||||
size 421056
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:cb5c700d003936779455353bfa4ed9432410c0975c46e2dfd30c6a1abccd1727
|
||||
size 23320
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:99d6b5146be85fa408dc5432883c3e2745cffe890133854a97dcf22f5c5962d4
|
||||
size 47564
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0a4de6e502cae7542f1f311bcdbea6bb145f960f0d27d86a03160d1a60133778
|
||||
size 301310
|
||||
Reference in new issue
Block a user