This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc.OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos
Intel® Parallel Studio XE Intel® Media SDK / Media Server StudioIntel® System Studio Intel® Distribution of OpenVINO™ toolkit
What’s Inside Intel® Distribution of OpenVINO™ toolkit
OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc.OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos
Intel® Architecture-Based Platforms Support
OS Support CentOS* 7.4 (64 bit) Ubuntu* 16.04.3 LTS (64 bit) Microsoft Windows* 10 (64 bit) Yocto Project* version Poky Jethro v2.0.3 (64 bit) MacOS
Intel® Deep Learning Deployment Toolkit Traditional Computer Vision
Model Optimizer Convert & Optimize
Inference EngineOptimized InferenceIR
OpenCV* OpenVX*
Optimized Libraries & Code Samples
IR = Intermediate Representation file
For Intel® CPU & GPU/Intel® Processor Graphics
Increase Media/Video/Graphics Performance
Intel® Media SDKOpen Source version
OpenCL™ Drivers & Runtimes
For GPU/Intel® Processor Graphics
Optimize Intel® FPGA (Linux* only)
FPGA RunTime Environment(from Intel® FPGA SDK for OpenCL™)
Bitstreams
Code Samples
An open source version is available at 01.org/openvinotoolkit (some deep learning functions support Intel CPU/GPU only).
OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc.OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos
Integrate Deep learning
Maximize the Power of Intel® ProcessorsCPU, GPU/Intel® Processor Graphics, FPGA,VPU
Intel® Deep Learning Deployment Toolkit For Deep Learning Inference
Caffe*
TensorFlow*
MxNet*.dataIR
IR
IR = Intermediate Representation format
Load, infer
CPU Plugin
GPU Plugin
FPGA Plugin
Myraid Plugin
Model Optimizer
Convert & Optimize
Model Optimizer
What it is: A python based tool to import trained models and convert them to Intermediate representation.
Why important: Optimizes for performance/space with conservative topology transformations; biggest boost is from conversion to data types matching hardware.
Inference Engine
What it is: High-level inference API
Why important: Interface is implemented as dynamically loaded plugins for each hardware type. Delivers best performance for each type without requiring users to implement and maintain multiple code pathways.
Trained Models
Inference Engine
Common API (C++ / Python)
Optimized cross-platform inference
OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos
GPU = Intel CPU with integrated graphics processing unit/Intel® Processor Graphics
Simple & Unified API for Inference across all Intel® architecture
Optimized inference on large IA hardware targets (CPU/GEN/FPGA)
Heterogeneity support allows execution of layers across hardware types
Asynchronous execution improves performance
Futureproof/scale your development for future Intel® processors
Transform Models & Data into Results & Intelligence
MovidiusPlugin
OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc.OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos
GPU = Intel CPU with integrated graphics/Intel® Processor Graphics/GENGNA = Gaussian mixture model and Neural Network Accelerator
GNA API
Intel® GNA
GNA* Plugin
Optimal Model Performance Using the Inference Engine
OpenVINO™ toolkit includes optimized pre-trained models that can expedite development and improve deep learning inference on Intel® processors. Use these models for development and production deployment without the need to search for or to train your own models.
Speed Deployment with Intel Optimized Pre-trained Models
Intel Confidential
Age & Gender
Face Detection – standard & enhanced
Head Position
Human Detection – eye-level & high-angle detection
Detect People, Vehicles & Bikes
License Plate Detection: small & front facing
Vehicle Metadata
Human Pose Estimation
Vehicle Detection
Retail Environment
Pedestrian Detection
Pedestrian & Vehicle Detection
Person Attributes Recognition Crossroad
Emotion Recognition
Identify Someone from Different Videos – standard & enhanced
Intel® Deep Learning Deployment Toolkit For Deep Learning Inference
Caffe*
TensorFlow*
MxNet*.dataIR
IR
IR = Intermediate Representation format
Load, infer
CPU Plugin
GPU Plugin
FPGA Plugin
Myraid Plugin
Model Optimizer
Convert & Optimize
Model Optimizer
What it is: A python based tool to import trained models and convert them to Intermediate representation.
Why important: Optimizes for performance/space with conservative topology transformations; biggest boost is from conversion to data types matching hardware.
Inference Engine
What it is: High-level inference API
Why important: Interface is implemented as dynamically loaded plugins for each hardware type. Delivers best performance for each type without requiring users to implement and maintain multiple code pathways.
Trained Models
Inference Engine
Common API (C++ / Python)
Optimized cross-platform inference
OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos
GPU = Intel CPU with integrated graphics processing unit/Intel® Processor Graphics
• Launch the Model Optimizer for the Caffe bvlc_alexnet model with the output IR called result.* in the specified output_dir:
• Launch the Model Optimizer for the Caffe bvlc_alexnet model with multiple inputs with scale and mean values specified for the particular nodes:
• Launch the Model Optimizer for the Caffe bvlc_alexnet model with reversed input channels order, specified mean values to be used for the input image per channel, and specified data type for input tensor values
python3 mo.py --input_model bvlc_alexnet.caffemodel --model_name result --output_dir /../../models/
/** cpu_extensions library is compiled from "extension" folder custom MKLDNNPlugin layer implementations. These layers are useful for inferring custom topologies. **/plugin.AddExtension(std::make_shared<Extensions::Cpu::CpuExtensions>());
}
if (!FLAGS_l.empty()) {// CPU(MKLDNN) extensions are loaded as a shared libraryauto extension_ptr = make_so_pointer<IExtension>(FLAGS_l);plugin.AddExtension(extension_ptr);
}if (!FLAGS_c.empty()) {
// clDNN Extensions are loaded from an .xml description and OpenCL kernel filesplugin.SetConfig({{PluginConfigParams::KEY_CONFIG_FILE, FLAGS_c}});
Load the network to the plugin and get the ExecutableNetwork// --------------------------- Loading model to the plugin ------------------------------------------
ExecutableNetwork executable_network = plugin.LoadNetwork(network, {});network = {}; //release the network datanetworkReader = {}; //release the network reader
auto data = input->buffer().as<PrecisionTrait<Precision::U8>::value_type*>();…
Inferinfer_request.Infer();
Post-Process
const Blob::Ptr output_blob = infer_request.GetBlob(firstOutputName);auto output_data = output_blob->buffer().as<PrecisionTrait<Precision::FP32>::value_type*>();…Many output formats. Some examples:• Simple classification: an array of float confidence scores, # of elements=# of classes in the model• SSD: many “boxes” with a confidence score, label #, xmin,ymin, xmax,ymax
Optimize Vision/Smart Video Solutions from Edge to Cloud
Smart Cameras
Video Gateways
Data Center and Cloud
Clients
Intel® Software Tools Help Developers Accelerate, Innovate, and Differentiate
Speed Video Encode/Decode & Image Processing, CompressionIntel® Media Server Studio & Intel® Media SDK
Boost Performance, System Bring-up & Power Efficiency, Debug with Intel® System StudioAccelerate Data Center/Cloud Workloads with Intel® Parallel Studio XE
Accelerate Computer Vision Solutions, Integrate Deep Learning InferenceIntel® Distribution of OpenVINO™ toolkit
Customize Solutions, Optimize Compute, Heterogeneous ProgrammingIntel® SDK for OpenCL™ Applications & Intel® FPGA SDK for OpenCL™ software technology
Deliver Fast, Efficient, High Quality Video/Computer Vision Processing End to End
Key Vision Solutions Optimized by Intel® Distribution of OpenVINO™ toolkit
Intel teamed with Philips to show that servers powered by Intel® Xeon® Scalable processors & Intel® Distribution of OpenVINO™ toolkit can efficiently perform deep learning inference on patients’ X-rays & computed tomography (CT) scans, without the need for accelerators. Achieved breakthrough performance for AI inferencing:
188x increase in throughput (images/sec) on Bone-age prediction model.
38x increase in throughput (images/sec) on Lung segmentation model.
“Intel®Xeon®Scalable processors and OpenVINO toolkit appears to be the right solution for medical imaging AI workloads. Our customers can use their existing hardware to its maximum potential, without having to complicate their infrastructure, while still aiming to achieve quality output resolution at exceptional speeds."— Vijayananda J., chief architect and fellow, Data Science and AI, Philips HealthSuite Insights, India
The Intel® Distribution of OpenVINO™ toolkit helped GE deliver optimized inferencing to its deep learning image-classification solution. By bringing AI to its clinical diagnostic scanning, GE no longer needed an expensive 3rd party accelerator board, achieving:
5.9x inferencing performance above the target
14x inferencing speed over the baseline solution
Improved image quality, diagnostic capabilities, and clinical workflows
With the OpenVINO™ toolkit , we are now able to optimize inferencing across Intel® silicon, exceeding our throughput goals by almost 6x,” said David Chevalier, Principal Engineer for GE Healthcare. “We want to not only keep deployment costs down for our customers, but also offer a flexible, high-performance solution for a new era of smarter medical imaging. Our partnership with Intel allows us to bring the power of AI to clinical diagnostic scanning and other healthcare workflows in a cost-effective manner.”
GE Healthcare*5
Intel-GE Healthcare, Intel® Software Development Tools Optimize Deep Learning Performance for Healthcare Imaging
Key Vision Solutions Optimized by Intel® Distribution of OpenVINO™ toolkit
Intel® Pentium® processor N4200/5, N3350/5, N3450/5 with Intel® HD Graphics Yocto Project* Poky Jethro v2.0.3 (64 bit)
Iris® Pro & Intel® HD Graphics 6th-8th generation Intel® Core™ processor with Intel® Iris™ Pro graphics & Intel® HD Graphics 6th-8th generation Intel® Xeon® processor with Intel® Iris™ Pro Graphics & Intel® HD Graphics
(excluding e5 product family, which does not have graphics1)
FPGA Intel® Arria® FPGA 10 GX development kit Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA operating systems OpenCV* & OpenVX* functions must be run against the CPU or Intel® Processor Graphics (GPU)
Linux* build environment required components OpenCV 3.4 or higher GNU Compiler Collection (GCC) 3.4 or higher CMake* 2.8 or higher Python* 3.4 or higher
Microsoft Windows* build environment required components Intel® HD Graphics Driver (latest version)†
OpenCV 3.4 or higher Intel® C++ Compiler 2017 Update 4 CMake 2.8 or higher Python 3.4 or higher Microsoft Visual Studio* 2015
External Dependencies/Additional Software View Product Site, detailed System Requirements
421Graphics drivers are required only if you use Intel® Processor Graphics (GPU).
Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision #20110804
43
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks.
INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS”. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc.OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos