Developing Video Signal Processing Algorithms for Embedded Vision Systems

PowerPoint Presentation

Developing Video Signal Processing Algorithms for Embedded Vision SystemsShogo MURAMATSUDept. of Elec. & Electronic Eng.Niigata University

1

ContentsIntroduction to Embedded Vision Tutorial Material with MATLAB/Simulink EmbVision TutorialExample ActivitiesSmart Dormitory Program (PBL Example)NSOLT Project (Research Example)Conclusions

2015/8/26SISA 2015 @ Chiba Institute of Tech.Introduction to Embedded Vision Tutorial Material with MATLAB/SimulinkEmbVision TutorialExample ActivitiesSmart Dormitory Program (PBL Example)NSOLT Project (Research Example)Conclusions

2

The IoT Era is Beginning.Diversification of Sensing Environment

2015/8/26SISA 2015 @ Chiba Institute of Tech. Embedded Systems will be Deployed Broadly.

3

Seeing is Believing. Rapid Proliferation of Vision Systems

2015/8/26SISA 2015 @ Chiba Institute of Tech.Embedded Vision Systems gather Attention.

ContentsIntroduction to Embedded Vision Tutorial Material with MATLAB /Simulink EmbVision TutorialExample ActivitiesSmart Dormitory Program (PBL Example)NSOLT Project (Research Example)Conclusions

2015/8/26SISA 2015 @ Chiba Institute of Tech.

5

From DSP First to SI FirstDSP First: A Multimedia Approach (1998)Presents basic DSP concepts in an intuitive style by multimedia signals with MATLAB.

Evolution of MATLAB makes the System Integration (SI) First Approach available.Embedded System DevelopmentWeb + Database Management Network Connection

DB

2015/8/26SISA 2015 @ Chiba Institute of Tech.

EmbVision TutorialEducational Material for new members in our Lab. (Junior or Senior) Estimated to take 12H

SISA 2015 @ Chiba Institute of Tech.2015/8/26Today, condensed to50min!http://msiplab.eng.niigata-u.ac.jp/embvision/en/

Demo with Raspberry PiGradient Filtering on Raspberry Pi (Final Exercise)

2015/8/26SISA 2015 @ Chiba Institute of Tech.

Overview of EmbVision TutorialOne can implement user-defined System objects on Raspberry Pi.2015/8/26SISA 2015 @ Chiba Institute of Tech.Part 1Part 2Part 3Part 4Part 5Part 6IntensityAdjustmentColor spaceConv.Grad.FilterSystemObjectVideo StreamProcess.MATLABSystemblockUnit TestRaspberry Pi

MATLABSimulink

Part 1: Image I/O and Pixel ProcessingTargetRead, display, and write imagesSimple pixel processingExercisesIntensity Adjustment

Color Space Conversion

2015/8/26SISA 2015 @ Chiba Institute of Tech.

RGB2HSVHSV2RGB

Part 2:Filtering and Frequency AnalysisTarget1-D signal and 2-D image filteringFrequency analysisExercisesHorizontal Differential Filter

Magnitude and Direction of Gradient

2015/8/26SISA 2015 @ Chiba Institute of Tech.

Part 3:Class Definition and Unit TestingTarget Object-oriented programming in MATLABUnit testing framework in MATLAB.ExercisesHSV2RGB ClassGradient Filter Class

2015/8/26SISA 2015 @ Chiba Institute of Tech.

12

Part 4: Video Stream Processing - MATLAB -TargetRead, display and write videos in MATLABVideo stream processing in MATLABExercisesSobel Gradient FilterFrame Difference2015/8/26SISA 2015 @ Chiba Institute of Tech.

Part 5: Video Stream Processing - Simulink -TargetRead and display videos with SimulinkUse MATLAB System blocks on SimulinkVideo stream processing with SimulinkExercisesPrewitt Gradient FilterSobel Gradient Filter

2015/8/26SISA 2015 @ Chiba Institute of Tech.

Part 6: Video Stream Processing - Raspberry Pi -TargetSimulate Simulink model in external modeDeploy Simulink model on Raspberry PiExercisesSobel Gradient FilterDeploy a created new model(Option)

2015/8/26SISA 2015 @ Chiba Institute of Tech.

DIYList of Expected Extension ExamplesRaspberry Pi Camera Module ControlFixed-point ImplementationParallel Implementation (for Quad Core on RasPi2)Detection & Recognition of ObjectsSpeech & Audio Signal ProcessingGPIO/I2C ControlsNetwork Application Development etc.

2015/8/26SISA 2015 @ Chiba Institute of Tech.

ContentsIntroduction to Embedded Vision Tutorial Material with MATLAB/SimulinkEmbVision TutorialExample ActivitiesSmart Dormitory Program (PBL Example)NSOLT Project (Research Example)Conclusions

2015/8/26SISA 2015 @ Chiba Institute of Tech.

17

Smart Dormitory ProgramProject-Based Learning (PBL) Program at Facul. of Eng., Niigata Univ.Financially Supported by MEXTMEXT: Ministry of Education, Culture, Sports, Science and Technology15 Research Groups are working on their research activities.The 2nd group is Cyber Physical System (CPS) DormitoryMembers are from different grades and departments.Develop an IoT App. with embedded vision systems2015/8/26SISA 2015 @ Chiba Institute of Tech.

CPS DormitoryDevelops a System for Estimation and Visualization of Customer Congestion at a Student CafeteriaBeagleBone Black boards as Sensor NodesThingSpeak as an IoT data collection plathome

Installation of a Sensor NodeDeployment Map of Sensor Nodes2015/8/26SISA 2015 @ Chiba Institute of Tech.

19

Presentation at the 4th MEXT Science Inter-College 2015/8/26SISA 2015 @ Chiba Institute of Tech.

@Kobe Convention Center29th Feb. - 1st Mar., 2015

Nonseparable Oversampled Lapped TransformSparsity-Aware Image and Volume Data Restoration

(a) (b)

NSOLT Project[Muramatsu,ICASSP2014]2015/8/26SISA 2015 @ Chiba Institute of Tech.

Simulink Model

21

Zynq ImplementationXilinxs All Programmable SoCARM CPU and FPGA are connected to each other through AXI4 interface on a single chip

2015/8/26SISA 2015 @ Chiba Institute of Tech.

Xilinxs Zynq

Inside of Zynq

Workflow of Co-implementationGeneration of HDL(HDLCoder)Generation of C(Embedded Coder )ConfigurationBuildZynq2015/8/26System Model of NSOLT(MATLAB/Simulink)Isolation of HW and SW in the modelSISA 2015 @ Chiba Institute of Tech.

23

ContentsIntroduction to Embedded Vision Tutorial Material with MATLAB/SimulinkEmbVision TutorialExample ActivitiesSmart Dormitory Program (PBL Example)NSOLT Project (Research Example)Conclusions

2015/8/26SISA 2015 @ Chiba Institute of Tech.

24

ConclusionsThis tutorial explainedHow to develop and evaluate video signal processing algorithms with MATLAB/SimulinkHow to implement visual applications on embedded systems with MATLAB/SimulinkTwo example activities were introducedCPS Dormitory Prog. as an Educational ActivityNSOLT Project as a Research Activity2015/8/26SISA 2015 @ Chiba Institute of Tech.