Non-intrusive Apnoea / Hypopnoea Detection System via MS Kinect captured Depth Video Analysis Cheng Yang # , Yu Mao ∗ , Gene Cheung ∗ , Vladimir Stankovic # , Kevin Chan % # Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow, UK ∗ The Graduate University for Advanced Studies, National Institute of Informatics, Tokyo, Japan % School of Medicine, University of Western Sydney, Camden and Campbelltown Hospitals, Sydney, Australia Simon Fraser University, Nov 13, 2014 1
20
Embed
Non-intrusive Apnoea / Hypopnoea Detection …research.nii.ac.jp/~cheung/sfu2014_sleep_handout.pdfNon-intrusive Apnoea / Hypopnoea Detection System ... We propose a dual-ellipse model
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Non-intrusive Apnoea / Hypopnoea Detection System
via MS Kinect captured Depth Video Analysis
Cheng Yang#, Yu Mao∗, Gene Cheung∗, Vladimir Stankovic#, Kevin Chan%
#Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow, UK
∗The Graduate University for Advanced Studies, National Institute of Informatics, Tokyo, Japan
%School of Medicine, University of Western Sydney, Camden and Campbelltown Hospitals, Sydney, Australia
Simon Fraser University, Nov 13, 2014
1
Motivation Sleep-disordered breathing is common in the general population.
Repeated episodes of apnoea / hypopnoea can significantly disturb a person’s sleep.
Existing sleep monitoring systems:
1. Vibration-sensing wrist-bands (Fitbit, Jawbone UP) – Minimally intrusive; Mostly record sleep time only.
2. Full multi-sensing monitoring device (Philips Alice PDx) – Accurate in detecting vital signs; expensive and intrusive.
A previous non-intrusive sleep monitoring system w/ depth video:
M.-C. Yu et al., “Breath and position monitoring during sleeping with a depth camera,” in Int’l
Conference on Health Informatics, Vilamoura, Portugal, Feb 2012.
1. Torso movement detection - cannot distinguish chest/abdomen movements individually.
2. Not robust: mount a MS Kinect on the ceiling and measure the distance to the body – will not work if
the patient sleeps sideway.
We propose a dual-ellipse model (to be discussed) to detect chest/abdomen movements individually –
possible to track the breathing cycle even if the patient is sleeping in sideway.
3
Our Sleep Monitoring System
GOAL: a non-intrusive apnoea / hypopnoea detection system using
MS Kinect depth video.
System site: Bondi Junction Private Sleep Laboratory in Sydney,
Australia.
Targeted subjects: patients admitted for diagnostic sleep studies.
Operate in a completely dark room (active infrared sensing).
Unobstructed view of the patient’s upper body.
3 components:
1. 11-to-8-bit depth video coding.
2. Temporal depth video denoising.
3. Sleep-event classification.
pillow
patient
Kinect depth video
4C. Yang, G. Cheung, K. Chan, V. Stankovic, “Sleep Monitoring via Depth Video Recording & Analysis,” 5th IEEE
International Workshop on Hot Topics in 3D (Hot3D), Chengdu, China, July, 2014.
graph signal processing
Depth Video Coding
Given 11-bit depth images captured by MS Kinect 1.0, alternately encode 8 LSBs / MSBs as 8-bit images via H.264 video.
At receiver, recover missing 3 MSBs in each block using block-based motion estimation (overlapped bit matching / MV smoothness criteria).
MSB frame LSB frame
5C. Yang, G. Cheung, K. Chan, V. Stankovic, “Sleep Monitoring via Depth Video Recording & Analysis,” 5th IEEE
International Workshop on Hot Topics in 3D (Hot3D), Chengdu, China, July, 2014.
Microsoft Kinect
Temporal denoising & event classificationTwo tasks leveraging on recent advances in graph signal processing (GSP) [1]:
1. Graph-based temporal denoising:
Reduce temporal flicker in depth video via a graph-signal smoothness prior.
2. Graph-based classification:
Robustly classify apnoea / hypopnoea vs. normal breathing via graph-signal interpolation.
[1] D. I. Shuman, et al., “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,” IEEE Signal Processing Magazine, vol. 30, no.3, May 2013, pp. 83–98.
C. Yang, Y. Mao, G. Cheung, V. Stankovic, K. Chan, "Graph-based Depth Video Denoising and Event Detection for Sleep Monitoring," IEEE International Workshop on Multimedia Signal Processing, Jakarta, Indonesia, September, 2014.
6
Common thread: a piecewise smooth signal x has small graph Laplacainregularization term xx LT
Related Work (cont’d)
Spatial denoising based on Graph Fourier Transform (GFT), but not temporal:
W. Hu et al., “Depth map denoising using graph-based transform and group sparsity,” in IEEE MMSP, Italy, Oct 2013.
Data classification using GSP tools:
A. Sandryhaila and J. Moura, “Classification via regularization on graphs,” in IEEE GlobalSIP, Austin, TX, Dec 2013.
We propose a more intuitive and less complex graph-based classification for event detection.
7
Problem: temporal flicker across Kinect depth frames in time.
Proposal: Graph-based temporal denoising
Piecewise smoothness (PWS) of graph-signal x can be measured using , where L is graph Laplacian.
Signal prior: Motion vector (MV) field is PWS!
Draw graph for MVs of pixel blocks in denoised frame t-1 and noisy frame t.
Use in denoising objective as regularization term.
Temporal Depth Video Denoising
Frame t - 1 Frame t
Target blocksPredictor blocks
𝑤𝑖,𝑗 = exp −𝐯𝑖 − 𝐯𝑗 2
2
𝜎𝑣2
MV: 𝐯𝑖 = 𝑥𝑖 , 𝑦𝑖
MV field: 𝐯 = 𝐯1, … , 𝐯𝑁
𝐋 = 𝐃 𝐀−
xLxT
xLxT
ji,
2
ji,
T wxLx ji xx
8
Temporal Depth Video Denoising (cont’d)Objective: Find the optimal Motion Vector (MV) field and denoise blocks in frame t given:
i) denoised frame t-1 and ii) noisy frame t.
Graph Construction:
Draw an edge between two spatial neighbouring blocks of same frame (intra-frame edge).
Draw an edge between target and predictor blocks of different frames (inter-frame edge).