人類內在感受與外在表現落差時 之語音、生理特徵表現 Group 05 B05901192 張晁維 B05901017劉昶樂 B05901021 賴明緯
人類內在感受與外在表現落差時之語音、生理特徵表現
Group 05
B05901192 張晁維
B05901017劉昶樂
B05901021賴明緯
問題意識
Stefano Sandrone et al.” Weighing brain activity with the balance: Angelo Mosso’s original manuscripts come to light”, Brain, Volume 137, Issue 2, 1 February 2014, Pages 621–633
初代測謊機
International League of Polygraph Examiners
J E REID; F E INBAU. TRUTH AND DECEPTION - THE POLYGRAPH ('LIE-DETECTOR') TECHNIQUE, 2D ED.
認知心理學
Vrij A. et al.( 2008). Increasing cognitive load to facilitate lie detection: The benefit of recalling an event in reverse order. Law and Human Behavior, 32, 253–265.
研究現況與面向
• Deception and behavior
• Cognitive load approaches
• Systematic verbal lie detection approaches
• Psychophysiological detection of deception
Deception and Behavior
• observation of behavior is hard to detect deception
(Charles F. Bond Jr., Bella M. DePaulo (2006))
• connection between lying and non verbal cues is weak
(Sporer, S. L., & Schwandt, B. (2007))
• meta analytical evidence question the utility of behavior as a source of deception markers
Cognitive load approaches
• Aim at designing interview strategies oriented to produce behavioral differences between truth tellers and liars
• Vrij et al. (2010) argued that creating a lie might require more cognitive effort than just describing an episodic memory
• Strategies like imposing cognitive load(e.g. describe the event in reverse order, stare at the interviewer’s eyes, etc.), encouraging interviewee to say more, and asking unexpected questions.
(Vrij, Fisher, & Blank, 2017)
• Pros: stronger theoretical bases (Walczyk et al.’s (2014))
Systematic verbal lie detection approaches
• Reality Monitoring(RM)
• Criteria-Based Content Analysis(CBCA)/Statement Validity Assessment(SVA)• a clinical assessment procedure rather than a standardized psychometric test.
• Both RM and CBCA/SVA are based on the notion that the verbal descriptions of self-experienced events differ from those of imagined or invented events.
• Pros: high-quality field studies for forensic cases
Psychophysiological detection of deception
• Comparison Question Test (CQT) and Concealed Information Test (CIT) with Polygraph• CQT: examinee are asked with a series of relevant, irrelevant, and comparison questions, and
they are expected to display strong physiological responding just after relevant one.• CIT: examinee are asked with a series of multiple-choice questions, and only those who have
knowledge about the crime details show stronger physiological action to correct alternatives than incorrect ones.
• Event-Related Potentials(EPRs):• examine EEG waves that appears accompanied with the recognition of meaningful.(Iacono, 2015; Rosenfeld, 2011)
• fMRI:• Compare and identify truth-teller and liar by observing fMRI brain activated area during
deception(Langleben et al., 2016)
Psychophysiological detection of deception
• Cons: • All of these method are vulnerable to countermeasures, and thus measure
methods need revisions
• sensitivity and specificity of polygraph depends on examiner using CQT or CIT
• ERPs and fMRI classification rate are not always better that those obtained by polygraph.
• Pros: • fMRI and ERPs provides more data for further analysis
• Developing potentials
語音壓力偵測Voice Stress Analysis (VSA)
• Jitter• 微肌群震顫(micro-muscle tremors,MMT)-Lippold 1971• 隨意肌的收縮正常時,以每秒大約 10 個週期小幅振盪• 說謊時,由於壓力,使肌肉協調性改變(喉嚨肌肉拉緊),導致在 8~12Hz 的生理性震顫減少
• PSE、CVSA、LVA系統
• Pitch• 75~500Hz• (a)無壓力 (c)有壓力(平坦)
Lippold, O., "Physiological tremor", Scientific American, vol. 224, pp. 65-73, 1971.
CSC DatasetDistinguishing deceptive from non-deceptive speech – Hirschberg et al.
• 32 hours of speech data from 32 English speakers
• Task: To convince the interviewer that they possessed thecharacteristics of a successful entrepreneur. Anytime they answeredthe interviewer’s question with a lie, the subject would press a pedalunder the table indicating the truth value of their statement.
• Without ”high stakes” scenarios (the subject is very likely toexperience fear or shame, such as testifying in court after swearing onthe Bible)
• Only able to marginally improve their error rate with further featureconstruction/selection
Detecting Lies via Speech Patterns – Chow et al
LR = Logistic regression, SVC = support vector classifier, GB = gradient boosting classifier, MLP = multi-layer perceptron classifier, BAG = bagging classifier and RNN = recurrent neural network;
Neural Lie Detection with the CSC Deceptive Speech Dataset-Shloka Desai et al.• PCA features
• RNN-based
Neural Lie Detection with the CSC Deceptive Speech Dataset-Shloka Desai et al.
A Deep Learning Approach for Multimodal Deception Detection – Krishnamurthy et al(Mar/2018)
• Use textual, audio and visual features.• Visual Feature Extraction-3D CNN
• Textual Features Extraction- pretrained Word2Vec + CNN
• Audio Feature Extraction – openSMILE (open-source toolkit used to extract high dimensional features from audio)
• Micro-Expression Features
• MLP_U: Unimodal
• Data Fusion: MLP_C – Concatenation; MLP_H+C - Hadamard + Concatenation
What We Want To Do?
• Detecting deception use ONLY audio signals by deep learning.
Detection of Pathological Voice Using CepstrumVectors: A Deep Learning Approach - Fang et al
預估使用器材
全指向性電容麥克風 NTD 289 3支
USB音效卡 NTD 80 3件
HUB集線器 NTD 339 1件
Reference
• https://www.ncbi.nlm.nih.gov/pubmed/26794034
• https://onlinelibrary.wiley.com/doi/full/10.1111/lcrp.12088
• http://www.psicothema.com/PDF/4376.pdf
• https://www.ncbi.nlm.nih.gov/pubmed/26787599
• http://psycnet.apa.org/record/2007-13995-005
• http://psycnet.apa.org/record/2007-01724-001
• http://journals.sagepub.com/doi/abs/10.1207/s15327957pspr1003_2
• https://people.ok.ubc.ca/stporter/Publications_files/Pitfalls%20and%20Opportunities-FINAL.pdf
• https://www.sciencedirect.com/science/article/pii/S0732118X14000142