Web and Social Media Image Forensics for News Professionals Markos Zampoglou, Symeon Papadopoulos, Yiannis Kompatsiaris Centre for Research and Technology Hellas (CERTH) – Information Technologies Institute (ITI), Thessaloniki Greece Ruben Bouwmeester, Jochen Spangenberg Deutsche Welle, Bonn/Berlin, Germany Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
20
Embed
Web and Social Media Image Forensics for News Professionals
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Web and Social Media Image Forensics for News Professionals
Centre for Research and Technology Hellas (CERTH) – Information Technologies Institute (ITI),
Thessaloniki Greece
Ruben Bouwmeester, Jochen Spangenberg
Deutsche Welle, Bonn/Berlin, Germany
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Social media for news reporting (1/2)• “Eyewitness media”: an essential component in news
reporting– Photos/videos/audio of unfolding events captured by non-
professionals – Mostly disseminated via social media
• News agencies cannot shun such content, or they might:– miss out on “being first” – lose opportunity to establish contact with sources– lose important sides of the story– miss out on opportunities for user feedback
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Social media for news reporting (2/2)• But, eyewitness media can be untrustworthy
– Material may be manipulated to serve personal ambition or propaganda
– e.g. reusing past images from unrelated contexts, or explicitly tampering with media content
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Sharks in the streets of Puerto Rico (or New Jersey, or Houston…)
Picture of a Moroccan woman claimed to show a Paris attacker
Image verification: tools of the trade
• Metadata analysis– E.g. do the dates/locations match? Is the image already
copyrighted? By whom?• Reverse image search using e.g. Google or TinEye
– Has the image been posted elsewhere? Does it originate from a different context?
• Content analysis for tampering localization– Most commonly, Error Level Analysis (ELA)
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
– Search for self-similarities within the image– Minimize the computational cost of search– Maximize flexibility with respect to transformations
• Splicing– Search for trace inconsistencies within the image– Color Filter Array patterns, JPEG DCT coefficients, camera noise patterns– Can be used to detect many copy-move forgeries as well
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
The pitfalls of image forensics
“MH17 – Forensic Analysis of Satellite Images Released by the Russian Ministry of Defence”
• A report by Bellingcat1 using (amongst other tools) ELA to prove that the Russian MoD had forged the images
• N. Krawetz: Bellingcat “misinterpreted the results”2
• Real-world examples of detections and non-detections• Tutorials and explanations of algorithm mechanics• “Magnifying glass” for examining fine details• Export analysis as a personalized PDF document
including all data and comments by the investigator
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Current challenges (1/2)• Are new protocols necessary?
– E.g. automatic cross-checking of specific metadata fields• Scalability is still an open issue• Interpretation of results is not always intuitive, even in
untampered images
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Current challenges (2/2)
• When applied in real-world cases, algorithms perform significantly worse than in research datasets– This could partly be attributed to the effects of Social Media
dissemination
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Quantitative evaluations (1/3)
• To assess the performance of the implemented algorithms, we evaluated their performance on a large number of images
• Benchmark datasets:– Columbia Uncompressed (Hsu and Chang 2006, “COLUMB”), First
Image Forensics Challenge (“CHAL”)1, Fontani et al 2013 (“FON”)• Additional datasets
– The Wild Web Dataset (Zampoglou et al, 2015)– The Deutsche Welle Image Forensics Dataset
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Quantitative evaluations (3/3)
• Performance on the Wild Web Dataset:
• Performance on the Deutsche Welle Image Forensics Dataset:– Successful detection for 2 out of 6 cases (4 out of 12 images)
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Algorithm DQ GHO BLK ELA DWHF MEDDetections (out of 80) 3 12 3 2 3 1
Time (sec) 0.27 6.12 13.40 1.29 122.13 0.54
Conclusions• Automatic evaluations demonstrate certain limitations
in the state-of-the-art in tampering localization• But:
– Automatic evaluations not the same as human investigation– Tampering localization algorithms are only one verification
tool alongside reverse search and metadata analysis• The REVEAL Media Verification Assistant combines
virtually all known methods currently used for Social Media verification
• So, what next?
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Future steps
• Re-examine the requirements using more formal processes involving large numbers of people
• Evaluate the implemented methods using user feedback instead of automated processes
• Extend the current toolset with novel functionalities– Make outputs more easy to interpret– Keep up-to-date with the state-of-the-art– Incorporate novel proposals by investigators
• Keep improving stability, speed, GUI responsiveness• It will all depend on feedback!
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
References• Ferrara, Pasquale, Tiziano Bianchi, Alessia De Rosa, and Alessandro Piva. "Image forgery
localization via fine-grained analysis of cfa artifacts." Information Forensics and Security, IEEE Transactions on 7, no. 5 (2012): 1566-1577.
• Farid, Hany. "Exposing digital forgeries from JPEG ghosts." Information Forensics and Security, IEEE Transactions on 4, no. 1 (2009): 154-160.
• Fontani, Marco, Tiziano Bianchi, Alessia De Rosa, Alessandro Piva, and Mauro Barni. "A framework for decision fusion in image forensics based on dempster–shafer theory of evidence." Information Forensics and Security, IEEE Transactions on 8, no. 4 (2013): 593-607.
• Lin, Zhouchen, Junfeng He, Xiaoou Tang, and Chi-Keung Tang. "Fast, automatic and fine-grained tampered JPEG image detection via DCT coefficient analysis." Pattern Recognition 42, no. 11 (2009): 2492-2501.
• Mahdian, Babak and Stanislav Saic, “Using noise inconsistencies for blind image forensics,” Image and Vision Computing, vol. 27, no. 10, pp. 1497–1503, 2009.
• Zampoglou, Markos, Symeon Papadopoulos, and Yiannis Kompatsiaris. "Detecting image splicing in the wild (WEB)." In Multimedia & Expo Workshops (ICMEW), 2015 IEEE International Conference on, pp. 1-6. IEEE, 2015.
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany