Presentazione standard di PowerPoint · 2019-11-05 · SAM Saliency /context detection Soft-attentive LSTM Text generation LSTM. Toward an explainable AI: What the Machine pays attention

Visual Intelligence

Prof. Rita Cucchiara

AimageLab, Dipartimento di Ingegneria «Enzo Ferrari»Università di Modena e Reggio Emilia, ItalyDirector of the National CINI Lab AIIS

“Like the steam engine or electricity in the past, AI is transforming our world, our society and our industry.

Growth in computing power, availability of data and progress in algorithms have turned AI into one of the most strategic technologies of the 21st century.”

Artificial Intelligence for Europe - Brussels, 25.4.2018

Artificial Intelligence

“AI refers to systems that display intelligent behaviour by analysing their environment and taking actions – -with a certain degree of autonomy- to achieve a specific goal.”

Artificial Intelligence for Europe - Brussels, 25.4.20182018

Artificial Intelligence

Machine Learning

Deep Learning

Game theory

Knowledge representation

Automated Reasoning

Logics

Computer Vision

Pattern Recognition

Natural Language Processing

Cognitive Robotics

IntelligentIoT

Speech Recognition

Multi-Agents

Fuzzy systems

…systems that display intelligent behaviour by analysing their environment and taking actions

National Lab CINI AIIS:

51 nodes ( 47 universities, CNR, IIT, FBK)910 members

>100 Labs>700 projects>80 spinoff

National CINI Lab AIISArtificial intelligence andIntelligent systems

UNIMORE Modena , Italy

AImageLab Dipartimento di Ingegneria «Enzo Ferrari» & Modena Technopole

AIRI AI Research & Innovation Center; 36 People working in AI, ML and CV

ComputerVision

PatternRecognition

MachineLearning

Deep Learning with (Artificial)

Neural NetworksIntelligentInference

Action

Display intelligent behaviour

Analyse the environment

Take actions

Visual Intelligence

Where do you put your attention?

What do you predict while driving?

Saliency or task-driven attention?

Visual intelligence

Visual Intelligence for a better human and machine mutual Comprehension

..l’arte suprema di saper vedere..

Visual intelligencecan helpmachines

Interacting with AI

Viual Intelligence

Interacting with AI

by collaborative robotics

Interacting with AI

AI, data understanding and visual Intelligence

can helphumans in controlling Machines in cyber

Visual Intelligencefor secuirty

Imagine to haveImagination

Fakes from Art

Art2Realby GANs

[M. Tomei, M. Cornia, L. Baraldi, R. Cucchiara. “Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation”. CVPR 2019]

AI recognizes

YOU and your

ancestors!

Visual Intelligence is Imagination and Hallucination

DeepFakes

https://www.cnn.com/interactive/2019/01/business/pentagons-race-against-deepfakes/

REAL FAKE

AI can helpdesigners

AI can helpall of usin security and smart cities

PrEVUE

..l’arte suprema di saper vedere..

Saliency

SALIENCY [Itti Koch PAMI 1989]

[ Itti and Koch PAMI ’89, Nature Reviews 2001]“Saliency map”: an image map representing areas of saliency

Saliency: data-driven, perceptual or semantics driven?

• SAM

Saliency Attentive Model (SAM) @ AImageLab

M.Cornia, L.Baraldi, G.Serra, R.Cucchiara

Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model

IEEE Transactions on Image Processing, 2018Ranked #1 at LSUN Competition CVPR2017

SAM

Refine with an iterative (LSTM-based) model the saliency detection

Trained with generic images

(SALICON MIT300).. Now

SAM can explore the world

Saliency and Attention

[A.Palazzi,D. Abati, Davide S. Calderara, and R.Cucchiara Predicting the Driver's Focus of Attention: the DR(eye)VE Project IEEE Transactions on Pattern Analysis and Machine Intelligence 2018]

Tell me what you see

Saliency and Attentionin image captioningfrom Vision to Language

Saliency and Captioning

[M Cornia, L. Baraldi, G. Serra, R. Cucchiara Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention ACM TOMM 2018 ]

ResNet -50, trained with Imagenet;

SAM Saliency /context detectionSoft-attentive LSTM

Text generation LSTM

Toward an explainable AI: What the Machine pays attention of when is describing the scene

Attention to details

Where people are, what people are doing, what the people see.

Ball in hand?

Learning to put Attention in details

People detection

People Join detection

People Join detection

[M. Fabbri, F. Lanzi , S. Calderara, R. Cucchiara, Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World ECCV2018]

Hallucinating occluded joints

Hallucinating third dimension of (occluded) joints

For Machine and Human Mutual Comprehension

Human Behavior Understanding

Future HMI

ControllableExplainableCorrectable

Captioning for Explainable Reasoning

A penny for your thoughts

Captioning for Explainable Reasoning

A penny for your thoughts

Captioning for Explainable Reasoning

Imagine to understandwhat the robot sees

Captioning for Explainable Reasoning

L. Baraldi, R.CucchiaraExplainable Robot-World interactionArxiv 2019.

AI ALGORITHMS & ARCHITECTURESAI DATAAI HARDWARE

What do you need?

What you don’t need..

1 ignorance2 negligence3 malevolency4 skepticism

Ma c’e’ una magia che e’ opera divinaLa’ dove la scienza di Dio si manifestaattraverso la scienza dell’uomo…

(U.Eco 1984)

THANKS

Thanks to all AImageLab UNIMORE