Prof. Roberto VezzaniLaboratorio Imagelab, http://imagelab.ing.unimore.it

Dipartimento di Ingegneria «Enzo Ferrari»

Università degli Studi di Modena e Reggio Emilia, Italia

Lo stato della Computer Vision

Responsabile del laboratorio: Prof. Rita CucchiaraMembri: 4 strutturati, circa 10 dottorandi e 10 assegnisti

http://imagelab.ing.unimore.it

Projects and Collaboration (2017)

• Industrial Projects: • Ferrari, Voilap, Sata, …

• Regional projects: • Jump, Sacher, (EU FESR 2015-2017)• FAR Driver attention ( UNIMORE- Stanford project)• HAxIA• Vaex Augmented experience in cultural heritage

• National Projects: • Cluster Smart City and Community «Città Educante» (Italian MIUR- Project 2016-2018)

• International collaborations:• Facebook Artificial Intelligence Research : selected as 15 FAIR European Labs• ISCRA Italian SuperComputing Resource Allocation – CINECA 2016.2017,• EU Horizon 2020 Advance• Panasonic CA, USA Phd Student exchange

Research activities

• Videosurveillance & Human Behavior Understanding

• Computer Vision, Pattern Recognition, Machine Learning and Deep Learning

• Multimedia and Visual Big Data

• Automotive

• Sensors, mobile and embedded vision

From traditional CCTV …

… to computer vision based systems

Surveillance applications

Surveillance applications

’70: 1975 DARPA Image Understanding Program in USAthe use of simple statistic analysis (‘79 Otsu threshold)pixel processing

’80: David Marr: vision and psychology perception, GestaltBorn of pattern recognition communityRobot vision (Haralick Shapiro) Bottom-up approaches for machine vision

’90: Multi-classifiers (OCRs..) model-based visionVision with multiple features (color, shape, texture, motion)Machine learning for parameter settingMultimedia JPEG and MPEG-4

’00: Open source toolsreal video surveillance, biometry.. (MoGs,)local keypoints (SIFT), tracking and detectors

’10: Multi-sensors IoTDeep learning and CNNsVision on big dataEmbedded and cloud

Evoluzione della

Computer Vision

Dove siamo oggi?

A common processing pipeline

Input dataPre-

processing

Featurecomputationand selection

Classificationor

Reasoning

Toy problem

• Obiettivo: dato un veicolo, indicarne la tipologia (suv, monovolume, compatta, ecc…)

• Come procedere?• Input: veicolo

• Pre-processing: isolo dal contesto, tolgo distrattori, ecc…

• Feature: ne misuro le dimensioni, conto il numero di porte, guardo la forma, ecc…

• Classificazione: provo ad ipotizzare soglie o parametri per ogni caratteristica

• Cosa affidare ad un algoritmo di Intelligenza Artificiale?

Learning tradizionale: l’uomo calcola le features, la macchina le interpreta

PEOPLERE-IDENTIFICATION

People re-identification

Roberto Vezzani, Davide Baltieri, and Rita Cucchiara. 2013. People reidentification in

surveillance and forensics: A survey. ACM Comput. Surv. 46, 2, Article 29 (Dec. 2013).

People re-identification

• Data una immagine di una persona, ritrovarla in una galleria di immagini

Unarticulated 3D-model

• We proposed a 3D body model, called “SARC3D” for re-identification tasks

• A set of vertices is regularly sampled from a sarcophagus-like surface.

• Each person is described with a global scale factor and

appearance information (color histogram) for each

vertex

Re-identification

PEOPLETRACKING WITHOVERLAPPED CAMERAS

PEOPLESOCIAL INTERACTION ANALYSIS

• Single target trackingTRANSDUCTIVE LEARNING

• Multi target trackingBIPARTITE GRAPH MATCHING

• Groups detection in crowdsSUPERVISED CLUSTERING

• Leaders identificationLEARNING TO RANK

Groups detection

Our approach• Define socially grounded features• We cast it as a correlation clustering task where affinity measure is learned depending on the scenario• We introduce a loss function specifically designed to obtain plausible social groups and a way to optimize it

Leaders identification in crowds

Deep learning

Artificial neural networks

Artificial neural networks

Input dataPre-

processing

Featurecomputationand selection

Classificationor

ReasoningDeep learning

Quali requisiti per un algoritmo deep?

• Una buona risorsa di calcolo, meglio se ad elevato parallelismo (GPU)

• Un po’ di pazienza in fase di training

• Fantasia, esperienza e pazienza nella definizione della rete

• Tanti, tanti, tanti dati annotati

Evoluzione delle GPU (es. Nvidia)

ALCUNI CASI DI SUCCESSO

Microsoft Kinect – body joints

Bo

dy

join

tsfr

om

sin

gle

RG

B im

age

Driver monitoring

POSEidon

A demo

Original unseen RGBDEPTH RGB from DEPTH

POSEidon learned something more…

Iearnedimages

To have a mental image on what it did’nt seeTo imagine face by depth!

A TRANFER LEARNED EXPERIENCE:

«Depth-to-face»: an impressive side effect

3D Pandora dataset @Imagelab

2D Pandora dataset @Imagelab

Learnt by the Poseydon Net @Imagelab

DR(EYE)VE

VIDEO SALIENCY

Video saliency

Dr(eye)ve Neural Networks learned what the drivers should pay attention on…

AUTONOMOUS GUIDANCE

DISTANCES FROM SINGLE CAMERA

Real-time distance prediction (front camera)with Computer Vision, Geometry and Deep Learning

Real-time distance prediction (rear camera)with Computer Vision, Geometry and Deep Learning

This is what a car sees around (easy situation)

NUOVI SENSORI

PER LA VIDEOSORVEGLIANZA

This work has been funded by Florim Ceramiche SpA, a leading

manufacturer of ceramic tiles in Italy.

The goal was to “transform” a traditional and stylish fashion floor into

an innovative sensing floor, able to meet these requirements:

• low cost: cost of sensing floors comparable to traditional ones

(about $200/square meter);

• high scalability: coverage of wide areas is allowed;

• high reliability and robustness

• temporal and spatial resolutions: one sensor each 10cm;

sampling at 20Hz

• non-invasive and invisible: the sensing layer must be invisible to

users

This is a

sensing floor!

Research prototype

• 6 modules

• Size: 3 x 4 meters• Sensors: 768 (24x32)

MUST - HISTORICAL MUSEUM – LECCE (Italy)

http://www.techrepublic.com/trends for artificial intelligence

1. Deep Learning 2. AI replace workers

3. IoT

4. Breakthroughs in emotional understanding 5. AI in shopping and customer service 6. Ethical questions

7. A problem with (gender) representation

And then?

Pubblicità: Master Mumet

• We are inviting you to collaborate with the Second edition of the international Master on Visual Computing and Multimedia Technologies. 12 students - five months of university training, six months of stage in a company partner of the project

THANKS

Dipartimento Ingegneria «Enzo Ferrari»

Thanks to all the Imagelab

Staff Rita Cucchiara,

Costantino Grana, Roberto Vezzani,

Simone Calderara, Giuseppe Serra,

PhD students and Research Assistants Francesco Paci, Francesco Solera,

Patrizia Varini, Lorenzo Baraldi, Stefano Aletto,Fabrizio Balducci, Guido Borghi, Davide Abati,

Marcella Cornia, Andrea Palazzi, Federico Bolelli, Andrea Corbelli, Fabio Lanzi,

Riccardo Gasparini, Silvia Calio, Paolo SantinelliPast members

Andrea Prati, Massimo Piccardi, Marco Manfredi, Davide Baltieri, Giovanni Gualdi, Rudy Melli,

Daniele Borghesani, and many others