ImageLab : Attività di ricerca e sviluppo in video-sorveglianza

Università degli Studi di Modena e Reggio EmiliaDipartimento di Ingegneria dell’Informazione

ImageLab:ImageLab:Attività di ricerca e sviluppo in Attività di ricerca e sviluppo in video-sorveglianzavideo-sorveglianza

Prof. Ing. Rita CucchiaraProf. Ing. Rita Cucchiara

ImageLab ImageLab

Dipartimento di Ingegneria dell’InformazioneDipartimento di Ingegneria dell’Informazione

Università di Modena e Reggio Emilia, ItalyUniversità di Modena e Reggio Emilia, Italy

Rita Cucchiara-Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia ImageLAB-DII Università di Modena e Reggio Emilia

ImagelabImagelab

Imagelab Laboratory of Image Analysis for Computer Vision and Multimedia

Dipartimento di Ingegneria dell’InformazioneUniversità di Modena e Reggio Emilia, Italy

http://imagelab.ing.unimo.it

StaffStaff

Prof. Rita Cucchiara, PhD (Faculty staff)

Ing.Andrea Prati, PhD (Faculty staff)Ing. Costantino Grana,PhD (Faculty staff)

Ing. Roberto Vezzani,PhD (post-Doc )Ing. Simone Calderara (PhD student)Ing. Giovanni Gualdi (Ph-d Student)Ing. Daniele Borghesani (fellow)Dr. Amit Tilvankar (visiting researcher)

Ing, Rudi Melli, Vision-EIng. Giuliano Pistoni, Vision-EIng. Emanuele Perini Vision-E

Prof. Massimo Piccardi (University of Technology, Sidney)

10 Un-graduated students…

Research Activities at ImageLabResearch Activities at ImageLab


Computer Visionfor robotic automation

Transcoding for Multimedia

Image analysisfor Medical Imaging

Video analysisfor telemetry

People and vehicle surveillance

Video analysisfor indoor/outdoorsurveillance

Multimedia: video annotation


CV for robotic automationCV for robotic automationProjects in collaboration with companies with local and regional Projects in collaboration with companies with local and regional

support: some examplessupport: some examplesVision-Based Navigation for AGV Vehicle (2005-2006)Vision-Based Navigation for AGV Vehicle (2005-2006)

Research activity in vision-based localization, navigation and Obstacle detection Research activity in vision-based localization, navigation and Obstacle detection for Automated Guided Vehicle.for Automated Guided Vehicle.

System for automatic creation of ceramic mosaics (2005-2006)System for automatic creation of ceramic mosaics (2005-2006) Research activity in the development of a complete robotic system for the Research activity in the development of a complete robotic system for the

automatic creation of ceramic mosaics.automatic creation of ceramic mosaics.

Detecting borders in labels for textile implants (2006-2007)Detecting borders in labels for textile implants (2006-2007) Research activity in the development of a complete robotic system for the Research activity in the development of a complete robotic system for the

automatic creation of ceramic mosaics.automatic creation of ceramic mosaics.

Analysis of car vibrations (2006-2007)Analysis of car vibrations (2006-2007) analysis of head vibration in Ferrari carsanalysis of head vibration in Ferrari cars



Medical Imaging Medical Imaging

Medical Imaging:Medical Imaging:

Dental tomography (1999-2002)Dental tomography (1999-2002)

Echocardiography ( Burhnam Institute, CA, USA) (2002-2004)Echocardiography ( Burhnam Institute, CA, USA) (2002-2004)

Dermatological image analysis Dermatological image analysis (1999-2007)(1999-2007) SegmentationSegmentation Feature analysisFeature analysis Mosaicing Mosaicing RetrievalRetrieval



Multimedia Multimedia

Cache performance analysis (2000-2004)Cache performance analysis (2000-2004) MIUR ITALIAN Project (2002-2004)MIUR ITALIAN Project (2002-2004)

Video Transcoding (2002-2008)Video Transcoding (2002-2008)

Video Annotation (2004-2008)Video Annotation (2004-2008) EU VI FP DELOS Project (2004-2008)EU VI FP DELOS Project (2004-2008) EU VII FP VIDI-VIDEO Project (2007-2009)EU VII FP VIDI-VIDEO Project (2007-2009)



Video surveillance- old projectsVideo surveillance- old projects

Experiments since 1996 ( University of Ferrara, University of Experiments since 1996 ( University of Ferrara, University of Bololgna)Bololgna)

Projects with Bologna Municipality (1997-1999)Projects with Bologna Municipality (1997-1999)

Sakbot project (University of Modena) (1998-2001)Sakbot project (University of Modena) (1998-2001)

““Domotica per Disabili” Project indoor surveillance (2001-2003)Domotica per Disabili” Project indoor surveillance (2001-2003)

Sakbot for Traficon (2003-2006)Sakbot for Traficon (2003-2006)


Video-surveillance now (1/2)Video-surveillance now (1/2)

LAICA (Laboratorio di Ambient Intelligence per una Città Amica)LAICA (Laboratorio di Ambient Intelligence per una Città Amica) funded by funded by Regione Emilia-Romagna, Italy (2005-2006-?)Regione Emilia-Romagna, Italy (2005-2006-?)

Collab. With WTI Techn., Regulus, Comune Reggio Emilia…Collab. With WTI Techn., Regulus, Comune Reggio Emilia…

Surveillance in public parks with distributed and ptz cameras.Surveillance in public parks with distributed and ptz cameras.

BE-SAFE Bheavior lEarning foir Surveillance application with feature BE-SAFE Bheavior lEarning foir Surveillance application with feature extractionextraction –NATO Science for Peace program (2007-2008)–NATO Science for Peace program (2007-2008)

With Israel (Hebrew Yniv, Magal)With Israel (Hebrew Yniv, Magal)

Automatic real-time detection of infiltrated objects for security of airports Automatic real-time detection of infiltrated objects for security of airports and train stations and train stations (2006-2008) Australian Research Council(2006-2008) Australian Research Council

with University of tecnlogy, Sidneywith University of tecnlogy, Sidney

FREE-SURF: Free Surveillance in a Privacy Respectful way FREE-SURF: Free Surveillance in a Privacy Respectful way

Funded by Italian MUR PRIN Project (2006-2008) Funded by Italian MUR PRIN Project (2006-2008)

With Univ. of firenze, Palermo, IRST TrentoWith Univ. of firenze, Palermo, IRST Trento


Video-surveillance now (2/2)Video-surveillance now (2/2)

Collaboration with Municipal Policy of Modena for forensis analysis Collaboration with Municipal Policy of Modena for forensis analysis (2006-2007)(2006-2007)

Mobile remote SurveillanceMobile remote Surveillance (2006-2007) (2006-2007) funded by ER PRIITT funded by ER PRIITT project project

with Sistemi integrati spawith Sistemi integrati spa

Traffic analysis on highway (2007)Traffic analysis on highway (2007)

with Digitek spawith Digitek spa

VISOR Project in Vidivideo (2007-2009)VISOR Project in Vidivideo (2007-2009)

Smoke detection (WTI 2007)Smoke detection (WTI 2007)



People real time remote surveillancePeople real time remote surveillance

2003 -2004 “2003 -2004 “Domotica per Disabili” FCR Modena Domotica per Disabili” FCR Modena Posture detection for domotic applicationPosture detection for domotic application 2006 PRIITT “2006 PRIITT “Videosorveglianza mobileVideosorveglianza mobile” Sistemi Integrati remote ” Sistemi Integrati remote

surveillance with h264 streamingsurveillance with h264 streaming

Remote SurveillanceRemote Surveillance

Mobile surveillance Mobile surveillance Int. Symp. On Multimedia 2007Int. Symp. On Multimedia 2007



Projects at imagelabProjects at imagelab

2004-2006 2004-2006 vehicle and traffic surveillance - Traficonvehicle and traffic surveillance - Traficon (B) (B)

S.Diego highway


Projects at ImageLabProjects at ImageLab

2005-2006 2005-2006 LAICA ProjectLAICA Project Laboratorio di Ambient Intelligence per una Città Laboratorio di Ambient Intelligence per una Città Amica Piano Telematico ERAmica Piano Telematico ER

Public park in Reggio Emilia, Public park in Reggio Emilia, Modena CampusModena Campus


Projects at ImageLabProjects at ImageLab

(2006-2008) Automatic real-time detection of infiltrated objects for security of Automatic real-time detection of infiltrated objects for security of

airports and train stationsairports and train stations Australian Research Council


Project at imagelabProject at imagelab

NEWNEW 2007-2009 2007-2009 Free SURFFree SURF free surveillance in a privacy respectful way MIUR free surveillance in a privacy respectful way MIUR

PRIN Project (national coordinator)PRIN Project (national coordinator) 2007-2009 2007-2009 BE SAFEBE SAFE Behavioral lEarning in Surveilled Areas with Feature Behavioral lEarning in Surveilled Areas with Feature

Extraction NATO Science for Peace project with IsraelExtraction NATO Science for Peace project with Israel

Multiple-people Multiple-camera surveillanceMultiple-people Multiple-camera surveillance

1) single fixed camera 1) single fixed camera SAKBOT: Moving object segmentation (PAMI 2003)SAKBOT: Moving object segmentation (PAMI 2003) Shadow analysisShadow analysis (PAMI2003) (PAMI2003) Tracking with occlusions (PR-submitted)Tracking with occlusions (PR-submitted) Posture analysisPosture analysis (T SMC 2004)(T SMC 2004) Face detection (PRL 2005)Face detection (PRL 2005)

2) single moving PT camera2) single moving PT camera MosaicingMosaicing Motion clusteringMotion clustering (RTI 2004)(RTI 2004)

3) multiple (overlapped FoVs) cameras3) multiple (overlapped FoVs) cameras HECOL Homography and Epipolar based consistent labeling (PAMI Submitted, HECOL Homography and Epipolar based consistent labeling (PAMI Submitted,

CVIU)CVIU)

4) multiple (not overlapped FoVs) cameras4) multiple (not overlapped FoVs) cameras Color analysis and retrievalColor analysis and retrieval

5) Trajectory analysis5) Trajectory analysis• trajectory clustering for abnormality detectiontrajectory clustering for abnormality detection



Video surveillanceVideo surveillanceSurveillance ofSurveillance of

EnvironmentEnvironment

Traffic Traffic

PeoplePeople

Natural environment (fire detection, fog detection…,)Urban Environment ( crowd monitoring in stations, metro..)Abnormal situations detection

Road traffic control: Statistical control ( vehicles monitoring, classification ) Accident detection ( queue, incidents..)Urban control: people at road crossing, privileged path..Naval, aerial traffic control

Security: people detection in specific areapeople tracking for abnormal behavior detectionpeople interaction analysis

Safety: people behavior monitoring in house and officespeople behavior analysis in public placepeople behavior analysis in car


Human-based video surveillanceHuman-based video surveillance

Most of commercial available video surveillance systems still provide Most of commercial available video surveillance systems still provide acquisition, storage (and access) onlyacquisition, storage (and access) only

modellimodelli

DataStorage

DataCompression

internet/ intranetData Acquistion

DataCompression

UniversalMultimedia Access



System generationsSystem generations 1) analog cameras and CCTV video monitors1) analog cameras and CCTV video monitors



System generationsSystem generations

2) Digital video recorder system

3) Digital PC basedor embedded encodingsystem



System generationsSystem generations

4) LAN cameras

5) LAN cameras with streamingWireless cameras


Human-based video surveillanceHuman-based video surveillance Effort in new hardware solutionsEffort in new hardware solutions

Distributed vs centralizedDistributed vs centralized New camerasNew cameras

• PTZ, infrared, thermal, omnidirectional panoramic camerasPTZ, infrared, thermal, omnidirectional panoramic cameras

StandardStandard vs. vs. stand alonestand alone encoding solutions encoding solutions User friendlyUser friendly (multimedia) human interface (multimedia) human interface Exploiting Exploiting motion detectionmotion detection

Improving human attentionImproving human attention Storage space savingStorage space saving

Data Acquisition

modellimodelli

DBMS

Human/machine based Control center

Intra-inter net

Kn

ow

led

ge

extra

ction


(Automated) (Automated) computer vision-basedcomputer vision-based video surveillancevideo surveillance

Exploiting image processing, pattern recognition and computer vision Exploiting image processing, pattern recognition and computer vision techniques in order totechniques in order to

ExtractExtract moving objects moving objects Detect variationsDetect variations of objects’ motion status of objects’ motion status MeasureMeasure object flow object flow ClassifyClassify people and other objects people and other objects TrackTrack people and objects people and objects Consistent TrackConsistent Track in distributed system in distributed system RecognizeRecognize Bheavior, posture, interaction Bheavior, posture, interaction


Multiple-people Multiple-camera surveillanceMultiple-people Multiple-camera surveillanceat ImageLabat ImageLab

Sakbot

Ad hoc

Hecol

PPC

Projects and prototypes since 1996


Related Imagelab bibliographyRelated Imagelab bibliography

TRAFFIC CONTROL R. Cucchiara, P. Mello, M. Piccardi,"Image Analysis and Rule-Based Reasoning for a Traffic Monitoring,"

IEEE Transactions on Intelligent Transportation Systems, vol. 1, n. 2, pp. 119-130, 2000

SAKBOT R. Cucchiara, C. Grana, M. Piccardi, A. Prati “Detecting Moving Objects, Ghosts and Shadows in Video

Streams“, IEEE Transactions on Pattern Analysis and Machine Intelligence, july 2003

SHADOW DETECTION Prati, I. Mikic, M.M. Trivedi, R. Cucchiara, "Detecting Moving Shadows: Algorithms and Evaluation," IEEE

Transactions on Pattern Analysis and Machine Intelligence, July 2003• POSTURE ANALYSIS

R. Cucchiara, C. Grana, A. Prati, R. Vezzani, "Probabilistic Posture Classification for Human Behaviour Analysis" in IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans , vol. 35, n. 1, pp. 42-54, 2005

• MULTISENSOR R. Cucchiara, A. Prati, R. Vezzani, L. Benini, E. Farella, P. Zappi, "An Integrated Multi-Modal Sensor Network

for Video Surveillance" Journal of Ubiquitous Computing and Intelligence (JUCI), 2006

DISTRIBUTED SURVEILLANCE S.Calderara, A.Prati, R. Cucchiara, HECOL Homography and epipolar based consistent labeling for outdoor

park surveillance in press on Computer vision and Image Understanding 2007

FACE DETECTION R. Cucchiara, A. Prati, R. Vezzani, "A System for Automatic Face Obscuration for Privacy Purposes" in

Pattern Recognition Letters, vol. 27, n. 15, pp. 1809-1815, 2006


Ghostshadow

ghost MVO

MVO shadow

Moving object segmentation:Moving object segmentation: SAKBOTSAKBOT

1) Statistical and knowledge based background modeling 2) color based background suppression with shadow detection 3) Classification and MVOs segmentation

TrackingObjectsHistory

t

Scene

BackgroundSuppression

BackgroundUpdate

Segmentation

It

MVOMVO shadow

DBt

B t

Object validation

Shadow detection

GhostGhost shadow


SAKBOT RESULTSSAKBOT RESULTS Experiments against PETS2001, PETS2006 VSSN dataset First classified in

background suppression competition ACM Multimedia VSSN06 S.Barbara USA Future implementation in Intel OpenCV Used

in indoor and outdoor applications (people detection) In traffico control In sea surveillance


Multiple people Tracking with occlusions: Multiple people Tracking with occlusions: Ad HocAd Hoc

Ad hoc : Appearance-based, with occlusion classification trackingAd hoc : Appearance-based, with occlusion classification tracking Probabilistic model of trackingProbabilistic model of tracking Appearance basedAppearance based (probabilistic at pixel level) (probabilistic at pixel level) DiscriminativeDiscriminative: detect the best position and appearance at each frame: detect the best position and appearance at each frame Handling occlusionHandling occlusion

Occlusion classificationOcclusion classification Keep object appearance also during occlusionKeep object appearance also during occlusion


ExampleExample

Example of tracking without occlusion updateExample of tracking without occlusion update


ExampleExample


HECOLHECOL: distributed surveillance: distributed surveillance

HECOL Homography and epipoles HECOL Homography and epipoles based consistent labeling, in a based consistent labeling, in a probabilistic framework..probabilistic framework..

1)1) Compute Compute overlapped camera FoVoverlapped camera FoV2)2) Construct the Construct the Camera Transition GraphCamera Transition Graph3)3) Sakbot + Ad Hoc tracking for each Sakbot + Ad Hoc tracking for each

cameracamera4)4) when one or more new objects are in when one or more new objects are in

the overlapped areathe overlapped area1)1) Prune the search space with CTGPrune the search space with CTG2)2) Construct the hypotheses spaceConstruct the hypotheses space3)3) Construct the Construct the forward and forward and

backward probabilitybackward probability to link to other to link to other single or group of people single or group of people

4)4) Solve with a double MAP the Solve with a double MAP the consistency consistency

building Main entrance

benches

park

road

building sec. entrance

c1c

1

c2


Homography and epipolar linesHomography and epipolar lines


A priori and conditional probabilityA priori and conditional probability

Matching With occlusionWith multiple people

Rita Cucchiara-Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio ImageLAB-DII Università di Modena e Reggio EmiliaEmilia


New and Future worksNew and Future works

Tracking people with PT camerasTracking people with PT cameras

Bheavior analyisBheavior analyis

Face tracking for identificationFace tracking for identification

Surveillance video annotation (VIDI Video Project)Surveillance video annotation (VIDI Video Project)

Smoke detectionSmoke detection



Future eventsFuture events

ICIAPICIAP http://localhost/iciap07/

International Conference onInternational Conference on Pattern recognition and computer visionPattern recognition and computer vision MultimediaMultimedia Human centered applicationHuman centered application Security and surveillaince Security and surveillaince Industrial applicationsIndustrial applications

WorkshopsWorkshops Computational Color imagingComputational Color imaging DELOS Visual and Multimedia digital library DELOS Visual and Multimedia digital library

The Industrial ForumThe Industrial Forum European project forum European project forum

For more information…

http://imagelab.ing.unimo.it

Rita Cucchiara

Tel. 059 2056136

[email protected]

ImageLab : Attività di ricerca e sviluppo in video-sorveglianza

Documents

Transcript of ImageLab : Attività di ricerca e sviluppo in video-sorveglianza