Le nuove frontiere del Multimedia Forensics: contraffazione, localizzazione e visual understanding

Le nuove frontiere del Multimedia

Forensics: contraffazione,

localizzazione e visual

understanding

Prof. Sebastiano BattiatoDipartimento di Matematica e Informatica,

Università di Catania

Image Processing LAB – http://iplab.dmi.unict.it

iCTLab - www.ictlabsrl.it

[email protected]

Scuola Superiore della Magistratura - 2 Febbraio 2017

http://iplab.dmi.unict.it/

http://www.ictlabsrl.it/


Outline• Intro FIVE (CSI effect, etc.)

• Guidelines & Best Practices

• Technical details and related topics (fundamental,

acquisition issues, etc.)

• Some recent Case Studies:

– Enhancement and Dynamics Reconstruction

– Forgery

• Discussions about LR and its «practical» usage

• New trends & Challenges

– Visual GeoLocalization

– Social

– Body Worn Cameras



Forensic science (often shortened to forensics) is the practical

application of science to matters of the law. Use of scientific

methods for gaining probative facts (from physical/analog or

digital evidences )


Multimedia

Forensics

- Source identification

- Integrity verification/tampering detection

Techniques from multimedia forensics merely provide a way to

test for the authenticity and source of digital sensor data. In this

sense is not about analyzing the semantics of digital or

digitized media objects.

“Forensics Image (Video) analysis is

the application of IMAGE SCIENCE

and DOMAIN EXPERTISE to interpret

the content of an image or the image

itself in legal matters” (SWGIT –

www.fbi.gov)


Image/Video Forensics (in practice)

• Enhancement/Restoration

• Interpretation and Content Analysis– Plate Recognition

– Dynamic Reconstruction (car crashes, etc.)

– Antropomethric issues

– …

• Source Identification

• Integrity/Authenticity


Original File: Special Cases

• Recapture: create a fake and then take a

picture with the camera we want to

pretend the picture was taken with

• Staging: the image file is authentic, but

the content has been staged

In these cases an authentic file does not

imply an authentic content.Scuola Superiore della Magistratura - 2 Febbraio 2017


• SWGDE Best Practices for Photographic Comparison for All Disciplines(Version: 1.0 23/06/2016)

• SWGDE Image Processing Guidelines (Version: 1.0 28/02/2016)

• SWGDE Proposed Techniques for Advanced Data Recovery from SecurityDigital Video Recorders (Version: 1.2 23/06/2016)

• SWGDE Training Guidelines for Video Analysis, Image Analysis andPhotography (Version: 1.1 28/02/2016)

• SWGDE Recommendations and Guidelines for using Video SecuritySystems (Version: 1.0 29/05/2015)

https://www.swgde.org/

https://www.swgde.org/documents/Current Documents/SWGDE Best Practices for Photographic Comparison for All Disciplines

https://www.swgde.org/documents/Current Documents/2016-02-08 SWGDE Image Processing Guidelines

https://www.swgde.org/documents/Current Documents/2016-02-08 SWGDE Training Guidelines for Video Analysis, Image Analysis and Photography_v1-1

https://www.swgde.org/documents/Current Documents/SWGDE Training Guidelines for Video Analysis, Image Analysis and Photography

https://www.swgde.org/documents/Current Documents/SWGDE Recommendations and Guidelines for Using Video Security Systems


https://s-five.eu/

The final draft of the FIVE Best Practice Manual is publically

available from December 8, 2015 ("October/DIWG2015

version"): DRAFT_BPM_FIVE_20151009

https://s-five.eu/

https://www.s-five.eu/sftp/BPM/DRAFT_BPM_FIVE_20151208.doc

ISO Guidelines


Fantasy/Fiction


CSI Effect


Esper Blade Runner


Reality


I Need That Plate! No Way...

Fantasy

• We cannot introduce new information

(there is no data..)

• But we can enhance and extract only

something already present



Fattibilità del miglioramento

Esempi:

Da una singolo fotogramma in cui si vede una targa composta da tre pixel bianchi

non sarà mai possibile ottenere nulla.

Per quanto riguarda il miglioramento di

targhe, che è senza dubbio una delle

richieste più comuni, l’esperienza ci

consente di affermare che se la risoluzione

verticale della targa non è almeno 12-15

pixel, non è possibile ottenere alcun

miglioramento significativo.


Fattibilità del miglioramento

Da un video molto buio caratterizzato da un rumore elevato, spesso se si

hanno a disposizione abbastanza fotogrammi è possibile recuperare un

dettaglio.

Se la risoluzione è adeguata e la

compressione non eccessiva, anche con

sfocature molto pesanti è possibile

ottenere un’immagine nitida .

• What is the minimum quality for video? “Minimum quality”

doesn’t exist.

• The success of the enhancement depends on several

factors:

Main goal (video captured with an HD camera but the

license plate we need to extract is too far away)

Technical related details: Resolution of the area of

interest, Level of compression, Presence of blur /

focus, Number of available frames, Noise / brightness

and contrast

• It’s important to understand which defects are

present in order to apply the proper tools.

Understand When It’s Possible To Get

Something


Perspective Correction

Geometrical transformation


Bridge (2016)



(source Interpol)

Image/Video Forensics:

Some details


Recent Case Studies



Bergamo (2013)

Video


Output

Failure Case 1


Failure Case 2


Failure Case 3


Failure Case 4


Anthropometric Issues and not

only (Simmi Rome)

- proprietary format encoding

- Low resolution 702x540 e 768x576

- 5 fps


PTZ and Involved Dynamics of

events


Albino(BG) - 2015

Loris Stival (2015)


Salomon (2016)


Drago (2016)


Investigative scenarios

• Criminal offenses are often captured by video-surveillance

systems

• Need for verification of the identity of the persons involved in

the criminal offenses

• Identification based on facial or other somatic traits


Credits: Dott. Tessitore: Pol. Scient. – ROMA -

One-to-one comparison result

Totally

Compatible

Partially compatible

Not compatibleAt least one facial feature

that doesn’t match

All facial features match.

Moreover we have at least onecharacteristic sign in both photos under

examination



Bayesian framework

𝑃(𝜃𝑝|𝑠)

𝑃(𝜃𝑑|𝑠)=𝑝(𝑠|𝜃𝑝)

𝑝(𝑠|𝜃𝑑)×𝑃(𝜃𝑝)

𝑃(𝜃𝑑)

Likelihood

ratio

PriorsForensic

expert

Judge

Hypothesis Θp: materials come from the same source

Hypothesis Θd: materials come from different sourceScuola Superiore della Magistratura - 2 Febbraio 2017


Bayesian framework

𝑃(𝜃𝑝|𝑠)



𝑃(𝜃𝑑)

Typicality

term

Similarity

term


Hypothesis Θd: materials come from different source

Scuola Superiore della Magistratura - 2 Febbraio 2017 Credits: Dott. Tessitore: Pol. Scient. – ROMA -

Bayesian framework

𝑃(𝜃𝑝|𝑠)



𝑃(𝜃𝑑)

Likelihood

ratio

1

Same

sourceDifferent

sources

−∞ +∞

The evidence is more likely to occur under the

hypothesis of …

Strength of the evidence

with respect

to the competing

hypotheses


Hypothesis Θd: materials come from different sourceScuola Superiore della Magistratura - 2 Febbraio 2017

Likelihood ratio to verbal scaleLikelihood

RatioLLR Verbal equivalent

+5 > 10.000 > 4 Very strong evidence to support

Same-so

urce

hyp

oth

eses

+4 1000 – 10.000 3 – 4 Strong evidence to support

+3 100 - 1000 2 – 3 Moderately strong evidence to support

+2 10 - 100 1 – 2 Moderate evidence to support

+1 2 – 10 0.3 – 1 Limited evidence to support

0 1 0 Inconclusive

-1 0.5 - 0.1-0.3 to -

1Limited evidence to support D

ifferent-So

urce

hyp

oth

eses

-2 0.1 - 0.01 -1 to -2 Moderate evidence to support

-3 0.01 - 0.001 -2 to -3 Moderately strong evidence to support

-4 0.001 - 0.0001 -3 to -4 Strong evidence to support

-5 < 0.0001 < - 4 Very strong evidence to support



Credits: Prof. Didier Meuwly Principal Scientist, NFI


Credits: Prof. Didier Meuwly Principal scientist, NFI

But the overall quality of the

input data is fundamental..


Boston Marathon

“The FBI, reportedly has more than 2,000 agents looking at the publicly

available evidence,”

GTTI - Thematic Meeting on MMSP 2017, Courmayer (AO)

Boston Marathon

With so many cameras, whether personal or

surveillance, in use, crimes like the Boston Marathon

bombing will be documented with terabytes of images

and footage. Analytics tools that help investigators sort

through the events will be increasingly important in

solving those crimes

J. Klontz, A.K. Jain, “A Case Study on Unconstrained Facial Recognition Using the Boston Marathon Bombings Suspects”. Computer 46(11):91-94, 2013.

1a

1b

2a

2b

2c


Visual Comparison (3D model)


Catania (2017)

Two brothers


Torino (2016)

Image Manipulation: Case “Mozzarella Blu”

Scuola Superiore della Magistratura - 2 Febbraio 2017Scuola Superiore della Magistratura - 2 Febbraio 2017

Scuola Superiore della Magistratura - 2 Febbraio 2017Scuola Superiore della Magistratura - 2 Febbraio 2017

Evidence on the web


Le tre alterazioni cromatiche

presentate in Figura sono state

ottenute mediante l’utilizzo del

software ImageJ per il ritaglio

manuale del particolare grazie allo

strumento “Polygonal Selection”, e

facendo uso del software AMPED5

2010 per l’applicazione della

variazione di tinta


Forgery on Biomedical Images

Corriere della Sera – Ottobre 2013


Eagle Grabs Baby?: Video Of

Bird Snatching Child In Montreal

Baffles Viewers (2012)


Eagle and Baby

A blog post on the Montreal animation school Centre NAD website

claims that the video was a hoax created by their students. The post

states:

The “Golden Eagle Snatches Kid” video, uploaded to YouTube on

the evening of December 18, was made by Normand Archambault,

Loïc Mireault and Félix Marquis-Poulin, students at Centre NAD, in

the production simulation workshop class of the Bachelors

degree in 3D Animation and Digital Design.

According to the post, “Both the eagle and the kid were created in 3D

animation and integrated in to the film afterwards.”


WorldPhotoPress 2012: faked?



When is an image fake, and when

is it merely enhanced?


Final Report

• 1. XMP Analysis. The XMP analysis reflects an incomplete understanding of the Photoshop

metadata and also paraphrases the contents in a misleading way. The referenced block of

metadata merely indicates that the file was adjusted in the Adobe Photoshop Camera Raw

module on multiple occasions before it was opened in Photoshop and then saved out as a

JPEG. In fact, this metadata does not track whether multiple files were composited.

• 2. Error Level Analysis. The forensic analysis of the JPEG compression as performed by

error level analysis (ELA) does not provide a quantitative or reliable analysis of photo

manipulation. This analysis frequently mis-identifies authentic photos as altered and fails to

identify altered images, and as such is not a reliable forensic tool.

• 3. Shadow Analysis. The shadow analysis is flawed in its logic and conclusions. It is true

that linear constraints that connect points on an object with their corresponding points on the

shadow should intersect at a single point (assuming the presence of a single light source).

The location of this intersection point, however, cannot be used to reason about the

elevation of the light in the scene. The intersection point is simply the projection of the light

source into the image plane. This projected location can be anywhere in the image

(including below the ground plane) depending on where the photographer is oriented relative

to the sun.“

• Dr. Hany Farid

WPP Report:The integrity of the

Image (Nov. 2014)


Current practices and accepted

standards relating to the

manipulation of still images in

photojournalism and documentary

photography.

Media organizations

prohibit the alteration of

images beyond

traditional darkroom

techniques.

It means – as first – that the

alteration of images – where

alteration means the digital

addition or subtraction of

elements is forbidden

The Case of Steve McCurry: What Is

'Truth' in Photography?


Faida di Locri: un falso alibi

basato su un video



Current Trends (Use cases)

and Challenges

Future of Imaging

Nikon


Current Trends: Point&Shoot

and Share



Social (Multimedia) Forensics

• Image and Video Phylogeny

ReVeal project

Social (Multimedia) Forensics

• Uploading an image on a Social Network

- The process alters images

- Resize

- Rename

- Meta-Data deletion/editing

- Re-Compression

- NEW JPEG file Structure

M. Moltisanti, A. Paratore, S. Battiato, L. Saravo - Image Manipulation on Facebook for Forensics

Evidence – ICIAP 2015, LNCS 2015;

O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of

Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347

Social (Multimedia) Forensics (2)

• Uploading an image on a Social Network- The process alters images

- Each Social Network Service makes differentalterations

Social Network

Fingerprint

on Uploaded

Images

Next Steps


Next Steps (2)


Social/Visual

GeoLocalization


Battiato, S., Farinella, G. M., Milotta, F. L., Ortis, A., Addesso, L., Casella, A., Torrisi, G.

(2016, June). The Social Picture. In Proceedings of the 2016 ACM on International

Conference on Multimedia Retrieval (pp. 397-400). ACM.

The Social Picture

S. Battiato, G.M. Farinella, F.L.M. Milotta, A. Ortis, L. Addesso, A. Casella, V.D’Amico, G. Torrisi, “The Social Picture”, In Proceedings of ACMInternational Conference on Multimedia Retrieval (ICMR) 2016, New York.


The Social Picture

S. Battiato, G. M. Farinella, F. L. M. Milotta, A. Ortis, L. Addesso, A. Casella, V. D'amico, G. Torrisi, The Social Picture, ACM International Conference on Multimedia Retrieval 2016


Body-Worn camera


Corriere.it 29/5/2016

Body Worn Camera


Furnari, Antonino, Giovanni Maria Farinella, and Sebastiano Battiato. "Recognizing Personal Contexts from Egocentric Images"Proceedings of the IEEE International Conference on Computer Vision Workshops. 2015

A. Furnari, G. M. Farinella, S. Battiato, “Segmenting Egocentric Videos to Highlight Personal Locations of Interest”, IEEE InternationalWorkshop on Egocentric (First-Person) Vision – in conjunction with the IEEE Conference on Computer Vision and PatternRecognition, Las Vegas, (2016)

A. Furnari, G. M. Farinella, S. Battiato, "Recognition of Personal Locations from Egocentric Videos" IEEE Transactions on Human-Machine Systems, 2016

First Person Vision

Recognizing Personal Contexts

Datasets are available online: http://iplab.dmi.unict.it/PersonalContexts/



Final Tips

• Ogni singolo caso va attenzionato in

maniera “dedicata”

• E’ frequente anche il caso in cui non si

riesca ad estrarre alcuna info utile

• Evitare approcci approssimativi ;)


Conclusioni

• Le tecniche di Image (video) Forensic costituisconosicuramente un ulteriore strumento di indagine adisposizione degli investigatori per poter estrarre ed inferire,utili informazioni dalle immagini (e dai video) digitali anchenel caso di dispositivi mobili.

• Per essere in grado di recuperare o di inferire delle evidenzedi prova è comunque necessaria una adeguata competenzaspecifica che richiede uno studio sistematico deifondamenti della teoria dell'elaborazione delle immaginie dei video digitali. I software esistenti agevolano il lavorodegli investigatori ma non riescono per forza di cose adautomatizzare in maniera sistematica ed efficiente talioperazioni e richiedono l'ausilio di professionisti esperti.

Prof. Sebastiano Battiato

Dipartimento di Matematica e Informatica

University of Catania, Italy

Image Processing LAB – http://iplab.dmi.unict.it

iCTLab - www.ictlabsrl.it

[email protected]



http://www.ictlabsrl.it/

mailto:[email protected]

Referenze (1)


Forensics Uses of Digital

Imaging – Second Editiondi John C. Russ

CRC Press

ISBN 978-1498733076


Referenze (2)

Elaborazione delle Immagini

Digitali - terza edizionedi Gonzalez, Woods - Ottobre 2008

Pagine:840

Euro 53,00 (on line 45 euro)

ISBN 9788871925066

Capitoli 1-5, 10

http://hpe.pearson.it/gonzalez

http://www.imageprocessingplace.com

http://hpe.pearson.it/gonzalez

http://www.imageprocessingplace.com/


Referenze (3)

Fondamenti di Image

Processingdi S. Battiato, F. Stanco

Ottobre 2006 Pagine: 150 Prezzo: 20 Euro –

EdiArgo ISBN: 88-88659-49-8

Survey

• Matthew C. Stamm, Min Wu and K. J. Ray Liu,

Information Forensics: An Overview of the First

Decade (2013), in: IEEE Access, 1(167-200)

• Alessandro Piva, An Overview on Image Forensics

(2013), in: ISRN Signal Processing, 2013 (Article ID

496701, 22 pages)

- C. Baron - Adobe Photoshop Forensics – Sleuths,

Thruts, and Fauxtography – Thomson Course

Tehcnology - 2009


On line ResourcesTutorial by Prof. Hany Farid - Digital Image Forensics:

lecture notes, exercises, and matlab code for a survey

course in digital image and video

forensics. http://www.cs.dartmouth.edu/farid/downloads/tutor

ials/digitalimageforensics.pdf

Special Issue on Multimedia in Forensics, Security and

Intelligence - IEEE Multimedia Magazine Vol. 19, Issue 1,

pp. 17-19, 2012

SOFTWARE: Amped5, Authenticate, Adroit, Four&Six, Izitru,

Ghiro, …


http://www.cs.dartmouth.edu/farid/downloads/tutorials/digitalimageforensics.pdf

Main Scientific PublicationsM.Moltisanti, A.Paratore, S. Battiato, L. Saravo - Image Manipulation on Facebook for

Forensics Evidence – ICIAP 2015, LNCS 2015;

F. Galvan, G. Puglisi, A. R. Bruna, S. Battiato, First Quantization Matrix Estimation from

Double Compressed JPEG Images, IEEE Transactions on Information Forensics and

Security, 2014

S. Battiato, G. M. Farinella, E. Messina, G. Puglisi - Robust Image Alignment for Image

Authentication and Tampering Detection – IEEE Transactions on Information Forensics

& Security, Vol. 7 – Issue 4, pp. 1105-1117, 2012.

S. Battiato, G. M. Farinella, G. Puglisi, D. Ravì – Aligning Codeboooks for Near

Duplicate Image Detection – Multimedia Tools and Applications - Springer 2013.

S. Battiato, G. Messina - Digital Forgery Estimation into DCT Domain - A Critical Analysis

- In Proceedings of ACM Multimedia 2009 - Workshop Multimedia in Forensics - Bejing

(China), October 2009.

S. Battiato, G.M. Farinella, G.C. Guarnera, T. Meccio, G. Puglisi, D. Ravì, R. Rizzo - Bags

of Phrases with Codebooks Alignment for Near Duplicate Image Detection – In

Proceedings of ACM Multimedia – Workshop Multimedia in Forensics, Security and

Intelligence (MiFor 2010) – Florence (Italy), October 2010;


IISFA memberbook• S. Battiato, G. Messina, R. Rizzo - Image Forensics - Contraffazione Digitale e Identificazione

della Camera di Acquisizione: Status e Prospettive - Chapter in IISFA Memberbook 2009

DIGITAL FORENSICS - Eds. G. Costabile, A. Attanasio - Experta, Italy 2009;

• S. Battiato, G.M. Farinella, G. Messina, G. Puglisi - Digital Video Forensics: Status e Prospettive -

Chapter in IISFA Memberbook 2010 DIGITAL FORENSICS - Eds. G. Costabile, A. Attanasio -

Experta, Italy 2010

• S. Battiato, G.M. Farinella, G. Puglisi - Image/Video Forensics: Casi di Studio - Chapter in IISFA

Memberbook 2011 DIGITAL FORENSICS - Eds. G. Costabile, A. Attanasio - Experta, Italy 2012.

• S. Battiato, M. Moltisanti – Tecniche di Steganografia su Immagini Digitali – Chapter in IISFA

Memberbook 2012 DIGITAL FORENSICS - Eds. G. Costabile, A. Attanasio - Experta, Italy (2013)

• S.Battiato, F. Galvan, M. Jerian, M. Salcuni - Linee Guida per l'autenticazione Forense di

Immagini – Chapter in IISFA Memberbook 2013 DIGITAL FORENSICS - Eds. G. Costabile, A.

Attanasio - Experta, Italy (2013)

• S. Battiato, A. Catania, F. Galvan, M. Jerian, L.P. Fontana – Acquisizione ed Analisi Forense di

Sistemi di Videosorveglianza - Chapter in IISFA Memberbook 2014 DIGITAL FORENSICS - Eds.

G. Costabile, A. Attanasio - Experta, Italy 2015

• S. Battiato, O. Giudice, A.B Paratore - "Social" Image Forensics: Status e Prospettive - Chapter

in IISFA Memberbook 2016 DIGITAL FORENSICS - Eds. G. Costabile, A. Attanasio, M. Ianulardo -

Edizioni In Magazine/Menabò Group, Italy 2016


Sicurezza e Giustizia• S.Battiato, F. Galvan - Introduzione alla Image/Video Forensics - Sicurezza e

Giustizia - Numero I/MMXIII - pp. 42-43 – 2013.

• S.Battiato, F. Galvan - La Validità Probatoria Delle Immagini e dei Video-

Sicurezza e Giustizia - Numero II/MMXIII - pp. 30-31 – 2013

• S.Battiato, F. Galvan - Ricostruzione Di Informazioni 3d A Partire Da Immagini

Bidimensionali - Sicurezza e Giustizia ( n.IV_MMXIII ) – 2014

• S.Battiato, F. Galvan - Verifica dell'Attendibilità di un Alibi Costituito da

Immagini o Video - Sicurezza e Giustizia - Numero II/MMXIV - pp. 47-50 – 2014.

• Rundo, E. Tusa, S. Battiato - Medical Image Enhancement nei Procedimenti

Giudiziari Medico-Legali in ambito Oncologico - Sicurezza e Giustizia - Numero

I/MMXVI - pp. 53-56 – 2016

• S. Battiato, G. Tessitore - La Stima dell'Errore nella Determinazione dell'Altezza

di un Soggetto - Sicurezza e Giustizia - Numero 4/MMXVI - pp. 38-41 - 2016

• See more at: http://www.sicurezzaegiustizia.com/


Conclusions

• Forensics Image and Video Enhancement

is now a consolidated field but new

intriguing challenges emerge every day.

• Among other, current trends include:

– Big Data analysis (e.g. Social Network) by

«deep» paradigm?

– Advanced Video Synopsis (First-person-

Vision)

– Semantic Exploitation of user-generated

content


Further Credits

• Amped (www.amped.it)

• Undergraduate course: Computer

Forensics – University of Catania

(http://www.dmi.unict.it/~battiato/CF.html)


http://www.amped.it/

www.iciap2017.com

AREA: Information Forensics and Security

DEADLINE: 22 March, 2017

Invited Speaker: Prof. Fernando Peréz-Gonzalez


Main Contacts

Further Info

Image Processing Lab

Università di Catania

www.dmi.unict.it/~iplab

Email

[email protected]


Le nuove frontiere del Multimedia Forensics: contraffazione, localizzazione e visual understanding

Law

Transcript of Le nuove frontiere del Multimedia Forensics: contraffazione, localizzazione e visual understanding