Post on 02-May-2015
ISTITUTO DI SCIENZE E TECNOLOGIE DELLA COGNIZIONE
Piero Cosi
SEZIONE DI PADOVA - “FONETICA E DIALETTOLOGIA”Via Martiri della libertà, 2 – 35137 adova (Italy)
e-mail: cosi@pd.istc.cnr.it - www: http://www.pd.istc.cnr.it
Facial AnimationGeometry Manipulation Based on a Pseudo Muscle Model
Lucia an Italian Talking Head
Copyright, 2009 © ISTC-SPFD-CNR
Incontro con Frank H. GuentherDirector of CNS Speech Lab, Boston University, Boston MA
"Neurolinguistica: produzione e percezione del parlato e controllo senso motorio"
Martedì 22 settembre 2009 - ore 10,30Sala delle Statue, Palazzo Rospigliosi
Via XXIV Maggio, 43 - Roma
Copyright, 2009 © ISTC-SPFD-CNR
ISTC Applications
SONICSPHINXCLSU Toolkit
ILT – Italian Literacy Tutor
Outline
Motivation
Who is working on similar … stuff? Movie Companies & Studios Animation Software Companies Research Institutes and Universities
ISTC-SPFD – LUCIA & FESTIVAL Emotions (Audio & Video)
and….what about the future!?
Motivation
human-like conversational agents(Virtual Characters, Computer Characters, Virtual Agents,
Conversational Agents, Interface Agents…)
human-like characters
they interact naturally, using facial gestures, eye contact, speech patterns, posture and gesture, just like we might use them when conversing with other people
one might see them in computer games or in the media
users can interact and cooperate with them to complete a shared task
Soggetti Coinvolti
RICERCA(modelli fisici)
LUCIA
GRETA
BALDI
ecc.
PC, web, telefonia, ...
animazione
Movie Companies & Studios
Games
ecc.
ecc.
ecc.
http://www2.pd.istc.cnr.it/LUCIA
LUCIA a new emotive/expressive Italian talking head
ELITE System
ELITE Video Processor(100Hz)A / D
(16KHz)A/D
Articulatory signals
Acoustic signals
Track Animation example
Facial Animation
FacialModelling / Animation
Geometrymanipulations
Parametrization
Pseudomusclemodel
Wrinklegeneration
Free formdeformation
Splinemodel
Pure vector based model
Facial Animation & TTS
Input Text
FESTIVALItalian TTS
segmental/lexical/semanticinformation
AVengine(TXT2Animation)
FAPFAP
WAVWAV LUCIA
http://www2.pd.istc.cnr.it/FESTIVAL
http://www2.pd.istc.cnr.it/LUCIA
emotions and expressionsuniversal expressions (fear)
Audiovisual Emotive/Expressive Speech
LuciaLucia
neutral joysadness
disgustfear
Emotion Player
Copyright, 2007 © ISTC-SPFD-CNR
“ILT” – Italian Literacy Tutor
“Interactive Books
Video
Video
(7yr
s)(7
yrs)
(13y
rs)
(13y
rs)
WELCOMEto
INTERSPEECH 2011
Firenze27-31 August 2011
MANY THANKS!
and