MAGA: a Mobile Archaeological Guide at Agrigento 1 DINFO - Dipartimento di ingegneria Informatica...

MAGA:MAGA:a Mobile a Mobile ArchaeologicalArchaeological

Guide at AgrigentoGuide at Agrigento

1DINFO - Dipartimento di ingegneria InformaticaUniversity of Palermo

Viale delle Scienze - 90128 Palermo - Italy

2ICAR - Istituto di CAlcolo e Reti ad alte prestazioni

branch of Italian National Research CouncilViale delle Scienze - 90128 Palermo - Italy

Agnese Augello1, Antonella Santangelo1, Salvatore Sorce1, Giovanni Pilato2,Antonio Gentile1, Alessandro Genco1 and Salvatore Gaglio1,2

SpeechTek ’07 - New York City - 21 august 2007SpeechTek ’07 - New York City - 21 august 2007

MAGA: A Mobile MAGA: A Mobile Archaeological Archaeological Guide at AgrigentoGuide at Agrigento

SpeechTek ‘07, NYC 21/8/2007SpeechTek ‘07, NYC 21/8/2007

SummarySummary

Aim of the workEnabling TechnologiesMAGA System architectureImplementation detailsExamplesConclusions



Aim of the workAim of the work

To build smart multimodal systems for information retrieval

user-friendly

suitable for different mobile devices

We propose the use of a verbal interface (for interactions) along with RFID (for in-door positioning purposes)

Requirements:

Natural language interface

Availability of the system in environments where mobility of the user is essential (e.g.: the Archaeological Museum of Agrigento)



Enabling technologiesEnabling technologies

Personal Digital Assistants (PDAs)

Multimodal Browsers (Verbal Interaction)

Conversational Agents (Chatbots)

Semantic Networks (Reasoning Capabilities)

Radio Frequency Identification (RFID)



Personal Digital Assistant (PDA)Personal Digital Assistant (PDA)Mobile devices and wireless communication allow people to access information everywhere

Information retrieval on small devices is limited

Boring interaction and large waste of time during information research!!!

I/O interface



Verbal interactionVerbal interaction

Experimental results demonstrate that verbal interaction allows people to reduce the time spent on manual input (Lyons et al, 2005)

Vocal interaction is a more direct approach, but it comes with a set of constraints:

processing powerspeakers adaptationnarrow dictionariesbounding grammars



Conversational AgentsConversational Agents

A simple approach to build a dialogue system

Pattern-matching based rules

Chatbots can be used as interfaces for several applications (e.g.: entertaining, educational, e-learning …)



Semantic networksSemantic networks

Allow to: describe a specific domain

Concepts, properties of concepts, relations between concepts are clearly defined in that domain

represent commonsense knowledge covering aspects of everyday life

make inferencemake inference to generate new knowledge from the existing one

share and reuse common understanding of the structure of information

Es: OpenCyc, OpenMind, WordNet…



RFIDRFID

Passive tags: limited reading range -> limited position estimation error

Useful in environments with spot-areas



MAGAMAGAIntegration of chatbot, commonsense reasoning, speech recognition/synthesis and RFID technologies to build a virtual guide accessible:

in a cultural heritage environment (such as the “Archaelogical Museum of Agrigento”) by means of different mobile devices

by simulation on a multimodal browser



MAGA System ArchitectureMAGA System Architecture



RFID tagRFID tag detection detection

Tag #1

Tag #2

Tag #3

Tag #4

Database



Verbal interaction engineVerbal interaction engine



Verbal Interaction:Verbal Interaction:X+V TechnologyX+V Technology

XHTML+Voice (X+V) as implementation language

Multimodal Tools 4.1.2.2 for WebSphere Studio V5.1.2: IBM Multimodal Toolkit IBM Multimodal Browser, (X+V interpreter)

X+V page



X+V Client-server architectureX+V Client-server architecture



X+V and Grammar CreationX+V and Grammar Creation

PHP for X+V creation

php request (question)ChatBot

XHTML code

ChatBot (question)

question in textual form

VXML code with the replyreplay in textual form

VXML code with grammar

XHTML code PHP for Grammar creation

MXML

Rule Pre

php request (domain)

Rule Key-Word

Rule Post

DomainDatabase

GRXML

info domain

info domain

info domain



Grammar exampleGrammar example

<rule id="main_rule"> <item repeat="0-1"> <ruleref uri="#pre"/> </item> <ruleref uri="#parola_chiave"/> <item repeat="0-1"> <ruleref uri="#post"/> </item></rule>

<rule id=“parola_chiave">

<one-of>

<item> Archeological Museum </item>

<item> Talamone </item>

<item> Acropolis </item>

</one-of>

</rule>

<rule id="pre"><one-of> <item repeat="0-1"> tell me </item> <item repeat="0-1"> i need information </item> <item repeat="0-1"> to when </item></one-of><item repeat="0-1"> about </item><item repeat="0-1"> the </item></rule>

<rule id="post">

<item repeat="0-1"> is dated </item>

<item repeat="0-1"> in Agrigento </item>

</rule>



Chatbot Knowledge BaseChatbot Knowledge Base



Chatbot technologyChatbot technology: ALICE: ALICE

Chatbots systems: dialogue based on natural languageALICE free software technology

ALICE is an artificial intelligence, natural language chat robot

The knowledge base is composed of question-answer modules, called categories and structured with Artificial Intelligence Mark-up Language (AIML)



Each category is a rule for matching an input and converting to an output, and consists of a pattern, which represents the user input, and a template, which represents the chatbot’s answer.

<category> <pattern>WHAT IS A

CIRCLE></pattern> <template>A circle is the set of

points equidistant from a common point called the center</template>

</category>

The chatbots’ knowledge is stored in a tree called GraphMaster

Alice’s brain

Chatbot technologyChatbot technology: ALICE: ALICE



OpenCyc OntologyOpenCyc Ontology

Knowledge base: logic assertions, common sense rules and heuristics for the reasoning about quotidian life objects and events

Inference system: is based on a series of heuristic related to a reasoning techniques about microtheories

CycL: the language in which Cyc (and OpenCyc) has been written, is a formal language from Lisp language



The “AgrigentoMuseumMt” The “AgrigentoMuseumMt” created in OpenCyccreated in OpenCyc



Example: a MicrotheoryExample: a Microtheory



ALICE-KB and CycALICE-KB and CycInference capabilitiesInference capabilities

We have written a java application that bridges the AIML interpreter with the OpenCyc inference engine (based on the CyN project).

The knowledge base of the chatbot is then enriched with new AIML tags that enable the ontology querying, executing and asserting CycL/SubL statements.

This allows (common sense) reasoning capabilities inside Alice chatbot.

A template can be seen as a meta-answer that must be processed to compose the appropriate response.



ALICE-KB and Cyc: Inference ALICE-KB and Cyc: Inference CapabilitiesCapabilities

The created ontology enables the composition of answers that are not present in the traditional AIML knowledge base.

It is possible to:a) extend the number of AIML categories to obtain a

more fluent dialogue;

b) extend the ontology with new concepts, facts and relations of the specific domain to provide the chatbot a smarter behavior.



An example of AIML Category…<pattern>I NEED INFORMATION ABOUT THE * </pattern>

<template><cycsystem>(cyc-query '(#$Comment <cycterm><star/></cycterm></cycsystem>It is located at<cycsystem>(cyc-query '(#$located_in <star/> ?X)) </cycsystem> and has been created by <cycsystem>(cyc-query '(#$createdBy <star/> ?X)) </cycsystem></template>

…and the resulting interaction

User: I need information about the Archeological Museum in Agrigento.Chatbot: It is one of the most modern museums in Sicily and it contains archeological

finds from the provinces of Agrigento and Caltanissetta, extending from prehistoric to Roman times. It is located at Agrigento, and has been created by the architect Franco Minissi.



An example of AIML Categories using the RFID information

<category><pattern> LOCALIZATION DETECTED</pattern><template>You are in <set name=“showcase”><system>java RFID “showcase”</system></set> <srai> INFO SHOWCASE <get name=“showcase”/> </srai></template></category>

<category><pattern>INFO SHOWCASE *</pattern><template>The showcase contains <set name=“artifact”><cycsystem>(cyc-query '(#$Contains

<star/> ?X)) </cycsystem></set>. <srai> INFO ARTIFACT <get name=“artifact”/> </srai></template></category>

<category> <pattern> INFO ARTIFACT </pattern><template> belongs to <cycsystem>(cyc-query '(#$CreatedDuring <star/> ?X)) </cycsystem>, it is attributed to <cycsystem>(cyc-query '(#$CreatedBy <star/>?X)) </cycsystem> and and represents <cycsystem>(cyc-query '(#$Representing <star/>?X)) </cycsystem> </template></category>



A dialogue exampleA dialogue example

User: I need information about the Archeological Museum in Agrigento.

Chatbot: It is one of the most modern museums in Sicily and it contains archeological finds from neighbors of Agrigento and Caltanissetta, ranging from prehistoric to Roman times.

Chatbot: You are in room #3, in front of the showcase #17. Do you want more information?

User: Yes.Chatbot: In this showcase you can see an Attic amphora. It

belongs to Arcaic Greek period, it is attributed to Dikaios and represents Athena.

User: What is in the next room? Chatbot: In the room #4 there are Capitals and gargoyles with

lion's heads from various temples.



DemoDemo

QuickTime™ e undecompressore Microsoft Video 1

sono necessari per visualizzare quest'immagine.



Conclusions and future workConclusions and future work

Chatbot with “reasoning capabilities” and speech recognition technologies have been embedded in a Personal Digital Assistant to assist user in visiting historical sites

Natural interaction overcomes the limitation of traditional pre-recorded audio/visual guides

Future work will regard the enhancement of the system:Enhance the chatbot reasoning capabilities and natural language interaction;Use of different wireless technologies for context-aware service provisionUse of user profiles for ad-hoc service composition



Thank you!Thank you!

Vocal User Interface, System Architecture Design: Antonella Santangelo {[email protected]} Antonio Gentile {[email protected]}

Information retrieval, reasoning: Agnese Augello {[email protected]} Giovanni Pilato {[email protected]} Salvatore Gaglio {[email protected]}

RFID, pervasive access, HCI: Salvatore Sorce {[email protected]} Alessandro Genco {[email protected]}

MAGA: a Mobile Archaeological Guide at Agrigento 1 DINFO - Dipartimento di ingegneria Informatica...

Documents

Transcript of MAGA: a Mobile Archaeological Guide at Agrigento 1 DINFO - Dipartimento di ingegneria Informatica...