Laboratorio di analisi di dati linguistici

Laurea specialistica in Linguistica Teorica e Applicata, Università di Pavia

Andrea Sansòsanso@humnet.unipi.it

A.A. 2005-2006

Corso progredito

10 CFU

Laboratorio di analisi di risorse linguistiche

Parte quartaLessici

Risorse per la linguistica tipologicaStrumenti e tecnologie per la

creazione di risorse linguistiche

Lessici

Una definizione:

“A computational lexicon is a very complex – and expensive – component to be built adequately. It must contain, in an explicit and formalised way, all the information which a native speaker uses in everyday situations, from the simpler orthographic, phonetic, morphologic information, to the more complex syntactic, semantic, pragmatic, logical, ontological, multilingual information. A ‘complete’ lexicon should practically incorporate our ‘knowledge of the world’, and represent it in an explicit and formal way”

N. Calzolari, “Computational lexicons and corpora. Complementary components in human language technology”, in P. van Sterkenburg (ed.), Linguistics Today –

Facing a Greater Challenge, 89-107. Amsterdam-Philadelphia: J. Benjamins, 2004.

Lessici

Lessici corpora

Questa conoscenza del mondo è un oggetto mutevole e continuamente in accrescimento, impossibile da “congelare” in un lessico statico.

The only way of reflecting and capturing all the potentialities of a language relies on trying to extract the linguistic and lexical information not only from ‘experts’, i.e. native speakers or linguists, but from the texts themselves in which the language is actually used, with a continuous process of enrichment. From these considerations the importance of corpora obviously emerges.

N. Calzolari, ibidem

Lessici

Lessici corpora

L C POS tagging / lemmatisation

C L frequencies of different linguistic objects

C L proper nouns / named entity recognition

L C syntactic parsing

C L updating / tuning a lexicon

C L collocational data

Lessici

Lessici corpora

C L semantic clustering and ‘nuances’ of meaning

L C semantic mark-up

C L lexical knowledge acquisition

L C word sense disambiguation

C L validation of lexical models

C L corpus-based computational lexicography

Lessici

Lessici corpora

Esempio: italiano chiedere vs domandare

Dal punto di vista teorico (introspettivo) sono sinonimi; i dizionari cartacei utilizzano la stessa definizione

Ma:

• domandare è utilizzato quasi sempre in senso interrogativo (ask to know); chiedere è utilizzato spesso in senso imperativo (ask to have);

• chiedere è molto più usato di domandare;

FrameNet

FrameNet (FN) is a corpus-based lexicon-building project that documents the links between lexical items and the semantic frame(s) they evoke; it accomplishes this by annotating sets of sentences that exemplify the items being described, and performing various operations on the resulting annotations. The basic units in FN descriptions are the frame and the lexical unit (LU), the latter understood as the pairing of a “word” with just one of its meanings; thus, a word with four meanings is treated as four lexical units. In most cases, for a word to have more than one meaning implies that it belongs to more than one frame.

Charles J. Fillmore, Collin F. Baker, and Hiroaki Sato, “FrameNet as a ‘Net’”, in Proceedings of the 4th International Conference on Language Resources and Evaluation, Lisbon 2004, pp. 1091-1094.

FrameNet

Main components of the FrameNet database

(1) the frame ontology,

(2) the set of annotated sentences, and

(3) the set of lexical entries.

The basis of the ontology is the set of frames, each of which consists of an informal characterization of a situation type (the frame definition), together with a collection of frame elements (FEs). The FEs are the semantic roles of the entities involved in each frame. FE names are used as labels for the words or phrases that are in grammatical construction with the L(exical) U(nit)s that evoke that particular frame. For example, the frame that includes the English verb inform has as its core FEs SPEAKER, ADDRESSEE and MESSAGE.

FrameNet

• The example sentences are selected by FrameNet annotators as representing the typical uses of the LUs belonging to individual frames. Each set of annotations is centered around a particular LU; the sentence’s constituents are labeled (with FE names) according to the ways in which they fill in information about the frame. For example, sentences (1) and (2) have SPEAKER appearing as subject, and ADDRESSEE as object; the MESSAGE FE appears as a that clause in sentence (1), and as an event-naming nominalization introduced by of in sentence (2).

(1) [SPEAKER We] informed [ADDRESSEE the press] [MESSAGE that the prime minister has resigned]

(2) [SPEAKER We] informed [ADDRESSEE the press] [MESSAGE of the prime minister’s resignation]

FrameNet

The lexical entry for each LU is a summary of what has been recorded in its annotations, presented as valence descriptions, showing all the ways in which its frame elements can be realized, such as the alternative syntactic realizations of the MESSAGE just shown for the verb inform. The collection of annotated sentences is made available in the database as evidence for the analysis.

The first and most obvious way in which LUs are related to each other is through membership in the same frame. Thus inform shares a frame with the verbs notify and announce, and also with the nouns notification and announcement, and the verb resign shares frame membership with its nominal partner resignation, and with verbal expressions like abdicate, step down and stand down. But LUs can also be related to each other in other ways, either because their frames are related to other frames, or through semantic properties (called semantic types in the FN database) assigned to LUs individually rather than through their frames.

FrameNet

Semantic types:

The FrameNet database allows the assignment of semantic types to LUs, FEs and frames. The perception verbs hear vs. listen are distinguished as passive versus active perception verbs, and so, respectively, are see vs. look. Hearing and seeing are things that happen to you, listening and looking are things that you do, and this difference is considered important enough to merit entry into separate frames. In the FN database, hear and see and the passive perception uses of other sensory words, such as feel, taste and smell, belong to the Perception experience frame; the verbs look and listen belong to the Perception active frame, along with the corresponding active uses of feel, taste and smell.

FrameNet

Subframes

Subframes are used for representing subevents; frames that represent complex processes have subframes representing their subparts. To take a simple example, the Motion scenario frame has three subframes, Departing, Motion, and Arriving. In this case, the subframes are temporally ordered, but in general, subframes need not be completely ordered with respect to each other. For example, the Commercial transaction frame has two subframes Commerce goods-transfer and Commerce money-transfer, but these are not ordered with respect to each other. In some commercial transactions, you pay in advance, in others, only after receiving the goods or services.

FrameNet in azione…

http://framenet.icsi.berkeley.edu/index.php

Tutta la documentazione si trova in un manuale:

http://framenet.icsi.berkeley.edu/index.php?option=com_wrapper&Itemid=126

FrameNet in altre lingue

Salsa Project – FrameNet in German

http://www.coli.uni-saarland.de/projects/salsa/

Spanish FrameNet

http://gemini.uab.es/SFN/index.html

WordNet

Sistema di riferimento lessicale disponibile online:

http://wordnet.princeton.edu

I significati delle parole sono rappresentati da gruppi di sinonimi (synsets). Sono rappresentate anche relazioni quali meronimia, iperonimia, antonimia, etc.

Bibliografia aggiornata:

http://mira.csci.unt.edu/~wordnet/

Altri lessici multilingui

Mimida:

http://www.gittens.nl/SemanticNetworks.html

Un lessico multilingue basato su WordNet e su vocabolari liberamente disponibili sul web.

MultiWordNet:

http://multiwordnet.itc.it/english/home.php

Un lessico multilingue (italiano, spagnolo, ebraico, rumeno) in cui i synsets sono allineati, laddove possibile, con i synsets del WordNet di Princeton. Sviluppato all’IRST-ITC di Povo (TN).

EuroWordNet

http://www.illc.uva.nl/EuroWordNet

Un progetto analogo per le lingue europee: è possibile scaricarne una demo

I vari WordNets sono collegati ad un Interlingual index che è basato sul Wordnet americano e che permette di passare da una parola in una lingua a una parola analoga in un’altra. Questo index consente anche di accedere a un’ontologia

condivisa di 63 distinzioni semantiche, che fornisce una base semantica comune per le varie lingue

Altre iniziative

Progetto EAGLES (Expert Advisory Group for Language Engineering Standards): http://www.ilc.cnr.it/EAGLES96/home.html

• development of standards in morphosyntax, syntax and semantics

• awareness of the interdependence between lexical specifications and corpus tagsets / syntactic annotations

• gli standard sviluppati sono serviti nella creazione di risorse (sia corpora che lessici) creati all’interno dei progetti europei Parole e Simple

Altre iniziative

Progetto ISLE (International Standards for Language Engineering) – Computational Lexicon Working Group:

http://www.ilc.cnr.it/EAGLES96/isle/ISLE_Home_Page.htm

• una continuazione del progetto EAGLES

• sviluppo di uno schema generale per la codifica dell’informazione lessicale multilingue (MILE; Multilingual ISLE Lexical Entry)

• impegno a raggiungere consenso su standard di fatto attraverso una procedura bottom-up

• impegno a massimizzare l’interazione e le sinergie con chi lavora nell’ambito del semantic web

Altre iniziative

Progetto PAROLE: http://www.ub.es/gilcub/SIMPLE/simple.html

• obiettivo: produrre in Europa un nucleo iniziale di corpora e lessici armonizzati (catalano, danese, olandese, inglese, finlandese, francese, tedesco, greco, italiano, portoghese, spagnolo, svedese)

• Informazione codificata:

• Morfologia: written forms, including stems and variants; morphosyntactic category; inflected forms; morphological features; derivation; abridged forms

Altre iniziative

Progetto PAROLE:

• Informazione codificata:

• Sintassi: subcategorization patterns; grammatical relations of subcategorised complements; control; diathesis and lexical alternations; pronominalization; linear order constraints; constraints on the syntactic context where the lexical entry is inserted; idioms and collocations

Altre iniziative

Progetto SIMPLE: http://www.ub.es/gilcub/SIMPLE/simple.html

• Aggiunta di un livello semantico a PAROLE

• “The first attempt to tackle harmonised encoding of semantic types and semantic (subcategorisation) frames on a large scale, i.e. for so many languages and with wide coverage”

Altre iniziative

Progetto SIMPLE:

• Informazione semantica: semantic type; domain information; lexicographic gloss; argument structure for predicative semantic units; event type, to characterise the aspectual properties of verbal predicates; links of the arguments to the syntactic subcategorization frames; ‘qualia’ structure, represented by a very large and granular set of semantic relations and features; regular polysemous alternations (e.g. container for content); hyponymy, synonymy, etc.

Due tipi di database tipologici

Databases that collect and document primary language datae.g. Agreement database

Autotyp Reflexives and intensifiers databaseStresstyp...

Databases documenting secondary language datae.g. Noun Phrase Universals Database (Edinburgh)

The Universals Archive (Konstanz)Das grammatikalische Raritätenkabinett (Konstanz)

http://ling.uni-konstanz.de/pages/proj/sprachbau.htm

Database tipologici

http://www.lotschool.nl/Research/ltrc/databases/index.htm

contiene un elenco dei database tipologici elaborati all’interno del progetto LTRC (Utrecht)

Particolarmente user-friendly:

Typological Database of Intensifiers and Reflexives (TDIR): http://noam.philologie.fu-berlin.de/~gast/tdir/index.htm

Reduplication database: http://ling.uni-graz.at/redup/

The SMG databases: http://www.smg.surrey.ac.uk/

Database tipologici

World Atlas of Language Structure

The World Atlas of Language Structures consists of 142 maps with accompanying texts on diverse features (such as vowel inventory size, noun-genitive order, passive

constructions, and 'hand'/'arm' polysemy), each of which is the responsibility of a single author (or team of authors). Each maps shows between 120 (35) and 1110

languages, each language being represented by a dot, and different dot colors showing different values of the features. Altogether 2,650 languages are shown on

the maps, and more than 58,000 dots give information on features in particular languages

Tools per la ricerca tipologica:

http://lingweb.eva.mpg.de/fieldtools/tools.htm

Strumenti e tecnologie per la creazione di risorse

Tools specializzatiFieldwork:

Shoebox: http://www.sil.org/computing/shoebox

Fieldworks Data Notebook: http://fieldworks.sil.org (open source)

Speech analysis:

Praat: http://fonsg3.hum.uva.nl/praat (gratuito)

SpeechAnalyzer: http://www.sil.org/computing/speechtools/speechanalyzer.htm

(versione 2.1 non gratuita; versione 1.5 gratuita)

Annotation tools:

CLAN: http://childes.psy.cmu.edu (gratuito)

Altri strumenti si possono trovare sulla pagina del LARL, nei link (categorie: concordancing tools e altre risorse linguistiche)

Strumenti e tecnologie per la creazione di risorse

Tools specializzatiTagger morfologici:

Morph-it – tagger morfologico dell’italiano ; disponibile una demo in rete sul sito:

http://sslmitdev-online.sslmit.unibo.it/linguistics/morph-it.php

POS taggers:

CLAWS:

www.comp.lancs.ac.uk/computing/research/ucrel/claws/trial.html

TREE tagger:

www.ims.uni-stuttgart.de/projekte/corplex/TreeTagger/DecisionTreeTagger.html

Strumenti e tecnologie per la creazione di risorse

Tools specializzati

Codifica di testi

DBT (DataBase Testuale): software di analisi testuale e interrogazione full-text sviluppato da E. Picchi (ILC, CNR, Pisa)

http://www.ilc.cnr.it/pisystem/demo/index.html

Il LARL possiede un corpus di italiano L2 e il corpus del LIP (Lessico di frequenza dell’italiano parlato) interrogabili attraverso il DBT