POLITECNICO DI MILANOCorso di Laurea Magistrale in Ingegneria Informatica

Dipartimento di Elettronica e Informazione

RHYTHM COMPLEXITY ANALYSISFOR ODD TIME SIGNATURES

Supervisor: Prof. Augusto SartiAssistant supervisors: Ing. Bruno Di Giorgi,

Dr. Massimiliano Zanoni

Master graduation thesis by :Jacopo Foglietti, ID 819536

Academic Year 2015-2016

POLITECNICO DI MILANOCorso di Laurea Magistrale in Ingegneria Informatica

Dipartimento di Elettronica e Informazione

ANALISI DELLA COMPLESSITA’RITMICA PER TEMPI DISPARI

Relatore: Prof. Augusto SartiCorrelatori: Ing. Bruno Di Giorgi, Dr. Massimiliano Zanoni

Laurea di Tesi di :Jacopo Foglietti, ID 819536

Anno Accademico 2015-2016

Alla mia famiglia.

Abstract

Nowadays, Music Information Retrieval applications are rapidly increasing,with particular attention to classification of musical content into user-friendlydescriptors, starting from the music signal or a representation of the relatedmusical properties. In this thesis we focus on rhythm, one of the mostimportant aspects of music, and in particular, we aim at the automaticestimation of rhythmic complexity. This problem has been addressed byresearchers for a long time as attempt to better understand the process ofentrainment, which represents the human’s inner instinct of synchronizationto an external perceived rhythm. However, since the complexity dependsalso on factors like listener’s musical experience and education, it is highlysubjective, and the process of automatic measuring is not trivial.

Although the state of the art offers several methods for rhythmic com-plexity analysis, at the best of our knowledge, none of them has a straight-forward application for the evaluation of rhythmic patterns with unconven-tional time signatures, especially if we consider experimental musical genreslike jazz and progressive rock; and non-Western musical cultures.

This thesis proposes a quantitative model able to estimate rhythmic com-plexity related to unconventional time signatures. The model starts from asymbolic representation of the rhythm and combines perceptual and math-ematical features, taking advantage of both approaches.

A perceptual test was designed and conducted in order to validate themodel. For this test we generated a novel dataset that includes only rhythmswith odd time signature. As shown by the gathered results, our model’sestimates of complexity are more accurate than the set of models foundin the state of the art. Although the test focuses only on unconventionalrhythms, we provide empirical evidence that our model can be successfullyapplied also to standard meters.

I

Sommario

Al giorno d’oggi le applicazioni di Music Information Retrieval si diffondonosempre più rapidamente, con particolare attenzione alla classificazione delcontenuto musicale tramite descrittori di facile comprensione per l’utente, apartire dal segnale musicale o dalla rappresentazione della proprietà musicaliassociate al segnale stesso. In questa tesi ci concentriamo sul ritmo, unodegli aspetti più importanti in musica, e in particolar modo, ci proponiamodi stimare in maniera automatica la complessità ritmica. Per molto tempo iricercatori hanno analizzato questo problema, con il tentativo di fornire unacomprensione migliore del processo di entrainment, ovvero il meccanismobiologico attraverso il quale l’uomo sincronizza l’organismo con uno stimoloritmico esterno. Il processo di classificazione automatica rimane comunquemolto complesso, dato che il concetto di complessità è fortemente soggettivoe dipende da fattori che derivano anche dall’educazione e dall’esperienzamusicale dell’ascoltatore.

Nonostante lo stato dell’arte offra numerosi metodi di analisi della com-plessità ritmica, troviamo che nessuno di questi fornisca una chiara appli-cazione della valutazione di strutture ritmiche con tempi non convenzionali,soprattutto se consideriamo generi musicali sperimentali come il jazz o ilprogressive rock; e culture musicali non occidentali.

Questa tesi propone un modello quantitativo in grado di stimare la com-plessità ritmica legata a tempi non convenzionali. Il modello parte da unarappresentazione simbolica del ritmo e combina aspetti percettivi e matem-atici, sfruttando le qualità di entrambi gli approcci.

Infine, è stato progettato un test percettivo al fine di validare il modello.Per questo test abbiamo generato un nuovo dataset che include solo ritmi contempi dispari. Come dimostrano i risultati ottenuti, le stime di complessitàdel nostro modello sono più accurate rispetto ai modelli presenti in letter-atura superando lo stato dell’arte. Sebbene il test si concentri solo su ritminon convenzionali, dimostriamo empiricamente che il nostro modello puòessere applicato con successo anche alla valutazione di metriche standard.

III

Contents

Abstract I

List of Figures VII

List of Tables XIII

1 Introduction 11.1 Context and Problem Statement . . . . . . . . . . . . . . . . 11.2 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 State of Art 72.1 Theoretical Background . . . . . . . . . . . . . . . . . . . . . 7

2.1.1 Rhythm, Meter and Beat Induction . . . . . . . . . . . 72.1.2 Subjective Rhythmization and Metrical Accent . . . . 82.1.3 Syncopation . . . . . . . . . . . . . . . . . . . . . . . . 112.1.4 Odd Time Signature . . . . . . . . . . . . . . . . . . . 122.1.5 Geometric Rhythm Analysis . . . . . . . . . . . . . . . 12

2.1.5.1 Polygon Representation . . . . . . . . . . . . 122.1.5.2 Evenness . . . . . . . . . . . . . . . . . . . . 13

2.2 Rhythmic Complexity . . . . . . . . . . . . . . . . . . . . . . 142.2.1 General Overview . . . . . . . . . . . . . . . . . . . . . 142.2.2 Complexity Measures . . . . . . . . . . . . . . . . . . 15

2.2.2.1 Rhythm Syncopation . . . . . . . . . . . . . 152.2.2.2 Pattern Matching . . . . . . . . . . . . . . . 192.2.2.3 Distance Measures . . . . . . . . . . . . . . . 212.2.2.4 Information Theory . . . . . . . . . . . . . . 222.2.2.5 IOI Histogram . . . . . . . . . . . . . . . . . 222.2.2.6 Mathematical Irregularity . . . . . . . . . . . 24

2.2.3 The Clock model . . . . . . . . . . . . . . . . . . . . . 25

V

3 System overview 313.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.2 Complexity Model Architecture . . . . . . . . . . . . . . . . . 383.3 The Non-Constant Clock Model . . . . . . . . . . . . . . . . . 383.4 Almost Even Hierarchy Evaluation . . . . . . . . . . . . . . . 46

4 Experimental Results 514.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.1.1 Experimental data . . . . . . . . . . . . . . . . . . . . 514.1.2 Test Modeling and Procedure . . . . . . . . . . . . . . 52

4.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564.2.1 Outlier detection . . . . . . . . . . . . . . . . . . . . . 564.2.2 Model Validation . . . . . . . . . . . . . . . . . . . . . 57

5 Conclusion and Future Works 755.1 Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Bibliography 79

List of Figures

2.1 Eight equivalent ways of representing an example rhythm (knownas the clave Son rhythmic pattern). The first four notationsare more common among classical musicians, while the lastfour are mainly used by drummers and non-musicians. Points5,6,7 are pulse-based rhythmic notations, they differ only inthe way pulses and onset are represented. In this work weadopt the binary notation (n.7), where pulses are the 0’s andonsets the 1’s. Notation number 8 is called inter-onset inter-vals (IOIs) list and describes a rhythm in terms of number ofpulses between successive onsets. Image taken from [1]. . . . . 9

2.2 Clock diagram, divided into sixteen intervals of time, and in-ternal polygon representation of the clave Son rhythm (seefigure 2.1 for other representations of the same pattern). Im-age taken from [2]. . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3 Toussaint’s weighted metrical hierarchy for 16-pulses rhythms.Prime factorization of p = 16 gives a list l1 = [2, 2, 2, 2] forwhich just 1 unique combination exists. Metrical weights areassigned dividing recursively the grid by 1, 2, 4, 8 and 16.Syncopation measure is computed by summing the weightscorresponding to onset positions in the rhythm. . . . . . . . . 16

2.4 Toussaint’s weighted metrical hierarchies for 12-pulses rhythms.Prime factorization of p = 12 gives a list l2 = [2, 2, 3] forwhich 3 unique combinations exists: l2.1 = [2, 2, 3], l2.2 =

[2, 3, 2], l2.3 = [3, 2, 2]. Metrical weights are assigned to eachhierarchy dividing the grid in the order given by the elementsin the corresponding list. Syncopation measure is computedby summing the weights corresponding to onset positions inthe rhythm and taking the mean value among the three hier-archical evaluations. . . . . . . . . . . . . . . . . . . . . . . . 17

VII

2.5 LHL’s weighted metrical hierarchy for 16-pulses rhythms. Treegeneration is based on the same principle, from Toussaint, ofprime factorization of the number of pulses p. Perceptualweights are computed as the negative of the slowest metricallevel index a pulse belongs to. Image taken from [13]. . . . . . 18

2.6 Example of LHL syncopation evaluation for an example rhythmrex = [1 0 0 1 0 1 0 0 0 0 0 1 0 0 0 0 ]. A syncopation is foundwhen a rest or tied note is preceded by a sounded note oflower weight. Syncopation of rex is the sum of the individualscores. At the bottom, the calculation of each syncopationscore is shown. Image taken from [25]. . . . . . . . . . . . . . 19

2.7 Pressing reference patterns with the relative complexity weights.Rhythm is divide in sub-rhythms of the same size of referencepatterns and then each sub-rhythm is assigned with the cor-responding cognitive values. Final complexity value is givenby the sum of each cognitive cost. Image taken from [39]. . . 20

2.8 Example of elaborations of a quarter note. Image taken from[9]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.9 (a) shows the polygon representation of the clave Son rhythmr= [1 0 0 1 0 0 1 0 0 0 1 0 1 0 0 0]. The corresponding localIOIs list is [3 3 4 2 4]. Local IOIs histogram is depicted infigure (b), where the frequency of each interval is representedas the height of the corresponding bin. Image taken from [13]. 23

2.10 (a) shows the polygon representation of the clave Son rhythmr= [1 0 0 1 0 0 1 0 0 0 1 0 1 0 0 0]. The corresponding globalIOIs list is [3 3 4 2 4 7 6 7 6 6]. Global IOIs histogram isdepicted in figure (b), where the frequency of the intervals ofall the unique pairs of onsets is represented as the height ofthe corresponding bin. Image taken from [13]. . . . . . . . . . 24

2.11 Representation of a rhythmic pattern and the candidate clockswhich may be associated together with the pattern itself. Eachclock is defined by a time unit and a phase (location). Fromthe list of possible candidates, one or more clock may bechosen as the most representative of the beat structure in-duced by the rhythm. Selection of the best clock(s) is appliedthrough the counter-evidence (C-score) computation. Imagetaken from [42]. . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.12 A flow chart of the clock model together with its output at thevarious stages. The system is initially fed with the IOIs rep-resentation of the input rhythm, which is successively trans-formed into the corresponding accent notation. All clocks oflength l with time unit in the range [2, (p/2) − 1] and phasegoing from 0 to l − 1 are then generated. Strength of theinduced clocks is given by the C-score computation equals toC = (W ·k-) + (1 ·k0), where +ev = k+= number of accentedelements coinciding with clock, 0ev = k0= number of non ac-cented elements coinciding with clock, −ev = k- = number ofsilences coinciding with clock. Divisor div is a binary valueindicating whether the time unit is a divisor of p or not. Theunit with both div = 1 and minimum C-score is the best clock.Image taken from [42]. . . . . . . . . . . . . . . . . . . . . . . 28

3.1 Weighted metrical hierarchy for 16-pulses rhythms. . . . . . . 323.2 Weighted metrical hierarchies for 14-pulses rhythms. Each

hierarchy is the result of a subdivision of p into equal parts.Each subdivision is applied following the order given by theunique combinations of prime factors diving p. . . . . . . . . . 33

3.3 Weighted metrical hierarchy for 13-pulses rhythms. Since pis prime, prime factorization of p simple results in p itself,as consequence only no subdivision and subdivision by p isallowed, and the hierarchy is almost totally flat. . . . . . . . . 33

3.4 Average chord length for different onsets configurations (imagetaken from [3]) . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.5 Overview of the novel complexity model architecture describedin section 3.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.6 Flow chart of the C-Clock model for rhythm r1 = [1 1 0 0 00 1 1 1 0 0 1 0]. Input rhythm is initially transformed intothe corresponding accents sequence, then C-score evaluation isapplied. We can’t consider the distinction made in the originalmodel between time unit divisors and non divisors of p, sincein this case p is prime and no divisors exists. . . . . . . . . . . 41

3.7 Flow chart of the NC-Clock model for rhythm r1 = [1 1 0 0 0 01 1 1 0 0 1 0]. For this rhythm, there are in total 126 possiblecombinations of periods in range [min_per,max_per] whosesum is equal to p. We skipped some of them, for convenience. 44

3.8 Almost maximally even rhythm positions, for a pattern withn=5 onsets and p=16 pulses (image taken from [2]). . . . . . 46

3.9 16 almost maximally even rhythms with n=5 onsets amongp=16 pulses (image taken from [2]). . . . . . . . . . . . . . . . 47

3.10 Almost even hierarchical decomposition for p = 13 pulses.Resulting AEH tree is symmetric, in this case we obtain 12leaf in total, where each leaf provides a list of elements givingthe order to recursively subdivide the grid of pulses. . . . . . 48

3.11 One of the 12 AEH hierarchies resulting from almost maxi-mally even decomposition method applied to p=13. . . . . . . 49

4.1 Landing page of the complexity test. . . . . . . . . . . . . . . 524.2 UI of a test case shown during the listening/tapping phase.

No visual response of the rhythm is given. . . . . . . . . . . . 544.3 UI shown after the tapping of a rhythm. The user can listen

back to the rhythm and checking the corresponding box no-tation. Before proceeding to the next test case, he/she mustprovide perceptual and performance complexity judgments. . 55

4.4 Mean value of the mean tap correlation vector, drawn in func-tion of the first 5 removed sessions, whose evaluations worsecorrelate with the average ratings. . . . . . . . . . . . . . . . 58

4.5 Mean value of the mean mds correlation vector, drawn in func-tion of the first 5 removed sessions, whose evaluations worsecorrelate with the average ratings. . . . . . . . . . . . . . . . 58

4.6 Boxplot of the tap accuracy ratings computed for the 10 se-lected test sessions and relative to 14-pulses rhythms. . . . . . 59

4.7 Boxplot of the tap accuracy ratings computed for the 10 se-lected test sessions and relative to 13-pulses rhythms. . . . . . 60

4.8 Boxplot of the mds ratings computed for the 10 selected testsessions and relative to 14-pulses rhythms. . . . . . . . . . . . 60

4.9 Boxplot of the mds ratings computed for the 10 selected testsessions and relative to 13-pulses rhythms. . . . . . . . . . . . 61

4.10 Correlation graphs between mean tap and SYNC models. Tapaccuracy CC value = -0.013. . . . . . . . . . . . . . . . . . . . 62

4.11 Correlation graphs between mean mds and SYNC models.Mds CC value = -0.198. . . . . . . . . . . . . . . . . . . . . . 62

4.12 Correlation graphs between mean tap and inv STD models.Tap accuracy CC value = 0.679. . . . . . . . . . . . . . . . . 63

4.13 Correlation graphs between mean mds and inv STD models.Mds CC value = 0.670. . . . . . . . . . . . . . . . . . . . . . . 63

4.14 Correlation graphs between mean tap and E models. Tapaccuracy CC value = 0.513. . . . . . . . . . . . . . . . . . . . 64

4.15 Correlation graphs between mean mds and E models. MdsCC value = 0.543. . . . . . . . . . . . . . . . . . . . . . . . . 64

4.16 Correlation graphs between mean tap and E_C models. Tapaccuracy CC value = 0.076. . . . . . . . . . . . . . . . . . . . 65

4.17 Correlation graphs between mean mds and E_C models. MdsCC value = -0.031. . . . . . . . . . . . . . . . . . . . . . . . . 65

4.18 Correlation graphs between mean tap and E_NCmodels. Tapaccuracy CC value = -0.364. . . . . . . . . . . . . . . . . . . . 66

4.19 Correlation graphs between mean mds and E_NC models.Mds CC value = -0.231. . . . . . . . . . . . . . . . . . . . . . 66

4.20 Correlation graphs between mean tap and AEH models. Tapaccuracy CC value = 0.514. . . . . . . . . . . . . . . . . . . . 67

4.21 Correlation graphs between mean mds and AEH models. MdsCC value = 0.497. . . . . . . . . . . . . . . . . . . . . . . . . 67

4.22 Correlation graphs between mean tap and AEH_C models.Tap accuracy CC value = 0.167. . . . . . . . . . . . . . . . . 68

4.23 Correlation graphs between mean mds and AEH_C models.Mds CC value = -0.095. . . . . . . . . . . . . . . . . . . . . . 68

4.24 Correlation graphs between mean tap and AEH_NC models.Tap accuracy CC value = 0.769. . . . . . . . . . . . . . . . . 69

4.25 Correlation graphs between mean mds and AEH_NC models.Mds CC value = 0.682. . . . . . . . . . . . . . . . . . . . . . . 69

4.26 Rhythm perceptual and performance complexity judgmentsthe user is required to answer after the tapping of the rhythm. 72

List of Tables

3.1 Three example rhythms and the corresponding box and IOI’slist notations. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.2 Toussaint’s measure for rhythms in table 4.2. . . . . . . . . . 34

3.3 Std evaluation for rhythms in table 4.2. . . . . . . . . . . . . 35

3.4 Std and toussaint evaluation for 16-pulses rhythms, where cor-responding IOI’s lists have the same elements but in differentpositions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.5 Evenness measure for rhythms in table 4.2 with correspondingmaximally even configurations. . . . . . . . . . . . . . . . . . 37

3.6 Three examples of C-score computation for C-Clock model. . 40

3.7 Best clocks list for rhythm r2 = [1 0 1 0 1 0 1 1 0 0 0 1 0]computed with the C-Clock method. Minimum value of opti-mal C-score is 4, this means that for r2 there is no a constantsampling able to perfectly fitting the accents sequence. A con-sequence, these beat sequences are just an approximation ofthe induced meter. . . . . . . . . . . . . . . . . . . . . . . . . 42

3.8 Best clocks list for rhythm r2 = [1 0 1 0 1 0 1 1 0 0 0 1 0] withNC-Clock model. Minimum value of C-score=0. All of thesenon constant clock fit without approximations the sequenceof accents, and all of them are possible candidates for beinginduced as beat pulses to the listener’s perception. . . . . . . 45

4.1 Rhythms and corresponding IOI’s pattern included in thedataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.2 The 3 example rhythms we described in section 3.1. . . . . . . 70

XIII

4.3 AEH_NC inverse complexity estimation for rhythms in table4.2, compared with the other models evaluation. Models arenotated as in the following: SYNC = Toussaint’s syncopa-tion; E = evenness; inv STD = inverse standard deviation:AEH_NC = AEH evaluation of induced non-constant beatpattern. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

4.5 Pearson coefficients matrix computed for all the describedmodels. We shortened the models notation in the followingway: ta = mean tap, md = mean mds, sy = SYNC, ist =inv STD, e = E, ec = E_C, en = E_NC, a = AEH, ac =AEH_C, an = AEH_NC. Gray highlighted columns refer toground truth vectors we used for models validation. BoldCC’s values are relative to the models that better describethe perceived complexity, that are inv STD and AEH_NC. . 71

4.4 AEH_NC inverse complexity measures for rhythms in table4.2 plus the “for-on-the-floor” rhythm r4. Even though r4 isthe pattern with the smallest number of onsets, it is still esti-mated as the most simple with respect to the other rhythms. 71

4.6 Perceptual correlation coefficients, studied with regard bothto the number of rhythm loop listened before starting tapping,to the evaluations of the AEH_NC model and to mean tapand mds rating models. We shortened the models notationin the following way: ta = mean tap, md = mean mds, pc= perceval, pf = perfeval, ll = listenloop, an = AEH_NC.Gray highlighted columns refer to ground truth mean mod-els. Both perceval and perfeval show high CC’s values withthe mean models. AEH_NC model is a good predictor ofthe perceived performance complexity, while no relevant cor-relation is shown between AEH_NC model and perceptualcomplexity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Chapter 1

Introduction

1.1 Context and Problem Statement

Because of the widespread availability of the Internet and increasing storagecapabilities, information technology has to deal with a very large amount ofmultimedia data and accessibility has become a central issue. MultimediaInformation Retrieval (MMIR) is a modern research discipline which pro-vides a coherent link between low level audio/video information and higherlevel description of the same content. General aim of this type of study isdescribing and summarizing the media content through feature extractiontechniques, filtering these data and classifying them according to high leveldescriptors. This is not a trivial task, since objective information, as au-dio/video signal, has to be converted into a subjective abstract language.For this reason, MMIR is considered an interdisciplinary science, stretchingfrom computer science subfields (pattern analysis, machine learning, sig-nal processing, interaction design) to psychological areas (perception theory,cognitive neuroscience, psychoacoustics, sociology).

Music Information Retrieval (MIR) is a branch of MMIR that is focusedon the development of computational patterns able to retrieve attributesfrom audio signal meaningful to the human listener. This research allowsprograms to process information according to human perception principlesand may lead to a better formal understanding of music. In this thesis we fo-cus on one of the most intuitive aspect of music: rhythm. Rhythm is definedas the cyclic timing of acoustic events with some repetitive structure [4]. Itis the musical feature that triggers our inherent ability of synchronization toperiodic stimuli, known as entrainment process, and it is strictly related toperceived emotions such as excitement. The main issue we address in thiswork is the perceived complexity of rhythm.

The reason why we focus on complexity estimation is mainly given by thearousal system theory from psychologist Berlyne [5], who confirms that com-plexity is a very relevant semantic descriptor of the music content . Arousalis a region in the human brain that involves many different neural systemsand is important for regulating activities as attention, reactivity and excite-ment. Arousal is also involved in the preference process when a person isintroduced to an acoustical stimuli, and the innate capability of a stimu-lus to induce arousal is called arousal potential. There are various variablesaffecting this property and Berlyne identifies complexity, novelty and sur-prise effect of the stimulus as the most significant. Moreover, he shows thatrelationship between complexity and pleasantness normally takes the formof an inverted U-curve, where the peak of the curve represents the optimaldissonance of a musical piece [6]. Therefore, a good listening is mixed with afair amount of difficulty, because on the other hand we would have whethera boring or annoying perception of that. Other interesting studies in scien-tific and medical area are proposed by Vuust [7]. He deeply investigates theeffect of musical groove and polyrhythms [8] on human feeling and the powerof communication these features have in musical language, illustrating howrhythm strongly influences the complexity parameter. In addition to this,using neuroimages, he provides models able to simulate neuronal processesthe brain activates when it is involved in musical experience and even thesemodels display the existence of a maximal appreciation point at the top of aninverted U-shaped curve. As consequence, rhythmic complexity is deeply in-volved in human activities of listening, liking and learning, furthermore it isworth investigating for the development of applications including automaticranking, enhanced querying, rhythm recognition, ethnological analysis. Theproblem is that the optimal dissonance is a totally subjective concept and,for this reason, it’s hard to give a formal definition of rhythmic complexity.

In general, an object’s complexity reflects the amount of informationembedded in it [9], however this can’t be considered an exhaustive explana-tion of the concept, simply because notion of complexity is relative to thecontext we are considering. Streich [10] proposes an interesting categoriza-tion of the meaning of complexity, dividing the definition in two differentperspectives: the formal view and the informal view. The former relies oninformation theory principles and allows the formulation of computationalexpressions, with the advantage of being totally independent from the ob-server. There are different approaches in this direction (Shannon entropy,Kolmogorov complexity, Stochastic complexity), where the key idea is thatcomplexity is proportional to the amount of information carried by the signal,in terms of length of coding and extent of data transmission. On the other

2

hand, informal perspective of complexity refers to empirical experience andimpression we have about something which is complicated, counterintuitive,unpredictable and difficult to understand in its whole. In this view is moreimportant investigating about perception of complexity, rather than finding atheoretical model, also because, especially in the rhythmic patterns domain,considering only objective phenomena would be a strong limitation. In fact,the formal models assume that signal information is generated through cod-ing principles, but perceptual information can’t be processed in the sameway. This fact is stressed by the subjective rhythmization phenomenon, aneffect for which, when listening for example to a monotone metronome se-quence of “tic” sounds, some events are perceived more accented than others.In this work, when considering the informal view, we refer to cognitive com-plexity. According to Pressing [11], “complexity is dealt with humans by thedevelopment of automatic routines, information bases and heuristics (for in-terpretation, decision-making, action etc.) that both circumvent the impactof normal limitations in memory, attention and control”. Hence the cognitivecomplexity reflects the difficulty of recognizing, processing and reproducinga given temporal sequence.

Many other attempts of rhythmic complexity formalization have beenproposed in the literature and several complexity metrics have been described[12, 13]. They differ in the way rhythm is analyzed, but consistently startfrom a symbolic representation of rhythm. This means that the methodsapply directly to the rhythmic score, which is a symbolic encoding describ-ing the set of rhythmic events. In this work we adopt the standard musicnotation known as box notation, a format equivalent to binary sequence rep-resentation.

Given a rhythmic pattern representation, different properties may bestudied in order to explore its relation with the complexity parameter. Afamily of methods are focused on the evaluation of syncopation, a typical fea-ture of jazz and progressive music. It’s a kind of disturbance of the regularrhythmic flow and amplifies the sense of excitement in the listener’s percep-tion, stressing his ability of memorizing and also reproducing that pattern.For this reason, syncopated rhythms are associated to a greater complexityvalue. Although there exist various methods of syncopation measuring, onlyfew of them have been tested against human perception. Another importantvariable for complexity evaluation is the evenness of a rhythm. This prop-erty measures how evenly and regularly onsets are distributed among pulses.Maximally even rhythms, i.e. the ones where onset are spread out as muchas possible, can be generated by the famous Euclidean algorithm for find-ing the greatest common divisor and are very popular in traditional world

3

music [14]. These rhythms give a very natural and comfortable listening tohumans, and so evenness is considered as one of the most significant featureswhich influence human perception of complexity in a rhythmic groove.

The goal of this thesis is considering some of the computational methodsof syncopation and human perceptual complexity present in the literature,extending their validity to unconventional time signatures and generic num-ber of pulses. Time signature is a Western musical notation that specifiesthe number (signature numerator) and the duration (signature denominator)of pulses that are perceived as strong events inside each bar or loop. If weconsider a pulse value of 1/16 and for example a 4/4 time signature, thismeans that events are mentally grouped in 4 groups of duration 4/16 each.4/4 is the most used in all pop, rock and dance musical genres. Unconven-tional time signatures are uneven sequences that can’t be reduced to timesignatures such as 2/4, 3/4 or 4/4, for which the timing of strong perceivedevents may be not constant in duration. At the best of our knowledge, notextensive studies have been presented in the literature about unconventionaltime structures, even tough they are popular in other Western genres (jazz,prog, electronic) and also in non-Western musical cultures (African, Asian,South-American). Most known examples are given by musical pieces like“Take Five” from Dave Brubeck, which is in 5/4, or “Money” from PinkFloyd, which is in 7/8. On the other hand, for what concern world music,in the southern Balkans one finds time signatures such as 7/16 or 11/16, inIndia there exist more than 150 rhythms called “tals” which are played bybasic construction of 2’s and 3’s, while Turkish folk music adopts the “turkisjidioms”, that are arranged using 5/8 time signature patterns. With the termodd time signatures we refer to all that kind of patterns whose evolution intime repeats every uneven number of pulses (for example 13/16 or 14/16).These kind of metric structures are the focus of our research, in particular wewant to investigate the cognitive complexity experienced by humans whenlistening to them. In this regard, we will propose a method inspired by Poveland Essens’s clock model theory and exploit a modified version of the even-ness property, in order to estimate the complexity of best clock classificationwe found as result of the adopted clock model. Finally, we developed a Web-based performance test that allowed us to analyze correlation between beatinduction, evenness, reproduction ability and complexity judgments.

1.2 Thesis Outline

The remainder of this thesis is structured as follows. In Chapter 2 we intro-duce the rhythmic properties relevant for our work and discuss about some

4

studies related to rhythmic complexity analysis, which are representative ofthe state of art. In Chapter 3 we address the problems the existing complex-ity measures have in the evaluation of non conventional rhythmic patterns,then we describe our model with its two main computational stages. InChapter 4, we provide both a detailed description of the Web-based test wedeveloped in order to gather experimental data and a comparison of resultsthrough the validation of a set of considered models. Finally, in Chapter5, we summarize the main contributions of our work and finally suggestsome future developments related to the automatic estimation of rhythmiccomplexity.

5

6

Chapter 2

State of Art

In this chapter we will present some of the main existing approaches aboutrhythmic complexity evaluation. We introduce in 2.1 the musical propertiesand the terminology that we are going to deal with. Afterwards, in 2.2, wewill give an overview of the state of art in rhythmic complexity estimation.

2.1 Theoretical Background

2.1.1 Rhythm, Meter and Beat Induction

Rhythm is a general concept that can be used to describe any periodic tem-poral pattern in the universe. Its definition implies an organization of eventsin time, but this representation is not sufficient to exhaustively explain thenotion of musical rhythm. Parncutt defines musical rhythm as “an acousticsequence evoking a sensation of pulse” [15] and establishes that expectancyand periodic perceptual grouping are the most appropriate basis for distin-guishing musical rhythm from the other rhythm expressions. This definitioninvolves cognitive aspects of music and highlights the existence of an ab-straction level which goes further the acoustic signal and which is inducedto the mind of the listener. This sensation is called pulse sensation andrepresents the human feeling coming from the expectancy of events whenlistening to a rhythm. In general, pulse sensation is used as blanket term fordescribing all the rhythmic levels a single rhythmic sequence can evoke inthe listener perception. This effect justifies why we have to distinguish theidea of sounding rhythm from the one of perceived rhythm, distinction fromwhich meter concept arises.

London provides an extensive study about rhythm perception, underly-ing the difference between rhythm and meter [16]. The first is the annotated

structure of temporal stimulus, while the latter is the way our cognition at-tends to the rhythmic stimulus. Meter is the listener’s entrainment to therhythmic surface, hence the mechanism representing the biological synchro-nization activity to external stimuli coming from the environment. The at-tending process is therefore the foundation for the inner musical participationof humans. The induction property of rhythm leads the listener to adoptingan anticipatory schema as basis of interpretation for the pulse sensation. Fora deeply understanding of this perceptual schema, London suggests focusingon entrainment models, since “entrainment models of temporal attending arevery attractive for musical meter as they provide a ready means of account-ing for subdivision, durational judgments, syncopation, accented rests, andexpectancy violations and other temporal anomalies” [17].

The interaction between rhythm and meter relies on what Honing callmetrical structure [18], the reworking framework used for internalize a rhyth-mic sequence. Perceived rhythmic levels are mentally processed through ahierarchical structure commonly based on three metrical levels [19]: tatum,the lowest level, beat (or tactus), the middle level, measure, the highest level,which is defined by a given number of beats. Meter is thus the result of ahierarchical organization based on the perceived temporal regularities. Dif-ferent levels are rarely heard equivalently in significance [20], Lerdahl andJackendoff state indeed that the listener tends to focus on the intermediatelevel [21], and, as consequence, beat is assigned to be the reference time towhich the other levels are correlated. Similarly, Parncutt identifies the beatas the pulse sensation with the highest perceptual relevance and shows thatits significance depends on both tempo and rhythmic pattern [15]. Beat in-duction is the cognitive process through which a person feels a sensation ofregular pulse when listening to a rhythm [18], and this timing effect can bepractically measured by observing the period at which most humans entraintapping their foot or dancing to that rhythm. In order to mentally establisha basic beat, we must be able to distinguish a sequence of regularly spacedshort-notes events from the sound signal. These rhythmic events are the beatpulses, which are usually slower in time than the rhythm pulses (or simplypulses).

2.1.2 Subjective Rhythmization and Metrical Accent

One of the most used rhythmic notation in music is the one based on pulsesand onsets. A pulse is the atomic time unit of a rhythm (i.e. the tatum),or, in other words, “one of a series of regularly recurring, precisely equivalentstimuli” [22]. It gives the structure for having continuous short-duration

8

Figure 2.1: Eight equivalent ways of representing an example rhythm (known as theclave Son rhythmic pattern). The first four notations are more common among classicalmusicians, while the last four are mainly used by drummers and non-musicians. Points5,6,7 are pulse-based rhythmic notations, they differ only in the way pulses and onsetare represented. In this work we adopt the binary notation (n.7), where pulses are the0’s and onsets the 1’s. Notation number 8 is called inter-onset intervals (IOIs) list anddescribes a rhythm in terms of number of pulses between successive onsets. Imagetaken from [1].

signals perceived as points in time. The minimun duration between twosuccessive events is given by the temporal size we assign to the pulse timeunit. An onset is an acoustic event with instantaneous attack which marksthe beginning of the stimulus. The rhythmic pattern is generated by a groupof onsets that “create a durational patterning, which is measured in terms ofthe pulses contained in each duration” [13]. In figure 2.1 are depicted eightequivalent common ways of representing a rhythm, points 5,6,7 of the figureare pulse-based notations.

Since both pulse sensation and meter are abstract concepts, not neces-sarily present in the auditory signal, but rather inferred by the listener [23],the representations we reported in figure 2.1 miss the perceptual informa-tion. As a matter of fact, it is well known that even the most simple onsetpattern, made of isochronous identical events, is never perceived as constant,because of the subjective rhythmization mechanism induced by the metricalstructure. Moreover, the metrical schema defining the meter is a framework

9

based on a cultural and educational background that is a kind of listeningkey for the understanding of a temporal sequence and that lets us classifydifferently patterns of equivalent sounds. For this reason, we need more toolsto properly describe the musical perception. In this regard, we introduce thenotion of metrical accent, which describes the perceptual grouping of equiv-alent sounds. This type of accent is not a phenomenal or physical accent,which are given for example by loudness or timbre changing, but it resultsfrom the perception of beat in its metrical context [21]. It is like havinga kind of mental timekeeper which lets us perceive some rhythmic eventsmore relevant than others, according to their position in the pattern. Inview of this considerations, we may establish that the beat is the structureof accented events defining the meter.

Metrical structure properties have to be deeply investigated in order toexplore the effects of temporal perception. Internal rhythmization is indeeda subjective process and meter construction may differ between listeners,therefore it is difficult to find an objective method for extracting the beatstructure from sound signal. In addition to this, rhythm can be metricallyambiguous and more than one structure could fit the sequence of sounds [16].A common approach to afford this task is applying the distinction betweenstrong or weak beat. According to Parncutt, meter is a kind of periodicgrouping [15], where the pulse sensation stage is followed by a time-relativesimultaneous perception of different pulses, whose effect results indeed in aregular alternation of strong and weak beat. For example, in a 4/4 timesignature (meter notation where each measure is divided in 4 beats and eachbeat is said to have a 1/4 time interval) the first beat is called the downbeatand it is usually the strongest accent in the sequence, while the third beatis the next strongest. Second and fourth beat are perceived are less relevantand for this reason they are weak. Positions between beat pulses that arisefrom a temporal subdivision of the beat (like 1/8 notes) are even weaker,therefore there is a range of values that can be assigned to metrical accents.

Studying the beat distribution of a rhythmic pattern is the best methodto extract informations about the induced meter. Rhythmic sequences plac-ing physical accents in correspondence of weak beat positions are usuallycalled off-beat sequences. Somehow, these sequences are not in harmonywith the expectation of the listener and as consequence they increase theamount of tension related to the rhythmic perception. For this reason in2.1.3 we explain the notion of syncopation, a property reflecting this idea oftension in a musical piece and strictly related to perceived complexity.

10

2.1.3 Syncopation

In the Harvard Dictionary of Music, Randel defines syncopation as “a mo-mentary contradiction of the prevailing meter or pulse” [24]. The word “mo-mentary” worths particular attention: syncopation exists in the context ofregularity given by the meter, however it does not challenge the prevailingmeter, but plays just a momentary disturbance of the rhythmic regular flow[25]. Syncopation is often associated to the “racing heart” physical reaction,described by feelings of anxiety and tension [26]. It is the musical propertyformalizing the degree of surprise generated by off-beat rhythmic sequences.As seen before, most music is organized around an abstract framework con-sisting of accented and unaccented events, consequently a greater level ofsyncopation is associated to those sequences which accent more weak po-sitions in the metrical structure while leave nearby strong positions emptyor devoid of stress [23, 27]. Syncopation is therefore inversely proportionalto the listener expectation, nevertheless the interplay between expected andsurprising events is a necessary and critical condition to the generation ofmusical expressions [28]. This feature is used in many musical styles as in-dicator of groove and human feeling, especially in electronic musical pieces.Not by chance, syncopation manipulation has been addressed in many ap-plication for augmented and automatic rhythmic performances, to enableartificial performers to “fail” as humans do [29, 30, 31], and also in feature-based analysis for automatic semantic description [32].

Concerning our research, syncopation is a fundamental feature for com-plexity analysis: syncopated sequences heavily stress listener’s ability ofmemorizing and reproducing the pattern [23], therefore they are associatedto a greater complexity value, both for musicians and non-musicians [33].The greater is the congruity between inferred meter and sounding signal,the easier is to understand (perceive, process, encode) and subsequently toreproduce and remember a temporal sequence. Several experiments havebeen conducted in order to investigate how syncopation affects the percep-tion and reproduction of rhythmic pattern [23] and to discover the bestassumptions a computational model should adopt for a formalization of acomplexity measure evaluation [34]. Moreover, being syncopation strictlyrelated to complexity and arousal potential of the stimuli, applications re-lated to machine recognition for musical patterns [35] and to feature-basedanalysis for automatic semantic description of liking judgments [32] con-firmed that syncopated rhythms plays a fundamental role in the perceivedcomplexity of music.

11

2.1.4 Odd Time Signature

A rhythm with an odd time signature is a sequence of notes that repeatsafter an odd number of pulses. The oddity feature implies the generationof complex patterns, because perceptual grouping of events can’t always fitthe sequence of notes into symmetric subdivisions. Even if most of the timebeat pulses are perceived to be isochronous, in this case nonisochronousbeat sequences may be induced and also the reproduction of odd rhythmsbecomes more difficult. London says that “the asymmetrical meters do makeyou work a little harder to make you stay along with them, and that’s partof their appeal, attraction and char” [36]. When faced with odd sequences,listeners have to focus more on temporal attending of rhythmic features, andthen use the resulting metrical informations as a generator for subsequentexpectations, while in a standard entrainment model listeners can start witha partially mapping of rhythmic properties and then build additional lev-els through the already known hierarchical framework [17, 37]. Cognitivecomplexity of odd rhythms is therefore deeper than the even ones, howeverfactors related to cultural and musical experience can strongly influence thefeeling of these rhythms, for this reason many musicians and researchersaddress great interest to the analysis of the odd time signature.

2.1.5 Geometric Rhythm Analysis

Analysis and manipulation of rhythm require different kinds of visual rep-resentation than traditional Western music notation, for this reason othergraphic tools have been proposed in the literature. In 2.1.5.1 we describethe polygon representation, while in 2.1.5.2 we explain the set theory featureknown as evenness and why it is relevant in the musical rhythmic domain.

2.1.5.1 Polygon Representation

In general a rhythm may be thought of a series of consecutive pulses, wheresome of them are onsets and others are silent pulses. In this work, thesmallest music note value considered is the sixteenth note. The Time UnitBox System (box notation) is a binary notation that well fits the idea ofan onset pattern over a pulse structure. In this work, we use the binary listversion of box notation, where the onsets are marked by a 1, while remainingpulses are marked by a 0 (see figure 2.1, point 7).

Throughout this thesis, rhythms are understood in a cyclical manner, soit is like they repeat infinitely, unless otherwise specified. To better inducethis notion of cyclical timeline, we can connect the two end of the box nota-

12

Figure 2.2: Clock diagram, divided into sixteen intervals of time, and internal polygonrepresentation of the clave Son rhythm (see figure 2.1 for other representations of thesame pattern). Image taken from [2].

tion and draw a circle in which pulses span regularly along the circumference.In this view, we have a clock pattern where time flows in clockwise direction,moreover, connecting the rhythm onsets, a convex polygon is depicted insidethe circle. An example of polygon representation is illustrated in figure 2.2.

The cyclic view of a rhythmic polygon explicits the relative lengths ofnote intervals and offers powerful visualization advantages for the extractionof discriminant geometric features, which wouldn’t be so intuitive in theclassic linear representation of the box notation.

2.1.5.2 Evenness

Rhythmic complexity formalization can be studied through mathematicalanalysis and geometric techniques, which are very popular especially amongresearchers involved in ethnological studies [38, 39, 2, 40]. One of the mostimportant features coming from geometric study is the evenness of a rhythm.This property measures how largely onsets are distributed among pulses andits relevance has been investigated in several studies [14]. In particular, Eu-clid introduced for the first time the evenness concept during the developingof an algorithm for the computation of the greatest common divisor of twonumbers, and more recently Toussaint demonstrated how the structure ofthe Euclidean algorithm may be used for efficiently generating a large familyof rhythms very popular in world music [41] and also how evenness propertyis related to a human feeling of liking and attraction [2]. There are several

13

algorithms for describing the problem of maximally even set, which is a setwhere the elements are spread out through intervals as distant as possible,and almost all of them, to properly represent the cyclic nature of rhythms,adopt the graphical clock diagram (figure 2.2). One example is the evennessmeasure proposed by Clough and Douthett based on the sums of the inter-val arc-lengths (geodesics along the circle) between all pairs of onsets [3].In chapter 3 we will give more extensive details on evaluation of evenness,including a novel computation of the evenness property that we apply onodd rhythms, taking inspiration by the concept of “almost maximally evenrhythms” introduced by Toussaint in [2].

2.2 Rhythmic Complexity

2.2.1 General Overview

Complexity is a feature that can be understood in several contexts. It isa property describing a system which stands between a condition of orderand total randomness. Something is considered complex if we both havedifficulty describing and can’t fully appreciate it [10]. We are interested inthe musical sense of complexity, with special focus on temporal patterns.There are two main approaches for addressing rhythmic complexity analysis[10]: the formal method, mainly based on information theory principles andcoding efficiency, and the informal method, which is focused on cognitiveexperience and practical experiments. Formal measures aim to generate anobjective formulation of a complexity theoretical model, but even tough theyare invariant with respect to listeners and external environment, they yieldvery bad results, due to the special nature of temporal patterns. This factis argued in [42], where Povel and Essens state that temporal patterns area special subclass of serial patterns, however, while serial patterns can beperceptually represented thanks to coding model incorporating a finite setof elementary transformations (repetition, transposition, mirroring), in thetemporal domain these operations don’t apply (for example, a specific in-terval repeated later in a sequence is not usually recognized as such). Thecrucial point is that IOIs (figure 2.1, point 8), which annotate a rhythmicsequence as a list of number of pulses between successive onsets, don’t formalphabets and for this reason they can’t be efficiently processed by humanperception. On the other hand, informal methods result in a deeper under-standing of the cognitive complexity, since they make alternative predictionsabout the structure and depth of meter, tough is quite hard defining anobjective computational model from subjective experiences. In conclusion,

14

temporal sequences need a special processing which ideally combines boththeories.

Pressing introduces three possible notions of complexity: hierarchical,dynamic and generative complexity [11]. The first refers to the structure ofa musical composition, where the amount of complexity is proportional tothe levels density and correlation. This approach is coherent with the na-ture of metrical levels, providing multiple references and sequential degreesof complexity to which pay attention. The second reflects the amount oftime changing behaviours given by unpredictable conditions, improvisatoryperformance and/or adaptive conditions environment. Finally, “the gener-ative complexity is the length of the shortest program which can generatethe object in question, when the program is written in a universal program-ming language” [11]. This notion falls within the area of computer scienceknown as information theory, which, providing formal method, is not reallyuseful in the study of perceived complexity. However, Pressing reworks thislast approach and proposes a method for rhythmic complexity evaluationfocused on generation of music data from a human perspective rather thana computer program.

In 2.2.2 we will see the details of the just mentioned method and ofthe family of algorithms which mostly influenced our model of complexityanalysis for unconventional rhythmic patterns.

2.2.2 Complexity Measures

“Measuring the Complexity of Musical Rhythm” from Thul [13] is proba-bly the most updated and exhaustive listing of all the methods aiming toevaluate complexity of temporal sequences. There are six main categoriesin which the analysis of methods is partitioned: rhythm syncopation, pat-tern matching, distance measures, information entropy, interonset intervalhistogram, mathematical irregularity. We will give an overview of the mainmethods for each family.

2.2.2.1 Rhythm Syncopation

Mathematical syncopation measures are very good predictors of the rhythmiccomplexity, specially of the perceived meter complexity, which is obtainedby “measuring how well the human subjects are able to track the underlyingmetric beat of a rhythm” [43]. We consider two kind of syncopation measures:Toussaint’s Metrical Complexity and Longuet-Higgins and Lee Complexity(LHL). Both of them pertain to the metrical hierarchy framework of Lerdahl

15

Pulse Index

Met

rical

Wei

ght

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

5

4

3

2

1

Figure 2.3: Toussaint’s weighted metrical hierarchy for 16-pulses rhythms. Prime fac-torization of p = 16 gives a list l1 = [2, 2, 2, 2] for which just 1 unique combinationexists. Metrical weights are assigned dividing recursively the grid by 1, 2, 4, 8 and16. Syncopation measure is computed by summing the weights corresponding to onsetpositions in the rhythm.

and Jackendoff [21], which is a temporal and psychological structure thatconsider a rhythmic pattern at all its metrical levels.

Toussaint’s complexity is computed in two stages: the first one takes ininput the number of pulses of the rhythm and generate the correspondingmetrical hierarchy, the second one uses metrical weights to calculate thesyncopation level of the rhythm. For a rhythm of p pulses, metrical hierarchyis a vector of length p constructed considering all the possible ways in whicha meter can divide a rhythm in an equal number of pulses. The structureestablishes how beats are distributed over the grid of pulses according tothe current meter, and the metrical weights assigned to each position arecomputed by applying recursive division of p for a set of combinations ofprime divisors of p itself. There is a different hierarchy for every uniquecombination of prime divisors and each hierarchy defines a specific sequenceof perceptual weights, for each of the p pulses. For example, if p = 16 wehave a unique hierarchy (figure 3.1), while if p = 12 we have three hierarchies(figure 2.4). The hierarchy is dependent just from the number of pulses andnot from the number of onsets.

In the second stage, syncopation measure is computed by summing theweights corresponding to onset positions in the rhythm. If more than onehierarchy results from prime factorization, final mean value is considered.Intuitively, the more syncopated positions will have a lower value, thereforeToussaint’s method is an inverse complexity measure, where a smaller scoringmeans a more complex rhythm.

16

Pulse Index

Met

rical

Wei

ght

0 1 2 3 4 5 6 7 8 9 10 11

5

4

3

2

1

(a) prime factors list l2.1 = [2, 2, 3]

Pulse Index

Met

rical

Wei

ght

0 1 2 3 4 5 6 7 8 9 10 11

5

4

3

2

1

(b) prime factors list l2.2 = [2, 3, 2]

Pulse Index

Met

rical

Wei

ght

0 1 2 3 4 5 6 7 8 9 10 11

5

4

3

2

1

(c) prime factors list l2.3 = [3, 2, 2]

Figure 2.4: Toussaint’s weighted metrical hierarchies for 12-pulses rhythms. Primefactorization of p = 12 gives a list l2 = [2, 2, 3] for which 3 unique combinations ex-ists: l2.1 = [2, 2, 3], l2.2 = [2, 3, 2], l2.3 = [3, 2, 2]. Metrical weights are assigned toeach hierarchy dividing the grid in the order given by the elements in the correspond-ing list. Syncopation measure is computed by summing the weights corresponding toonset positions in the rhythm and taking the mean value among the three hierarchicalevaluations.

17

Figure 2.5: LHL’s weighted metrical hierarchy for 16-pulses rhythms. Tree generationis based on the same principle, from Toussaint, of prime factorization of the number ofpulses p. Perceptual weights are computed as the negative of the slowest metrical levelindex a pulse belongs to. Image taken from [13].

LHL model adopts a metrical scheme as well, but in this case it is formu-lated in terms of a tree structure where metrical weights are assigned to eachleave. The root-node of the tree represents the whole rhythmic bar, and itsleaves represent the pulses of the rhythmic pattern. Also in this case metri-cal hierarchy is computed taking in input the number of pulses p, computingits prime factorization, considering each unique combination and generatinga tree (or more trees) over the selected permutation(s). Having a 4/4 timesignature with p = 16, prime factorization list is equal to 2-2-2-2, and soa binary tree with 5 levels is formed (2.5). At any of these metrical levels,tied notes or repeated rests are considered as a single longer note, so thatonly attacks are considered (LHL assume that longer notes are perceivedas beginning on metrically stronger beats). To each terminal node is thenassigned a perceptual weight based on its metrical unit. Weight for eachpulse is computed as the negative of the slowest metrical level index a pulsebelongs to. For example, for a 4/4 meter, the weight of the downbeat wouldbe 0, of the half bar -1, the second and fourth quarter notes -2, the eightnotes -3, and so on.

Finally, a syncopation is found when a rest or tied note is preceded bya sounded note of lower weight. Score of syncopation is equal to the weightdifference between the rest or tied note and the preceding sounding note(figure 2.6). Total syncopation of the rhythmic pattern is given by the sumof the individual scores.

18

Figure 2.6: Example of LHL syncopation evaluation for an example rhythm rex = [10 0 1 0 1 0 0 0 0 0 1 0 0 0 0 ]. A syncopation is found when a rest or tied noteis preceded by a sounded note of lower weight. Syncopation of rex is the sum of theindividual scores. At the bottom, the calculation of each syncopation score is shown.Image taken from [25].

The results of the studies in [43, 23] indicate the psychological relevanceof both syncopation measures, since these are strong predictors of the user’sperformances and are also coherent with the human judgments obtained in[44], however they still miss a complete understanding of perceived effects ofsyncopation [34]. Moreover, in the case of odd time signature and odd num-ber of pulses, weight assignment through metrical hierarchy decompositiongive very poor results, since for example a prime factorization of a primenumber x simply return a list of two factors, 1 and x. This issue will be fullyaddressed in section 3.1.

2.2.2.2 Pattern Matching

Pressing’s model we mentioned in 2.2.1 is both a cognitive complexity mea-sure and a pattern matching method, since it is based on comparison be-tween patterns within a musical rhythm and reference patterns with knownassigned weights. He argues that the attending process in the listening ac-tivity consists in developing routines to overcome the attention limitation.This routines have different difficulty levels and the complexity of a pat-tern is proportional to the difficulty of generating the corresponding routine.Pressing defines 10 types of patterns (figure 2.7), which he weights accordingto how syncopated they are [11], therefore each pattern has a cognitive cost.

19

Figure 2.7: Pressing reference patterns with the relative complexity weights. Rhythm isdivide in sub-rhythms of the same size of reference patterns and then each sub-rhythmis assigned with the corresponding cognitive values. Final complexity value is given bythe sum of each cognitive cost. Image taken from [39].

We point the reader to [13] for a precise description of weights calculation.Due to the structure of reference pattern (two onset/four pulses combi-

nations) we need to divide the rhythm in a hierarchical manner, using theprime factorization of the number of pulses we have seen for the generationof the metrical unit. After that, we select each sub-rhythm and we look forthe corresponding pattern, scoring the sub-rhythm with the correspondingweight. The cognitive complexity value is given by the sum of each cognitivecost.

Another matching measure is the Tanguiane’s complexity [45], whichmeasures the degree of complexity of a hierarchical structure according tothe level of interaction between low-level and high-level patterns. The goal ofthe proposed model is to represent a rhythm with the least amount of overallcomplexity [46]. The idea is that some patterns are the elaboration of moresimple patterns: root patterns are those patterns not being elaboration ofany other figures (figure 2.8).

Thul shows how bitwise AND operation between binary rhythmic se-quence can be used to evaluate the kind of patterns interaction [13]. Todetermine the complexity value we have to apply sequential binary divisionsof the original pattern, obtain a hierarchical representation of all metricallevels, consider all pairs of metrical levels and count the existing number ofroot patterns. The maximum number of root patterns over each metricaldivision level is the Tanguiane’s complexity ratings [46].

Pressing cognitive complexity results are in good agreement with teachingand performing experience [39], Tanguiane’s complexity yields good, even ifnot optimal [9], results as well. So they are efficient evaluations metrics,

20

Figure 2.8: Example of elaborations of a quarter note. Image taken from [9].

however they have been tested just for a restricted dataset of rhythms andoverall are not supposed to work with an odd number of pulse, since binarydecomposition breaks down to an unbalanced metrical structure and patternmatching techniques can’t be properly applied.

2.2.2.3 Distance Measures

Here the notion of distance is used to evaluate rhythmic complexity in termsof distance between a given rhythm and a target simple rhythm given by ameter with even beat distribution. Most known methods in the literatureare the Directed Swap Distance from Toussaint [47, 48] and the WeightedNote-to-Beat Distance (WNBD) from Gómez [49]. The former compute thenumber of swaps (interchange between two adjacent elements) to transforma certain rhythm into the target rhythm. The Swap Distance is computedas the minimum number of swaps to transform the input sequence into theideal one, therefore the more a rhythm is distant from a target rhythm, themore is complex. The latter relies again on syncopation theory, where asyncopation is identified in each onset that does not occur on the beat. TheWNDB score is based on the distance between rhythm onset and the neareststrong positions of a target rhythm in a way such that onsets nearest toa strong beat are perceived as more syncopated (see [49, 43]) for details).Final complexity measure is the results of the sum of all such scores dividedby the number of onsets.

Complexity analyses carried out with distance measures, in particularusing the WNBD distance, show that they form a tight cluster with the

21

metrical measures of Toussaint and LHL [43], thus their perceptual validityis confirmed. On the other hand, if we consider odd time signatures, we arenot able to define a target pattern, because the most simple beat distributionin a odd sequence of pulses is indeed what we aim investigating.

2.2.2.4 Information Theory

Information theory is a research field studying the storage, processing andcommunication of information. A key feature of information theory is en-tropy : supposing of having a source who generates information through se-quential outcomes of a random variable, entropy is the measure of the amountof uncertainty of this process. If we consider a rhythm as a binary sequencegenerated by a random process, entropy may be considered a rhythmic com-plexity measure. We are not going to expensively speaking about this kind ofcomplexity evaluations since several studies prove that pure information the-oretic measure are not able to capture well the human perceptual, cognitiveand performance complexity of rhythms [39, 9]. As a matter of fact, Press-ing acknowledge that these metrics will always rate random sequences as themost complex ones, which is not perceptually true. We refer to [13, 10, 9]for detailed implementations of a few information coding-based complexitymetrics.

2.2.2.5 IOI Histogram

Histogram is a graphical representation of the probability density functionof a variable. To construct it, we have first to divide the range values inequal spaced non-overlapping intervals, called bins, and then assigning aweight (graphically represented by the bin tall) to each bin, counting thenumber of times each observation of the variable falls in a certain interval.The probability of a new element falling into a bin is determined by thenumber of elements in a bin divided by the total number of elements in allbins [50]. In this context, a similar approach can be adopted for evaluatingthe distribution of the frequency of the inter-onset intervals of a rhythm. Weremind that IOIs are an alternative rhythm notation expressing the durationof the onsets as number of pulses between adjacent onsets. Ideally, a greatvariety of IOIs values implies a major level of information to be processedand memorized by the listener, therefore a rhythm associated to a flat anduniform histogram indicates an high level complexity. Thul presents twoways for constructing histogram given a rhythm. The first is the local IOIshistogram (figure 2.10 and measures the frequency each duration between

22

Figure 2.9: (a) shows the polygon representation of the clave Son rhythm r= [1 0 0 10 0 1 0 0 0 1 0 1 0 0 0]. The corresponding local IOIs list is [3 3 4 2 4]. Local IOIshistogram is depicted in figure (b), where the frequency of each interval is representedas the height of the corresponding bin. Image taken from [13].

sequential pairs of onsets occurs. Thus each bin of the histogram is theoccurrence number of the corresponding rhythm IOIs.

The second construction is the global IOIs histogram and consider theduration between all unique pairs of onsets in a rhythm. If we have a rhythmwith p pulses and n onsets, there are

(k2

)unique pairs, therefore each bin of

the histogram is the occurrence number of the corresponding IOI over thecombination of all the possible interval pairs.

There are three measures that can be applied to both histogram rep-resentation for rhythmic complexity evaluation, where all of them aim toevaluate the uniformity of the histograms.

The first is the standard deviation (std), which measures the amount ofvariation of the IOIs set. A low standard deviation means a flat histogram,or in other words a large presence of several onset durations, thus a morecomplex rhythm.

Secondly, we can measure the information entropy of the bins after theyare normalized such that the sum of frequencies is 1. In this way, the nor-malized histogram may be considered a discrete probability distribution. Xis the discrete random variable taking the bin value a number of time equalsto the corresponding normalized frequency. We can use Shannon’s informa-tion entropy formula: H(X) = −

∑x∈X p(x) log2 p(x). In this way we are

measuring the uncertainty of X, which is maximum when the probability

23

Figure 2.10: (a) shows the polygon representation of the clave Son rhythm r= [1 0 01 0 0 1 0 0 0 1 0 1 0 0 0]. The corresponding global IOIs list is [3 3 4 2 4 7 6 7 66]. Global IOIs histogram is depicted in figure (b), where the frequency of the intervalsof all the unique pairs of onsets is represented as the height of the corresponding bin.Image taken from [13].

distribution is uniform, but, as before, this condition is given by a flat his-togram, thus this is just another method for indicating the uniformity ofthe normalized IOIs histogram and associating the corresponding value ofcomplexity.

Finally, the third measure computes the tallest bin, after the same proce-dure of normalization used for the information entropy method. If the tallestbin has small height, histogram has a flat distribution, thus the correspond-ing rhythm is more complex than a rhythm with few IOIs values.

2.2.2.6 Mathematical Irregularity

These kinds of metrics exploit geometrical properties of rhythm to evalu-ate how irregular and asymmetric a rhythm is. There are two measuresboth discussed by Toussaint [47], whose name are off-beatness and rhythmicoddity.

Given a rhythm r with n onset and p pulses, off-beatness measure in-scribes regular polygons with number of vertexes between 1 andm (moreoverthe number has to be divisor of m) inside a circle of fixed radius where pulsesare evenly distributed along the circumference. Each pulse which does notcorrespond with any of the polygons vertexes is marked off-beat, and off-

24

beatness of r is equal to the number of onsets falling in off-beat positions.The oddity property is a feature discovered by Simha Arom [51] and

is shown by those rhythm whose cyclic representation does not have anycouple of onsets which connected together partition the rhythm into twosub-units with equal number of pulses. Toussaint developed an algorithm formeasuring this property, establishing that rhythm with zero or few coupleof onsets able to regularly dividing the rhythmic circle are more complexsince they have an asymmetric and odd structure. However he also statesthat “the rhythmic oddity property is only relevant to rhythms with an evennumber of pulses, since an odd number of pulses implies the property” [49],thus this measure is not addressable for the evaluation of rhythms with oddtime signature. In addition to this, both rhythmic oddity and off-beatnessmeasure are very poorly correlated with perceptual judgments [43].

2.2.3 The Clock model

In [52] Povel investigates the perceived organization and experienced com-plexity of temporal sequences. The resulting hypothesis theorizes that, whenlistening to a rhythmic pattern, listener generates an internal clock made ofisochronous timings that match the beat structure of the sounding sequence.Parncutt points out a little difference between internal clock and pulse sen-sation [15]: both refers to the same concept, but the internal clock alludes toan underlying neurophysiological mechanism, while pulse sensation alludesto the listening and performing experience of the listener. Pulse sensation isan approach closer to the idea of entrainment, because feelings are regardedas byproducts of the interaction between an organism and its environment[53]. However, the clock model, estimating the internal clock induction, isthe most approved theory in literature affording the task of theoreticallyformalize the internal representation of temporal sequences [9, 15, 54].

The clock model theory is described by Povel and Essens in [42]. Clockis a time structure composed by two parameters: the unit (fixed intervalbetween ticks) and the phase (synchronization with respect to the rhythmicsequence). In figure 2.11 is depicted a rhythmic pattern and the series ofpotential clocks with different units and phases which may be associatedtogether with a temporal sequence.

Two factors are critical in the selection of the best clock: the accentedbeat structure of the sequence and the matching quality of the potentialclock with the sequence of accents perceived in the sequence. Povel andOkkerman studied the perception of series of onsets in equitone sequences[55] and established the existence of phenomenal accents, a kind of rhythmic

25

Figure 2.11: Representation of a rhythmic pattern and the candidate clocks which maybe associated together with the pattern itself. Each clock is defined by a time unit and aphase (location). From the list of possible candidates, one or more clock may be chosenas the most representative of the beat structure induced by the rhythm. Selection of thebest clock(s) is applied through the counter-evidence (C-score) computation. Imagetaken from [42].

26

accents evoked by changes in IOIs rather than changes in loudness or pitch.According to this research, an onset is perceptually accented if it is:

1. isolated (no other onsets on the next left and right pulse);

2. the second of a two cluster onsets;

3. the first and the last of a cluster of onsets greater than 3;

An example of bext clock computation is illustrated in figure 2.12In order to evaluate which is the best clock that segments a sequence, the

algorithm first transforms the list of onsets and pulses into a list of accentedtones. In the final list, accented onsets are marked with 2, the not accentedones with 1, and silences with zero. Subsequently, the method considers allthe clock units with length l in the range [2, (p/2)− 1] and phase going from0 to l − 1 that divide the sequence (this is like subsampling the sequencewith both different periods and different starting points) and then for eachclock calculates: k+ = number of accented elements coinciding with clock,k0 = number of non accented elements coinciding with clock, k- = numberof silences coinciding with clock. Finally the counterevidence (referred asC-score) for each clock is computed as C = (W ·k-)+(1 ·k0), with weightingfactor W set to 4. Label div indicates if the clock unit is a divisor of thenumber of pulses of the rhythm.

The period with the lowest C-score and div=1 (meaning that the clockunit is a divisor of p) is the best clock, that is the metrical unit with thehighest probability will be selected for structuring the sequence. The va-lidity of using C-score as a criterion for clock selection has been proved byEssens in [54]: differently from the model proposed by LHL [56], which isa process model operating from left to right through a sequence developinghypothetical clocks in the process, the clock model is a global model, whichexhaustively tests all potential clocks and selects that one most stronglyinduced.

Once the best clock unit has been defined, Essens passes to a second stageand applies a subdivison model to define a complexity factor by evaluatingthe types of subdivision the clock makes applied to the rhythm. This is doneto construct a measure which combines perceptual complexity, expressed asthe strength of the induction of the best clock, with the efficiency of codingof the rhythm. There are three types of subdivisions:

1. subdivision into equal parts

2. unequal subdivision

27

Figure 2.12: A flow chart of the clock model together with its output at the variousstages. The system is initially fed with the IOIs representation of the input rhythm,which is successively transformed into the corresponding accent notation. All clocks oflength l with time unit in the range [2, (p/2)− 1] and phase going from 0 to l − 1 arethen generated. Strength of the induced clocks is given by the C-score computationequals to C = (W · k-) + (1 · k0), where +ev = k+= number of accented elementscoinciding with clock, 0ev = k0= number of non accented elements coinciding withclock, −ev = k- = number of silences coinciding with clock. Divisor div is a binaryvalue indicating whether the time unit is a divisor of p or not. The unit with bothdiv = 1 and minimum C-score is the best clock. Image taken from [42].

28

3. empty (no intervals in that clock unit)

Thus, given a best clock with a fixed sampling period, rhythm segmen-tation is performed to analyze the subdivisions. The different number ofIOIs inside each subdivision give the degree of equality, and a greater com-plexity is assigned to the segmentation with the greater number of unequalsubdivisions, since it requires more load of information to be coded.

Finally both subdivision and clock model must be combined and weightedindependently. The contribution of each factor to the perceived complexityin temporal sequences is assessed in [54, 15, 9] thanks to the support ofperforming experiments and complexity judgments.

29

30

Chapter 3

System overview

In this chapter we will describe our model for complexity evaluation ofrhythms with unconventional time signature. In 3.1 we will explain whythe complexity measures explained in 2.2.2 are not able to effectively quan-tify the effort needed to learn and repeat an odd rhythmic pattern. In section3.2 we will illustrate the pipeline of operations for computing the cognitivecomplexity of a rhythm, while in section 3.3 and 3.4 we will give the detailsof the two main stages of our model.

3.1 Motivation

After the analysis of the state of art and of the theories about music per-ception, we came up to the idea that metrical hierarchies and symmetricproperties of a rhythm are the key features for the estimation of perceivedrhythmic complexity. These are indeed the attributes that better describe atemporal pattern in function of its onsets position and IOIs, therefore theyprovide a valid framework for finding discriminant properties between fam-ilies of rhythms. For this reason, we first tried using three simple rhythmiccomplexity measures based on syncopation, standard deviation and evenness.Let’s consider for example three temporal patterns and their correspondingnotations (table 4.2): the first rhythm has got 16 pulses, the second one 14pulses and the third one 13 pulses.

We start evaluating the Toussaint’s complexity of each of these rhythmswith the syncopation measure. This method initially constructs one or moremetrical hierarchies, in order to establish the strength of beat positions infunction of the number of pulses. We refer to section 2.2.2.1 for the con-struction of metrical hierarchy (figure 3.1) relative to r1, which has got 16pulses.

id rhythm IOI’s list pulsesr1 [1 0 0 1 0 1 0 1 0 0 0 1 0 1 1 0] [3 2 2 4 2 1 2] 16r2 [1 0 0 1 0 1 0 1 0 0 0 1 0 1] [3 2 2 4 2 1] 14r3 [1 0 0 1 0 1 0 1 0 0 0 1 0] [3 2 2 4 2] 13

Table 3.1: Three example rhythms and the corresponding box and IOI’s list notations.

Pulse Index

Met

rical

Wei

ght

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

5

4

3

2

1

Figure 3.1: Weighted metrical hierarchy for 16-pulses rhythms.

Concerning r2, prime factor divisors of p = 14 are 2 and 7 and there aretwo different combinations of the factors list, l2.1 = [2, 7] and l2.2 = [7, 2], asconsequence there are two distinct hierarchies (figure 3.2).

Finally, r3 is a rhythm with a number of pulses both odd and prime.This means that prime factorization of p = 13 yields to a factors list of justone element l3 = [13], corresponding to the metrical hierarchy illustrated infigure 3.3.

Once metrical hierarchies have been computed, one can uses metricalweights to calculate the syncopation level of a rhythm. Syncopation measureis obtained comparing the onset positions in the rhythm with the indexedpositions in the hierarchy and summing the corresponding weights. If thereis more than one hierarchy, mean value is considered, as suggested in [13].Intuitively, the more syncopated positions will have a lower value since theyare not aligned with the strongest beat positions, therefore a smaller valuedescribes a more complex rhythm. The complexity evaluations are shown intable 3.2:

In this case, r3 rhythm is the most syncopated and therefore also themost complex among the example rhythms we have considered. This resultis theoretically obvious not only for the odd number of pulses, but alsobecause of the different number of weights contained in each hierarchies.

32

Pulse Index

Met

rical

Wei

ght

0 1 2 3 4 5 6 7 8 9 10 11 12 13

5

4

3

2

1

(a) prime factors list l2.1 = [2, 7]

Pulse Index

Met

rical

Wei

ght

0 1 2 3 4 5 6 7 8 9 10 11 12 13

5

4

3

2

1

(b) prime factors list l2.2 = [7, 2]

Figure 3.2: Weighted metrical hierarchies for 14-pulses rhythms. Each hierarchy is theresult of a subdivision of p into equal parts. Each subdivision is applied following theorder given by the unique combinations of prime factors diving p.

Pulse Index

Met

rical

Wei

ght

0 1 2 3 4 5 6 7 8 9 10 11 12

5

4

3

2

1

Figure 3.3: Weighted metrical hierarchy for 13-pulses rhythms. Since p is prime, primefactorization of p simple results in p itself, as consequence only no subdivision andsubdivision by p is allowed, and the hierarchy is almost totally flat.

33

id rhythm toussaintr1 [1 0 0 1 0 1 0 1 0 0 0 1 0 1 1 0] 12r2 [1 0 0 1 0 1 0 1 0 0 0 1 0 1] 8r3 [1 0 0 1 0 1 0 1 0 0 0 1 0] 6

Table 3.2: Toussaint’s measure for rhythms in table 4.2.

As a matter of fact, rhythm having a greater number of pulses tend to beevaluated as more simple (actually, some kind of normalization may appliedto compare at the same time rhythms with different number of pulses and/oronsets [13]). In addition to this, there is a more important problem tohighlight: for any other 13-pulses rhythm we wanted to analyze with thesame number (but different position) of onsets x, we would always obtain acomplexity value equal to x + 1 (we are assuming that every rhythm startswith an onset in the first pulse position). In fact, as shown in figure 3.3,the metrical hierarchy we obtain from the factorization of a prime numberis almost flat, giving no chance of discriminating complexity in a fair rangeof values, and no normalization strategy would solve it. For even numberof pulses but with few integer divisors, as for p = 14, Toussaint’t measuremight give flat results as well, since weights distribution over the grid ofpulses is quite uniform. This means that for many time signatures thismeasure is poorly sensitive to onset distribution, while we know from meterperception theory that onsets position is essential in entrainment activitiesand cognitive complexity evaluation. Even tough we would use the LHL’ssyncopation measure (presented in 2.2.2.1), we would have the same problem,since the construction of the hierarchical tree is based on prime factorizationof the number of pulses p as well.

Another complexity measure we may use is the standard deviation (std)of a rhythm, as suggested in 2.2.2.5. Given a set of values, std quantifiesthe amount of dispersion of these values with respect to the average value.A low std means that data points tend to be close to the mean, while anhigh std indicates that data points are spread over a wider range of valuesfar from the mean point. The sample std of a set of values is given by:

σ =√

1N

∑Ni=1(xi − x)2, where xi is the i-th data point and x its mean

value. In the rhythmic domain, we may set xi with the intervals from theIOIs list of a rhythm and compute its std. IOIs list represents indeed thetemporal structure the listener has to code during the listening process, andsince coding aims at efficiency of description, it is a common rule assumingthat “the efficiency of a code is inversely related to the number of symbols

34

needed” [42]. As consequence, a rhythm with high std implies that is madeof a great variety of note intervals, therefore is more complex to memorizeand reproduce. If we consider the example rhythms in 4.2, the correspondingstd values are:

id IOI’s list stdr1 [3 2 2 4 2 1 2] 0.95r2 [3 2 2 4 2 1] 1.03r3 [3 2 2 4 2] 0.89

Table 3.3: Std evaluation for rhythms in table 4.2.

In this case r2 is classified as the most complex rhythm among the others.Std is indeed advantageous for comparing rhythm with different number ofpulses, however it does not distinguish rhythms with same IOIs but differentonset positions. For example, if we look to table 3.4, we have that r1.1 andr1.2 both have the same std value but different Toussain’t complexity (thatis still good measure for evaluating rhythm of 16 pulses). As a matter offact, r1.1 is more syncopated than r1.2, but the std measure is not able tocapture this feature.

id rhythm IOIs list std toussaintr1.1 [1 0 0 1 0 1 0 1 0 0 0 1 0 1 1 0] [3 2 2 4 2 1 2] 0.95 12r1.2 [1 0 0 0 1 0 1 1 0 1 0 0 1 0 1 0] [4 2 1 2 3 2 2] 0.95 17

Table 3.4: Std and toussaint evaluation for 16-pulses rhythms, where correspondingIOI’s lists have the same elements but in different positions.

Finally, we consider the evenness property as inverse measure of complex-ity. Given a set with p pulses and n onsets, the evenness of this set measureshow evenly the onsets are distributed over the pulses. The rhythms of a setwhich is maximally even contain the onsets pushed as far apart as possibleand, in terms of coding efficiency, this condition implies an optimization ofthe information storage, since symbols are distributed in order to reach themost regular possible structure. From a perceptual point of view, the moreIOIs are symmetrically ordered, the lower is the variance associated to IOIslist, therefore given two rhythms with the same number of pulses and onsets,we expect that the one with greater evenness value is the perceptually sim-pler. To quantify the evenness of a rhythm, we use the first method describedby Kanz in [3], which is based on computation of the averaged chord length

35

Figure 3.4: Average chord length for different onsets configurations (image taken from[3])

between pairs of onsets. In a clock diagram representation where p pulsesare equally spaced points around a circumference of a unit circle and onsetsare n selected points of this circular grid, the length of the chord connectingtwo onsets that are d pulses apart is given by: chord = 2 · sin(π·dp ). Usingthis equation, we can compute the chord length for each pair of onsets andthen finding the average chord length value of a given rhythmic pattern assum of the single chord lengths divided the number of pair of onsets. Mini-mum and maximum average chord length of a rhythm having p pulses andn onsets correspond to the minimally and maximally evenness configurationof it, and they may be computed as:

Ave(d, p) =1

|d|·|d|∑k=1

2 · sin(π · dkp

) (3.1a)

Avemin(n, p) =4

n · (n− 1)

n−1∑k=1

(n− k) · sin(π · kp

) (3.1b)

Avemax(n, p) =2

(n− 1)

n−1∑k=1

(2(

1−{p · kn

})−[n|p ·k

])sin(π · bp · k/n)c

p

)(3.1c)

where, d is the vector containing the rhythm IOIs. In 3.1c, floor functionbxc is the greatest integer less than or equal to x, fraction function {x} is thedecimal part of x and the function [x|y] is 1 if x divides y and 0 otherwise.In [3] there is an example where all the configurations (excluding rotationand reflection of a given setting) for p = 8 and n = 3 are listed with thecorresponding averaged chord length, as illustrated in figure 3.4.

Given thus p = 8 and n = 3, Avemin = 0.98 indicates the average chordlength of the minimally even rhythm [1 1 0 0 0 0 0 1] corresponding to theIOI’s list configuration [1 6 1] , while Avemax = 1.70 indicates the averagechord length of the maximally even rhythm [1 0 0 1 0 0 1 0] corresponding to

36

the IOI’s list configuration [3 3 2]. If we consider again the example rhythmspresented at the beginning of this section in table 4.2 and we associate toeach rhythm its corresponding maximally even configurations, we may definethe evenness measures, normalized with respect to the evaluation of themaximally and minimally even rhythm with the same number of pulses andonsets. Normalization is applied computing evenness(rj) = Avej(d, p) −Avemin(n, p)/Avemax(n, p)−Avemin(n, p). Normalized evaluations are givenin table 3.5.

id rhythm evennessr1 [1 0 0 1 0 1 0 1 0 0 0 1 0 1 1 0] 0.94

Avemax(16, 7) [1 0 1 0 1 0 0 1 0 1 0 1 0 1 0 0] 1.00r2 [1 0 0 1 0 1 0 1 0 0 0 1 0 1] 0.94

Avemax(14, 6) [1 0 0 1 0 1 0 1 0 0 1 0 1 0] 1.00r3 [1 0 0 1 0 1 0 1 0 0 0 1 0] 0.97

Avemax(13, 5) [1 0 0 1 0 1 0 0 1 0 1 0 0] 1.00

Table 3.5: Evenness measure for rhythms in table 4.2 with corresponding maximallyeven configurations.

Evenness measure can be used for comparing rhythms with different num-ber of pulses and onsets. Moreover, we described in section 2.1.5.2 its rele-vance for perceived liking. Among the three measures we have seen in thissection, evenness is the most flexible, nevertheless we have to point out thatit can’t be a stand-alone complexity measure, basically for the fact that it isa pure geometric property which does not exploit the theories about meterperception. This effect is evident in table 3.5: we expect that a 13-pulsesrhythm (r3) would be more complex than a 16-pulses rhythm (r1), just be-cause a 4/4 time signature is a kind of trademark for pop and rock Westernmusic, while a 13/16 time signature rhythm would sound “strange” to many.Nevertheless, the evenness value is similar for both rhythms, with r3 beingslightly more even than r1, this shows that evenness measure is affected bysimilar problems of invariance as std.

From this considerations, we can state that toussaint, std and evennessare complexity measures efficient just for special cases and strictly dependenton the selected dataset. On the contrary, our goal is developing a model notonly able to capture the perceived complexity of a rhythm independentlyfrom its number of pulses and onset, but also allowing an efficient com-parison between rhythms with different characteristics, as we will show inchapter 4. The idea we will propose in section 3.2 takes inspiration from the

37

just described methods, but is based on the the implementation of a com-pound model that combines both information theory principles and musicalperception theories.

3.2 Complexity Model Architecture

The diagram depicted in figure 3.5 illustrates the architecture of our imple-mented model for rhythmic complexity evaluation. The system is fed withan input rhythm r of n onsets and p pulses and provides as output the fi-nal complexity value of r, C(r). It is mainly based on two novel stages: thenon-constant clock model (NC-Clock model) and the almost even hierarchies(AEH) evaluation. Such architecture is desirable because we exploit the sen-sitivity to onset positions given by the former, and to the number of pulsesgiven by the latter.

In the first stage, rhythm r is initially processed by the NC-Clock modeldescribed in section 3.3, whose result is a list of best clocks. These clocks aresequences of aperiodic intervals that sample in different ways the sequenceof accents derived from r, using the Povel and Essens’s rules for evaluationof phenomenal accents (section 2.2.3). Each best clock is a candidate non-isochronous beat sequence induced by the rhythm to the listener, thereforein the second stage we evaluate the metrical complexity of each of this se-quence through a special kind of weighted hierarchies that we call almosteven hierarchies (AEH), described in section 3.4. Complexity value Ci foreach clock sequence is taken as mean value of the hierarchies evaluations, atthe end the final complexity value for r is computed as the maximum valueamong all the Ci complexities.

3.3 The Non-Constant Clock Model

In literature, one of the most influent model about perception of temporalpatterns is the one from Povel and Essens [42], but we found out that aconstant subdivision model is not able to properly match the sequence ofaccents perceived in odd rhythmic patterns. For this reason, in this sectionwe adapt the original clock model in order to optimize the matching betweena clock and the accented list computed from an odd rhythm. We want alsoto understand how a mental clock reacts in front of non equal subdivisionsof the rhythmic pattern.

Let’s consider for example a rhythm of p = 13 pulses, r1 = [1 1 0 0 00 1 1 1 0 0 1 0], and the sequence of operations of the Povel and Essens’s

38

input rhythm: r

NC-Clock Model

NC-Clock_1 NC-Clock_2 NC-Clock_k. . .{List of best NC-Clock with lowest

C-score

beat sequence fromNC-Clock_1

beat sequence fromNC-Clock_2

beat sequence fromNC_Clock_k

AEH Evaluation

AEH Evaluation

AEH Evaluation

list of hierarchiesevaluations (lhe_1)

list of hierarchiesevaluations (lhe_2)

list of hierarchiesevaluations (lhe_k)

C_1 = mean(lhe_1) C_2 = mean(lhe_2) C_k = mean(lhe_k){Complexity

values, one for each beat

sequence from NC_Clock

C(r) = maxi C_iFinal complexity value

. . .

. . .

. . .

. . .

. . .

Figure 3.5: Overview of the novel complexity model architecture described in section3.2.

39

model (here called C-Clock model) for the evaluation of the best constantclock structuring the sequence of accents (figure 3.6).

Input rhythm, described as a list of onset bins, is initially transformedinto a sequence of accents according to the rules presented in section 2.2.3.Then, we have to parse the sequence of accents with given fixed samplingperiod and starting location given by phase, and counting the type of occur-rences, as illustrated in table 3.6.

period phase accents seq. k- k0 k+ C-score

2 0 [1 2 0 0 0 0 2 1 2 0 0 2 0] 4 1 2 17

3 2 [1 2 0 0 0 0 2 1 2 0 0 2 0] 2 0 2 8

5 1 [1 2 0 0 0 0 2 1 2 0 0 2 0] 0 0 3 0

Table 3.6: Three examples of C-score computation for C-Clock model.

As we noticed, there are problems when uneven sequence are fed in inputto the C-Clock model. The most relevant is that, in case of a sequence of ppulses with p prime number, the constant clock induction process formalizedby Povel and Esse’s has no a proper solution. There is indeed a distinctionbetween “clocks having a time unit that is a divisor of the sequence (..whichfit a sequence properly), and clocks with a non-divisor unit” [42]. Non divisortime units are inadequate to specify the temporal structure in the sequence,therefore the constant method solution is represented by the most inducedclock, that is the one with the lowest C-score, among all the clocks thatare divisor of the rhythmic sequence. As a matter of fact, there may existsmore than one clock with minimum C-score, however a sequence as the oneillustrated in figure 3.6 can’t have divisor units time since 13 is prime, thusis like none induced beat pattern is evoked by the sequence. We may tryto ignore this aspect and force the solution being the one we found in theexample, with a best clock of period=5, phase=1, C-score=0. However theinconveniences of this best clock are quite evident (see table 3.6): a non-zerophase clock, changing the start onset position, gives a bad perception of thereal rhythmic sequence; clock subdivisions are unequal and obviously don’tfit properly the sequence; an accented onset is completely bypassed in thesampling step (this is why here greater clock periods have lower C-scores,

40

input rhythm

(n onsets, p pulses)[1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0]

accents sequence [1, 2, 0, 0, 0, 0, 2, 1, 2, 0, 0, 2, 0]

- Generate all clocks

with period in range

[2,p/2]

- Apply weights and

compute C-score

period phase C-score

2 0 17

2 1 13

3 0 13

3 1 9

3 2 8

4 0 9

4 1 8

4 2 8

5 0 9

5 1 0

5 2 9

5 3 4

5 4 8

6 0 5

6 1 1

6 2 4

6 3 8

6 4 8

6 5 4

select best clock(s)

(minimun C-score) 5 1 0

Figure 3.6: Flow chart of the C-Clock model for rhythm r1 = [1 1 0 0 0 0 1 1 1 0 0 10]. Input rhythm is initially transformed into the corresponding accents sequence, thenC-score evaluation is applied. We can’t consider the distinction made in the originalmodel between time unit divisors and non divisors of p, since in this case p is prime andno divisors exists.

41

since there are more likely avoiding the sample of non accented onset). Inaddition to this, constant clocks may result in C-score whose minimum valueis not 0, as we can see in table 3.7 for the evaluation of the 13-pulses rhythmr2 = [1 0 1 0 1 0 1 1 0 0 0 1 0]. In this case, independently from divisor andnon-divisor time units, it is impossible finding a sequence ables to sampleonly accented onset, thus the induced beat sequence suggested by the modelis just an approximative representation of perceived meter.

period phase accents seq. C-score

4 3 [2 0 2 0 2 0 1 2 0 0 0 2 0] 4

5 2 [2 0 2 0 2 0 1 2 0 0 0 2 0] 4

5 4 [2 0 2 0 2 0 1 2 0 0 0 2 0] 4

6 1 [2 0 2 0 2 0 1 2 0 0 0 2 0] 4

6 2 [2 0 2 0 2 0 1 2 0 0 0 2 0] 4

6 4 [2 0 2 0 2 0 1 2 0 0 0 2 0] 4

6 5 [2 0 2 0 2 0 1 2 0 0 0 2 0] 4

Table 3.7: Best clocks list for rhythm r2 = [1 0 1 0 1 0 1 1 0 0 0 1 0] computedwith the C-Clock method. Minimum value of optimal C-score is 4, this means that forr2 there is no a constant sampling able to perfectly fitting the accents sequence. Aconsequence, these beat sequences are just an approximation of the induced meter.

To resume, C-Clock method is not efficient for the following reasons:

• when p is prime, a proper solution does not exist;

• when p is odd, beat sequences provided by C-score computation do notproperly fit the rhythmic pattern and are counterintuitive;

42

• it is not alway possible to find a beat sequence whose C-score is equalto 0, that is defining clock pattern able to capture only accented eventsand gaining reliable information about the metric structure of therhythm;

To overcome the problems, we propose a method called non-constantclock model (NC-Clock model), which is based on a variable period sam-pling of the accents structure. Thanks to this kind of approach, we are ableto determine the non-isochronous sequences of beat we perceive during thelistening of uneven sequences. This is a key information for knowing how arhythm is perceptually complex: once the beat structure has been establishedin our perception, “we then have the ability to internally maintain and/orexternally reproduce it in the absence of (or even in contradistinction to)external stimuli” [17], therefore meter analysis is necessary to evaluate thedifficulty level of the learning process.

The main difference in NC-Clock model is that the matching with ac-cented onsets is optimized trough a sequence of samples described by a listof index values, rather than a single constant period. For a rhythm r withp pulses, we first define a set of periods in the range [min_per,max_per]where min_per = 2 and max_per = p/2 if p is even or max_per = p/2 + 1

if p is odd, then we generate all the combinations of periods in that range,whose sum is equal to p. This set of combinations is the new group of non-constant clocks we want to test against the temporal pattern. In figure 3.7is depicted the sequence of operations for best C-score evaluation in NC-Clock model, where the input rhythm is r1 (the same used in in figure 3.6).Here we set the phase of each clock to 0, without considering other kindof synchronization, since we want keeping aligned both rhythm and beatsequence.

Even in this case the method results in a list of best clock whose C-scoreis different from 0, but this is quite obvious if we look at accents structureand the condition we impose to the phase. On the other hand, we get asresult a list of dynamic clocks that fit properly the temporal sequence, havingboth long and short intervals parsing the sequence, in this way the modeloutput is a complete list of optimal beat structures.

For a better comparison of evaluation models, we may consider again therhythm r2 = [1 0 1 0 1 0 1 1 0 0 0 1 0], for which C-Clock model gave poorperceptual results. Using the NC-Clock model, we get a list of 10 best clockwith C-score = 0, as illustrated in figure 3.8.

Thanks to the non-constant sampling of accents sequence, we can prop-erly extend the method from Povel and Essen’s to the analysis of rhythms

43

input rhythm

(n onsets, p pulses)[1 1 0 0 0 0 1 1 1 0 0 1 0]

accents sequence [1 2 0 0 0 0 2 1 2 0 0 2 0]

- Generate all clocks

with sequence of

periods in range

[2,p/2], whose sum

is equal to p

- Apply weights and

compute C-score

set of

periodsphase C-score

[6,7] 0 1

[7,6] 0 2

[2,4,7] 0 5

[2,5,6] 0 6

[2,6,5] 0 5

[2,7,4] 0 9

[3,3,7] 0 5

[2,2,2,7] 0 9

[2,3,5,3] 0 13

[2,4,2,5] 0 5

[5,2,2,2,2] 0 10

[4,3,2,2,2] 0 10

[2,2,3,2,2,2] 0 14

select best clock(s)

(minimun C-score)

[6,7] 0 1

. . .

. . .

. . .

[6,2,5] 0 1

[6,5,2] 0 1

[6,2,3,2] 0 1

Figure 3.7: Flow chart of the NC-Clock model for rhythm r1 = [1 1 0 0 0 0 1 1 10 0 1 0]. For this rhythm, there are in total 126 possible combinations of periods inrange [min_per,max_per] whose sum is equal to p. We skipped some of them, forconvenience.

44

set of periods phase accents seq. C-score

[7, 6] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[2, 5, 6] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[4, 3, 6] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[4, 7, 2] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[7, 4, 2] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[2, 2, 3, 6] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[2, 2, 7, 2] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[2, 5, 4, 2] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[4, 3, 4, 2] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

[2, 2, 3, 4, 2] 0 [2 0 2 0 2 0 1 2 0 0 0 2 0] 0

Table 3.8: Best clocks list for rhythm r2 = [1 0 1 0 1 0 1 1 0 0 0 1 0] with NC-Clockmodel. Minimum value of C-score=0. All of these non constant clock fit withoutapproximations the sequence of accents, and all of them are possible candidates forbeing induced as beat pulses to the listener’s perception.

45

with generic number of pulses. How we can show in figure 3.8, the modelproposes a series of possible solutions for structuring the listening patternand, even tough we need a unique complexity value, it would be quite ineffi-cient trying to extract a final value from a sort of processing applied directlyto C-score value, since best scorings are always of the same value. For thisreason, we introduce in the following session a further processing stage.

3.4 Almost Even Hierarchy Evaluation

Almost even hierarchy (AEH) is a kind of metrical weighted hierarchy thatwe introduce for measuring the structure properties of the beat sequencesresulting from the NC-Clock model. Computation of hierarchies is inspiredby the theory about almost maximally even rhythms described by Toussaintin [2]. Given a sequence of n = 5 onsets and p = 16 pulses, the numberof different rhythms may be created from the various pattern configurationsare Cp,n =

(pn

)= p!

n!·(p−n)! = 4, 368. This is a huge number, however it turnsout that 16 combinations of this set are more relevant than the others, sinceare the most attractive and popular among musicians. These sequences arethe ones close to the optimal maximally even rhythm given by rmaxev = [10 0 1 0 0 1 0 0 0 1 0 0 1 0 0]. We can visualize these patterns in figure 3.8.

Figure 3.8: Almost maximally even rhythm positions, for a pattern with n=5 onsetsand p=16 pulses (image taken from [2]).

The diagonal line, starting from zero indexed pulse ad arriving to itscyclic self-image, together with the horizontal line, representing the onsetnumber, create a set of intersection points indicating the pulse time at whichan onset should be played to generate a maximally even rhythm. If we haven = 5 and p = 16, k-th onset should be assigned to k-th pulse pk = p·k

n

46

Figure 3.9: 16 almost maximally even rhythms with n=5 onsets among p=16 pulses(image taken from [2]).

with k ∈ (0, n − 1), thus assigned positions are: 0.0, 3.2, 6.4, 9.6, and 12.8.Approximation of these value to the nearest pulse position yield to indexes0, 3, 6, 10, 13, or equivalently to the IOI’s list [3 3 4 3 3], that is indeedthe corresponding IOI notation of rmaxev. Now, the almost maximally evenrhythms are all that configurations with 5 onsets and 16 pulses where thefirst onset is at pulse 0, while the others are at the nearest left or right pulseof each maximally even position (black dots assignment in figure 3.8). Asconsequence, the 16 selected rhythms are the one with onsets in selectable po-sitions: 0, (3,4), (6,7), (9,10), (12,13). Box notation of the resulting rhythmsis illustrated in figure 3.9 (some of them are already known drum patternsequence).

Popularity of these structures let us think that evenness property isfundamental in the analysis of perceptual experiences imposed by the hu-man brain during listening and performing activities. As we said in sec-tion 3.3, studying the strength of beat induction results in a list of bestnon-isochronous clocks candidates to describing the sequence of perceivedaccents. In order to elaborate a complexity measure, we choose to process

47

{13}

{6, 7}

{3, 3, 3, 4}

{3, 3, 3, 2, 2}

{3, 3,4, 3}

{3, 3,2, 2, 3}

{2, 4, 3, 4}

{2, 2, 2, 3, 2, 2}

{2, 4, 4, 3}

{2, 2, 2, 2, 2, 3}

{4, 2, 3, 4}

{2, 2, 2, 3, 2, 2}

{4, 2, 4, 3}

{2, 2, 2, 2, 2, 3}

{7, 6}

{3, 4, 3, 3}

{4, 3, 4, 2}

{3, 2, 2, 3, 3}

{2, 2, 3, 2, 2, 3}

. . .

. . .

Figure 3.10: Almost even hierarchical decomposition for p = 13 pulses. Resulting AEHtree is symmetric, in this case we obtain 12 leaf in total, where each leaf provides a listof elements giving the order to recursively subdivide the grid of pulses.

the structure of induced clocks rather than working directly on the rhythm.In section 2.2.2.1 we found that Toussaint’s syncopation measures was verygood in describing the complexity level of a rhythm, assigning proportionalweights to onsets positioned in off-beat pulse indexes. However, the problemdescribed in section 3.1 highlights that the weighted hierarchies describedby Toussaint are not usable in the analysis of odd sequence, and especiallyfor sequences with a prime number of pulses. This is the reason why weintroduce AEH evaluation: we want to construct new weighted hierarchies,based on almost maximally even subdivision, and use these metrical grids toassign a complexity value to each best induced clock. The idea is that thefarthest are the beat sequences from the condition of maximally evenness,the harder is the complexity of the sounding rhythm.

Let’s consider for example a 13-pulses rhythm r=[1 1 0 0 0 0 1 1 1 0 01 0]. After the analysis of the NC-Clock method, we get a list of inducedbeat sequences that are 13 pulses long as well. Instead of computing metri-cal weights through prime factorization technique applied to the number ofpulses p, as done in [13], we apply AEH decomposition of p, as depicted infigure 3.10.

The subdivision is called almost even because we force each pair of seg-ments not having a distance greater than 2, where distance is defined in termsof number of pulse units. Therefore, subdivisions like {13} -> {5,8} or {6,7}-> {3,3,5,2} are not allowed, while {13} -> {6,7} or {6,7} -> {3,3,4,3} areadmitted. 2’s and 3’s are the minimal elements in the hierarchy, with re-

48

Pulse Index

Metr

ical W

eig

ht

0 1 2 3 4 5 6 7 8 9 10 11 12

5

4

3

2

1

Figure 3.11: One of the 12 AEH hierarchies resulting from almost maximally evendecomposition method applied to p=13.

spect to the theory the duple and triple meters are the simplest temporalstructure in metrical hierarchies [17]. It is worth noting that decompositionprocess depends only on p, moreover the result is a list of weighted grids,one for each leaf of the hierarchical tree, as consequence for any 13-pulsesclock (or rhythm in general), there are 12 almost even hierarchies, while forexample for any 14-pulses clock there are 34 almost even hierarchies. Eachbranch of the tree is indeed a path for generating metrical weights by recur-sive decomposition of the subsets at every step. In figure 3.11 we can see forexample the metrical hierarchy corresponding to the path: {13} -> {6,7} ->{3,3,3,4} -> {3,3,3,2,2}. For weights assignment we actually considered bothno subdivision and p-equal subdivision options (see [13], pg.9 for details)

Once we compute the list of AEH’s corresponding to a given p-lengthsequence, we can proceed to the evaluation of clock sequences in the sameway indicated in section 2.2.2.1: summing the weights corresponding to beatpositions in the temporal pattern. Since we get more than one hierarchy,final mean value is assigned as complexity evaluation for a given best clock.While the aim of Toussaint’s method was evaluating the syncopation level ofa rhythm through the analysis of the off beatness of the onsets position, herewe study the correlation of the induced meter with all the possible almosteven structure distributions. According to this method, a clock assignedwith an lower value of AEH evaluation is a clock whose beat structure isirregular, asymmetric and, as consequence, more complex from a cognitivepoint of view.

In the last stage of our model, we select the maximum complexity valueC(r) among the list of AEH evaluations assigned to each induced non-

49

constant clock, since the greatest value correspond to the most even sequenceable to capture the accents distribution of the input rhythm. We establishthat, by transitive property, a rhythm that evokes a clock sequence thatin turn has a complex structure, therefore also the rhythm is complex toentrain. For this reason, C(r) is the final value we propose as measure ofperceived complexity for a given rhythm.

50

Chapter 4

Experimental Results

In section 4.1 we will describe the performance test we developed in orderto gather data about the listener’s perception of odd rhythms. In particularwe will explain how we chose the rhythms, the performance measures andfew details about the implementation. After that, in section 4.2 we willanalyze the experimental results collected form the user test sessions and wewill show their correlation with some of the complexity models discussed inprevious chapters.

4.1 Setup

The experiment reported here concerns both reproduction and complexityjudgments of temporal patterns. We chose to implement a Web-based per-formance test to make it accessible as more as possible, without a selectionof participants based on specific characteristics.

4.1.1 Experimental data

The rhythms we included in the dataset for the perceptual test are generatedtaking example from the permutation method [54]. This procedure computesall the permutations of a sequence of IOI’s. We have considered:

• periods = [1, 2, 3, 4, 5] - list of periods, defining a set of possible IOI’sthat can be used to generate the rhythmic pattern;

• p1 = 14, p2 = 13 - two groups of pulses;

• n = 6 - fixed number of onsets (with given p1 and p2, number 6 is agood candidate for generating interesting pattern structures, otherwise

Figure 4.1: Landing page of the complexity test.

less or more onsets would respectively occupy poorly or largely the timespace).

From this settings, we have generated all the sequences resulting fromthe combination of elements in periods list, whose sum is equal to p1 orp2. We got 666 unique combinations of 13-pulses rhythms, while 951 uniquecombinations of 14-pulses rhythms. For each group of pulses we computedstandard deviation (std) ordered the sequences according to std values andsampled uniformly a number of rhythms. For reasons related to overall testduration and greater interest in odd lengths, we finally composed a datasetof 28 rhythm, which 12 are 14-pulses rhythms and 16 are 13-pulses rhythms.The final dataset is illustrated in table 4.1.

4.1.2 Test Modeling and Procedure

A test session is a sequence of 28 test cases, each session is different fromthe other in the order into which rhythms are drawn randomly from thedataset described in table 4.1. Test can be completed in approximately 30minutes, everyone is allowed to join it. Figure 4.1 shows the introduction tothe complexity test.

Before the experimental session, a training session is run, where the lis-tener is free to get confident with the interface. The training is identical tothe experimental session, except for the resulting data that are discarded.User is asked to perform at least three test cases, after which he can chooseto continue practicing or switch to the testing phase.

The rhythms are reproduced as wav files with sample accurate loop-ing, using the webaudioAPI library in javascript. The tap events are timestamped using javascript timer, which is accurate in the order of millisec-onds. While the onset is a simple sinusoidal click, a triangle instrumentsound is reproduced at the start of each loop, to sync the bar metric levelat the start of the rhythm, avoiding the possible induction of a clock withnonzero phase. The audio files have been created using a pulse durationTp = 200ms. In figure 4.2 is depicted an example of the user interface.

The subject is asked to listen to the rhythm playing in loop until he/she

52

Table 4.1: Rhythms and corresponding IOI’s pattern included in the dataset.

id rhythm IOI’s list n. onsets n. pulsesr0 [1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1] [3 2 2 4 2 1] 6 14r1 [1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1] [3 2 5 2 1 1] 6 14r2 [1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0] [1 1 3 5 2 2] 6 14r3 [1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0] [2 1 1 4 3 3] 6 14r4 [1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0] [1 5 2 1 3 2] 6 14r5 [1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1] [2 3 5 1 2 1] 6 14r6 [1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1] [1 5 2 2 3 1] 6 14r7 [1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0] [2 3 2 2 1 4] 6 14r8 [1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0] [2 3 2 2 2 3] 6 14r9 [1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0] [1 1 1 4 2 5] 6 14r10 [1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0] [1 1 3 5 1 3] 6 14r11 [1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0] [2 2 4 1 3 2] 6 14r12 [1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0] [2 1 3 1 1 5] 6 13r13 [1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0] [1 5 1 1 3 2] 6 13r14 [1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0] [5 1 1 1 2 3] 6 13r15 [1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0] [3 3 1 3 1 2] 6 13r16 [1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0] [1 2 3 1 1 5] 6 13r17 [1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1] [4 1 2 2 3 1] 6 13r18 [1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0] [5 2 2 1 1 2] 6 13r19 [1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0] [1 2 1 2 4 3] 6 13r20 [1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1] [1 3 1 2 5 1] 6 13r21 [1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0] [2 2 3 2 1 3] 6 13r22 [1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0] [1 2 2 3 1 4] 6 13r23 [1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1] [2 2 5 1 2 1] 6 13r24 [1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1] [5 1 1 4 1 1] 6 13r25 [1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0] [3 2 1 3 2 2] 6 13r26 [1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1] [3 3 2 2 2 1] 6 13r27 [1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0] [2 2 2 1 4 2] 6 13

53

Figure 4.2: UI of a test case shown during the listening/tapping phase. No visualresponse of the rhythm is given.

feels confident. Then, he/she starts reproducing the rhythm by pushing abutton at the onset times while the rhythm is still playing. Data recordingwindow starts in correspondence of the first tap assigned to the first onsetof a given loop and ends after the rhythm has been cyclical playing for fourtimes. Since we want the user being totally focused on the listening, no visualresponse is given during the tapping, and we provide a simple animatingrepresentation of the rhythm just after the performance of the single testcase (see figure 4.3). Here the subject can visually check and listen again tothe rhythm he just tapped, while the animation follows the ongoing rhythm.

In order to proceed to the next test case, the user must provide two kindof complexity judgments: perceptual complexity (perceval) and performancecomplexity (perfeval). The first refers to an abstract perceived complexity,while the former refers to a practical complexity related to tapping difficulty.These two concepts may overlap each other, but their relationship is notobvious.

Two measures have been chosen to evaluate the tapping performanceof the test. Both of them compare the sequence of taps with the rhythm,represented by a sequence of onset times:

54

Figure 4.3: UI shown after the tapping of a rhythm. The user can listen back to therhythm and checking the corresponding box notation. Before proceeding to the nexttest case, he/she must provide perceptual and performance complexity judgments.

θ[kNo + i] = (oi + kNp)Tp k ∈ R+, i = 0, . . . , No, (4.1)

where Np is the number of pulses, No the number of onsets, Tp the pulseduration and k the repetition counter.

The first measure is called tap accuracy Sta, and measures the sum ofthe distances of the tap events τi from the onsets, using a Gaussian kernelfunction for the distance:

Sta =1

No

Nτ∑i=0

∑j

κ(τi − θ[j]), (4.2)

where k increases up to the last repetition of the tapping performance.The chosen kernel function κ = e− x2

2·σ2 is a Gaussian kernel with standarddeviation σ = 30ms, therefore it does not overlap among adjacent onsets.

The second performance measure Smds is called mean deviation scoreand is conceptually similar to the one used in [42]. It evaluates the accuracyof the intervals between adjacent taps:

Smds =

No∑i=0

‖(τi+1 − τi)− (θ[Θ(τi+1)]− θ[Θ(τi)]‖, (4.3)

where Θ(t) indicates the index of the onset time that is nearest to thetime t:

55

Θ(t) = arg mini‖θ[i]− t‖ (4.4)

These measures have been inverted for making them more intuitive: Sta isa sum of distances already inverted by the nature of kernel function, for whicha tap event more distant from an onset time is weighted lower, therefore anhigh Sta indicates a better tapping performance; Smds is computed as theinverse of the sum of differences between taps and onsets distances, thereforealso in this case better higher values of Smds describe a better performance.

4.2 Evaluation

In the current section, we first provide a description of the data gathered fromthe test and how we dealt with the outliers, in order to retain only meaning-ful samples. Then we validate our rhythmic complexity model studying itscorrelation with the ground truth, which is build on the average performanceratings computed among the user test sessions.

4.2.1 Outlier detection

Although the landing page specified clearly that the test is mainly targetedto musicians, we decided not to restrict the access and let anyone executethe test. While this allowed us to gather a significantly larger amount ofdata, it also had the effect of generating some spurious data. Therefore, weperformed an outlier detection procedure as a preprocessing stage, with thepurpose removing only the meaningless data.

A test case is marked as outlier if one of two conditions is satisfied. Thefirst condition is trivial and check if the number of taps is less than thenumber of onsets, that is Nτ < No. We remind that Nτ , in the optimalcase, is equal to 4 · No, since evaluation metrics refer to the tapping of arhythm repeated for 4 times- Therefore the first condition defines a minimumrequired number of taps equals at least to No. The second condition focuseson the temporal density of the tap events:

d =1

Nτ

Nτ∑i=0

Θ(τi+1)−Θ(τi), (4.5)

and marks the test as outlier if d < 1/2 or d > 2, which means thesubject systematically skipped more than onset or tapped more than twiceper onset.

56

4.2.2 Model Validation

At the end of testing phase, we collected a total of 15 sessions (made by 15different subjects), which in total correspond to 420 test cases (28 rhythmsfor each session). Very few of them got some outliers, that we discarded forthe analysis (the procedure of outlier removed about the 1% of the data).We want to test the correlation of the average ratings given by Sta andSmds with a group of complexity evaluation methods, studying the differentbehaviours. We received some feedback from users who did the experiment,suggesting that there are different ways to perceive rhythmic complexity. Forthis reason, we perform an additional analysis of outliers, this time amongthe sessions (not the test cases, as described in section 4.2.1), in order tofocus on the largest cluster of sessions with similar ratings. We measurethe linear relationship betweens the sessions evaluations with the Pearsoncorrelation coefficient.

When we evaluate the correlation of the ratings of a user session withthe vector of the average ratings, we obtain a certain correlation coefficient(CC). If we repeat this operation for all the sessions, we obtain a vectorof CC’s. Removing iteratively the smallest coefficient from this vector, itis like removing the sessions that worse correlate with the average ratings,that is discarding the sessions that are far from the performances of theother users. Since we defined two evaluation measures, Sta and Smds, thereexists two vectors of CC’s, that we call mean tap correlation and mean mdscorrelation. Now, for each vector, we iteratively remove the session with thesmallest CC, and then we compute the corresponding vector mean value.Doing so, we can select the sessions that, in mean, correlate with the averageratings more than a certain coefficient. This procedure is depicted in figures4.4 and 4.5, where, for both Sta and Smds, we report the increasing meanvector value in function of the first 5 removed sessions (we assume 5 as themaximum number of removable sessions, otherwise data for analysis wouldbecome insufficient).

Even if mean tap correlation and mean mds correlation vectors showtogether an high correlation (i.e. CC = 0.60), they do not perfectly sharethe final set of removed sessions, hence we define a unique set by imposinga threshold τ = 0.35 and keeping sessions with CC > τ both for mean tapmodel and mean mds model. In the end, indexes of the removed sessions are[1,2,9,12,14] and 10 test session remains, that correspond to 280 test cases (28rhythms for 10 test sessions). In figures 4.6, 4.7, 4.8 and 4.9 are illustratedthe boxplots of the tap accuracy and mds measures for the selected testsessions, sorted by the relative average ratings and divided by the number of

57

[]

[12]

[12,

2]

[12,

2, 1

][1

2, 2

, 1, 1

4][1

2, 2

, 1, 1

4, 3

]

removed sessions

0.48

0.50

0.52

0.54

0.56

0.58

0.60

mean c

orr

ela

tion v

alu

e

mean value of the mean tap correlation vector

Figure 4.4: Mean value of the mean tap correlation vector, drawn in function of thefirst 5 removed sessions, whose evaluations worse correlate with the average ratings.

[]

[12]

[12,

1]

[12,

1, 1

4][1

2, 1

, 14,

9]

[12,

1, 1

4, 9

, 13]

removed sessions

0.45

0.50

0.55

mean c

orr

ela

tion v

alu

e

mean value of the mean mds correlation vector

Figure 4.5: Mean value of the mean mds correlation vector, drawn in function of thefirst 5 removed sessions, whose evaluations worse correlate with the average ratings.

58

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

3

2

1

0

1

2

3

tap a

ccura

cy

Figure 4.6: Boxplot of the tap accuracy ratings computed for the 10 selected testsessions and relative to 14-pulses rhythms.

pulses. We called the average ratings mean tap model and mean mds modelrespectively (mean models if addressed together), and we assume these twomodels as the ground truth. In general, a good performance is described byhigh values both for tap accuracy and mds, mean models are indeed highlycorrelated (CC = 0.869).

In the following, we list the eight methods notation we are going to testtogether with the mean models:

1. SYNC : rhythm syncopation measure, introduced by Toussaint [13];

2. STD : rhythm standard deviation measure [13];

3. E : rhythm evenness measure, computed as described in section 3.1 [3];

4. E_C : evenness measure applied to the best constant beat sequence in-duced by a rhythm. If there is more than one best sequence, maximumevenness value is taken;

5. E_NC : evenness measure applied to the best non-constant beat se-quence induced by a rhythm. If there is more than one best sequence,maximum evenness value is taken;

6. AHE : almost even hierarchy evaluation applied to the rhythm;

59

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27rhythm (p=13)

4

3

2

1

0

1

2

3ta

p a

ccura

cy

Figure 4.7: Boxplot of the tap accuracy ratings computed for the 10 selected testsessions and relative to 13-pulses rhythms.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

4

3

2

1

0

1

2

mds

Figure 4.8: Boxplot of the mds ratings computed for the 10 selected test sessions andrelative to 14-pulses rhythms.

60

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27rhythm (p=13)

3

2

1

0

1

2

3

mds

Figure 4.9: Boxplot of the mds ratings computed for the 10 selected test sessions andrelative to 13-pulses rhythms.

7. AHE_C : almost even hierarchy evaluation applied to the best constantbeat sequence. If there is more than one best sequence, maximum AEHvalue is taken;

8. AHE_NC : almost even hierarchy evaluation applied to the best non-constant beat sequence. If there is more than one best sequence, max-imum AEH value is taken.

61

1. SYNC

SYNC and mean models graphs are illustrated in figures 4.10 and 4.11.

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

2.5

evalu

ati

on

mean tap model

SYNC model

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27

rhythm (p=13)

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean tap model

SYNC model

Figure 4.10: Correlation graphs between mean tap and SYNC models. Tap accuracyCC value = -0.013.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

2.5

evalu

ati

on

mean mds model

SYNC model

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27

rhythm (p=13)

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean mds model

SYNC model

Figure 4.11: Correlation graphs between mean mds and SYNC models. Mds CC value= -0.198.

Resulting CC’s values are almost null, -0.013 for the tap accuracy and-0.198 for the mean deviation score. As we explained in section 3.1, Tou-ssaint’s measure can’t basically exploit the information carried by rhythmswith a number of pulses different from a multiple of 4, for this reason synco-pation evaluation of 14-pulses rhythms is very poorly meaningful, while for13-pulses rhythms is totally flat.

2. STD

STD model measures the complexity of a rhythm describing the amount ofsymbols needed to represent it. We explained in section 3.1 that it is not

62

always able to efficiently evaluate rhythmic complexity, but the strong cor-relation values between mean and inverse STD (inv STD) models (0.679 fortap accuracy and 0.670 for mean deviation score, figures 4.12 and 4.13) indi-cate that, for this dataset, std measure is a good predictor. We provide theinv STD model for a more intuitive comparison of the models: an IOI’s listwith a small std values describe a simpler pattern, since less information isneeded to be elaborated in the listening process, as consequence STD mea-sure is inversely proportional to the goodness of performance and negativecorrelation coefficients would result without inverting it.

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

2.5

evalu

ati

on

mean tap model

inv STD model

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27

rhythm (p=13)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

evalu

ati

on

mean tap model

inv STD model

Figure 4.12: Correlation graphs between mean tap and inv STD models. Tap accuracyCC value = 0.679.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

2.5

evalu

ati

on

mean mds model

inv STD model

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27

rhythm (p=13)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

evalu

ati

on

mean mds model

inv STD model

Figure 4.13: Correlation graphs between mean mds and inv STD models. Mds CCvalue = 0.670.

3. E

Rhythm evenness is an inverse complexity measure and a reliable descriptorof cognitive complexity, as we can see in figures 4.14 and 4.15, where correla-

63

tions values are 0.513 for the tap accuracy and 0.543 for the mean deviationscore. These values confirm that distribution of onsets over the grid of pulsesis an important complexity feature, but we showed in section 3.1 that it isnot sufficient for discriminating rhythmic complexity features related to thenumber of pulses.

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean tap model

E model

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27

rhythm (p=13)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean tap model

E model

Figure 4.14: Correlation graphs between mean tap and E models. Tap accuracy CCvalue = 0.513.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean mds model

E model

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27

rhythm (p=13)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean mds model

E model

Figure 4.15: Correlation graphs between mean mds and E models. Mds CC value =0.543.

4. E_C

In this case we measure the evenness of the best constant clock sequence(s)induced by a rhythm. If the result of the C-Clock model is a list of clocks,then mean evenness value is considered. Correlation graphs are shown infigures 4.16 and 4.17

64

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

evalu

ati

on

mean tap model

E_C model

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27

rhythm (p=13)

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean tap model

E_C model

Figure 4.16: Correlation graphs between mean tap and E_C models. Tap accuracy CCvalue = 0.076.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

evalu

ati

on

mean mds model

E_C model

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27

rhythm (p=13)

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean mds model

E_C model

Figure 4.17: Correlation graphs between mean mds and E_C models. Mds CC value= -0.031.

There is no correlation between models (tap accuracy = 0.076, meandeviation score = -0.031). This method is inefficient mainly because constantclock sequences are generally defined from a few number of onsets, mostlydistant one from the other. In fact, when constant sampling is used, theperiods almost equal to the half of the rhythm length are the ones with bestC-score, since in this way unaccented events are discarded (see figure 3.6). Asconsequence, evenness measure is quite similar for all the final beat patternsinduced by rhythms, and cognitive complexity can’t be properly estimatedamong different rhythms.

5. E_NC

Here we evaluate the evenness of best clock induced sequences, but this timeusing non-constant sampling. Also in this case models are poorly correlated

65

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

1.5

1.0

0.5

0.0

0.5

1.0

evalu

ati

on

mean tap model

E_NC model

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27

rhythm (p=13)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

2.5

evalu

ati

on

mean tap model

E_NC model

Figure 4.18: Correlation graphs between mean tap and E_NC models. Tap accuracyCC value = -0.364.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

evalu

ati

on

mean mds model

E_NC model

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27

rhythm (p=13)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

2.5

evalu

ati

on

mean mds model

E_NC model

Figure 4.19: Correlation graphs between mean mds and E_NC models. Mds CC value= -0.231.

(tap accuracy = -0.364, mean deviation score = -0.231), as illustrated infigures 4.18 and 4.19.

In general, when using the NC-Clock model, the resulting best clockslist has a great number of candidate sequences, therefore the assigned meanevenness value is well distributed among the rhythms and there is no asaturation of values as in the E_C model. However, it is clear that theanalysis of beat patterns through evenness feature is not indicative of theperceived rhythmic complexity, both with constant and non-constant clockperiods.

6. AEH

AEH complexity estimation is a procedure similar to the Toussaint’s synco-pation measure, but construction of the metrical grids is based on almost

66

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean tap model

AEH model

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27

rhythm (p=13)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

evalu

ati

on

mean tap model

AEH model

Figure 4.20: Correlation graphs between mean tap and AEH models. Tap accuracy CCvalue = 0.514.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean mds model

AEH model

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27

rhythm (p=13)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

evalu

ati

on

mean mds model

AEH model

Figure 4.21: Correlation graphs between mean mds and AEH models. Mds CC value= 0.497.

evenness principle rather than prime factorization of the number of pulses.As the SYNC model, AEH is an inverse complexity measure, therefore agreater AEH rating describe a less complex rhythm (see section 3.4 for de-tails). The AEH complexity rating of a rhythm is equal to the mean valueamong the metrical hierarchy evaluations, and from figures 4.20 and 4.21 wecan observe that CC’s values between models are quite good (tap accuracyCC value = 0.514, mean deviation score CC value = 0.497), meaning thatAEH weighting schemes are nice predictors of perceived rhythmic complex-ity.

7. AEH_C

This is a compound model where AEH evaluation is computed for the bestinduced clock sequences, as described in figure 3.5, with the difference that

67

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

2.5

evalu

ati

on

mean tap model

AEH_C model

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27

rhythm (p=13)

1.0

0.5

0.0

0.5

1.0

1.5

2.0

evalu

ati

on

mean tap model

AEH_C model

Figure 4.22: Correlation graphs between mean tap and AEH_C models. Tap accuracyCC value = 0.167.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

2.5

evalu

ati

on

mean mds model

AEH_C model

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27

rhythm (p=13)

1.0

0.5

0.0

0.5

1.0

1.5

2.0

evalu

ati

on

mean mds model

AEH_C model

Figure 4.23: Correlation graphs between mean mds and AEH_C models. Mds CC value= -0.095.

here we use constant period sampling of the rhythm accented sequence. Infigures 4.22 and 4.23 are shown the correlation graphs, which basically havealmost null CC’s values (tap accuracy CC value = 0.167, mean deviationscore CC value = -0.095). We may conclude that constant clock sequencesare not able to properly describe the perceived complexity of rhythms withuneven time signature, both if we describe them trough evenness feature (asdone in E_C model) or AEH feature.

8. AEH_NC

We finally present the model described in chapter 3 and modeled in figure3.5. The AEH evaluation of the best non-constant clock sequence(s) resultingfrom the NC-Clock analysis gives the most high CC’s values with the meanmodel, as we can see in figures 4.24 and 4.25, where values are: tap accuracy

68

r10 r9 r5 r6 r2 r4 r11 r0 r3 r1 r7 r8rhythm (p=14)

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean tap model

AEH_NC model

r24 r14 r20 r12 r13 r19 r23 r17 r15 r16 r22 r21 r26 r18 r25 r27

rhythm (p=13)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

evalu

ati

on

mean tap model

AEH_NC model

Figure 4.24: Correlation graphs between mean tap and AEH_NC models. Tap accuracyCC value = 0.769.

r10 r1 r2 r5 r9 r6 r4 r0 r7 r11 r3 r8rhythm (p=14)

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

evalu

ati

on

mean mds model

AEH_NC model

r20 r14 r24 r12 r13 r19 r15 r23 r16 r22 r17 r25 r26 r18 r21 r27

rhythm (p=13)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

evalu

ati

on

mean mds model

AEH_NC model

Figure 4.25: Correlation graphs between mean mds and AEH_NC models. Mds CCvalue = 0.682.

CC value = 0.769, mean deviation score CC value = 0.682. We remind thatAEH_NC is an inverse complexity measure, therefore a greater AEH_NCrating describe a less complex rhythm.

We thus can establish that AEH_NC computational model provides aperceptually significant rhythmic complexity evaluation, combining aspectsrelated both to formal and informal notion of complexity.

Goodness of AEH_NCmodel can be confirmed also resuming the 3 exam-ple rhythms introduced in section 3.1, where none of the considered modelswas able to capture complexity information carried by the rhythmic struc-ture. Rhythms table 4.2 is here reported for convenience.

We apply the AEH_NC model to the rhythms in table 4.2 and we reportthe standardized ratings in table 4.3.

SYNC method evaluates r2 as the most complex rhythm, but we knowthat this measure is strictly dependent on the number of rhythm onsets. E

69

id rhythm IOI’s list pulsesr1 [1 0 0 1 0 1 0 1 0 0 0 1 0 1 1 0] [3 2 2 4 2 1 2] 16r2 [1 0 0 1 0 1 0 1 0 0 0 1 0 1] [3 2 2 4 2 1] 14r3 [1 0 0 1 0 1 0 1 0 0 0 1 0] [3 2 2 4 2] 13

Table 4.2: The 3 example rhythms we described in section 3.1.

id SYNC E inv STD AEH_NCr1 12 0.94 -0.95 1.34r2 8 0.94 -1.93 -1.05r3 6 0.0.97 -0.89 -0.28

Table 4.3: AEH_NC inverse complexity estimation for rhythms in table 4.2, comparedwith the other models evaluation. Models are notated as in the following: SYNC = Tou-ssaint’s syncopation; E = evenness; inv STD = inverse standard deviation: AEH_NC= AEH evaluation of induced non-constant beat pattern.

measure can’t distinguish r1 from r2, and classifies r3 rhythm as the simplestone. Inv STD method evaluates r2 as the most complex rhythm as well,but then establishes that r1 is more complex than r3. AEH_NC measuredoes not depend on the number of pulses and estimates that r2 is morecomplex than r3, which in turn is more complex than r1. In our opinion,since AEH_NC model relies on informations regarding both the strengthof the induced beat pattern and the quality structure of the beat patternitself, it provides the most perceptually reliable classification. Moreover,even though the nature of hierarchical grids tend to evaluate a rhythm withmore onsets with an higher score and thus with a lower complexity value, ourmodel still provide a proper complexity estimation when different rhythmicpatterns are compared. For example, if we add to the dataset of table 4.2the 16-pulses “four-on-the-floor” rhythm r4, whose onset pattern is largelydiffused in disco and dance music genres, the most complex sequence remainsr2, and r4 is evaluated as the most simple, even if its pattern has got thesmallest number of onsets with regard to the other rhythms (see table 4.4)

70

CC ta md sy ist e ec en a ac antamd 0.86sy -0.01 -0.19ist 0.68 0.67 -0.04e 0.51 0.54 -0.005 0.78ec 0.07 -0.03 -0.29 0.01 -0.197en -0.36 -0.23 -0.43 -0.008 0.035 0.02a 0.51 0.49 -0.01 0.52 0.248 0.06 -0.49ac 0.16 -0.09 0.27 -0.08 -0.13 0.07 -0.11 -0.06an 0.77 0.68 0.06 0.71 0.495 0.07 -0.49 0.77 -0.01

Table 4.5: Pearson coefficients matrix computed for all the described models. Weshortened the models notation in the following way: ta = mean tap, md = mean mds,sy = SYNC, ist = inv STD, e = E, ec = E_C, en = E_NC, a = AEH, ac = AEH_C,an = AEH_NC. Gray highlighted columns refer to ground truth vectors we used formodels validation. Bold CC’s values are relative to the models that better describe theperceived complexity, that are inv STD and AEH_NC.

id rhythm AEH_NCr1 [1 0 0 1 0 1 0 1 0 0 0 1 0 1 1 0] 0.26r2 [1 0 0 1 0 1 0 1 0 0 0 1 0 1] -1.10r3 [1 0 0 1 0 1 0 1 0 0 0 1 0] -0.66r4 [1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0] 1.50

Table 4.4: AEH_NC inverse complexity measures for rhythms in table 4.2 plus the“for-on-the-floor” rhythm r4. Even though r4 is the pattern with the smallest numberof onsets, it is still estimated as the most simple with respect to the other rhythms.

In order to give a general overview of the models we have analyzed inthis chapter and how their complexity ratings relate each other, we providein table 4.5 the Pearson CC’s matrix of the models. AEH_NC and inv STDare the most correlated models with the average rating of the mean models,and they also evidence a strong correlation each other (i.e CC = 0.71).

Even if standard deviation is one of the most simple measure we con-sidered in this work, it is able to give very good results when describingthe user tapping performances. In view of this, we selected a new datasetof rhythms in order to have a better understanding of the correlation be-tween AEH_NC and inv STD models. The dataset is composed by the 1617unique combinations we obtained from the permutation method we describedin section 4.1.1, using the same experimental settings. Thus there are 66613-pulses rhythms and 951 14-pulses rhythm, and each rhythm is described

71

Figure 4.26: Rhythm perceptual and performance complexity judgments the user isrequired to answer after the tapping of the rhythm.

by a pattern of 6 onsets with minimum and maximum of IOI’s value in therange [1,5]. The CC value that results from the variance analysis of thetwo evaluation vectors is basically null (CC = 0.122), meaning that the twomodels understand rhythmic complexity in different both valid ways. Forthis reason, AEH_NC and inv STD models might be combined together forthe development of a new compound complexity estimation model.

Finally, we want to give a quick overview of the perceptual (perceval)and performance (perfeval) complexity (figure 4.26) judgments the subjectis asked to provide for each rhythm (figure 4.3).

These judgment vectors are build computing the mean ratings amongthe 10 sessions for each rhythm of the dataset. We can observe the resultsin table 4.6. The correlation between perceval and perfeval is very high(CC = 0.869), this means that users did not perceive a substantial dif-ference between the listening and the performance complexity. There is aquite strong negative correlation between AEH_NC model and performancecomplexity (C = −0.52), that is the result we expected, since an higherAEH_NC score implies a more simple cognitive structure of the rhythmicpattern and perceived performance complexity is rated with a lower value.On the other hand, perceptual complexity is not correlate with our com-putational model (some feedback from users let us believe that distinctionbetween perceptual and performance complexity may be confusing). Bothperfeval and perceval show high correlation with the mean tap and mdsmodels (also in this case coefficients are negative since the less the rhythm isjudged complex, the better is performed). As conclusion, we also check if anysignificant correlation exists between model evaluations and the number ofrhythm loop listened (listenloop) before starting to tap, however, since thereis no correlation both with AEH_NC model and perceptual/performancecomplexity judgments, it seems that users start tapping the rhythm as soonas they feel ready, without care about mistaking or not. For the same reason,

72

CC ta md pc pf ll antamd 0.86pc -0.43 -0.49pf -0.67 -0.69 0.869ll 0.12 0.07 0.196 -0.16an 0.77 0.68 -0.32 -0.52 0.36

Table 4.6: Perceptual correlation coefficients, studied with regard both to the numberof rhythm loop listened before starting tapping, to the evaluations of the AEH_NCmodel and to mean tap and mds rating models. We shortened the models notation inthe following way: ta = mean tap, md = mean mds, pc = perceval, pf = perfeval,ll = listenloop, an = AEH_NC. Gray highlighted columns refer to ground truth meanmodels. Both perceval and perfeval show high CC’s values with the mean models.AEH_NC model is a good predictor of the perceived performance complexity, while norelevant correlation is shown between AEH_NC model and perceptual complexity.

no correlation exists between listenloop and mean models.

73

74

Chapter 5

Conclusion and Future Works

In this work we provided a computational model for the automatic estimationof rhythmic complexity, with particular attention to the analysis of odd timesignature rhythmic patterns. The model is based both on mathematical andperceptual musical features extracted from a symbolic representation of therhythm.

At the beginning, we reviewed some preliminary studies aimed at under-standing what perceived musical complexity is and how it can be modeled.In this regard we mentioned subjective rhythmization, metrical accent andsyncopation. Then, we focused on more complex rhythm structures, verypopular in tribal world music and experimental Western genres, but ne-glected by research. We focused our research on how odd metric rhythmsare perceived as more complex with regard to the even ones, how peopleinternalize and perform them. We proposed a novel model (AEH_NC) inorder to answer these questions.

First model stage consists in adopting the clock model, originally pro-posed by Povel and Essens, to the processing of temporal patterns withgeneral length. It is a perceptual model that infers information on internalrepresentation of rhythmic sequences by first converting a rhythm into itscorresponding perceived accents notation, then computing a list of candi-date clocks whose structure fits the beat sequence induced by the rhythm tothe listener. We improve this fitting using a non-constant sampling of theaccents sequence, so that we are able to generate a proper list of best clocksthat works for odd time signature. Once clocks list has been generated,we exploit the structural properties of the resulting beat sequences to pre-dict them. Beat pulses are the metrical structure that define the rhythmicmeter, and we assume that the final beat pattern coincides with the mostinduced clock. In this respect, we implement a second model stage where

AEH analysis is applied: each beat sequence is evaluated measuring its de-gree of correlation with a list of almost even hierarchies. These are weightedhierarchical masks, build on the principle of maximally even distribution,that measure the geometrical distribution of onsets placement. As for themetrical hierarchies of Toussaint, we assume that a rhythm is simpler if itis assigned with a lower AEH value, because it means that rhythm inducessequence of beat which are asymmetrically distributed. At the end, we takethe maximum value among the AEH scorings assigned to beat patterns asthe final estimation of the inverse of complexity.

AEH_NC model is validated trough a Web-based experimental test,where we test the subject’s ability of tapping a pattern in synchronous withthe playing rhythm. Perceived and performance complexity judgments arealso required for each rhythm. Rhythms are drawn from a pattern genera-tion technique, from which we chose a dataset of 28 rhythms (12 14-pulsesrhythms, 16 13-pulses rhythms) through uniform distribution sampling. Cor-relation results show that our model is very good at describing the perfor-mance sessions of the users, STD and E measures also give good results, butthey are not always able to evaluate properly complexity, while AEH_NCmethod still distinguishes structural rhythmic features, even when rhythmswith different lengths are compared. The power of our model is in the re-moval of some assumptions about the nature of the the data. Finally, forwhat concern complexity judgments, we find that our model shows goodcorrelation with the performance complexity ratings, while there is no sig-nificant correlation with the perceptual complexity ratings.

In conclusion, we can summarize the main contributions of the work inthis thesis saying that for the first time an investigation of the perceptionof unconventional time signatures has been provided. We have been ableof giving an objective formalization of the observed phenomena, showingthat a pattern will be poorly reproduced or judged complex when the pulsesensation induced by the rhythm is given by a beat pattern that shows aweak symmetry property. Moreover, AEH_NC model allows the similaritycomparison between different families of rhythm, which differ in the dura-tional length. This feature suggests the possibility of taking existing rhythmsdataset and making a detailed analysis of the complexity measures result-ing from other computational models. In this work we have seen that asimple purely objective measure such as the inverse of STD shows an highcorrelation both with the experimental data and the AEH_NC complexityestimate. Therefore, besides testing the models against different datasets,we believe we also need more data to provide a more robust validation ofour model.

76

5.1 Future Works

Many scenarios might be interesting for the future developments of this work.First, more experimental test sessions would lead to a more robust percep-tual model. Here, for the first time in literature, a dataset of unconventionaltime signatures have been used, therefore validation of AEH_NC methodneeds to be tested against a large number of user performances. Since wehave implemented a Web-based experimental test, this task has been greatlysimplified, everyone indeed can easily join the test and having its perfor-mance integrated in our data analysis system. However, we believe that sinceevaluation measures are very sensitive to external conditions (listening envi-ronment and devices, listener focus and musical background, understandingof test rules), cross-validation of the test platform is advisable, in order toavoid spurious data. Moreover, a special focus should be given to the testof rhythms generated from a unique IOI’s set, where STD model evaluationcan’t give significant results, and study the accuracy of our AEH_NC model.

The main extension is inherent to the input domain: our complexityevaluation method could be used starting from the acoustic signal as thegiven input. In particular, onset detector might be used both for detectingonsets and weighting their acoustic intensity, replacing the set of rules weused in the NC-Clock model for defining accents.

Musicology would also benefit of the findings of this work. The studies fo-cused on the geographically and cultural provenience of families of rhythmicpatterns could adopt AEH_NC model in order to investigate their com-plexity similarities. Especially in the Asian and African cultures, odd timesignatures are part of the popular musical background, therefore our methodcould give important contributions to the future ethnological analyses. An-other interesting feature used in genres such jazz and African tribal music isthe polyrhythm, which is the simultaneous use of two or more metrical con-flicting rhythms. It would be interesting extending the validity of AEH_NCmodel to the complexity estimation of polyrhythmic structures, where singlepatterns are combined together in order to generate a new level of complexity.

77

78

Bibliography

[1] Godfried T Toussaint. A comparison of rhythmic similarity measures.In ISMIR, 2004.

[2] Godfried T Toussaint. The rhythm that conquered the world: Whatmakes a “good” rhythm good? Percussive Notes, 2:52, 2011.

[3] Richard Krantz, Jack Douthett, John Clough, et al. Maximally evensets. In Bridges: Mathematical Connections in Art, Music, and Science,pages 193–200. Bridges Conference, 2000.

[4] Peter Desain and Luke Windsor. Rhythm perception and production.swets & Zeitlinger Lisse, The Netherlands:, 2000.

[5] Daniel E Berlyne. Aesthetics and psychobiology, volume 336. JSTOR,1971.

[6] Paul C Vitz. Preferences for rates of information presented by sequencesof tones. Journal of Experimental Psychology, 68(2):176, 1964.

[7] Peter Vuust and Maria AG Witek. Rhythmic complexity and predictivecoding: a novel approach to modeling rhythm and meter perception inmusic. Frontiers in psychology, 5:1111, 2014.

[8] Peter Vuust and Andreas Roepstorff. Listen up! polyrhythms in brainand music. Cognitive Semiotics, 3(fall2008):134–158, 2008.

[9] Ilya Shmulevich and Dirk-Jan Povel. Complexity measures of musicalrhythms. Rhythm perception and production, pages 239–244, 2000.

[10] Sebastian Streich. Music complexity: a multi-faceted description of au-dio content. PhD thesis, Citeseer, 2006.

[11] Jeffrey Pressing. Cognitive complexity and the structure of musical pat-terns. In Proceedings of the 4th Conference of the Australasian CognitiveScience Society, 1999.

79

[12] Seth Lloyd. Measures of complexity: a nonexhaustive list. IEEE ControlSystems Magazine, 21(4):7–8, 2001.

[13] Eric Thul and Godfried T Toussaint. Rhythm complexity measures: Acomparison of mathematical models of human perception and perfor-mance. In ISMIR, pages 663–668, 2008.

[14] Erik D Demaine, Francisco Gomez-Martin, Henk Meijer, David Rap-paport, Perouz Taslakian, Godfried T Toussaint, Terry Winograd, andDavid R Wood. The distance geometry of music. Computational geom-etry, 42(5):429–454, 2009.

[15] Richard Parncutt. A perceptual model of pulse salience and metri-cal accent in musical rhythms. Music Perception: An InterdisciplinaryJournal, 11(4):409–464, 1994.

[16] Justin London. Hearing in time: Psychological aspects of musical meter.Oxford University Press, 2012.

[17] Justin London. Some examples of complex meters and their implicationsfor models of metric perception. Music Perception: An InterdisciplinaryJournal, 13(1):59–77, 1995.

[18] Henkjan Honing. Structure and interpretation of rhythm in music. Psy-chology of music, pages 369–404, 2013.

[19] Bruno Di Giorgi, Massimiliano Zanoni, Augusto Sarti, and StefanoTubaro. Automatic chord recognition based on the probabilistic model-ing of diatonic modal harmony. In Multidimensional Systems (nDS),2013. Proceedings of the 8th International Workshop on, pages 1–6.VDE, 2013.

[20] Harald Krebs. Some extensions of the concepts of metrical consonanceand dissonance. Journal of Music Theory, 31(1):99–120, 1987.

[21] Fred Lerdahl and Ray Jackendoff. A generative theory of tonal music.1987.

[22] Grosvenor Cooper and Leonard B Meyer. The rhythmic structure ofmusic, volume 118. University of Chicago Press, 1963.

[23] W Tecumseh Fitch and Andrew J Rosenfeld. Perception and productionof syncopated rhythms. Music Perception: An Interdisciplinary Journal,25(1):43–58, 2007.

80

[24] Don Michael Randel. The Harvard dictionary of music, volume 16.Harvard University Press, 2003.

[25] George Sioros and Carlos Guedes. Syncopation as transformation. InInternational Symposium on Computer Music Modeling and Retrieval,pages 635–658. Springer, 2013.

[26] John A Sloboda. Music structure and emotional response. Psychologyof music, 19:110–120, 1991.

[27] Stephen Handel. Listening: An introduction to the perception of audi-tory events. The MIT Press, 1993.

[28] Leonard B Meyer. Emotion and meaning in music. University of chicagoPress, 2008.

[29] George Sioros, Marius Miron, Diogo Cocharro, Carlos Guedes, and Fa-bien Gouyon. Syncopalooza: manipulating the syncopation in rhythmicperformances. In Proceedings of the 10th International Symposium onComputer Music Multidisciplinary Research, pages 454–469. Citeseer,2013.

[30] George Sioros, André Holzapfel, Carlos Guedes, et al. On measuringsyncopation to drive an interactive music system. In ISMIR, pages283–288, 2012.

[31] George Sioros and Carlos Guedes. Automatic rhythmic performance inmax/msp: the kin. rhythmicator. In NIME, pages 88–91, 2011.

[32] Robert Mitchell Parry. Musical complexity and top 40 chart perfor-mance. 2004.

[33] O Ladinig and H Honing. Complexity judgments as a measure of eventsalience: Are there indeed differences between musicians and nonmusi-cians. Music Perception, x, xx, 2008.

[34] Chunyang Song, Andrew JR Simpson, Christopher A Harte, Marcus TPearce, and Mark B Sandler. Syncopation and the score. PloS one,8(9):e74692, 2013.

[35] Ilya Shmulevich and D-J Povel. Rhythm complexity measures for mu-sic pattern recognition. In Multimedia Signal Processing, 1998 IEEESecond Workshop on, pages 167–172. IEEE, 1998.

81

[36] Evelyn Lamb. Uncommon time: What makes dave brubeck’s unortho-dox jazz stylings so appealing?, 2012.

[37] Mari Riess Jones. Learning and the development of expectancies: Aninteractionist approach. Psychomusicology: A Journal of Research inMusic Cognition, 9(2):193, 1990.

[38] Mohammad Tanvir Irfan, Md Akhtaruzzaman, Md Saiful Islam, andMd Mahabubul Alam. Mathematical representation and analysis ofrhythms from various regions. In 2007 International Conference onInformation and Communication Technology, pages 13–16. IEEE, 2007.

[39] Godfried T Toussaint et al. A mathematical analysis of african, brazil-ian, and cuban clave rhythms. In Proceedings of BRIDGES: Mathemat-ical Connections in Art, Music and Science, pages 157–168, 2002.

[40] Neil McLachlan. A spatial theory of rhythmic resolution. LeonardoMusic Journal, 10:61–67, 2000.

[41] Godfried T Toussaint et al. The euclidean algorithm generates tra-ditional musical rhythms. In Proceedings of BRIDGES: MathematicalConnections in Art, Music and Science, pages 47–56, 2005.

[42] Dirk-Jan Povel and Peter Essens. Perception of temporal patterns. Mu-sic Perception: An Interdisciplinary Journal, 2(4):411–440, 1985.

[43] Francisco Gómez, Eric Thul, and Godfried T Toussaint. An experimen-tal comparison of formal measures of rhythmic syncopation. In Proceed-ings of the International Computer Music Conference, pages 101–104,2007.

[44] Ilya Shmulevich and D-J Povel. Measures of temporal pattern complex-ity. Journal of New Music Research, 29(1):61–69, 2000.

[45] Andranick S Tanguiane. Music recognition based on artificial perceptionapproach 1. Journal of New Music Research, 22(2):133–149, 1993.

[46] AS Tanguiane. A principle of correlativity of perception and its ap-plication to music recognition. Music Perception: An InterdisciplinaryJournal, 11(4):465–502, 1994.

[47] Godfried T Toussaint et al. Classification and phylogenetic analysis ofafrican ternary rhythm timelines. In Proceedings of BRIDGES: Mathe-matical Connections in Art, Music and Science, pages 25–36, 2003.

82

5.1. Future Works 83

[48] Miguel Dıaz-Banez, Giovanna Farigu, Francisco Gómez, David Rappa-port, and Godfried T Toussaint. El compás flamenco: a phylogeneticanalysis. In Proceedings of BRIDGES: Mathematical Connections inArt, Music and Science, pages 61–70, 2004.

[49] Francisco Gómez, Andrew Melvin, David Rappaport, Godfried T Tou-ssaint, et al. Mathematical measures of syncopation. In BRIDGES:Mathematical Connections in Art, Music and Science, pages 73–84,2005.

[50] Bernard W Silverman. Density estimation for statistics and data anal-ysis, volume 26. CRC press, 1986.

[51] Simha Arom, Martin Thom, Barbara Tuckett, and Raymond Boyd.African polyphony and polyrhythm: musical structure and methodology.Cambridge university press, 2004.

[52] Dirk-Jan Povel. A theoretical framework for rhythm perception. Psy-chological research, 45(4):315–337, 1984.

[53] James J Gibson. The ecological approach to visual perception: classicedition. Psychology Press, 2014.

[54] Peter Essens. Structuring temporal sequences: Comparison of modelsand factors of complexity. Perception & Psychophysics, 57(4):519–532,1995.

[55] Dirk-Jan Povel and Hans Okkerman. Accents in equitone sequences.Perception & Psychophysics, 30(6):565–572, 1981.

[56] H Christopher Longuet-Higgins and Christopher S Lee. The perceptionof musical rhythms. Perception, 11(2):115–128, 1982.