Algorithmic Music Composition Using Linear Algebra

By Serhat Yelkenci, Bachelor of Science

A Thesis Submitted in Partial

Fulfillment of the Requirements
for the Degree of
Master of Science
in the field Industrial Engineering

Advisory Committee:

Xin Chen, Ph.D., Chair

Sohyung Cho, Ph.D.

Hoo Sang Ko, Ph.D.

Graduate School
Southern Illinois University Edwardsville
May 2017




ProQuest Number: 10275073




All rights reserved

INFORMATION TO ALL USERS
The quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.






ProQuest 10275073

Published by ProQuest LLC (2017 ). Copyright of the Dissertation is held by the Author.


All rights reserved.
This work is protected against unauthorized copying under Title 17, United States Code
Microform Edition © ProQuest LLC.


ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
 ABSTRACT

ALGORITHMIC MUSIC COMPOSITION USING LINEAR ALGEBRA

by

Serhat Yelkenci

Chairperson: Xin Chen

Sound, in its all forms, is a source of energy whose capabilities humankind is not yet fully aware of.

Composition – the way of aggregating sounds into the form of music – still holds to be an unperceived

methodology with many unknowns. Methodologies used by composers are generally seen as being

innate talent that cannot be used or shared by others. Yet, as any other form of art, music actually is

and can be interpreted with mathematics and geometry. The focus of this thesis is to propose a

generative algorithm to compose structured music pieces using linear algebra as the mathematical

language for the representation of music. By implementing linear algebra, a practical data structure is

obtained for analysis and manipulation of music. Instead of defining a single structure from a certain

musical canon, which limits the frame of music, the generative algorithm proposed in this paper is

capable of learning all kinds of musical structures through linear algebra operations. The algorithm is

designed to build musical knowledge (influence) by analyzing music pieces and initiating from a new

melody as an inspirational component to produce new unique and meaningful music pieces.

Characteristic analysis features obtained from analyzing music pieces serve as constraints during the

composition process. The proposed algorithm has been successful in generating unique and meaningful

music pieces. The process time of the algorithm varies depending on the complexity of the musical

structure it has been taught. Yet, the free nature of the generative algorithm and the capability of

matrical representation offer a practical linkage between unique and meaningful music creation and

any other mathematical foundation.

ii
 ACKNOWLEDGEMENTS

I am very much thankful to my thesis advisor Dr. Xin Chen who always shared his wisdom and
supported me in my explorative intensions. I would sincerely like to thank Dean Dr. Cem Karacal who
believed in me against all the odds and gave me a chance. I would like to thank Dr. Sohyung Cho for
being supportive no matter what and his efforts for the faculty of Industrial Engineering.
Lastly, I want to express my gratitude to my mother who raised me to be free and my father, who started
my interest in music.

iii
 TABLE OF CONTENTS
ABSTRACT........................................................................................................................................... ii
ACKNOWLEDGEMENTS .................................................................................................................. iii
LIST OF FIGURES ................................................................................................................................v
LIST OF TABLES ................................................................................................................................ vi
1.INTRODUCTION ...............................................................................................................................1
Music Composition & Mathematics .......................................................................................... 1
Purpose of Study...................................................................................................................... 2
The Nature of the Generative Algorithm .................................................................................... 2
2.LITERATURE REVIEW ....................................................................................................................5
3.MATHEMATICAL REPRESENTATION OF MUSIC ......................................................................9
Melody Matrices ...................................................................................................................... 9
Note vector ....................................................................................................................................10
Duration vector (Rhythmic aspect)..............................................................................................11
Power vector .................................................................................................................................12
Combining vectors into a melody matrix.....................................................................................12
The Characteristic Analysis Features on Melody Matrices ........................................................ 13
Cosine similarity...........................................................................................................................14
Duration binary vector .................................................................................................................16
Scale value vector .........................................................................................................................16
Complexity coefficients ................................................................................................................17
Behavioral analysis vectors .........................................................................................................18
Relationship of the characteristic analysis features ...................................................................19
4.GENERATIVE ALGORITHM .........................................................................................................22
Generation of the Core ........................................................................................................... 25
Transition Matrix ................................................................................................................... 29
Generating New Melodies Using the Transition Matrix & Core Melodies .................................. 33
5.RESULTS OF THE STUDY .............................................................................................................38
6.CONCLUSION, DISCUSSION & FUTURE RESEARCH ..............................................................42
Conclusion ............................................................................................................................ 42
Discussion & Ideas about Future Research .............................................................................. 43
“Good” Music ....................................................................................................................... 45
REFERENCES .....................................................................................................................................47
APPENDICES ......................................................................................................................................50
A. Music-to-Matrices Transformation ......................................................................................50
B. Matlab Scripts for MIDICSV Language to Matrical ...........................................................52
C. Python Scripts for MIDI to EXCEL ....................................................................................57
D. Matlab Code for Matrical Language to CSV Language ......................................................58
E. Python Scripts for MIDICSV Language to MIDI................................................................61

iv
 LIST OF FIGURES

1. Classical Representation of Music (Turkish March By W.A. Mozart) ………....................…9

2. Matrical Representation of a Melody From the Music Piece “Lightly Raw”…………..…...12
3. Four Consecutive Melodies from the Music Piece “Lightly Raw””...………………..……..13
4. Notes vs Cumulative Time…………………………………………………………….…….14
5. Continuous Graph for Notes……………………………………………….……...................15
6. Labels for Similarity Measures………………………………………….…………………...15
7. Behavioral Analysis Vectors for the Melodies from “Lightly Raw”……………………..….18
8. Hierarchical Relationship of Characteristic Analysis Features……………………………...20
9. Partial (Two-Three Columns) Similarity Between two Melodies……..………….…........…21
10. Relationships Between Melodies…………………………………….…………………...….23
11. Projection of the Music Generation within the Generative Algorithm……….…………...…24
12. Generation Road Map……………………………………………………..........................…25
13. Illustration of a Music Piece Before the Analysis………………………...............................29
14. Matrical Structure of the Music Piece Shown in Figure 11……………….………...….……30
15. Characteristic Analysis Features Recorded from the Four Melodies……….……………….30
16. Similarity Vectors to be Placed in Transition Matrix…………………….………….…….…31
17. The Transition Matrix Containing Similarity Vectors for a Single Instrument………………32
18. Symbolic Transition Matrix………………………………………..………………………...32
19. Loop Regarding the Duration Vector ……………………………………….…….…………34
20. Loop Regarding the Power Vector…………………………………………….…………..…36
21. Loop Regarding the Note Vector………………………………………………………….…37
22. Generated Matrices After the Algorithm Finished Operating……………….…………..…..38
23. Conventional Representation of a Generated Music Piece and Core Melody…………….….41
24. Conventional Representation of a Generated Music Piece and Core Melody…….……..…..42
25. Output Music Piece with Process Time 382 Seconds (6.37 Min)……………………...........40
26. Illustration of the Dimensions of Analysis…………………………………………………..46
27. Representation of MIDICSV Language ………………………………………….……….…52
28. MIDICSV into Matrical Representation By MIDI2MY……………………….………….....53
29. Matrical Representation Into MIDICSV Language By MY2MIDI………………………….53

v
 LIST OF TABLES

1. Summary of the Algorithmic Composition Methodologies…………………………..……….8

2. Actions taken regarding the Combination of Constraints for the Duration Vector……….….35
3. Actions taken regarding the Combination of Constraints for the Power Vector……….…….36
4. Actions taken regarding the Combination of Constraints for the Note Vector…....................37

vi
 1

CHAPTER I

INTRODUCTION

Music Composition & Mathematics

Music known to be a major form of art, defined as a “meaningful combination” of sounds

(Godøy & Leman, 2011). These combinations are produced by the engineers of sounds, known to be

the composers, who built many pieces throughout the history, same as another engineer built systems

in their own discipline. Although being very good instrument players, they were capable of structuring

what they can play and create unique representations using their imagination. Even now, so many

people can play instruments but so few are able to compose. It is because composers are strong

mathematicians and analysts with a musical talent (Shah, 2010). Yet, when it is discussed how

composers do compose, their methodology is regarded as natural talent, which made it hard to be

mastered by all. In fact, composers continuously built a form of influential methodology (musical

knowledge/taste) for themselves, which led them to use this knowledge during the compositions of their

music pieces. Certain rhythmic patterns or harmonies became prominent in their music pieces, defined

as style, due to the significance they assign to them in their musical knowledge (Janssen et al., 2013).

It can be observed that their music pieces bear significant mathematical patterns or geometries

underneath that reveals their characteristics (Xenakis, 1971). This leads to the idea “that structuring

sounds into each other to form music” is a cultivation of mathematics. It is the hidden mathematical

structures that we save in our minds, eventually makes us remember a melody, feeling a groove and so

on (Burgoyne et al, 2013). Music genres can be distinguished from each other from the different

patterns they include in their mathematical behavior (Klapuri, 2010). While a reggae song mostly

repeats the guitar chord all over the music piece, a progressive metal song uses chords in much variety.

The variety of patterns indicate the existence of unlimited composing methodologies.

 2

Purpose of Study

The main purpose of this study is to build a practical system to generate/compose unique and

meaningful music. By stating unique and meaningful, an algorithm that seeks novelty while harboring

sense. Yet, in order to be able to generate music, first was to analyze and understand the mathematics

underneath music. To achieve this goal, the initial step was to find an appropriate mathematical

representation of music. A mathematical language, that can represent the conventional musical notation

in computers, suitable for mathematical operations and analysis, found to be linear algebra, which

concerned with features of vector spaces. When sound is considered as a form of energy, vector spaces

offered practical representation of its characteristics (Xenakis, 1971). Vectors and matrices are highly

capable of representing melodies and thus music pieces, when aggregated to each other. Similarities

can be measured, manipulations can be made and extended analysis can be held. Thus, the scientific

framework in this research is mainly linear algebra and its extensions. Using vectors and matrices as

the mathematical language for music representation enabled the creation of a generative algorithm by

manipulating these matrices and vectors in a meaningful manner, which is actually composing music

pieces. Instead of seeking a single structure or a methodology of composing, and since using linear

algebra with an extended mathematical capability, a methodology that can learn any musical structure

became the goal of this research. Methodology would be adapted to any genre, even a genre that is yet

unknown.

The Nature of the Generative Algorithm

The balance held between “creativity” and the “hidden customs” associated with a music genre

(funky, depressed, inspirational etc.), or any other form of art, is currently held to be the resort of

determining the method of generating music. Simply, while an art form is intended to be unique, it has

to follow several rules to be part of a genre or sound “familiar” in listener’s mind. It is fair to point out

that the preference of a genre over another is simply caused from the different mathematical customs

associated with them. Composers are the individuals who are aware of this fact and they shape their
 3

composition techniques accordingly. Taking into consideration that composers have a strong musical

perspective in their mind, these “hidden customs” evolve in their minds before generating music. To

illustrate the point, one would agree with the fact that a painter would not be able to draw a tree without

knowing what a tree looked like (Hoff, 2016). Likewise, a classical music composer could not compose

a classical piece without knowing how it sounds like. These ideas lead to the emergence of pre-installed

knowledge/perspective if one needs to create/generate. Yet, influential methodology needs an

inspirational aspect to possess a character. This inspirational aspect can be defined as the notion of

creativity, which is in accordance with the fact that a composer would start building its pieces inspired

from a melody he or she just created. Then other instruments would come to subject and other measures

would be created, yet still influenced by the first melody composer created randomly. This is taught to

be the part how creativity affects the composition. Thus, the music composition methodology offered

in this research can be illustrated as;

The input “x”, a randomly generated melody, which can be stated as the creativity feature

(inspiration) while the generative function “f”, can be stated as the musical knowledge of a composer

(influence). The generative function is created by analyzing and influencing a music piece, meaning

assimilation of the music structure such as the similarities between melodies, number of

measures/instruments, repetitions of certain parts. The composition knowledge obtained is saved in

binary vectors in order to route the generative algorithm to process a given new input. It will process

the randomly generated melody, in a manner that the output would be a unique piece due to the

inspirational aspect, the input, which is created without referencing anything. The generative function

is consisted of binary similarity vectors, which are information/knowledge gathered from a music piece.

These information vectors, which represent the relationship between given two melodies from a music
 4

piece, later to be defined within a transition matrix, can be classified such as; repetition of a melody,

similar melodies (more complex, simplistic, etc.) or completely different melody. These vectors will

serve as constraints during the composition process of new melodies, and the output would be a piece

that has meaningful interrelations within its instruments/melodies. The methodology is proposed by

taking human composers as a reference more than referencing any other methodology. As will be

mentioned in the literature review, many other methodologies use either Markovian Stochastic Models

or Artificial Neural Networks where each has characteristics that makes their methodology farther from

a human composer where this study stands. By defining linear algebra as the representation of music,

generative algorithm is created in the manner of matrix manipulation, which makes it a thoroughgoing

analysis from bottom to top.

Chapter 2 introduces research and work conducted in algorithmic music composition. In

Chapter 3, matrical representation of music will be introduced alongside the characteristic analysis

methods on the matrices. Chapter 4 will cover the structure and the application of generative algorithm.

In Chapter 5, results of the experiments conducted with the generative algorithm will be introduced.

Chapter 6 will conclude the research; provide discussion points and future recommendations for

interested researchers.
 5

CHAPTER II

LITERATURE REVIEW

The interpretation of music with mathematics dates back to Pythagoras (around 500 B.C.), who

is believed to be the pioneer of this linkage. He pointed out the concept of Musica Universalis (Music

of the Spheres) by interpreting the proportional movement of the spheres with mathematics, which

evolved into a mathematical understanding of sounds evolving into music during the Renaissance

period (Papadopoulos & Wiggins, 1999). With the developments in computer science in recent history,

algorithmic composition has been investigated by many. Cope described algorithmic composition as “a

sequence (set) of rules (instructions, operations) for solving (accomplishing) a [particular] problem

(task) [in a finite number of steps] of combining musical parts (things, elements) into a whole

(composition)” (Cope, 1992). The difference between research projects, held in the framework of

algorithmic composition, appears in the mathematical concepts used within methodologies.

Probabilistic models served as one of the first types of algorithmic composition by using

random events within the music composition (Stochastic Modeling). Prominent examples of stochastic

algorithms were Markov chains and the use of Gaussian distributions. Hiller and Isaacson were one of

the first who used a sequential computational model using random number generators and Markov

chains for algorithmic composition. They programmed the ILLIAC (Illinois Automatic Computer) to

compose music by generating pitches using random numbers. Via using probabilities through Markov

Chain, they used the past to determine the probabilities of future. They tested them according to the

rules of classical counterpoint where a pitch may be replaced by another if it does not fit in the customs

(Hiller & Isaacson, 1959). Following Hiller and Isaacson (1959), Xenakis proposed a Markovian

stochastic approach in composition for determining selection of a succeeding note (Xenakis, 1971).

Due to his interest in architecture, he focused on visualizing sounds in three-dimensional spaces. He

interpreted the characteristics of sounds and melodies due to their shapes in these spaces. “Cybernetic

Composer” was invented by Ames and Domino to compose pieces in different genres using a slightly
 6

different technique then others, which deduced the rhythm of the melody using Markov chains, and

then choose the pitches accordingly (Ames, Domino, 1991). The performance of mathematical models

however, left limited due to subjectivity during the determination of probabilities and the density of

effort needed when complexity of the music tends to be higher.

Another type of algorithmic composition is defined as the knowledge based systems (KBS).

Knowledge Based Systems exceled an excessive usage of constraints and pre-defined rules that lets the

algorithm pretend as it is an educated musician. An advantage was stated as they had reasoning and

explanative choice of actions (Papadopoulos & Wiggins 1999). A prominent example is Ebcioglu who

invented a backtracking Specification Language (BSL) to implement a rule-based system for

constructing harmonization in the style of J.S. Bach. Harmonies, the interrelation between notes that

sounds “pleasant” is used as a constraint that has to be satisfied in the algorithm (Ebcioglu, 1988;

Balaban & Ebcioglu, 1992). Other examples of KBS was put forward by making intention-based music.

Instead of using randomness to affiliate creativity, potential actions were defined at the initiation stage

of the composition (Ramalho & Ganascia, 1994; Robertson et al., 1998; Zimermann, 1998)

Construction and application of KBS’s ended up being exhaustive due to complex nature of music.

Although they try to program the notion creativity by pre-determined possibilities of initiation, it

ideologically fails due to lack of randomness. Manipulating the rules become difficult when level of

complexity increases as it is in Markovian stochastic models.

Recently, the most popular type of algorithmic composition in scientific environment is defined

as “Systems Which Learn” (Papadopoulos & Wiggins 1999). Unlike the KBS or Markovian Models

where the programming of knowledge plays the biggest part, systems that learn consist of flexible

algorithms that can absorb “what” music is from music piece(s). Framework includes Artificial Neural

Networks (ANN) and Machine Learning (ML), which have been introduced for musical composition

by many. Todd and Loy, Griffith and Todd are one of the first ones who proposed the usage of ANN’s

(Todd & Loy, 1991; Griffith & Todd, 1999). Todd used a feed-forward ANN with feedback for melody

generation. Mozer used a recurrent auto-predictive connectionist network to excel note-by-note

 7

prediction for composition (Mozer, 1994). Most recent trend in mathematical music composition

gathers around using recurrent neural networks for composition. While it was put forward in 1997, a

specific type of recurrent neural network’s called LSTM (Long-Short Term Memory) became famous

in 2015, due to its high performance as a sequential learner. The advantage of LSTM is stated as the

capability of choosing to remember or forget memory by containing different input, output and memory

gates (Karpathy, 2015). Currently there are many programs, which use LSTM as the processing tool

for algorithmic composition. Some can be stated as Google’s open source deep learning music project

Magenta, DeepJazz, BachBot, FlowMachines (uses Markov constraints as a neural network technique),

WaveNet (uses Convolutional Neural Networks which is widely used in image classification and

generation) and GRUV (Brinkkemper, 2016). Although ANN’s are highly good learners, there are

open-ended discussions such as choosing number of training, number of hidden layers or the

architecture of the cell memories, which make it harder to generalize the performance of the

methodology. Outcome is unpredictable when any manipulation is done in the structure of the model,

due to artificial neural network’s hidden characteristics.

Table 1: Summary of the Algorithmic Composition Methodologies

Papadopoulos and Wiggins state the need of “multiple, flexible, dynamic, even expandable

representations” due to the need of a more explicit interpretation of the characteristic of “human”

behavior in music. They point out the need of an essential way of musical cognition, from perception
 8

to composition and recommend a system “which absorbs new knowledge (being able to combine

different styles, showing creativity)” (Papadopoulos & Wiggins 1999). The algorithmic composition

method proposed in this paper is developed by adapting the good sides and trying to dispose of the

weaknesses of the methodologies mentioned above.

 9

CHAPTER III

MATHEMATICAL REPRESENTATION OF MUSIC

Melody Matrices

In order to analyze and manipulate the mechanics beneath music, the first step was to find a way

to represent the behavior in a mathematical way. Instead of classical representation of music (A, A#,

B, C, etc.), mathematical representation/notations supposed to provide extensive means for analysis

and modification. In this project, matrical representation (vectors and matrices) is suggested to be the

base of musical analysis and algorithmic composition. Vectors and matrices, the base of linear algebra,

promises a broad opportunity of research and analysis by their nature.

As the inception of the ideology, a melody was considered as a significant and repetitive part

of a music piece, which evolves into a piece by aggregating with other melodies. A melody can be

defined as the common stage or meeting point where deeper features (notes) and shallower features

(music piece) can be interpreted. One can analyze a melody and find the notes and rhythm, or can

combine melodies into a music piece. This can be referenced by the incidents where people sing or

whistle a melody when they want to express a song. A melody may be supported by other melodies

from different instruments. In the figure 1 below, each melody is separated by bars. Thus, these bars

may be stated as the junction point for analysis between smaller aspects as notes/rhythm and bigger

aspects as traffic/structure of the piece. That makes a melody matrix very crucial aspect in terms of

being a link. Due to this perspective, melodies are represented by matrices.

 10

Figure 1: Classical Representation (Turkish March by W.A. Mozart) (Neumann, 2010)

A melody is a combination of features that builds its characteristic. These features are

considered as notes in a given piece, duration value of each note from its inception to wither and power

(physical appreciation) of a note, which creates a three-dimensional space for a melody. This three

dimensional space has supposed to bear the characteristics of music as a whole with no other feature

needed. These components were considered as mathematical vectors within the melody matrix. A

melody is a combination of vectors, eventually evolving as a 3xN matrix where 3 represents note,

duration and power vectors and N represents the rhythm key of the piece. Rhythm key, as it is in the

classical music theory, represents the rhythmic aspect of a melody, the number of elements in a row in

matrical representation. It is a crucial aspect, which puts each matrix on a common ground.

Note vector

Notes are actually defined to be special sounds in a single pitch that produce unique

signals/vibrations for the receiver. When one goes up or down in pitch, one can find the same note with

a different signal but same perception as its higher/lower note. Some instruments provide seven notes

in a single pitch while some provide twelve or more notes. In Anatolian music culture, number of notes

in a pitch range is even higher than the western music style. Actually, in terms of a scientific

perspective, you can divide a finite line into infinite number of pieces, which would be very small, yet

different. Meaning, a pitch range may have infinite number of notes within. The important fact in
 11

defining specific notes, is the perception capability (vibration extend) of the listener. Since the focus is

western audience for this research, as an example we can consider twelve-note pitch range, which are

the base of western music. {C, C#, D, D#, E, F, F#, G, G#, A, A#, B} In mathematical language, they

can be represented with numbers such as {0,1,2,3,4,5,6,7,8,9,10,11…}, as increasing in the pitch.

Scales are defined to be the note directory of a melody associated with different styles of music. Scales

may be used to define different note libraries or elements for note selection purposes. An example; {G,

D, E, A}, meaning a melody with the given notes, the note vector can be defined as [7, 2, 4, 9], in a C

key where C to be selected as 0 or 12. Even the note vectors may contain column vectors within,

indicating more than one note at a given time. A chord is represented in a column vector within the

note vector.

Note Vector: - Rhythm Key: 4

Duration vector (Rhythmic aspect)

Each note in a melody has its own duration, which narrates how a specific note will last before

passing to the next note. In classical musical theory, these durations are defined as a whole, half, quarter,

eighth, sixteenth etc. When combined these durations create a measure and combined measures create

a music piece. With that perspective, durations can also be represented by vectors such as [1/32, 1/16,

1/8, 1/7, 1/6, 1/5, 1/4, 1/3, 1/2, 1]. Melodies that attached together in a piece must have the common

ground where modifications and analysis can be done, emerging the need of an equal base. It can be

said that each melody in a piece take the same time with others. Therefore, it can be considered that

each melody is a single bar and the note durations within the measure is divided into what rhythm that

melody follows. If there are only three notes with equal duration in a melody, each have 1/3 duration

of that melody. Therefore, in a duration vector, it is determined that an element would not have a

duration value which can exceed 1 when summed with its current duration. Duration vector examples

below have rhythm keys 4, 6 and 3 consecutively:

 12

If there are more than one note at a given time, meaning multiple durations at a given time, the

constraint of one should still be satisfied by looking at its position. It is fair to see that a duration value

in the vector cannot take value more one where a duration value of one indicates a whole note. This

constraint will be satisfied within the generative algorithm. The important aspect of how many

elements/cells are there in the duration vector is associated with the rhythm key of the given melody,

as defined earlier. If the duration vector has different elements, the least common multiple becomes the

rhythm key. For the examples above, the keys are; four, six, and three respectively, indicating the

number of elements in each vector.

Power vector

The last dimension (3rd row) of a melody matrix that contains power value of each note exists in

the melody. The physical identity of a note, which determines the magnitude of the perception for the

listener (strong hit, smooth hit, reverb, vibrato, silence and so on). The values within the power vector

can be determined on a range from [0-1] with random values or with values which can be derived from

functions y=f(x). (E.g. elements derived from y=sin(x) function would provide vibrato sound,

imagining its behavior between [0-1]).

Power Vector:

Combining vectors into a melody matrix

With the definition of its components (notes, durations and power), melody can be represented

as a 3-row matrix, as it can be seen in Figure 2.

 First Row: Notes

 Second Row: Duration

 Third Row: Power

Figure 2: Matrical Representation of a Melody a Melody Matrix

 13

Each column in a matrix represents a possible cell for a note to exist, including its characteristic

features (note, duration and power). (Column 1: A quarter (1/4) note A (12) with a physical magnitude

of 1). Number of columns in the melody matrix is determined by the rhythm key. The above matrix can

be emphasized as such; a melody, with the notes in the sequence of {A, F, F, F} with durations (two

quarter and one half note) and each having the same power of physical representation with a rhythm

key 4 (Time Signature: 4/4).

The Characteristic Analysis Features on Melody Matrices

To conceive intelligence from music pieces and use them during the generation process,

characteristic analysis features were defined, each focusing on a characteristic of melodies. There are

used to identify and analyze possible structures (relationships) between melodies. These features are

actually vectors and matrices representing a characteristic or relation between melodies. A well-known

music piece “Lightly Raw” was used in the experiments for obtaining these characteristic analysis

features. Matrical representations of the first four melodies can be seen in Figure 3 below. As defined

earlier, each bar was transformed into matrices containing note, duration and power vectors within.

MATLAB was used to conduct operations on the matrices. Melodies in the piece is structured in a 4/4

time signature, which indicates the number of columns for each melody as four. After defining each

melody as 3x4 matrices, all of the matrices were stored in an array list for easiness in operations. Several

methods were used to conceive intelligence on the relationships within and between melodies.

Figure 3: Four Consecutive Melodies from the Music Piece “Lightly Raw”
 14

Cosine similarity

As a common method, cosine measure is used to analyze similarities in linear algebra

operations. The product of each melody, divided by their norms, provided the cosine value between [0-

1], indicating the similarity measure between given two melodies. By obtaining cosine similarity

between melodies, information is gained about the structure of the music piece, as if a part always starts

with a similar melody or a melody comes after a certain type of melody with a high frequency. The

advantage of dot product is that it provides each relationship between rows of each melody, not only

the whole melody.

Figure 4: Discrete Graph for Notes

(Melody 1: Blue Line with Circles, Melody 2: Turquoise Line with Stars)

The strong similarities between note-note, duration-duration and power-power are stored by

determining boundaries between [0-1]. The results indicate high and low similarities between melodies

yet; the numerical values between [0-1] are hard to interpret regarding the reason of similarity due to

the number of dimensions, except “1” which indicates an identical melody. This evolves the emergence

of alternative characteristic analysis for the given melodies. (Example: 0.9997 is the value indicating
 15

the similarity between two melodies represented in Figure 4. Though it indicates a strong similarity,

alternative detailed analysis is needed to observe the cause of similarity). Their derivative behavior can

be seen in Figure 5.

Figure 5: Continuous Graph for Notes

(Melody 1: Blue Line with Circles, Melody 2: Turquoise Line with Stars)

Visualizing the behavior of a melody emerged from the need of observing the reason of

similarity. The visualization puts forward the idea whether melodies can be formed by a certain

equation providing such structure depending on the time. Possible relationships between melodies

would be interpreted through this structure. The idea of creating a transition matrix came up from this

base of perspective. To obtain such a structure, one is able to analyze a music piece and determine the

possible patterns. Thus, transition matrices (described in Section 4) are formed out of music pieces to

serve as a course of action for the generative algorithm. A given melody would have the type of

relationships with other melodies as represented in Figure 6.

 16

Figure 6: Labels for Similarity Measures

Duration binary vector

To have a deeper analysis regarding the duration values, Duration Binary Vector was defined

which includes values of “1” and “0” indicating if there is a start of note hit at the given point. This

characteristic feature is the strongest indicator of the rhythmic aspect of the melody. It shows how note

hits are placed within the possible locations within the duration vector.

Two Melodies:

Duration Binary Vector:

Scale value vector

Scale Value Vector is another characteristic analysis feature that stores the information about

which notes from scale library are used in the melody. This analysis takes scale library and notes used

in the melody into consideration. Given a scale library defined at the beginning of the generation as a

vector containing all the possible notes that melody can include, some or all notes may be placed in the

melody. Scale value vector is a binary vector in the size of the scale library. If a certain element of the

library is used within the melody, scale value vector contains “1” in the corresponding place with that

notes position in the scale library.

 17

Melody I:

Scale Library:

Scale Value Vector:

Complexity coefficients

To observe the complexity behavior in a given melody, mainly for note and duration vector, a

coefficient was created. To indicate the complexity of the note vector used in the melody, note

coefficient determines the number of notes used within the melody. If there are two notes used,

coefficient becomes two. This characteristic analysis can be stated respective to a given melody, instead

of regarding a relationship with another melody.

A Given Melody:

Note Coefficient:

On the other side, rhythm coefficient focuses on the duration binary vector in a given melody

matrix. This coefficient is serving also in the purpose of pattern analysis in melodies and the whole

music piece. It can be defined as the complexity level of the rhythm of a melody. Rhythm coefficient

determines how many note hits (1’s) exist with a duration binary vector.

Melodies:

Duration Binary Vector:

Rhythm Coefficient:
 18

It can be seen that, the melody with the rhythm coefficient on the left bears smaller duration

values than the melody with more durations on the left. This factor is used as a significant tool during

generation of a new melody to control the organization of the piece.

Behavioral analysis vectors

Another type of characteristic regarding the matrical melodies are the behavioral analysis

vectors. These vectors contain information about how note and duration values change directions while

picking their succeeding element. These is a type of pattern analysis unbiased from the values selected

for the note or duration vectors. The figure below is an illustration for the behavioral analysis vectors.

Figure 7: Behavioral Analysis Vectors for the Melodies from Lightly Raw

Given two melodies share the same attributes when selecting their succeeding values in both

duration and note vectors. This type of characteristic explains deeper similarities, which cannot be

explained by cosine similarity measure or complexity coefficients. Information again is saved regarding

these two vectors that share the same value selective behavior in the whole melody, which will be used

as an information when generating a new melody out of a random melody.

 19

Relationship of the characteristic analysis features

The characteristic analysis features regarding a melody may not be limited to the ones defined

in this chapter. There may be more information, which can serve in explaining a melody

mathematically. Yet, during the definition of these characteristics, there were three major points come

out of the experiments, which led the way to limit them as they are for this project.

One important subtopic concerning these characteristic analysis features is the transmissibility

of this characteristic information into other music pieces with different note and rhythm structures,

meaning matrices with different sizes. The rhythm key of the melody to be generated may be different

from the rhythm key of the melodies above. That is the reason these similarities are recorded as binary

information instead of characteristic analysis features themselves. The goal of obtaining these

characteristic analysis features is to determine whether two melodies share deeper characteristics or

not. If they do share, generative algorithm will try to create a new melody in a correlative manner. The

algorithm does not concern the column-to-column changes instead; information is recorded regarding

how two melodies share a characteristic. This bases the ideology under transition matrix, which will be

introduced in the next chapter.

Another important aspect about the characteristic analysis features is that they should not

crosscut each other. The purpose of this characteristic analysis is to serve as router during the generation

procedure. However, in order them to be applied in an efficient manner, the aforementioned

characteristic analysis features should have a hierarchical structure to prevent loops that go to infinity

within the generation. A plethora of this information may make it impossible for generative algorithm

to process a music piece in an acceptable time interval. When all the constraints need to be analyzed,

and randomness defines the core of the algorithm, the optimized search space may become so complex

that generative algorithm may not even find the space in a long time.
 20

Figure 8: Hierarchical Table for the Relationship of Characteristic Analysis Features

As an example, from Figure 8, cosine similarity measure is a broader type of analysis that is

used to identify if a melody is identical with another melody. If the cosine measure is 1, there is no way

that two melodies can have different characteristics within. Yet, a note vector may have the same scale

value vector and note behavior vector while not being identical. Behavioral rhythm characteristic

vectors may be same while duration binary vectors are not identical. It is possible to generate a new

melody by applying only a hierarchical procedure of these characteristic analysis features. That is one

reason that these characteristic analysis features are limited.

The last important aspect about the relationship of these characteristic analysis features is the

partial similarity. A partial similarity may exist between two melodies yet they are not represented due

to inequality of these characteristic analysis features. While the melodies do not share a certain

characteristic, some parts of the melodies may partially have strong similarities. These type of

characteristic analysis features may evolve the definition of partial or deeper patterns within a music

piece.
 21

Partial Similarity has a more complex search algorithm unlike comparing characteristic

analysis vectors if they are equal or not. To identify the pattern occurrences, all sequential combinations

of the elements within the vectors are checked. In order to conceive this information, a loop is generated

to create (n(n+1))/2 separate vectors to be analyzed afterwards regarding similarity occurrences, where

n is the number of elements (rhythm key). This search procedure exponentially increases the storage

place, which reduces the speed of the generative algorithm. Examples of partial similarities can be seen

in Figure 9.

Figure 9: Partial (Two-Three columns) Similarity between Two Melodies

Partial Similarity is also highly subjective aspect. There may be different views on how one

defines a pattern in a sequence of values. It can be defined as repetitive elements, frequent sequences

in different locations or existence of at least three or more elements. Yet, the purpose of this analysis is

to reveal the existence of a pattern as an information to be used during the generation of a new music

piece.
 22

CHAPTER IV

GENERATIVE ALGORITHM

To provide the integrity of all measures in a created piece, melodies must have relationships

preferably. As aforementioned, music has a trade-off between creativity and certain musical behavioral

rules. These rules, to be learned, are the conventions made by our history of music. Being energized,

cooling down, repetition and so on. A music piece may be created with nothing but full of randomness

depending on the purpose of its creation as it is in certain genres like free jazz. Yet, even these genres

bear some musical characteristics underneath which points out the necessity of hidden rules alongside

creativity. On the other way, if the piece were created fully according to musical rules for a certain

audience, as it is in the knowledge-based systems, it would be no more than a replica and lack

uniqueness/identity. If there is no randomness, then each song would be similar. These two notions

make this trade-off so important.

One way to balance these two concepts is to simply taking “human behavior in composing” as

the main reference. A composer starts by creating a certain melody. If he or she is satisfied with the

melody, composing continues with building the other melodies, eventually the traffic of the song, in a

complementary manner with the first melody. Precisely, this is one dimension of a musical piece where

composing goes horizontal. The second dimension is the creation of the other instruments, which is the

vertical dimension. Here, the interrelations between melodies bear a different characteristic where

instruments have different nature of expressing themselves, which is also due to our musical

background. A drum rhythm on a guitar melody puts its focus less on the notes but more on the duration

of notes (rhythmic aspect). On the other hand, a new string or a piano melody is influenced from the

notes equally or more from the rhythmic aspect of a given melody. (Here, we define drum rhythms as

also melodies).
 23

Figure 10: The Relationships between Melodies

 Each square indicates a melody belong to that certain instrument.

 Arrows indicate the relationship between melodies, where characteristic comparison takes place.

Relationships are both vertical and horizontal for a given melody.

Thus, it is highly important for generative algorithm is to generate new melodies according to

a recourse, which includes these relationships of each square with other squares in the graph above. A

melody in a measure must be in a meaningful manner with its predecessor, posterior, same located

measure from other instruments. More truly, each square/melody in the x-y graph must have a specific

relation with other squares in its vertical line or horizontal line, which makes the song in a meaningful

manner. Thus, it was determined that all the new melodies in a row in the graph below would be

generated using a single melody from their row as an input.

Figure 11: Projection of the Music Generation within the Generative Algorithm
 24

In this research, the way proposed to build a music piece is as following; first step is to create

a random melody associated with an instrument, defined as the “core”. The core, a sample melody,

which is supposed to be created with a generative algorithm, should be unaffected by any paradigms or

rules like in knowledge-based systems. After obtaining this core, which is a melody matrix, other

melodies can be generated taking the core as an input and transition matrix as the router for the

generative algorithm. The process can be emphasized with a mathematical function:

The input “x” is a randomly generated melody constructed with pseudo random number

generator and “f” is the generative algorithm, which contains the transition matrix within. The transition

matrix is created by influencing from another music piece, meaning assimilation of the music structure

such as # of measures, # of instruments, traffic and possession of the musical customs. It will serve to

structure and place the new melodies created out of the core, in a manner that the output would be a

unique piece with meaningful interconnections. Thus, in other words, randomness will be placed within

customs learned from music piece(s), where output will eventually bear both characteristics of

uniqueness and convention. This was possible due to the definition of linear algebra as the

representation of music where generative algorithm is created in the manner of matrix manipulation,

which makes it a thoroughgoing analysis from bottom to top.

The characteristics relationships between the melodies are contained within the transition

matrix. This is done through recording the characteristic analysis features from a single or multiple

music pieces. These aspects point out the importance for the cooperation of transition matrix with the

generative algorithm. The generation is visualized in the figure below:

 25

Figure 12: Generation Road Map

Thus, the process of generation is mainly divided into three parts as such; generation of

random melodies (core) to serve as the inspirational feature, forming the transition matrix by

conducting characteristic analysis from a music piece as the influential aspect of the composition and

using the generative algorithm to output a music piece using the core melodies and transition matrix.

Generation of the Core

After defining the ideology, parts of the generative algorithm are defined for the music

composition. As aforementioned, a random melody is placed in the generative algorithm as an input,

which is defined to be a core melody. The output, a music piece will be constructed by processing the

generated melody with the transition matrices obtained by the music piece(s) it has been taught.

The vectors defined below are how parts of melody matrices and characteristic analysis features

are defined in MATLAB programming language. (See Appendix B)

 26

Formation of the Melody Matrix:

RKEY: the rhythm key for the piece/melody

SCALE: the library for note values that can be selected

GNOTE: vector containing the notes included in GMEL

GDUR: vector containing the generated durations of each note

GPOW: generated power vector included in GMEL

GMEL: “generated melody” aggregated version of note/duration/power vectors

Characteristic Analysis Features:

GSCALEVALUE: Notes used from the Scale Library

GNOTEC: Coefficient indicating number of notes used within the melody

GNOTEBEHAVIOR: Behavioral Analysis Vector for Note Vector

GDURBIN: Binary Duration Vector showing how note hits are placed

GCUMDUR: generated melody’s cumulative durations regarding time

GRHYTHM: Coefficient indicating complexity of the duration

GDURATIONBEHAVIOR: Behavioral Analysis Vector for Duration Vector

Generative Procedure of the Core

The following procedure creates a melody to be used as the core during the composition process.

The sequence for the creation of vectors is Duration-Power-Note, which are combined afterwards.

Because the core melody is created fully randomly without constraints, the procedure is more simplistic

compared to creation of new melodies to be described in the following sections. To illustrate, an

example of generation will be presented. In the set of actions above, melody is created between first

seven operations and characteristic analysis are recorded from thereafter.

 27

Generation of Rhythm Key (Building the Column Space)

RKEY = random: 4

Generation of Binary Duration Vector

GDURBIN = random (0,1)1xRKEY:

Determine Duration Vector Values

GDUR = adjust (GDURBIN1xRKEY):

Generate Power Vector Randomly

GPOW = random (0,1)1xRKEY:

Define the Scale for the Note Vector

SCALE: random (0,1)1xRKEY:

Generate Note Vector Randomly regarding GDURBIN & SCALE

GNOTE = random (SCALE)*GDURBIN:

Build Melody Matrix by aggregating GNOTE, GDUR, and GPOW Vectors:

Determine the Cumulative Duration Vector

GCUMDUR = adjust (GDUR):

Define the Rhythm Coefficient according to GDUR

GRHYTHM = adjust (1,0) ~GDUR:

Define the Note Coefficient

GNOTEC = adjust (1,0)~GNOTE:

Determine the Scale Value Vector

GSCALEVALUE = adjust (1,0)~GNOTE:

 28

Determine the Note Behavior Vector

 GNOTEBEHAVIOR = adjust (1,0)~GNOTE:

Determine the Duration Behavior Vector

 GDURATIONBEHAVIOR = adjust (1,0)~GDUR:

The characteristic features of the generated melody are saved in order to be used during the

generation of new melodies. These characteristic features can be defined as an initial reflection of the

music piece. Features as Rhythm Key and Note Scale are defined within the procedure that are universal

for the whole music piece.

Core Melody:

Scale:

Duration Binary Vector:

Scale Value Vector:

Duration Behavior:

Note Behavior:

Rhythm Coefficient:

Note Coefficient:
 29

Transition Matrix

Transition Matrix is the matrical representation of necessary musical knowledge that is obtained

from a music piece. The purpose of transition matrix is to serve as a reference to generative algorithm

when a new melody needs to be generated. The characteristic analysis features explained earlier in the

methodology part, shapes the elements within the transition matrix. This is the aspect of influence,

which is regarded as the musical knowledge necessary for the composing algorithm. Thus, transition

matrix serves as a structure of the music piece to be created according to the Core.

Figure 13: Illustration of a Music Piece before the Analysis

Each line in the figure represents a note-duration value on a cumulative time scale, where

instruments are separated with different colors. There is a single instrument on the left side of the

vertical line, yet there are three instruments on the other side. The grey lines are chords, which indicates

multiple note values in the same time, which are recorded with column matrices within the note vector.

The structure of the music piece can be observed by analyzing the figure where melodies are stationed

differently. Purpose of transition matrix is to record this information in a mathematical manner to be

used in generating new melodies.

A music piece is transformed into comma-separated values (CSV) for text processing and later

transformed into matrical representation by operations through MATLAB. (See Appendix A). Matrical

representations obtained from the musical piece can be seen in Figure 11.
 30

Figure 14: Matrical Structure of the Music Piece shown in Figure 11

Once the matrical representations are obtained, characteristic analysis is held for each melody

in the music piece. These include, cosine similarity measures between each vector of the melodies (1)

(2) (3), duration binary vector (4), rhythm coefficient (5), note coefficient (6), scale values vector (7).

The structure of their storage differs according to the nature of information. Cosine similarity measures

are information between melodies, which make them stored in structures that contain relationship of

each melody with another. Other characteristic analysis is personal for a given melody, which makes

them stored in the same structure as the music piece is formed. The record process of characteristic

analysis features can be seen in Figure 15.

Figure 15: Characteristic Analysis Features Recorded from the Four Melodies

After all analysis vectors are obtained, each characteristic of a melody is compared with other

melodies characteristics. This is the stage where transition matrix is beginning to be created. Similarity

vectors are created within the transition matrix as an element of the matrix. Melodies relationship with

each other are saved as binary values in similarity vectors within the transition matrix. Comparisons
 31

are held to determine whether two melodies share the same characteristic value or not which are labeled

as 0 or 1. To give an example, if two different melodies bear the same characteristic feature, the

corresponding binary value inside a similarity vector becomes “0”, indicating there is not a need for

modification. On the contrary, binary value becomes 1 in the similarity vector, when corresponding

characteristic features are different. Similarity vectors that reflects an identical, similar and completely

different melody consecutively can be seen below;

Figure 16: Similarity Vectors to be placed in Transition Matrix

After each characteristic is checked whether they are equal or different, binary values are assigned

in similarity vectors, and the vectors are placed within the transition matrix in an index that reflects the

relationship of those two melodies, which can be seen in figure below.

 32

Figure 17: The Transition Matrix Containing Similarity Vectors for a Single Instrument

After transition matrix is fully obtained another matrix is created for the symbolic representation

of transition matrix, which is for user to have a visual of the structure of the music piece. If all the

elements of the similarity vector are same, these two melodies are defined to be identical and labeled

with “I”. If there is 1’s and 0’s in the similarity vector at the same time, they are defined as similar

melodies and labeled with “S”. If all the elements are 0, these two melodies are defined as completely

different, labeled with “R”. The 1’s and 0’s within a similarity vector will control how the generative

algorithm will produce new melodies in the later section. The symbolic transition matrix can be

observed below;

Figure 18: Symbolic Transition Matrix

 33

Generating New Melodies Using the Transition Matrix & Core Melodies

The necessary generative algorithm for the composition of the music piece is consisted of

repeat-continue procedure of actions. Unlike the generation of the core melodies, this algorithm has

loops, which may start processing or elapsed according to corresponding binary values in the similarity

vector. The necessary creativity aspect of the composition method is included both in the creation of

the core melodies and part where generation of new melodies takes place. There are three common

scenarios where parts of the algorithm are authorized to process;

1. Algorithm simply generates a new melody randomly - If all the similarity values within

similarity vector are 1’s. (R: No Relation between Melodies)

2. Algorithm manipulates the core by replacing and generating random values until it satisfies the

constraints – If there are both 1’s and 0’s in the similarity vector. (S: Similar Melodies)

3. Algorithm copies the core melody – If all the similarity values within similarity vector are 0’s.

(I: Identical Melodies)

As it is in the generation of the core, this algorithm follows a creation procedure in the sequence

of duration-power-note. Consecutive three loops control the process of generation, which have initial

conditions held by cosine similarity measures.

Beginning from the duration loop, cosine measure is checked whether the melody to be

generated contains an equal duration vector or not. If cosine similarity value between two melodies are

“0” (indicating melodies with equal duration vectors), melody surpasses the duration loop of the

algorithm and proceeds to power loop. The important fact about these paths is that the characteristic

analysis constraints are naturally concentric; meaning satisfying cosine similarity constraint

automatically satisfies duration binary vector duration behavior and rhythm coefficient constraint. This

evolves to creation of less than 2n paths where n is the number of constraints. If otherwise, meaning two

melodies (core melody and melody about to be generated) do not share the equal duration vectors,

duration vector enters into the loop to be processed according to other values in the similarity vector
 34

regarding the duration vector. In the figure below, box containing one in the column of cosine similarity

measure is the part which duration vector enters the loop. The pathways within the duration loop is

illustrated below.

Figure 19: Duration Loop within the Generative Algorithm

This part holds necessary actions, containing random generation, permutation and elementwise

manipulation, depending on the path duration vector has to pass. All the boxes containing one starts an

operation to the duration vector to be created. The new vector may have same duration binary vector

and only need new duration values, while protecting the note coefficient characteristic equal. Regarding

the combinations that algorithm has to process, following table emphasizes the actions to be taken;

Table 2: Actions Taken regarding the Combination of Constraints

 35

For each constraint regarding the duration vector, if the corresponding binary value is 1 in the

similarity vector, algorithm applies generation and/or permutation until all constraints are satisfied to

be proceeded to the loop regarding the power vector. After each action taken, the resulting new vectors

are compared with the old ones whether they satisfy the binary value constraint in the similarity vector.

If not, same actions take place again until they are satisfied. Same procedure is followed in the power

area to be proceeded to the note area, which can be seen in Figure 20.

Figure 20: Loop regarding the Power Vector

Power vector has the least complex procedure of actions due to the mass of identical vectors.

Almost all the melody matrices have the same power vector containing equal values, which indicate

stable hits. Yet, even the cosine similarity measure for the power vector is not equal, a random

generation takes place until new vector and core vector do not share a cosine similarity of 1. The

necessary actions can be seen in the table below.

Table 3: Necessary Actions taken in the Power Loop:

 36

Following the power loop, note vector is passed to the note loop to be checked by cosine

measure at the beginning. Note loop has the same architecture pretty much like the duration loop due

to the nature and number of the characteristic constraints. As it is in the duration loop, note vector is

checked whether it satisfies the cosine similarity measure. If not, it enters the loop to be processed

according to the actions defined by the combinations of the constraints, which is illustrated in Figure

21 and Figure 22.

Figure 21: Loop regarding the Note Vector

Table 4: Necessary Actions Taken in the Note Loop

 37

After all constraints are satisfied, three vectors (duration, power and note vectors) are

aggregated to each other to form the new melody and placed within the corresponding place in a matrix

regarding similarity vector in the transition matrix used. After all melodies are generated, matrix that

contains the music piece are processed into MIDI format through text processing for listening. (See

Appendix A)

Figure 22: Generated Matrices after the Algorithm Finished Operating

In this research, different instruments were generated only from a horizontally direction,

meaning each row that belongs to an instrument was generated by using a core melody belonging to

that row. These other core melodies however, were created by manipulating the initial core melody

with assumptions referenced by Knowledge-Based-Systems. A core melody for the drum rhythm was

created from initial core melody according to vertical transition matrix defined earlier. Yet other

melodies belong the drum rhythm was created regarding the core melody generated and horizontal

transition matrix belonging to them.

The intended analysis is to generate these melodies looking at both vertical and horizontal

directions while being inside the generative algorithm. This increases the necessary constraints within

the algorithm, changing the architecture of the generation. There are several other constraints are

needed to link melodies horizontally to each other. Yet, the architecture of the algorithm is adaptable

for new constraints if only if constraints have a hierarchical relationship that do not crosscut each other.

Correlation of these characteristic analysis (how duration binary vector acts with the rhythm coefficient

and e.g.) is crucial for the practicality of the algorithm.

 38

CHAPTER V

RESULTS OF THE STUDY

The last stage of the generative algorithm was evolved after many experiments. In most of the

early experiments, the generated music pieces did not consist of inter-related melodies where they

supposed be, or the process time was incredibly long due to disharmony of the characteristic analysis

constraints. Different musical structures caused different problems during composing. Yet, these

problems made the generative algorithm evolve into a level of flexibility in order to respond to various

scenarios. After improving the architecture of the algorithm and re-designing the correlation between

the characteristic constraints, the last state of the generative algorithm was successful in generating

unique and meaningful music pieces out of various music structures in considerable time intervals. The

generated pieces were validated by comparing them with core melodies and transition matrices, which

turned out as intended. While the music piece follows musical behavior offered from the transition

matrix, it produces a unique and inter-related music piece.

The basic idea was different from learning notational knowledge as a certain note follows a

specific note. In fact, the purpose was to learn the compositional knowledge that includes information

about similarities of melodies, repetitions and song structure, which was at the end achieved. The

generated pieces were containing the same compositional knowledge as they have the same structure.

Yet pieces were built upon the new melody that has completely different nature, which is reflected into

the whole music piece.

Due to randomness aspect of the algorithm, even when the same core melody is processed by

the same transition matrix, the generated music piece was completely different, which indicates that

the search area within the generative algorithm is extensive. This is due to reaching a certain space of

satisfaction instead of finding an optimized value. Cause if it was so, the algorithm would take much

longer hours of operation to generate a music piece, even not be successful by completing. The figures

below contain two generated music pieces with their core melodies.
 39

Figure 23: Conventional Representation of a Generated Music Piece and Core Melody
 40

Figure 24: Conventional Representation of a Generated Music Piece and Core Melody

The range of the process time can be highly large due to the structure of the transition matrix.

The procedure of the generative algorithm was designed to reduce the process time by manipulating

only the necessary vector (note, duration, power) when a constraint was not met, instead of

manipulating all. When parameters are complex in terms of music, (e.g. large rhythm key value, a big

scale or a complex transition matrix where there are many differences between melodies), it takes

comparably longer time for algorithm to generate a whole music piece. It is due to the fact that

constrains are harder to satisfy when the new generated song bears large rhythm key or complex melody

structure within. This aspect makes the process time of the algorithm an unpredictable value. As an

example if one needs to generate a music piece inspired by a complex melody and influenced by a

complex genre, process time may take a longer time.

 41

Figure 25: MIDI Illustration of a Generated Music Piece with Process Time: 382 Seconds

Another feature that is believed to increase the process time is the relationship of the

characteristic analysis features. As an example, note coefficient, note behavior, scale value vector and

cosine similarity values are characteristic information regarding the note vector. During the

experiments, it was observed that infinite loops were evolved from the correlation of these constraints.

The sequence of applying such constraints directly affects the process time. Defining the constraints in

concentric manner prevented these infinite loops occurring. As an example, when two melodies have a

cosine similarity measure of “1”, they are identical which prevents the generative algorithm taking note

scale vector and note coefficient into account.

 42

CHAPTER VI

CONCLUSION, DISCUSSION & FUTURE RESEARCH

Conclusion

As a new form, using linear algebra as the mathematical representation of music have been

successful in several ways. The most important gain from using vectors and matrices is the extended

capability of analysis on music pieces. Instead of using a sequential language, as it is used in other

projects, containing musical information in matrices offered highly practical data structure open for

analysis and manipulation. The operational capability of linear algebra provided many ways to analyze

and interpret the behavior of music, observe the similarities and patterns between melodies. In addition

to analysis, matrical representation opened up a way for algorithmic composition method to evolve.

One of the very big issue regarding algorithmic composition methods so far was the inability of

producing meaningful music pieces. The term “meaningful” should not be put on par with sounding

pleasant. Meyer defines musical meaning as “the planned deviation from the norm and how it is

accomplished” which emphasizes the necessity of musical knowledge (Meyer, 1961). It cannot

expected of computers to plan and structure music by themselves, who lack feelings, intentions or way

of interpreting our lives. This idea was the base of designing an algorithm, which needs to absorb

knowledge. Generative algorithm learns the way of expressing from human-compositions, which in a

way fills the gap of natural expressionism. By defining randomness as creativity and inserting it within

the generative algorithm with a transition matrix that can master musical customs, the notion

“meaningful” is to be achieved in the maximum possible way. As a new algorithmic composition

method, algorithmic composition method proposed in this paper had a good performance on composing

meaningful music pieces. It is due to the capability in combining creativity and convention in a natural

way.
 43

One of the most advantageous features of the generative algorithm is its flexibility, which is due

to the usage of linear algebra. Compared to other famous algorithmic composition techniques, the

algorithm needs much less initial knowledge, which has to be pre-determined. The field of operation

for the algorithm is not limited to a certain genre, music piece structure or amount of instrument. The

fact that algorithms learns this information provides an extensive range of work area. Another

advantage linked to using linear algebra is that the capability of interpreting music alongside geometry.

Because linear algebra is the mathematical field to interpret geometrical meanings, the algorithm

proposed in this paper can be used to generate songs by even analyzing paintings, designs or any other

geometric objects. This linkage should open up new possibilities for the music composers to find new

styles, genres and music cultures that are yet unknown.

Discussion & Ideas about Future Research

The understanding of composing in this research was based on human-composer behavior at the

most. This was held by analyzing a human composition knowledge and applying it into a new

composition by generating a random core melody, as composers start their composition with a melody.

Yet, composers do not have a musical knowledge that relies on a single music piece, different from the

generative algorithm where composition knowledge is obtained from a single music piece. This is a

main research topic that builds upon this research. It should be investigated how compositional

knowledge, or transition matrices in this research, can be obtained from many music pieces, which is

the exact case for the composers. Whether the transition matrices should be mixed into each other or

transition matrices that have a high correlation may partially effect the construction of the transition

matrix according to a fitness test that acts as an authority on selection. This topic of research not only

related with music composition but should also interpreted through composer’s cognition of

composition using their musical background. It takes this research into a higher level where justification

is difficult yet a certain methodology that has sense on these selections is achievable. It is so; by

analyzing many transition matrices and breaking transition matrices down as melodies are broken down
 44

to obtain knowledge in this research. Thus, it is actually the possibility of another dimension of analysis.

The figure below illustrates the idea.

Figure 26: Illustration of the Dimensions of Analysis

Another important topic for future research is the capability of transition matrices on identifying

high similarities between music pieces. This is a powerful that marketing and business analysts may be

interested in. By this capability of analysis, hidden knowledge can be obtained on why certain audience

prefers a music structure to another; how certain music pieces or campaign music share an identical or

highly similar transition matrices with each other, even issues with plagiarism may be identified

between music pieces. The usage of transition matrices are unique in that manner where people try to

identify similarities regarding the physical appearance of music pieces (notes selected in a sequence),

where the knowledge transition matrices reflect is completely underwater.

Another important subject to be improved after this research is the vantage point aspect within

the algorithm. As defined earlier, generation progress evolves through manipulating a core melody and

generating new melodies according to transition matrix. While all the similarity relationships between
 45

melodies exist within the transition matrix, core melody is manipulated from the perspective of a certain

melody. If there are four melodies within a music piece, there are four different ways to create the 4th

melody during the generation since 4th melody has relationship with all the other melodies including

itself. Yet, due to placing core melody as any melody from the music piece, the generation procures the

perspective of that certain placement. An example would be as such; given there are three melodies to

be created and the first melody is similar to second and third one while second and third melodies are

identical. If the core melody is placed as the first melody and others to be generated by manipulating

the first, generation will create two different melodies for the second and third since using randomness

when generating a similar melody. This aspect puts forward the vantage point problem where creation

of identical melodies, which are similar to the core melody, is not achieved now. This as another topic

of research can be solved by involving more than one perspective during the generation process of the

algorithm where two identical melodies that are similar to the core can be generated identically in the

generation. It may even reduce the process time of the algorithm where some certain melodies would

be placed by cloning instead of generated randomly.

“Good” Music

The opinion on a music piece whether it is beautiful or not, is considered subjective for its

audience. There are examples of songs that are accepted as masterpieces by almost everyone or songs

just admired by a specific community. Almost no song is admired by everyone. Even the sounds that

are not accepted as music; sound of a wave, crowd or water can and are appreciated by people. Thus,

seeking a common method to generate a piece that would be admired by all is not the purpose during

generating music, where again, justification is unachievable. Yet, computers lack the sense of listening

music, which makes it even harder to describe them while we as humans are still in the dark about this

aspect.

To put forward an idea, there is a certain goal that has to be chased while composing, the aspect

that has to be maximized, is the harmony, concord of all elements with each other to provide the
 46

necessary output to the audience. Given there are limitless type of preferences in music industry, there

should be limitless type of pieces to be built with the help of computerized generation. The only

common feature within this variety is the harmony of the musical elements; how the left hand in piano

supports the solo played from the right hand, how a drum player can start playing the expected rhythm

on a bass solo that makes everyone clap their feet and so on. One can define this this as the art of

cooperation. Some people define it as “groove” and some people even define it as religion. Even

speaking a language to communicate can be a proof of the power of harmony. How a word is placed

within a sentence can change the effect of a given sentence. For music, people have the evidence that

how this feeling, defined as the effect of harmony, can affect people to feel and react extraordinarily

efficient, can help plants grow faster or it can help Alzheimer patients remember and react to the music

and most importantly, it is what our brain records and never forgets (Norton et al, 2009). It is the

chemistry of each element with another, which raises the question about the possibility of finding it

through mathematics.

Thus, generating, in a mathematical manner, is to find out how to model the numbers in such a

manner to obtain the best possible relation out of them, which again is hard to justify. In order to do

this, detailed analysis has to be conducted on the relations, positions of the numbers with each other,

so that generation can be based on maximizing such relations, the harmony. This point is the most

challenging and least-illuminated part of this project, yet offers great satisfaction during the chase. This

is why; music generation will be improved and influenced by the mathematicians in the future.
 47

REFERENCES

Ames C., Domino M. (1992). Cybernetic Composer: An Overview. Understanding Music with

AI: Perspectives on Music Cognition. Pages 186–205. The AAAI Press/ MIT Press

Balaban M., Ebcioglu K., (1992). Understanding Music with AI: Perspectives on Music Cognition.

Cambridge, Mass: AAAI Press, Print.

Brinkkemper F. (2016). Analyzing Six Deep Learning Tools for Music Generation. [Online]. Available:

http://www.asimovinstitute.org/analyzing-deep-learning-tools-music/. [Accessed: 3- March -

2017].

Burgoyne, J. A., Bountouridis, D., Van Balen, J., Honing, H. (2013) Hooked: A Game For Discovering

What Makes Music Catchy. Proceedings of the 14th International Society for Music

Information Retrieval Conference

Cope D. (1992). Computer Modeling of Musical Intelligence in EMI. Computer Music Journal, pp. 16

(2):69–83

Ebcioglu K., (1988). An Expert System for Harmonizing Four-part Chorales, Computer Music Journal,

12 (3):43–51

Godøy, R. I., & Leman, M. (Eds.) (2011). Musical Gestures: Sound, Movement, and Meaning. New

York (N.Y.): Routledge

Griffith N., Todd P. M. (1999). Musical Networks: Parallel Distributed Perception and Performance.

Cambridge: MIT Press.

Hiller L., Isaacson L. M. (1959). Experimental music; Composition with an Electronic Computer,

Publisher New York, McGraw-Hill

Hoff A., (2016). In convergent: A Study of Generative Algorithms [Online]. Available:

http://inconvergent.net/. [Accessed: 17 - March - 2017].

 48

Janssen B., Haas B., Volk A., Kranenburg P. (2013). Finding Repeated Patterns in Music: State of

Knowledge, Challenges, Perspectives. International Symposium on Computer Music Modeling

and Retrieval. CMMR 2013: Sound, Music, and Motion pp 277-297.

Karpathy A. (2015). The Unreasonable Effectiveness of Recurrent Neural Networks, [Online].

Available: http://karpathy.github.io/2015/05/21/rnn-effectiveness/. [Accessed: 22- March -

2017].

Klapuri, A. (2010) Pattern induction and matching in music signals. Exploring Music Contents, 7th

International Symposium, CMMR 2010, Málaga, Spain, pp. 188–204

Meyer L. B. (1961). Emotion and Meaning in Music, University of Chicago Press

Mozer M. C. (1994). Neural Network Music Composition by Prediction: Exploring the Benefits of

Psychoacoustic Constraints and Multi-Scale Processing. Connection Science, 6 (2-3): 247–

280

Neumann M., (2010). La forme rondo. [Online]. Available: http://www.clg-daubigny-auvers.ac-

versailles.fr/spip.php?article1007 [Accessed: 4- April - 2017].

Norton A., Zipse L., Marchina S., Schlaug G. (2009) Melodic Intonation Therapy. Music, Stroke

Recovery, and Neuroimaging Laboratory, Beth Israel Deaconess Medical Center/Harvard

Medical School, Boston, Massachusetts, US

Papadopoulos G., Wiggins G. AI Methods for Algorithmic Composition: A Survey, a Critical View and

Future Prospects. Proceedings from the AISB’99 Symposium on Musical Creativity,

Edinburgh, Scotland. 1999, pp. 110–117, 1999

Ramalho G., Ganascia J. G. (1994). Simulating Creativity in Jazz Performance, B. Hayes-Roth and R.

Korf, editors, National Conference on Artificial Intelligence 1994, Volume 1, Pages 108–113.

The AAAI / the MIT Press

 49

Robertson J., de Quincey A., Stapleford T., and Wiggins G. (1998). Real-Time Music Generation for a

Virtual Environment in ECAI98 workshop on AI/Alife and Entertainment, Brighton, UK.

Shah S. (2010). An Exploration of the Relationship between Mathematics and Music. Manchester

Institute for Mathematical Science. School of Mathematics. The University of Manchester

Todd P. M., Loy G. (1991). Music and Connectionism. Cambridge MIT Press, 1991.

Walker J. (2008). MIDICSV, [Online]. Available http://www.fourmilab.ch/webtools/midicsv/

[Accessed: 24 - March - 2017].

Xenakis I. (1971), Musiques formelles, Revised English edition Formalized Music: Thought and

Mathematics in Composition. Harmonologia Series No. 6. Stuyvesant, NY: Pendragon Press.

ISBN 1-57647-079-2. Reprinted Hillsdale, NY: Pendragon Press

Zimermann D. (1998), Modelling Musical Structures. Aims, Limitations and the Artist’s Involvement”.

Workshop at ECAI’98. Constraint Techniques for Artistic Applications, Brighton, UK

 50

APPENDICES

Appendix A

Music-to-Matrices Transformation

Transforming mathematical representations into music and the other way around needed multiple

actions. First was to find a musical extension that can be transformed into data. MIDI (Musical

Instrument Digital Interface) was determined to be the necessary computer extension, which is a sort

of protocol or language between computers and musical instruments. It enabled songs transform

mathematical analysis and computations to into music. After determining the computer extension to be

used, transformation of MIDI into data was needed.

The transformation is held through two applications called MIDICSV and CSVMIDI created by

John Walker (Walker, 2008). He defined a CSV Language that can be transformed into a MIDI file and

vice versa. In this research, the MIDI music files are transformed into MIDICSV Language to be later

processed into matrical representation by MATLAB Code (MIDI2MY). After obtaining the MIDICSV

type of data, which can be seen in the figure below, a text, processing was needed to transform them

into matrical representation.

Figure 27: Representation of MIDICSV Language

 51

Two MATLAB scripts were written in order to transform MIDICSV language into forms of

vectors and matrices. The transformation can be seen in the figure below and the code is provided in

the following section.

Figure 28: MIDICSV into Matrical Representation by MIDI2MY

When the transition matrix is obtained, generative algorithm begins to compose a new music

piece. After the composition is over, metrical representations again transformed into MIDICSV

Language by inverse text processing and later to be transformed into music that audience can listen.

The MATLAB code is provided in the section.

Figure 29: Matrical Representation into MIDICSV by MY2MIDI

 52

Appendix B

Matlab Scripts for MIDICSV Language to Matrical

MIDI2MY (Matlab Code for MIDICSV Language to Matrical Representation)

53
54
55
56
 57

Appendix C

MidiCsvScript (Python Code for transforming Midi to MIDICSV Language)

TXT2EXCEL (Python Code for importing CSV into EXCEL File)

 58

Appendix D

MY2MIDI (Matlab Code for Matrical Language to CSV Language)

59
60
 61

Appendix E

CsvMidiScript (Python Code to transform CSVMIDI to MIDI)