Last modified:
My current research area is in computer music, artificial intelligence, and computational linguistics. Specifically, I am investigating automated performance and composition. It is interdisciplinary work involving machine learning, artificial intelligence, programming languages, computational linguistics, and music theory. Musical problems exist in a vast, multidimensional space that is very difficult to traverse. Many of the algorithms and algorithm categories that are useful computer music also have relevance to other tasks, such as natural language processing and data mining.
As a composer, I am also interested in understanding both my own and others’ compositional processes, as well as exploring new and interesting ways to make music with computers. The Euterpea package for Haskell is ideal for these applications, since it supports note-level and also sound-level manipulation. The image on the right is a score-view of a few measures of MIDI output produced by some of my very early Euterpea-based software.
Most of my work currently is towards the development of an interactive system called MUSICA, which is part of the DARPA Communicating with Computers program. The goal of the program is to create a collaborative agents that can communicate and interact with the user across multiple domains, including music composition and improvisation.
Another one of my ongoing projects is a system called Kulitta. It is a modular algorithmic composition framework that can both generate novel music as an artificial intelligence agent as well as learn musical features from musical corpora – a task that mirrors learning by listening. Kulitta’s features include a new class of musical, parameterized grammars that incorporate some features of programming languages and an iterative approach to composition using topological chord spaces, which are a type of geometric representation for certain music theoretic concepts. Kulitta is implemented in Haskell and uses the Euterpea library.
Algorithmic Composition and Performance
There are many topics that fall under the broad label of “computer music research.” Two such areas are in algorithmic composition and performance. The following diagram gives an idea of the sort of systems that interest me:
Performance
Although not all music can be easily divided into a composition and a performance as distinct objects (soundscapes, for example, would not make this distinction), it is quite common in a lot of music that can be put into traditional Western notation. The performance and the composition are distinguished on the basis that there may be many good ways to interpret the composition’s dynamics, tempo, etc. The following two MP3s demonstrate the impact performance can have on a phrase.
The phrase in the examples above was algorithmically created (by Kulitta – see the next section). The computer performance sounds stiff and clunky, while the human-performed version flows more naturally. Because a composition can have many performances, some good and some bad, it is tricky to evaluate the quality of the underlying composition in a qualitative way. This is particularly true when pieces must be evaluated by ear rather than by analyzing a printed score. Trying to find a quantitative way to evaluate algorithmic compositions has been a subject of my research for the last year.
Composition
Even if we just consider Western music, or even just classical Western music, the box marked “Algorithm 1” in the figure above has a lot of sub-problems to consider. Generating harmonies, melodies, and rhythms are all potentially very difficult problems that may need to be addressed individually. Algorithmic possibilities include generative grammars, Markov Chains, fractals, and many others. Just as human composers must often be trained to improve compositional ability, machine learning approaches may also be employed, allowing a model to learn from some number of samples. This kind of learning ability may be especially useful for algorithms that are expected to cover many styles of music.
Another problem for both sides of the generative process is what kind of input data to use: existing music examples, rules from music theory, mathematical models, streams of random numbers, etc. Considering the wide number of input data and algorithmic possibilities (as well as the number of data structure and other representational issues involved) the fields of algorithmic composition and performance present a very large number of potential solutions to explore.