Machine (Un)intelligence: Aesthetics, interaction and improvisation

Henrik Frisk (PhD) - professor of music

2022-01-20

Created: 2024-07-10 Wed 16:08

Goodbye Intuition

An artistic research project hosted by the Norwegian Academy of Music in Oslo that includes a range of improvising musicians:

Ivar Grydeland

Morten Qvenild

Sidsel Endresen

Andrea Neumann

as well as:

David Toop and myself

The project

is a platform to explore the meaning of intuition, musical structure and aesthetics by means of playing with a newly developed improvisation machine

(nick-named Kim Auto, or KA in the following)

What is KA, and why?

a composer can assign a variety of roles to a computer in an interactive music environment. The computer can be given the role of instrument, performer, conductor, and/or composer. These roles can exist simultaneously and/or change continually, and it is not necessary to conceive of this continuum horizontally.

Lippe, C., Real-time interaction among composers, performers, and computer systems, Information Processing Society of Japan SIG Notes, 2002(123), 1–6 (2002).

What does KA do?

It collects material from whoever is playing with it and constructs an archive of material that it then uses when it plays. It has four different personalities that may be configured and that defines how it responds to input. These are used to shape its output based on musical concepts such as high/low pitch, and structural aspects such as dense or sparse.

By interpolating between the different personalities the responsiveness of the system and its output may be varied.

Method

Laboratories with experiments and discussions
Performances
Studies performed on the process in the workshops

In summary, an experimental practice with a layer of reflection and analysis.

Example laboratory

On music as a principle for organisation

"[t]here is a long standing tradition of seeing jazz, particularly free and avantgarde jazz, as the expression of an ideal society"

Cook, N., Scripting social interaction: improvisation, performance and "art" music, In: Born, G., Lewis, E., & Straw, W. (Eds.). (2017). Improvisation and Social Aesthetics, (pp. 59–77) (2017). : Duke University Press.

That the field of music is also affected by the huge interest in artificial intelligence is not surprising. The practice of improvisation in music is sometimes seen as a model form for organisation

Given such an assumption, musical improvisation may appear to be the perfect case for evaluating the functionality of humanoid. Would it prove to be possible to write a software that can improvise with musicians in a manner that is indistinguishable from that of a human player, not only is this a significant engineering task, it may be assumed that this software can also function in other social contexts. Music as a test bed for intelligent technology. There are a number of criteria that would need to be met and the computer system's ability to respond to input is a challenge both in deciding what constitutes valid input, and what is a useful response.

On intelligent music behaviour

There ha been a number of attempts to design a Turing test for evaluating the successfulness of intelligent systems for music processing:

Rencon piano competition
Live performance evaluation
Evaluation of interactive systems such as The Continuator

What is intelligent musical behaviour?

"[w]hat purpose would be satisfied by creating qualitative or quantitative metrics for musical intelligence, given the lack of successful similar criteria for natural musical intelligence, musicality, or even music per se"

Belgum, E., Roads, R., Chadabe, J., Tobenfeld, E. T., & Spiegel, L., A turing test for "musical intelligence"?, Computer Music Journal, 12(4), 7–9 (1988).

Examples

"[s]ince 2001 the NIME series of conferences has seen the presentation of a wealth of interface and instrument design ideas"

Bowers, J., & Archer, P., Not hyper, not meta, not cyber but infra-instruments, In , Proceedings of the 2005 conference on New interfaces for musical expression (pp. 5–10) (2005). : .

Schnell and Battier introduced "composed instruments"

Schnell, N., & Battier, M., Introducing composed instruments, technical and musicological implications, In , Proceedings of the 2002 conference on New interfaces for musical expression (pp. 1–5) (2002). : .

Antropomorphising the instrument

Interpreting the attitude of the performer of a composed instrument with the help of categories from the traditional way music is created leads to various metaphors such as that of playing a musical instrument, conducting an orchestra, playing together (ensemble) with a machine, acting as a one-man band.

An instrument or a composition?

"the work is replaced by the interface"

As for KA:

the system works quite well as long as one goes along with it. If you resist it, however, and oppose to its playing or the material it uses, it is very slow to response: In effect, what is emerging is a kind of composition, or a compositional frame.

One of the topics raised early on in the workshop in Stockholm was the notion of KA as a composition rather than a co-improviser, or an instrument. This may at first seem odd, after all, GI is an improvisation group, what purpose serves the notion of a composition in this context? Despite the conceptual contradiction, this is related to the notion of composed instruments, as was discussed above: Some of the performance instructions, the score, is encoded in the system. Or, more plainly: "the work is replaced by the interface"

But if the responses from the system are not responsive enough, so to speak, if the co-player KA is not listening attentively enough, then it becomes difficult to make meaningful commentaries about either the system, or of one's own strategies interacting with the system (even though, as will be shown in the next section, not listening may actually be a perfectly valid strategy). If the perfect playing companion is not available within KA, perhaps it is possible to instead look at it as a compositional frame that affords certain musical behaviour?

Aesthetics and ethics

… art practices in general offer a context in which experimentation and play is possible.

the aesthetics of the practice offers a set of value judgments that an engineering context may not provide, nor a traditional artistic practice.
is not a property of the field of digital art but rather a function of it
the framework of artistic practice does not by itself guarantee sound values and reasonable ethics, it is merely a possibility, a potential,
The emphasis here is on the practice and the way the practice organises itself when the value system is primarily aesthetical

Digital art?

"a item is a work of digital art just in case (1) it's art (2) made by computer or (3) made for display by computer (4) in a common digital code" [1]

"The idea becomes a machine that makes the art. This kind of art is not theoretical or illustrative of theories; it is intuitive, it is involved with all types of mental processes and it is purposeless." [2]

[1] Lopes, D., A philosophy of computer art (2009), : Routledge, Oxon.

[2] LeWitt, S., Paragraphs on conceptual art, Artforum, 5(10), 79–83 (1967).

What is the nature of the machine?

the ethics of improvisation is negotiated through the aesthetics of the context, what is it actually that guides the aesthetics of KA?

When the machine is accepted as a member, what is its identity and who is control?

the code
the hardware running it
the sounds it produces
is it just an extension of the programmer that created it?

Machine aesthetics

Are we prepared to allow the machine to develop its own aesthetics, as we would with a human co-player?

A machine player may develop extremely fast, much faster than a human player.
It may than be uninterested in playing with human performers.
For that reason we may introduce restrictions to the code to avoid this scenario
What are the aesthetical consequences of introducing this kind of limitations on a machine?
Are we not introducing a structural divide between humans an non humans?

One may imagine that within not too long there are tools, like KA, but more advanced, that exhibits some notion of what we can call musical intelligence. Are we in that situation prepared to allow the machine to develop its own aesthetics, as we would with a human co-player? It is not difficult to imagine that such a machine can develop extremely fast, much faster than a human player. In this case, perhaps the machine eventually grows to be uninterested, not only in the music that humans play, but even loses interest in playing with humans at all. If the machine is still programmable by us we could of course put some limitations on the way its knowledge about music develops, thus avoiding this scenario. This kind of limitation could also be part of the design of the system from the beginning, which may appear to be a sensible thing to do, but it would nevertheless be to narrow its freedom in a way that is not often done for human musicians. If these systems in fact are intelligent in some manner, what are the consequences if we introduce constraints on the machines potential for development? Introducing a structural divide between one class of performers (humans) and another (machines) may have an effect on the way