Representation and Modelling

In computer music theory, the capacity of synthesis to either imitate the world or produce something new is sometimes framed in terms of a standard versus nonstandard approach to mathematical modelling. Standard synthesis simulates a sound described by an acoustic model; nonstandard synthesis specifies an algorithm whose internal relationships are themselves the description of the sound. WP3 asks whether this distinction, and the theories of representation that underwrite it, need to be rethought in light of neural synthesis algorithms that traverse both sets of terms.

Neural synthesis algorithms model audio on the basis that sound can be anything a neural network is capable of learning to recognise and synthesise. In their concatenated translations, from sound to data to text and back again, these methods call for a new understanding of the relation between world, model, and synthesised sound, beyond the binary of imitation and novelty inscribed by theories of standard and nonstandard. Yet the consequences of this shift for electroacoustic music studies have gone unaddressed.

Approach

WP3 is a practice-based inquiry into the use of neural synthesis methods in composition. The work proceeds in two stages. First, audio datasets are built from contrasting sound sources to model a population of speculative sound entities that traverse representational and abstract qualities. Second, these synthetic agents are activated within a network of interconnected processes involving a further level of machine learning, modelling morphological interactions among the agents and their orientations in space. Interaction is modelled as a circular system of speculative creativity through which a human creator can discover new synthetic sound ecologies, departing from unidirectional approaches where humans simply teach machines.

Autoethnography will build an alternative analytical vocabulary for synthetic media, extending attempts to move beyond spectromorphology and engaging with Joanna Demers' notions of reproduction, construction, and deconstruction.

Methods

Practice-research (composition and evaluation), autoethnography, neural synthesis (Realtime Audio Variational Autoencoder), supervised and unsupervised machine learning (multi-layer perceptrons, UMAP, KMeans).

Outputs

Two large-scale compositions premiered internationally; a joint-authored book, Electronic Music After Spectromorphology, co-authored with researchers from WP1, WP2, and WP5.