Theorising Synthetic Models

This study addresses theories of representation and modelling. It has two aims: first, to consider the extent to which relations of representation vs non-representation, like sampling vs synthesis, need to be rethought in light of neural synthesis algorithms which traverse both sets of terms; and second, to consider how the refashioning of this relationship impacts key tenets in electroacoustic music composition and analysis.

In computer music theory, the capacity of synthesis to either imitate the world or produce something new is sometimes framed in terms of a standard versus nonstandard approach to mathematical modelling. Standard synthesis simulates a sound described by an acoustic model; nonstandard synthesis specifies an algorithm whose internal relationships are themselves the description of the sound. This study asks whether this distinction, and the theories of representation that underwrite it, need to be rethought in light of neural synthesis algorithms that traverse both sets of terms.

Neural synthesis algorithms model audio on the basis that sound can be anything a neural network is capable of learning to recognise and synthesise. In their concatenated translations, from sound to data to text and back again, these methods call for a new understanding of the relation between world, model, and synthesised sound, beyond the binary of imitation and novelty inscribed by theories of standard and nonstandard. Yet the consequences of this shift for electroacoustic music studies have gone unaddressed.

Approach

As a practice-based inquiry into the use of neural synthesis methods in composition, this study proceeds in two stages. First, audio datasets are built from contrasting sound sources to model a population of speculative sound entities that traverse representational and abstract qualities. Second, these synthetic agents are activated within a network of interconnected processes involving a further level of machine learning, modelling morphological interactions among the agents and their orientations in space. Interaction is modelled as a circular system of speculative creativity through which new synthetic sound ecologies can be discovered, departing from unidirectional approaches where humans simply teach machines.

Autoethnography will build an alternative analytical vocabulary for synthetic media, extending attempts to move beyond spectromorphology and engaging with Joanna Demers' notions of reproduction, construction, and deconstruction.

Methods

Practice-research (composition and evaluation), autoethnography, neural synthesis, supervised and unsupervised machine learning.

Working bibliography

Reading for this study is being added as it proceeds.