Immersive Pipeline

Immersive Pipeline at Goldsmiths University


With the recent interest in virtual reality, questions of immersive experience have focused on VR and associated hardware typified by the headset. While VR headsets are able to provide the simulation of total immersion by delivering high resolution stereoscopic image unfettered by environmental conditions, it is those very environmental aspects that are lacking – a sense of physical space, a sense of embodiment, and a sense of the social. In this sense, VR in the classical sense is an isolated experience for the user.

Creating immersive audiovisual environments in physical space us give back spatial, corporeal, and social senses. A number of formats for immersive projection have existed, including the CAVE automatic virtual environments of the 1990s, and the more recent full dome facilities. We can even think of planetariums as immersive theatres for creative content beyond constellations

Immersive Pipeline is a research project led by Atau Tanaka initially funded for a period of 6 months by the AHRC/EPSRC Research and Partnership Development call for the Next Generation of Immersive Experiences.  Artist researcher,  Dr. Blanca Regina joined the team in January 2018, and the software developer Pierre Tardif.

Partners include:
Société des Arts Technologiques, Montréal
–  Institute for Sound and Music, Berlin
–  Recombinant Media Labs, San Francisco
–  Sonic Womb, London
–  GarageCube, publisher of MadMapper and Modul8, Geneva
–  Derivative, publishers of TouchDesigner, Toronto
–  NSC Creative, UK distributors of Front Pictures, Leicester

Immersive Pipeline Workshop - 12&13 Feb 2018

The project its focused on the research and production of pipelines and translators for the authoring, sharing, and  touring of immersive media performance works. The aims are to:

1) Create content production pipelines to allow artists to concentrate on the creative aspects of their immersive works, and

2) Facilitate the conversion and translation of existing works from one format to another, for example, CineChamber to Hexadome. SIML could become a leading UK studio for the production of new immersive works, and a screening facility to introduce UK audiences to the catalogue of immersive works created for the CineChamber, Satsophere, and ISM-Hexadome.

The main goals of this project are to provide research platform and to answer to the following questions:

       Can we identify best practice in immersive production techniques in order to define an authoring workflow for new immersive audiovisual performance works?

       Can we draw upon the wealth of existing practice for it to drive technology development, instead of technology determining creativity?

       Can we create technologies for the translation of creative work from one immersive format to another in such a way to maintain original creative intent, and liberate works from being inadvertently site specific? 


Immersive Pipeline –  Research session – 12&13th February 2018 at Goldsmiths London
Coordination: Atau Tanaka & Blanca Regina

Monday 12th February from 15:00 to 18:30
15:00 – Presentation by Atau Tanaka (Goldsmiths)
15:15 – Talk withPaul Mowbray (NSC Creative)
15:30 – Talk with Naut Humon (Rml CineChamber)
15:45 – Talk with Luc Courchesne (SAT)
16:00 – Tea time – short break –
16:30 – Talk with Julian Henriques (Goldsmiths)
16:45 – State of the Art and identifying needs
(Group discussion)
17:45 – Demonstration of SIML with Julian Henriques
18:30 – End of the session

Tuesday 13th February from 10:00 to 14:00
10:00 – Presentation by Blanca Regina (Goldsmiths)
10:15 – Talk by Nick Meehan (Hexadome)
10:30 – Talk by Francois Wunschel (MadMapper)
10:45 – Talk by Karel Doing
11:00 – Tea time – short break –
11:15 – Brainstorming, problem solving
(Group discussion)
12:45 – Lunch
13:15 – Hands on session with participants & artists
13:45 – Closing considerations with Atau Tanaka
14:00 – End of the session

Photos by Alessandro Vangi
Edited by Pierre Bouvier Patron

         Event 12/04/2018 . Link to Full information


This is the first public event of Immersive Pipeline. The programme is curated by Atau Tanaka and Blanca Regina and presents immersive audiovisual performances and a screening. It showcases some of the work in progress of international artists and Goldsmiths students. Includes a presentation with one of the pioneering figures in the field of immersive audiovisual, Naut Humon from (RML), San Francisco.


LILLEVAN + ATAU TANAKA present Le Loup, Lifting & Myogram
ALBA CORRAL + ALEX AUGIER present Germination
BRYAN DUNPHY present Ventriloquy II
Screening introduced by Naut Humon
Halveplane by MASAKO TANAKA + OVAL (RML CineChamber)

Lillevan and Atau will perform three short audiovisual pieces looking at the body as a source of sound, image, and corporeal expression. A sensor system detects the electromyogram (EMG) signal, electrical impulses from the nervous system causing muscle tension, and renders a musical instrument the performer’s own body, allowing him to articulate sound through concentrated gesture. Live image manipulation parallels this, situating the performer in an immersive sound/image space. Le Loup: A single short recorded sample of howling wolves are stretched and transposed, creating a continuous growling. Lifting: Whistling oscillators are modulated in frequency and amplitude, recalling the classical Theremin. Myogram: A direct sonification of muscle activity where we hear the neuron impulses of muscle exertion as data. Throughout the piece, the raw data is first heard, then filtered, then excite resonators and filters. This piece was created in musical collaboration with Miguel Ortiz. After meeting at Tacheles in Berlin 25 years ago, this performance is Atau and Lillevan’s first collaboration. The groundwork was laid in London in 2015 when Lillevan performed with Adam Parkinson in the Sonics Immersive Media Labs (SIML) at Goldsmiths.
Lillevan is an animation, video and media artist. He is perhaps best known as founding member of the visual / music group Rechenzentrum (1997-2008).Lillevan has performed and collaborated with many artists from a wide array of genres, from opera to installation, from minimal electronic experimentalism to dance and classical music; performed and exhibited all over the globe, and at all the major media festivals.
Atau Tanaka conducts research in embodied musical interaction. This work takes place at the intersection of Human Computer Interaction and gestural computer music performance. He studies our encounters with sound, be they in music or in the everyday, as a form of phenomenological experience. This includes the use of physiological sensing technologies, notably muscle tension in the electromyogram signal, and machine learning analysis of this complex, organic data. At the other extreme, he studies user experience through ethnographic methods of participatory design where activities of workshopping, scenario building, and structured brainstorming lead an understanding of a mediums affordances in bottom-up, emergent ways. Atau has previously been Artistic Ambassador at Apple, researcher at Sony Computer Science Laboratory, and professor and guest professor in Japan, France, and northeast England. His work has been supported by the European Research Council (ERC), Horizon2020, and both science and humanities sections of Research Councils UK (RCUK).

The audiovisual work that brings together the aesthetic and technical researches of Alex Augier and Alba G. Corral to converge towards a coherent transversal artistic proposal, emphasizing what appears to be an original and specific path to hypermedia. Sound, image, space and time merge in a poetic, dreamlike and organic hypermedia, deployed in space, time of performance and under the gestures of the two performers. The conception of this hypermedia implies that one dimension does not take precedence over the other but is completed and enriched. The proposal puts aside achievements where the sound or image is used in their aspects of illustrations, in their informative or narrative functions. The project also integrates the issue of correspondence with the actions of the performers, the stakeholder of the work.

Alba G. Corral is a Visual Artist and creative coder based in Barcelona. With a background in computer engineering, Corral has been creating generative art using software and coding for the past decade. Her practice spans across live performance, video, digital media and installation, exploring abstract narratives and expressing sensitivity and taste for colour. By combining generative systems with improvised drawing techniques, her digital language becomes organic, creating mesmerising digital landscapes. Corral is known for her stunning live audio-visual performances where she integrates real-time coding and drawing in collaboration with musicians. . Her works has been exhibited at festivals and events in Europe, México, Japan & EEUU.

Alex Augier
is an electronic musician based in Paris. His work focuses on hybrid digital aesthetics in a musical and transversal perspective, including sound and visual elements. These elements interact with the space and take mainly the form of singular audiovisual performances. He defends an overview of the creative process where design, programming and technology are an integral part of the artistic project. His works have been presented at international festivals including Ars Electronica (Linz/AT), SAT (Montreal/CA), L.E.V (Gijon/ES), Scopitone (Nantes/FR), Media Ambition Tokyo (Tokyo/JP), Mapping (Geneva/CH), Open Source Art (Gdansk/PL), Multiplicidade (Rio/BR), Athens Digital Arts Festival (Athenes/GR), Nemo (Paris/FR), Sonica (Glasgow/UK)…Music work is edited by DAC Records, Kolonia Artystow.

Ventriloquy II

Bryan’s work uses interactive machine learning techniques to explore the relationships between sound and image. As a demonstration of the Immersive Pipeline, Bryan will present a work for six projectors and surround sound that explores our perception of cross modal ventriloquism. The audio and visual elements are generated simultaneously with parameters mapped to the performer’s position in 3D space using a neural network performing regression analysis. This allows for the expressive control of both sound and visuals in real time. The audio and visuals in this work are given equal importance with neither one merely supporting the other. Furthermore, there is a fundamental dependency between the audio and visual elements, neither one could exist in isolation and still form a coherent artistic statement. These elements collide in the audience’s perception to become more than the sum of their parts.

Bryan Dunphy is an Irish audiovisual artist based in London. His work has been performed and screened at venues across the UK and Ireland such as Seeing Sound (Bath), The National Concert Hall (Dublin), the Samuel Beckett Theatre (Dublin) and the Darklight Film Festival (Dublin). Recently, his digital artwork was used in the BBC documentary Weapons of Mass Surveillance (2017). He holds a B.Mus from NUI Maynooth and an M.Phil from Trinity College Dublin. He has been a recipient of An Chomhairle Ealaíon’s (The Arts Council of Ireland) Travel and Training Award for 2016 and 2017, which has allowed him to pursue a PhD in Audiovisual Composition at Goldsmiths, University of London.

Naut Humon
is the Founder and Artistic Director of International Creative Operations for Recombinant Media Labs (RML) whose spatial AV media project the RML- CineChamber propagates a panoramic performance production platform for image and sound surround residencies & festival formats. During the 90’s thru the main 2000’s Naut was also curating artist content for Asphodel Records in New York and SF and for select portions of the annual ARS Electronica Festival in Austria where he also was the US coordinator of their Digital Music and Sound Art jury category for ten years. Having performed in the eighties with the avant- garde music group Rhythm & Noise, he later formed the Surround Traffic Control network; the aural optic incubator that gave birth to today’s CineChamber apparatus that recently returned from overseas to California in 2016 / 17 with the annual autumn
‘Captured in the data -bloodstream of our auraloptical organs arrives the meticulous weavings of Tokyo born Masako Tanaka, a visionary panoramic pioneer of the CineChamber’s ten channel format . The animated fibers that she threads together feed a formation fabric of chaotic complexity and gravitational mass that pervade every surface of RML’s screen arena. The marvel and full measure of her spatial detailings reference what an insect with multiple eyes might experience as these clouds of granulation organisms of thousands of cinematic fragments flock and swarm with wild and unpredictable behaviors. One can view these immersive molecular mappings over many repeated showings and witness new unforeseen elements each and every time. Masa’s optically insistent continuum depicts a transformational habitat of time shifted frames through rotating layers of a visual kind of energy – the room is alive, impulsive, off axis, always breathing, as the floodgates of image offsprings and particle fibrillations subdivide and spill forth. Masako Tanaka also worked as the main video production head for the Multiple Otomo Project on the Asphodel Label where she was the primary editor, assembler and visual composition coordinator & adjuster. 13 Markus Popp’s sound nuggets which form the sonic underpinning for Masako Tanaka’s video personifications stem from Markus’s Oval Commers audio file archives from the early 2000’s era. Performed and released under the name Oval, Popp’s music has been at the forefront of experimental electronica since the early ’90s. Originally a three-piece , the Oval group became known as propagators of the so-called ‘glitch’ movement; disdaining conventional electronic instruments, their early albums were composed through the creative abuse of compact disc players. Deliberately mutilated discs were used to create a remarkable range of fragmentary sounds, which would then be painstakingly processed, compiled and assembled into musical pieces with a strange allure all of their own. As Markus moved on with Oval alone he adopted the Apple Powerbook to run his specially commissioned software which has been used to create a series of interactive ‘installation objects’ for public spaces. On the hard drive of his Powerbook, Popp had created an archive consisting of tens of thousands of tiny sound fragments which Masa dipped into a smaller portion of to re- compile from to utilize a longer reoccurring & evolving musical soundtrack.’