1. Introduction of the themes
In this respect, the aim of the present dossier is to define a framework for inquiring into the notion of audiovisual environment on the contemporary electro-scene, by linking electro acoustic and electronic music to visual image, installations, and other emerging projects in the area of media arts [AA.VV. 2010]. This research intends to address a rather common misunderstanding concerning the relationship between sound, music and images. In fact, we aim to demonstrate that audiovisual art is not just the simple combination of music and images, but it has to do with a space opened by an active interaction between different forces. It is, in other words, a space preserved for something that may come unexpectedly. In this sense, what is unexpected is something that may not have an immediate meaning, but nevertheless acquires an indescribable inner pulse that vibrates through a tension. At the bottom of this pulsation it is possible to recognize a particular energy that must be developed and can take the shape of a sound or even of something else, organising a set of different points of convergence between various dimensions. A sound pulse, for example, can pass under the treatment and control of the image and vice versa [Hegarty, 2014]. Thus, relationships between these elements are not just a matter of contiguity, but they rest on a process of composition that will be at the core of our thinking and conversations with artists.
2. On perception
The idea of presence is not limited to the visible sphere. In order to identify the presence – and all the entities deriving from it – we need a deeper and more acute attention to all the phenomena that stimulate our senses. In fact, there is a sort of landscape full of visual and sonic inputs determining – unconsciously – our perception. In this sense, I’m referring to those tiny perceptions [Leibniz, 1921; 39] that are felt at the limit of our senses, and yet can shape our experience. These intangible implicit perceptive faculties modify our field of experience and give it a tangible quality. There will never be – paraphrasing Leibniz – a conscious perception without the inclusion of an infinite series of tiny perceptions capable of altering and modifying the process of macro-perception taking place at any one point, and at the same time preparing for the next perception.
From the point of view of aesthetical categories, the work of artists such as Granular Synthesis (Kurt Hentschlager and Ulf Langheinrich), Ryoji Ikeda or Ryoichi Kurokawa, Herman Kolgen, or Elio Martusciello is marked by a deep inquire into territories that deal with the discussion of sight and sound perception, working on the boundaries of subliminal processes that are able to organise visual and auditory architectures so to create an immersive ambience for the spectator. In other words, these works develop an organic and compositional dimension, where the subliminal part plays a central role in giving consistency to the whole work, and flows into the spectators’ perception like an underground current. This subtle dimension takes shape in the folds of sound and images, drawing a meaningful constellation and letting a tension emerge between an evident and tangible dimension – that of the visible electro-scene – and an intangible and underground one that makes it possible.
With the precious help of artists’ interventions and contributions, one of the things this project aims to call into question is the profound qualities and the nature of the relationship between sound and image. In fact, in areas such as that of sound installations or video arts, a common misunderstanding arises when artists think they can create an artwork by simply mixing together image and sound.
The perspective we propose, instead, is based on the fact that we believe that an audio-video synchronization alone cannot be considered a work of art in itself. On the contrary, it is necessary to imagine a process of composition in which the aesthetic sensibility should work freely and harmonically on the two layers. This is the first point in order to locate our work within the field of contemporary audiovisual performance and installations, and it is a point of view that reflects the artistic scene we want to focus on.
To think in terms of composition means to be aware that the intensity of a live electronic performance is reached through musical and visual elements, which are nothing but the surface of the work. A deeper and more complex process lays underneath: it doesn’t operate neither in visual nor in acoustical terms; but rather it is based on time, space, movement, forms and organic qualities of matter. In other words, this is the definition of an invisible background texture, which makes the visible and the audible stand out and be perceived. What emerges here is a subliminal dimension of the artistic communication, coming out of the artwork and overwhelming the audience (J. Demers, 2010; 45 and F. Dyson, 2014; 56 sgg).
In this context we will consider the work of artists that focus their research on certain themes of particular interest. We will tackle three main themes of central importance: a first element is the creation of immersive environments, that turn the scene into an amorphous environment of perception and alteration, similar to those designed, for example, by Granular Synthesis and Edwin van Der Heide; a second aspect concerns the reinterpretation of the structural models of nature as a metaphysic dimension of the matter, as for the perspective adopted by Ryoji Ikeda or Ryoichi Kurokawa; a third perspective examines the strategies to extend the visibility and audibility, as for works by Herman Kolgen, Thomas McIntosh and Elio Martusciello, in which electronic devices are employed in order to “visualize” a invisible and inaudible dimension of the real.
2. 1. Reconfiguration: the effects of immersive environment
a) Performances and installations by Kurt Hentschlager and Ulf Langheinrich’s Granular Synthesis collective have gradually become a paradigmatic model for immersive works. A work such <360> (2002), for example, is a spectacular audio-visual architecture that completely immerses the audience in a digital sound and video ambience – what the artist considers to be the electronic counterpart to a natural maelstrom. Originally conceived as a hybrid structure it combines material and immaterial components to create a new form of virtual theatre. This environment surrounds the audience with 12 screens of video projections perfectly synchronized with the soundtrack. By means of modulations of light, colour and sound, the entire ambience flickers and whirls: sometimes it creates a soothing space of meditation and sometimes a heightened sense of intense excitement. The sound is partly displaced by multiple numbers of small hidden loudspeakers, integrated with a battery of subwoofers that together produce a sound mass surrounding the spectator, and prevent him or her to recognize distinct sound events at localizable positions. In this immersive field, the perception of the human body is modified, and the result is a direct immersion into an electronic flux. Thanks to our perception, and to the subsequent process of analysis, we gradually combine events that at first have nothing to do with each other, except for the fact that we happen to sense them at the same moment in time.
b) The Dutch artist Edwin van der Heide explores the limits of the composition focusing his research on the different ways in which an immersive environments can affect the perception of space and time. His work DSLE (2012) is an audiovisual environment that takes us to the limits of our visual and auditory perception. By exploring all the transitions permitted and created by electronic devices, it concentrates on the possible interrelations between sound and light. In presenting the work on his web-site, van der Heide wrote: “Although theoretically all sounds can be seen as sums of multiple sine waves, music in general is often too complex to result in interesting visual patterns. Different models are being used to give light a spatial and time-based component comparable to the spatial behaviour of sound. DSLE is using an octophonic loudspeaker setup and forty-four independently controlled led lights. The lights are shining towards and therefore lighting up the screen that surrounds the audience. The spatial nature of the setup and the detailed level of control creates the possibility to control and manipulate our perception of space”.
By combining audio and visual elements the spatial perception of sound is often reduced because of the two-dimensional nature of the images, in opposition to the three-dimensional nature of sound.
The behaviour and artefacts of our senses represent the starting point for the composition of a transforming space. Moments where sound and light seem to interrelate with each other perfectly are interrupted by moments where the spatial perception of sound and light contradict with each other, and lead to an ambivalent way of perceiving space. Parts where changes are imperceptible and almost invisible alternate with parts in which the retina itself is overstimulated. In such a work presences are visually and acoustically organized by means of figures of light and sound architectures that pulsate according to a rhythmic pattern. The audience is immersed in this architecture in constant mutation and every variation on the environment produces a re-orientation of the spectator’s perception.
2.2. Transfigure the nature – the meta-physical dimension of matter
a) Ryoji Ikeda’s new project supersymmetry (2014) presents an artistic vision of the reality of nature offering an immersive and sensory experience to the visitors. All those visuals and sounds result from a composition of an immense quantity of data that exist in our world, though they are mostly invisible and inaudible. The key element is always composition – or re-composition –, the structure of the nature, the matter that organises the nature. In this context, Ikeda works around the algorithmic dimension of things. He composes any kind of elements, from numbers or phenomena to material “objects”. His process for each new work is a trial-and-error and back-and-forth way.
This project is a series of new works conceived of as “installation” versions of his performance work “superposition” (2012 till today), and represents a platform where to update the progresses and outcomes of his forthcoming residence during 2014-15 at CERN in Geneva, the world’s biggest centre for particle physics. On his web-site, Ikeda wrote about the project: “In particle physics, Supersymmetry is a proposed extension of space-time symmetry that relates two basic classes of elementary particles: boson and fermion, and predicts a partner particle in the Standard Model, to help explain why particles have mass. The Standard Model has worked beautifully to predict what experiments have shown so far about the basic building blocks of matter, but physicists recognize that it is incomplete. Supersymmetry is an extension of the Standard Model that aims to fill some of the gaps. It predicts a partner particle in the Standard Model. These new particle would solve a major problem with the Standard Model – fixing the mass of the Higgs boson. If the theory is correct, supersymmetric particles should appear in collisions at the LHC of CERN, Geneva”.
According to the theoretical perspective we follow, this project represents a real immersion into the matter. Such invisible substances have a phantom-like behaviour, a spectral nature. Phantom-like as well are the data-derived images and sounds that inform the intelligent production of this resonant machine which Ikeda has put into play, starting from a mathematical programming of equations driven and related within a cosmos that is, on its turn, organised by the electronic order spectres and the pulse of their matrix: images and sounds synthesizers that work directly upon the code [Canty, Bonin, Chatonsky, 2010; 89 sgg]. In this manner, Ryoji Ikeda presents a series of devices that create a landscape constantly re-constructed. The point is to make palpable all the data that invades our world, and to realize a stratified translation of them from one medium to another. Sound enters the image and vice-versa, and what can be perceived is the vibratory moment of this passage.
b) In the same context, concerning an “aesthetic of transfigurations”, we can include Rheo (2009), the project by Ryoichi Kurokawa. This work is composed of concrete images, field recordings and generative audio/animation, and all these elements are abstractly and poetically translated and re-mediated. Everything is shown by means of 3 vertically long screens of High-Definition video and 5.1ch surround sound. The title Rheo refers to the constant flow in nature that is based on the circle of transitions. All things change and flow with the time. Everything runs and nothing remains the same, and what seems to be immutable mutates. We find the beauty and the life (the being) in this flux. Then, the nature of things would lurk in such an alteration.
In nature the alternative patterns of regular and irregular forms create the harmony of alteration and unity. It might be said that we could find the rule of the essential beauty in this continuous alteration. Kurokawa shots the motif of flowing in nature as he depicts the patterns of nature. He reconstructs the nature and represents it in an abstract form. Then, the water, it is the most symbolic motif. It represents the organic circulation in the nature. And also the water relates closely with us as well. We have been in the water before we were born. The water unconsciously evokes our memories. These alterations are something invisible and we cannot recognize it.
Kurokawa’s work incorporates the organic form of nature and traces the motion and the colour. Then, he reconstructs the landscape of nature that has a fluent alteration of time, and he re-presents it with electronic devices. The basis of this design is a volume of human’s memory that hides under unconsciousness. It is horal continuation from time immemorial and the circle of alteration. The organic model, the form, structure, and dimension of time in the alteration of nature creates beauty with some method. “Water” plays an important role in this work as a figural motif. Kurokawa depicts a sort of solidity in its work and incorporates an inter-community of biorhythm with nature. The composition of three screens reminds the traditional painting. The sight can rest on all directions. It enables to apprehend the moving, the flux of time, and the expansion of space. The sound itself undergoes the flowing time with the same process as images. It depicts the nature as a sound-scape. Kurokawa seeks to transform the space as a device into the place where a body as medium makes a proportion with the environment.
2. 3. Extend the visibility | Explore the audibility
a) With another perspective we can approach a composition of digital audiovisual performance and environment such as that by Herman Kolgen. Among the most interesting figures of the artistic panorama, he is based in Montreal (Quebec, Canada). Since 2008 Kolgen has inaugurated a personal research focused on a radical interrogation about audiovisual systems. His versatile work is characterized by an approach towards audio and visual material that we might define “radiographic”, in which the dimensions of the invisible and inaudible take shape by being directly manifested to spectator’s sight. Attracted by an organic view of things, Herman Kolgen is inspired by the internal tension that produces the matter (trans)formation. His work explores liminal territories where boundaries are permeable, where sound and vision fold in on each other and memory and immediate experience blur. Many of his works exist in multiple forms; composed and recomposed as installations, film-poems, performances and collections that are open to being experienced in many ways. Neither his work nor his process is hermetic, he creates open systems where sound and vision blur and contaminate each other; where hearing, seeing and experience occur. Hearing is our most basic and elemental sense. It is intimate, it is preverbal and previsual. We start hearing even before we are born — the sound of our mother’s heartbeat, the rush of her blood as it flows through us. We start to hear well before we see, and the noises, rhythms and murmurs of the world pass through flesh to reach us in the womb. Hearing is instinctual, emotional, and subjective. It is a sort of narrative; it reveals meaning as well as the place of objects in the world. Like vision, it has volume and depth; it describes the physical dimensions of our world. Sound is representational, concrete.
In Inject (2008) – for example – a human body is immersed in a tank. In the course of 45 minutes, the liquid pressure exerted on it increases the neurosensory changes. From its fibre in the epidermal nervous system, the body reacts to changes in viscosity of the liquid chamber. Its cortex gradually loses all knowledge of reality because of a lack of oxygen. The body of the performer becomes a guinea-pig: a material body whose states are the subject of psychophysical kinetic paintings. During this period the body of the performer is embedded in a tank full of water for more than 8 hours per day, ranging from the absence of gravity and lack of oxygen. With the help of various techniques of digital video recording systems and different cameras, Herman Kolgen assembled several sequences in as many captured moments in time, resulting in a series of images that he then met a body of flexible and modular. His research works to make the imperceptible elements perceptible. He is fascinated by what the human eye can’t see; in this direction, he uses macros, high-speed cameras, particles and grains, to satisfy that urge. The neo-real universe emerges, but the general truth will remain believable. The relationship between elements gives rise to tension, magnetism, and phenomenological pressure. Everything appears to be chaotic, but under the microscope, you can see a new dimension of the real.
b) In this same framework, I would introduce Ondulation (2002), a work by the Canadian artist Thomas Mcintosh, developed in collaboration with composer Emmanuel Madan and the Finnish light designer and sound artist Mikko Hynnimen.
In a general way, Ondulation is what might be considered as a composition for water, sound and light. The scenic device is composed of a basin full of water where concentric waves on the surface are created by the sound produced by speakers placed under the structure. The movement of the water, thus, is directly caused by the soundscape of the work that also influences the gradual change of lighting on the device. In this case, the water wave depends on the acoustic wave. Water acts like a mediumor a transformation vector. It becomes a sensitive plate on which the acoustic waves develop and compose the figures of light projected on the platform. At the same time, every variation in the intensity of sound or frequencies represents a different movement on the water surface. On its turn, the light on the screen is modulated by the movement of water into complex visual forms perfectly synchronized with their musical source. The resulting fusion of sensory experiences is a temporal sculpture: a construction of water, sound and light that evolves as a composition in time. Ondulation thus is a time sculpturewhere the ephemeral dimension produced by the device allows time to become visible through a series of vibrations. This game of appearance/disappearance makes it possible to see the sound and to listen to the image; to make it perceivable and to vanish, through a series of gradual changes that call into questions the spectator’s perception.
c) In this operative framework, the project composed by the Italian artist Elio Mastusciello – To extend the visibility (2006-2008) – audiovisual installation – presents an important aspect because it is entirely devoted to the theme of a complete incorporation of technological, natural and cultural resources that interact together in a challenging and complex way. This whole work is based on satellite images of our planet. The view of such images obviously overruns our usual gaze, it reaches the outer limits of the stratosphere and then turns to look back on us and our geographies. In this torsion we can’t recognise anything but every image can be read at multiple levels: the micro and macro have become the same thing. This is a new formula for visualising things. The satellite images are lights refracted through the tears of the floating eyes of new empires. We had expected “space” to be turned outwards, but it turns out to meet itself round the back, and now what we are getting closer to is our own life – a moon of Sirious or the light of a new constellation…While watching, we become finally invisible, a barely perceptible sign of a track in the snow on a thousand kilometre long frozen tundra that could just as well be the surface of the desert or a leopard skin [Martusciello, 2009; 3 sgg.].
The project offers two different modalities for its fruition: on one hand, the work is presented as a video, on the other, as an installation environment. In this second modality, the spectator walks through the space looking to the videos on the screens displaced all around the ambience. The audience is on the earth and at the same time can look the planet in an abstractive resolution. These images of our planet, multiplying during the “ecological data” era, seem to darkly foresee a disappearance: the critical situation related to irreversible pollution. This is a way for presenting in a new form what we already have under our eyes and ears, but that we are not able to see and hear or feel yet.
In this context, this dimension is mainly found through the folds of images and sounds, and it puts a strain between the perceptible and imperceptible parts [Ikoniadou, 2014; 45-66].
There is a geological metaphor we consider important. In this respect, the dossier behaves like a seismograph, for it is able to record the vibrations passing through the audiovisual composition deep into the ground, and finally it detects their superficial tracks on the environment of contemporary scene.
The materials we are going to collect – according to the panorama presented in the previous pages – come mainly from two trends recognisable on the multimedia scene: on one hand, an attention to the internal organic line in which the materials are organized in order to outline a narrative fabric; on the other hand, an external organic line in which the materials operate towards abstraction. Nevertheless, the two layers above mentioned (the sound and the visual) are not opposed to each other, but they’re both inscribed into a system of shades. These are two different strategies to approach the matter [Salter, 2010; 103].
The organic line, common to both perspectives, is also – and above all – present in the quality of the materials that will be crossed. It is a horizon that opens to the dimension of the ‘organic sound’, moving closer to the electro-acoustic sounds, or taking shape in a visual imagination that looks at microbiology or at process of transfiguration of the matter. More than that, the structure of this dossier would provide a reflection upon the concept of time itself. To this extent, the point is to follow the development of an operative trend that goes from the forms of time to the time of forms.
3. On the Atmosphere
Immersion, as an aesthetic concept, offers an overwhelming experience to the audience, more involving than a casual look or a listening from a safe distance. Thus immersion suggests a heightened state, and asks for an involvement, wherein the visitor is becoming integral part of the environment, and the work can be considered complete only with- and in the experience of the visitor. Consequently, at least at first sight, we are talking about a solitary moment, an existential yet sensual state. As we hear with our entire body, not just the ears, the sensation of being surrounded by sound quickly takes on a quality of “being” the sound itself. Depending on whether the content resonates with us, as much as the sound waves, we might have either a joyous or annoying experience. The psychological effects of an immersive work become prominent as analytical perception is bypassed, or in better words is delayed to after one has left the “sphere”.
In this sense, the environment thus acquires an immersive quality. Sound and light create a subliminal effect by radically influencing the audience’s perceptual system through the use of low and high frequencies, and of flashing lights. The spectator is thus immersed in an environment marked by continuous vibrations, like in the case of the works by Ikeda and van der Heide. Here, the inaudible and the invisible have an effect on the audible and visible because every single light or sound (no matter how imperceptible), contributes to the creation of a tension in the basic texture of the works [Thompson & Biddle, 2013; 73 and Jones, 2006; 65].
In conclusion, they are spread presences that become atmosphere.
This atmosphere is nothing else but the temperature of the stage work, the transient way of encounter between bodies and other entities. It’s a state of elements, that offers itself not really in a visible way: is more like a sensation to be experienced. The typologies of atmosphere created by the scene represent different ways to affect the audience, producing some effects on it, by resting on images that speak directly to the emotive perception and survive in the memory of the stectator. What impress people the most are the immaterial architectures of light, the pulsation of sound – colour and glow: a perfect disposition of environmental sensations like that produced by the immersive space projected by Granular Synthesis or Martusciello.
It is in this way that the affective space creates atmospheres.
Atmosphere – just like the presence of light and sound events caught at the limits of perception – is related to the simple fact that something exits. It is something impalpable, elusive, operating at a different level from that of things. Inside reality perception never really stops. In this sense we could argue that a tangible and distinct atmosphere cannot exist, unless it integrates a potentially infinite number of smaller atmospheres – of different quality – capable of dislodging each other [see Canty & Bonin & Chatonsky, 2010; 75 sgg.]. An atmosphere cannot be rationally understood; it is felt just like a sensation. It is like a presentiment that will never come to pass, or a sudden intuition – like a glimpse – or nothingness. So the spectator’s role is that of pre-sensing, of glimpsing. Thus, in order to react to these stimuli, the spectator too has to undergo a re-organization of his/her sensorial geography; they have to learn to perceive differently and feel in a new way.
Bibliographie
– AA. VV., Espaces, Paris, Editions IRCAM, 1994.
– A.A, V.V., Permanence through Change. The Variable Media Approach, New York, Guggenheim Museum Publications, 2003.
– AA.VV., Live A/V performance audiovisuelles, «mcd», hors-série, 2010.
– Alberganti, Alain, De l’art de l’installation : La spatialité immersive, Paris, Harmattan, 2013, 410 p.
– Bernas, Steven et Catherine Bouko, Corps et immersion, Paris, Hammatan, 2012, 164 p.
– Bishop Claire, Installation Art: a Critical History, Londres-New York, Routledge, 2005, 144 p.
– Canty, Daniel, Vincent Bonin et Grégory Chatonsky, Angles arts numériques Elektra 10_ Essais | Angles digital arts Elektra 10_ Essays, Montréal, edition festival Elektra, 2010.
– Cox, Christopher et Daniel Warner, Audio Culture, New York-Londres, Continuum, 2006, 472 p.
– De Oliveira, Nicolas, Nicola Oxley et Michael Petry, Installation Art in the New Millennium: The Empire of the Senses. Londres, Thames & Hudson, 2004, 208 p.
– Demers, Joanna, Listening through the Noise: The Aesthetics of Experimental Electronic Music, Oxford, Oxford University Press, 2010, 248 p.
– Dyson, Frances, The Tone of Our Times: Sound, Sense, Economy, and Ecology, Cambridge, MIT Press, 2014, 208 p.
– Goodman, Steve, Sonic Warfare: Sound, Affect, and the Ecology of Fear, Cambridge, MIT Press, 2012, 270 p.
– Grayson, Cooke, Liveness and the Machine, Improvisation in Live Audio-Visual Performance, « Screen Sound», no2, 2011.
– Hentschlager, Kurt & ZKM, Granular Synthesis. Immersive works, Ostfildern, Hatje Cantz Verlang, 2004, 46 p.
– Hegarty, Paul, Rumour and Radiation: Sound in Video Art, Londres-New York, Bloomsbury Academic, 2014, 216 p.
– Ikeda, Ryoji, Formula, Londres, Forma, 2005.
– Ikeda, Ryoji et Carsten Nicolai, cyclo, Id, vol.1, Berlin, Die Gestalten Verlag GmbH & Co, 2011.
– Jones, Caroline, Sensorium: Embodied Experience, Technology, and Contemporary Art, Boston, MIT Press, 2006, 268 p.
– Leibniz, Gottfried Wilhelm, Nouveaux essais sur l’entendement humain, Paris, Flammarion, 1921, 448 p.
– Lockwood, Annea, «Sound Explorations: Windows into the Physicality of Sound», Leonardo Music Journal, vol. 19, 2009, p. 44-45.
– Martusciello, Elio, To Extend The Visibility, Surrey, ReR Megacorp, 2009.
– Meulen B. C., D. Tavy et B. C. Jacobs, «From Stroboscope to Dream Machine: A History of Flicker-Induced Hallucinations», European Neurology, vol. 62, no5, 2009.
– Moulon, Dominique, Art contemporain nouveaux médias, Paris, Scala, 2011, 128 p.
– Nicolai, Carsten, Unidisplay, Berlin, Die Gestalten Verlag GmbH & Co., 2013, 104 p.
– Roads, Curtis, Microsounds, Cambridge, MIT Press, 2001, 423 p.
– Sakamoto, Ryuichi et Shiro Takatani, Life. Inivisible, inaudibile…, Tokyo, NTT publishing, 2007.
– Sakamoto, Ryuichi et Shiro Takatani, Life. TEXT, Tokyo, NTT publishing, 2010.
– Salter, Chris, Entangled: Technology and the Transformation of Performance, Cambridge, MIT Press, 2010, 500 p.
– Toop, David, Ocean of Sound, Londres, Serpent’s Tail, 1995, 320 p.
– Thompson, Marie et Ian Biddle, Sound, Music, Affect: Theorizing Sonic Experience, Londres-New York, Bloomsbury Academic, 2013, 288.