Artist statement
I am an interaction design researcher. My research revolves around design experiments in interaction design. I engage in experimental design processes, resulting in the production of a variety of interaction designs with a strong focus on affective experiential qualities.
Philosophical Insights into Actual Affective Design
To start this Conversation, as an artist of interactive and immersive art, what role does affect and/or emotion play in the production of your works of art or design?
The processes and the designs both become vehicles for knowledge production as a kind of research through design (Frayling19931) or research-creation, feeding back into the general field of interaction design, affect theory and the coupling between the two in the exploration and design of what I term affectively engaging interfaces. Within interaction design and Human-Computer Interaction HCI more generally, a number of people have been exploring affective aspects of the interaction, most notably under the heading of Affective Computing2.
I work with a different notion of affect, primarily fueled conceptually by the philosophy of experience of Brian Massumi which presents a concept of affect building on Spinoza, Bergson, James, Simondon, Deleuze & Guattari, Stern and Whitehead. In my work, I have been transforming the philosophical insights into actual affective design concerns to be experimented with in the design work. As a consequence, to come back to the question, all my design experiments are inherently affective-led inquiries in interaction design, where the affective concerns feed into the theoretical and practical explorations.
Then I suppose that you refer to the following definition from Brian Massumi in his introductory notes of Thousand Plateaus by Deleuze and Guattari (2005 (1989)?
« AFFECT/AFFECTION. Neither word denotes a personal feeling (sentiment in Deleuze and Guattari). L ‘affect(Spinoza’s affectus) is an ability to affect and be affected. It is a prepersonal intensity corresponding to the passage from one experiential state of the body to another and implying an augmentation or diminution in that body’s capacity to act. L’affection(Spinoza’s affectio) is each such state considered as an encounter between the affected body and a second, affecting, body (with body taken in its broadest possible sense to include « mental » or ideal bodies). »
Yes, exactly – affect as a prepersonal – or preindividual – intensity, which cause a transition between states of capacitation. The very basic idea about « an ability to affect and be affected » is, I believe, at the core of contemporary concerns around interactivity. It’s so simple, and still offers such a rich rethinking around some of the basic notions we are dealing with in interaction design, opening a new field of questioning in the discipline, as Massumi also talks about :
« The notion of affect does take many forms, and you’re right to begin by emphasizing that. To get anywhere with the concept, you have to retain the manyness of its forms. It’s not something that can be reduced to one thing. Mainly because it’s not a thing. It’s an event, or a dimension of every event. What interests me in the concept is that if you approach it respecting its variety, you are presented with a field of questioning, a problematic field, where the customary divisions that questions about subjectivity, becoming, or the political are usually couched in do not apply3. »
Exemplification with Ekkomaten
To be more concrete about ‘transforming the philosophical insights into actual affective design concerns’, may you share an example of this process please. Then would you say that they emerge more during the creation, the production or the exibition stage?
That’s a very good question indeed. Here is an example. Ekkomaten is an interactive sound installation, which was designed to give people an experience of the 18th century city of Aarhus through six auditory echoes from the past, available through a listening machine situated in the city4.
This project was undertaken in the research center Digital Urban Living, where we entered into a collaboration with a historical festival. The brief was, very generally speaking, to use interactive technologies to bring the 18th century to life using interactive means. We started out with the basic idea of designing an affectively engaing interface for listening to the 18thcentury city of Aarhus through an interactive sound design.
From the outset we were working directly with affective design concerns. On a more concrete level, this fed directly into the experiments with the crafting of the interaction, i.e. the design of the physical Ekkomaten machine, and the feeling of interaction with the digital soundscape. It also fed into the actual creation of six different radio plays that were accesible as echoes from the past in a larger soundscape.
Concerning the interaction, we spent a lot of time creating an interface that would demand bodily activation, would, in fact, resist interaction, as opposed to the seamless swiping on a smartphone. Here, the affective outset was used to conceptually unfold the richness of experiential qualities that might be pursued in the design work, opening, as I was talking about above, a problematic field for experimentation. In the creation of the soundscapees, the notion of affect resulted in a move away from creating factual stories – and to instead focus on craftin sonic situations that would immediately engage the listeners affectively in the auditory universe of the 18th century.
When Ekkomaten was on display, and in the work we have been doing afterwards to make sense of how people reacted to and interacted with the installation, we have been able to empirically unfold the different social, experiential and narrative spaces emerging around the affective interaction with the machine. Clearly, people have different experiences when interacting; some only touch the machine and do not engage, others mostly explore the interaction by turning around the machine rather than listening, others really take the time to listen to all the echoes. We do believe, however, that it can be argued that – at least to some – Ekkomaten perceptually and affectively engages its users in the sonic exploration of a narrative space emerging at the intersection between fact and fiction, echoing people’s everyday lives from the 18th century in our present day society. We have also made the argument that Ekkomaten can be understood as a designed electronic object that can powerfully activate a design fictional potential concerning the use of technology in our everyday society in a more poetic sense.
In the Ekkomaten project, affect theoretical concerns set the stage for design experiments, focused on a distinct set of experiential qualities – but after seeing how people actually interacted with Ekkomaten, through observations and interviews, we have been able to add a more elaborate conceptual layer to account for the actual experiential effects and affective exchanges catalyzed through people’s interaction – and this knowledge has been used to sketch out new design experiments. In another project, Echoes from Møllevangen, the Ekkomaten machine, for instance,has been used to orchestrate a collective listening process in a residential area, Møllevangen, in Aarhus, to activate the auditory sensibility of the inhabitants and use the technology to state a differnet kind of participatyr, community storytelling.
Affect and / or Emotion?
What difference(s) are there for you between affect and emotion?
The ‘easy’ answer to this would be something like; emotion is qualified or recognized affect; the affective hits you non-consciously, and the emotive is the conscious recognition of particular affective reactions.
A lot of time has been spent separating affect from emotion, to better build a vocabulary for articulating how affect works, and this has been extremely productive in a number of ways. However, I think it’s important to continuously reflect on the full ‘experiential continuum’:
« Experience is a continuum. All its dimensions are always all there, only differently abstracted, in different actual-virtual configurations, expressing different distributions of potentials5. »
Starting from the idea of a full experiential continuum entails looking more closely into the relation between the affective and emotive. In an article from 2012, From Signal to Signification in Interactive Environments, I have been exploring this, building on the work of Gilbert Simondon.
Simondon states that affectivity can be considered the foundation of emotion by taking the charge of the preindividual nature and making it into a support for the collective individuation. In turn, emotions are for affectivity the discovery of a superior order, of a synergy moving to a higher or more stable metastability. There is resonance between both affect and emotion. In fact, Simondon argues that emotion is integrative and ‘‘more rich’’ than affection. I think it’s important to not fall into an either/or discourse here – the same, I would say, might go for the non-representational and representational – it’s really about unfolding the different exchanges and dynamics going on here. Simondon has an interesting take on how this might be understood, as the relation between signal and siginfication, which I take as a starting point for analyzing the experiential dynamics in relation to this larger experiential continuum. On the basis of this, I argue that « …focusing on signification opens a conceptual path for exploring interactive technologies and environments in terms of the individuations and lived relations that occur through our interactions with them in experiential terms6. »
What difference(s) are there for you between affect and emotion?
I would rather say that I use the affective vocabulary to address particular experiential qualities – through interactive sound or poetry production and various forms of physical interaction. Rather than producing specific affects, the affective becomes a starting point for unfolding rich experiential fields, creating conditions of emergence for affective exchanges. But future projects would be able to more thoroughly investigate the particular affects or affective reactions emanating from the interaction.
By ‘affective exchanges’, do you mean with participants, with collaborators or with devices?
Yes! At least from e.g. the work with Ekkomaten (see above) and Ink (see below), it is clear to see that these affective exchanges happen on a number of levels : between a person and her interaction with the machine, between a person and her relation to e.g. the 18th century of Aarhus or digital literature, between different persons coming together through the interaction – but also between people and us as researchers; being in ‘the wild’, actually engaging with the people who experience the installations, talking to them about the installation, about the future if digital technologies, literature, observing how they react – you get an immediate feel about what actually succeeds in affecting people, about the capacitations (or not) catalyzed by the installations (or not).
Would you say that your approach is rather intuitive, instinctive or in link with cognitive or psychological sciences or philosophical dimensions, if so which ones?
I am definitely working from a range of (experience) philosophical dimensions, but also from more general foundations from interaction design theory, focusing on experiential aspects of digital technologies, for example McCarthy & Wright’s 2005 book on Technology as Experience from 20047. The experiments are also informed by general themes within interaction design research today, such as Urban Interaction Design, physical/tangible interfaces, critical design – or related to technologies such as 3D projections, Media Facades, interactive tables, sensor technologie and so forth.
Could you elaborate how experience feeds your engagment with interaction design and, if I understand well, how it generates new philosophical foundations?
I might have touched upon this above : my field of inquiry is interaction design, and I always think about the knowledge generated through the design experiments through this field, so somehow I would say that my main contribution is to design theory and philosophy – but I have also been publishing articles and giving papers at aesthetic and philosophy conferences.
I would never say, however, that I am generating new philosophical foundations. Rather, I try to explore through design processes how the different philosophical concepts can feed into the actual design of new interactive technologies and environments, focused on the experiential effects catalyzed on an affective level. Currently, however, I am moving into writing and project collaborations where this knowledge is put to work in different transdisciplinary contexts (e.g. in the Immediations project) – and, possibly new philosophical foundations.
What emotions do you feel towards your work of art and what emotions have you observed in the public and concerning which work of art?
That’s an interesting question, you don’t get that a lot as an interaction design researcher!
The interesting part is that I have developed a quite close relationship to the Ekkomaten machine, which I described above. As mentioned before, it’s a physically engaging and very big installation that we have been taking down and putting up so many times that I have developed a very close relationship to it; it might sound strange, but there is something about the hands-on experiences with this digitally enhanced object that has translated into a sort of caring, which I have not ususally experienced – so in that sense it has been a very affectively engaging installation for me personally.
But the same goes for the public’s reactions to Ekkomaten; it is definitely one of my more succesful designs in terms of creating conditions of emergence for affectively engaging with e.g. the history of Aarhus in the 18th century or the life of everyday people in the neigbourhoods Møllevangen, the two projects where Ekkomaten has been used.
During a conference at the research group « Performativité et effets de présence », UQAM, a few years ago, Zaven Paré, an artist who works with robots, described a similar personal engagment with a robot he used to work with. He developed a ‘friendly’ relationship spending time with it…
Yes, I am not quite sure whether it was a ‘friendly’ relationship, though. Right now, it’s more of an exhausted relationship, since we have been spending so much time with the machine, and are now looking for new design opportunities – but I feel we have not heard the last of Ekkomaten, though.
Do these emotions make you change your artistic intentions, and if so how?
I am not really sure about this; of course, the experiences with these projects feed into new projects; I have a lot of ideas and perspectives I would like to pursue in future experimentation, but whether this has changed my artistic intentions, I am not so sure. Maybe it’s because I rarely talk abouth my artistic intentions; it’s not usually something that goes down very well with the interaction design research community, where we pursue research interests more collectively. But of course my engagement with the different projects feed into new ideas I want to explore in future projects.
Atworks inducing emotions the most
Which ones of your artworks have induced emotions the most?
If Ekkomaten was defenitely the first one, I would like to mentionthe interactive poetry generation machine INK and the 3D projection work we did for a Danish national icon statue Holger Danske. The Journey of Holger the Dane exemplifies affective design concerns in relation to the activation of the statue of Holger the Dane, a national icon, through 3D projections. In this experiment, the concerns were primarily developed as a way to conceptualize affective tensions surrounding both the socio-cultural myth behind the statue as well as the physical location of intervention. According to the legend, Holger the Dane will wake up and become alive if Denmark – or Danishness – is threatened, to fight for his country. Of course, doing a project concerned with bringing the statue ‘to life’ immediately raises a range of challenges; if we really woke the statue up, we would to some extent be implying that Denmark was under threat, which would tap into a very heated cultural and political discussions (that have only intensified in the last months with the refugee crisis in Syria, for instance). As a consequence, we decided to bring the statue of Holger the Dane to life – without waking him up.
This experience was particularly strong at one point in the projection, and also very visible in people’s accounts of the interactive experience. It comes near the end, where Holger the Dane looks as if he is sleeping (around 4.20 in the video). The 3D projection here subtly shifts the contours of the statue so it looks like Holger the Dane’s chest is moving. At the same time, the soundscape plays breathing sounds. This was perceived as being a very organic illusion which made the statue seem alive, although not awake. It seems that here the 3D projection functions as more than a digital layer. Instead it is fusioned with the physical statue to create an affectively engaging field of experience. This fusion of the physical and digital triggers a relational event that is immediately felt as microperceptual shocks. Even though you know it is just a projection, it feels as if the statue is coming to life. This feeling of plasticity feeds into the next sequences non-sensuously and changes your experience of the setting. In this particular sequence, I would argue that the installation functions as a technology of emergent experience (Massumi 2002, p. 192).
INK after Print is an interactive literary installation, which is designed to attract the public into an affective engagement with digital literature’s ergodic qualities. Through their engagement with Ink, people can – individually or collaboratively – produce poems by manipulating three books embedded with a custom-made sensor system. Ink has been designed as an immediately engaging, interactive installation for public spaces that uses performative strategies to draw people into a literary interaction.
In the design of INK, we were working with the resonance between affective and ergodic engagement and the relation between sense and sense making tied to the experience of digital literature, so here’s a different trajectory, but still focusing on designing affectively engaging interfaces. The idea is that the interaction can develop into a deep, continued engagement with the installation through the production of user-generated content – through the resonance between affective and ergodic engagement – drawing users into a performative writing and reading process. Here, the notion of affect is used activate the non-conscious and immediate physical activation of the senses before one can reflect consciously on it. Affective engagement is used to describe how the body is activated in a given situation of interaction, making people act and feel differently.
The affective engagement continuously modulates and backgrounds people’s interaction with Ink. In this way, the affective engagement is developed from the sensorial experience of the installation’s physical dimensions (e.g. the books, the interface and the kinesthetic feedback) and through the interaction directed towards the verbal dimensions of reading, reflection and interpretation. Simultaneously, the ergodic engagement starts from the verbal and develops through the interaction with the physical, sensorial.
Before closing, wich articles would you like to propose to our readers that deal with affect or emotion in regards with your works of art?
From my list of publications, here’s a small selection of articles that I believe can further contextualize what we’ve been talking about in this conversation.
In Understanding Affective Engagement as a Resource in Interaction Design. Proceedings of « Engaging Artifacts » (Fritsch, 2009), I propose affective engagement as a possible resource when designing interactive environments. I mobilize the theory of Brian Massumi and argue that his notion of affect provides a non-informational understanding of affect as distinct from emotion. This leads to proposing a processual and relational account of affective engagement to inform the practical work when designing experiential fields as conditions of emergence in interaction design.”
In Between Experience, Affect and Information – Experimental Urban Media in the Climate Change Debate (Fritsch and Brynskov, 2011), we analyze two interactive climate projects designed at CAVI (Center for Advanced Visualization and Interaction) at Aarhus University and point to some issues that we have identified to be central to designers and users of urban media and interfaces from an affective point of view. The two projects are: Climate on the Wall and CO2nfession/CO2mmitment. The analysis is established through three different theoretical “lenses”: engagement, affect, and information. We conclude with some observations about experimental urban interfaces in general, based on our experiences from this project framed by the theoretical lenses.
In Affective Experience as a Theoretical Foundation for Interaction Design (Fritsch, 2011), it must be the most coherent argument I have made concerning affective interaction design so far. Basically, I mobilize Massumi’s experience philosophy to sketch out a theoretical foundation that contributes to understanding and addressing experiential concerns on an affective level on interaction design. In particular, I propose an affective vocabulary building on this theoretical foundation, to uncover new experiential values in the design work. I argue that an affective approach to interaction design emphasizes processes of change before fixedness and formalization; situational conditions of emergence rather than contextualizing structures of determinacy. Importanlty, it is not possible to design “an” affective experience. Rather, affective experience generally describes an experiential dynamics that can be addressed in design work. Through the theoretical development of the affective vocabulary I argue that it is possible to experiment with new affective concerns in the design of interactive environments and products.
This article From Signal to Signification in Interactive Environments (Fritsch, 2012) attempts to rethink the relation between signal and signification as well as affect and emotion in the design of interactive environments based on the work of French philosopher of technology Gilbert Simondon. Through an analysis of the minimal media installation Touched Echo, I argue that it is necessary to account for the dynamics of a larger experiential continuum to uncover the affective–emotive relations that occur through the transindividual workings of the signal and signification in interactive environments.
In Ekkomaten: Exploring the Echo as a Design Fiction Concept (Fritsch, Breinbjerg and Basballe, 2013) we present the Ekkomaten project in detail. We describe the design process and results from the display of the installation, and argue that Ekkomaten is an affectively engaging listening machine that offers an auditory and situated experience of the 18th century city of Aarhus. The six site-specific echoes engage the users of the installation as protagonists in the exploration of an imagined narrative space emerging from the intersection of fact and fiction and the infra- and extraordinary. As an electronic design object, we show how Ekkomaten points both back and forward in time, questioning our current understanding of the 18th century, science fiction’s previous visions of the future, and current ideas about possible post-digital futures.
Notes
[1] See Christopher Frayling, « Research in Art and Design », in Royal College Of Art
[2] There are a number of other approaches to working with affect in HCI as well, I have made a thorough review in my dissertation (Fritsch 2011) and in a more condensed form in an article on affective engagement (Fritsch 2009).
[3] See Brian Massumi and Joel McKim, « On Microperception and Micropolitics. An Interview with Brian Massumi », Inflexions 3, August 2008, online
[4] See Ekkomaten and Fritsch et. al 2013.
[5] See Massumi 2008, p. 18.
[6] See Jonas Fritsch, « From Signal to Signification in Interactive Environments », Journal of Aesthetics and Culture, vol. 4, 2012, online
[7] See John McCarthy and Peter Wright McCarthy, Technology as Experience, MIT Press, 2004, online
Bibliographie
– Fritsch, Jonas, «Understanding Affective Engagement as a Resource in Interaction Design», Proceedings of « Engaging Artifacts », Nordic Design Research Conference, 2009, en ligne, <https://archive.nordes.org/index.php/n13/article/view/57>.
– Fritsch, Jonas et Martin Brynskov, «Between Experience, Affect and Information – Experimental Urban Media in the Climate Change Debate», dans Marcus Foth, Laura Forlano, Martin Gibbs, et al., From Social Butterfly to Engaged Citizen: Urban Informatics, Social Media, Ubiquitous Computing, and Mobile Technology to Support Citizen Engagement, Cambridge, MIT Press, 2011, p. 115-135.
– Fritsch, Jonas, «Affective Experience as a Theoretical Foundation for Interaction Design», thèse de doctorat, Département de l’information et des études médiatiques, Université d’Aarhus, 2011.
– Fritsch, Jonas, «From Signal to Signification in Interactive Environments», dans Journal of Aesthetics and Culture, vol. 4, 2012, en ligne, <https://www.tandfonline.com/journals/zjac20>.
– Fritsch, Jonas, Morten Breinbjerg et Ditte Amund Basballe, «Ekkomaten: Exploring the Echo as a Design Fiction Concept», dans Journal of Digital Creativity, vol. 24, no1, 2013, p. 60-74.