{"id":710,"date":"2018-03-01T19:04:13","date_gmt":"2018-03-01T19:04:13","guid":{"rendered":"https:\/\/archee.uqam.ca\/?p=710"},"modified":"2022-10-28T19:04:31","modified_gmt":"2022-10-28T19:04:31","slug":"mars-2018-film-acting-and-performance-capture-the-index-in-crisis","status":"publish","type":"post","link":"https:\/\/archee.uqam.ca\/mars-2018-film-acting-and-performance-capture-the-index-in-crisis\/","title":{"rendered":"Mars 2018 – Film Acting and Performance Capture: The Index in Crisis"},"content":{"rendered":"\n

Since 2002, every time Academy Award nominating season rolls around, it is guaranteed that journalists will once again raise the question, \u201cWhen Will a Motion Capture Actor Win an Oscar?\u201d (Hart). This discussion began when\u00a0New Line Cinema, the company that released Peter Jackson\u2019s\u00a0The Lord of the Rings: The Two Towers<\/em>, made a concerted effort to garner a Best Supporting Actor award nomination for\u00a0Andy Serkis, the British actor who played Gollum in the film \u2013 a nomination that was not forthcoming.\u00a0Twentieth Century Fox\u00a0would repeat the gesture in both 2012 and 2014, again on behalf of Serkis, this time for his performances as the ape Caesar in\u00a0Rise of the Planet of the Apes\u00a0<\/em>and\u00a0Dawn of the Planet of the Apes<\/em>. Again, there were no nominations, even though critics and audiences alike have praised all of Serkis\u2019s performances as Gollum and Caesar, for which he has won other awards.\u00a0\u00a0<\/p>\n\n\n\n

\"\"
Serkis and Golum<\/em><\/figcaption><\/figure>\n\n\n\n

Before proceeding any further, I want to address the fact that I just described Serkis, who has become the most famous actor to work extensively with performance capture as well as a staunch advocate for the practice, as having\u00a0played<\/em>\u00a0Gollum and given a\u00a0performance\u00a0<\/em>as Caesar. Serkis has said, \u201cReviewers have a strange way of describing my performances. [\u2026] They\u2019ll say things like, \u2018Serkis lent his voice to\u2019 or \u2018inspired the emotions\u2019 or \u2018lent his movements to\u2019 or \u2018emotionally retained the backbone of,\u2019 as opposed to \u2018performed the role.\u201d\u00a0(Hart, 2012) A film actor myself, I choose to pay Serkis the respect of avoiding such circumlocutions and describing him simply as an actor who plays roles through performance capture even though the status of motion capture performance as acting is precisely the question at issue.<\/p>\n\n\n\n

\u201cThe Gollum Problem\u201d is how\u00a0Derek Burrill\u00a0summarizes the questions raised about performance capture and film acting and the reluctance of the Academy of Motion Picture Arts and Sciences (AMPAS, the body that awards the Oscar), to recognize what Serkis and others do as\u00a0acting<\/em>. He captures the debate in a series of questions: \u201cWas Serkis present enough in the performance? At what point is something too digitized? If something is partially digitized, what of its ontology, its presence? Can someone (or something) perform, in the traditional sense, in the digital?\u201d (Burrill, 2005, p. 492)\u00a0These are+\u00a0far-reaching questions that go well beyond the realm of film acting, to the general status of performance in the Digital Age. Here, I will restrict myself to examining the problematic status of performance capture in film acting primarily through Burrill\u2019s first and third questions \u2013 those concerned with presence and ontology. My point of departure is yet another question: Why is it that the voting membership of the Academy, made up of film industry professionals of all types, is apparently unwilling to nominate actors who perform through performance capture?<\/p>\n\n\n\n

Note that there is nothing in the Academy\u2019s rules governing the Oscars to prevent actors working in performance capture from being nominated for acting awards. In fact, when the Academy modified its definition of an animated film in 2010, it specifically included new language to the effect that \u201cmotion capture by itself is not an animation technique2<\/sup>.\u201d\u00a0This clause can be interpreted legalistically as a way for the Academy to shift films using performance capture out of the animation category, thus paving the way for such performances to be considered in the acting categories.<\/p>\n\n\n\n

It is the case, however, as an\u00a0LA Times<\/em>\u00a0study of 2012 revealed, that the nearly 6,000 voting members of the Academy\u00a0<\/p>\n\n\n\n

are markedly less diverse than the moviegoing public, and even more monolithic than many in the film industry may suspect. Oscar voters are nearly 94% Caucasian and 77% male,\u00a0The Times<\/em>\u00a0found. Blacks are about 2% of the academy, and Latinos are less than 2%. Oscar voters have a median age of 62, the study showed. People younger than 50 constitute just 14% of the membership. (Horn, 2012)<\/p>\n\n\n\n

It would not be surprising if this group were to prove to be conservative in its understanding of what film acting is, and what kinds of things that show up on screen should be eligible to be awarded as such. However, the fact that the Academy\u2019s membership is dominated by aging white men in no way explains exactly why<\/em> performance capture may be objectionable from their presumed conservative standpoint.<\/p>\n\n\n\n

Discussing the\u00a0\u201cGollum Problem\u201d on Salon.com<\/u>\u00a0\u00a0in 2003, Ivan Askwith suggests that Serkis was not nominated because Academy voters could not see Serkis in Gollum, just as they could not see\u00a0John Hurt\u00a0 in the title character of\u00a0The Elephant Man<\/em>, a performance with which Serkis\u2019s are frequently compared.<\/p>\n\n\n\n

Although nominated for best actor, John Hurt did not win in 1980, a loss he has partly attributed in recent interviews to the fact that the audience could not recognize him at all. It seems almost certain that Serkis, had he been nominated, would face the same problem.<\/p>\n\n\n\n

Defining the problem in terms of the actor\u2019s recognizability, Askwith frames it as a problem of iconicity, to use a semiotic term. Recall\u00a0C. S. Peirce\u2019s definition of an iconic sign or icon: \u201cMost icons, if not all, are\u00a0likenesses\u00a0<\/em>of their objects. A photograph is an icon\u201d (Short, 2007, p. 215)\u00a0(emphasis in original). Gollum is not a likeness of Serkis \u2013 they look almost nothing alike. Bert States suggests that we need to be able to perceive the actor in the character in order to appreciate the performer\u2019s artistry: \u201cWe always recognize Olivier in Hamlet or Olivier behind the dark paint of Othello. [\u2026] When the artist in the actor comes forth, we are reacting to the actor\u2019s particular way of doing his role.\u201d (States, 2002, p. 26)\u00a0This process is short-circuited, however, if we cannot recognize Serkis in Gollum, leading to a situation in which the performance is not perceived as creditworthy and therefore not eligible for an award. I use the term\u00a0creditworthy<\/em>\u00a0in the way\u00a0Stan Godlovitch\u00a0does when talking about musical performance, to mean that we as audience can \u201cread back from what [we perceive of the performance] to the [performer\u2019s] creditworthiness and hence skilfulness.\u201d (Godlovitch, 1998, p. 26)\u00a0Gollum\u2019s lack of iconic resemblance to Serkis seriously inhibits this process of reading back, and therefore appreciation of the actorly skill Serkis brought to his performance.<\/p>\n\n\n\n

However, iconicity is not the only semiotic dimension of the Gollum Problem. I shall argue here that although it is reasonable to characterize the actor as an iconic sign for the character he or she plays, there is a deeply embedded historical discourse around film acting that sees the actor\u2019s gestures and, especially, facial expressions on screen not primarily as iconic signs for the character\u2019s gestures and expressions, but as indexical signs of the actor\u2019s interiority. In what follows, I will investigate the indexicality of film acting, first by examining the relationship of the Gollum Problem to current debates over the ontology of digital imagery, then by contextualizing it in relation to entrenched assumptions about the nature of film acting. First, however, I will offer a brief overview of performance capture as a technology and process.<\/p>\n\n\n\n

What is Performance Capture?<\/h2>\n\n\n\n

Commentators regularly trace the history of motion capture back to the studies of animal and human motion through sequential photographs undertaken by\u00a0Edweard Muybridge\u00a0in the last quarter of the 19th<\/sup>\u00a0Century. Fast-forwarding to the motion picture era, another important precedent for motion capture is the rotoscope, a device invented in 1915 by the celebrated animator\u00a0Max Fleischer, whose studio produced both Betty Boop and Popeye cartoons. The rotoscope allowed animators to draw over filmed performers by hand to transform them into cartoon characters, as the Fleischer Studio did with band-leader Cab Calloway, who appears as a ghostly dancing walrus in the 1932 animated Betty Boop short, \u201cMinnie the Moocher.\u201d This technique is still in use.<\/p>\n\n\n\n

Although the field began to develop rapidly in the late 1980s, electronic motion capture actually began in 1967 with\u00a0Lee Harrison III\u2019s Scanimate. For the first time, a performer wearing a \u201cdata suit\u201d could directly control the movements of an animated figure on a television screen through her own movements. (Sturman, 1998, p. 38)\u00a0This remains the basic procedure of motion capture to this day: a performer is outfitted with sensors at a number of points on the body. As the performer moves, high-resolution digital cameras track the displacement of the sensors and relay this information to a computer. The data can then be used to animate an onscreen CGI (Computer Generated Imagery) figure created by designers and animators, as was the case for Sirkis\u2019s portrayals of Gollum, Caesar, King Kong in Peter Jackson\u2019s 2005 film, and Capt. Haddock in Steven Spielberg\u2019s\u00a0The Adventures of Tintin<\/em>. Tom Hanks portrayed many of the characters in\u00a0The Polar Express<\/em>\u00a0through motion capture and CGI. It is the combination of motion capture data extracted from an actor\u2019s performance and CGI that allows the screen to be populated not only with fantastic creatures but also with digital doubles that resemble actors or with huge crowds of extras that exist only as data. A digital double performed some of Val Kilmer\u2019s stunts in\u00a0Batman Forever<\/em>; the assembled masses in Ridley Scott\u2019s\u00a0Gladiator<\/em>\u00a0were constructed from motion capture data taken from 2,000 human performers.<\/p>\n\n\n\n

Originally, motion capture systems captured only full-body movements. Around 2005, motion capture systems that allowed the capturing of facial expressions came into use. At this point, motion capture (or MoCap) was renamed as \u201cperformance capture\u201d (or PerfCap) because the sense was that the technology could now engage with a fuller range of an actor\u2019s expressive capabilities. Because motion capture still does not deal well with hands and feet, a sub-specialty has evolved called \u201chand-overing\u201d (the analogy is with voice over) in which data captured separately from hand movements is added to full-body motion capture.<\/p>\n\n\n\n

An important innovation in performance capture occurred between\u00a0The Two Towers<\/em>\u00a0(2001) and James Cameron\u2019s\u00a0Avatar<\/em>\u00a0(2009), another film to make extensive use of the technology. Serkis\u2019s first portrayal of Gollum entailed \u201ca three-stage process. [\u2026] First, Serkis would perform his scenes on set with the other actors, the same scenes would then be performed again without Serkis, and Serkis would finally replicate these same scenes back in the WETA studio while wearing his motion-capture suit and markers.\u201d (Romano, 2014)\u00a0(Weta Digital is the New Zealand effects company founded by Jackson that is currently at the cutting edge of PerfCap technology). Ultimately, the performance seen on screen was constructed from the data collected during the studio sessions and added into the film. Technological innovations developed for\u00a0Avatar<\/em>\u00a0made it possible to largely eliminate the second and third stages since enough motion capture data could be extracted from the actors when they were performing with one another to go straight into animating their virtual characters. This streamlined process is the one used for the recent\u00a0Planet of the Apes<\/em>\u00a0movies and has strengthened the arguments Serkis and others make to defend the idea that performances created through performance capture should be understood simply as acting and, therefore, worthy of Oscar consideration.<\/p>\n\n\n\n

Digital Images, Motion Capture, and the Question of Indexicality<\/h2>\n\n\n\n

The issues raised by performance capture overlap with questions raised about the general differences between analog (chemical) photography and digital photography. One of the chief debates in this area surrounds the question of indexicality. Whereas Peirce defines icons as referring to their objects through resemblance, he defines the index as<\/p>\n\n\n\n

A sign [\u2026] which refers to its object not so much because of any similarity [\u2026] nor [by association] [\u2026] as because it is in dynamical (including spatial) connection [\u2026] with the individual object. (Short, 2007, p. 219)<\/p>\n\n\n\n

Analog photography is thus both iconic and indexical in that photographs resemble their subjects, they derive from a spatial relationship between those subjects and the camera, and they point to the subjects\u2019 existence in the real world. In order for something to be photographed, it has to have been present before the camera. A painting of a unicorn, for example, is iconic but not indexical because we recognize it as a likeness of a unicorn, but it does not point to a real entity that actually posed for the painting.<\/p>\n\n\n\n

D. N. Rodowick\u00a0argues that whereas we perceive analog photography as \u201cthe automatic transcription of a past state of affairs\u201d and therefore as indexical, with digital photography<\/p>\n\n\n\n

\u2026 the logic through which indexicality is effected changes fundamentally. Digital capture involves a discontinuous process of transcoding: converting a nonquantifiable image into an abstract or mathematical notation. In digital capture, the indexical link to physical reality is weakened, because light must be converted into an abstract symbolic structure independent of and discontinuous with physical space and time. (Rodowick, 2007, p. 117)<\/p>\n\n\n\n

Others argue, however, that indexicality persists in digital images. Tanine Allison, in one of the very few academic articles on performance capture published to date, stakes out this position: \u201cI consider motion capture to be an example of \u2018digital indexicality,\u2019 a blend of computer-generated images and material recorded from reality.\u201d (Allison, 2011, p. 326) Allison appears to have borrowed the concept of \u201cdigital indexicality\u201d from Philip Rosen. Without disputing the ontological distinctions between analog and digital photography emphasized by Rodowick and others, Rosen argues that digital images can nevertheless function in the same manner as analog photographs by asking,<\/p>\n\n\n\n

How is it that a digital camera can be sold not as a displacement, but as a replacement for a conventional still camera? How is it that, in the appropriate context, a digital photograph may take on the functions of an indexical photograph, such as family testament or photo-journalism? The answer is that the digital camera does not necessarily exclude all operations of a photochemical camera; it even uses a lens to gather light. [\u2026] [I]t retains indexical import as a light-sensing device. [\u2026] This means that the digital camera is a machine for configuring \u201cpure data,\u201d according to a certain range of prior pictorial norms, namely those identified with photography. But this does not make it nonindexical. (Rosen, 2001, p. 308)<\/p>\n\n\n\n

By shifting the ground of the debate from ontology to function, Rosen is able to make a persuasive case that we habitually use digital media to replicate the indexicality of analog media. But this argument is hard to sustain in the context of performance capture.<\/p>\n\n\n\n

At the level of the technology, it is significant that so-called motion capture \u201ccameras\u201d are not \u201clight-sensing devices\u201d in the same sense that photo-chemical or even digital still cameras are. As described by\u00a0Qualisys, a manufacturer of motion capture equipment, these cameras \u201cemit infrared light onto the markers [on the performer\u2019s body] that reflect the light back to the camera sensor. This information is then used to calculate the position of targets with high spatial resolution\u201d (\u201cMotion Capture\u201d). In order to have enough data to reconstruct movement in three dimensions, each motion capture subject has to be shot from at least three different angles simultaneously. A motion capture camera is not, then, a machine for configuring \u201cpure data\u201d\u2014it is a machine for\u00a0generating<\/em>\u00a0the pure data from which the performer\u2019s movement can only be configured by using another data-driven machine, a computer. In short, the motion capture camera does not capture anything that resembles an image in itself, as the digital camera can still be argued to do. As\u00a0David Saltz\u00a0observes, \u201cMotion capture simply records the changing positions of a discrete set of points on the performer’s body. [\u2026] The real product of a motion capture session is simply a long list of numbers.\u201d (Saltz, 2003, p. 3)<\/p>\n\n\n\n

Describing motion capture as \u201ca performance extractor,\u201d Saltz further points out that \u201cmotion capture, in effect, captures verbs without subjects, performance without identity.\u201d (Saltz, 2003, p. 10)\u00a0Whereas an analog film or video recording of a performer in motion both iconically represents that person and indexically points back to the person whose movements were recorded, motion capture captures only the motion, not the person, and therefore does neither. Rather, it renders what were once movements produced by a specific subject as abstract information. This data set only becomes an image when the data is used to animate a figure on a screen, and this figure, understood as a sign, does not point directly back to reality. This image neither is, nor even can be perceived to be, an \u201cautomatic transcription\u201d of some real occurrence.<\/p>\n\n\n\n

The effect Rodowick ascribes to digital photography, that of weakening \u201cthe indexical link to physical reality\u201d is thus exacerbated in the case of performance capture. Commenting on Peirce\u2019s definition of the index, T. L. Short observes that the \u201crelation of index to object depends on the existence of the latter. As [Peirce] wrote in 1902, \u2018An\u00a0index\u00a0<\/em>is a sign which would, at once, lose the character which makes it a sign if its object were removed.\u2019\u201d (Short, 2007, p. 219) This removal of the object of the sign is precisely what happens through performance capture\u2014the performer, whose actions would normally ground the sign in reality, disappears, leaving behind only a trail of numerical data. The indexical link is thus damaged beyond repair. As an indexical sign, the only object to which a figure on screen created through performance capture could possibly point is the motion capture data set.<\/p>\n\n\n\n

The Indexicality of Film Acting<\/h2>\n\n\n\n

The disappearance of the performing subject in performance capture is even more problematic when considered in the context of film acting. Actors on screen are iconic signs for the characters they portray, of course. But there is a deeply entrenched historical discourse around film acting that defines how it is understood among people who work in the medium. In the terms of this discourse, actors on screen are indexical signs that point to the actor\u2019s own interiority, to the thoughts and emotions that underpin the actor\u2019s work. This discourse can be traced from the early days of film theory up to present day acting pedagogy.<\/p>\n\n\n\n

Both\u00a0B\u00e9la Bal\u00e1zs\u00a0and\u00a0Siegfried Kracauer, notable names in the history of film theory, contributed to the discourse in question through their joint emphases on the camera\u2019s ability to do more than reproduce the surface of reality. According to both, the camera penetrates below the surface to reveal hidden truths, including those lurking in the psyches of actors. \u201cClose-ups are often dramatic revelations of what is really happening under the surface of appearances. [\u2026] They show the faces of things and those expressions on them which are significant because they are reflected expressions of our own subconscious feeling.\u201d\u00a0(Bal\u00e1zs, 1952, p. 56) Bal\u00e1zs famously described the power of the camera by saying,<\/p>\n\n\n\n

The language of the face cannot be suppressed or controlled. However disciplined and practisedly hypocritical a face may be, in the enlarging close-up we see even that it is concealing something, that it is looking a lie. For such things have their own specific expressions super posed on the feigned one. It is much easier to lie in words than with the face and the film has proved it beyond doubt. (Bal\u00e1zs, 1952, p. 63)<\/p>\n\n\n\n

Bal\u00e1zs implies that since no one has the ability to completely control facial expressions and what they communicate, film actors must be able to perform in ways that will not be unmasked as false by the camera. Kracauer expresses a similar idea:<\/p>\n\n\n\n

The film actor is less independent of his physique than the stage actor, whose face never fills the whole field of vision. The camera . . . reveals the delicate interplay between physical and psychological traits, outer movements and inner changes. [M]ost of these correspondences materialize unconsciously. (Kracauer, 2004, p. 21)<\/p>\n\n\n\n

More directly than Bal\u00e1zs, Kracauer proposes that since the camera reveals connections between the film actor\u2019s interiority and an external appearance that the actor does not fully control consciously, the actor\u2019s primary concern is with achieving an inner state that will result in the appropriate external appearance. This suggests, in turn, that the actor\u2019s facial expressions and emotive gestures on screen are indexical signs pointing to the actor\u2019s internal psychological or emotional state.<\/p>\n\n\n\n

Although Bal\u00e1zs and Kracauer both published their definitive works of film theory after the Second World War, their formative experiences of cinema occurred in the 1920s in Germany. As\u00a0Cynthia Baron\u00a0has shown, with the arrival of sound in film, the Hollywood studios became increasingly interested in training actors, resulting in the establishment of a number of acting schools around Los Angeles in the 1930s and 1940s. The studios actually ran some of them, while others were independent but nevertheless served the film industry. The principles taught in these schools derived largely from those imparted by the first generation of European and American actors and acting teachers to bring versions of the Stanislavskian approach to the United States, such as expatriates like\u00a0Maria Ouspenskay\u00a0and\u00a0Richard Boleslavsky, and figures associated with the Group Theater like\u00a0Morris Carnovsky(a founder of one such school, the\u00a0Actors\u2019 Laboratory\u00a0in Los Angeles, in 1941) and\u00a0Stella Adler.\u00a0<\/p>\n\n\n\n

The precepts governing the way film acting was taught in these schools often reiterate assertions concerning the nature of the apparatus found in early film theory. For example, Josephine Dillon begins her book\u00a0Modern Acting: A Guide for Stage, Screen and Radio\u00a0<\/em>(1940) which became a standard text, by discussing the implications of the close-up for acting and the consequent importance of the actor\u2019s eyes. (Dillon, 1940 p. 3)\u00a0Lillian Burns, who coached actors for MGM, describes the camera as \u201ca truth machine.\u201d (Baron, 2004, p. 88)\u00a0Comparing acting on stage with acting on film in a 1949 essay for the magazine\u00a0Theater Arts<\/em>, the actor Hume Cronyn, who was associated with the Actors\u2019 Laboratory, states, \u201cthe camera will often reflect what a man thinks, without the degree of demonstration required in the theater.\u201d (Cronyn, 2008, p. 373)<\/p>\n\n\n\n

One of the main acting techniques advocated by many of these teachers was the inner monologue, a silently verbalized stream of the character\u2019s fictional consciousness. Dillon, in her book, describes the value of this technique to the film actor:<\/p>\n\n\n\n

Since the expressions of the eyes in acting are to represent the emotions of the part played, the actor should, in studying the part, improvise the probable mental conversations of the person portrayed, and memorize them as carefully as the written dialogue. Then the eye expressions will be right; they will be convincing or believable. [A]lso, if the actor is thinking actual words during a close-up in which there is no spoken dialoguer, the timing of his glance and of the changes of expression in his eyes will be perfect. (Dillon, 1940, p. 9)<\/p>\n\n\n\n

In other words, the actor\u2019s external expressions arise from maintaining the appropriate internal state. Or, as Burns so succinctly puts it, \u201cYou cannot say \u2018dog\u2019 and think \u2018cat\u2019 because \u2018meow\u2019 will come out if you do.\u201d (Baron, 2004, p. 88)\u00a0In this way, the aspects of the film actor\u2019s performance perceptible to the audience serve as indexical signs pointing to a process taking place inside the actor that gives rise, perhaps unconsciously and beyond the actor\u2019s full control, to the facial expressions and gestures that render the character.<\/p>\n\n\n\n

This way of thinking about film acting persists to the present day, particularly in the context of actor training.\u00a0Jeremiah Comey\u00a0declares in the pages of\u00a0The Art of Film Acting<\/em>, a textbook published in 2002, that \u201cthe camera sees everything and demands absolute truth. [\u2026]\u00a0 The camera sees what you truly are at the moment.\u201d\u00a0(Comey, 2002, p. 14) He elaborates:<\/p>\n\n\n\n

All acting in film takes place in close-up . . . where the camera sees your emotions. You can\u2019t hide anything because the camera sees everything\u2014fear, happiness, anxiety, lack of confidence, nervousness, whatever is going on inside you. On film you cannot \u201cact\u201d in love, you have to\u00a0be\u00a0<\/em>in love. (Comey, 2002, p. 18)<\/p>\n\n\n\n

I can attest from my own current experience as a film actor that this strong belief that if the actor is feeling the right thing, the camera will pick it up more or less automatically remains dominant. I have heard both directors and casting people regularly say things along the lines of, \u201cIf you are feeling it, I will see it.\u201d<\/p>\n\n\n\n

It is clear that this understanding of film acting, in which the surface manifestations of the actor\u2019s work are indexical signs pointing to what is happening in the actor\u2019s interior in the moment of performance, challenges those who would argue for seeing performance capture as acting. The reason is self-evident: we as audience cannot read back from the gestures and expressions of Gollum or the ape Caesar to their internal states since CGI figures possess no interiority: they are simply particular ways of displaying data on a screen. In defending performance capture as a kind of acting, it therefore becomes necessary to argue that the actor\u2019s expressions and gestures can be successfully captured and transferred to the CGI figure, to be read either as indexing this figure\u2019s presumed emotional state or pointing back to the actor whose performance was captured. As Allison points out, this is precisely the tack taken by Serkis and Jackson in their attempt to frame Serkis\u2019s performance as King Kong as acting:<\/p>\n\n\n\n

Although Allison acknowledges through her use of the word \u201cpretend\u201d that the position adopted by Jackson and Serkis is problematic, she goes on to advance a similar argument by saying,\u00a0<\/p>\n\n\n\n

The character of Kong stands as both an instantiation of the indexical that takes the form of animation (Serkis\u2019 recorded movement animates Kong) and an instantiation of animation that takes the form of the indexical (a digitally constructed creature that literally takes the shape of Serkis\u2019 body in motion (Allson, 2011, p. 335)).<\/p>\n\n\n\n

The difficulty is that Kong does not \u201cliterally take the shape of Serkis\u2019s body in motion\u201d and, in fact, could not possibly do so. This is not because of some deficit in the technology that will eventually be overcome. Rather, it has to do with the difficulty of transferring movements across entities with different physiognomies. As Marc Boucher points out in an essay on the use of motion capture in dance, since \u201ca giraffe does not move like an anteater, mapping motion data from one onto the other results in something that is physically impossible because of the extreme anatomical and morphological variance.\u201d (Boucher, 2011)\u00a0Even though apes and humans are physically much more similar to one another than are a giraffe and an anteater, the problem persists, especially when it comes to the subtleties of facial expression. An article on the technical dimensions of performance capture in\u00a0Dawn of the Planet of the Apes<\/em>\u00a0describes the process of constructing the facial expressions of the CGI apes as one of reworking the actors\u2019 expressions for the structure of an ape\u2019s face \u2013 a laborious process that is partly achieved automatically by software and partly manually by animators:<\/p>\n\n\n\n

the capture is stabilized, and then translated not only from human to ape facial shape, but also into a system that allows the animators to adjust lip sync, correct expressions and better match the acting choices of the MoCap artists when shown on an Ape face. [\u2026] It is also possible for certain combinations of blend shapes or expression to be created by the data and or the hand animation. These are not character achievable or \u201coff-model.\u201d When this occurs a corrective shape is automatically triggered that avoids the bad combination and produces a substitute expression. (Seymour, 2014)<\/p>\n\n\n\n

In other words, the expressions we see on the screen apes\u2019 faces are not identical to those Serkis and the other performers made when captured, but had to be adjusted to fit the shape of an ape\u2019s face. Furthermore, if an expression cannot be made to work on the CGI figure, a different one, generated by the software, is automatically used.<\/p>\n\n\n\n

This points to the other major problem with the argument that CGI gives us direct access to gestures and expressions captured from live performers: it completely elides the contributions of the animators and designers who intervene in the process by using the motion capture data to create the characters we actually see. There are, for example, many compelling \u201cbefore and after\u201d style video clips that place footage of actors in motion capture suits side-by-side with the corresponding scene from the finished film. Such presentations make it seem as if the transfer is fairly direct, but only by leaving out everything that had to happen to get from one state to the other. Serkis himself has been roundly criticized for pronouncements in favor of seeing performance capture as acting that often seem to either ignore or denigrate the animators\u2019 work. The intervention of animators between the actors whose performance is captured and the figure we ultimately see on screen considerably muddies the question of indexicality. Precisely what does this figure point to: its own fictional interiority? That of the actor? Or the work of the animators who actually created it? Commentators like Allison would like to have it all ways, but this is not viable in relation to a tradition of acting that insists that the facial expressions of the entity we seen on screen in close-up are indexical signs for processes taking place within the same entity at the time of filming.<\/p>\n\n\n\n

Conclusion<\/h2>\n\n\n\n

I trust it is clear that it has not been my purpose here to take a stand on the question of whether or not the Academy should consider granting Oscars to performance capture actors, a question on which I remain ambivalent. I have used Gollum\u2019s Problem as a heuristic for attempting to think through the question of why the idea that such performances should be accepted unproblematically as instances of screen acting is controversial. I have pursued this inquiry along two lines, both having to do with the indexicality of the screen image. Following the first line, I suggested that although digital photography has put the ontology of the image into question, arguments that digital images can at least function indexically are credible. Such arguments do not, however, extend to the context of motion capture, where the technology undermines indexicality by divorcing performance (in the form of movement) from the materiality, identity, and presence of the performer. <\/p>\n\n\n\n

Following the second line, I proposed that film acting today is defined by a deeply entrenched discourse whose history can be traced through both film theory and practice, a discourse that posits that the gestures and, especially, facial expressions we see actors make on screen are indexical signs for processes taking place within their minds and psyches \u2013whether those processes are understood as spontaneous emotional reactions or constructed inner monologues. Inasmuch as CGI figures possess no interiority, they cannot be perceived this way, and arguments that the expressions of the actor whose performance was captured and the interior states to which they point simply transfer to the CGI figure are highly questionable in light of the design decisions and technical routines involved in bringing the two together into a screen performance.<\/p>\n\n\n\n

I am not suggesting that the discomfort around the idea that performance capture in film should be seen as acting, of which the Academy\u2019s reluctance is symptomatic, is necessarily permanent. I think it is quite possible that in the not-too-distant future we as audiences, and perhaps even the Academy, will find a way of understanding and classifying such performances. However,\u00a0Andrew Darley\u00a0describes the current situation accurately:<\/p>\n\n\n\n

A certain sense of ambiguity haunts the imagery of such films. This is related to uncertainty as to the status of the imagery in terms of origination: is it cartoon animation, three-dimensional (puppet) animation, live action or, perhaps, a combination of all three? One becomes fascinated with the imagery precisely because of this uncertainty, seduced by the ways it recasts, amalgamates and\u00a0confuses\u00a0<\/em>familiar techniques and forms. (Darley, 2000, p. 84)<\/p>\n\n\n\n

There is pleasure in this seductive ambiguity, but it also makes it impossible for us to read back from what we see on screen to the creditworthy actions of those who created it. Returning to one of Burrill\u2019s foundational questions \u2013 \u201cCan someone (or something) perform, in the traditional sense, in the digital?\u201d \u2013 the answer, at least where performance capture and film acting are concerned, is \u201cNo.\u201d<\/p>\n\n\n\n

When we do reach the point of creating a category for motion capture\u00a0\/\u00a0CGI performances, I suspect we may not perceive them as a species of screen acting at all. Saltz suggests that \u201cmotion capture is, at root, a form of virtual puppetry\u201d (Saltz, 2003, p. 12)\u00a0and he is not alone in characterizing it in this way.\u00a0David J. Sturman, in an essay that chronicles early uses of motion capture for entertainment purposes, describes it as \u201ccomputer puppetry\u201d and notes that puppeteers make better performers in this medium than actors or mimes. (Saltz, 2003, p. 42)\u00a0Even Serkis has said that thinking of his relationship to Gollum as that of a puppeteer to his puppet made it easier for him to perform using motion capture. Searls, 2014, p. 304)\u00a0Since AMPAS does not give awards for puppetry, and performers like Serkis would much prefer to think of themselves as actors rather than puppeteers, this characterization of perfcap is probably not going to get much traction in Hollywood. Arguably, it nevertheless provides a more accurate account of the process than seeing these performances as constituting a variety of screen acting.<\/p>\n\n\n\n

Comparing CGI figures like those in the\u00a0Planet of the Apes\u00a0<\/em>movies with traditional physical puppets, Donald Crafton writes, \u201csuch films use [\u2026] motion capture to choreograph puppets (or puppetlike simulations) without wires or other visible human agency so artfully that they seem to be animals performing on their own. [\u2026] In their ability to affect audiences, these puppet figures have their own agency.\u201d (Crafton, 2012, p. 68)\u00a0This claim is part of Crafton\u2019s larger argument that audiences experience animated figures on screen not as performances by animators but as performers in their own right \u2013 a claim that applies equally to performances like Serkis\u2019s and to the classic animation Crafton primarily discusses.<\/p>\n\n\n\n

Although I, too, wish to emphasize the autonomy of perfcap figures, the idea that we perceive these figures on screen not primarily as performances by actors or animators but as performers themselves, I will couch the point in different terms. In his book\u00a0The Play of Nature<\/em>, the philosopher\u00a0Robert Crease\u00a0offers an extended analogy between scientific experimentation in laboratories and theatrical performances as a means of developing a philosophical account of experimentation. Scientific theories, in his view, are analogous to play scripts in that both are merely \u201cabstract testimonials to the possibility of the presence of phenomena.\u201d\u00a0(Crease, 2012, p. 68)\u00a0 In the case of theater, the phenomenon is the world of the play populated by characters described in a script. In the case of experimentation, the phenomena are scientific events hypothesized in theory. In both cases, the object of the performance, whether on stage or in a laboratory, is to bring the phenomenon forth, to make it present and perceptible to what Crease repeatedly identifies as \u201ca suitably prepared\u201d audience. (Crease, 2012, p. 158)<\/p>\n\n\n\n

Perfcap CGI figures bridge Crease\u2019s two categories of phenomena. On the one hand, they are characters, inhabitants of fictional worlds described in dramatic texts (film scripts). While they are not scientific phenomena strictly speaking, they are technological phenomena of some significance. Coaxing them into existence requires just as much theory, apparatus, and personnel as the staging of any major scientific breakthrough, such as the identification of a particle that may be the \u201cHiggs boson\u201d at the\u00a0CERN\u00a0(Conseil Europ\u00e9en pour la Recherche Nucl\u00e9aire) laboratory in Switzerland in 2012. Crease\u2019s view of performance is expansive\u2014beyond the immediate procedures and products on view to an audience, he emphasizes the significance of production, which he defines as all of the decisions, materials, locations, institutions, funds, and people involved in revealing the phenomenon. From this perspective, the perfcap figures we see on screen do not appear as performances by individual actors but, rather, as techno-dramatic phenomena brought forth through a complex process of production that includes actors, writers, designers, animators, and many others, working together. The resulting phenomena appear to their audiences as largely autonomous figures with their own agency, as Crafton suggests, just as the audience for the Higgs particle perceives it as existing independently of the production process that made it perceptible to them2<\/sup>. <\/p>\n\n\n\n

Notes<\/h2>\n\n\n\n

[1] This text has been published in a first version in\u00a0PAJ: A Journal of Performance and Art<\/a>\u00a0Volume 39, Number 3, September 2017 (PAJ 117)<\/a>\u00a0 pp. 7-23<\/p>\n\n\n\n

Bibliographie<\/h2>\n\n\n\n

\u2013 Academy of Motion Picture Arts and Sciences.\u00a087th Annual Academy Awards of Merit<\/em>, Los Angeles, Academy of Motion Picture Arts and Sciences, n. d.<\/p>\n\n\n\n

\u2013 Allison, Tanine, \u00abMore than a Man in a Monkey Suit: Andy Serkis, Motion Capture, and\u00a0Digital Realism\u00bb,\u00a0Quarterly Review of Film and Video<\/em>, vol.\u00a028, 2011, p.\u00a0325-341.<\/p>\n\n\n\n

\u2013 Askwith, Ivan, \u00abGollum Dissed by the Oscars?\u00bb, Salon<\/em>, 18 f\u00e9vrier 2003, en ligne, <https:\/\/www.salon.com\/2003\/02\/18\/gollum\/>. <\/p>\n\n\n\n

\u2013 Bal\u00e1zs, B\u00e9la,\u00a0Theory of the Film: Character and Growth of a New Art<\/em>, Londres, Dennis Dobson, 1952, 318 p. <\/p>\n\n\n\n

\u2013 Baron, Cynthia, \u00abCrafting Film Performances: Acting in the Hollywood Studio Era\u00bb dans\u00a0<\/em>Pamela Wojcik,\u00a0Movie Acting: The\u00a0<\/em>Film\u00a0Reader<\/em>, New York, Routledge, 2004, p.\u00a083-94.<\/p>\n\n\n\n

\u2013 Boucher, Marc, \u00abVirtual Dance and Motion-Capture\u00bb,\u00a0Contemporary Aesthetics<\/em>, vol.\u00a09, 2011,\u00a0en ligne, <https:\/\/quod.lib.umich.edu\/c\/ca\/7523862.0009.011?view=text;rgn=main>. <\/p>\n\n\n\n

\u2013 Burrill, Derek Alexander, \u00abOut of the Box: Performance, Drama, and Interactive Software\u00bb,\u00a0Modern Drama<\/em>, vol.\u00a048, no<\/sup>\u00a03, automne 2005, p.\u00a0492-512.<\/p>\n\n\n\n

\u2013 Comey, Jeremiah,\u00a0The Art of Film Acting: A Guide for Actors and Directors<\/em>, Oxford,\u00a0<\/em>Focal Press, 2002, 304 p. <\/p>\n\n\n\n

\u2013 Crafton, Donald,\u00a0Shadow of a Mouse: Performance, Belief, and World-making in Animation,\u00a0<\/em>Berkeley, UC\u00a0<\/em>Press, 2012, 392 p. <\/p>\n\n\n\n

\u2013 Crease, Robert P.,\u00a0The Play of Nature: Experimentation as Performance<\/em>. Bloomington, Indiana University Press, 1993, 220 p. <\/p>\n\n\n\n

\u2013 Cronyn, Hume, \u00abNotes on Film Acting\u00bb dans Senelick,\u00a0<\/em>Laurence,\u00a0Theater Arts<\/em>\u00a0on Acting<\/em>,\u00a0New York, Routledge 2008, p.\u00a0370-376.<\/p>\n\n\n\n

\u2013 Darley, Andrew, Visual Digital Culture: Surface play and spectacle in new media genres<\/em>, New York, Routledge, 2000, 236 p. <\/p>\n\n\n\n

\u2013 Dillon, Josephine,\u00a0Modern Acting: A Guide for Stage, Screen and Radio<\/em>, New York, Prentice-Hall, 1940, 313 p. <\/p>\n\n\n\n

\u2013 Godlovitch, Stan,\u00a0Musical Performance: A Philosophical Study<\/em>, New York, Routledge, 1998, 184 p. <\/p>\n\n\n\n

\u2013 Hart, Hugh, \u00abWhen Will a Motion Capture Actor Win an Oscar?\u00bb,\u00a0Wired<\/em>, 24 janvier 2012,\u00a0en ligne, <https:\/\/www.wired.com\/2012\/01\/andy-serkis-oscars\/>. <\/p>\n\n\n\n

\u2013 Horn, John, Nicole Sperling et Doug Smith, \u00abUnmasking the Academy: Oscar Voters Overwhelmingly White, Male\u00bb,\u00a0LA Times,\u00a0<\/em>19 f\u00e9vrier 2012, en ligne, <https:\/\/www.latimes.com\/entertainment\/envelope\/oscars\/la-et-unmasking-oscaracademy-project-20120219-story.html#page=1>. <\/p>\n\n\n\n

\u2013 Kracauer, Siegfried, \u00abRemarks on the Actor\u00bb in Robertson,\u00a0<\/em>Pamela Wojcik,\u00a0Movie Acting: The Film<\/em>\u00a0Reader<\/em>, New York, Routledge, 2004, p.\u00a019-27.<\/p>\n\n\n\n

\u2013 \u00abMotion Capture Technology\u00bb,\u00a0Qualisys<\/em>, n.d.,\u00a0en ligne, <http:\/\/www.qualisys.com\/company\/motion-capture-technology\/>. <\/p>\n\n\n\n

\u2013 Rodowick, D. N.,\u00a0The Virtual Life of Film<\/em>, Cambridge, Harvard University Press, 2007, 216 p. <\/p>\n\n\n\n

\u2013 Romano, Nick, \u00abHow\u2019d They Do That? A Brief Visual History of Motion-Capture Performance on Film\u00bb,\u00a0Screencrush<\/em>, 14 juillet 2014, en ligne, <https:\/\/screencrush.com\/motion-capture-movies\/?trackback=tsmclip>.<\/p>\n\n\n\n

\u2013 Rosen, Philip,\u00a0Change Mummified: Cinema, Historicity, Theory<\/em>, Minneapolis, University of Minnesota Press, 2001, 472 p. <\/p>\n\n\n\n

\u2013 Saltz, David, \u00abThe Ontology of Motion Capture\u00bb, unpublished text presented at the Annual Meeting of the American Society of Aesthetics, January 2003.<\/p>\n\n\n\n

\u2013 Searls, Collette, \u00abUnholy Alliances and Harmonious Hybrids: New Fusions in Puppetry and Animation\u00bb dans Dassia N. Posner, Claudia Orenstein et John Bell,\u00a0The Routledge Companion to Puppetry and Material Performance<\/em>. Abingdon, Routledge, 2014, p.\u00a0294-307.<\/p>\n\n\n\n

\u2013 Seymour, Mike, \u00abApe Acting\u00bb,\u00a0FX Guide<\/em>, 17 juillet 2014, en ligne, <https:\/\/www.fxguide.com\/fxfeatured\/ape-acting\/>. <\/p>\n\n\n\n

\u2013 Short, T. L.,\u00a0Peirce\u2019s Theory of Signs<\/em>, Cambridge, Cambridge University Press, 2007, 396 p. <\/p>\n\n\n\n

\u2013 States, Bert O., \u00abThe Actor\u2019s Presence: Three Phenomenal Modes\u00bb dans\u00a0<\/em>Phillip Zarrilli,\u00a0Acting (Re)Considered\u00a0: A Theoretical and Practical Guide<\/em>, New York, Routledge, 2002, p.\u00a023-39.<\/p>\n\n\n\n

\u2013 Sturman, David J., \u00abComputer Puppetry\u00bb,\u00a0IEEE Computer Graphics and Applications<\/em>, janvier\u00a0\/\u00a0f\u00e9vrier 1998, p.\u00a038-45.<\/p>\n","protected":false},"excerpt":{"rendered":"

Since 2002, every time Academy Award nominating season rolls around, it is guaranteed that journalists will once again raise the question, \u201cWhen Will a Motion Capture Actor Win an Oscar?\u201d (Hart). This discussion began when\u00a0New Line Cinema, the company that released Peter Jackson\u2019s\u00a0The Lord of the Rings: The Two Towers, made a concerted effort to … Continued<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[56],"acf":[],"_links":{"self":[{"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/posts\/710"}],"collection":[{"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/comments?post=710"}],"version-history":[{"count":2,"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/posts\/710\/revisions"}],"predecessor-version":[{"id":713,"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/posts\/710\/revisions\/713"}],"wp:attachment":[{"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/media?parent=710"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/categories?post=710"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archee.uqam.ca\/wp-json\/wp\/v2\/tags?post=710"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}