My March 9th posting on motion capture, “Oh Motion Capture, What Art Thou?,” elicited an interesting comment from Vita Berezina-Blackburn, an animation specialist at Ohio State University, who finds motion capture

to be closer to traditional puppetry than cel animation and wish there would be more films featuring experimental use of motion capture which has infinite possibilities in terms of setting up virtual rigs driven by human movement.

Her wish that more artists would use motion capture for experimentation is not often heard, but did ring a bell. Back in 1999, in doing a story for Animatoon on the University of Southern California’s Division of Animation and Digital Arts, I interviewed Vibeke Sorenson, its founding chair, who mentioned she first developed an interest in the area in graduate school, when computer animation was still in its infancy; she recalled, “the real time approach was important because of the roll of the spontaneous gesture in the act of creation.” And in the “Philosophy Statement” she wrote about the program she sent me said,

The computer provides unprecedented opportunities for data transformation, both in real-time and not in real-time. It allows animators to work with both 2 and 3-D animation, in real-time interactive virtual environments. They are a hybrid form of filmmaker, functioning at various times as directors, actors, cinematographers, and editors. Computers are transformative instruments providing vast new spaces and possibilities for animators.

Sorenson is now Chair of the School of Art, Design, and Media at Nanyang Technological University in Singapore.

Berezina-Blackburn’s feeling that motion capture is a form of puppetry is also a view strongly held by John Clark Matthews, the award-winning puppet filmmaker (The Mouse and the Motorcycle trilogy, Frog and Toad are Friends, Mouse Soup, etc.), who I recently talked to about the topic. (I must note John and I are friends and in 1992 I presented a paper on his films, “Experiments in Style: the Animated Puppet Films of John Matthews,” at the Society for Animation Studies conference at CalArts.)  When his studio went under in the mid-90s, he took a job as a computer animator with Sony Imageworks, where he was a lead/supervising animator on such films as Stuart Little (the design of the title character was based on the ones he did for The Mouse and the Motorcycle films) and Polar Express; he retired five years ago, but has not lost his interest in films and performance capture.

Before Polar Express, John experimented with motion capture at Sony Imageworks (samples of this work can be found here) and realized that “performance capture is nothing more than puppeteering.” As a puppeteer he is a big booster of the process and feels there is considerable room for creativity using the process.

(In commenting on the complaints animators had with Wes Anderson’s problems had with his decision to direct Fantastic Mr. Fox long distance, he feels it “is much better [using performance capture] than an animator trying to figure out what a director wants, especially when the director is not present.”

I’m writing this from Edinburgh, Scotland, where my wife and I have been enjoying a really wonderful Society for Animation Studies conference. A full report will follow when I get back home, but I can’t help responding to the Motion Picture Academy’s new rules for defining what is animation (see press release here), which states in part that,

a sentence regarding motion capture was added to clarify the definition of an animated film. The language now reads: “An animated feature film is defined as a motion picture with a running time of greater than 40 minutes, in which movement and characters’ performances are created using a frame-by-frame technique. Motion capture by itself is not an animation technique. In addition, a significant number of the major characters must be animated, and animation must figure in no less than 75 percent of the picture’s running time.”

It was mentioned during one of the conference’s many discussions of motion capture and drew some incredulous responses from the packed room (the person reporting it wasn’t sure if it was correct), but a comment by Sheridan Institute of Technology’s Tony Tarantini made around this time about James Cameron’s assertion that there’s no animation in Avatar is worth reporting. He basically felt that at a time when animation is becoming the dominant mode of production, Cameron is try to take it [the field] away from animators.

In the paper my wife Vickie and I gave yesterday, we discussed how live-action directors, like Cameron, liked motion capture because it enabled them to do animation in a way similar to the way they film live-action (i.e., they direct actors instead of animators). For whatever reason, he does not want to see himself as an animation filmmaker and I suspect the new rules regarding motion capture were added in part to assuage people like him; it would also please Pixar, DreamWorks Animation and Blue Sky, as it would reduce possible future Oscar competition. (Needless to say, I feel motion capture is animation.)

In a discussion about the new rules at Cartoon Brew, a number of people felt that motion capture films could still be considered animation if the data was finished by animators frame-by-frame, while Ryan McCulloch asked whether this would disqualify Happy Feet, which won the Oscar for Best Animated Feature several years ago (I believe it was the same year that another mocap film, Monster House was also nominated)? And the ever sane Floyd Norman said, “This is only going to get crazier.”

These are wonderful times for animation bloggers, what with all the controversy raging about whether or not motion capture/performance capture is or is not animation. I have long said that it is, but would like to amplify my feelings a bit on the matter. The cause for this is a recent posting from the ever thoughtful Mark Mayerson, who criticizes Cartoon Brew’s Jerry Beck and Amid Amidi’s acceptance of the technique as animation; Mayerson argues that it is a postproduction technique, and thus should not and cannot be considered animation (which, he says, is a production technique).  He concludes by saying:

I’ve written extensively on how fragmented the process of making an animated film is and how so many of the acting decisions are made before the animator starts work. The character designs, the storyboard and the voice performance all make acting decisions that constrain the animator’s interpretation. There is no question that motion capture is yet another constraint, probably larger than all the others. To insist that Avatar is an animated film is to marginalize animators even more than they are in what are generally considered animated films. Is this the direction we want things to go? Better to agree with James Cameron [that it’s not animation] and focus our attention on films where animators create, not enhance, performances.

His argument is not a new one and I’m sure that any number of animators feel that motion capture work demeans them because it reduces the animation to a postproduction process. And similar arguments have long been lodged against rotoscoping. But if we take an historical approach, which I think can be useful, then the evidence is strongly in favor of both rotoscoping and motion capture being animation.

Remember, Max Fleischer invented the rotoscope in 1915 as a way to create more fluid animation; and though I have not done much research in this area, I would be surprised if anyone could find comments by any other animation pioneer that derided the process as being something other than animation. It is said that early animators struggled to have their characters move in a realistic manner, which arguably created an opening for Fleischer’s invention.

One of the earliest examples of motion capture used in lieu of animation in a mainstream production was the Brilliance commercial Robert Abel and Associates did in 1984 for the Canned Food Information Council. In the film describing its production posted above, it is clearly labeled as an animation process. And it should be noted that the company used the technique at a time when computer animation seemed incapable of easily producing realistic human movement.

Bill Kroyer, recalled in an interview with me that,

When we did Tron, all you could do is move one object, like a light cycle, and it had one thing on top, like a moving turret as in a tank. Having multiple movements was a big deal, because nobody had really written software which structures movement in a hierarchy; so when you move the shoulder, it moves the elbow, the wrist and the fingers; then you can move the elbow and it moves the wrist.

At Digital Productions, [in 1984] they wrote a program that created a hierarchy. They set up this hierarchy of a human body, but the objects were mere blocks — the head was a square and the torso was a kind of a little pyramid — but at least it had all the joints; it had a neck, back, hip, knee and everything. Then they gave me this block woman as we called her and said, “Just see if you can make it move.” And I just started creating key frames and animating; I started with the center of gravity and the hips, then I kept adding on and adding on and created this dance scene.

In other words, Robert Abel, one of the pioneers of computer animation, not having the technology available to Digital Productions (or perhaps feeling it was inadequate) turned to motion capture in much the same way that Max Fleischer turned to rotoscoping.

Thanks to Amanda Kieffer.

‘

Well, the wait is over and, whether one likes it or not, Avatar looks like the game changer that James Cameron, Jeffrey Katzenberg and other promoters of 3D movies said it would be, quieting critics who said the technology would never really work in live action. It also looks like it will be the film which legitimatizes motion/performance capture, especially as a way for live-action directors to enter the wonderful world animation (though sometimes without necessarily admitting it’s animation). It also helps that, despite its occasionally comical mixture of Star Wars and FernGully, it’s a pretty good movie.

As I wrote a year ago, “I suspect 3D will not go away anytime soon; the question , I believe, is whether or not it will go beyond being a niche market.” Avatar’s success certainly solidifies 3D’s place in the cinematic mainstream, though calling it a live action is problematic. (In this regard, do read Brad Brevet’s “Should ‘Avatar’ Be Considered for Best Animated Oscar?” on RopeofSilicon.com here and Steve Hulett’s follow-up comments on The Animation Guild blog here.) Thus, Kristin Thompson’s comments on Beowulf that “It’s still fiendishly difficult and expensive to shoot live action material in digital 3-D, so most projects are animated,” perhaps still seems to hold.

In regards to his use of motion capture, Cameron has been especially boastful about how he has overcome the last obstacle to the technology’s acceptance, that of being able to reproduce not only the reference actor’s bodily actions, but their exact facial expressions as well. As a result we are left with the spectacle of critics gushing over how, for example, Sigorney Weaver’s avatar face looks just like Sigorney Weaver’s actual face (see comparison below). This, as Brevet points out, is something that animators have been doing since Snow White and the Seven Dwarfs (actually since Otto Messmer’s pre-Felix the Cat work on Charlie Chaplin cartoons).

Also, the film really does not fully address the problem of the uncanny valley, as the mocap characters are not meant to be realistic humans, but highly stylized humanoids; a better test would be to see how Cameron would do on a follow-up to Final Fantasy: The Spirits Within.

Cameron also boasts that his work on performance capture technology will eventually lead it to becoming more commonplace and cheaper. I suppose so, but less expensive approaches already exists. For instance, director Neill Blomkamp in an interview about his District 9 with Todd Gilchrist notes:

Pretty much in any shot with an alien interacting with a human, which 99 percent is Christopher interacting with Wikus, there was Jason Cope, who was the actor who plays Christopher and who also plays all of the other aliens in the film. He was always on set in a lycra, light-reflective suit, and he would be interacting with Sharlto. It was not performance capture from a data-recording standpoint; like, there were no motion-capture cameras around. But once our live-action camera was tracked, the animators at Image Engine would sort of trace-animate the motion of Jason, almost literally like tracing him. That rotomation would become the essence of the performance of this digital creature, and then they would paint Jason out and put the digital one in, and you would have both performances and they would both be real and they would both be interacting with one another. It’s just very difficult and very expensive to paint someone out of a moving-camera [image] and then replace them with something, but we factored that in.

And despite District 9’s $30 million budget, it doesn’t suffer much in comparison with Avatar and, I would argue, is the better film.

The comparison between the two films is also interesting in that Blomkamp’s training and experience was an animator and special effects artist, while Cameron’s was not. (True, Cameron can draw, a skill which is often considered the holy grail of qualifications to becoming an animation artist or special effects artist, he never had any particular training in either craft.)

… The idea was breathtaking. Picasso’s love for American comic strips was mentioned in Gertrude Stein’s book, The Autobiography of Alice B. Toklas. He was now thinking about making an animated version of Don Quixote! Since he knew nothing about the intricate process of making animation, Picasso had left it up to his courtiers to find someone who could help him make the picture.

One of those people was a friend of the producer, so here we were sitting over a beer as I faced this mind-jolting possibility. A stream of thoughts were jostling each other through my head. Imagine working with Picasso on a storyboard! … Where could I get an animation crew in France? Would Picasso do more than just draw a storyboard? Could he learn to animate?

— Shamus Culhane, Talking Animals and Other People,  p.385

Fantastic Mr. Fox is the latest example of the recent trend of live-action filmmakers into animation, something that would have been considered an anomaly only a few years ago, or the stuff of Shamus Culhane’s shattered dream. If there is something anomalous about Fantastic Mr. Fox it is not that it is animated, but that he chose to do it using stop motion rather than motion capture, the current technique of choice of former live-action directors like George Miller (Happy Feet), Robert Zemeckis (Polar Express, Beowulf and A Christmas Carol), and the team of Steven Spielberg and Peter Jackson (for their forthcoming Tintin trilogy). (Mocap, of course, is increasingly used for such live-action/animation hybrids as James Cameron’s Avatar, while I suppose the low budget choice would be Flash, as Ari Folman did with Waltz with Bashir.)

Though this paradigm shift is probably all to the good, it has not always been greeted with enthusiasm by the animation community. After all, motion capture is often seen as something other than real animation, which live-action folk seem to latch onto as a poor substitute for “the intricate process of making animation.” Amid Amidi in a recent post on Cartoon Brew, made a similar point with regards to Flash in giving advice to Jonathan Demme about a possible animated version of Dave Eggers’ novel Zeitoun:

…I beg you not to use cheap Flash/AfterEffects-style animation. Don’t Waltz with Bashir this film, and compromise the personal impact of the story with mechanical movement. Maintain the integrity and vitality of the graphic illustration that initially drew you to the project, and bring it to life with the nuance and lushness that only traditional hand-drawn animation can provide.

Though Anderson’s film has been largely given a pass, it encountered some unusual public grumbling from some crew members. Thus, in August, the Spectacular Attractions blog reported on the reaction by cinematographer Tristan Oliver to Anderson’s decision to direct the film long distance from Paris, rather than working alongside the film’s crew in London’s Three Mills Studios communicating via email and sending copies of his favorite films on DVD “to give an impression of what he’d like to see.”

I think Wes doesn’t understand what you can do, and he often wants us to do what you can’t do, and the length of time the process takes … I don’t think he quite comprehends that, and how difficult it is to change something once you’ve started. It takes a big amount of someone’s time to change a very small thing. I think he also doesn’t understand that an animator is a performer. An animator is an actor. And this is the secret to animation: you direct your animator, you do not direct the puppet, because the puppet is an inanimate object. You direct an animator as if you’re directing an actor, and they will give you a performance. So we’ll get a note back from Wes saying “that arm movement is wrong.” But that arm movement is part of a fluid performance. And that has been really quite difficult for the animators.

            Later on, a story in the Los Angeles Times further noted

The move did little to endear Anderson to his subordinates. “It’s not in the least bit normal,” director of photography Tristan Oliver observed at the production’s East London set last spring, when production on “Mr. Fox” was about three-quarters complete. “I’ve never worked on a picture where the director has been anywhere other than the studio floor!”

Moreover, Anderson had no idea that his ignorance of stop-motion … and exacting ideas concerning the film’s look would so exasperate his crew.

“Honestly? Yeah. He has made our lives miserable,” the film’s director of animation, Mark Gustafson, said during a break in shooting. He gave a weary chuckle. “I probably shouldn’t say that.”

Now that the film has been released to general critical acclaim, all seems forgiven. And I must say I found the film quite charming and very much a piece with other Anderson films — perhaps a bit too self conscious but nevertheless likeable.

But the episode brings up the question of how live-action filmmakers will adapt to animation when their knowledge of the medium is deemed less than adequate. The reaction by Tristan Oliver and Mark Gustafson to Anderson’s methods is nothing compared to the reactions I heard regarding director Joe Dante’s handling of the animated segments of Looney Tunes: Back in Action.

In live-action, first-time directors with little or no training pose a similar problem; and over the years, producers have learned to deal with such situations. I believe Elia Kazan once noted that when he went on the set of his first Hollywood movie, A Tree Grows in Brooklyn, he didn’t have a clue what he was supposed to do; however, the cameraman, Leon Shamroy, told him he should stage the action and he would handle the camera. Something similar seems to have been the case with Orson Welles on Citizen Kane with cinematographer Gregg Toland. (Welles would acknowledge his debt to Toland by giving him equal billing in the film’s credits.)

A Hollywood cinematographer once confided to me that he was dubious about taking a high-profile assignment because he was tired of the sometimes thankless task of educating first-time directors. As thankless as these sorts of tasks might be, Hollywood has adjusted to the process and not a few of these first-timers have gone on to long careers behind the camera. It would seem the animation industry is in the process of learning to adapt in a similar fashion; the process might not be without pain, but as Fantastic Mr. Fox shows, the results need not be all bad.

P.S. (December 3rd): Another low budget choice for live-action filmmakers doing animation would, of course, be Bob Sabiston’s Rotoshop, a computerized rotoscope process used by Richard Linklater in Waking Life and A Scanner Darkly.

Brian Henson, Jim Henson’s son and co-chair of The Jim Henson Company, showed up today at Atlanta’s Center for Puppetry Arts to give a fascinating presentation about his company’s use of “digital puppetry” in their new PBS preschool series, Sid the Science Kid. The talk coincided with the Center’s new “Jim Henson: Wonders From His Workshop” exhibition.  Yesterday, he gave a similar talk at the American Film Institute Theater in Washington, DC.

Henson explained how the company’s approach to puppetry, which has always been designed with television in mind, has evolved over the years; starting with the hand puppets of shows like Sesame Street to the animatronics in films like Dark Crystal and Labyrinth to the real-time motion capture of Sid the Science Kid. For Henson, the evolution seemed natural, as he and his father have always looked to the latest technology to make their work more effective.

 Sid the Science Kid is not the first Henson project to use motion capture, but it was the first to be released. (The first was Frances, another preschool series, based on the Russell Hoban books.) One of the attractions for the company, according to Henson, was the fact that the process enabled them to show their characters from head to toe for the first time, something not really possible with their usual puppetry methods. Both puppeteers and puppets are used as motion capture actors, rather than using actor actors. Henson claims that shooting for each episode takes about two-and-a-half days, which, of course, does not include postproduction process. As the show is being done on a low budget, one can expect that the economics of the process factored into their decision.

To judge from the clips shown (on a big screen), the resulting animation is a mixed bag: while the general look is good, the characters tended to lack weight. I remember this being a problem with such early mocap shows as Nelvana’s Donkey Kong Country; the problem here is not as acute as the earlier show, but it is nevertheless still annoying. It is also interesting since Henson made a point, in demonstrating how he manipulated a Muppet for the TV camera, to give a sense of weight. (There is also a problem with lip synch, though this not really a significant issue.)

The fact that The Jim Henson Company has given its imprimatur to motion capture is certainly important for proponents of this technology. However, those who see mocap as something akin to the bubonic plague, are more likely to feel a growing sense of unease.

By the way, a few days ago, The Jim Henson company announced, “two all-new innovative CGI-animated series, Dinosaur Train and The Skrumps, at
MIPCOM Jr.” However, there was no indication whether or not they will be using mocap.

September 30th Update: Alan Louis, the Center for Puppetry Arts’ Director of Museum & Education Programs, sent over the following image of Brian Henson after his presentation when he was signing autographs. (By the way, the top image is from the reception held before the event.)

Seeing Robert Zemeckis’ Beowulf, I was reminded of the time when director John Frankenheimer came down to the University of Southern California in 1962 to show All Fall Down. During the Q&A session which followed, a student asked why he had used an elaborate tracking shot in the opening sequence. Frankenheimer replied simply that he did it because he could; after all, he pointed out that as a director in live television dramas, such shots would have been extremely difficult or impossible. I suspect that if the same question were asked of Zemeckis about the wild zoom and traveling shots in Beowulf, his answer might be much the same, except he would substitute live-action films for live TV.

And if I had to focus on Beowulf’s greatest failing, it would certainly be its self-conscious use of not only dizzying camera moves, but also its ridiculously exaggerated camera angles and perspectives. I must assume Zemeckis is a fundamentalist who believes that you should only do in animation what you absolutely cannot do in live action, whether it makes sense of not. (Orson Welles seemed to take a similar approach viz-à-viz the theater and radio when he made Citizen Kane.) This sort of thing is further aggravated by the film’s in-your-face use of stereoscopic 3D.

This is unfortunate since Zemeckis has shown, in films like Forest Gump, that he is more than just a competent filmmaker and the script for Beowulf is not as bad as his direction makes it seem. The tale, nominally based on the classic Early English poem, of how Geatsman Beowulf rescues Denmark by slaying the monster Grendel (above) and eventually becomes king, does not really need to hide behind all the film’s cinematic pyrotechnics. And in terms of subject matter, it deals with topics many in animation have long been hoping Hollywood would tackle in animated movies.

The motion capture animation, a technique that causes much of the animation blogosphere to foam at the mouth, is generally acceptable, despite the vacuous nature of some of the characters’ expressions. It is no secret that one of the attractions of motion capture to directors like Zemeckis is that they see it as a less threatening way to do animation; performance capture animation, which is is their preferred terminology, has enough similarities to live-action to make it comfortable. However, in the process he seems to have forgotten how to make a decent film; in comparison, Happy Feet, another motion capture effort from a live-action director (George Miller of Mad Max and Babe fame), is a masterpiece.

Now I’m more curious than ever to see Grendel, Grendel, Grendel, Alexander Stitt’s 1981 Australian animated version of John Gardner’s novel, which tells the story from the monster’s point of view.