Dances with Machines by Rebeccla Zacks

Dances with Machines by Rebeccla Zacks

Section:

Humachines

The movements of the lanky man of the videotape mesh perfectly with the undulating rhythms and cascading tones that accompany his dance. As the music swells, his gestures grow pronounced and emphatic; as the sound dwindles to the pulse of a synthesized bass or the flutter of an electronic clarinet, his motions diminish to the twitch of a hand or the slow weep of an arm. The choreographer, it seems, must have worked closely with the dancer and the composer to make such a seamless piece. The reality is more complex; This dancer is, in fact, also choreographer and composer, choosing his moves on the fly while simultaneously making the music to match in a intimate collaboration with a video camera and a homemade computer system.

Sprawled shoeless on the living room floor in his Toronto home, 38-year-old David Rokeby watches the 28-year-old version of himself on a small TV set. Though his worn jeans, wire-rimmed glasses and only slightly scruffy hair make him look like the math professor his parents wanted him to be, Rokeby has instead become an internationally known interactive artist--his multimedia installations invite gallery goers and exhibition attendees to become active participants in the artistic process.

In language that shifts easily between the professorial and the poetic, Rokeby explains both the technology and the artistic intentions behind his work. In many ways, his career sounds like that of a researcher. Rokeby thinks of each of his installations as an experiment; observing the hundreds of thousands of people who have participated with his pieces has given him an invaluable opportunity to learn about humans, machines and the very complicated relationships between them.

Through these artistic explorations, Rokeby has begun to understand how people's interactions with computers change as technogadgetry becomes more and more common. And he has uncovered some ways that machines can subtly distort human perceptions. After years of investigating such ideas, Rokeby worries that our increasing interaction with the Internet and "intelligent" technologies might cause us to devalue some of the attributes that make us human. So while others work toward a transparent interface between person and machine, Rokeby aims to expose the quirks, foibles and rough edges of that relationship. "Because I've programmed a lot, because I've built computers, I know what it's like to write a program and then watch people deal with it, and watch how my decisions change people's experiences" says Rokeby. "For me, it's important that I somehow articulate the importance of that act."

Rokeby played the videotape of his dance on a sunny January afternoon to demonstrate his best-known project: Very Nervous System. The name is an umbrella term for an ongoing series of installations the project's technological roots date back to some fiddling around with light sensors and a synthesizer that Rokeby did in the early 1980s. Over the years, Rokeby has used the technology behind Very Nervous System not only in his artistic endeavors, but also to support them; reduced to its initials, VNS is an image-processing device he builds and sells to performers, composers, researchers and other artists.

What VNS does, essentially, is translate the motion captured in a live video image into a digital signal. That signal can, via a Macintosh computer, drive electronic equipment such as synthesizers, video players and lights--all in real time. In a typical Very Nervous System installation, a body moving in the camera's field of vision becomes an integral part of the work, triggering and modulating sounds or other effects.

Rokeby develops software and hardware for projects such as Very Nervous System with little outside help, and no formal technical training. As a teenager growing up in southern Ontario in the 1970s, he taught himself programming in order to indulge a fascination with electronic music and computer graphics. At 19, with an offer on the table for a lucrative but uninspiring job in data processing, Rokeby instead embarked on a "five-year plan"--he would focus on the things that interested him and avoid those that "smacked of career." If it didn't work out, he figured, he could always go back to school and get a computer science degree.

After a stint at the Ontario College of Art, almost five years to the day after he hatched his plan, Rokeby received an invitation to show his work at the Venice Biennale, arguably the world's premier art show. The list of his artistic honors has grown steadily since.

Rokeby isn't the only artist exploring the gray area between the body, the mind and the computer ('see "Virtual Plants," p. 62), but he began doing this kind of interactive work long before most of the other artists currently on the scene, says Finnish media scholar Erkki Huhtamo, a visiting professor in the department of design at the University of California, Los Angeles. What's more, Huhtamo says, Rokeby is one of few to have constructed his own technological tools. "He's wonderfully capable of doing that," says Huhtamo, "but on the other side he has applied those tools for various artworks--a career that combines these two sides meaningfully and interestingly is rather rare."

Virtuosity in both technology and art has given Rokeby a unique perspective on the evolving ways people relate to machines. Audiences of the early installations, shown at a time before many people used much computing power outside arcade video games, were "more open to the raw experience," Rokeby recalls. They focused on the physical, and felt as though they were bumping into invisible objects that made noise. As time went on, though, people became more interested in the "geewhiz" technical aspects of the installations, and in tying the experience into a rapidly expanding computer culture.

But even as PCs became ubiquitous and "virtual reality" and "interactive media" attained buzzword status, there was an ecstatic quality to how people reacted to the piece. "Very Nervous System is very exciting," Rokeby says. "To show it is very satisfying on a certain level because people love it, and come up to you and tell you that it's brilliant and fabulous and it has changed their way of looking at something." But Rokeby began to worry that his work was too exciting, that people were so blown away by the real-time physical experience that they weren't stopping to ask the questions he had hoped they would: "'What happened between me and the machine? What does that mean for my relationship with my word processor?'"

To Rokeby, the answers to these questions have implications far beyond artistic concerns. He noticed that people tended to credit Very Nervous System with more than its fair share of responsibility for certain effects; they might, for example, synchronize their movements unconsciously to a particular preprogrammed rhythm in the mix but believe it was the technology that adjusted to them.

"Given people's general sense that machines are very smart," Rokeby says, "they have a strong tendency to attribute the smarts to the machine, even if it's their own smarts reflected back." Rokeby believes that as interactive technologies, particularly the Internet, begin to play a central role in communications, commerce and civic activities, "the sense of where the control is and where the intelligence is becomes more significant, more politically and socially important."

To get away from the distracting excitement of Very Nervous System, in the 1990s Rokeby began working on pieces that were less physical and lacked the frenzied feedback between audience and computer. In 1995, he started showing a piece that turned his early installations inside out, giving audiences the chance to watch the computer's image-processing operation as it happened and to see what the machine had been seeing for all those years. "One of the things that was always weird about Very Nervous System," Rokeby says, "is that it is a surveillance system, but no one ever felt threatened by it--people didn't feel like they were being watched."

So in Watch, Rokeby created an overtly voyeuristic experience. Video projectors shine two images side-by-side, each a processed version of a surveillance camera's view of a nearby public space. In Very Nervous System, the computer extracts motion from a video signal by comparing one frame with the last and determining which pixels have changed, but that whole procedure is invisible to the viewer. The image-processing techniques used in Watch are a dissection of VNS's internal workings. On one side only the things that are moving show up, white ghosts gliding through a black void; the other side shows only what's still, a seemingly normal but frozen black-and-white video image.

To these images, Rokeby adds a soundtrack: The occasional noise of a camera shutter or electronic beeping interrupts soft hypnotic sounds of breathing, a heartbeat and a ticking clock. It's a reminder, Rokeby says, that there might be something wrong with spying on people in this way.

Watch also serves as a reminder of how different the world can look when seen through varying technological lenses. In the early days of developing the piece, Rokeby aimed the camera out his studio window at a busy intersection. The two different video filters--one catching motion, the other stasis--became socioeconomic filters: In one image, members of a vibrant crowd moved swiftly about their business, in the other, panhandlers appeared to be sitting quietly alone on a deserted sidewalk.

Rokeby again draws from art a lesson about the impact of technology on our perceptions. The image-filtering techniques he employs in Watch are very similar to those used to compress video for storage or transmission. (Programmers save digital space by recording or sending only the changing pixels in successive frames of a moving image.) The more we use such techniques in daily life, he says, the more we wear inherently biased lenses. Rokeby says he is particularly concerned by the large number of design decisions being made "by programmers in startup companies working on intense deadlines, with very little experience of philosophy and politics."

Though the insights rokeby has gained through his art may put him in a better position to make such programming decisions, he has no desire to tie himself to his own startup company. He builds and sells only a few VNS units a year, though many more people would like to get their hands on one, according to Todd Winkler, a music professor at Brown University. "In the computer music world, his system is very well known and people talk about it, want to learn about it all the time" says Winkler, who has used his own VNS setup for more than three years in installations, performances and demonstrations. Still, Winkler understands Rokeby's decision to focus primarily on art rather than commerce. "Getting into the business of making little metal boxes that everybody in the world wants could really consume you completely," Winkler says.

On the contrary, what is consuming Rokeby these days is his latest project, The Giver of Names. It's a concept that came to the artist almost instantaneously on the day after his birthday in 1990. "The idea was there would be a computer and objects and you could present the objects to the computer and it would talk about them," he recounts. Realizing this seemingly straightforward notion, however, has taken the better part of the decade.

Part of the motivation behind The Giver of Names was what Rokeby, perhaps presciently, saw as a shift in the interplay between people and technology. As he wrote in an e-mail quoted in the catalogue for the 1998 premiere of The Giver of Names, in the 1980s it was the body that was "most challenged by the computer. ... In the '90s it seems to be the notions of intelligence, and consciousness."

Rokeby worries that as we grow accustomed to such phenomena as intelligent agents on the Internet and computerized phone systems, we may devalue certain human attributes. To talk to that computerized receptionist, for example, we often have to exaggerate and mechanize our speech--the change in enunciation is a "subtle dumbing-down process." So rather than trying to make The Giver of Names a flawless facsimile of human thought, Rokeby wanted to leave it rough, exposing the "quirky textures" of a strictly mechanical intelligence rather than using clever programming to paper them over.

In action, The Giver of Names is quirky indeed. The installation space is spare: A video camera aims at a black pedestal around which a variety of objects are strewn. Off to one side is a Macintosh G3. Visitors can select objects from the pile, or items they've brought with them, and arrange them on the pedestal; the computer captures an image and processes it, identifying colors, outlines and shapes. The system then begins a mechanical version of free-association, pulling up words that are somehow connected to the details culled from the image. The Giver of Names'"state of mind" in this process is a relational database of 100,000 objects, words and ideas.

An object on the pedestal, Rokeby explains, "is like a pebble dropped in a pond of memory, and the associations are like ripples moving away from the initial object and exciting or stimulating different parts of the memory." The words most "stimulated" in this process become the palette from which the computer chooses in forming sentences that appear on the computer screen. At the same time, male and female voices fill the installation space as they utter the words.

Presented with a soda bottle and an apple, for example, the system might pick up on the red of the apple and the shape of the bottle--these would probably stimulate the word "wine," among others, says Rokeby. "As for the sentence, it could be anything from 'The wine spilled' to something completely off the wall like 'Red aliens from inner cities flopped sumptuously on the wine-stained sofa.'"

Early on, The Giver of Names tended to talk about war. The system's fixation on generals and grenades prompted Rokeby to consider the fact that many of the databases he used were developed for military-sponsored artificial intelligence and natural-language processing research. "It's kind of interesting," he says, that the tools "used to train artificial intelligences about language will inevitably have a strong defense bias, because the best resources right now were funded by the Defense Department."

Rokeby is the first to admit that such specific lessons aren't likely to be obvious in his artworks, that most people won't listen to The Giver of Names talking about a piece of fruit and say, "Gee, I should really think about the effects of military funding on the future of artificial intelligence." But by seeing ourselves in collusion with and in contrast to the mechanical perceiving, thinking and speaking systems that Rokeby builds, we can all begin to think about, as he puts it, "how much of what we do is basically mechanical and how much of what we do does imply something richer and more complicated." And Rokeby takes great satisfaction in the unique intensity with which interactive art allows him to communicate such ideas. Not everyone gets the point of each installation, he says, "but when they get it, boy do they get it."

~~~~~~~~

By Rebecca Zacks

VIRTUAL PLANTS AND OTHER (ONLINE) CREATURES 

Though David Rokeby and other artists who create interactive installations are starting to gain a foothold in the mainstream art world, it's still unlikely that you'll be able to find their work at a museum near you. You can, however, readily find these folks on the Internet, where their combination of computer savvy and artistic sensibility produces Web sites that are well worth exploring. Here's a small sample of what's out there:

Rokeby himself provides an extensive catalogue of his pieces, along with some of his writings, at http://www.interlog.com/~drokeby/.

Austrian-born Christa Sommerer and French-born Laurent Mignonneau teamed up in 1992, and now work at the ATR Media Integration and Communications Research Laboratories in Kyoto, Japan. At their ATR Web site (http://www.mic.atr.co.jp/~christa/) you'll find images from and explanations of the elaborate virtual ecosystems they've created for installations and Web-based pieces. Sommerer and Mignonneau have built a number of unique viewer/machine interfaces: Audiences can create new plants or creatures and influence their behavior by drawing on touch screens, sending e-mail, moving through the installation space, and even by touching real plants wired to the computer.

Janine Cirincione and Michael Ferraro, both faculty members at New York's School of Visual Arts, founded their design studio, Possible Worlds, in 1992. The company's elegant site (http://www.possibleworlds.com/) provides a glimpse both of commercial projects (which include a new animated show for MTV) and of interactive installations--joystick-controlled journeys through surreal computer-generated landscapes populated with quirky characters.

New York performance and installation artist Toni Dove has shown a number of virtual-reality and video-laser disc pieces. A viewer's gestures drive the sound and images in Dove's interactive movie, Artificial Changelings, which tells the parallel tales of a 19th-century kleptomaniac and a 21st-century hacker. Read more about Dove and Artificial Changelings at http://www.funnygarbage.com/dove/, and be sure to click on the small moving pictures at the bottom of the opening screen for an archive of images from the installation.

The Ars Electronica Center in Linz, Austria, is a home for interdisciplinary investigation of art, society and technology. At the center's somewhat labyrinthine site (http://web.aec.at/), you can explore the institution's "Museum of the Future," as well as archives from its annual festival and from the Prix Ars Electronica--an international computer art competition that has had a special category for interactive art since 1990.

Finally, installation artist Stephen Wilson, a professor in San Francisco State University's Conceptual/Information Arts Program, has compiled an encyclopedic list of links on "Intersections of Art, Technology, Science & Culture" at http://userwww.sfsu.edu/~infoarts/links/ wilson. artlinks2. html. From here, you can get to pages on a vast number of artists, events, organizations and areas of research. Wilson's book, Information Arts, is due out soon.

Copyright of Technology Review is the property of MIT Technology Review and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

Copied from UMaine Folger Library Database on 05.10.17

Chicago Style Citation

Zacks, Rebecca. "Dances with Machines." Technology Review 102, no. 3 (May 1999): 58. Academic Search Complete, EBSCOhost (accessed May 10, 2017).