Section:
Humachines
The movements of the lanky man of the
videotape mesh perfectly with the undulating rhythms and cascading tones that
accompany his dance. As the music swells, his gestures grow pronounced and
emphatic; as the sound dwindles to the pulse of a synthesized bass or the
flutter of an electronic clarinet, his motions diminish to the twitch of a hand
or the slow weep of an arm. The choreographer, it seems, must have worked
closely with the dancer and the composer to make such a seamless piece. The
reality is more complex; This dancer is, in fact, also choreographer and
composer, choosing his moves on the fly while simultaneously making the music
to match in a intimate collaboration with a video camera and a homemade
computer system.
Sprawled shoeless on the living room
floor in his Toronto home, 38-year-old David
Rokeby watches the 28-year-old version
of himself on a small TV set. Though his worn jeans, wire-rimmed glasses and
only slightly scruffy hair make him look like the math professor his parents
wanted him to be, Rokeby has instead
become an internationally known interactive artist--his multimedia
installations invite gallery goers and exhibition attendees to become active
participants in the artistic process.
In language that shifts easily
between the professorial and the poetic, Rokeby
explains both the technology and the artistic intentions behind his work. In
many ways, his career sounds like that of a researcher. Rokeby thinks of each of his installations as
an experiment; observing the hundreds of thousands of people who have
participated with his pieces has given him an invaluable opportunity to learn
about humans, machines and the very complicated relationships between them.
Through these artistic explorations, Rokeby has begun to understand how people's
interactions with computers change as technogadgetry becomes more and more
common. And he has uncovered some ways that machines can subtly distort human
perceptions. After years of investigating such ideas, Rokeby worries that our increasing interaction
with the Internet and "intelligent" technologies might cause us to
devalue some of the attributes that make us human. So while others work toward
a transparent interface between person and machine, Rokeby aims to expose the quirks, foibles and
rough edges of that relationship. "Because I've programmed a lot, because
I've built computers, I know what it's like to write a program and then watch
people deal with it, and watch how my decisions change people's
experiences" says Rokeby.
"For me, it's important that I somehow articulate the importance of that
act."
Rokeby
played the videotape of his dance on a sunny January afternoon to demonstrate
his best-known project: Very Nervous System. The name is an umbrella term for
an ongoing series of installations the project's technological roots date back
to some fiddling around with light sensors and a synthesizer that Rokeby did in the early 1980s. Over the years,
Rokeby has used the technology behind
Very Nervous System not only in his artistic endeavors, but also to support
them; reduced to its initials, VNS is an image-processing device he builds and
sells to performers, composers, researchers and other artists.
What VNS does, essentially, is
translate the motion captured in a live video image into a digital signal. That
signal can, via a Macintosh computer, drive electronic equipment such as
synthesizers, video players and lights--all in real time. In a typical Very
Nervous System installation, a body moving in the camera's field of vision
becomes an integral part of the work, triggering and modulating sounds or other
effects.
Rokeby
develops software and hardware for projects such as Very Nervous System with
little outside help, and no formal technical training. As a teenager growing up
in southern Ontario in the 1970s, he taught himself programming in order to
indulge a fascination with electronic music and computer graphics. At 19, with
an offer on the table for a lucrative but uninspiring job in data processing, Rokeby instead embarked on a "five-year
plan"--he would focus on the things that interested him and avoid those
that "smacked of career." If it didn't work out, he figured, he could
always go back to school and get a computer science degree.
After a stint at the Ontario College
of Art, almost five years to the day after he hatched his plan, Rokeby received an invitation to show his work
at the Venice Biennale, arguably the world's premier art show. The list of his
artistic honors has grown steadily since.
Rokeby
isn't the only artist exploring the gray area between the body, the mind and
the computer ('see "Virtual Plants," p. 62), but he began doing this
kind of interactive work long before most of the other artists currently on the
scene, says Finnish media scholar Erkki Huhtamo, a visiting professor in the
department of design at the University of California, Los Angeles. What's more,
Huhtamo says, Rokeby is one of few to
have constructed his own technological tools. "He's wonderfully capable of
doing that," says Huhtamo, "but on the other side he has applied
those tools for various artworks--a career that combines these two sides
meaningfully and interestingly is rather rare."
Virtuosity in both technology and art
has given Rokeby a unique perspective
on the evolving ways people relate to machines. Audiences of the early
installations, shown at a time before many people used much computing power
outside arcade video games, were "more open to the raw experience," Rokeby recalls. They focused on the physical,
and felt as though they were bumping into invisible objects that made noise. As
time went on, though, people became more interested in the "geewhiz"
technical aspects of the installations, and in tying the experience into a
rapidly expanding computer culture.
But even as PCs became ubiquitous and
"virtual reality" and "interactive media" attained buzzword
status, there was an ecstatic quality to how people reacted to the piece.
"Very Nervous System is very exciting," Rokeby says. "To show it is very
satisfying on a certain level because people love it, and come up to you and
tell you that it's brilliant and fabulous and it has changed their way of
looking at something." But Rokeby
began to worry that his work was too exciting, that people were so blown away
by the real-time physical experience that they weren't stopping to ask the
questions he had hoped they would: "'What happened between me and the
machine? What does that mean for my relationship with my word processor?'"
To Rokeby,
the answers to these questions have implications far beyond artistic concerns.
He noticed that people tended to credit Very Nervous System with more than its
fair share of responsibility for certain effects; they might, for example,
synchronize their movements unconsciously to a particular preprogrammed rhythm
in the mix but believe it was the technology that adjusted to them.
"Given people's general sense
that machines are very smart," Rokeby
says, "they have a strong tendency to attribute the smarts to the machine,
even if it's their own smarts reflected back." Rokeby believes that as interactive
technologies, particularly the Internet, begin to play a central role in
communications, commerce and civic activities, "the sense of where the
control is and where the intelligence is becomes more significant, more
politically and socially important."
To get away from the distracting
excitement of Very Nervous System, in the 1990s Rokeby
began working on pieces that were less physical and lacked the frenzied
feedback between audience and computer. In 1995, he started showing a piece
that turned his early installations inside out, giving audiences the chance to
watch the computer's image-processing operation as it happened and to see what
the machine had been seeing for all those years. "One of the things that
was always weird about Very Nervous System," Rokeby says, "is that it is a
surveillance system, but no one ever felt threatened by it--people didn't feel
like they were being watched."
So in Watch, Rokeby created an overtly voyeuristic
experience. Video projectors shine two images side-by-side, each a processed
version of a surveillance camera's view of a nearby public space. In Very
Nervous System, the computer extracts motion from a video signal by comparing
one frame with the last and determining which pixels have changed, but that
whole procedure is invisible to the viewer. The image-processing techniques
used in Watch are a dissection of VNS's internal workings. On one side only the
things that are moving show up, white ghosts gliding through a black void; the
other side shows only what's still, a seemingly normal but frozen
black-and-white video image.
To these images, Rokeby adds a soundtrack: The occasional noise
of a camera shutter or electronic beeping interrupts soft hypnotic sounds of
breathing, a heartbeat and a ticking clock. It's a reminder, Rokeby says, that there might be something
wrong with spying on people in this way.
Watch also serves as a reminder of
how different the world can look when seen through varying technological
lenses. In the early days of developing the piece, Rokeby aimed the camera out his studio window
at a busy intersection. The two different video filters--one catching motion,
the other stasis--became socioeconomic filters: In one image, members of a
vibrant crowd moved swiftly about their business, in the other, panhandlers appeared
to be sitting quietly alone on a deserted sidewalk.
Rokeby
again draws from art a lesson about the impact of technology on our
perceptions. The image-filtering techniques he employs in Watch are very
similar to those used to compress video for storage or transmission.
(Programmers save digital space by recording or sending only the changing
pixels in successive frames of a moving image.) The more we use such techniques
in daily life, he says, the more we wear inherently biased lenses. Rokeby says he is particularly concerned by
the large number of design decisions being made "by programmers in startup
companies working on intense deadlines, with very little experience of
philosophy and politics."
Though the insights rokeby has gained through his art may put him
in a better position to make such programming decisions, he has no desire to
tie himself to his own startup company. He builds and sells only a few VNS
units a year, though many more people would like to get their hands on one,
according to Todd Winkler, a music professor at Brown University. "In the
computer music world, his system is very well known and people talk about it,
want to learn about it all the time" says Winkler, who has used his own
VNS setup for more than three years in installations, performances and
demonstrations. Still, Winkler understands Rokeby's
decision to focus primarily on art rather than commerce. "Getting into the
business of making little metal boxes that everybody in the world wants could
really consume you completely," Winkler says.
On the contrary, what is consuming Rokeby these days is his latest project, The
Giver of Names. It's a concept that came to the artist almost instantaneously
on the day after his birthday in 1990. "The idea was there would be a
computer and objects and you could present the objects to the computer and it
would talk about them," he recounts. Realizing this seemingly
straightforward notion, however, has taken the better part of the decade.
Part of the motivation behind The
Giver of Names was what Rokeby,
perhaps presciently, saw as a shift in the interplay between people and
technology. As he wrote in an e-mail quoted in the catalogue for the 1998
premiere of The Giver of Names, in the 1980s it was the body that was
"most challenged by the computer. ... In the '90s it seems to be the
notions of intelligence, and consciousness."
Rokeby
worries that as we grow accustomed to such phenomena as intelligent agents on
the Internet and computerized phone systems, we may devalue certain human
attributes. To talk to that computerized receptionist, for example, we often
have to exaggerate and mechanize our speech--the change in enunciation is a
"subtle dumbing-down process." So rather than trying to make The
Giver of Names a flawless facsimile of human thought, Rokeby wanted to leave it rough, exposing the
"quirky textures" of a strictly mechanical intelligence rather than
using clever programming to paper them over.
In action, The Giver of Names is
quirky indeed. The installation space is spare: A video camera aims at a black
pedestal around which a variety of objects are strewn. Off to one side is a
Macintosh G3. Visitors can select objects from the pile, or items they've
brought with them, and arrange them on the pedestal; the computer captures an image
and processes it, identifying colors, outlines and shapes. The system then
begins a mechanical version of free-association, pulling up words that are
somehow connected to the details culled from the image. The Giver of
Names'"state of mind" in this process is a relational database of
100,000 objects, words and ideas.
An object on the pedestal, Rokeby explains, "is like a pebble
dropped in a pond of memory, and the associations are like ripples moving away
from the initial object and exciting or stimulating different parts of the
memory." The words most "stimulated" in this process become the
palette from which the computer chooses in forming sentences that appear on the
computer screen. At the same time, male and female voices fill the installation
space as they utter the words.
Presented with a soda bottle and an
apple, for example, the system might pick up on the red of the apple and the
shape of the bottle--these would probably stimulate the word "wine,"
among others, says Rokeby. "As
for the sentence, it could be anything from 'The wine spilled' to something
completely off the wall like 'Red aliens from inner cities flopped sumptuously
on the wine-stained sofa.'"
Early on, The Giver of Names tended
to talk about war. The system's fixation on generals and grenades prompted Rokeby to consider the fact that many of the
databases he used were developed for military-sponsored artificial intelligence
and natural-language processing research. "It's kind of interesting,"
he says, that the tools "used to train artificial intelligences about
language will inevitably have a strong defense bias, because the best resources
right now were funded by the Defense Department."
Rokeby
is the first to admit that such specific lessons aren't likely to be obvious in
his artworks, that most people won't listen to The Giver of Names talking about
a piece of fruit and say, "Gee, I should really think about the effects of
military funding on the future of artificial intelligence." But by seeing
ourselves in collusion with and in contrast to the mechanical perceiving,
thinking and speaking systems that Rokeby
builds, we can all begin to think about, as he puts it, "how much of what
we do is basically mechanical and how much of what we do does imply something
richer and more complicated." And Rokeby
takes great satisfaction in the unique intensity with which interactive art
allows him to communicate such ideas. Not everyone gets the point of each
installation, he says, "but when they get it, boy do they get it."
~~~~~~~~
By Rebecca Zacks
VIRTUAL PLANTS AND OTHER (ONLINE) CREATURES
Though David Rokeby
and other artists who create interactive installations are starting to gain a
foothold in the mainstream art world, it's still unlikely that you'll be able
to find their work at a museum near you. You can, however, readily find these
folks on the Internet, where their combination of computer savvy and artistic
sensibility produces Web sites that are well worth exploring. Here's a small
sample of what's out there:
Rokeby himself provides an extensive catalogue
of his pieces, along with some of his writings, at
http://www.interlog.com/~drokeby/.
Austrian-born Christa
Sommerer and French-born Laurent Mignonneau teamed up in 1992, and now work at
the ATR Media Integration and Communications Research Laboratories in Kyoto,
Japan. At their ATR Web site (
http://www.mic.atr.co.jp/~christa/)
you'll find images from and explanations of the elaborate virtual ecosystems
they've created for installations and Web-based pieces. Sommerer and Mignonneau
have built a number of unique viewer/machine interfaces: Audiences can create
new plants or creatures and influence their behavior by drawing on touch
screens, sending e-mail, moving through the installation space, and even by
touching real plants wired to the computer.
Janine Cirincione and
Michael Ferraro, both faculty members at New York's School of Visual Arts,
founded their design studio, Possible Worlds, in 1992. The company's elegant
site (
http://www.possibleworlds.com/)
provides a glimpse both of commercial projects (which include a new animated
show for MTV) and of interactive installations--joystick-controlled journeys
through surreal computer-generated landscapes populated with quirky characters.
New York performance and
installation artist Toni Dove has shown a number of virtual-reality and
video-laser disc pieces. A viewer's gestures drive the sound and images in
Dove's interactive movie, Artificial Changelings, which tells the parallel
tales of a 19th-century kleptomaniac and a 21st-century hacker. Read more about
Dove and Artificial Changelings at
http://www.funnygarbage.com/dove/,
and be sure to click on the small moving pictures at the bottom of the opening
screen for an archive of images from the installation.
The Ars Electronica Center
in Linz, Austria, is a home for interdisciplinary investigation of art, society
and technology. At the center's somewhat labyrinthine site (
http://web.aec.at/), you can explore the
institution's "Museum of the Future," as well as archives from its
annual festival and from the Prix Ars Electronica--an international computer
art competition that has had a special category for interactive art since 1990.
Finally, installation artist
Stephen Wilson, a professor in San Francisco State University's
Conceptual/Information Arts Program, has compiled an encyclopedic list of links
on "Intersections of Art, Technology, Science & Culture" at
http://userwww.sfsu.edu/~infoarts/links/
wilson. artlinks2. html. From here, you can get to pages on a vast number of
artists, events, organizations and areas of research. Wilson's book,
Information Arts, is due out soon.
Copyright of Technology Review is the property of MIT
Technology Review and its content may not be copied or emailed to multiple
sites or posted to a listserv without the copyright holder's express written
permission. However, users may print, download, or email articles for
individual use.
Chicago
Style Citation
Zacks, Rebecca. "Dances with Machines." Technology
Review 102, no. 3 (May 1999): 58. Academic Search Complete, EBSCOhost
(accessed May 10, 2017).
