From A to D and back again:The emerging aesthetics of Interactive Art, Simon Penny

First published in Leonardo Electronic Almanac april 1996
and
NextWave catalog, May 1996, Melbourne Australia

Interactive art represents a radical phase-shift in western esthetics. Artists are confronting unexplored territory: the esthetics of machine mediated interactivity. Designing the interactive experience adds an entire dimension to the esthetic endevour, one without precedent in the visual and plastic arts. In the west, the visual arts have no tradition of an esthetics of interactivity. Six hundred years of painting has resulted in a rich esthetics of the still image, of color and line, shape and area, of representational geometry and perspective. The effect of six hundred years of enculturation is that we know how to read images (which observe the conventions of renaissance perspective) before we can read text. One hundred years of moving image has given us a culturally established set of cinematic conventions: we can read cinema. But as yet we have no culturally established esthtetic of real time interaction.

The implications of this observation are resounding. Jonathan Crary cogently argues that meaning in an artwork is constituted between the viewer and the work, that the `techniques of the observer' are as important as the techniques of the artist. Artists are struggling to establish a new canon, a new genre, but not only are understandings about the dynamics of the interactive experience very limited among artists, but the `techniques of the user' are non-existant. What results is a crisis of meaning: the work cannot `mean' because the user doesn't speak the language.

What I hope to do here is lay out some tentative definitions and contingent categorisations, and question some obfuscating notions, in order to stimulate closer, more rigorous thought on the subject. In the past, some people have proposed to me that a photo or a painting is interactive, I find that infuriating. So I'd like to define `interactive' for the purpose of this discussion: An interactive system is a machine system which reacts in the moment, by virtue of automated reasoning based on data from its sensory apparatus. An Interactive Artwork is such a system which addresses artistic issues. A painting is an instance of representation. A film is a sequence of representations. Interactive artworks are not instances of representation, they are virtual machines which themselves produces instances of representation based on real time inputs.

There are two new esthetic tasks in interactive art. The first is to discover the nuances and modalities of the interactive dynamic, and to find out how to apply these to esthetic goals. The second is the integration of the esthetically manipulated interactive dynamic with the other components of the work, be they physical objects, images or sounds, into an integrated esthetic whole.

Interactivity implies real time, now. The difference between an interactive application and handwritten correspondence is essentially one of time. There is a time frame which is percieved by humans as `real-time' response, it is the time frame of physiological reaction. Slower (or faster) than this is not `real time'. Switching a light switch produces a real time response, but it is simplistic. Interactivity allows reactions by a machine in real time which are intellectually and cognitively complex. This defines the medium. This capability is made possible by high speed data processing.

Observation of altered of spatial and temporal relationships has been one of the staples of new media theory for decades. These changing relationships are effecting critical social and cultural changes, and it is to be expected that artists will get interested in such changes.

Peter Callas observed some years ago that the televised spectacle was in the process of taking on the functions historically assigned to the architectural monument. The architectural monument is localised in space but persistent over time. The broadcast media spectacle, on the other hand, is localised in time but spread wide over space. A book is a medium in which time is collapsed (you read after I write) but space remains stable (I cannot access the book unless I am sharing my physical space with the book). Conversely, telematic works collapse space but `maintain' time. A book is like a monument. A conventional artwork is like a monument. Interactive artworks may be inherently Warholian, they may have significance only for their "15 minutes of fame". On the other hand some interactive artworks present themselves in the guise of static, permanent `monumental' things (this may only be a transition phase).

The advent of artworks which combine electronic image (and sound) technologies, sculptural utilisation of space and digitally coordinated automation of parts; often with electromechanical elements and recourse to data from sensors or other user input devices; presents us with a new esthetic complex, the conformations of which are only marginally amenable to the techniques of traditional artistic analysis. Therefore it seems necessary to attempt an evaluation of the esthetic modalities and formal qualities of various types of interactive work, by reference to some well known examples.

Two preliminary distinctions must be made. In the first case I would like to distinguish between systems which embrace the `top-down', serial processing single cpu model. These works are usually built on or for desktop computers. Another genre of work, inspired by robotics, complexity theory and artificial life models, utilises discrete or networked multiple processors, and are usually installations or performances with mobile components. This paper will concentrate on the former category (although my own work concentrates on the latter)

A second distinction must be drawn within the `single cpu' category: between screenal and spatial interactivity. The first extends the idea of the painting or the cinema/tv screen, the second engages the kinesthetic aspects of embodiment, the sensibilites of sculpture and dance. Hypertextual works offers through the screenal window a navigable dataspace, a space which is experienced intellectually rather than physically: Henry See's A Memory Project and Lynn Hershman's Deep Contact were early examples of this paradigm which has become ubiquitous in what is now called `interactive multimedia'.

Many interactive artworks explore the relationship between real space and virtual space. One way to understand this phenomenon is to `see' it as a visualisation of the relationship between the physical world and the imperceptible digital world. The entire phenomenon of VR may be seen this way, as a metaphorisation of digital data'space' into the visual. Certainly the conflation of digital communication with some kind of non-existent `space' is a consistent trope from the `cyberspace' of Gibson's Neuromancer to Lawnmowerman. Thankfully, VR's fifteen minutes of fame came and went. Now we look back in wonder at all the utopian brouhaha that was generated around what basically amounts to a viewing device. I note that many projects which are presented as `VR' these days have dumped the HMD and are presented on monitors or projections screens, they amount to navigable CAD projects.

Two of Jeffrey Shaw's works: Legible City and Virtual Museum, are paradigmatic of works which map a virtual space onto a real space. In these works physical movement in real space navigated the viewpoint through a superimposed virtual world.

In legible city, speed of travel through the virtual world was controlled by peadlling a stationary bicycle. There was a one-to-one correspondance between the rate of movement through the virtual world and the speed the bicycle would have been travelling, were it mobile. In effect, the bicycle is a mouse. It is worth noting that the interfaces to virtual worlds are seldom mapped one to one. Generally a small movement in the real world produces a large movement in the virtual world. This tendency replicates the paradigm of the labour saving machine. This paradigm is taken to the absurd extreme in the `flying eye' type of VR interfaces in which the gaze seems to be rocket-propelled. In Virtual Museum, simply leaning in your seat provided the motion through the virtual world.

In Legible City, the bicycle analogy allows for the dissociation between steering and linear travel. This dissociation of rotational and linear modes is carried to extreme in EVE, (Experimental Virtual Environment, Jeffrey Shaw et al). In EVE the three degrees of rotational movement are mapped directly from the movement of the head, ie, I turn ninety degrees to the right, the image also moves ninety degrees to the right. But the three degrees of linear movement are abstracted, mapped through the use of a joystick.

Another type of relation between virtual space and real space is the `paradoxical map', in which the virtual space replicates and is aligned point for point with the real space, except for certain digital phenomena. Gideon May's `Table with the Spirits'(1993) is one of several examples of this type. In most HMD (Head Mounted Display) type VR projects, Virtual Space and Real Space are totally separated.

David Rokeby's Very Nervous System presents us with a paradigm of non-visual interactivity in which human physical movement illicits responses (in this case sound) in a more or less marvellous way (in this case video image interpretation). There is no metaphor of travel here, you are (only) where you are. Yet a kind of invisible `virtual world' is mapped over the real world. Interaction with that virtual world produces sound. In some circles this instrumenting of an architectural space is referred to as `ubiquitous computing', and is discussed as the inverse of VR.

Screen-based `hypertextual works', `instrumented physical spaces' and `mapped virtual and real environments' are three new genres. Another `dimension' can be added to each of these by the inclusion of fast, wide bandwidth digital comunications technologies. We might call this tele-interactivity. There are identifiable sub-genres in which the interaction is: between people geographically separated; between a person and a machine, geographically separated; or between people geographically separated at a virtual site.

The first we might call `teleconferencing art'. Paul Sermon has produced provocative works in this vein, such as Telematic Dreaming. In this work, a video camera sends the image of Paul, one a bed, to another site at which it is projected onto a bed. At the second site, a guest is invited onto the bed. Paul can see the remote site by a returned video image. Pauls image interacts with the guest on the bed. The piece also includes two way audio contact. Through its direct eroticism pieces generates a host of questions about prescence.

A second sub-genre utilises the idea of teleoperation. Eduardo Kac and Ed Bennett's Ornithorinco allows a user to teleoperate a robot (over phone lines) to explore an enviroment. More provocatively, Stelarc's recent Fractal Flesh project allows his body to be teleoperated over the net. In both these works some aspect of the user (vision, volition) is extruded over the communication network to `be' in another place. A third sub-genre (exemplified by Agnes Hegedus' Fruit Machine) allows multiple remote users to cooperate in tasks in a shared virtual environment.

The sudden explosion of networked multimedia (via the World Wide Web) has finally realised the dreams of the pioneer network artist of the mid eigthies, (though this realisation has a decidedly commercial cast to it). A recent network project by the Berlin based Art+Com group, T-Vision is on one hand chillingly panoptical, but on the other it demonstates coordinated global data retrieval in a way that the WWW only hints at. T-Vision offers a radical new paradigm of computation. In this work, a user rolls a beach-ball sized trackball, and a globe of the world presented on the screen, rolls correspondingly. This image is made up of a patchwork of satellite and aerial photos. This world can be zoomed. In some cases one can zoom from the entire globe down to a city street in one smooth swoop. In one case, one can zoom into the Art+Com office, and look through a video camera pointing out the window, and see real time video action! T-Vision can utilise the entire internet, drawing on dispersed databases for its images, so that the globe is continually updated, even to the extent of real time video, if available.

As cinema emerged as a technology and an esthetic, it also built its institution its cultural niche. Cinema did not find its home in the museum. It built another institution, where the notion of the `original' was absurd. (No one asks if the particular print of the film you're watching is an `original or a copy'.) In the same way, interactive art will forge its own cultural niche. Is the web the environment where Interactive art will settle? Only time will tell.

Simon Penny. Pittsburgh PA 1996

Above copied from: http://ace.uci.edu/penny/texts/atod.html