Showing posts with label text/image. Show all posts
Showing posts with label text/image. Show all posts

Friday, May 7, 2010

KEY CONCEPTS OF HOLOPOETRY, Eduardo Kac



Experimental poetry followed many directions in several countries in the twentieth century. Each new direction attempted to address the historical, cultural, and often political needs of its own time. Between 1978 and 1982 I worked with countless experimental poetic styles, trying to develop my own direction. I explored traditional versification, recitation, body-based performance, visual poetry, graffiti, collage, typography, color, object-poems, sound, and a number of other possibilities. As a result of this relentless experimentation, I felt on the one hand that the printed page imprisoned the word within its two-dimensional surface, thus creating specific limits to poetic expression. On the other hand, I realized that the construction of solid three-dimensional objects gave the word a permanence and a physical presence that contradicted the dynamics of language. I was looking for a poetic language that would be malleable, fluid, and elastic. It was clear that I had to work with a new medium, beyond the page and the object -- a new medium that would still allow for the private experience of reading a poem. My conclusion was that the solution might lie somewhere between the two-dimensional surface and the three-dimensional volume -- in thin air. I envisioned in my mind's eye a poetic form that would exist beyond the page without being embodied on tangible objects. A poetic form that would be flexible, buoyant, and oscillatory as the thought process itself, and that could give new communicative power to the word. As I projected with great enthusiasm in my mind's eye what such a poetry would be like, I also thought that this dream was unachievable since it founded the principles of this new syntax in new media that -- at least for me, at the moment -- did not exist yet. My goals seemed, at first, anything but within reach. Holography was on my mind. I had read about it, but could not quite visualize what a hologram was like -- until I saw one. The experience of seeing a hologram for the first time early in 1983 was intense. I immediately recognized in this new medium the immaterial and kinetic solution to the poetic problem I had developed. I spent the next couple of years making the first holopoems and developing the theory of holopoetry. This work resulted in the first international exhibition of holopoems, in 1985, at the Museum of Image and Sound, in São Paulo. From the start the breaking down of the immaterial space of holography, as well as the development of non-linear temporal systems, have been the basis of my holographic syntax. My objective has always been not to use holography for its obvious three-dimensional qualities. I asked myself: what would be the difference between a sculpture of letters and a hologram of this sculpture? The difference was not significant. I immediately realized that holography was much more complex than the touted "illusion of three-dimensional space." This new medium has an incredible power not only to create an immaterial visual poetic experience, but to manipulate temporal systems, and to store information in ways that can be carefully controlled to generate fascinating new perceptual experiences. That is what I was after, and that is what I have been exploring since then. I must make it clear that I do not consider holographic poems those holograms that record or reproduce verbal material already successfully realized in other form or media. It is important to explore the unique qualities of the holographic medium itself and to develop a truly genuine "holographic writing." In order to clarify some of the unique aspects of my holopoetry, and also to help delineate some of the new compositional elements I have developed since 1983, I will discuss in what follows some of the key concepts of holopoetry. This will also work as a glossary of sorts, which can be used as a reference in the reading of my other texts as well as in the discussion of the holopoems themselves.

°°°

ANIMATION

Animation in holopoetry refers to the fact that the words employed in a piece are set in motion. This is usually produced on a computer and then transferred to the hologram, although purely holographic animations are also used occasionally. Computer animations are created specially for the syntax of the holopoem. This involves a complex pre-visualization experience. Computer animations that are created for video or film do not work well in a hologram. This is due to the differences between the monoscopic surface of screen-based animations and the stereoscopic space of the hologram. A holographic animation must be created taking into account the stereoscopic perception of the viewer.

BEHAVIOR

In visual poems created for print, letters and words can be said to have a specific position on the page. These letters and words are arranged into a unified visual composition. In holopoetry, letters and words cannot be said to have a specific position or composition. Instead, they exhibit a particular kind of behavior. Something happens to letters and words as they are read by the viewer. Active behavior replaces static structure.

BINOCULAR READING
I call binocular reading the process according to which some holopoems present different letters and words to each eye simultaneously. This feature is unique to holopoetry, and transforms the reading process in an intense experience. Normally, when looking at objects around us, we perceive two different points of view of the very same object. Binocular reading takes place when we read one word or letter with the left eye and at the same time a completely different word or letter with the right eye. Many holopoems –– Amalgam, for example –– rely on this principle for their syntactic and semantic efficiency.

COLOR

In holopoetry color is not fixed. It is relative. One viewer can see a letter in one color and immediately see it change into another. Two readers looking at the same word could see it in different colors simultaneously. While many holographers are disturbed by this uncontrollable behavior, I find it perfectly appropriate to stress the ungraspable nature of meaning. The oscillatory nature of color in my holopoems moves away from traditional symbolism and from the use of color as a structuring visual element. The chromatic system of each holopoem is created within certain parameters, which I specify. The creation of viewing zones and the behavior of color in a holopoem are intrinsically related, since form and relative position of viewing zones affect the diffraction of light.

DIGITAL HOLOPOEMS

Computer holopoems, or digital holopoems, are holopoems created from digital data, instead of physical letters made of metal, wood, and other materials. My first digital holopoem (Quando?, When?) was created between 1987 and 1988. Since 1989, all of my holopoems have been created with computers. If a holopoem is not made with the aid of a computer, I call it 'optical holopoem.'

DISCONTINUOUS SPACE

Discontinuous space is created in a holopoem when the homogeneity of the three-dimensional volumetric space of the hologram is broken down into discrete spaces that may or may not overlap in space, or time.

DISCONTINUOUS SYNTAX

The holopoem organized in a discontinuous space takes advantage of the logic and topology of this new poetic space. It presents the verbal material with a syntax of actual, perceptually real leaps and oscillations.

EMPTY SPACE

Quite literally, in holopoetry 'empty space' refers to the fact that the poem is read in an immaterial and empty space, visually located between the recording medium (holographic film) and the viewer –– and not on the surface of the page. This implies that holopoetry does not operate within the logic of traditional visual poetry inherited from Mallarmé, according to which the white on the page represents silence and the black type represents sound. Holopoetry undermines the subjugation of written language to phonetic systems and affirms the verbal experience based on the possible appearance or disappearance of graphemes within empty spaces. The white on the page which represented silence is removed and what remains is empty space, an absence of (printing) support which has no primary symbolic value. The vacuous gaps between words and letters do not represent positively absence of sound, because the photonic inscriptions don't stand essentially for its presence. We are in the domain of spatiotemporal writing, four-dimensional writing, if we wish, where spatial gaps don't point to anything except for the potential presence of graphemes. The voids are not to be "seen", unlike the white on the page. They are a quite literal interplay of absence and presence.

FLUID SIGN

A fluid sign is essentially a verbal sign that changes its overall visual configuration in time, therefore escaping the constancy of meaning a printed sign would have. Fluid signs are time-reversible, which means that the transformations can flow from pole to pole as the beholder wishes, and they can also become smaller compositional units in much larger texts, where each fluid sign will be connected to other fluid signs through discontinuous syntaxes. Fluid signs can also operate metamorphoses between a word and an abstract shape, or between a word and a scene or object. When this happens, both poles reciprocally alter each others' meanings. A transfiguration takes place and it produces in-between meanings that are dynamic and as important in holopoetry as the meanings produced momentarily at the poles. Fluid signs create a new kind of verbal unit, in which a sign is not either one thing or another thing. A fluid sign is perceptually relative. For two or more viewers reading together from distinct perspectives it can be different things at one time; for a non-stationary reader it can reverse itself and change uninterruptedly between as many poles as featured in the text. The holopoem Souvenir D'Andromeda is an example of this.

HOLOPOEM

A holographic poem, or holopoem, is a poem conceived, made and displayed holographically. This means, first of all, that such a poem is organized in an immaterial three-dimensional space, with complex non-linear temporal characteristics, and that even as the reader or viewer observes it, it changes and gives rise to new meanings. Thus as the viewer reads the poem he or she constantly modifies the text. As distinguished from traditional visual poetry, it seeks to express dynamically the discontinuity of thought; in other words, the perception of a holopoem takes place neither linearly nor simultaneously but rather through fragments seen at random by the observer, depending on the observer's position relative to the poem. Perception in space of colors, volumes, degrees of transparency, changes in form, relative positions of letters and words, animation, and the appearance and disappearance of forms is inseparable from the syntactic and semantic perception of the text.

HOLOPOETRY

Holopoetry is the word I coined in 1983 to name the new poetics I then introduced. By virtue of necessity, holopoetry can only be fully experienced via the creation of experimental works with the medium of holography. Today holopoems are stored on film. In the future, however, digital holopoems will be stored optically on discs. The exact storage media will change. That is not what defines a holopoem. Holopoetry is defined by unstable spaces, immateriality, four-dimensionality, interactivity, movement, relative perception, and related concepts.

HYPERPOEM

A hyperpoem is a digital interactive poem based on a system (hypertext) that branches out as the reader makes choices along the way. Hyperpoems promote a disengagement of the textual distribution characteristic of print. The node – and not the syllable – from which links irradiate is the new unit of measurement. The writer now defines the work as crisscrossing axes of combination. The reader has to make selections in a way that is similar, albeit not identical, to the way the writer has. The reader is now presented not with one narrowed-down selection of words in strings or in graphic layouts, but with an electronic field that is a complex network with no final form. In each node the poet will deploy text or add sound and moving images to it. In the future, when holography becomes digital, holographic hyperpoems will become possible.

IMMATERIALITY

In holopoetry, immateriality refers to the fact that the verbal material is organized in a space made of diffracted light, and not on any tangible or concrete form, such as the printed page. This new space, defined by photons, has no mass or tangible expression.

INTERACTIVITY

A holopoem is interactive in the sense that the natural movement of the viewer in front of the holopoem is enough to change what he or she reads. Every new movement reveals new reading possibilities, including the appearance or disappearance of verbal forms. In the future, when digital holograms become scriptable, it will even be possible to modify or add to the elements in the holographic text.

NON-LINEARITY

Holopoems are not organized with a beginning, middle, and end, as a poem in verse commonly is. Neither are holopoems printed on a page, with its suggested reading from left to right and top to bottom, or its opposite, the simultaneous ideogram. Discontinuous holopoems are read in leaps. Sequential holopoems are based on the principle of temporal reversal.

PARALLAX

An apparent change in the direction of an object, caused by a change in observational position that provides a new line of sight. Many holopoems explore parallax semantically. For example: in Omen, the word eyes spins inside a cloud of smoke. As the viewer moves from left to right and vice versa, the word appears and disappear, suggesting multiple readings.

PERCEPTUAL SYNTAX

If visual poetry developed a visual syntax –– based on the rejection of traditional syntax and on the elaborate visual treatment of the words on the page, holopoetry develops a perceptual syntax –– based on the rejection of the static syntax of print and on the development of complex and dynamic spaciotemporal verbal systems. A holopoem calls for non-linear perceptual responses to the words, which are experienced in time –– and not for the simultaneity of gestalts.

PSEUDOSCOPY

The opposite of 'orthoscopy', or the correct optical representation of a holographic image. Under certain conditions, a hologram can be made to reverse its image in space and time. A concave object is perceived as a convex pseudoscopic image. An object that rotates to the right is seen rotating to the left. Objects that appear in front of other objects are seen behind these objects in the pseudoscopic image. Objects that are seen behind the holographic plate float freely in pseudoscopic space in front of the plate. This feature is unique to holography and has been explored in holopoetry since the beginning. The first holopoem, Holo/Olho, from 1983, is based on this principle, and so is Chaos, and Wordsl 2.

SEMANTIC INTERPOLATION

In certain works, as the viewer moves relative to the holopoem, he or she perceives that each graphic line that renders the visual configuration of each letter starts to actually move in three-dimensional space. The viewer then perceives that as the lines and points go under an actual topological transformation, they slowly start to reconfigure a different letter. In Astray in Deimos, what was read as an adjective is becoming a noun, for example. I call this semantic interpolation. If the viewer happens to move in the opposite direction, the noun is transformed into the adjective. The shifting of grammatical forms occurs not through syntactical dislocations in a stanza, but through a typographic metamorphosis that takes place outside syntax. The meanings of in-between configurations can not be substituted by a verbal description, or by a synonym. Neither can it be replaced by a specific word, as gray suggests a specific intermediary position or a meaning between black and white. In holopoetry transient clusters of letters or ephemeral shapes that lay between a word and an image aim to dynamically stretch the poetic imagination and suggest meanings, ideas and feelings that are not possible to convey by traditional means.

TEXTUAL INSTABILITY

By textual instability I mean precisely that condition according to which a holographic text does not preserve a single visual structure in time as it is read by the viewer, producing different and transitory verbal configurations in response to the beholder's perceptual exploration.

TIME-REVERSIBILITY

Time-reversibility takes place in holopoems, such as Zephyr, which are made so as to be read from any temporal pole with equal semantic efficiency. This means, for example, that if one starts reading an animated holopoem from right to left (or top to bottom, or back to front), this holopoem can also be read from left to right (or bottom to top, or front to back). The time vector of the piece is reversible.

TRANSITIONAL DISCONTINUITY

In most holopoems, discontinuity is explored via leaps and gaps between the verbal material. In some cases, as in "Shema" (1989), letters are embedded in color fields that operate verbal discontinuity via visual transitions of colors. I call this 'transitional discontinuity.'

VIEWING ZONE(S)

A viewing zone is a non-physical zone, located in front of the hologram, through which the reader can actually see the words in the poem. When I create a holopoem, it is part of my writing process to decide how wide, tall and deep the viewing zones will be. I also decide the shape and relative position of these viewing zones. I can decide how many will there be and what gaps might there be between them. I can combine multiple viewing zones and edit them in many ways. I can decide on a number of viewing-zone parameters, which I use to create the unique quality of each work. The reader never sees a visual representation of these viewing zones. They are invisible. Viewing zones can be rendered sequentially and discontinuously, which helps create the space and the syntax of each holopoem.

Suggested Further Reading: Kac, Eduardo. Holopoetry: Essays, Manifestoes, Critical and Theoretical Writings; 1983-1995 (New Media Editions: Lexington, 1995).

above copied from: http://www.ekac.org/keyconcepts.html

Tuesday, April 8, 2008

Art and the Semiotics of Images: Three Questions About Visual Meaning

George L. Dillon
University of Washington
July 1999

In the last five years, the Internet has vastly enhanced our ability to display images to each other, and we can now think of ourselves not just as viewers and consumers of images but as makers and users of them ourselves. Indeed, if on the Internet we do not use images, we appear stuck in print culture and oblivious to the possibilities of the new medium. We can of course avoid giving these impressions by including some wallpaper and a few bits of eye candy, without thereby getting very far at all into graphics as a mode of conveying meaning. Schools and colleges certainly offer very little guidance outside of the area of technical communication. At present we have more questions than answers, among which three seem quite fundamental:

  1. how language-like are images?
  2. how do images and words work when they are both present?
  3. how do scenes of people gazing and posing convey visual meaning?

I will expand briefly on each of these questions and then take them in order.

Some say that images work via a second communicative system, one fully as expressive as natural language, but separate and structured independently of it. Others find visual and verbal meanings more dissimilar than similar, with the visual lacking a kind of determinacy for which verbal language seems better suited.

This question of the nature and indeterminacy of visual meaning will be the first point we will take up. The question is obviously related, namely, how do the two signalling systems work when they are placed together? In principle, visual meanings may be entirely separate from verbal ones, but as a practical matter, we rarely find pure images with no text attaching to them. Some 35 years ago, Roland Barthes wrote of our very common practices of surrounding images with words which help to specify and stabilize the interpretations of particular images:

all images are polysemous; they imply, underlying their signifiers, a 'floating chain' of signifieds, the reader able to choose some and ignore others. Polysemy poses a question of meaning and this question always comes through as a dysfunction....Hence in every society various techniques are developed intended to fix the floating chain of signifieds in such a way as to counter the terror of uncertain signs; the linguistic message is one of those techniques.

Among these "linguistic messages" are captions, labels, placards, guidebooks, brochures, and fliers—all bits of institutional apparatuses which select and present texts and images for the public. They are the tools of curators, teachers, and editors. They in turn are parts of an even larger body of institutions and practices which stabilize how images are to be interpreted and used. That is, when an image is used in a textbook or a treatise, we assume it is there to illustrate and support the meanings and information provided by the text. When an image occurs in an advertisement, we assume that it is there to help sell a product, as by depicting an instance of someone enjoying possession and consumption of the product. Thus we have in these standard deployments of text and image the harmonious relations of explication (by text) and illustration (by image).

For that reason, many who have dealt with the semiotics of images have based their discussions on images in textbooks and above all in advertising. Barthes did in "Rhetoric of the Image" saying that the intention of the advertising image is anything but elusive or problematic. Gunther Kress and Theo van Leeuwen build their semiotics of the visual on such a stable corpus of adverts and texts, and it is an entirely reasonable way to proceed—except that in studying the fenced-in image, some of the signifying potentials suppressed by the standard cases will go unrecognised. Conceptualist artists in recent decades have worked to foreground and overturn the standard canons and to explore possibilities of tension and struggle between images and text.

"It could be argued that the heart of Conceptual art in the late 1960s was not, as is often stated, the notion of the artwork being essentially linguistic, but rather the notion that it was simultaneously linguistic and visual. It is certain true that the combination of text and photograph became increasingly its archetypal form" (Godfrey, pp. 301-2).

Even the process of labelling itself, which was foregrounded rather lightheartedly by Rene Magritte, has been pushed in disturbingly directions, as Willie Doherty (see Godfrey: pp. 367-72). Relations between text and image—whether contentious or harmonious—will be the second question we will take up.

The standard scriptings of instruction and advertising also allow the viewer to place herself outside the human scenes that may be depicted. Kress and van Leeuwen describe a two-valued relation to people depicted: either they look at the viewer, and so make a "demand" for recognition, acknowledgement, response, or they are not looking at the viewer, and in a sense "offer" themselves for viewing as "third persons" ( Reading Images, pp. 121-130.) But artists and critics of recent decades have questioned the innocence of the beholder and for that matter of the subject and artist as well. Once we begin to think in terms of gaze and pose, demand/offer gets complicated in a hurry. Looking, then, is the third question to be taken up.

This little survey of graphic signification will draw on painting, photography, and digital graphics, there being no sharp line distinguishing the latter two and all three appearing via reproductions, on the Web. To be sure, some (the "post photography" folks like William J. Mitchell) have argued that the case is quite altered with digitally manipulated images which give up the claim or even appearance of representing some part of the material world, and J. David Bolter and Richard A. Grusin argue for a line of development in Western graphic culture toward pure presentation (unmediated by a representer artist) which culminates in contemporary Net graphics. There is some point to this—digital artists take their images where they find them, whether in a box of old photographs, scans of objects sitting on top of the scanner, stock photos, their browsers' caches—and we may imagine the gaze of digital taking/making as directed not through a viewfinder or past an easel, but at a monitor screen. But just as we imagine ourselves in scenes of seeing (though at one remove from the photographer's or painter's seeing) we can continue to do so at two removes, perhaps more.

One set of very substantial differences remains in the sheer torrent of unvetted images that pour down through the Net neither selected by editors nor labelled and explained by curators nor "shown" and reviewed in galleries. The mass media have already filled our lives with a vast eclectic profusion of styles and meanings, and now even amateurs can display their images on monitors around the world. The danger is not so much of an anarchy of signifying practices, however, as much as a vastly lowered expectation of signification in web graphics. If we do not pause and look and reflect along some of the lines traced here, all the great effort to build bandwidth to disseminate graphics and hardware to display them will have been for naught.

1. The (relative) indeterminacy of image meanings

For Barthes and for our discussion, language functions as a medium with relatively explicit, determinate meanings to which the "meanings" of images may on the whole be contrasted. Images "say" nothing—they are mute, they make no propositions about the world—and for that reason have been valued by modernist poets as a mode of meaning or apprehension that does not use discursive reason. To articulate this difference, I will develop a point suggested by Barthes and noted as well by Victor Burgin, namely that images, like texts, have a rhetoric of arrangements which signify, but there is no syntax that articulates their parts and binds them into a whole.

Though pictures are quite different from texts of natural language, they are not wholly different, and many have sought parallels between the two media. Like texts, most pictures are composed of parts, though the parts are bits of image (and perhaps words) arranged on a surface. When the various shapes in a picture wash and flow and blend into each other and the background, they do not seem very much like words, but when they have crisp edges, as for example in the Dada photomontage introduced here, they have attracted the term "word" and their arrangement likened to a syntax.

For example, Dawn Ades, in her overview Photomontage (Thames and Hudson, revised and enlarged edition, 1996) says of this famous piece by Hannah Höch [left] "disparate elements, photographs and scraps of text are thickly scattered over the surface, but still remain legible, like words on a page" (p. 30)—but a page, crucially, with words arranged on it, not placed in sentences. Further such montage is, as they say, flat, which means that there is no topography of concepts, no arranging into a space ordered by perspective, but only a topology of relatedness conveyed by touching and separation and spatial order. (See John Willats, Art and Representation, Princeton University Press, 1997, p. 13 and c.3.) It is hard to tell what relative size or overlapping indicates. Nonetheless, these placements signify—here by contrast, oxymoron, antithesis, and incongruity (catechresis) principally—but not by virtue of their grammatical role in sentences. That is, there is arrangement and composition of the parts, and these arrangements signify after the fashion of the artful patterning of words (the figures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logic. Rhetorical signifying is also notoriously polysemous: words arranged in a list, for example, can convey plenitude even to the point of overflowing (epitrochasm), or equivalence, or precise, detailed attention, or hierarchical ordering. And so, we may say, can images. But for language, these rhetorical figures of arrangement are a secondary signifying system; for images, they're all we've got. As long as the meanings we have to convey pertain to objects in space, a graphic display is fully as adequate, perhaps superior to, a verbal description (we often draw diagrams to clarify such meanings). But, as Paul Messaris argues (using syntax metaphorically):

as soon as we go beyond spatiotemporal interpretations, the meaning of visual syntax becomes fluid, indeterminate, and more subject to the viewer's interpretational predispositions than is the case with a communicational mode such as verbal language, which possesses an elaborate set of explicit indicators of analogy, causality, and other kinds of connections between two or more concepts (Visual Literacy (1994): p. xiii).
When the edges of the parts are blurry, or they are overlaid and merge one into the other, then figures of identity, duality (amphibole), and metaphor come more to mind. Graphics that do this sort of thing move away from representation of objects in a physical space (with defined light source) toward what Kress and van Leeuwen call "lowered" or less realistic modality—they ask to be taken more abstractly as a schematic diagram of the way the world might be or ought to be ideally or is in a certain underlying aspect). Of this well-known self-portrait by the Russian Constructivist El Lissitsky (1924) Edward Tufte says:
Overlapping images express a multiplicity of links and metaphors: the mind's eye, the hand of creation, the coordination of hand and eye, the hand and tool, the integration of person and work, the wholeness of artistic creation—and, possibly, even a halo for its saintly constructor. ( Visual Explanations (1997): p. 140.)

One can only agree with this, but Tufte plunges forward into syntactic metaphor:

By showing steps between the ideas in the mind to the reality of the paper, Lissitsky illustrates the process of graphic thinking and creation. Each visual bridge acts as a verb to link up the nouns (mind, eye, hand, compass, image, type, grid, paper) of artistic work. That work on paper then reflects back (via the pointing arrow) to eye and thought. The grid of the graph paper orders both worlds. (p. 141)

Note that the metaphor "the mind's eye" has now sprouted "mind" as a separate object "in" the picture. If the visual bridges are verbs, what verbs are they? "ISA"? "Flows forth?" Tufte's flight of syntactic metaphor obscures the difference between images and words and suggests a precision of articulation that the picture does not have. (Note that it only suggests that articulation: he doesn't spell the sentences out; language, we are reminded, can be used to intimate as well as to declare, and often is in artcriticism.) To be sure, Tufte's words are as much enthusiastic celebration of the picture as shrewd analysis of it, but they do illustrate one of society's techniques of fencing in the image, namely, by critical commentary, here specifically by turning the image into a quasi-statement. And it is to these techniques and institutional arrangements thatwe now turn.

2. Text and/versus Image

Whether or not images are inherently more polysemous than words, it is very common to find (and seek) words around exhibited or published images--titles, labels, placards, guides, "the artist's words" and so on. Classically, however, the words are peripheral to the work and confined to background information and perhaps a few interpretive hints and pointers to notable features of the work. Artists are notoriously sparing of words, preferring to let the image "speak for itself." In mass media, however, as Barthes noted, words are everywhere, from speech bubbles to voice over to writing overlaid on the image (poster or slogan fashion), and when conceptualist artists started writing extensive commentaries next to or on their images, they simultaneously broke down the image/text and High/Mass culture dividers.

To see how much energy and interest can be generated from splitting of placard and image, consider the "Statuary" series by Jacqueline Hayden on www.zonezero.com; the first one of 10 is here in the margin. These pictures are presented one by one in a highlighted oval (museum lighting) against a rich dark maroon field; each comes with a little placard button that when pressed opens a window, as here, with the placard. (The picture also can be enlarged.) The placard text in each case seems utterly unaware of the modification Hayden has made to the antique torso and thus enacts the obliviousness of the Western fine arts tradition to the look of bodies past the age of fifty. The images are rather small platinum prints done with great care and fine finish, and the exhibition is not a joke or mockery of age by youth or of museum culture by the realities of the aging body or preposterous vanity of those past their physical prime. These tensions are evoked but not resolved (since images don't say anything); rather the gaze they call forth is a compassionate one seeking andfinding a certain kind of beauty.

But that is getting ahead of the story, which begins with the standard arrangement whereby text may discreetly assist us in getting the image to float in the right directions.

To begin with the simple determining function of text, compare the following two images from an exhibit catalog from which superimposed words have been removed so that you can experience their "float" without words; you can then add the words by clicking the "Add Text" button. This first is an abundant display of supermarket prepared food, and one could imagine several possible lines of intention (they are all Kraft food products, they run heavily to cheese and preserves, they are a riot of color, shape and detail that severely challenges computer resolution, they are unbounded in all directions) but (you've clicked it already, haven't you?) the words (enlarged for legibility) anchor the display to a very conventional dismissal of "American processed food."

In this second graphic, the words"Post Human" seem to point to some kind of future world or tendency; it echoes the other "posts"—certainly poststructualism is post humanist—but what part of the "post human" world are we contemplating, and with what attitude? The image is also a bit hard to make out because of the angles; the woman may be partially submerged (but upside down?) and the light is no help either. Is this some kind of cryosleep in zero gravity? There are a lot of things that might be called post human....

There are better clues available than the words on the image: this graphic, like the preceding one, comes from an exhibition catalog for a show sponsored by the Deste Foundation for Contemporary Art in Athens, USA (Ohio?) in 1990. Called "Artificial Nature," the catalog pursues the phrase post human through many pictures of the artificial replacing, altering, and glossing over traditional human limits. It even provides another view of the striped lady, who apparently is lying in a few inches of water at the bottom of a whirlpool bath. Clearly the text does not close down interpretation here, or even give it much assistance.

In these first, rather simple cases, one has the impression that the image came first, and the words were added to interpret what was already there. When we speak of illustration, however, we are usually thinking of adding an image to an already existing text, and this relation too would seem to anchor the image. At times, however, the image seems to interpret the text quite broadly or even undermine it. Consider for example the following work from Wired magazine.

Each issue of Wired includes a 4 page (2 double) spread before the Contents page which cites a line or two from a featured article later in the magazine and functions as a teaser (or highly graphic "abstract") for the article. The sentence to be quoted and graphicked is usually long enough to support the two stage setup (double page one followed by double page two), as for example additive or contrastive pairings, or cause and effect.

The "Data" set of pages is built on lines from anarticle about a Seattle company that recovers old email, even deleted email. The lines seem rewritten over themselves. The line in "Data 1," "Backups containing millions of email messages are the digital equivalent of formaldehyde," offers a simile which is the basis of the green liquid look with its bit of magnified mosquito or crane fly in it.

Turning the page, the color changes to fiery red and hotter yellow, to a lake of fire or furnace with old disks, a key, some more cranefly wing, numbers and labels. The text says explicates the simile: "a medium where nothing decays." The fire could be taken as what puts companies in the hot seat, but it can also attract traditional connotations of Hell, the place where nothing is forgotten or forgiven. For me, seeing a sort of doll's face or mask in the fire invites this human association with the digital eternally unforgotten. This I should add carries the significance of the graphic far into a spiritual dimension that has little to do with the content of the article, which mostly turns on CYA for corporations. If the reader turns to the indicated page and begins to read the article, she likely will be disappointed by the absence of metaphysical grandeur. Which is to say that the artist takes the lines out of context and composes a visual meditation upon them; the graphic, however, is still an illustration of what the words propose.

Usually Wired's graphic serves the bit of quoted text; the next example is unusual in its relation to the quoted words. Gary Wolf's featured article in June 1999 Wired profiles Sir John Templeton and his investments in religion, specifically in showing that good religion is good business. The two double-page spread is built on lines from one of Templeton's operatives and is neither explicitly endorsed nor derided in the text of the piece. In context, it both celebrates the triumph of world capitalism and outlines the next area for it to annex, namely the realm of moral values.

On the first two pages, the two spray cleanser containers on the right margin seem to express the result of the end of the struggle for markets. Photographed in hard focus and bright light against dead black with nothing but the text to support them, they illustrate what Kress and van Leeuwen call "hyperreal" modality, which in this case links to sensual pleasure focussing on the consumer object typical of food and drink adverts (p. 169). (see also John Berger, Ways of Seeing, pp. 140-141) When we match these pages with their text declaring the settling of the fundamental battle over the free market, it is hard to avoid the effect of severe understatement (or underrepresentation) amounting to ironic deflation ("the late consumer capitalist market economy as epitomized by the choice of cleansers now dominates the scene"—with Bruce Springsteen's "57 channels and nothin' on" in the background). In the second pair of pages, the two packs of cigarettes (on sale in Japan, I hear) fill the position of the cleansers and would appear to represent the not yet realized victory of the market in the sphere of morals. (And here they bear their own texts ("Peace" and "Hope") which push even beyond "Fantastic" and "Fabulous" as Orwellian perversions of the words.) The graphics thus mock the words from Templeton's agent by reducing the grand phrases to their practical consequences in daily life: "capitalist marketing of morality would offer us immoral commodity choices packaged with positive words"—how backward can people be to withhold assent! In this display from Wired, graphics comes as close as it can to making a counter statement.

This degree of distance between graphic and text is typical of political cartoons and demonstration placards. The graphic style, however, is usually not realistic but exaggerated and parodic, which is to say of lower (less realistic) modality (as if: "this is the way the world would be if these views were real"—conditional if not irrealis, one might say, not indicative). John Heartfield's "The Butter is Gone" (1935) is a famous exemplar. The text is a quotation from a speech of Hermann Göring's, "Bronze has always made a nation strong; butter and fat at best make a people plump." And so, the butter being gone, the family is dining on metal. Although the graphic is a montage of photos, the swastika wallpaper and general absurdity of the actions lower the modality.

Victor Burgin developed exactly the opposite relation of text and graphic in his political work of the 1970s: here the image is "appropriated" from an advert and the text written on it is social critique or theory. One quite well-known one ("Possession") was done at the time of an exhibit of contemporary artists in Newcastle. The Arts Council asked for some publicity posters, and Burgin responded with "Possession" 200 copies of which were pasted up on the streets of Newcastle. Burgin intended for the diametrical opposition of text and image to catch the gaze and trigger thought. Follow-up research indicated that not many passersby remembered what the posters said, much less what they implied. For a few more years, Burgin continued to exhibit large photographs with substantial text (often over 100 words) at odds in various ways with the image. The effect is sometimes a rather professorial and preachy enumeration of the "contradictions" of late capitalist consumer society, but at other times it is more suggestive, enigmatic, or tensely ironic, as when he quotes Foucault's description of the Panopticon in a picture of a Berlin peep show with circular stage.

In "Life Demands a Little Give and Take" text and image are in the opposite relation to "Possession," namely, the text is from the commercial advert and the image is from the street. I am not sure how readily the image would make sense with no context, but in a collection of pictures that deal with the contradiction between manipulative, obfuscating culture (ideology) and real material conditions, it is not hard to see this picture as an exposure of the racist overtones of pale=beautiful. That is, we have ordinary people waiting for a bus on a nondescript street corner in modern Britain, among whom the camera's gaze falls on a woman who is distinctly not pale and who does not qualify as one of the targeted audience of the fashion magazine spiel.

Burgin was certainly not alone making text+image displays in the 1970s; much conceptualist art would fall under this rubric. Keith Arnatt, for example, exhibited a similar display, this time with a philosophic theme. Tony Godfrey, who cites this work, says, "It is uncertain whether the photograph acted as a critique of the philosophy or was merely the pretext for quoting it" (Conceptual Art, p, 172), which is a way of saying he is not sure whether the image illustrates or undermines the text. He finds the text unmemorable and finally unnecessary, saying that it serves "ultimately only to underline what is implicit." In a sense you can always say that, even when you don't say what is implicit, but for me it does highlight certain themes inherent in the situation ("the contradictoriness of all self-authenticating gestures," "uncertainty (a la Austin) of what accusation is being denied," "the making of such a photograph declaring oneself to be a real artist—is it real art?" "is the art more real with the accompanying text?" The photo might in itself cast the viewer into its reflexive abyss, but the text certainly does help. This is art that makes you think.

Not all Conceptualist artists played the big discourses of politics and philosophy off against images; some, like Karen Knorr played bits of self description or art cliches off against exquisitely photographed interiors to engage the viewer in reflection. Berger's and Richon's own contributions to the collection are even more oblique in the relation of text to image, as if the textual material is slipping out of alignment with the visual. The texts certainly do not dominate over these images, and this may partly have to do with the extreme degree of deliberation and high degree of technical finish the images exhibit. Without the texts, however, I am not sure we would have much of a clue as to what context to place the images in (they do appear in sets in these publications.)

3. The scene of looking

Three Scenes of Visual Appreciation
David Seymour (Chim): Bernard Berenson, renowned art critic and author of classic works   such as Venetian Painters of the Renaissance(1894), The Study and Criticism of Italian Art (1901, 1902, 1915,) and Essays in Medieval Art, at the age of 90 in the Borghese Gallery, Rome (1955). Natalie Bookchin and Lev Manovich: Porno_Pictorialism (1995) from Digital SnapshotsVictor Burgin: Graffitication, 1977

David Seymour (Chim): "Bernard Berenson, renowned art critic
and author of classic works such as Venetian Painters of the Renaissance (1894), The Study and Criticism of Italian Art (1901, 1902, 1915,) and Essays in Medieval Art, at the age of 90 in the Borghese Gallery, Rome" (1955).

Natalie Bookchin and Lev Manovich: "Porno_Pictorialism" (1995) from "Digital Snapshots"
Victor Burgin: "Graffitication" (1977)

The first image depicts looking as the classic scene of "art appreciation" which authorizes among other things the refined and learned connoisseur Mr. Berenson to gaze upon the statue of a largely unclad woman. We are safely at a second remove, standing behind the statue watching Mr. Berenson gaze (with "yearning" the Chim memorial website has it) at the figure that does not meet his gaze (this is Antonio Canova's Paola Borghese as Venus and she is staring off down the length of her couch). The second image, which has been digitally manipulated, has us once again gazing upon a scene of gazing, though this time we infer the gazer's view from her legs. The oval framing the scene suggests either a peephole or a classic oval frame. We do not see her expression to tell us what she makes of her collection of images of women. The title suggests erotic reverie. The third image we owe to Victor Burgin, complete with a lecture upon the voyeur as fetishist. It is unmistakably the scene of guilty viewing, unauthorized by anything. Photographs, even manipulated ones, give us very strongly the impression that we are seeing some part of the world and sharing the view of it with the photographer who saw it in his viewfinder. We can very easily be drawn in to imagined scenes of picture making, and a good bit of the meaning these pictures hold for us has to do with how we play out the roles they cast us in. These enterings into the scene are by no means confined to photographs; the art critic Michael Fried has developed extensive and detailed theories about it in relation to nineteenth century French painting (and hence in relation to modernism gnerally); but camera's automatic vanishing point perspective offers us a familiar world in which our own viewing point is always readily apparent.

Photography offers us two stories about the making of photographs. One, call it the "frozen moment of life," is associated with photojournalism, street photography, candids, and snapshots. It capitalizes on modern photography's ability to capture some part of the way the world looks in a given place and instant ("modern" because you need decently fast emulsions and sometimes good flash). The photographer may take many exposures from numerous angles and lens settings, but she will look for and try to seize "the decisive moment" in which the fullest significance of the scene is manifest. There can thus be only limited planning; graininess, high contrast,cropping which breaks objects, and blur give authenticating testimony to the unplanned "catching" of the unstaged life of the moment.

The alternate story of the scene of taking photos contrasts on most of these points, bringing it closer to studio-composed oil painting. Here nothing is left to chance—nothing occurs by chance—and the viewer may ponder as long as he wishes why this or that detail is exactly as it is. It is a tableau vivant. There is still the difference from painting that all objects are seen in the camera's eye in one exposure, none in the artist's imagination only, so that the "actual moment of time" assumption is still maintained. This is perhaps why photography is so effective as a medium of pornography: the photographer must have been just a few feet away from the subjects who were doing exactly what you see to each other (or to themselves). (It is sometimes suggested that as people begin to realize what digital manipulation of photos can do—that the participants may never have been together in one place, exchanged looks, or bodily fluids—they will lose their appeal as a focus for fantasizing.)

On either version of the basic story, then, there was a moment when the photographer looked into the viewfinder and saw the scene that ultimately appeared in a print or transparency. The photographer is thus the first viewer of the scene, and we as viewers imagine ourselves with our eyes at the place of the taking lens—where, that is, we infer the lens to be. This positioning in the scene is not just physical, however, but moral as well: that is, we can easily put on what we think to be the artistic (or salacious, or reportorial...) attitude of the photographer—his or her gaze. This line of thought seems to be heading toward suggesting that there is something dubious, at least in plenty of cases, about looking and freezing the appearance of someone or ones for public distribution. Didn't your mother teach you not to stare? Above all, not to stare at cripples, wounds, beggars, deformities, private parts, rotting food, tubes protruding from the body, and people talking with no listener in sight—as if looking (so the psychoanalytic story goes) for what is wrong, what is missing, or for reassurance that it isn't really missing ("the fetish"). So Victor Burgin gives us one image of the scene of seeing—the voyeur's peek into the lighted room of an adjacent dwelling—(along with a little lecture about the paradox of the photographic image as fetish). This is the classical viewer/voyeur scene of unlicensed, transgressive seeing—the subject is exposed to our gaze unbeknownst to them. We assume they would not want us to be looking at them this way, and the exposure is all on one side. There is something transgressive here. A border is being crossed.

But, so the argument goes, are we not being too hard on ourselves? Art is not a quarter a minute peep show, and anyway, aren't they supposed to know to pull down the blinds (and put some up!)? If they don't do it, and wander around at night naked with the lights on, are they not exhibiting their nakedness? Are they not perhaps even posing for us? Thus we come to Michael Fried's rich and celebrated discussions of absorption: the idea gained some currency in France from the time of Diderot that the viewer's pleasure in a painting with human figures, its ability to wholely engross the viewer, depends on the assumption that the subjects are not posing for us; rather, they are absorbed in whatever they are about, and this absorption is a condition for our absorption in viewing them. Since painters have always used models, this absorption effect is essentially an illusion, the conjuring of which posed a continuing problem for painters which they dealt with in various ways.

Among the most inventive were Gustav Courbet, Fried argues, and Édouard Manet, whose most famous paintings are radical and modern precisely because they overturn the whole absorption game; Olympia and his famous picnicker are naked and do not merely vaguely look toward the viewer (is this "demand" or "offer"?); rather, they stare back in a fashion usually felt to be challenging or socially amused. Suddenly the old, stable arrangement of viewer and object is up for renegotiation, and has never been settled since. Indeed, a great deal of art photography and commentary in the last thirty years has worked over and upon this theme.

One way for a photographer to work these themes is to revisit the key paintings, especially those of Manet, and remake them (so we overlap the topic of intertextuality here). Here I will look a some photographs by Jeff Wall, who seems to have set himself the goal of becoming the portrayer of modern life in late twentieth century Vancouver that Manet was in mid-nineteenth century Paris.

Jeff Wall has been working mainly in large transparencies (which indicate a strong liking for the translumination that we are now used to in Net display of graphics). Stereo (here shown as installed/exhibited and also closer up) alludes doubtless to many of the nude ladies of the great oil tradition, and strikingly by way of contrast to Manet's Olympia, who as noted breaks out of the tradition to engage the viewer. Here we have the new absorption of the Walkman which disengages the young man from any sense of being viewed. The couch too contrasts with Olympia's—it is a $50 Salvation Army special with hairoil stains, tattered piping and a nice, prominent stain. He doesn't need anybody or anything as long as he has his head space.

Walkman-induced neoabsorption also caught the eye of Jan Saudek, who gave Velasquez's Rokeby Venus a similar makeover. The Velasquez original fits nicely into this theme of absorption and gaze. In it, Venus turns her back toward us and appears to be entirely absorbed in her own image in the mirror held by Cupid. But wait, if we can see her image, then she cannot: she sees our image, and so, more indirectly and discreetly than Olympia, she gazes back. (Click on the thumbnails—the filiation between the pictures is not evident otherwise.)

Clearly Saudek's take on the irruption of Walkmans into modern life is similar to Wall's: once again,a gaze that existed in the original is absorbed by the black hole of the "personal listening device." The idea of these images spoke so much to Saudek that he did a second "Walkman" version with a classical Narcissus image. Note here the very close attention to replicating the inner and outer fabrics and the position of the feet, which is just different enough to make it clear the whole assembly was photographed anew.

Another of Jeff Wall's depictions of modern life (in particular, modern life in the Pacific Northwest) turns on a remaking of Manet's famous "Dejeuner sur la Herbe" (which keeps peeping throught the moving "reader" slit in the online version of this paper) as the very large transparency "The Storyteller" (229 x 437 cm). Here too we have gatherings in public park spaces, though the setting on the landscaped banks of a freeway overpass is a far cry from the Paris "herb" and the temperature is cooler, judging by the clothes of the figures and their little fire. Clearly, it is Vancouver (Wall taught Art History at Simon Fraser University for many years). The principle point of contact with Manet's Dejeuner is the group of three, most particularly the posture of the man, elbow on knee. Manet's grouping is directly lifted from Marcantonio Raimondi's The Judgment of Paris (—see Fried, é Manet's Modernism , p. 56). But the relations are strikingly different: the three members of the group are engaged in the woman's story, and no one, naked or otherwise, has any awareness of or interest in us. I find this a salutary treatment for those who might yearn to go to Paris and live in the Impressionist period.

(We should perhaps note that Manet's "Dejeuner" is a very strange painting—very hard to make narrative sense of (why is she sitting there nekid, the men clothed, and no one paying the slightest attention, except us?). At least one reader, namely the Barbie parodist Dean Brown, has visually shown another story painted over in the picture as we have it today.

3.2 including the shooter

The set of engagements (and non-engagements) is further enriched when the photographer includes himself or herself in the scene. I am not thinking primarily of Cindy Sherman, who includes herself as the main subject, but of photographers who depict themselves depicting. Such acts require mirrors and break the conventional twining of viewer's and photographer's eyes. That is, the viewer cannot be the "implicit photographer" when she sees the photographer represented behind the camera (assuming it is the camera that took the picture, shooting into a mirror). If she sees the photographer viewing through the taking lens, where is she viewing from? The classical precedents for such pictures are the grand canvases of Velasquez (Los Meninas may it rest in peace) and Courbet (The Painter's Studio), but as paintings, the "viewer as painter" is less compelling. That is, we know that the painter can paint himself into the scene any day he pleases, but the sense of "shared instant of time" is so much stronger that these reflexive pictures are disorienting. The one resolution, I think, is to back the viewer away from imagining himself as interacting in a scene of photographing and promoting a kind of detached analysis (and perhaps admiration) of the artifice—or amusement at what can easily come off as self-deprecating. Jonathan Miller's On Reflection includes a couple of pages (pp. 184-5) of photographer's self-portraits with taking camera; one, by Andreas Kertesz uses a distorting lens and model to suggest the queerness of the situation. But perhaps the most copious and now well-discussed body of such self-portraits is by Helmut Newton.

An introductionary essay by Urs Stahel to Helmut Newton: Selections from his Photographic Work ("Participating without Consequences: Rules and Patterns of Newton's Voyeurism, "pp. 19-30) discusses a number of Newton's pictures of himself at work photographing nudes. Among these is one ("Self Portrait with Wife June and Models," Paris, 1981) upon which Victor Burgin has lavished much semiotic and psychoanalytic attention. (see In/Different Spaces , University of California Press, 1996, cc. 2 and 3). Although Burgin begins with a textbook application of Barthesian semiotic analysis (first denotation—the non-codified description of the scene and then connotation—the cultural codes and associations of raincoats, FM spiked heels, pinup posture, followed by "rhetorical" patterning of antithesis and repetition), he moves toward explication of the feminist psychoanalytic argument of Laura Mulvey's work (and toward personal themes engaged by the picture). What both Burgin and Stahel ignore is Newton's opening up of the scene of the work and the consequences of glamour photography. This is a scene for dramatic imagining: what can the model be thinking as Newton's wife sits watching like a casting director? Is she turning toward him to receive instructions? What can Newton be thinking as he positions people (and make no mistake, they are all positioned) and dons a raincoat? Why does he make himself so short? What exactly might June be thinking? Is this a proper use of the Vogue Paris studio? Who's paying the model? and when we have finished all that, what about the other model? It seems to me this picture works exactly against Stahel's title: it drops the screens and baffles to expose relations that do have consequences—personal and material—that visual eroticism attempts to bracket and conceal.

The last of these pictures thematizing the acts of viewing, making, and seeing is a near contemporary of Newton's "Self Portrait," namely, Jeff Wall's equally wellknown "Picture for Women." Like many other Walls, it has a precursor in Manet, namely "The Bar at the Folies Bergère." This too appears to have a mirror, this time behind the subject, in which her reflection, along with that of a patron, appears. The geometry, as has been noted by a number of critics, does not seem to be quite right: if we are standing more or less directly in front of her (though not meeting her gaze), then it is hard to know where the other customer is located, or else where we are. (One critical cartoon of the times drew the scene up supplying what M. Manet had "forgotten" to put it, namely the figure of the other customer standing to the right, back to our view. In a sketch for the painting, Manet posed the girl looking sharply to her left across the viewer's gaze to the customer.) It is above all the woman's posture that echoes Manet. Here we note a bit of illusionism even in classic realism—it is hard to imagine, given the scene Manet wants to evoke, where he would set his easel, or how it would look if he chose to paint it in.

Wall, however, drops the illusion of being anywhere but his studio and also opens up the full apparatus of enhanced warehouse lighting and wiring, all of which set up superb parallel line grids to assist the eye in perspective. The light stands partition the composition into a triptych rather classically occupied by the the three principle persons: the subject, the photographer, and the camera eye/I (but the light favors her). The woman, once again reversing Manet, is looking directly at the viewer in as level a gaze as one could imagine—not challenging, or flirtatious, or submissive, supplicating, the list goes on. Well, of course she isn't looking at you, she's looking at the camera, but Wall stands a good distance away from the camera and farther forward (that is a very long cable release he has there). He appears to be looking, off the mirror, at her. But the effect of moving away from the camera is to vacate the space of the viewing eye, which is then free for the viewer to fill. The central protagonist is the camera, and the camera is you.

In his "Survey: The Mainstream and the Crooked Path" to Jeff Wall, (ed. Thierry de Duve, Arielle Pelenc, and Boris Groys, Phaidon, 1996) Thierry Duve celebrates this photograph as a breakthrough modernist photograph. For him this means broadly "self-critical and self-referential" and narrowly "conscious of the medium," which in this case is the transparency of the picture's surface (p. 29). But I do not think we are made aware of the materiality of the photograph's (or transparency's) surface; rather, I think that our awareness that we are looking at a photograph collapses. Our brain tells us the woman is posed in an utterly contrived position with her hands resting on the edge of a plywood sheet not more than 4 feet wide facing directly into a large plate glass mirror. But perceptually one or two (incompatible) conclusions seem evident: either she and her assistant Mr. Wall are waiting for you to come to the camera to take the shot, or they are about to take your picture. This completes the turning of the tables on the viewer, who becomes, finally, the viewee. Surely the title, "Picture for Women" is some sort of pointer. Then her remarkable gaze becomes The Gaze, the regard classically directed from the male observor toward the female object, now here reversed.

Conclusions

Throughout we have been working with the modern notion of art as de-automatization—as making conscious and evident the grounds of normal day to day viewing—through the violation of conventions, some of them conventions of practical graphics and some of them conventions of classical art appreciation. On the issues of rhetorical signfication, tension between text and image, and the scene of viewing, we have been able to tease out interpretations according to regular and one hopes transportable principles, using text and figures of rhetorical form, though with no hope of a syntax-semantics mapping strings of images onto logical forms. The general point seems fairly evident that insofar as a certain image does de-automatize, it obtrudes its own making and functioning in ways that would interfere with its use in advertising or instruction. Hence these are not the images and ways of signifying that will be found in your basic corpus of practical working images.

Over and over Tony Godfrey says of conceptualist art that its purpose is not to be beautiful but to make you think. Such art should resist adaptation to advertizing or instructional uses, especially the former, since the purpose of advertising is to focus your thoughts on the object for sale, not to make you think beneath the surface. But of course the industry employs many very clever people and its appetite for a new look is insatiable. Even locating the viewer as the maker of the image can be brought off, say, in a camera ad.

Here is one last image—an advert for Agfa's digital camera from the August 1999 edition of Wired. In broad outline, of course, this is conventional to and beyond the hackneyed point, selling the camera as a sex-appeal-enhancing possession. But there is a special twist—this happy encounter occurs as the camera is being used, not just displayed. Assuming the picture is what "you" see, "you look up" seems to refer to the moment when you look up through the camera's viewfinder to shoot the woman in the second story window (this is why the window casement is appears so tipped inward at the top); "she sees you" in the act of shooting, approves of your somewhat cyborgian mien (which of course is not depicted), and blows you a kiss. The crucial clue for this interpretation is the slight vertical pinching in the middle of the picture (i.e., the top and bottom edges are not straight but curve inward, then outward again). This gives "viewfinder" look. So you want us to think about the scene of shooting? OK, we can use that to sell cameras too! "incredibly easy to use ePhoto digital cameras."

Wednesday, February 6, 2008

Semiotics for Beginners: Articulation



By, Daniel Chandler.

This is only an excerpt of the larger site.

By the book.

Articulation

Semiotic codes vary in their complexity of structure or 'articulation'. The term articulation, as used by semioticians with reference to 'code structure', was derived from André Martinet's structural linguistics. The term can be misleading, since everyday usage might lead one to assume that it relates to how 'articulate' individuals are. Presumably because of this potential confusion, theoretical linguists have largely abandoned the use of the term articulation in the structural sense, preferring to refer to 'duality of patterning', but semioticians continue to use the term. Its semiotic usage is rather more closely related to the sense in which lorries (trucks) can be 'articulated' - that is, having separable sections which are linked together. Pierre Guiraud comments on semiotic articulation:

A message is articulated if it can be broken down into elements which are themselves significant. All semiotic elements must be significant. Thus the lorry on the traffic sign can be broken down into wheels, chassis, cabin, etc., but the presence of these elements does not modify the sign. On the other hand, the absence of a jacket or its permutation with a jersey changes the significance of the way someone is dressed. (Guiraud 1975, 32)

Following the model of verbal language, an articulated code has a 'vocabulary' of basic units together with syntactical rules which can be used to generate larger meaningful combinations (Innis 1986, 88-9, 99-102). A semiotic code which has 'double articulation' (as in the case of verbal language) can be analysed into two abstract structural levels: a higher level called 'the level of first articulation' and a lower level - 'the level of second articulation' (Nöth 1990; Eco 1976, 231ff). The traffic sign lacks double articulation, but rather than having no articulation, it would more commonly be referred to as having first articulation only.

At the level of first articulation the system consists of the smallest meaningful units available (e.g. morphemes or words in a language). In language this level of articulation is called the grammatical level. The meaningful units at this level are complete signs, each consisting of a signifier and a signified. Where codes have recurrent meaningful units (such as the Olympic sports pictograms and textile care symbols), they have first articulation. In systems with double articulation, these signs are made up of elements from the lower (second) level of articulation.

At the level of second articulation, a semiotic code is divisible into minimal functional units which lack meaning in themselves (e.g. phonemes in speech or graphemes in writing). These purely differential structural units (called figurae by Hjelmslev) are recurrent features in the code. They are not signs in themselves (the code must have a first level of articulation for these lower units to be combined into meaningful signs). These lower units are nonsignifying sign elements. In a code with both levels (a 'double articulated' system) the function of these lower units is purely to differentiate the minimal meaningful units. In language, the phonemes /b/, /p/ and /t/ are elements of second articulation, the function of which is to distinguish between words, such as /pin/, /bin/ and /tin/, which are elements of the first articulation of language. In language, the level of second articulation is thus a phonological level.

Semiotic codes have either single articulation, double articulation or no articulation. Double articulation enables a semiotic code to form an infinite number of meaningful combinations using a small number of low-level units (offering economy and power). The infinite use of finite elements is a feature which in relation to media in general has been referred to as 'semiotic economy'. Traditional definitions ascribe double articulation only to human language, for which this is regarded as a key 'design feature' (Hockett 1958; Hockett 1960; Hockett 1965). Louis Hjelmslev regarded it as an essential and defining feature of language (Hjelmslev 1961). Double articulation is seen as being largely responsible for the creative economy of language. Language is a semiotic system which is highly economical - employing only a small number of signs. Amongst other advantages, linguistic economy facilitates learning and recall. As for creativity, language is infinitely productive. The English language, for instance, has only about 40 or 50 elements of second articulation (phonemes) but these can generate hundreds of thousands of words. Similarly, from a limited vocabulary we can generate an infinite number of sentences (subject to the constraint of syntax which governs structurally valid combinations). Consequently, as Noam Chomsky has noted, the creative economy of language gives us the power to endlessly generate sentences which we have never encountered before. It is by combining words in multiple ways that we can seek to render the particularity of experience. If we had individual words to represent every particularity we would have to have an infinite number of them, which would exceed our capability of learning, recalling and manipulating them. John Lyons comments that 'duality, as it operates in language, is also bound up with arbitrariness. If each phonological element in a given form had to bear an identifiable iconic relationship, whether conventional or natural, to some aspect of its meaning, it is obvious that there would be severe constraints upon the possibility of combining phonological elements with one another' (Lyons 1977, 74). Roman Jakobson observed that

    In the combination of linguistic units there is an ascending scale of freedom. In the combination of distinctive features into phonemes, the freedom of the individual speaker is zero: the code has already established all the possibilities which may be utilized in the given language. Freedom to combine phonemes into words is circumscribed; it is limited to the marginal situation of word coinage. In forming sentences with words the speaker is less constrained. And finally, in the combination of sentences into utterances, the action of compulsory syntactical rules ceases, and the freedom of any individual speaker to create novel contexts increases substantially, although... the numerous stereotyped utterances are not to be overlooked. (Jakobson & Halle 1956, 74)

As Jakobson notes, even beyond the level of the sentence, the ways in which we use words are subject to linguistic conventions which limit the possibilities open to us. If we depart too far from the norms, we may fail to communicate.

Double articulation does not seem to occur in the natural communication systems of animals other than humans. As to other human semiotic systems with double articulation, Nöth notes that these include systematic codes used in library or warehouse catalogues, and 'many codes of data-processing'. He adds that 'there has been much discussion about the structure of codes such as architecture, photography, film, sign language and narratives, but no convincing conclusion has been reached concerning the articulation of these codes' (e-mail 12/8/97). Susanne Langer claims that whilst visual media such as photography, painting and drawing have lines, colours, shadings, shapes, proportions and so on which are 'abstractable and combinatory', and which 'are just as capable of articulation, i.e. of complex combination, as words', they have no vocabulary of units with independent meanings (Langer 1951, 86-7).

    A symbolism with so many elements, such myriad relationships, cannot be broken up into basic units. It is impossible to find the smallest independent symbol, and recognize its identity when the same unit is met in other contexts... There is, of course, a technique of picturing objects, but the laws governing this technique cannot properly be called a 'syntax', since there are no items that might be called, metaphorically, the 'words' of portraiture. (ibid., 88).

Rather than dismissing 'non-discursive' media for their limitations, however, Langer argues that they are more complex and subtle than verbal language and are 'peculiarly well-suited to the expression of ideas that defy linguistic "projection"'. She argues that we should not seek to impose linguistic models upon other media since the laws that govern their articulation 'are altogether different from the laws of syntax that govern language'. Treating them in linguistic terms leads us to 'misconceive' them: they resist 'translation' (ibid., 86-9).

Some codes have first articulation only. These semiotic systems consist of signs - meaningful elements which are systematically related to each other - but there is no second articulation to structure these signs into minimal, non-meaningful elements. Where the smallest recurrent structural unit in a code is meaningful, the code has first articulation only. Many semioticians argue that nonverbal communication and the various systems of animal communication have only first articulation. Nöth notes that although bird calls make use of basic units, each of these is a complete message, so bird calls have first articulation only (Nöth 1990, 151). Other examples include hotel and office room numbers where the first digit indicates the floor and the second indicates the serial number of the room on that floor. The system of related traffic signs (with red borders, triangular or circular shapes, and standardized, stylised images) is a code with first articulation only (Eco 1976, 232). Some semioticians (such as Christian Metz) argue that codes based on motivated signs - such as film and television - lack second articulation. Metz declared that in film, 'it is impossible to break up the signifier without getting isomorphic segments of the signified' (cited in Nöth 1990, 469).

Other semiotic codes lacking double articulation have second articulation only. These consist of signs which have specific meanings which are not derived from their elements. They are divisible only into figurae (minimal functional units). Nöth suggests that 'the most powerful code with second articulation only is the binary code of information theory' (e-mail, 12/8/97): this has only 2 minimal functional units, 0 and 1, but these units can be combined to generate numbers, letters and other signs. A rather less powerful system with second articulation only is that of accession codes for books, which are simply serial numbers.

Codes without articulation consist of a series of signs bearing no direct relation to each other. These signs are not divisible into recurrent compositional elements. The folkloristic 'language of flowers' is a code without articulation, since each type of flower is an independent sign which bears no relation to the other signs in the code. Unarticulated codes, which have no recurrent features, are 'uneconomical'.

Some commentators have proposed more than two levels of articulation in verbal language, but Guiraud argues that:

    The two articulations are not to be thought of in terms of syntactical levels. In fact, several levels can be distinguished in the first articulation: sentence, proposition, syntagm, word, morpheme; but each of these complex signs are simply successive combinations of the basic signs which carry the elements of meaning that are picked up at each level. (Guiraud 1975, 32)

Umberto Eco argued that cinema has a triple articulation: iconic figures; semes (combinations of iconic figures); and kinemorphs (combinations of semes) (Stam 2000, 114).

The notion of articulation is, in short, a way of dividing a semiotic system into basic levels: in the case of verbal language the levels can be termed those of sound and meaning. This clearly relates to the Saussurean analytical division of the sign into a 'sound-image' (signifier) and a concept (signified). In a semiotic system with double articulation the levels of the signifier and of the signified are relatively autonomous.

Denotation, connotation and myth are also described semiotically in terms of levels (the 'orders of signification' of Hjelmslev and Barthes). More generally, Saussure noted that signs can themselves contain signs, as in the case of a 'complex sign' such as the word 'twenty-nine' which contains the 'simple signs' twenty and nine (Saussure 1983, 130; Saussure 1974, 131). On a larger scale, an entire text is a sign which may be composed of any number of other signs (Saussure 1983, 127; Saussure 1974, 128).

Related to the idea of levels of articulation in a semiotic system is the notion of modelling systems, since 'secondary modelling systems' are described as superstructures built upon 'primary modelling systems'. Some theorists, following Yuri Lotman, refer to language in these terms. Within this framework, writing is a secondary modelling system and written texts are built upon a primary modelling system which consists of spoken language. Secondary modelling systems are thus presented as if they constitute signs of signs or representations of representations (Culler 1985, 122). Saussure adopted this position, noting that 'a language and its written form constitute two separate systems of signs. The sole reason for the latter is to represent the former' (Saussure 1983, 24; Saussure 1974, 23). Since this stance grants primacy to the spoken form, it has been criticized (notably by Jacques Derrida) as phonocentric (Derrida 1976). If we grant that even perception involves encoding, then speech is as 'secondary' as any other sign-system. Despite his own phonocentrism, Marshall McLuhan emphasized the power of the written word, alluding to its basis in arbitrariness on both structural levels: 'By the meaningless sign [written letters] linked to the meaningless sound we have built the shape and meaning of Western man' (McLuhan 1962, 50). Other theorists have extended the notion of modelling systems to 'texts' in other media, seeing them as secondary modelling systems built out of a primary 'language'. Literary texts have been seen as a second order modelling system built upon the primary linguistic system or upon the modelling system of the written word (Silverman 1983, 27; Sturrock 1986, 103). Cinematic texts have sometimes been seen as built upon a primary modelling system of 'graphic language' (Altman 1999, 175). However, whether a graphical 'language' has basic building blocks and what these might be has been hotly disputed.