Visual signs in the age of digital reproduction, Göran Sonesson

Published in Ensayos Semióticos, Dominios, modelos y miradas desde el cruce de la naturaleza y la cultura. Proceedings of the 6th International Congress of the IASS, Guadalajara, Mexico, July 13 to 19, Gimate Welsh, Adrián, (ed.), 1073-1084. México: Pourrua.


Post-photography (like post-modernity) is really the name of a position: the distance from which it becomes possible to analyse photography. This position itself derives from a historical event: the arrival of the computer image. Photography, like many other terms employed in pictorial semiotics, is a common-sense notion, which it is the task of semiotic theory to reconstruct. As such it designates a particular way of producing, by means of a mechanical device, such marking on the surface which give rise to the illusion of seeing a scene of the experimental world projected onto a two-dimensional surface, as well as that peculiar granularity which was until recently immediately recognized as the expression plane resulting from such a process. The dissociation of these two (or more) concurrent qualities transforms the arrival of computer-aided picture construction into an element of social rhetoric.

Photography as texture and construction

Indeed, leading authorities of pictorial semiotics such as Floch (1986) and Groupe µ (1992) have denied the semiotic relevance of such putatively "socio-cultural" categories as photography. There is no reason to agree with such a judgement. First of all, it is difficult to see why society should be excluded form the semiotic domain: at least, analogously to what has been argued in the case of perceptual psychology by Groupe µ, we should incorporate as many social parameters as is necessary for the purpose of analysing signification. In the second place, photography is certainly not merely a social category. Rather, what is socially grounded is our expectation that certain properties should go together, in other words, that they should correspond to what the psychologist Eleanor Rosch (1978) calls a prototype, i.e. the most probable combination of properties (cf. Sonesson 1989a). And probability ("le vraisemblable" of the French structuralists) is certainly a social concept.

Elsewhere, I have suggested that we ordinarily distinguish pictures according to three kinds of categories: construction types, such as oil paintings, linear drawings, and photographs; function types, determined by socially anticipated purposes, such a caricatures, publicity pictures, and pornographic pictures; and circulation types, defined by the channels through which pictures are conveyed from a creator to a receiver, such as posters, frescoes, television pictures, and web-page pictures. Normally we expect certain construction types, function types, and circulation types to go together: art, in the sense in which it was conceived in the last century, and against which Modernism revolted, was ideally an oil painting (construction type), meant for aesthetic enjoyment (function type), circulating through galleries and museums (circulation types; cf. Sonesson 1996). The case of the photograph is more intricate: it involves a particular texture thought to be inseparably connected to a certain origin.

Unlike most other picture categories, photography has already engendered a small body of literature concerned to lay bare the specificity of its sign function (cf. Sonesson 1989b; 1994). According to Philippe Dubois (1983:20ff), the first semiotical theories of photography tended to look upon the photograph as a mirror of reality, or, in Peircean terms, as an icon; then came that most celebrated generation of iconoclasts who tried to demonstrate the conventionality of all signs, supposing even the photograph to present a "coded" version of reality, or, as Peirce (according to Dubois, at least) would have said, a symbol; and finally the photograph was seen for what it really is, in Dubois’ view: an index, more specifically, a trace left behind by the referent itself. Without subscribing to Dubois’ uni-linear story of progress, I will use his distinctions as a handy classification of the relevant epistemological attitudes.

The authorities quoted by Dubois from the first period are in fact largely pre-semiotical: Baudelaire, Taine, Benjamin, Bazin, but also Barthes. Most of the minor classics of semiotics are mustered for the part of the symbol-addicted team: Metz, Eco, Barthes, Lindekens, Groupe µ, and so on. In the part of the daring moderns, we find, apart from Dubois himself, such writers as Bonitzer, Krauss, Vanlier, but also Barthes, Benjamin and Bazin, when considered from another vantage point, and, of course, Peirce. Barthes here appears as a proponent of the iconic conception, because of having opposed the conventional, historically relative, and learned character of drawing to the "quasi-tautological" nature which photographic expression shows in relation to its content. His claim to be a vindicator of the symbol view probably rests on his listing of photographic "connotations". And he is considered a pioneer for the index theory for the reason that he has described each photograph as implying that "this has taken place" ("cela a été"). In fact, also Peirce may be considered as an authority for all conceptions: he sometimes tells us the photograph is an index, sometimes an icon, and elsewhere he observes that all real icons are somewhat conventional.

The mapping rules of chirography

Actually, Barthes’ (1964) defence for the iconicity view may not be as naive as has been claimed by Floch and others. It could be interpreted as the theory that drawing, but not photography, requires there to be a set of rules for mapping perceptual experience onto marks made with a pen on paper; and these rules imply a particular segmentation of the world as it is given to perception, picking up some (kinds of?) features for reproduction, while rejecting others, and perhaps emphasising some properties at the same time as others are underplayed; and all this takes place under given historical circumstances, which are responsible for varying the emphases and the exclusions. Against this, it might be argued that Renaissance perspective, and a lot of other principles of rendering, are built into the camera: but the point is precisely that they are incorporated into the apparatus, and thus not present to consciousness in the actual process of picture production.

The idea becomes more reasonable when expressed as a difference between the types of mapping rules involved in photography and hand-made pictures, respectively. If we look upon the relationship between the pictorial content and its referent in the outside world as a kind of indexicality, more in particular as a factorality (a relation of part to whole), we may interpret Barthes to claim that photography is able to pick up particular proper parts ("son sujet", "son cadre") and perceptual angles of vision ("son angle") of the whole motive, but cannot chose to render just a few of its attributes. In some all too obvious ways this is false: for essential reasons, photography only transmits visual properties, and it only conveys such features as are present on the sides of the object fronting the camera. Also, depending on the distance between the camera and the motive, only features contained in a particular range of sizes may be included.

As long as no trick photography is involved, however, it seems to be true that, without recurring to later modification of the exposed material, photography is merely able to pick up features, or restrict its selection of features, on the global level, whereas in drawing, local decisions can be made for each single feature (cf. Sonesson 1989b:36ff; Dubois 1983:96f). This also applies to all other rules of photographic transposition listed by Ramírez (1981: 158ff) and Gubern (1974:50ff): abolition of the third dimension, the delimitation of space through the frame, the exclusion of movement, mono-focal and static vision, granular, discontinuous structure of the expression plane, abolition or distortion of colour, limitation to scenes having a certain range of luminosity, and abolition of non-visual stimuli,.

The recent turn to an indexicalist position was taken together by Henri Vanlier (1983), Philippe Dubois (1983), and Jean-Marie Schaeffer (1987), yet the three theorists are very different in many respects. While Dubois and Schaeffer base their claims on Peirce’s theory, Vanlier’s notion of indexicality (split into the untranslatable opposition between "indice" and "index") is not really derived from Peirce; indeed, his "indice" is actually, in the most literal sense, a mere trace, of which he offers some very usefully descriptions. Schaeffer takes a less extreme stand than Vanlier and Dubois, arguing that the photograph is an indexical icon, or, in other cases, an iconical index (cf. Sonesson 1989b: 46ff).

Limitations of indexicality

When photographs are said to be indexical, it is contiguity, not factorality, which is meant, and a particular kind of contiguity at that: abrasion, i.e. the particular indexical relationship resulting from the fact that the object which is to become the referent has, on some prior moment of time, entered into contact with, and then detached itself from, what later is to become the expression plane of the sign, leaving on the surface of the latter some visible trace, however inconspicuous, of the event (cf. Sonesson 1989a,40; 1989b:46ff). In fact, as Vanlier (1983:15) notes, the photograph must be taken as a direct and certain imprint of the photons, and only as an indirect and abstract one of the objects depicted. Unfortunately, Vanlier (1983:23, 25) himself rapidly seems to forget this distinction, talking about the scene as being the cause of the picture. In any case, he fails to note that, if the indexicality obtains between the photons and the plate, it does not occur between the same relata as the semiotic function, i.e. the objects depicted and the picture. Dubois (1983:66) at least is more consistent with his conception of the photograph as being an index when he takes the photogram to be its most characteristic instance; yet, if this is the kind of photograph he is intent on explaining, he will fail to characterise what most people would consider prototypical photography.

Certain limitations are imposed on the photographic trace by the support on which it is inscribed. Some of these are mentioned by Vanlier: the quadrangular shape of the photograph, its digital nature, the information it leaves out, its inability to record the temporal aspects of the process giving rise to the trace, etc. This may be restated by saying that the photograph is not only an indexicality of the objects, or even the photons, but also of the properties of the film, of the lenses, of the photographic device generally, of the space covered by the photons, and so on. This observation is quite parallel to the one made in the study of animal traces, according to which the same animal will leave different traces on different ground (see Sonesson 1989a,I.2.6. and 1989d)

The trouble with a purely indexicalist account of photography is that it cannot explain what the photograph is a picture of. There is no intrinsic reasons for considering the cause producing a trace (and even so, we have seen than many more causes than the motif may be held responsible for the trace) to be a more important type of cause than the others. Indeed, we can only explain the importance of the motif, when we realise that a trace, in the most central sense of the term, contains not only indexical but also iconical aspects, and if we begin by admitting that a photograph is a kind of pictorial sign, and that all such signs are first and foremost grounded in the illusion of similarity.

Contrary to Vanlier and Dubois, Schaeffer (1987:101ff) thinks that the photograph may be an indexical icon in some cases, and, in other cases, an iconical index. It could be argued, however, that the photograph, contrary, for example, to a hoof-print, is always primarily an icon (Sonesson 1989b:68ff). While both the photograph and the hoof-print stand for a referent which has vanished from the scene, the signifier of the former sign continues to occupy the place that was that of the referent, and it stills remains temporally dated, whereas the photographic signifier, like that of the verbal sign, is omni-temporal and omni-spatial, tokens of its type being apt to be instantiated at any time and place (although only after the referential event and the time needed for development). In sum, in the case of a footstep, a hoof-print, etc., both the expression and the content are located at a particular time and place; in verbal language, none of them are; and in the case of photography, it is only the content (or, strictly speaking, the referent) which is bound up with spatio-temporality. Thus, the hoof-prints, present where before the horse was present, tells us something like "horse here before"; but the photograph of a horse, which most likely does not occupy the scene where the horse was before, only tells us "horse", and then we may start reconstructing the time and the place .

At this point, it may seem that we could say that, whereas the hoof-print is first and foremost an index, the photograph must originally be seen as an icon, before its indexical properties can be discovered. In fact, however, things may be still more complicated. Schaeffer is of course right in pointing out, against Peirce, that not all indices involve some iconic aspect, but it so happens that the hoof-prints, just like all other imprints and traces, in the narrow sense of these terms, also convey a partial similarity with the objects for which they stand. We have to recognise the hoof-print as such, that is, differentiate if from the traces of a man’s feet, or of a donkey’s, a well as from fake hoof-prints, and from accidental formations worked by the wind in the sand. Only then can we interpret the hoof-prints indexically. It remains true, however, that the essential meanings of the hoof-prints are embodied in indexicality: they tell us the whereabouts of the animal.

In the case of a photograph, on the other hand, we do not need to conceive of it indexically to be able to grasp its meaning. It will continue to convey signification to us, whether we are certain that it is a photograph or not. Indexicality, in photographs, really is a question of second thoughts and peculiar circumstances. It therefore appears that indexicality cannot be the primary sign relation of photographs, although it is an open potentiality present in their constitution, which is exploited in certain cases. First and foremost, the photograph is an iconical sign.

The proof of this is that of two pictures which look exactly alike, one may be a photograph and the other could have been constructed on the computer with the aid of some graphic application, either be combining elements of photographs scanned into the computer, or by using some algorithms for calculating the three-dimensional viewpoint and the position of the sources of light. But it is quite reasonable to claim that, at the present time, both pictures convey a connotation of photographicalness (as do, of course, to a lesser extent, some hyper-realistic paintings).

From chirography to technography

There is certainly some truth in Barthes’ intuition, however confusedly expressed, which locates the difference between photographs and hand-made pictures in the global and piece-meal character of the respective rules of transformation. One of the disturbing facts about post-photographic pictures, however, is that they are not hand-made, but still allow for local transformations.

While contemplating the prospects of a "science of depiction", in some ways analogous to linguistics, the psychologist James Gibson proposed a primary distinction between two large categories of picture signs, or, more generally, between those signs which constitute markings on surfaces: between photographic and chirographic pictures, that is, literally, pictures produced by the workings of luminosity on a surface, and pictures the markings of which are assembled by hand. According to Gibson (1978:228f; 1980) a picture is "a surface so treated that it makes available a limited optic array /---/ of persisting invariants of structure" at some point of observation. But he also speculates that to prehistoric man, just as to the child, the picture make up "a progressive record of movement", a layout receptive to traces, long before it is discovered also to "delineate something". If the record is of a stylus, brush, pen, pencil, crayon, marker or another hand-held tool, the result will be a chirographic picture; and if the traces have been produced by a camera, including its accessory equipment, we will have a photographic picture.

Considered in this way, chirographic pictures, just like photographic ones, are largely indexical: they are indexical of all forces contributing to produce them. It has been suggested that, to the toddler, the marks left on the paper are accidental traces of a motor activity which is at first experienced as rewarding in itself; only at about 18 months, with the emergence of the semiotic function, will the child react with disappointment when no strokes and dots result form the contact of the marker with the paper, and only at 3 years will he refuse to draw in the air (Cf. Gardner 1973:215ff; 1980ff). What was, in Hjelmslevean terms, at first accidental substance now becomes the very form of the act, defined by the principle or relevance known to us as the making of a drawing. Put in another way, chirographic pictures are indexical in origin: only later will iconicity come to the fore. Contrary to the case of photography, chirographic indexicality is thus entirely distinct from the iconic relation.

The "photograms" made by avant-garde photographers such as Moholy-Nagy, Man Ray, and Schaad, as well as preceding the invention of the common photograph in the experimental work of Niepce and Talbot, could be considered limiting-cases: they are actually comparable to the foot-prints left on the ground, light being the operating agent instead of mechanical pressure. When placed directly upon the photographic paper, without a camera obscura as an intermediary, two-dimensional objects will give rise to silhouettes, which can be easily identified; but when three-dimensional objects are used and the source of light is moved, the configurations which result are due to complex interactions, not only between the contiguous part of the object and the emulsion, but between the position of the light source and the non-directly contiguous parts of the object. Paradoxically, it is the camera obscura, which diminishes the contiguity between the object and the expression plane of the pictorial sign, which brings about the illusion of seeing a configuration, and which thus makes it possible to trace the configuration unambiguously back to its real-world source (Sonesson 1989a:64).

To grasp the nature of chirography, it may be necessary to oppose it to something which is vaster and less specific than photography. "Hard icons" is a term coined by Tomas Maldonado (1974) to describe signs which, in addition to bearing resemblance to that which they depict, are related to them as traces to that which produced them. Examples would be X-ray pictures, hand impressions on cave walls, "acoustic pictures" made with the aid of ultrasound, silhouettes, configurations left on the ground by people who were out walking in Hiroshima at the moment of the explosion of the nuclear bomb, thermograms, pictures made with "invisible light" to discover persons hiding in the woods — and ordinary photographs. The real contiguity between the picture and its referent is here taken to guarantee the cognitive value of the picture. It is important to note that "hard icons" cannot simply be signs which are both indexical and iconic, for that is true also of chirographs: there must be coincidence between their respective indexical and iconic grounds.

The case of the computer image

If photography is defined by the double relation of contiguity and similarity between its expression and its content or referent, a few surprising cases of photography, or perhaps rather some curious intermediary cases between photography and chirography, will turn out to exist. During the 18th century a device for producing drawings from silhouettes was in use: it consisted of a chair having a source of light on one side and a screen on which the shadow of the person sitting in the chair was cast on the other. The contours were conveyed by contiguity to the screen, but were not by themselves retained there, because of the lack of photographic emulsion, but had to be filled in by hand. In the case of the curious device known as a physionotrace, a view-finder was moved along the contours of the object, producing a contiguity between these contours and the gaze; thanks to another contiguity, this time between the view-finder and a stylus, the corresponding figure was concurrently traced onto a paper.

At this point, computer-generated pictures again seems to pose a problem. In some rather indirect way, contiguity may be said to play a role in the creation of certain computer images: the computer mouse, and even more clearly the digitalisation board, are clearly hand-held devices leaving a record on a surface, although this surface is not directly the monitor, and even less the print-out. Even though the very notion of a surface seems doubtful in this case, it is certainly true that by moving the mouse, we bring about a record of some kind of abrasion, which may even be accidentally produced by a toddler or a cat more interested in the movement for movement’s sake. The abrasion is not caused by physical pressure (apart from the first phase in which the hand grasps the mouse and presses it to the desktop) or by light, but by electronic impulses.

However, pictures produced, not by means of a mouse or some other hand-held device, but created entirely or in part by mathematical algorithms do not seem to involve indexicality in any essential way. On the other hand, 3D-scanners do not only rely entirely on indexicality, but also supposes a coincidence of the indexical ground and the sign function, in a way which is reminiscent of photography, and perhaps even more of the physionotrace.

As for the indexical relation of factorality between referent and content in computer images, it seems to be identical to that of truly chirographic, rather than photographic, pictures: continuity is suggested by what we know of the connections obtaining in our socio-cultural lifeworld, not by the acquaintance with some individual object of real-life experience. Yet the photographic connotations conveyed by some computer images tend to suggest that some real-world objects have been present around, as well as in front of, the camera.

According to another classification, proposed by Roman Gubern (1987b:46f), chirographic pictures, such as drawings, are distinguished from technographic pictures, which is a group comprising photographs as well as pictures produced by the cinematographic camera and the video. Among the technographic pictures, we might perhaps also locate those produced by the physionotrace and similar devices, and also what Gubern (1987a:73ff) elsewhere terms synthetic pictures, i.e. pictures produced by means of a computer. The disturbing fact about the latter is that they may look exactly like photographs, although they do not regulate themselves on contiguity, but are rather (indirectly) mediated by similarity. Traditionally all hand-produced pictures relied on similarity, since they depended on what Gibson calls the hand-eye-system, whereas all machine-made pictures were indexically derived – until this simple organisation was destroyed by computer graphics.

Espe (1983) has suggested a threefold division of graphics, which comprises all kinds of manipulations of two-dimensional surfaces: photographics, chirographics, and typographics. Like the term photographics, typographics here retains is ordinary sense, but it could perhaps also be conceived to mean, more broadly, the production of markings on surfaces by means of standardised implements. The case of computer pictures is ambiguous: it is one of the remarkable feats of desktop publishing that it de-standardises type-fonts, permitting them to the varied along a number of dimensions (size, obliqueness, etc.), thus bringing them closer to being pictures (Sonesson 1989b.34ff). But the facility with which documents are copied and combined on the computers also makes it possible to create pictures from standardised picture-elements (clip-art, etc.) or from fragments of individual pre-existing pictures, which serves to bring the production of pictures closer than ever to the methods of verbal and other sign production.

From mechanical reproduction and to digital production

When, in 1936, Walter Benjamin described our time as the age of mechanical reproduction, his diagnosis was not radical enough. By engendering ever new tokens, mechanical reproduction effectively reduces all tokens to their type, destroying the uniqueness of the characters of human history, and their infinitely ineffable creations, the nimbus of individual creation in its hic et nunc. Doing away with the "aura" of the work of art, it apparently only leaves the bare bones of categoricalness. Yet mechanical reproduction presupposes there to be an individual object to reproduce in the first place: a chirographic or photographic original, a first token which creates the type from which further tokens are derived. In verbal language, on the other hand, the type seemingly pre-exist to all its tokens, and this is also true, at least in some cases, of computer images: those which are combinations of standardised picture-elements, as well as those which are produced from mathematical algorithms.

It is for this very reason that we should distinguish digital from mechanical reproduction. In the former case, there is no first token, no real original which may be perceived as such. In that sense, digital reproduction, as opposed to mechanical reproduction, is not distinct from production.

If indexicality is seen as being on the side of Nature, and symbolicity on the side of Culture, iconicity could be thought of as bridging this opposition, since it depends on Human Nature, i.e. on the way we, as human beings, tend to perceive the world. The models of transformation built into graphic computer programs simulate the conditions of human perception. This means that the virtual world created by computer imagery remains an extension of the Lifeworld, of Greimas’ "natural world" and Gibsons "ecological physics".



Bibliography:

Barthes, R. 1964. "Rhétorique de l´image", in Communications, 4, 40-51.

Barthes, R. 1980. La chambre claire. Paris: Seuil & Gallimard 1980.

Dubois, PH. 1983. L’acte photographique. Paris & Bruxelles: Nathan/Labor.

Espe, H. 1983. "Realism and some semiotic functions of photographs", in Borbé, T., ed., Semiotics unfolding. Proceedings of the second congress of the International Association for Semiotic Studies., Vienna 1979. Berlin, New York, & Amsterdam: Mouton, volume III: 1435-1442.

Floch, J:.M. 1986. Les formes de l´empreinte. Périgueux: Pierre Fanlac.

Gardner, .H. 1973. The arts and human development. New York: Wiley & Sons 1973

Gardner, .H. 1980. Artful Scribbles, New York: Basic Books.

GIBSON, J.1978., "The ecological approach to visual perception in pictures, in Leonardo 11:3, 1978, pp. 227-235

GIBSON, J. 1980. "A prefatory essay on the perception of surfaces versus the perception of markings on a surface", in The perception of pictures: Volume I: Alberti’s Window. Hagen, M., (ed.), New York, Academic Press, xi-xvii.

Groupe µ 1992. Traté du signe visuel. Pour une rhétorique de l’image. Paris: Seuil.

GUBERN. R. 1974. Mensajes icónicos en la cultura de masas. Barcelona: Editorial Lumen 1974

GUBERN. R. 1987a. El simio informatizado. Barcelona: Ediciones Penínsulo.

GUBERN. R. 1987b. La mirada opulenta. Exploración de la iconosfera contemporánea. Barcelona: Gustavo Gili.

MALDONADO, T. 1974. Vanguardia e razionalità. Turin: Einaudi Editore 1974.

LINDEKENS, R. 1971. Eléments pour une sémiotique de la photographie. Paris & Bruxelles: Didier/Aimav.

RAMÍREZ, J. A. 1981. Medios de masas e historia del arte. Madrid: Cátedra. Second edition.

ROSCH, E. 1978. "Principles of categorization". In Rosch, E., & Lloyd, B., eds., Cognition and categorization. Lawrence Earlbaum Ass., Hillsdale, 27-48.

Schaeffer, J:M. 1987. L´image précaire, Paris: Seuil.

SONESSON, G. 1989a. Pictorial concepts. Inquiries into the semiotic heritage and its relevance for the analysis of the visual world. Lund: Aris/Lund University Press.

SONESSON, G. 1989b. Semiotics of photography. On tracing the index.. Lund: Institute of Art History.

SONESSON, G. 1994. "Prolegomena to the semiotic analysis of prehistoric visual displays", Semiotica, 100, 3, 267-332.

SONESSON, G. 1996. "Approches to the lifeworld core of pictorial rhetoric", Visio, 1,3, 49-76.

Vanlier, H. 1983. Philosophie de la photographie, Laplume: Les cahiers de la photographie.

above copied from: http://www.arthist.lu.se/kultsem/sonesson/PhotoPost.html