Thursday, December 13, 2007

Artificial Art: Computation and Cognition, Remko Scha

Computation and Cognition.

Could the digital computer have an essential influence on the art of the future, and be more than just another new technical tool? It is hardly possible to raise this question without thinking rightaway about the fantastic promises of cognitive science and artificial intelligence. These new branches of science and technology are committed to the development of precise mathematical models of human cognition -- models which are inspired by the digital computer and which, in their turn, can be implemented on such a computer. These promises, taken at face value, suggest the ultimate possibility of "artificial artists": completely automatic computer art, produced by carefully calibrated simulations of human emotion and human intelligence, equipped with extensive databases about for instance everyday life, international politics, art history, and the properties of paint.

Whether this would in principle be possible is a question which often gives rise to passionate philosophical discussions. More interesting, however, is the question about the practical feasibility of this idea: is the state of technology such that it makes sense to start to work on this? To set up, for instance, a research program spanning a few decades in order to develop the various algorithms and data bases that together will constitute the artificial artist? This practical question must be answered negatively. Artificial Intelligence research has failed to produce plausible models of human cognition so far, and there is no reason to expect that it will become much more successful very soon. The complexity of existing computational models can be gauged by looking at the "expert systems" which have recently become popular: relatively simple algorithms with a limited, precisely demarcated task. The amount of knowledge that a human expert can bring to bear on a problem, and the versatility that such an expert displays in applying this knowledge, are gigantic when we compare them to the capacities of current "expert systems".

For the time being, the project of emulating human cognition on digital computers is frought with fundamental problems. The results of A.I. research suggest that there are two kinds of cognitive processes, which differ in essential respects. On the one hand there are perceptual processes, which aim at the classification of sensory data; on the other hand there are the formal symbol manipulations that we employ in arithmetic, board games, legal reasoning, and other formally defined situations. The data structures and algorithms that are required to perform these two kinds of cognitive processes, turn out to be very different. Today's digital computer is primarily equipped for symbol manipulation; if one wants to implement, on this kind of machine, the associative thought processes that constitute perception, one must spell out these processes in an extremely cumbersome way. For human cognition, it is the other way around: humans are capable of both kinds of cognition as well, but in this case the perceptual processes are the more basic ones.

The most important challenge of A.I. research may be to implement, on digital computers, the human cognitive processes for which that kind of hardware is not optimally suitable. So far, this hasn't been very successful. A.I. rules in the areas where symbol manipulation plays a crucial role; but the computer's perceptual capabilities remain extremely meager. We may predict, for instance, with reasonable certainty that when the world champion chess will be a computer program, there will not yet be a program that can reliably distinguish twenty human faces, if lighting and camera angle are allowed to vary a little.

For this reason, emulating the decision processes of the human artist is not a realistic goal -- and that is not because artistic processes are necessarily more difficult to mimic than other mental processes, but because the existing models of all cognitive processes are inadequate. Even if 'creativity' would be a completely mythical notion, and genuine conceptual innovation were a negligible element in artistic practice -- even then, the artificial artist is not in the offing, and neither are the artificial engineer, the artificial pastry-cook or the artificial kindergarten pupil.

This conclusion may not seem very surprising. It resonates with a popular prejudice: that the artistic role of the computer should obviously be limited to that of a technical tool, because the inspiration that underlies all true art happens to be the prerogative of the human artist. Even artists who largely rely on the computer to design their work, and who are inspired by the computer's possibilities and limitations, often pride themselves on their creativity, and view the computer as "only a tool, like the sculptor's chisel". This received opinion, however, underestimates the unique new opportunities that the computer offers.

In certain kinds of cognitive skills, computers outperform people consistently by a large margin: for instance in making extensive calculations, in systematically checking long lists of possibilities, in manipulating intricate algebraic formulas, and in drawing random samples from large sets of items generated by complex definitions. These capabilities are not enough to emulate human thinking, but their implementation embues the computer with an independent, inhuman style of thinking, which is interesting because it gives rise to unforeseeable results. This becomes apparent if we take a closer look at the successes of Artificial Intelligence. These are mostly due to programs that solve limited, formally defined problems, which do not require the perceptual capabilities that are only possessed by higher animals. Examples are programs that play simple games, and expert systems for well-defined design problems, such as the lay-out of integrated electronic circuits. A brief glance at the behavior of such programs yields an interesting insight: they employ a rather different procedure than the human expert who solves the same problem. The computer program finds the best solution by systematically investigating all possibilities. The human expert, who is unable to face the full richness of the whole search space, thinks in terms of a limited repertoire of prototypes and conventions, and explores minimal variations on these if necessary.

The procedural differences also lead to different results. In dealing with a technical problem which allows different solutions that can be compared as being more or less optimal, the computer program will often come up with the better results, because it has looked more systematically at the different possibilities; that is why the electronics industry employs computers to design integrated circuits. Computer-generated solutions differ in interesting ways from human designs: they are often surprising and unconventional. When a computer program plays a demonstrably correct and winning chess end-game, it may very well seem bizarre and incomprehensible to a human observer. In such a case, the program shows that there are many possibilities which the chess game allows, but which the human perspective on chess overlooks.

If we do not care about emulating human behavior, but are interested in unexpected and unconventional results, the computer program may thus be superior. And art production is an obvious case in point. Though many artists have a hang-up about originality, they often produce works which look very much like those of their role-models and colleagues. We therefore propose a more analytic approach to art production: first define a mathematical system which charts all formal and perceptual possibilities as thoroughly as possible, and then implement computer programs which generate random images on the basis of this system. This approach may eventually yield more interesting results then the necessarily more limited method of the traditional individual artist.

Chance Art

To generate artworks by means of algorithms that make random choices from a mathematically defined class of possibilities, is not a new idea. There is a tradition known as "chance art", which aims at the production of art works which are not determined by the artist -- arbitrary artworks, random samples from the space of all possibilities. The attitude behind this resembles the stance of artists who exhibit nothing or who declare the whole world to be an artwork: one avoids choices and positions, one throws the observer back on his inalienable responsibility for his own esthetic experiences, and draws attention to the esthetic interpretability of everything.

So far, the project of making 'arbitrary' paintings has only been realized to a limited extent. In a way, painters like Picabia and Polke come closest, when they mingle intuively selected elements from various domains of life and art. But chance art based on clear, hard mathematical probabilities involves a paradox: more explicitly than any other genre, it displays the conceptual context that frames it. When, for instance, graphic designs are defined as grids of pixels that may be either black or white, drawing a random sample tends to yield a grey-looking plane (Morellet). One gets somewhat more variation by putting a random number of dots with a random size at random locations on the plane (De Vries), or by drawing a random number of lines with a random thickness through random points in random directions (Mandelbrot).

Constructions for making random shapes can also be more complex. For instance, Lévy's construction for random mountainscapes, as described by Mandelbrot: superimposing infinitely many infinitely small 'edges' which run across the plane through random points in random directions. (An 'edge' is a step-function on the plane: on one side of a straight line, the plane is elevated a certain amount.) Cutting a horizontal plane through such a random mountainscape, yields a 'map' with random coastlines.

To conclude this quick enumeration of elementary principles, I want to point to the phenomenon of Lissajous patterns, where a shape is created by multiplying different harmonic oscillations. More complex patterns arise when more complex movements are multiplied (cf. 'Machine Drawings').

So far, chance artists have been content with applying probabilistic operations within such simple systems. That is sufficient if all one wants to do is put forward the very idea of chance. But to really take on the project of the arbitrary painting, we need more; we need a formal language which allows us to assign distinct codes to perceptually different paintings, but also to assign the same code to perceptually equivalent paintings whose details may nevertheless differ considerably (as in the case of the different instantiations of Morellet's random pixels). Such a language is an algebra: it specifies a set of elementary shapes and a set of operations which map shapes onto other, possibly more complex shapes.

Algebras like this have been developed already for characterising specific styles. Harold Cohen, for instance, embued his drawing program AARON with an original style reminiscent of the COBRA painters. Programs which try to mimic existing artists have also been developed, for instance for Miró and Diebenkorn. The 'arbitrary painting' project, however, requires a system with a much richer repertoire of stylistic possibilities, and with the capability to exploit those possibilities in a very flexible way -- so that the degree of stylistic coherence within a painting (or within an exhibition) is itself a parameter whose value can be chosen at random.

From a completely different perspective, the psychology of Gestalt perception has also developed some coding languages which are relevant for our purpose -- for instance, in the work by Leeuwenberg and Buffart in Nijmegen on the mental representation of drawings built up out of straight line segments, and in the work by Lerdahl and Jackendoff in Boston on the perception of music.

It is clear that the machinery needed for the 'arbitrary painting' project would have to be substantially richer and more complex than any of these currently existing systems. New coding languages are needed to begin to conceptualize the range of possible images in a halfway adequate fashion. And drawing random samples from the space defined by such a language, and calculating the detailed execution of the various operations, will not be possible without a computational implementation.

Esthetic Implications

Art is often viewed as a means of communication, employed by high-minded individuals to convey profound ideas. In this view it is self-evident that artworks are designed, made, assembled, or at the very least chosen by artists: the artist is responsible for the artwork; the validity of the artwork is grounded in the artist's sense of life.

The actual production and reception of art in our society is a very different matter, however: works of art constitute material for an unconstrained process of esthetic reflection and interpretation, which is not subject to any rules. In Marcel Duchamp's words: 'the spectator makes the picture'. As a consequence, anything whatsoever can serve as input for the esthetic reflection process; the special status of the artwork is abolished.

Another point to note, is that the esthetic experience is in fact not a particular kind of extraordinary psychological state, but an ingredient of almost all experience --whenever objects, situations and processes which are not elements of a predefined code, are nevertheless employed by our mind as symbols for other (usually vaguer and more complex, perhaps intrinsically more important) things, circumstances and developments. Every object, every situation, every process thus has an infinite potential of meanings.

One of the first unequivocal articulations of this idea about esthetics is provided by Marcel Duchamp's 'readymades' -- simple objects from the real world, which he exhibited as works of art. Duchamp's gesture is sometimes interpreted as a celebration of the sublime autonomous creative power of the artist's Kunstwollen, but I would like to argue for a different interpretation. It is true that Duchamp chose his objects quite judiciously, but one should not be mistaken about the nature of his circumspection. Duchamp has made explicit statements about this, but one can also read it off the objects themselves. They are very ordinary, 'neutral' objects: schoolbook, coat-rack, hat-rack, bicycle-wheel, bottle-rack, snow-shovel, plastic bucket, coffee grinder, typewriter-cover: standard objects, drawing lesson examples.

Duchamp's own words: "It is very difficult to choose an object, because after a few weeks you start to like it or to hate it. One must approach a thing with indifference, as if one has no esthetic emotion. The choice of readymades is always based on visual indifference and, at the same time, on the complete absence of good or bad taste." Like the chairs and tables which always represent 'the object' in philosophical discussions, Duchamp's readymades are 'free variables', schemas that all other objects can substitute for, lacking specific properties which would block unification. [The preponderance of racks and containers might be taken to symbolize exactly this 'non-property'.]

Duchamp's readymades imply the esthetic interpretation of everything. They imply the abolishment of an esthetics in the traditional sense -- of an esthetics which assigns a different value to different things and experiences -- that appreciates paintings by brilliant artists more highly than children's drawings or technical diagrams -- which looks differently at bronze sculptures than at supermarkets or natural phenomena. They imply that esthetic perception is viewed as a cognitive process that is not tied to the art-context -- a process that has its origin and its justification in the observer, and that can be applied to arbitrary material.

The readymade implies that art history is finished: the artwork looses its special status, and art history its incentive. But at the same time it suggests a new beginning: to practise the esthetic interpretation of everything. Important movements in twentieth-century art betray an awareness of this: surrealism, nouveau réalisme, pop art, postmodernism all put forth some version of this idea. But they also take it back rightaway, by defining themselves as a style, with a focused interest in certain aspects of reality, and sometimes with preconceived ideas about how that reality is to be interpreted.

Other artists have thematized the esthetic interpretability of the real world in an explicit, somewhat humorous fashion: a socle with the whole world on it, a signature on a glass pane, paintings with the word "everything". Or, complementarily, they chose the uselessness and superfluousness of art as their topic: paintings with the word "nothing", mirrors, empty canvases, empty frames, empty rooms.

The project of artificial art shows that the awareness of the esthetic interpretability of everything is indeed a new beginning -- the beginning of an activity that is related to art, but at the same time clearly distinguished from traditional art practice. The crucial thing is that we do not know what 'everything' is! In our conventional, habitual thinking we tend to content ourselves with what we encounter already in the existing world, and what looks very much like it. The computer, however, makes it possible now to explore the combinatorics of the space that is defined by our full repertoire of visual elements and operations. Just as photography has been used for 150 years now to find out what we can see in the world we live in, the computer will systematically explore the esthetic possibilities of our cognitive faculties.

Chance and Architecture

Chance art faces the meaninglessness of art and life, and draws constructive conclusions from this state of affairs. It may thus be the only adequate reaction to the current impasse in autonomous visual art. But in the applied arts a similar step is necessary. In architecture, for instance, one must deal with the issue that style is unavoidable but that every style is problematic.

Functionalism is impossible. When the functions are fixed, many choices remain. The architect must confess to a personal esthetic choice or to a social/cultural tradition. In designing buildings for the general public or for unknown future users, such a confession is out of place.

And efficiency is not an objective criterion that defines the optimal solution among different functionally equivalent ones. Every efficiency implies an esthetics. At best, this is a stance concerning the true nature of materials and their proper use, and a normative stylization of daily life. In the worst case (efficiency as an 'economic' criterion: "always choose the cheapest solution"), it boils down to the mean esthetics of money. Architecturecannot avoid to symbolize something. Architecture which is economically efficient in the narrowest sense, symbolizes the Margin of Profit as the highest good.

Postmodernism is a neo-victorianism. The eclectic application of various historical styles, intended as a solution for the dilemma sketched above, results in a narrow, fashionably defined, neo-style. The call for a style-transcending meta-style remains unanswered.

The meta-style that is needed here, can be developed in the context of Artificial Art project described above, which should embrace a mathematical, perceptually oriented analysis of all known styles. After an explicit analysis of these styles, all possible syntheses, interpolations and extrapolations of them can be generated at will by means of computer programs.


Because the production of art is left to the arbitrary impulses and shortsighted ambitions of individual artists, most art is conventional and predictable. Human creativity is often over estimated. Well-designed generative algorithms can yield more surprising results than toiling individual artists.

For human persons it is very difficult to survey all possible combinations of a set of items. For this reason, many structures that are implicitly given in the visual language of our cognitive system have never been witnessed yet. The algorithmic exploration of explicit visual grammars will make these structures visible. I imagine the following division of labour: people define the elements and operations of the visual algebra, and thereby specify an infinite combinatorial space; computer programs draw random samples from this space. Progress by human abstraction and machine systematicity -- just as in science.

The interpretation of the artwork will be decoupled from all anecdotal information about the artist. Image-generation software will be developed through a collective, analytic, un-expressive effort. Art will not revolve around the individual artist any more. A Copernican turn.

copied from:

1 comment:

Dr. Flux said...


Indeterminacy in music is represented by three main tendencies:

Chance music - indeterminacy at the level of composition. During the writing of the piece, the composer employs a chance procedure. Once the work is finished, the score is followed exactly in the same way all traditional music scores are. Representative composer: John Cage.
Aleatory music - indeterminacy at the level of performance. The performer is asked to make decisions which will affect either details or even the form of the piece. Representative composers: Pierre Boulez, Karlheinz Stockhausen, Luciano Berio, etc.
In many instances elements of chance music and aleatory music coexist in the same work. (John Cage).
Stochastic music - indeterminacy at the level of composition but involving strict mathematical tools (stochastic distributions). Representative composer: Iannis Xenakis.
Chance Music

John Cage: Born 1912, died 1992.

Early works
The Wonderful Widow of Eighteen Springs
prepared piano
Schoenberg and Cage
on silence (the use of, the anechoic chamber)
the role of music according to the Indian master:
to calm the spirit, to get it ready for receiving the Divine inspiration
Chance music.
How it is done:
draw a list of elements to be used
determine some rules/limits in using them
to make choices, use a chance procedure such as:
flip coins
throw dice
I Ching (Cage)
computer program modeling I Ching (Cage/Hiller)
use impurities in manuscript paper to determine where the notes are
use star atlas: location of stars determine location of note in the score
Why would someone write music this way ?
to create a music which is formless, without structure
to put some distance between the composer (his/her personal taste, conditionings, etc.) and work
to write music which is not based on causal, deterministic relationships
to liberate the sounds (Cage) and let them be themselves.
to get away from the traditional role of the composer as an obnoxious person who tells everybody what to do.
When writing chance music, a composer leaves many aspects of the composition to chance but still has to make some subjective decisions such as: to determine the length of the piece or to specify (or not) the instrumentation. Cage wants us to listen to individual sounds out of the contex of a melody, texture, etc. and remarks that there would be no need for musicians if we had ears (i.e. if we were aware of our sonic environment). Implied here is also the idea that one does not need a special training to produce music and that music is whatever we decide to call by that name and listen to as such.

Cage's music is an example of music in which the philosophy, the ideas behind the composition, become more important. Instead of a narrative and a message of local importance (politics, emotions, etc.) the artist is concerned with delivering a world view
Zen Buddhism - religion, philosophy, way of life. John Cage became interested in and influence by it in the late 1940s. He attended D.T. Suzuki's lectures at Columbia University. Here are a few "features" expressed in a rather succinct and superficial way:
Big Mind: the existence of a universal potential for consciousness or understanding.
Small Mind: our mind, part of the Big Mind
Ta Tvam Asi, "you are all" or "all is me", a Vedic saying.
Errasing the difference between subject and object: a theme sponsored also by contemporary Physics (Quantum Theory). An object's existence depends on subjective perception (consciousness). Fields of mutual influence.
satori or sudden illumination
rational thinking, words, concepts, logic, causality/determinism get in the way of a true understanding
these are like "crutches" our mind needs in order to understand the world and not actual featurs of the world
spontaneity (see the Japanese tea ceremony, martial arts, etc.)
non-clinging/open mindness: one should be aware and receptive but not attach him/herself to anything
the absurd and the irrational as a (sometimes shocking) alternative.
non hierarchical, non discriminating, non judgemental thinking.
Zen "pedagogy": koans (meaningful anecdotes/short stories) and intellectual/physical shock.
D. T. Suzuki, his life and times. Cage's recillection of his lectures at Columbia University and his speech drowned by the airpalnes landing at LaGuardia.

Listen and interpret John Cage's anecdotes in Indeterminacy

Western roots: DADA

Satie and his Vexations, to be repeated 840 times.
Marcel Duchamp, a pioneer of chance music. His Bride Stripped Bare by Her Bachelors, Even: Erratum Musical
"The principle or practice in the arts and esp. painting that flourished chiefly in France, Switzerland, and Germany, from about 1916 to about 1920 and that were based on deliberate irrationality, anarchy, cynicism, and negation of laws of beauty and social organization" (Webster dictionary).
Critique of the above definition; its flaws
Main dadaists: Hans Arp, Tristan Tzara, Richard Huelsenbeck, Hans Ball, etc.
chance as a tool; irrationality = chance.
The art in "the sixties" as a dadaist revival.
John Cage's importance and influence on the arts
development of early electro-acoustic music
happenings: the Fluxus group
concept art: 4'33": testing the limits
multi-media works: errasing the boundaries between arts
music/art based on ideas and principles uncommon in Western culture
inspired the works of an entire generation and beyond

copied from: