David Rokeby in Conversation with Sabine Breitsameter

Interview - October 2002
»n-cha(n)t« - The Architecture of Language in a Networked Soundspace
David Rokeby in Conversation with Sabine Breitsameter

»n-cha(n)t« Interactive sound installation
by David Rokeby/Canada

The sound installation »n-cha(n)t« is based on language exchange through a computer network, and uses sophisticated language programming as well as voice recognition. - The visitor enters a gallery space, where a number of computers and monitors are hanging from the ceiling. On every monitor a person's ear is visible, which shows whether it is ready to listen or turning away. Below every monitor a microphone is installed. - What you hear is a concert of voices: Every computer you can see is represented by a human voice, speaking grammatically correct, sometimes enigmatic English sentences. - You as an interactor can speak through the mics to each computer, which then integrates your words into its speech, adapts the sense of its sentences according to your input, and after a while you can hear the whole computer community in the gallery space adapting your words, making them part of their speech.


installation shot of n-cha(n)t (2001) - Walter Phillips Gallery at the Banff Centre for the Arts



Step by step, their speech converges to the same overall topic and finally to the same words. It culminates in an unisono spoken chorus of voices. - The installation reveals itself to the patiently listening visitor, who is ready to follow the process of networked semantic convergence. The listener is rewarded by the installation's highly atmospheric sound environment and by observing the system's always exciting and surprising strategies to bring the different computers to consensus.
»n-cha(n)t« was awarded recently the Ars Electronica's »Golden Nica«, the first prize in the category »Interactive Art«.

Sabine Breitsameter:
David Rokeby, congratulations! Your prizewinning installation »n-cha(n)t« is one of the rare interactive audio works, that use language and its implications of semantics and meaning. - Looking at the title of your installation, it seems to be quite confusing. Could you explain it to us?

David Rokeby:
The title of the work is »n-cha(n)t«, which has a reference to the word in English, to enchant, to charm. That's however not such an important word in the title as the sub-sections, like the letter »n« which is a mathematical symbol for any number, if you think of algebra or in computer programming, for »n« equals 1 or 100 or whatever. »Chant« is because I have created this community of computers, that sometimes chant together, speak together in unisono, and then there is this complication where I put the parentheses around the »n« in chant, to imply that »n« is optional, without the »n« it's chat, so: to converse, to discuss - »n-cha(n)t«. There are seven computers in the installation, but it could be »n« computers. Seven is just a number. It could be one hundred, it could be three, that's where the title comes from.

Sabine Breitsameter:
So, can you describe us: What are the computers doing? What is happening sonically in your installation?

David Rokeby:
Well, what they are doing is quite complicated, they are a community, and a community who likes to feel some sense of togetherness. And I should carefully say that in fact computers don't have desires. It is not exactly that they as entities actually have this desire to speak together, but I have programmed them to have the desire, to feel some sense of togetherness with the other six computers. They also love to talk. They have the ability to use English in relatively complicated ways. They can formulate sentences that are in proper English grammar, they like to talk in a together way, they like to chant. They like to speak as they were reciting a poem together or in church. But they are also capable of being interrupted from the outside. Each has a microphone and a voice recognition system that allows them to listen to things you might say to them, and to be stimulated by that.
So, if you go up and talk to one of this chanting computers, it takes in that new information from you and it thinks about it, it tries to make some sense and contemplate the words you provided to it. That turns that particular computer into an individual, separate from the chanting community. So, it starts talking in a way that is relating to what you talked to it about. And it also starts sharing its information with one or two of its neighbors. So, it becomes a dissident. But it is a separate voice in the community. This creates a kind of chaos in the community, the whole community tries to deal with this information.

But then slowly if they are not interrupted or if they feel threatened, if they feel it is too much stimulation, they will find their way back by finding similar words, by finding communalities between them, to a point where they reach consensus again. And they come back to the chant.

Sabine Breitsameter:
As I already mentioned: Your »n-cha(n)t« is one of the very few works in the interactive arts' field that uses language and one of the main possibilities of language, which is: creating coherence. Why did you become interested in language for your artistic work?

David Rokeby:
I had avoided language as a way of working for many years, because I always felt as an artist, if I had to use language to describe my work, that some way I had failed, that my work had to transcend that. But language within the work is something quite different. So, then to approach the problem of making computers capable of dealing with language, I knew this was going to be a challenge, I didn't realize it was going to be so rewarding. Language is, I think, one of the most perverse human creations, and at the same time one of the most wonderful human creations. One of the amazing experiences especially of working with computer and language is that you are forced to take something, that is very natural to you, something we do all the time and we don't think about, and to try to make a computer do this sort of thing is to force oneself to think very deeply about it, and in a different way that a person involved in linguistics would do. Because you really have to go down to the bottom level again. You have to construct from the bottom and you run against a lot of amazing questions.

Sabine Breitsameter:
Constructing a networked process of finding consensus, implies enormous programming work as well as rather gigantic language databases...

David Rokeby:
Yes, they are very large. I used as many resources as I could that existed. For example I found very early on to my great pleasure something called »Word-Net« that is a project that came out of Princeton. It has been 30 years, I think, in development, which is a lexicon of English. It is more than a list of words or just a dictionary, it includes a certain number of links between ideas. So, for example mostly in the nouns a high hierarchical structure, so at the top you have the overall idea of noun, and then you have it split into entities, sensations, I don't remember exactly. It is divided into an ontology. This exists between the words. There are 500.000 words in »Word-Net« so, it is very large.
Then you have also the hierarchies of the natural world. You have mammals and sub-species. So, there are a bunch of existing structures. Not enough for my purposes, but a good start. I developed software to extract the most important words out of this and used the links that existed in »Word-Net« as a start to get me over the hump not to have to type in every word that I could think of or copy a dictionary into the knowledge base.
Then I had to develop a syntactical tagging for each of those entries. The syntactical information in Word-Net was not very useful, so I had to spend a lot of time researching syntactical models and grammatical expansion models. You start with the idea of sentence you expand from that to a noun phrase and a verb phrase
I had to research that, and then many words had special behaviors, I had to tag all that. Then I started to develop more and more connections between ideas and I tried in this case to use as many online resources that I could. Partly, because I thought, well I have a choice, I could make a self-portrait. I could only add connections that meant something to me. And I have done self-portraits with this kind of technology before. And I said, this time I want to see what would happen to imagine beforehand what knowledge might be accumulated by a computer that had access to the Internet. I used a lot of net based services, for example I used something called »The Bank of English» which is a website managed by »Collins«, which is - I think - a big dictionary company in the UK. They have a big pay-for service, where you get access to this huge data base of English, but they also have this free testing environment where you can type in any word and it will report back from a database of 9 Billion words of written, spoken, transcribed English. What are the 40 most common words to follow that word in English? In this later phase for »n-cha(n)t«, I have given the system the ability to read novels. So, if I can find an electronic version on a novel online, I ask it to read the novel and it will parse each sentence to discover the structure of the sentence and find out: this is the subject, this is the object, so say, if the sentence is »The captain ate the sweet tomato«, then from this sentence very simply, it can know, that the captain is capable of eating, and that a tomato is something that can be eaten, and particularly by the captain. It will know from its knowledge base, that the captain is probably a human being, it may know from its knowledge base, that a tomato is a kind of fruit, so eventually it may be able to generalize, that people eat food from this information.
It is not at this generalizing point yet, but it is at a point, where it gathers things like »a ship can sail«, and »it can sail on the ocean«. So,, by reading the novel it gains a sense of the world. And this is quite interesting to me, because in many ways this is similar to how I gained my sense of world as a child. And in a sense, these entities, that make up this piece, are purely literary.

Sabine Breitsameter:
And how does this process of semantic convergence take place in your installation? - Could you give us some general description?

David Rokeby:
For instance words spoken to the system by a visitor of the installation or words through the network from the other computers contribute to its shifting state of mind. For each incoming piece of information it resonates through the knowledge base a bit like if you drop a pebble into a pond and the waves ripple off from that.
If someone says to the system the word »orange« the computer will look at its associations. Because there are associations both to the fruit orange, to the color orange, also to the fact the Irish protestant in Ireland consider themselves »Orange Men« - so these are preliminary direct connections. But then the idea of fruit-orange connects to the idea of food, connects also a little bit less to other fruit: apples and bananas, it might also know that the orange fruit is round, so there is also a little bit that stimulation that leads to the idea of roundness, spheres, and the earth - I am making this up, because I don't know exactly, what the connections will be. But the connections in the knowledge base will mean, that the original stimulation of orange will spread to create a kind of pattern through the whole knowledge base, which in some way represents the stimulus.

So, this is quite a complicated landscape, a very individual state of mind, and the challenge to get the computers to actually chant together, was to find a mode of network communication between them, that would allow them to come to a state of what I would call a communion. Where they are not so much in a dialogue, but really synchronized, really drawn together to a similar state of mind. So, somehow all these individual levels of excitement must come to a point where they are all similar between all the computers. What happens is, at every moment, they tell each other, what is most important to them. So, they stimulate each other with what they are most stimulated by. And in the absence of outside information this creates a kind of re-enforcement between them to the point, where their states of mind get closer and closer and closer. As their states of mind gets closer, what they talk about gets more and more similar, not all the same words, but words that are somehow related. And then finally at some moment, they get to the point where they are so close, that what they say is almost exactly the same.

Sabine Breitsameter:
There are two layers of interactivity in your installation. One is the computers, interacting with each other. The second is the visitor of the space, speaking into one or several microphones and delivering language input to the network. What is your concept of this interactor?

David Rokeby:
This piece is quite different than many interactive pieces. In the 80s I did a lot of very directly, very intense directed mediated interaction. And I still love that, I still think it is wonderful. But I think it is only one narrow slice of what interaction means. Most of our interactions are not so clear. But still in interactivity there is a real impulse to create situations where the user is the god, the user is the whole focus. You go in and you do something, and there must be a response, and it must be very clear. And this is only one very narrow notion of interactivity. We interact all the time in much more ambiguous and complicated ways in the real world. And these are the ones that we would need an understanding of and to explore, I think. The pushing of a button, the clicking of the mouse is something that we understand completely. Emergent community behaviors don't reveal themselves in a second and won't respond to the click of the mouse. So, the challenge, I think, the difficulty is, that you as a visitor are not a god, but just a visitor to a community that has its own agenda.

Sabine Breitsameter:
So, your interactive concept is not based on a conversational or dialogical model. Which interactive principle does your installation follow?

David Rokeby:
Yes a lot of language-based stuff using computer relates to conversation as a model. I wasn't so much interested in dialogue in this work. To come back to the term »communion«: I think we are all somewhat lonely as human beings, at least some of the time, and feel a distance between ourselves and people around us. And there are those very special moments, where we find ourselves feeling suddenly, that the person we are talking to, really understands us in a way that's very moving and hard to describe and hard to repeat even. I sense, that in dialogue, we sometimes use language both to communicate and as a way to keep a sort of distance. To keep things polite, but distant. So, this idea of communion is a very important thing to me. A moment where you feel the miracle of communication more than the act of communication. The moment, when all those computers find their way to consensus and find their way to that place where they are together and not in dialogue is somehow related to that.

Sabine Breitsameter:
Thank you very much, David, for the interview.



Biography
David Rokeby, * 1960, is a sound and video installation artist based in Toronto, Canada. He has been creating interactive installations since 1982. He has focused on interactive pieces that directly engage the human body, or that involve artificial perception systems. His work has been performed / exhibited in shows across Canada, the United States, Europe and Asia. - He was awarded the first Petro-Canada Award for Media Arts in 1988, the Prix Ars Electronica Award of Distinction for Interactive Art (Austria) in 1991 and 1997 (with Paul Garrin), and the first BAFTA (British Academy of Film and Television Arts) award for Interactive Art. He has recently been awarded a 2002 Governor General's award in Visual and Media Arts and the Ars Electronica's »Golden Nica«, the first prize in the category »Interactive Art«.

Further links
http://www3.sympatico.ca/drokeby/
http://prixars.aec.at/2002/

above copied from: http://www.swr.de/swr2/audiohyperspace/engl_version/interview/rokeby.html