Musical Imagination: Sound and Image in Telematic Dialogue

1. Digital world, emptiness, nihilism and simulation

In the essay Vom Subjekt zum Projekt (Flusser 1998a) [From Subject to Project] Flusser develops the hypothesis that all things become fluid in the contemporary world. Computer science has penetrated all departments of existence, has exhausted the dogmas of faith and the theories of science, has transformed everything — man and world — into a network of fields of multiple relationships. Flusser points to a radical shift shaping our evolution: the change from linear to digital thinking. Linear thinking is based on alphanumeric codes — letters, numbers, etc. — which are the fundamentals of science and theories. Digital thinking deconstructs the theories of linear thinking, it transforms lines into points — into circuits, bits, pixels, etc. — and further reduces points to nothing. The digital world has no solid ground, no soil to stand on. We float in a cloud of random particles.

Flusser approaches the “nothing” — the zero dimension of the digital world — in different ways. In the previous essay O Chão que Pisamos (Flusser 1983) [The Soil we Stand On], Flusser invokes a sound metaphor to qualify our existential feeling: “the steps that move us toward the future sound hollow” (1983: 9). Flusser traces parallels between the emptiness of the contemporary world and that of the baroque. Both periods share a blend of rationalism and irrationalism. In the contemporary world, rationalism is articulated through cybernetics and computer science, irrationalism through the mysticism of mass culture, fanatical ideologies, etc. The baroque was a world of theater. Man represented conflicts on the stage — for instance, the loss of faith in the medieval dogmas. In the contemporary world we give up representation, we replace the stage with the farce. We cease to believe in ourselves.

Flusser is not the only philosopher to see the farce as the essence of the contemporary world. Baudrillard (1994), for example, develops the concept of simulation to characterize postmodern society. Simulation, according to him, is the opposite of representation; it is the radical negation of the sign as value (Baudrillard 1994: 6). The image becomes the simulacrum of itself in the absence of any representation and relation to any reality. The technologies of entertainment, information and communication accelerate the process of simulation in society. Simulation immerses human beings in a universe of intense and involving experiences that Baudrillard characterizes as hyper-reality. Since art and entertainment has also become simulation, art has lost its magic, its potential to provide us with new ways to see the world, to mediate access to reality.

In front of the abysm of nothingness, Flusser rejects pessimism. It doesn’t make sense to be a nihilist, he says (Flusser 1998: 17); he believes that man can overcome the crisis of the loss of faith by projecting alternative worlds. We lost faith in God and also faith in objective and subjective things. We have been pushed and thrown onto hollow soil. However, Flusser claims that we can raise ourselves by projecting alternative worlds ruled by their own laws. He proposes the practice of projection to promote freedom and the “telematic dialogue” as its ethical model.

In opposition to Flusser, Baudrillard sees himself as a nihilist. He defines nihilism as the passion that states the fascination for the mode of disappearance [Verschwinden in German]. In postmodern society, disappearance accounts, for example, for the social implosion of the growing masses and the implosion of meaning in the media. Melancholy is a crucial aspect of nihilism. We are all melancholic, says Baudrillard. Only terrorism can reverse the feeling of melancholy. Therefore, anything that creates meaning in the postmodern society can only be deadly.

2. Image, sound, linearity and superficiality

In the essay Die Geste des Fotografierens [The Gesture of Photographing] from the book Gesten. Versuch einer Phänomenologie (Flusser 1994) [Gestures. Attempt at a Phenomenology], Flusser defines photography as “method to fix on a surface of two dimensions subjects that exist in the space-time of four dimensions.” (1994: 100). The invention of photography was a revolution that transformed our thinking. The gesture of photographing is a philosophical gesture. As philosophy generates concepts, photography generates situations. The invention of the technical image — the image produced by technical apparatuses — reintroduced the power of imagination into the world. In the old societies, man communicated by means of traditional images and myths and so developed a power of imagination. The invention of writing weakened the power of imagination and developed conceptual, historical and linear thinking. The invention of technical images — photography, film, video, etc. — introduced a new power of imagination that is taking over the function of the linear codes of letters and numbers. In other words, we’ve started to think through images and are abandoning the texts. The power of the imagination, as suggested by Flusser, unfolds a double meaning: it defines both the capacity to create images and the capacity to think by means of images.

Flusser plays with words; he juggles with the etymology of words and their translation into many languages — German, Portuguese, English and French. As suggested by Guldin (2005), the idea of translation underlies Flusser’s philosophy. In his phenomenological account of gestures, Flusser develops many languages-games and metaphors of codes and technical apparatuses. For example, by comparing the gesture of writing with the gesture of photographing, he constructs metaphors based on the geometry of line and surface. The form of the gesture of writing is the line while the form of the gesture of photographing is the two-dimensional surface. In order to decipher a text, the eyes must follow a succession of signs organized in lines and unfolding a sensation of time. In order to decipher a photograph, the eyes carry through what Flusser calls “scanning”, a movement of traversing a surface that “deciphers a situation” (1998b: 25). Scanning is not a linear but a superficial action. The word “superficial”, which has here a double meaning, is a typical example of Flusser’s language-game. The term signifies both a form of decoding — the way we read a code — and a philosophical attitude — a form of thinking. The technical image technique fixes superficiality as paradigm of the digital world. But Flusser does not undertake a critical analysis of superficiality. Instead of interpreting meaning, he undertakes a phenomenological reduction that excludes cultural and social connotations. His account of the gesture of photographing raises issues such as what is the ontological and epistemological difference between photography and painting? How has the invention of photography affected painting? What is the effect of the invention of photography on science? What is the relationship between photography and the new forms of seeing such as slides, film, video, holography, etc.?

By comparing technical images to technical sounds we can approach the issue of the power of imagination in music. As photography rescued the power of imagination that had been eliminated by writing, the apparatuses of sound recording and sound reproduction introduced new kinds of consciousness of sound and music. The first device to record sound was the gramophone invented by Thomas Edison in 1877. The Portuguese word gravação [engraving] is perhaps the most appropriated to convey a concrete image of how the gramophone physically works. The sounds are memorized as analogical traces on a physical support; they are literally engraved on the surface of a wax cylinder. Therefore, the word gravação (Portuguese) evokes a much more accurate metaphor of the process of translating sound into (physical) media than for instance the words recording (English), enregistrement (French) or Aufnahme (German).

Another important metaphor in the understanding of the technical codes of sound is the signal-to-noise ratio (abbreviated as SNR). This technical term was introduced for comparing the level of a desired signal to the level of the background noise. It became a sort of reference for measuring the quality of transducers such as microphones, loudspeakers, and other electronic devices. As pointed by Kittler (1986: 72), all such devices operate on a noise background, because information travels through physical channels. As technology manipulates the signal-to-noise ratio it influences our consciousness of sound and music. At the end of the 19th Century, we learned to listen to the music and the noise produced by the first sound apparatus — the gramophone. Whoever listens today to the sound of an old gramophone would say that this machine produces an extremely high amount of noise. The magnetic tape and the tape recorder, invented during World War II, initiated the acoustics of the present. Beyond the functions of memorizing and reproducing, the tape recorder allows the manipulation of sounds as autonomous objects, alienated from their environment. For example, we can change the speed, cut the sound in fragments, change the sound spectrum, reorganize the sound fragments in different orders, etc. These manipulations create the “empire of simulation” (Kittler 1986:164).

The technology of digital signal processing (DSP) accelerates the process of sound alienation as it translates sounds in a sequence of numbers. Digital technology eliminates the background noise of analog devices — for example the rumble of the turntable and the scratches of the vinyl record — and increases the signal-to-noise ratio. The CD was the first digital machine for distributing mass music. However, the distribution of music in contemporary society is migrating to such digital networks of communication and information as iTunes, etc. The networks are also music apparatuses in the sense proposed by Flusser. Their specific functions and programs shape and change our musical experience. For example, the CD reproduces sound in a better sound quality (44.1Khz, 16 bits) than the mp3 standard, which compresses the music signal in the interest of saving space. A mp3 sound file encoded at 128 kilobits per second omits about 90% of the data found on a CD. Nevertheless, the mp3 format became popular in mobile digital devices such as the iPod and iPhone, which have limited amounts of memory. In other words: by the music distributed through digital networks it was vitally important to reduce signal-to-noise ratio in order to increase the storage capacity and the speed of communication.

Flusser’s opposition between surface and line help us to understand the main difference between the codes of image and sound. The technical image is a two-dimensional one; the technical sound is recorded and reproduced by linear codes. By performing a score, one accomplishes the symbolic gesture of translating graphic signs into sound. The musical score is a linear code, a sequence of graphs, letters, numbers, etc. that are read one after the other. The violinist performing a part of a symphony is decoding a single line that represents her/his instrument. The conductor reads a score consisting of several superposed lines representing all instruments. The beam of synchronous lines on the conductor’s score represents the musical flow of the symphony. This linear, temporal flow thus has a texture that varies from the sound of a single instrument to the sound of the whole orchestra. The sound texture determines, for example, the perception of melodies, chords, timbres, rhythms, etc. In other words, it constitutes what Wittgenstein considers the “surface” of music (Wittgenstein 1980: 8). From this point of view, sound is only the superficial image of music. The idea of superficiality is here related to sound texture, which constitutes the simplest dimension of music, according to Wittgenstein. The infinite complexity of music, he says, is hidden underneath the surface of the sound. Therefore, sound is what unfolds the power of imagination in music.

The comparison between cinema and music suggests another metaphor of linearity. Flusser considers film, in opposition to photography, as the artistic medium par excellence. By comparing the movie theater with Plato’s cave, he affirms that our culture makes us accept the film as the appearance of the truth. Film imposes the unreeling of the film as historical time. The linearity of film is related to the succession of photographic images on the ribbon. Film is a linear code that puts surfaces into motion. However, Flusser emphasizes that the images of film do not convey a scenic reality, as in traditional images. In opposition to these, film does not represent a phenomenon but a theory, an ideology, or a thesis, which is translated into phenomena by cinema. Therefore, film does not tell stories, but it presents stories. The film “makes history” (1998a: 124). Flusser considers cinema the “art” of our time. The gesture of film is the gesture of the “new man”, who, for Flusser, is not necessarily a pleasant creature.

In the movie theater, according to Flusser (1994: 121), we are immersed in sound and seated in front of the image. We watch the unreeling of the film, which is the material support of the technical images. The film is a linear support that creates the gesture of representing historical time. But the linearity of the film is disrupted by the gesture of cutting and pasting. The scissors cut the film, and the fragments of images are spliced to form sequences. Music invented the gesture of cutting and pasting before cinema. Beethoven’s music developed already sophisticated techniques of sound editing, for instance, in the piano sonata opus 57, the Appassionata. In his book Beethoven Proprietário de um Cérebro [Beethoven Owner of a Brain], Willy Correa de Oliveira (1979), analyzes the Appassionata from the point of view of editing. Beethoven’s composition uses cinematographic techniques such as take, cut, fusion, fade in, fade out, etc. “Beethoven concatenates the thematic fragments as if he operates, scissors in hand, with the sound takes on magnetic tape”, affirms Correa de Oliveira (1979: 103).

In fact, all music can be analyzed from the point of view of editing, which manipulates sounds and sequences of sounds. The sound material of a musical work can be segmented into different levels of temporality. We can consider the whole composition as a single sound or deconstruct it into a sequence of fragments that can be shorter and shorter up to the threshold of our spectral and temporal discrimination, the limit of what we define as a sound unit. By hearing a melody, we hear each single sound and the sequence of sounds that constitutes the melody as a whole. The musical consciousness operates both at the level of the individual sounds and at the level of the sequences of sounds that we define as units. We create the consciousness of each sound and also the consciousness of the melody as a whole. Flusser (1994: 112) affirms that the cinematographic gesture of cutting and pasting creates a history, an event, which is materialized in the film. Something similar happens to music. However, musical composition has the power to create stories for each sound. Each sound that we hear, each sound of a melody, as well as the sequence of sounds that forms the melody as a whole (or other sound units that we define), appears in our consciousness as a sensation, a phenomenon, an event. As film presents events as a succession of images, melody presents events as a succession of sounds. But each sound of a melody has also its story.

3. Sound, music, temporality and spectrum

When hearing a melody, we can focus our attention on different things. For example, we can hear each sound individually, each note of the melody; or we can hear larger temporal units, the sequences of sounds that we call phrases. We can also hear the melody as a whole, as a temporal unit captured by the memory and imagination. At each one of these levels, we create the consciousness of an individual sound that we call present. The sound-present is a temporal object, because each sound is different from another. The sound-present is also an event because sound is a dynamic temporal phenomenon. Sound perception requires that we perceive sounds at different levels of temporalities and that we synchronize these levels as a unit. Simultaneity and synchronicity on sound perception and cognition are fundamentals of polyphony and music.

In a recent paper (Chagas 2010), I proposed a semiotic account of temporality in sound and music. Based on ideas by Husserl’s phenomenology (Husserl 1966) and by Varela’s neurophenomenology (Varela 1999), I defined three levels of temporality for distinguishing sound phenomena: (1) the level of individual sound objects and events that are perceived in the world; (2) the level of acts of consciousness that are intentionally related to the temporal features of the perceived entities; (3) the level of the self-constituting flow of consciousness that makes possible the existence of sound in the world. The three levels of temporality can be related to both Peirce’s categories of firstness, secondness and thirdness and three types of signs: icons, index and symbols (Peirce 1998).

When we listen to music, we distinguish durations and units at all the three levels of temporality: at the level of the sound objects and events, at the level of the sound segments that we intentionally define, and at the level of the absolute melody of the world. The musical form is a self- constituting — and possibly autopoeitic — object that stand out as a distinct whole against the background of other events such as the sound environment. The duration and unit of a musical object — melody, song, symphony, concerto, opera, etc. — depend on intentional acts that define such things as a sound envelope, a crescendo or a diminuendo, an ascending or descending scale, the profile of a melody, its musical structure, etc. Sound music objects and events inhabit the world. At each instant we listen to a sound, and at each instant we listen to a different sound. At each instant we listen to sequences of sounds that constitute the melody of our consciousness.

In the book Das Universum der Technischen Bilder (Flusser 1996) [The Universe of Technical Images] Flusser projects the vision of a telematic society where humans and machines are engaged in a dialogue with the sole goal of producing information. The telematic society is ruled by dialogue, the only form of communication that can generate freedom in heterogeneous, post-human society. The crucial problem here is how information is created. According to Flusser, information emerges as a synthesis of previous information. In a telematic society, man creates information by playing with apparatuses. Human beings are not creators but players. Playing is a process of analyzing (decomposing) and synthesizing (composing) information in the world. Information has to be disintegrated before new information is generated. Information emerges as probability in a random environment made of de-information. The world tends naturally to de-information and man struggles against this process by literally informing the world. Flusser’s entropic conception of information — upon which the utopia of a telematic society is built — unfolds an evolutionary metaphysics deconstructing the concept of information in terms of oppositions such as chaos and order, randomness and pattern, etc. In this sense, it shares some Peirce’s ideas about the universe related to what he calls Cosmogonic Philosophy” (Peirce 1992: 297).

Spectral analysis is an exemplary method for generating information through sound analysis and synthesis, which particularly takes advantage of the possibilities of digital machines and tools. Figure 1 shows a sonogram of an excerpt of 13 seconds from a digital recording of the melody O Victoriosissimi Triumphatore by the medieval composer Hildegard von Bingen (1098-1179) (Bingen 2008). The sonogram analyzes the spectrum, the periodic structure of the sound vibrations translated in terms of frequencies, amplitude and phases. The horizontal dimension represents time and the vertical dimension frequencies: low frequencies at the bottom and high frequencies at the top. The acoustical energy is represented by line textures featuring variations in thickness and color that indicate the amplitude. A thinner line indicates a single frequency and a thicker line a frequency band. As a line becomes darker we hear the corresponding pitches or pitch bands more strongly. In this particular sonogram, for example, we see a strong acoustic energy on the frequency band around approximately 880 Hz and corresponding to the fundamental pitch of the note A5 sung by the soprano. The combination of lines representing the structure of frequencies, amplitude and phases determines the specific spectral characteristics of this excerpt.

The sonogram is a tool for sound analysis and synthesis that can be used in a “playful” way for both the study of sound and music composition. Through sound analysis we can generate information about the sound that can be manipulated for synthesizing a new sound, different from the original source. This type of manipulation has been extensively explored in the composition of electroacoustic music. Our experience with digital tools and apparatuses has transformed our relation to sound and music. Different kinds of consciousness — artistic, cultural, historical, etc. — emerge from this process. The apparatuses of the current digital society are used to hear, to compose, to perform and to distribute music. iTunes, the network of musical distribution, is one of the first successful models of mass communication of the digital society.

In my text A Música de Câmara Telemática: a Metáfora de Flusser e o Universo da Música Eletroacústica (Chagas 2008) [Telematic Chamber Music: Flusser’s Metaphor and the Universe of Electroacoustic Music] I approached some aesthetic questions related to the use of apparatuses in contemporary music. According to Flusser’s conception of information as synthesis of former information, the apparatus functions as a toy for generating new information. In the emblematic essay Ensaio sobre a Fotografia. Para uma filosofia da técnica (Flusser 1998a) [English translation: Towards a Philosophy of Photography (Flusser 2000)], he defines apparatuses as black boxes that simulate human thinking. But he says that in a very poetical way. Apparatuses are “black-boxes that play to thinking” (Flusser 1998b: 48). Flusser means that apparatuses do not play with thinking, but they inform thinking in a playful manner. This comparison between apparatus and toy is one of Flusser’s most original ideas. What makes an apparatus a toy are its virtual possibilities, that is, the functions for which it was not necessarily programmed. Playing with an apparatus is a subversive act that reverses its original meaning.

Flusser’s utopia of a telematic dialogue (Flusser 1996) is the joyful interplay of human beings and machines acting as partners and seeking new information. Music inspired Flusser to develop the idea of telematic dialogue. Music, according to him, is a concrete universe, based on calculation and computation as much as the universe of technical images. Music has anticipated certain tendencies of contemporary society dominated by technical images. Flusser proposes the concept of the power of musical imagination (Flusser 1996: 180) to define a new kind of creativity that accomplishes the synthesis between musical creativity and the power of imagination of technical images. He believes that technology will significantly approach visual and acoustics dimensions and ultimately eliminate the distinction between music and visual art. Therefore, music will become image and image will become music (Flusser 1996:180; Chagas 2008).

References

Baudrillard, Jean. 1994. Simulacra and Simulation. Translated by Sheila Faria Glaser. Ann Arbor: The University of Michigan Press.

Chagas, Paulo C. 2008. A Música de Câmara Telemática: a Metáfora de Flusser e o Universo da Música Eletroacústica. Ghregh 11. Available in http://revista.cisc.org.br/ghrebh/index.php?journal=ghrebh&page=article&op=view&path%5B%5D=300 accessed on January 14, 2013.

_____. 2010. Spectral semiotics: sound as enacted experience. A phenomenological approach of temporality of sound and music. In: L. Navickaite-Martinelli, (ed.) Before and After Music. Proceedings from the 10th International Congress on Music Signification. 117-126. Acta Semiotica Fennica XXXVII. Vilnius-Helsinki: Lithuanian Academy of Music and Theatre, Umweb Publications.

Flusser, Vilém. 1983. Pós-História. Vinte instantâneos e um modo de usar. São Paulo: Duas Cidades.

_____. 1994. Gesten. Versuch einer Phänomenologie. Frankfurt: Fischer.

_____. 1996. Ins Universum der technischen Bilder. (5ª ed.) Göttingen: European Photography.

_____. 1998a. Vom Subjekt zum Projekt. Menschwerdung. Frankfurt: Fischer.

_____. 1998b. Ensaio sobre a Fotografia. Para uma filosofia da técnica. Lisboa: Relógio D’Água Editores.

_____. 2000. Towards a Philosophy of Photography. Translated by Anthony Mathews. London: Reaktion Books.

Guldin, Rainer. 2005. Philosophieren zwischen den Sprachen. Vilém Flussers Werk. München: Wilhelm Fink Verlag.

Husserl, Edmund. 1966. Zur Phänemenologie des inneren Zeitbewusstseins (1893-1917)., R Boehm (ed.). Husserliana X. The Hague: M. Nijhoff.

Kittler. Friedrich. 1986. Grammophon, Film, Tapewriter. Berlin: Brinkmann & Bose.

Peirce, Charles S. 1992. The Architecture of Theories? In: Nathan Houser and Christian Kloesel (eds.). The Essential Peirce. Selected Philosophical Writings. Vol. 1. 285-297. Bloomington: Indiana University Press.

_____. 1998. What Is a Sign? In Houser, Nathan and Christian Kloesel (eds.). The Essential Peirce. Selected Philosophical Writings, Vol. 2. 4-10. Bloomington: Indiana University Press.

Varela, Francisco J. 1999. The Specious Present: A Neurophenomenology of Time Consciousness. In: J. Petitot & F. J. Varela, G. Pachoud, J. M. Roy (eds.) Naturalizing Phenomenology: Issues in Contemporary Phenomenology and Cognitive. 265-314. Stanford, CA: Stanford University Press.

Wittgenstein, Ludwig. 1980. Culture and Value. Translated by Peter Winch. Chicago: The University of Chicago Press.

CD:

Bingen, Hildegard von. 2008. Hildegard von Bingen. Celestial Harmonies. Responsories and Antiphons. Oxford Camerata, Jeremy Summerly. Naxos 8.557983.