"Tonality and Image"
A compilation of thoughts on acoustic and color properties of the image

This essay might be interpreted as a certain criticism of the Tonality/Image ideas and technologies when, in fact, it should be read as an account of what was experienced there by a musician-filmmaker with a dedication to the field of tonal imagery, a musician-filmmaker protective of the notion of visual tonalities (harmonies) as a generator of experiences, thoughts, ideas, beliefs/disbeliefs and processes that stand a good chance of being under-explored as well as stimulating, inspiring, revealing and moving.

'TONALITY and IMAGE' as an ATTEMPTED trans-disciplinary project involving FILM AND TEXT explores the sonic tonality of image and visual color tonality of sound. This project begins with the build up of three image sequences: 1) atonal silver bells - black and white 2) C# minor Red, Orange, Purple 3) F- blue.

As a film and text this project is evolutionary. Meaning its present running time (45 min) and presentation format (digital) is not final. As soon as I am able to overcome certain financial limitations I intend to continue this film on a 35mm print and 60/70min running time. Due to it's evolving nature in theory it need not end until I do.

These THREE SCENES mentioned above comprise a totality called:

"THE RIVER"

BASED on a personal SONIC VISUALIZATION of:

THE TIBETAN BOOK OF THE DEAD

The aspect of MUSIC/NOISE is an ATTEMPTED COMPOSITION based on the color and movement of IMAGES and TENSIONS that I perceive existing between TWO IDEAS:

"THE PHYSICAL IMPOSSIBILITY OF
DEATH IN THE MIND OF SOMEONE LIVING"
(Damian Hirst; 1996)
AND
"BARDO THODOL"
IE. LIBERATION BY HEARING OF WHAT YOU SEE
IE. THE TIBETAN BOOK OF THE DEAD

Inspirations

It is possible to define Tonality of Image as a form of experience, but, of course, evidence for the occurrence of any experience is mostly subjective, phenomenological. Following this exploration I begin to doubt there is a common system at this moment in time for viewing image (shape/movement/color) as a music tonality (sound/music).

However, if image and tonality serve certain functions in our mental life then study of it is beneficial for developing alternative ways of "hearing picture" and "seeing sound" for some if not for all as a kind of mind tuning. Of course many authors/creators have chosen different approaches of connecting Image and Sound. Or these approaches have chosen them.

I was also inspired by several streams of unconventional thinking: Buddhism, phenomenology, holistic, esoteric biology and exquisite instances of eastern art such as Rumi's poetry (The Spell). I wanted to construct a experimental foundation for a tonal understanding of the image as it might pertain to insights by my favorite artists such as those of multi-disciplinary painter M.K Ciurlionis, German writers Tomas Mann and Hermann Hesse, Neitzsche, the Bauhouse movement, Peter Brook, Louis-Bertrand Castel, John Cage, Gurdjieff, Goethe...

The above-mentioned artists/writers challenged me to step out of the linear, overly rational discourse that at times has conditioned my view of art. Almost every one of them one way or another has touched the subject of tonal representation of the image. Some of them agree that all aspects of the image such as color, shapes, movements can be directly translated into sound- tonality, chords, melody but as this essay will demonstrate, that's where the agreement ends. The study of image/tonality perceptions stands out from other creative and philosophical endeavors in that the field's great minds argue cogently with little common ground, and where nothing is safe from questioning. To my mind this situation presents a paradise on the frontiers of knowledge.

Experiencing tonality of image and image of tonality invites important questions about the nature of creativity and the mind of the artist. In the absence of consensus about tonality/image issues among artists and researchers of art I made an attempt to find a formula or framework for synaesthizing music colored imagery via pre-production and production stages of my film "The River".

"Viewing Music" and "Hearing Image"

The following quotes might serve as tuners into the mind-set of my purpose:

Different shapes of tones in a constant, all-permeating light…

(Stockhousen)

The soul of music has corridors and interconnecting corridors in it, there are caves, hiding places, dungeons in it; it is a hidden path from chaos to order; it is a path from disconnected thoughts to fascinating stories…they (corridors) all have different colors, all have different exits…

(Dostoyevsky)

"Tonality of Image" is my own definition of a controversial artistic phenomenon that is indicated by a whole range of expressions: "picturing sound", "visualizing music", "having/seeing a mental sound/picture", and, in some contexts, simply "hearing image". In some cases, all these expressions suggest a purely synesthesia phenomenon.

Synesthesia: Production of a sense impression relating to one sense part to the body by stimulation of another sense or part of the body.

The issue of tonality/image has been raised not only to bring a fresh perspective to the notoriously obscure art of filmmaking, but also to try to illuminate the infinite topic of imagination itself.

Music and Image in my Work

My work as a filmmaker testifies to my interest in the acoustical properties of image and the attempt to incorporate musical strategies into filmmaking. My activities as a director, coupled with my life- long interest in music, placed me in touch with the unconventional merger of image and tone (BTW I can also say: My activities as a musician, coupled with my life- long interest in film….). Much has been made of the film language, musical allusions, and possible musical structures to be found in many films, however, my attempt to conflate the codes of music tonalities and imagery in my short film "The River" was addressed differently. Given my interest in sound-imagery interrelationships, it is not surprising that "The River" draws heavily upon musical sources and exhibits certain musical characteristics. I wanted to create lyrical, romantic and melancholic flow of life imagery as perceived by a dying person.

When I was scripting the image sequences for "The River" I decided to experiment with some properties commonly associated with musical counterpoint: simultaneity, repetition, and autonomy versus interdependence. These contrapuntal properties are typically displayed in tonal music in the following manner:

a) Simultaneity is present in the concurrent interaction of two or more melodic voices in counterpoint.

b) Repetition is present in the way a tonal composition is limited to the twelve tones found in the chromatic scale and must employ these according to the conventional hierarchy of consonance and dissonance.

c) Autonomy versus interdependence is present in the way the concurrent voices in counterpoint are apprehended as two or more individual melodic statements and, at the same time, as a single integrated polyphonic texture.

All of these acoustical and rhythmic signifiers are introduced in the first 5 minutes of the film. The acoustic signifiers in the story show a progression from small black and white images/frames of the bridge, underpinned by consistent references to bell tones and dripping water.

This progression moves from passive hypnotic participation of mandala like sound/frames, intended to open up subconscious pathways in preparation for a more active response to musical performance and the experience of color saturated images.

d) Polyphonic images:

Just as musical counterpoint involves the simultaneous presentation of multiple melodic statements, imagery epiphany involves the simultaneous presentation of multiple perspectives (multi-layered imagery). Furthermore, the recognition that the imagery epiphany is both a composite structure and, at the same time, one integral thing is analogous to the way counterpoint is both a combination of autonomous melodic lines and an integrated, or interdependent, whole.

"The River" is an attempted approach to allegory, in which the imagery vehicle itself is allegorized by highlighting the semantic disparity between the way image points to the outside world and the conversely more personal, self-referential nature of music.

The single predicate upon which this allegorical strategy depends is the acknowledgement that image and music were once an inextricable unity, what the ancient cultures called Holy Art. This predicate is not only culturally determined, but is also phenomenologically justified inasmuch as image and music are both comprised of tonalities (sound/color) organized in time and space.

The Tibetan Book of the Dead:

I consider "The Tibetan Book of the Dead, to be a sonic decoder of visual illusions. It is read aloud to the dying and turns to musical chant, increasingly tonal, with rhythmic and bell accompaniment, following death, when the spirit is thought to be preparing itself for the next stage in its "life" cycle. It is designed to identify all those demons that are sensory products of our earthly existence, so that the soul may understand them for what they are. This ritual is an opportunity for the soul of the dying to awaken and embrace greater wisdom and choose away from the cycle of repetition. It's an opportunity in this way for healing, and comfort because a fundamental underlying assumption of Buddhist philosophy is that repetition is a spiritual sickness which keeps returning the soul to the same cycle of events until it denigrates and corrupts or until it works its way to a higher state of spiritual existence. The age-old practice of using sound for healing is fused with the isolation and focus upon individual and particular colors. This practice understands that the eyes and brain cannot not see something in the same way that the ears and brain cannot not hear and that these two sensual inputs are synesthetic in nature and should be treated together. 'The River' is an attempt to interpret these ideas.

Two Shores of "The River"

The following text concerns the two shores of "The River" intellectualism, and "Other ways of seeing and hearing" such as "gut feeling", and the stream between these two shores ie. creativity.

An unforgettable image is a revelation about the nature of reality. In letting this principle flow through us, like "The River" we are changed and our consciousness is transformed. Our physical sensory receptors i.e. our ears, eyes, etc., can well be thought of as information "transducers" which convert external stimuli i.e. changes in air pressure, light, etc., into nerve impulses read by the mind. Artists, scientists and philosophers have advanced many conceptual models of what the mind does with these impulses to derive knowledge and meaning.

Hirst's depiction of death in his installation, "The Impossibility of Death in the Mind of Someone Living" featuring a 14 foot shark floating in a tank of formaldehyde, provokes in me a meditation on sensory possibilities, as does "Bardo Thadol" only from a material perspective. According to this secular point of view of life it seems there are no ultimate and universal answers to our fears and dreads; there are only tensions that are either sustained or relieved. As a Russian immigrant swept up and along, bounced off an eastern weather front into a westernized whirlwind that has its own cultural, social, ethical, spiritual, that in its own way is contradictory, schizophrenic, privileged and terrified of death, these tensions are alive in me and the work I do. And this is why "The River" reflects contemplation, a journey, as opposed to a realization or a destination. "Bardo Thadol" concerns a visualization in which auditory experience is used to help decode visual illusions. So that hearing is perceived as truth and seeing as lies.

Gavinda said that it's strange that we're frightened to die but we're not frightened by the fact that there was a time when we were not here. We neurotically focus instead on the task of death avoidance and so in a sense live death constantly and painfully. Film too, in its own way, encourages this paradox in that it nearly always involves a presence as opposed to an absence; considers an object in a field of references. Assumes material forces in a material domain that more or less take center stage, even when it talks about spirit. Music on the other hand is experienced as spiritual perhaps because, as Tarkovsky neatly put it, music requires us to believe without seeing. He worked like a musical composer in his films. Instead of moving from location to location because of the primary demands of a typical film narrative e.g. character traveling in a train, moving the story from one location to the next - he foregrounded subordinate elements like space, color, tonality, tempo, rhythm. A friend of mine cinematographer once asked Andrei Tarkovsky if he would ever consider directing a music film. Tarkovsky replied that that's what he thought he'd been doing all his life.

The nature of sound and body:

Every soul is a melody. Every body is a rhythmic knot. Mallarme claims that every soul is a melody, which must be renewed, and every body is a rhythmic knot. In my opinion Mallarme really understood the relationship between sound and image. In one of his texts he writes of the sublime, generative aspect of Wagner's music: "… an audience would have the feeling that, if the orchestra were to cease exercising its control, the mime would immediately become a statue." My obsession with sound is actually an obsession with the mind's eye, the subconscious. I go from internal to external via sound. Therefore I find the sound (voice, music noise) magical - it can exist in both realms. "… in its phenomenal being it seems capable of dispensing with this exteriority within interiority, this interior space in which our experience or image of our own body spreads forth." (Alan Weiss; "Erotic Nostalgia")

In my own creativity sound/music comes always before the image/story. I have to hear it before I see it. In non-dialogue narrative I always have to assemble the sound track first in order to get a clear vision of how to layer/sequence my images on top. Though I might change the sound track at the later stage of post-production. I am particularly curious about the conceptions which might explain to me why sound and music are so tied to my (our) body in my own mentality. To hear is to connect with our "bodily co-ordinates", with that which we can sense, because we cannot perceive sound without both time and space. I understand that somehow our assumptions and experience of sound's emotional power rests in its happening in time and space. Just as we imagine our existence to be in both body (time) and soul (space), so sound makes a linkage between them real for us. Just as our person is both concrete and imagined, so is sound. Conversely, silence is equated with death, with absolute Otherness. However, as we have seen, silence perceived is actually sound. And if sound is a phenomenon that occurs and decays over time and space, ever decreasing in intensity, then the human body could also be described in such a way. We can only cut off hearing through death and so sound and music tie us to our mortality. Because sound decays, it gives us a sense of being there at its vortex, or not being there. Like the human body, a decaying sound - a sound moving toward silence - moves into a distance that is marked and measured by its absence from itself.

TOOLS

A brief overview of tonality/image technologies.

"Our writing instruments contribute to our thoughts"

I am sitting in front of my computer. I press a key and watch as the cursor floats through the waveform. I am using a program called Samplitude. It has a unique feature (Comparosonic) for coloring a wave file according to it's frequency. I hear the sound at the same time as I see its shape and color which is traversed by the cursor. 'Our writing instruments contribute to our thoughts,'1Nietzsche wrote in 1882. It is certainly the case with me. I always wanted to get a 'better look' at sound and study its visual nature. Now I can magnify the image of a sound and study it at the sample level. After years of practice (editing dialogues) it is possible for me to visually identify different sibilance in human speech in the course of using this program called Samplitude. I know the "look" of "s", "p", "k" But my plan was to go deeper then the physical level. I wanted to see the birth of the sound and then to experience how it gives birth to an image.

We do hear sound on the non material level as in the case with music, in our heads, in our feelings. And sound recording has introduced us to a new dimension to "the world of the dead", by manifesting the hallucinatory, paranoid, supernatural or presence of invisible. The effects of amplification, repetition, reversal, projection, broadcast, disassociation, and disembodiment equal those of the most profound theological fantasies.

COLOR/TONALITY SYSTEMS

In search of a

"HOLY ART"

The following sections represent an attempt to condense into a few pages information from different schools of philosophy, music composition and spirituality with regard to color/tonality systems.

Mini-intro: Paraphrasing Peter Brook

"…The artist searches vainly for the sound of a vanished tradition, and critic and audience follow suit. We have lost all sense of ritual and ceremony- whether it be connected with birthdays or funerals - but the words remain with us and old impulses stir in the marrow. We feel we should have rituals, we should do "something" about getting them and we blame the artist for not 'finding' them for us. So the artist sometimes ATTEMPTS to find new rituals (color/sound/image) with only imagination as his source: he imitates the outer form of ceremonies, pagan or Buddhist, unfortunately adding his own trappings - the result is rarely convincing. And after years and years of weaker and waterier imitations we now find ourselves rejecting the very notion of a HOLY ART…"

As much as I enjoy the force and wit of Peter Brook's thought I beg to differ and hold that imagination does not necessarily have to be imitation and can put us closer to original creativity, closer than we think i.e. we often imagine closer than we can think it.

The studies of color-music-image links during the last five thousand years in different parts of the world has resulted in an unprecedented variety of trends which have given us different aesthetic landscapes. One of the distinguishing marks of the modern period has been the tendency of creating a music for the eye (music-film) (not to be confused with film for music, or music for film) comparable with auditory music for the ears that dates to antiquity (Egyptian theater and pagan cults). But so far no universal technical formula has given it the durable, popular basis of the other (pure) art forms. Each major figure in color/image/ tonality had to spend a large part of his career inventing or struggling with clumsy, ultimately obsolete machinery, or following purely intuitive paths.

Image and Tonality are related in so many ways that it is necessary to categorize some of those relationships. First, there is the debate of whether image is itself a tonality. The belief that image possesses, characteristics of tonality leads me to attempt to apply tonality theories to the understanding of image in my film. These include music analyses, theory of harmony, specially invented theories and polyphonic theory. This category could thus be called "image as tonality".

Regardless of whether music actually is an image, our experience of music is evidently so subjective as to cause us not to be satisfied with the idea that perception of it is shared by others. This has led me to the practice of attempting to "translate" music into images and colors or to "describe" musical phenomena in images.

One of the first color/tonality systems was introduced by Pythagoras who had an understanding of a music emanating from celestial spheres as a natural consequence cosmic fusion or integration. According to this view the universe embodies a divine geometrical harmony that is mirrored in all natural phenomena, both in the microcosm and the macrocosm. The harmonies of celestial orbits parallel the seeming regularities of life on earth. The basis of these associations is mathematically precise vibrations that are manifested as light, sound, image, fragrance, and other sensual stimuli. Fusing one's perceptions of these apparently discrete sensory inputs constitutes synesthesia, which Pythagoras considered the greatest philosophical discovery and spiritual achievement as it reconciled the illusory everyday world with the authentic world of universal, enduring, abstract truths. From such beginnings the field of "specialized music/tonality perceptions" has evolved. These are invented, descriptive or explanatory systems, specially designed for the translating of music into color/imagery, and image into music/sound tonality. One of the best known and possibly most controversial music/color systems is a synesthetic concept developed by Alexander Scriabin.

MY PROCESS

How all of the above applies to my film

" It is impossible to imagine our own death; and whenever we attempt to do so we can perceive that we are in fact still present spectators"

Sigmund Freud

I wanted to create imagery inspired by the following: 1) readings in Russian and English of the TOBD 2) conversations with monks I had visited in the southeast region of Siberia who practiced a form of shamanic Buddhism, mainly handed down word of mouth, remarkable in this way for always remaining in the present. 3) Two years in the Russian army through which a main tenet of Buddhism, i.e. the transient nature of all experience, helped sustain me in a place that very close to death. 4) My journey to the west and subsequent challenge to everything I'd ever known a kind of death therein. 5) A realization of the phenomenological possibilities of colored music imagery following harmony rather than rhythm.6) A visualization of the Tibetan Book of the Dead that is both disturbing, beautiful, mysterious, romantic, lyrical and classical…

Any technique, knowledge or theory should be transparent in its application ie. a pathway to visibility. For this reason I abandoned the idea of following through on a folk concepts of color and sound described above. I found them far too mechanical and subsequently untrue in that they stifled my own accidental, spontaneous and subjective experience. Whilst adhering to the color/tonality I'd intended I found it impossible to bring all the elements of my work together into some form of cohesion. I was therefore forced to re-examine and explore the philosophies that influence my behavior, thoughts and feelings.

The last sequence of my film shows a foot falling back to the ground with a thud, a tiny inter cut, five or six frames of a bird in a cage, feet rising above a harmonium, and finally the film crops in synchronicity with the beginning. In contemplating yet another ending I considered what this could mean in relation to the tensions that reside between Hirst's work and The Tibetan Book of the Dead, in a last analysis, wondering how these tensions might be reconciled or laid to rest in me. Some words from 'Thus Spoke Zarathustra' later came to mind. "That time does not run backwards, that is his wrath. Revenge is the will's ill will against time and its 'it was'. 'It was' - that is the name of the will's gnashing of teeth and most secret melancholy. The will cannot will backwards; and that he cannot break time and time's covetousness, that is the will's loneliest melancholy. To redeem those who lived in the past and to recreate all 'it was' into a 'thus I willed it' - that alone I should call redemption. All 'it was' is a fragment, a riddle, a dreadful accident - until the creative will says to it, 'But thus I willed it."

Perhaps the material energy conveyed in an ontological conception such as Hirst's shark, with its suggestion of life as a stubborn quality, in denial of death, finds a counterpoint in Neitzsche's expression of spiritual release through acceptance of the world as illusion and through determination of the will to see, in his description of the "Eternal Return".

It's been observed that film and music as recording technologies allow for a radical plasticity of time and most vividly cater to paradoxical conditions of renewal and creativity, reversal and change. In a sense Hirst's work appeals to the filmmaker and musician in me as he nearly but not quite plastifies in his use of pickling solutions. There is also a terrible beauty there, partly conveyed in the implicitly temporal nature of his work. This body of work is corrupted, suspended as it were, as Damian plays God, slowing time's entropy. The works might also suggest the fetishized hand of a saint. They are in fact truly and radically not spiritual in and of themselves, but they might be goading us to look elsewhere for that. In some sense these spectacles beg rejection. I imagine the bodies beg release from their respective time warps. And yet the viewer is spellbound by their silence, strains to hear something, a breath or a sigh perhaps. Is Hirst not asking us to imagine a day we too would resound with that quality of silence? In his titles, 'The Impossibility of Death in the Mind of Someone Living' and 'Away From the Flock' he suggests that to imagine such a time and space is not possible.

The effort of imagining, a kind of purgatory in itself. Just that. No more.

I return again and again, different every time, to 'The Tibetan Book of the Dead' because of the promise of peace therein, the promise of life, if only I stop imagining and start listening. I return to its equation of sense of sight with illusion and sense of sound with truth. Hirst's body of work may convey the illusions of the body as finite and the sound as silence. But I believe the truth to be the opposite. Decomposition has its own sound. And a sound is no more plastic than a memory or a dream. This is certainly no less true for the object-value lent to sound by its recordability, sound that, given its complex nature, will never twice be the same. I concur with the idea that the recording studio creates a space where the body can experience a kind of death in the sense of a remembered impression or a haunting even, in that the body is separated from its sound, and experiences a sense of one less breath to breathe.

In recording the soundtrack to my film I was initially startled and disturbed by the plethora and speed of fleeting imagery in relation to the sound pallet I had before me. The sounds infused me in a way that the illusions could not. The ambient sounds in different tonalities came to me as true, though unlike the imagery, impersonal. They were intended to serve as a means of highlighting the color of the image by using corresponding tonalities of sound following Scriabin's system borrowed from the Theosophists interpretations of Eastern philosophy, essentially folk ideas, culturally and sensually derived from experience and imagination.

Microperception and microtones

I eventually realized upon viewing the first draft that the process did not add anything but instead tended to labor the message. Borrowing from this culturally specific system and tacking it onto my idiosyncratic sequences of imagery stylized the whole and proved moreover that single tones were inadequate to describing the actual presence of hundreds of microtones in the use of reds, oranges, yellows, golds ie. all those derivatives of red. Eastern music is far more suited to the task because of its integral use of microtones. I chose against the use of eastern sound and music because it is far too culturally specific and other and of course requires an acutely eastern educated ear. I would not have felt comfortable handling microtones I couldn't fully distinguish. So I decided to focus on motion and rhythm in articulating the footfalls of the journey and cut the film therefore according to this cloth.

The field of color/tonality undoubtedly needs further development. We don't know much about the internal structure of our perceptual and imaginative awareness. We also lack detail about what perceptual instruments there are, and how they might work (and details that are given are frequently speculative). Color/tonality is evidently a highly value laden and culturally charged concept and this does not provide universal parameters.

Before I made this film I had a piece of music which set me on a journey of visualizing harmonic structures. Once I had filmed the desired sequences and had edited the first draft I then began to tune the sound to the image. During this process it became apparent that if I simply highlighted the overall sound tonality of the picture then this would flatten the other microtones in terms of the color and the density of the image. As such I found there to be a correlation between the complexity and the compatibility of sound and image which frequently did not favor the application of Scriabin's system. By this I mean that there were simply too many varying tones of red to justify the simplistic application of C tone and C tonalities e.g. C7 C9 C7+5 in every instance. Perhaps this was apparent or at least more marked by the highly wavelike nature of the rhythm of editing as well as the multiple layered nature of the imagery.

In the final analysis I found the artistic merit of keeping to the color/tonality system in the making of the film "The River" hard to justify and sustain. As such the value of the color /tonality approach in my process was to be realized as experimental and problematic as opposed to artistic, spontaneous, and envisioned. However it did heighten my awareness of how things connect among themselves. It made me look at my own creative process, at macro and micro levels of color and sound on a scale previously unexplored, and raised a series of psychological and philosophical problems that led me further along a certain continuum. Finally, the theory of tonal imagery was not directly implemented in my film (although it was extensively used as a meditative and creative process in visualizing many "moods" behind it) and it certainly improved my personal qualities as a filmmaker and art practitioner. This is why the understanding of the nature, origin, and dynamics of color/tonality is a factor of critical importance for the theory and practice of filmmaking. It has deep implications and potential for audio/visual impact, the definition of the effective mechanisms of art and artist's transformation, and the choice of creative strategies.

This project was a difficult work of exploring and integrating perceptions, emerging technology, artistic principles, and natural links for the dynamic arts of sound and image. Also submerging my self into tonal meditations on the image helped me to realize the dangers of different forms of art materialism and the uncritical application of rules. Any technique or system is not the end of the road it is the beginning of it and this is precisely how I ended up using color/sound tonality in the production of my film. It inspired me, it moved me, it made me discover many things but in the end something else took over my creativity and I abandoned it in favor of my more idiosyncratic perceptions. The different systems I explored clearly left their mark in terms of the colors I subconsciously chose to highlight. In fact if these colors inspired Scriabin then they seduced me the more so through the exquisite nature of his music

The field of color/tonality is wide enough to fill a myriad of volumes uniting film, color music, metaphysics, philosophy, mysticism, physiology, psychology and neurology, with about two dozen other areas of knowledge. All of these volumes will deal with the simple fact that ton color/tonality is not just structured sounds, colors and shapes, but a synesthetic phenomenon that acts on human nervous systems through all senses in addition to having its cognitive appeal to the analytical musical/image mind….

More to come Copyright©Dimitri©KinZaZa 2001