SuperConductor:
The Global Music Interpretation and Performance Program
Manfred Clynes
19181 Mesquite Court
Sonoma, CA 95476
Contents
I. Interpretation: the Problem
The Interpretation Process with SuperConductor
1. Hierarchic Pulse
2. Predictive Amplitude Shaping
3. Organic Vibrato.
We describe how, without playing any notes oneself, one can in a few hours make first rate interpretations and performances of masterworks of the music literature, or of newly composed works. The principles discovered in the early nineteen eighties, chief among them hierarchical pulse and predictive amplitude shaping (Clynes 1983) make it possible to interpret music through global controls. Sectional controls supplement these. The problem how to provide a comprehensive microscore is analyzed and a solution given. A PC computer using a standard soundcard can now calculate and perform the score plus microscore in real time, thanks to brilliant programming by Steve Sweet - in a life-like manner. A musically sensitive person needs to guide the interpretation process: it is not a machine that makes its own fully meaningful music interpretations, nor do we wish to achieve this (it is inherently impossible we believe). Rather it is a program that powerfully enables the musicality of the user: with it music may be interpreted as never before possible, in utmost detail, with little effort, and achieve results that may challenge fine body-skilled performances. The interpretive process described here enlarges the musical understanding.
The music is totally demidified, the limitations of MIDI that until now have crippled the possibility of life-like music are gone. Over 300 pieces of chamber music and complete symphonies are included with their original scores, and have removable interpretations provided. About 50 pieces may be heard streaming from the internet and their scores followed, free, at http://ww.superconductor.com
The subtle anatomy of musical feeling is made available to those who hear it inwardly - to be shaped exquisitely now in great works, without overriding need for muscular athletics, practice and coordination that made it an elitist function hitherto.
I. Interpretation: The Problem
The interpretation of music is a special function that has arisen in Western music through the ability to write down a musical thought, imperfectly. Because this ability is imperfect, when music is performed the written text needs to be interpreted. Traditionally, this is linked with the search for meaning, Great interpreters arise, who are able better than others to reach musical meaning inherent and implied in the composition, corresponding largely to the musical thought of the composer - or, sifting this necessity with a finer sieve, going in detail beyond, and proceeding independently of what could be written. A given written work assumes many forms and meanings, as portrayed by different interpreters. And so the process of music interpretation also acts as a gauge to encompass the depth and breadth of the musical imagination of an age - and now that we have recordings, of successive ages.
The totality of these many interpretations can give us a more-sided view of the work than any one of them. But only one interpretation can be heard at any one time, either inwardly or outwardly.
The imperfections of the written notes are not slight: they are in fact huge. As a glaring example, consider that a typical instrumental score involving strings and/or woodwinds completely lacks written instructions about amplitude contours of the individual notes, and about vibrato. We have elsewhere reported (Clynes , 1986) that typically, 17 bits per note are provided by the interpreter vs. 10 bits by the score, so that roughly two thirds of the total information stream is 'arbitrarily' added on by the interpreter. Part of this additional information, which we call the Microscore, consists of higher resolution emendations to low resolution instructions provided in the score: note loudnesses, durations, crescendi and diminuendi, ritards and accelerandos - other parts are entirely new information, not even hinted at in the score, such as vibrato and note shaping for every individual note. We describe here how the Microscore in many aspects can be linked to the Score - the Microstructure linked to the Structure - and how a computer, now an ordinary PC, is able to provide this link in great detail, and provide organic interpreted performances in real time, guided by a human musical architect of the interpretation.
How is the flow of the supplemented stream, which we may call the interpretive stream, derived from and related to the score? In a meaningful, successful interpretation this stream does not appear arbitrary at all, but rather seems organically part of the concept of the piece. The continuing 'second guessing' involved is judged as successful or as failure (mistaken), depending on how meaningful, integrated and beautiful the whole appears.
Can a computer help to design and provide this
stream?
How can one be an architect of a music performance using a
computer?
We shall try to answer these questions in this paper. To other questions raised,
What artistic satisfaction does that give?
What does it teach us about music?
Who are going to be the architects of music performance?
What will this do to our society?
we can provide only hints at an answer here.
For their fuller treatment the reader is referred to the author's forthcoming book by MIT Press on Music Interpretation for the 21st Century (Clynes, in press). But we may say here that what is being described is opening up doors to a wider understanding and love of classical music through democratization of the process of music interpretation - made possible by the computer. Moreover, it opens a road for everyone to access their own inner feelings as they are activated or not depending on the settings they choose, a demonstration over and over of the universal nature and possibilities of the language of music - now at their fingertips. We hope that this will also make classical music a less endangered species in our society. Be that as it may, computer interpretation of entire works of classical music is a new endeavor, which we have espoused with a sustained effort. And the same process makes it possible for living composers to have their works interpreted and performed the way they want it.
By visiting the site http://www.superconductor.com the reader can actually hear for himself about 50 entire pieces so interpreted, played streaming in real time from the internet, after first downloading a free plug-in in about five minutes.
We can see that the stream of information can be broken down into five groups, each containing a number of variables.
For each note: 1. loudness, 2. duration deviation from the arithmetic printed notation value, 3. amplitude contour, 4. timbre variations within the note, 5. vibrato design. In addition expressive intonation may function at least some of the time, used for particular significant notes.
The act of interpretation requires the creation of the time forms and of subtle parametric control needed to fully realize this information stream, in all its richness and resolution, as the human brain can. It is orders of magnitude too complex to be printed and be followed on a score. Computers however can handle this complexity. But the wealth of detail they would need to supply would mean months or even years of computer work by a human to design each note with the computer according to the artistic meaning which they intend, just for one typical work involving a group of instruments.
Ideally, a good performance represents the musical thought of the performer, as he re-thinks the music written by the composer. If the composer is dead we cannot know to what extent this corresponds to the composer's own thought of his music. It could happen though, that a great interpreter can perform the music even better than imagined by the composer (although this may be rare). Some composers excel at performance details more than others. We consider "better" here as meaning not more accurate, or technically brilliant, but as revealing more meaning, a greater transforming power, a greater penetration of mysteries of the human psyche. The piece to be interpreted becomes a tool for revealing the richness of life, the beautiful - an entrance to a "better world" as the Schubert song "An die Musik" tells us. We are profoundly ignorant of what the word profound means, but we can feel it. In that sense interpreting a piece of music is a treasure hunt. And each treasure found is a portal to an even greater treasure!
Let us consider each category of note variables and how they contribute to interpretation.
What the score specifies, and the need - what one wishes it would, if one had the alacrity of a computer:
1. Loudness - Dynamics
In some music dynamic instructions in the score appear as somewhat as spice added to the music after it has been composed, adding to structural clarity, to interest, or to contrast. Some dynamic effects are linked to a basic musicality: they round the music and make it flow "naturally". Such is the crescendo and diminuendo with a rising and falling phrase, often appropriate, named by Casals the "rainbow" law.
Yet for other music, some dynamic aspects are totally part of the music itself, and such music is not conceivable without them. They are not matters of taste or of afterthought, but totally integral to the music. (This is so often for example in Beethoven's music. Clearly, the opening to the fifth symphony has to be fortissimo, or nothing at all. The musical conception includes, ab initio, the dynamics.)
A third class of dynamic effects are the modulations of loudness from note to note - unnotated microdynamics - that greatly contribute to give a live character to the music. Four consecutive sixteenth notes for example although denoted as belonging to piano will generally require four different degrees of loudness.
Special Accents
Fourthly, there may be special accents sometimes denoted by sf placed by the composer, on certain notes. These are not quantitatively specified. (Note that we are aware of special positive accents but not aware of negative accents - do they exist? ).
How are these these different dynamic requirements are designated and quantified in the score?
(i) Section Dynamics.
Composers use the designation pp, p, f and ff to indicate loudness, most often applied to sections of pieces. Such sections may be a number of bars long, and may apply differently to different voices. This designates four levels of loudness, or two bits. To make it more explicit, composers gradually extended this scale to ppp, pp, p, mp, mf, f, ff, fff (or occasionally even pppp or ffff). This increases the range to three bits resolution! But the needed resolution in reality is far greater, typically 8 bits* (not to be confused with the hi-fi requirement of 16 or 20 bits).
(ii) Crescendos and diminuendos.
Not quantified, other than by adjectives molto or poco, sometimes.
The need is that they should have their own non-linear trajectories. Their effect is different depending on the kind of curve they follow, or the precise beginning and end of the curve, and on the loudness level at the beginning and at the end of the curve. It is often desirable for crescendos to start from a somewhat lower loudness level than the immediately preceding, and for diminuendos to end softer than the immediately following, but if so, this is not quantized or specified. Resolution needed is the same as that of (i).
(iii) Balance
Between different voices and instruments to achieve the desired tonal character, in chords, and in simultaneous voices. Specified only by the letters p and f and their combinations as above, if specified at all.
Need is for seven bit resolution.
(iv) Microdynamics
The loudness relations between individual notes of a small note group, analogous to the relative accents of syllables of speech. Microdynamics apply to single voices as well as groups of voices, and operate at all levels of section loudness.
It is not specified at all: totally unnotated.
What are these loudness relations? The character of the pulse gives us a clue, as we shall see in what follows. The need is for seven bit resolution.
Since the dynamic microstructures of i, ii, iii, iv operate simultaneously, and the pulse and balance need definition even at the lowest loudness, the required resolution to realize all of this is preferably 22 bits.
2. Duration of individual notes.
Notation defines durations as whole number multiples and fractions, whole, half, quarter, eighths, sixteenths, notes and so on, as well as triplets and other whole number subdivisions of the beat. The tempo may be prescribed as a metronomic value. If so, then the duration of each note, in milliseconds, is determined. Played according to these values the music has no life, however. It is still born even though it moves through time. Why is this? In speech we give certain syllables more emphasis both in time and in loudness than others, in maintaining a flowing rhythm of speech, thereby producing nuances of meaning. We do not notate speech in fixed arithmetic time values. Music, which harbors song and dance, combines a repetitive pattern, the beat or pulse, with a story line, the structure of the music. Each of these streams makes demands on the durations of notes, modifying them. The modifications are subtle enough to defy quantitative printed notation.
The temporal modifications are of two kinds: pulse related, and tempo related.
Pulse related duration modulation is allied to microdynamics, and may be called microduration modulation, providing relative stress and meaningful pulse. Tempo related duration modulation is seen in rallentandos, ritards, accelerandos, and sectional tempo changes.
Fermatas are in the nature of demarcation guides to larger structural design, breathing pauses for the pulse. In addition to the notated fermatas, smaller breathing pauses may be required occasionally, relating to the larger structural details of the piece. These are generally not notated nor quantized - in more recent music they are indicated sometimes by a "comma". We call them micropauses.
Microduration modulation is totally unnotated, with the exception of a rare tenuto sign of uncertain duration significance. Rallentandos (and accelerandos) are specified only as to placement in the score but are not quantified. Their curves with time are crucial but totally unspecified, and unnotated. Fermatas are not quantified. A convention that they be considered as a 50 % extension of duration is rarely on the mark.
Microstructure requires all these durations to be appropriately modified to achieve interpretive goals. Needed resolution is one millisecond.
3. Individual Note Loudness contour (amplitude shape of each note.)
For instruments which can shape notes individually, such as string or woodwind instruments or the human voice, meaningful interpretation requires that each note be shaped individually. This is almost totally unspecified in scores. Only a very occasional unquantified hint is given such as fp or even more rarely pf .
The envelope shape however contributes more to vital, meaningful performance than most other variables - next to the pulse, it is the most powerful expressive element. This is a main reason why the violin and the human voice are such cherished instruments. (It is a bane of music made with MIDI that individual note shapes are mostly not produced, as this can only be done with highly extensive labor.)
Required resolution for defining the shape: eight bits and two milliseconds. This applies both to soft and to loud tones.
4. Vibrato.
Vibrato is never specified in the score. It is left up to the performer entirely when and how to use it. An overall uniform vibrato is notably counterproductive and even destroys an otherwise successful interpretation. Yet well placed, well timed vibrato of the right amount adds substantially to musical meaning and conviction. Interpretation requires that it be designed specially for each individual note. It helps affective communication to penetrate the listener's armour. It is a natural function related to voice tremor and more broadly to muscular tremor, and seems to activate receptors in the central nervous system and brain function. In these typically amplitude vibrato and frequency vibrato are combined.
The quantitative requirements are: vibrato amount 4 bits, vibrato frequency 4 bits, vibrato wave shape 4 bits (for definition, but this is not greatly varied), vibrato placement 4 bits, amplitude vibrato 4 bits, vibrato rise and fall times, each 4 bits per note. This represents a good deal of detail. Additionally, there may be a threshold of duration that determines that a note of smaller duration than the threshold receives no vibrato at all. The number of bits required for each of these aspects is not well known - 3 to 5 bits may be a fair estimate.
5. Timbre and intra-note timbre changes
The composer specifies which instrument to use, for what voice. This specifies timbre regions. Changes in timbre however are unnotated and unspecified, except for occasional indications such as sotto voce, which refers to loudness as much as to timbre. Other indications such as pizzicato refer to the manner of sound production with concomitant timbre, but do not try to specify timbre itself. Timbre changes are partly built into the instrument in the way timbre inherently changes with loudness for that instrument. Within the possibilities of each instrument timbre variations play a significant role. Timbre changes are hard to quantify, a condition not made easier by the multidimensionality of timbre, and by its dependence on rate of change for its nature. A steady timbre is an oxymoron: this makes it even a more complex problem to quantify.
It is clear, however - on the obverse side of the coin - that other microstructural variables, especially amplitude shaping and vibrato, balance, and to a degree also legato, multifariously affect the perceived timbre. Perhaps surprisingly, a quite substantial range of intranote timbre variations are effectively provided by amplitude shaping and vibrato. This is understandable since their subaudio transients modulate the harmonic spectrum, causing varying sidebands in the audio range that result in varying timbre being experienced. We find that much of perceived eloquent timbre changing effects within notes of are in fact the secondary result of amplitude contour and of vibrato substructure.
It is difficult to state requirements for intranote changes in timbre needed for interpretation apart from the timbre effects caused by vibrato and note shaping, and balance; the possibilities are also limited in different ways depending upon the choice of instrument made by the composer. But not only does a composer have no way to describe such timbre changes well, neither has a performer nor a computer programmer. Because the very concept depends on transients and rapid adaptation by the nervous system a way needs to be developed whereby this concept is either replaced by a less "fuzzy" one, or better understanding leading to one-one relationships are eventually achieved scientifically - in a manner that clearly separates intranote changes due to deliberately changing timbre from the timbre changes caused by amplitude contour shaping and vibrato fine structure, And to make things even more complex, acoustic properties such as masking, reverberation, and distance from the sound source have a strong influence on experienced timbre.
In the following, intra-note timbre variations will be treated as byproducts of other microstructural variables, ie the result of variations in attack, decay , vibrato legato and reverberation, yet without thereby restricting much in terms of musical meaning and to a considerable degree verisimilitude from what is humanly artistically achieved. (We will not consider the phenomena of Klangfarbenmelodie in this discussion.)
As is evident form the above, a creating a meaningful artistic performance on a computer, designing it note by note, replacing the skill but not the judgment of an artist and using all the variables an artist needs to use with appropriate resolution and coordination, requires an enormous amount of detail to be added to the printed score for its realization. To attempt to piece this together by designing and sculpting each note individually on a computer is clearly far beyond what any individual can do in a reasonable time - or even in an unreasonable time. It is in effect beyond human capacity for an average size classical chamber music work - and that is why it has not really been done. (MIDI performances generally do not provide individual note shapes, individual note vibrato, and is necessarily impoverished. Being derived from piano concepts, MIDI does better with piano pieces.)
Even the very musical thinking which such a detailed construction would require is also difficult to come by. In such a piece, every note is related to every other note. Changing an aspect of one note usually will require changes on other notes, and so on in cascade. The cascade will be different with every attempted change, throwing the previous work into disarray. So convergent changes are difficult to achieve. In the meantime, your concept of the piece may have undergone a small change: again everything has to be reworked.
The most demanding thing, probably, is to listen to all the miriad minute inflections and changes, to listen if each of them is really are what you want, without also losing the overall flow of the music: What then happens to spontaneity? How to get it back?
Faced with such a dilemma it is tempting to say to the reader at this point: "do not despair, now, SuperConductor comes to the rescue!"
As outlined in the above, the interpretive process would be impossible without organizing principles that replace the burden of individual note parameter choice with a choice of a process comprising many notes. This involves the change of emphasis from individual notes to groups of notes and their fine structure- analogous to a change of emphasis from letters to words, sentences and paragraphs, in speech.
Over many years of research, we have discovered organizing principles that act to bring order into the arrays of thousands of notes beyond the information provided by the score. We have studied how this information which we called microstructure relates to the structure given in the score (Clynes 1983, 1985, 1987, 1992,1994, 1995).
Basic to this view of organization of microstructure, is the double stream theory of music. It discerns that there are two simultaneous streams of information entering our central nervous system - through the ears in music that is heard, or in thinking music entirely in our brain. Stream I is the repetitive pulse (or beat). This is the unrolling canvas so to speak onto which the story of the music, Stream II, is written or painted. The canvas - the pulse - represents who it is that is telling the story. The pulse is repetitive, the story develops, changes and contrasts. It has a beginning, a middle and an end. The pulse however goes on throughout, repetitively, and has no middle that coincides with the middle of the story. The central nervous system treats those two streams differently. The pulse will tend to go to the feet, the second stream conveys the unfolding story.
(It is interesting that although a dog is extremely sensitive in many ways emotionally, you cannot tell a story to a dog (it may bark at the wrong place), nor does the pulse go to the dog's feet. It seems not to experience either musical stream like humans do)
The history of music may be regarded as a shifting balance between the two streams, from Gregorian chant to rock music of today - a change of predominance between the two streams. A remarkable balance between the two streams occurred in the classical period.
The pulse may be likened also to the gait of a person. One can tell who that person is, even from behind, by seeing their gait. When they slow down or makes a turn, it is still with their gait. (But the composer's musical pulse manifests itself on more than one time scale of organization, unlike the gait as far as we know). The story stream corresponds to the path that person is taking and what the person sees on the way.
Phrasing is part of the story stream. But a piece may be well phrased and otherwise well performed, but without the pulse it lacks vitality and presence.
The musical microstructure is organized differently - though concurrently - by the two streams. Knowing the manner of organization, a computer can successfully provide a microstructure, guided by a human interpreter, that emulates the human musical thinker and performer. It may even potentially do a better job, as it suffers less from distraction, accidental circumstance, and uncertainty. In essence, spontaneity can be provided by the real presence of the two streams. (cf. Casals' dictum "freedom with order... fantasy with order")
The Interpretation Process with SuperConductor
The interpretive process can be divided into
A. Global interpretive actions
B. Sectional actions such as section dynamics, crescendi, ritards
C. Actions on single notes.
In the interpretive process, the first step is to
adjust the global functions.
The second is to realize the sectional requirements.
The third, to adjust single notes, is hardly ever necessary, not
even for one note in many pieces.
Let us now consider each of the interpretive actions.
We have the service of three main organizing principles:
1. the hierarchic pulse,
2. predictive amplitude shaping
3. organic vibrato.
Further, pitch crescendo is a subtle but simple global adjustable functionality.
The first operates primarily within stream I, the others within stream II.
These "genies" do the job of organizing the microstructure according to the desires of the interpreter, in such a way as to realize his musical intentions.
Each of these functions has a knowledge base derived from research into the nature of microstructure. They represent aspects of musical thought that create the relationships between structure and microstructure. (Evidence and proof of the efficiency of these functions is that they work, not only in a few selected pieces, but in hundreds of major works, attested by a collection of some 300 entire pieces now included with SuperConductor, to which more are being added constantly. Many of these can be heard free on the internet with continuous streaming sound, using the free plug in that can be downloaded in about five minutes from the site http://www.superconductor.com )
1. Hierarchic Pulse. Composer's Pulse
The notion of a composer's pulse was first articulated qualitatively by Gustav Becking (1928) who noticed that when following a composition, "being conducted" with finger "conducting", the music seemed to conduct the finger in preferred, characteristic patterns for each composer. The motoric aspect of the composer's pulse was measured and confirmed by our work in the late sixties, in which we measured the sentographic motor patterns of conducting with finger pressure upon a pressure sensitive sensor (sentograph), and averaging the patterns over 50 beats with the CAT computer. We substituted finger pressure for movement in order to be able to get reliable quantification and measurement.. Subjects for these early studies were musicians of the highest caliber, including Pablo Casals, Rudolf Serkin, Murray Perahia, Yehudi Menuhin, Virgil Thompson and others (Clynes 1969, 1970, 1973). These measurements were consistent and provided evidence that there was an invariant form associated with each composer in the motoric output patterns governed by the music.
It was a big step from there to finding how the music itself in its microstructure represented the composer's pulse. This was achieved in 1982-83, largely through the realization that the pulse embodied a composer-specific combined amplitude and timing warp. Using a computer to generate experimental composer's pulses, with composer's melodies and eight bar phrases, deliberately restricted to sinusoidal - fundamental only - sounds it was determined what the required combined amplitude and duration modulation pattern was that would result in authentic, meaningful expression. Determined initially from playing four or five such themes and melodies, it was found that it could be then successfully generalized to other works of that composer.
Combining amplitude and timing deviations in a single composer-specific matrix was the key to the expressivity and identity that the pulse provides.
An important part of the success is the hierarchical nature of the pulse, a further realization. This second realization was that this combined time and amplitude warp would operate on more than one level of temporal organization. Experience has led us to the concept that it generally operates on three levels of organization, the fastest, intermediate, and slowest level and that they operate simultaneously in conjunction with one another (the three levels are multiplied together). This is not so surprising, if one considers that such a repetitive pattern would exhibit behavior analogous to a frequency response characteristic of dynamic system, that is, the effects would be expected to be most pronounced on one time scale and less pronounced at others, yet still present to a degree. At which time scale shall the effects peak?
It becomes a problem to be solved then: what is the hierarchic organization of the pulse for a particular piece? Experience teaches us that one organization works far better than alternative solutions. Discovering this configuration is a crucial part of the interpretive process. This, one conjectures, is the temporal organization that drove the composer's musical thinking. Interestingly, this organization is often independent of the meter that the composer uses to size a bar, other than its duple or triple nature. (When the meter of a piece changes, a new pulse configuration is required. SuperConductor is able to accommodate an unlimited number of meter changes and pulse configurations).
Let us give examples here from the works of Mozart and Beethoven. Consider allegro movements (considered generically here, they may include allegro vivace, allegro molto, allegro moderato, or just plain vivace, and so on -- ie outer movements). Many of these have sixteenth notes as the fastest notes. It is found, as a significant musicological finding, that the pulse configuration found for these is generally 2 x (4 + 4) x 4, going from the fastest to the slowest level. Typically there are two sixteenth notes, then two groups of 4 eighth notes, at the midlevel, and then four bars, or four groups of two bars, if there are only four eighth notes in a bar. The hierarchic pulse here has 64 elements, all different ( 2 x 8 x 4).
Interestingly, of the two groups of 4 eighths the second group typically is softer in Mozart, louder in Beethoven ( about 12-15 % louder on a linear scale).
Fast movements in which the shortest notes are eighth notes rather than sixteenth, typically have a similar structure, except quarter note elements replace the eighth notes in the midlevel pulse, and the top level maybe have two bars for each element.
Slow movements generally have a configuration of 4 x 4 x 4, where the fastest notes will be thirtysecond notes, mid level eighths, and the top level halfnotes. A second group of eighths notes may be present at the midlevel in some works, in that case the top level would be whole notes ( 4 x (4 + 4) x 4 ).
If the movements are in 3, the configuration generally becomes 2 X ( 3 + 3) X 4, or in slow movements 4 x 3 x 4. Fast 6/8 tend to be 6 x 4 x 4.
In every case, the amount of the pulse modulation that is present on all 3 levels needs to be adjusted for each piece. That is, the basic pulse patterns are suitably scaled for each piece. The scaling is done separately for the amplitude and the time aspects. The adjustment (slider) provides a proportion of that pulse pattern that is applied to that piece (separate adjustments for time and amplitude), using the interpreter's musical judgment . Even small changes in these slider positions produce significant musical results. For example, the joyous quality of a piece is brought out well by a particular setting, and a particular setting only.
Timing deviations of the pulse, as percentages, are largest on the fastest time scales (the timing slider on the highest level pulse( slowest time scale) is usually defaulted to a reduced .25). However, amplitude modulations are not generally smaller at the slower time scales, unlike the timing modulations. No changes in the pulse amplitude proportions appear to be necessary as the loudness level of playing is changed, nor as the music becomes louder or softer during the piece. As a second order phenomenon such corrections possibly may be made in the future, but the lack of them has not been felt.
Amplitude and timing scaling factors on the three levels should be adjusted for each piece. The most critical and significant pulse level is usually the midlevel. The adjustments are valid for the whole piece, as a rule, as long as the pulse configuration remains the same, (If the meter changes, or if the tempo changes quite radically, adjustment will probably be required for the new sections).
The reason for this lies in the different tempi that pieces have, and also to an extent differences in character of the piece. While we have never found a piece of a great composer that did not benefit from that composer's pulse, we have seen that every piece demands a slightly different scaling of that pulse. As mentioned above, even small differences in scaling are significant in affecting the mood, and eloquence of that piece. The ranges within which such adjustments are made are quite limited, however.
Is it possible that in the future these adjustments could be made from knowledge of the score? Not as long as the tempi chosen vary across different interpretations, and not as long as different aspects of the music may be preferred by different interpreters who have different concepts, eg some may prefer a more graceful rendition, some more robust etc. A good thing about this adjustment is that it allows varied interpretations with different meanings to exist while still maintaining the composer's 'presence'. Another is that it allows great refinements in the expressivity of a performance, an optimization for example of the grief or love in a piece - and optimization with respect to naturalness, power of conviction and energy.
The highest level pulse operates typically on the bar level. If a composer changes the organization of bars from groups of 4 to 3 or some other number, the pulse has to be reset to keep in synch. This means that you become very conscious of bar organization. Often a 3 bar or 2 bar phrase or section may be inserted between 4 bar sections, and the pulse needs to be appropriately reset. Often the highest level pulse does not begin with bar 1: the first bar or first small group of bars may me upbeat bars. That means the pulse array has to be started in the right way to be in phase with the structure of the work. In other pieces there occurs a consistent phase shift between the mid- and highest levels.
The musicological significance of hierarchic pulse and its configuration is substantial. Musicologist have long considered strong and week accents and bars, but "weak" and "strong" like p and f lack the necessary resolution and discrimination of quality, so the details in terms of time and amplitude that this opens up is exciting .
The highest level pulse contributes considerably to the "musical logic " experienced. Moreover, since it is composer-specific, it helps us to feel and understand the particular thought flow of that composer - it can in fact be correlated with various aspects of the musical structure, to induce a feeling for the relative emphasis and flow of passion of the composer (which in turn must have influenced the flow of his composition).
The pulse affects all voices of the piece in similar ways. This is appropriate, with few exceptions: long trills may need to have a reduced pulse amplitude modulation. If one considers that the pulse results in a motoric drive, so that it will tend to go to the feet, then it is clear that more than one pulse at a time can not function simultaneously in this way. While polyrhythms can well coexist, only one pulse will occur, ie the microstructure of the polyrhythms will tend to be governed by one overriding pulse. The delight in the polyrhythm derives from subsuming the other rhythms in a basic one, somewhat like the different levels of pulse do continuingly. Human nature allows one to dance only one dance at at time. (In this connection we can note also that the microstructural (and motoric) effect of eighths note triplets in quadruple time in an environment of four sixteenths is quite different from considering them in triple time - reason for this is the difference between the quadruple and triple pulse for that composer. )
The pulse constitutes stream I. However especially in its highest level it interacts with stream II, since the work incorporates the bar structure in its story -line. Cadences for example will coordinate well with the 4 bar structure. Even though the pulse array repeats, the points of repetition themselves are coordinated with key points of stream II. Because of this, the pulse array supports phrasing and musical 'logic'. Together with the global interpretation function to be discussed next, it contributes phrasing as a microstructure byproduct.
Stability of the pulse.
How stable is the pulse across the lifetime of a composer - and across years of research of an individual trying to identify it? Only very small adjustments have been required since first describing the pulse components of a number of composer's pulses in the 1983 publication. For example, it was found that the Beethoven pulse in the last quartets of Beethoven needed to have a slightly larger first component amplitude: about 10 % larger (on a linear scale). No changes in the time values were needed; everything else was unchanged. But then, when we retested the earlier Beethoven pieces with the newly adjusted pulse derived from the last quartets, the early works too were improved! So that the change apparently was not something that happened to Beethoven as he became older and developed his compositional style so incomparably, but something that became clearer to us only gradually: what the late works revealed applied also to the early works.
The Mozart Pulse underwent a small change since 1983 also: the first component duration value was changed from 105 to 107. This seemed to benefit a more loving, less superficially charming view. Also the amplitude slide factor is defaulted to .59 on the old values, resulting in a new default amplitude distribution for the four pulse of 1, .40, .63, .42.
A refinement also took place in the Schubert pulse, a slight reduction of the duration of the second component of the four-pulse.
Pulse elements were found for a number of composers in recent years, including Brahms, Scarlatti, Chopin, Handel, Franck in addition to the earlier values found for Bach, Mozart, Haydn, Beethoven, Mendelssohn and Schumann. SuperConductor today has over 300 works, and more are being added constantly. The validity of the pulse concept is overwhelmingly evident to the listener. And he can also experiment with the effects of changes in the pulse elements.
The pulse provides for an intimate "presence" of the composer. To those that hear this, it is a most valuable aspect of the performance, perhaps in a way the most essential. To preserve the composer's identity is not just doing service to the composer, it is giving access to a truth about the music, that goes straight to the heart. What this means is hard to describe in words but it is a verity that permits love as surely as disguise disables it.
2 Predictive Amplitude Shaping
This global function is orthogonal to the pulse microstructure and relates to melodic structure and Stream II. It applies to any voice where notes are individually shaped in amplitude contour, such as string and wind instruments and of course the human voice. In its basic form it does not apply to keyboard instruments.
Predictive amplitude shaping controls the shaping of notes so that they have individual, musically meaningful shapes related to the melodic line. It does not attempt to copy a way in which a particular artist shapes the notes, but is an algorithm which represents musicality with regard to this function, seemingly applicable to all well known composers. The interpreter adjusts the function to his requirements, and often separately for each instrument, eg violin 1, violin 2, viola, cello in a string quartet.
The musical insights incorporated in the function of Predictive Amplitude Shaping are twofold.
Step 1. A given note shape is stretched or compressed to fit all the notes of that voice; the same shape is applied to all notes. This shape, called the 'basic shape' is chosen by the interpreter, using the two parameters. This already has a more musical result than a constant ADSR.
Step 2. The basic shape is changed note by note through looking ahead what and when the next note is going to be. The shape of the present note is skewed forward or backward from its shape as produced in step 1, depending on the angle of a pitch-time tangent drawn form the beginning of the note to the beginning of the next note. So, if the next note is going to be of higher pitch, the present noteshape is skewed forward. If the next note shall be of lower pitch, it is skewed backwards. Moreover, the skewing is greater the sooner the next note is expected. These functions are calculated by the computer from knowledge of what the melodic structure is, ie it has to know what the next note is going to be and when, in that voice. The equations for this function have been published in Clynes, 1987.
This function was originally derived from the study of single melodies using sinusoidal sound only, in 1982. In 1993 it was applied for the first time to the sound of real instruments, such as violin, cello and flute, and it proved to be a successful for those as it was for the sinusoidal sound. Accomplished in 1993 in joint work with the Hewlett Packard Research Laboratories, using their UNIX workstations, it is now successfully realized by MicroSound International with current PCs and Power Macs.
Steps 1 and 2 are combined in a single functionality by the computer. The degree of skewing applied to that voice can be adjusted by the interpreter, but mostly requires little or no adjustment, across composers, pieces and voices.
To understand predictive amplitude shaping, we need to consider some preliminary functionalities and properties.
Firstly, the method of shaping amplitude used. Contrary to the usual ADSR method of amplitude shaping (Attack, decay, sustain, release) used for shaping in syntethizers, derived from the piano action (with its angular discontinuities) we consider musical tones as rounded shapes. We take the position that in the mind there is no sound shaping that corresponds to the piano mechanism - instead the shapes of musical thought have rounded forms that are virtually infinitely varied, to correspond to the expressive requirements. It turns out that such rounded forms are well represented by a family of beta functions. These need only two parameters to define their shape, in addition to the overall amplitude size, or loudness. By appropriate choice of these two parameters, virtually all the necessary amplitude contour shapes of tones can be produced.
To use predictive amplitude shaping globally, you need only to choose the basic shape by adjusting two sliders that activate the form, and possibly to adjust the degree of skewing by a third slider (mostly not required). The computer then calculates the shapes of all the notes of that voice, resulting in varied shapes for the individual notes. A similar choice is made for the other voices. The basic shapes chosen will typically depend on the tempo. Slow movements usually require more forward bending, less rapidly decaying basic shapes. Interestingly, the ear is more sensitive than the eye to small changes in shape of these forms.
The effect of proper choice of amplitude shaping parameters goes beyond mere musicality. It considerably affects the emotional qualities of the music. It is also composer-sensitive, Beethoven on the whole favors somewhat broader shapes than Mozart for example.
Beta function shaping for other musical purposes.
The shaping function can also be used to provide pizzicato. Sometimes, different sections of a piece will demand a different basic shape for some instruments, providing special effects. The sound of the tympani too and of other percussive instruments gain from being shaped both in their decay and attack.
Another usefulness of this function is for long notes, such as for sustained chordal backgrounds in symphonies for example. Such notes are not melodic notes, and they can be treated differently from those by the long note function which provides for a different shaping function for notes longer than a certain threshold. This modification of the shaping function serves a different purpose; to adjust the thickness of texture, the degree of masking that sustained notes cause in the ongoing musical material. (Obtaining the just decay for sustained notes in orchestral works is a function that an excellent conductor heeds carefully. It is also significant in chamber music.) This function also is useful to systematically provide rapid attacks for longer notes, when musically required.
Similarly not melodic are rapid tremolos (especially of wide intervals), and Alberti type basses, and need to be treated unlike typical melodic notes. This is accomplished by an automatic but adjustable reduction of the predictive shaping factor for rapidly skipping note patterns. Trills in string instruments may benefit from reduction in the predictive factor. Staccato notes, as well as notes before a rest, also do not take part in step 2 of the predictive amplitude shaping process. The last note if a piece likewise omits step 2, since there is no next note to come.
Predictive amplitude shaping cannot be regarded as "more legato ascending and less legato descending" - a dangerous oversimplification. Because the time interval to when the next note is heard determines the shape of the present note, as well as the change in pitch height to the next note, the shaping function is far more varied and complex. Even when the time intervals are identical such as in a scale, the shaping is non-uniform since some steps are whole tones and some are semitones.
The variety that predictive amplitude shaping provides however is not chaotic but musical. It is intimately linked to the musical structure, embodying both time duration and intervals. The hierarchic pulse function incorporates duration and amplitude, and ignores pitch intervals - predictive amplitude shaping incorporates time duration and pitch intervals, and ignores amplitude.
Small tempo changes are not significant in changing the parameters of either of the pulse function or of the predictive amplitude function. They are robust in regard to such changes, as second order effects in relation to them, and are not normally accommodated. With large sustained tempo changes the performance may benefit from corresponding changes in the shaping function parameters, and pulse attenuation sliders. Of special interest is the pulse enlargement feature of ritards, to be described under "sectional functions".
Pitch Crescendo as a global function.
As a sensitive, comparatively straightforward function pitch crescendo is very useful in subtly changing the character of the music, generally the entire piece. Pitch crescendo is the way in which a particular voice changes in loudness as the the pitch increases (apart from other influences on loudness). It is sometimes called the scaling of an instrument. It is used in SuperConductor as an adjustable slope per octave. Occasionally a negative pitch crescendo may be used, in which the loudness increases as the pitch decreases. Although pitch crescendo used may vary with different instruments, it is often quite acceptable to use a uniform pitch crescendo for all voices. This saves effort in setting individual pitch crescendos for each instrument or voice, which can be cumbersome when say 32 voices are used. Pitch crescendo is very helpful to set the "tone" of the music. It varies from composer to composer and also from piece to piece. Songful music like Mozart require higher values of pitch crescendo, of the order of 1.15 per octave. Schubert even higher, perhaps 1.20. Beethoven on the other hand "does not want to wear his heart on his sleeve", the pitch crescendo is rarely above 1.05, and in some pieces like the hushed slow movement of the Violin concerto is even less than 1, as low as .95. It is quite remarkable how a suitable pitch crescendo subtly transforms the otherwise already well interpreted music. It can give the music a natural singing quality, imbue it with a more life. In other music, it can destroy the story line, by trying to make singing music that is say processional or revelatory, which counteracts the effectiveness, rather than enhance it. Like all other interpretive functions, neither too little nor too much is needed.
Something closely related to pitch crescendo function was taught by Casals as his "rainbow" law, letting a phrase get louder as it rises and softer as it falls. Of course, no quantization could be given for that law in his teaching. If one studies and listens to fine singers, this is also readily observable. And like all such laws, exceptions could be found where it did not apply. But such exceptions had reasons, reasons that could be discovered if one regarded music as based on inner gesture, dancing and song.
Only quite rarely does one need to change the pitch crescendo parameters during the performance of a movement. A cadenza by a soloist might be such an occasion.
Phrasing arises surprisingly from the action of global functions: holonomic phrasing.
Together, the global functions of hierarchic pulse function and predictive amplitude shaping, aided by pitch crescendo, produce, virtually as byproduct, phrasing. This is one of the most striking aspects of global interpretation, and in a way, seems quite unexpected, even miraculous. Having adjusted the global parameters for these functions, one finds that individual phrases now appear shaped, developing like a polaroid pictures in front of our ears. The effect on phrasing gives one assurance in the musical quality of these functions, that they represent very real musical thought processes. This holonomic phrasing, as we may call it, is the result of inherent musicality applied to the performance as a whole, with no knowledge residing in these functions of what the specific phrases are. The functions interact with the phrases somewhat like light can illumine a structure making it visible - or audible in this case.
Accordingly, interpretive attention to individual phrases, and even more so, to individual notes, is transformed: it is no longer necessary to design each as a separate process. Instead, careful listening to a chosen phrase or phrases, is a guide to global adjustment, so that when that phrase or section has the right character, the other phrases of the piece will likewise have gained.
Holonomic phrasing is also aided by the addition of a small rest at the end of each phrase designated by the composer. This small rest is taken from the note value, not added to it (typically about 15 - 25% of the note value). It is adjustable globally; Mozart (and J.S. Bach when he indicates it) require a greater rest than Beethoven or Schubert. Nothing is done to indicate the beginning of the phrase. This accounts for a minor, if important part, of the phrasing effect produced by the global functions described above.
The hierarchic pulse function is helpful to phrasing also in that it contrasts successive phrases: the highest level of the pulse brackets repeated or contrasted phrases together as a group: there are lighter and heavier parts (bars or multiples of bars), phrases and answering phrases. It does this differently for different composers. In Mozart, the second phrase is often softer than the first in such a group, while in Beethoven the opposite may often be true. But such groupings result in a continuing "line" in music, a continuing, gripping interest. Music "makes sense" much more with it than without it. The alternation of strong and weak grouping in the midlevel pulse structure provides an intellectual satisfaction, gives the music a larger dimension intellectually. It may add to the impression of balance, proportion and clearly perceived flow and meaning. This alternation and progression on the slowest level is felt as linking integrally with the compositional structure, far more than does the pulse on the faster level. Yet the faster level pulse reveals much about the intimate nature of the composer, that is not revealed in the highest level. The fastest level provides a sense of presence of the composer, the slower a link to the particular work we are hearing, Both levels work simultaneously, hence the notion of a SuperConductor, supervising two conductors who are conducting the separate levels simultaneously, or three conductors for a three level pulse.
In addition the computer employs other "conductors" to shape each voice with Predictive Amplitude Shaping, so that there are a host of conductors being conducted by SuperConductor, not an implausible justification for its name.
A special global function for keyboard instruments
Keyboard instruments, in particular the piano, harpsichord and organ do not permit one to shape the notes like a violin does, so predictive amplitude shaping does not apply to them. However, instead of amplitude shaping per se, we can control a degree of expressiveness with a variable articulation and legato function that also looks at the next note to decide how to play the present note. The damper action is adjusted note by note, providing a "tail length" that is variable and a variable time for the tail to begin operating within the note. Like predictive amplitude shaping, this also helps phrasing.
The settable tail function is also useful in aiding to produce different characteristics of instrumental performance: a Mozart piano sound, or a Brahms piano sonority, Scarlatti or other characteristic keyboard sounds, both with piano and harpsichord. This function combines well with the pulse function.
Although we have had limited experience with the organ so far, it is clear that organ music also benefits from the global functions so far described. In fact it gains in the right kind of expressiveness, as does the harpsichord, in spite of the inability of the traditional instrument to alter loudness note by note. The organ can actually accommodate appropriate predictive amplitude shaping and benefit thereby without losing the character of organ music, somewhat surprisingly.
3 Individual Note Vibrato as a Global Function: Organic Vibrato
The subtleties of vibrato are known to the listener, at least subliminally, even though the saying is apt "if vibrato is clearly heard as such, it is already overdone". The listener cannot know in detail how vibrato is applied by the artist, at most they can see the shaking of the hand of a cello or violin performer. To the greatest part, vibratos, like the subtle emendations of duration of the pulse, or like the details of amplitude shaping, are felt by the listener and are not quantized either in the score, where there is no indication of vibrato at all, nor in teaching. Even for a computer, quantizing of vibrato is a different matter than the simple time durations and amplitudes of the elements of the hierarchic pulse, for example. Many parameters are necessary to specify vibrato appropriately. Yet it is possible to adjust these globally in a satisfying and successful manner. At least nine parameters are so adjustable, and others are built in, and need not be adjusted. Like with predictive amplitude shaping, the result is an individual vibrato for every note, designed according to the melodic structure. Quite often the global default settings of the parameters can be used for most voices (these produce different, structure-related vibrato for every note) and only certain voices need special global adjustment. Global settings produce different, structure-related vibrato for every note.
The vibrato functions combine predictive elements which look to the next note in determining the vibrato characteristics of the present note (dynamic parameters) with static parameters that relate the vibrato to selfreferential properties such as pitch height, loudness and duration. Predictive dynamic functions include: vibrato amplitude, vibrato frequency, vibrato rise time, vibrato fall time, and placement of the vibrato on the note: starting and ending times. Static functions include vibrato frequency related to pitch height, to note duration, to loudness, and vibrato amplitude related to pitch height, to loudness, to note duration.
The effective result is a combination of the static and dynamic vibrato design functions, which the computer can handle without difficulty. For vibrato envelope shaping beta functions are used, as for the predictive amplitude shaping of tones.
Two other functions included need to be mentioned:
1. the deliberate addition of a specific kind of distortion to the otherwise sinusoidal vibrato frequency. Judicious use of this results in a natural coloration of the vibrato, reproducing the effect of the nonlinear movements of the hand in producing vibrato.
2. The combination of amplitude and frequency vibrato. Most natural musical sounds that use vibrato do so as a combination of frequency and amplitude vibrato. Their relative proportions are variable. This functionality is also provided, so that the loudness of a tone is modulated with the same vibrato wavelet as is the frequency. The relative proportions need used not be adjusted during a particular piece, but may vary for different pieces. Phase relations between amplitude and frequency vibrato are invertible, a useful feature especially for romantic music.
Judicious use of all these parameters results in a natural sounding vibrato, different for each note, that can reinforce the meaning of the music and add conviction to it. We call such a vibrato design "organic vibrato".
It will not come as a surprise to most practicing musicians that wrong choices of vibrato can destroy an otherwise fine interpretation. Using vibrato can be beneficial or harmful to the music. As a uniform lacquer, it destroys the music - as organic vibrato it provides additional dimensions of vitality and meaning.
Predictive note shaping and organic vibrato reinforce one another. The kind of vibrato appropriate and required depends to a considerable degree on the composer, as well as on the piece. Slow movements will typically use different vibrato from allegro movements.
Now these distinctions both in quality and quantity can be documented and reproduced at will, leading to better understanding of the significance of vibrato as a multivaried musical linguistic element - this in spite of the total absence of notation in the score. Moreover, organic vibrato gives additional definition to phrasing beyond that already achieved by the other global functions described.
Ornamentation and trills.
Mordents, grace notes and other ornaments are treated as short notes, written out in the score. Trills however can be designed with varying speed. Biphasic speed trajectories allow the speed to be varied according to adjustable curves. Endings and beginnings of trills can be specially designed. Because notes succeed one anther so rapidly in trills, the predictive amplitude shaping factor sometimes may need to be reduced during a sustained trill. Longer trills may also require a decrease in the differences between the amplitude components of the pulse (ie more amplitude attenuation of the midlevel and low level pulse), but only a partial reduction.
Short note function
The relative prominence of trills and other ornaments, and some very fast runs or passages may be adjusted by the short note function. This permits increase or decrease in loudness of all notes shorter than a certain settable threshold. This is helpful if the ornaments and trills are too loud. On the other hand they can be made more prominent and brilliant in music like Scarlatti's if this scintillation is desirable.
B. Sectional Interpretation Functions.
For most music, having adjusted the global interpretation functions, the general character of the interpretation is defined. The next step in the interpretation process is then to add the sectional effects, which often can tend tend to seem like spice added to the dish that is already prepared in the main.
Those local sectional refinements consist of
1. Dynamic functions: crescendi and diminuendi,
terrasse dynamics, and balancing of voices for various sections,
if and as required. Various voices and groups of voices may
partake of these.
To obtain appropriate crescendos the curves of the crescendos are
adjustable with fractional power functions. Crescendos and
diminuendos are usually concave upwards (linear scales are used
in this representation).
2. Tempo changes: ritards and accelerandi, but also small terrasse tempo changes for an entire section, which are important to finer interpretation.
Many or most of these (except for the small terrasse tempo changes) are prescribed in the score, albeit in an unquantified manner. Often at the end of pieces no ritard is prescribed but a concluding ritard is still warranted, usually not longer than a bar or two.
The small terrasse tempo changes typically are not more than one metronome mark and usually less. Even a quarter of a metronome mark tempo change can have a marked effect that underscores the structure. The trios of minuets, for example, may have a slightly different tempo from the minuet (often a touch slower). The second subjects of sonata forms may have a slightly slower tempo. Even a small 1 to 2 % change in tempo is often quite sufficient, and will not be felt as disturbing the tempo, but letting it flow naturally. The changed tempo will persist for an entire section, and the former tempo resumed at a suitable juncture in the music, or eased into gradually. Such a 1 or even 2% accelerando over a number of bars is not generally felt as an accelerando per se, but becomes incorporated in the natural flow and vitality of the music.
These small sectional terrasse tempo changes are refinements that add to the subtlety of the music, help to define unconscious relationships, and give the impression of spontaneous but meaningful musical thought, without the listener knowing why. When one removes them, one immediately senses that something important has been lost. (Note that these sectional changes in tempo are additional to the duration changes of the pulse: the high level pulse will cause a small tempo change from bar to bar, that may exceed the sectional tempo changes, or be of similar magnitude. These however are ongoing and are subsumed into the general feeling of tempo. Just as a conductor may sway from left to right with the music at times, so the high level pulse makes the tempo sway slightly, so slightly that it is not experienced as tempo change, and lasts too short to be felt as sectional tempo change. Lower level changes of timing from element to element are of course much greater and often exceed 10 %, but are not felt as tempo changes because they occur on too small a time scale).
The editions of the Beethoven Piano Sonatas by Arthur Schnabel present a valiant attempt to systematize such suggested terrasse variations with metronome mark indications. However, the metronome marks' separation (5%, as has been the practice since Maeltzel invented it around 1807) is far too gross to indicate to a computer what needs to be done - humans can ignore the steps and play somewhere in between, as they may feel. (It may be recalled in this connection that our earlier studies of Toscanini timings revealed that while he too marked tempo indications in his score according to whole metronome marks, he was in fact far more consistently precise in his performances than the resolution of these marks (Clynes and Walker 1986, 1982). It was Bartok who started to divide these marks into fractional marks for his tempo indications.)
On Time in Music
It may be permitted here to insert a few remarks concerning the experience of time per se, so central to music. The ability of the brain and the central nervous system to sense the passing of time appears to comprise several different modalities, involving different clocks within the brain. We have called these modalities T1, T2, T3, and T4 (Clynes, 1994). T1 refers to sensing time on a large, diurnal or weekly scale, the sense that tells us when to wake up for example; T2 refers to knowing the size in time of a piece of the order of 10 - 50 minutes and its relative sectional proportions, built from phrase lengths of the order of 4- 10 seconds, with knowledge of their beginning, middle and end. The stability of this has been shown to be as high as 1 in 500 in measurements of many performances of entire pieces by quartets, and solo performances, Clynes and Walker 1986; T3 the sense of musical tempo, the beat, of the order of about 0.5 - 1.5 second, a repetitive time pattern with a limen of about 1 part in 200; and T4, the sense of note durations fractional seconds long, for which time is felt not as a beginning, middle and end, but as a single chunk of time, experienced as a single entity yet differentiated in experience according to duration (range about 0.3 to 0.1 seconds,limen about 1 millisecond).
Each of these time functions exhibits a relative independence from the others. A piece may be played in the morning or evening and still be experienced largely as the same piece, its relative proportions intact. It may be played at a somewhat different tempo, and yet largely keep its relative proportions; or, on the other hand, its proportions may change while keeping the same overall duration (see Clynes and Walker 1986). Changes of individual note durations and their proportions on a the T4 time scale do not affect the experience of any other time-functions. It is in this context that we may look at ritards and how they can be specific to a composer.
Composer specific ritards.
The execution of ritards is an art itself. Even among first rate artists we do not often encounter a true master at ritards. A few who come to mind are Landowska, Casals, Schnabel, Toscanini. In such a notable ritard, say at the end or near the end of a piece, the ritard is felt to frame the entire piece, in a way - suddenly the piece has found a frame that links it to the world we are in, suddenly one views the whole work from a perspective outside it. Or, to put it in another way, one feels in this overview as it were, that one is saying goodbye to the composer: "it has been wonderful to visit with you". It therefore seemed appropriate to focus one's attention on the personality, the identity, of the composer even more at such moments, when one is figuratively shaking their hand in farewell. And we found a method to do this, that seemed to work: an enlargement of the time aspects of the pulse, a temporary enlargement just for the duration of the ritard. Such an enlargement means that the pulse components that are longer get still longer and the ones that are shorter get still shorter relative to one another, in the course of a general slowing down of the music (note that the amplitude components are unaffected). The slowing of the music itself is governed by a power function curve that is fractionally adjustable. Such pulse enlargement for ritards was first described in (Clynes 1985, 1987)
The composer specific aspect of the ritard is adjustable and can be omitted from the ritard function. Our experience has been that it adds considerably to the natural flow of the ritard. It is a subtle difference, but such subtle differences make music what it is. So we have a Beethoven ritard, a Mozart ritard a Bach ritard, a Schubert ritard a Mendelssohn ritard and so on. The degree of pulse enlargement is about 20 to 30 % typically (ie 20 -30 % greater deviations from 100 than without the pulse enlargement). The pulse enlargement itself does not change the tempo - it is superimposed on the ritard curve chosen. Because of the differences in pulse configuration for different pieces, the composer-specific ritard is also different. We have by now implemented over 1000 such ritards in the 300 or more classical works contained in SuperConductor.
Although it has been variously suggested that ritards are typically quadratic curves (Todd, 1989) or cubic curves (Epstein, 1995), our experience with hundreds of pieces indicates that the appropriate ritard function varies along a spectrum of power exponent values from .3 to .8, depending on the length of the ritard, the tempo differences to be achieved and the nature of the music (.5 is the square root, and .333 the cube root curve). Rather than the example of a free falling or moving object that is being brought to rest following Newton's Laws, as is the basis for the theory of quadratic ritard behavior, we consider that ritards, like biologic functions, are subject to feedback control systems ( ie closed loop motor control systems rather than open loop) - and control system theory would lead one to consider the ritard curves to be formed of a sum of decaying exponentials (plus nonlinear effects, cf bang bang servo). The question of what constitutes the final tempo is a theoretical problem here: does one consider the beginning of the last note or its end to be the end of the ritard? If the latter, how does one determine the retarded tempo at the end of the last note?
We find empirically in practice that ritards can be well represented by fractional power law curves. This requires one parameter to define the curve (the power exponent) , and the desired beginning and end point. One has a choice of including the last note in the ritard or not, the difference being easy to handle. (It turns out further that upon resumption of the original tempo, in an a tempo resumption, when there are fast notes involved in the a tempo, it is often best to include the first note (or perhaps occasionally two) of the a tempo in the ritard: the transition is felt as smoother, more natural. The tempo is resumed not completely suddenly but with a very slight start-up phase that is not actually noticed as such, yet is favorable for a smooth transition.
An interesting problem is where to begin a ritard. The composer usually indicates this. However because of a lack of resolution even in the conscious mind, the actual ritard may often start a note or two earlier, without it being noticed. When no ritard indication is given by the composer, the choice of where it begins is entirely the interpreter's. How does he do this? It will be interesting to discover what systematic relation to structure this might show. Micropauses may also play a role in the conclusion of a piece, and their topical relation to ritards are another question. Placed earlier than the ritard, they can function as signals that the end may be approaching, or, placed near the end of the ritard, be part of a final gesture.
Accelerandos are handled in similar ways to ritards, but the curve tends to have an inverted curvature from the ritard curve.
Note that pulse effects are never handled by ritard or accelerando. Rubato as in Chopin can be produced largely by pulse deviation exaggerations, along the lines as used in composer-specific ritards, but more pronounced. When these are done in voices separately, it seems the kind of rubato not current today but described by Chopin (and Mozart and C.P.E. Bach) may be obtained, but more experimentation along those lines are needed.
Single note effects are only rarely implemented singly. More than 99 % of the time they are provided by the global and sectional adjustments.
Micropause
Micropauses are useful to indicate a temporary cessation of the pulse, such as at the end of a major section of a piece, or near the end of a work. It serves to make clear the largescale structure of the work. One "takes a breath" (Luftpause). Typically of the order of 140-200 milliseconds. (Note that this corresponds to the duration of the "present moment" as described for a number of sensory processes of the central nervous system (about 180 - 200 milliseconds) Clynes 1977, pps .21-23). Perhaps two or three such pauses might occur in a piece.
Staccato
The main editor contains a function to produce globally adjust staccato durations as an adjustable proportion of the nominal duration of the notes. Because the pulse causes variations of the actual note durations, the result is not a completely uniform staccato, but staccatos having a subtle degree of variation from note to note in relation to the pulse structure. The proportions of the staccato are globally adjusted for the composer and piece, mostly in the range of 38 - 55 % silence (needed resolution 0.5 % of the note duration), Beethoven, for example, as a rule needing longer sounding staccato notes than Mozart. But all staccatos are also individually adjustable in one voice, or as a vertical group, for several voices playing together.
Portamento
The editor allows similarly for portamento to be entered, typically in the range of 80-90% sound duration. Portamento is also globally adjustable.
Phrasing pauses
Phrasing pauses entered in the editor as rests taken from the last note of the phrase are globally adjustable, and are different for different composers. They tend to require higher values (about 25% rest) in Mozart than in Beethoven (about 15- 18 %).
Legato
Legato, as an overlap between two consecutive notes in the same voice, is an individual note function (non global) entered through the single note editor. Every note that has legato applied to it has a horizontal green legato line appended to its note picture on the monitor screen. Legato is adjustable as a percentage of the note duration. Note that it is now as easy to produce legato for a violin, or other instrument as it is for a piano. Normally a violinist cannot play "true legato", a note overlap in time, on the same string).
On a piano or harpsichord, in SuperConductor, the global tail function which adjusts damper action provides globally varied legato and articulation, apart from individual note legato, which may be also applied.
Fermata
Lengthening of any note or chord can be added on individually to all the other microstructure. This is useful for fermatas, and 2 occasionally for other notes. If a note is lengthened in one voice only, then some other note or notes of that voice have to be shortened correspondingly to maintain synchrony of all voices.
Single Note Amplitude Contour Shaping and Amplitude
The amplitude contour of any single note can be adjusted with the same two beta function variables as for global adjustment. Loudness of a note can also be adjusted. Very rarely used.
Pitch Bend
The pitch of any note can be shaped for melodic or harmonic purposes by an adjustable biphasic curve to obtain expressive intonation on that note, or to have a varied pitch sliding effect. These need to be adjusted individually when required. Resolution 1 cent. (Globally and sectionally applied well tempered tuning, as a selectable alternative to the normally applied equal tempered tuning is in preparation.)
Visual displays of the following can be selected, and viewed while playing, in full, solo or in any voice combination:
1. Composer's complete original score
2. Pulse component configurations, three level view. Bar numbers
showing position in the pulse array are given continuously while
playing.
3. Amplitude shape contours of all notes, one voice at a time,
together with the notes and bars of that voice. Changes in shape
are seen on all notes simultaneously with changes in the
parameters.
4. Frequency Vibrato on all individual notes, one voice at a
time. Changing vibrato wavelet shapes and their positions on the
notes can be seen simutaneously with changes in the parameters
5. Amplitude Vibrato on all notes together with their amplitude
contours, changes of amplitude vibrato and its postition on the
notes can be seen on all notes simultaneously with changes in
parameters.
These displays are useful to let one learn how the microstructure designed sounds in detail. They teach one's ear and one's sensibilities subtleties of meaning and microsotructure. Much of what is learned can be applied to one's own instrumental performance.
The complete implemented microscore can be listed in an extended table, along with the notes of the score, in vertical listing. The microscore itself, before being applied to the notes is also available. Microscores can be automatically compared with one another, giving the user information about what has been changed from one performance to another.
How does the computer handle all these functions?
SuperConductor 's task is to integrate all the functions
described above, and to generate music performances for real time
playback and recording. It is currently implemented on Windows
95, Windows NT on Intel platforms and under Macintosh OS on
Macintosh Power PC platforms, and requires 16 Mb of RAM on the
Intel versions and 32 Mb on the Mac.
It utilizes a 16 bit Soundblaster-compatible sound card under Windows, and the standard Sound Manager on the Mac, using only their DA converters to produce 16 bit 44.1 KHz stereo output. Sound is generated using instrument and human voice samples stored on hard disc as the source sound. Special caching algorithms have been developed so that near real time retrieval and accumulation of sampled sounds is possible. The method remarkably allows sample sizes only limited to the size of the hard disc. As each note is added to the stream of the audio output, it is first shifted in pitch according to the required pitch.
Predictive Amplitude Shaping functions are applied to the envelope of the sample, and Vibrato functions are added (thereby also modifying timbre). Panning and voice amplitude is adjusted in each voice. Adjustable millisecond range delays are set for a number of voices, to provide for auditory synchrony eg. cellos are often advanced about 25 milliseconds due to their slower auditory perception.
After all instrument sounds are accumulated in each of the stereo streams, reverberation is applied using adjustable nine parameter adjustable built-in software reverberation algorithms. Finally, the sound is output directly to the audio output channels (D to A converters) of the computer's soundcard or optionally saved as a WAV or AIFF file directly on the hard disc.
Instrument samples are stored on the computer's hard disc in a proprietary format designed to maximize the speed of loading and playback. Normally about 150 Mb of disc space are occupied by the samples. A sample-construction program is made available with the SuperConductor to allow users to build their own instruments from standard WAV or AIFF sound file formats.
As music is played, scanned versions of the composer's original scores (typically Breitkopf and Haertel or Bach Gesellschaft editions) are optionally displayed on the monitor screen. These images are stored in a compressed form on the hard disc and retrieved and displayed by SuperConductor on demand in real time. The images have been reduced in resolution and then enhanced so they can be optimally displayed on the monitor. A red pointer pointing at the bar being played lets the user follow the score, and pages are turned automatically.
Musical data (score and microscore) are stored in proprietary file format on hard disc. The special format allows for additional parameters necessary beyond what MIDI can accommodate. (File sizes are only slightly larger than MIDI files which contain detailed tempo maps, pedal and pitch bend information). Typically .5 mb to 1 mb are needed to store a five minute 12 voice piece, together with its complete interpretation, the microscore - the space depending on the number of notes and on the number of voices. Hundreds and even thousands of pieces can thus readily be stored on hard disk.
MIDI files can be imported readily into SuperConductor. When doing this however, voices in the MIDI version should be clearly separated and designed, since SuperConductor uses two voices for example, for violin double stops, to permit full control. SuperConductor can export to MIDI, but the MIDI file thus created will necessarily sacrifice vibrato and individual note shaping, although it will keep hierarchic pulse.
Music entry in SuperConductor is done by entering notes directly into SuperConductor's simple step-entry editor, or by first entering the notes into a MIDI sequencer or notation program and then importing the MIDI file into SuperConductor. SuperConductor can handle up to 128 voices
Music files together with their interpretations can be transmitted on the internet by e-mail to others and immediately performed by anyone who has SuperConductor. Transmission time is comparable to transmitting MIDI files. Using a modification of SuperConductor, SuperConductorWeb, music of SuperConductor can be adapted to stream from any internet web site, and be listened to by all those who have downloaded the free plug in from the web site, superconductor.com - even while running word processing programs.
In developing SuperConductor and the principles
that make it work, the author has had the help of many persons
and institutions over the years. Here I would like to thank them
all, and in particular the Rockland State Hospital Research
Center where the first measurements of composer's motoric pulse
were accomplished in 1967-8, with the help of Michael Kohn, the
New South Wales Conservatory Research Center where both
hierarchic pulse and predictive amplitude shaping were first
discovered in 1982-3 for the assistance of Brian McMahon, Nigel
Nettheim, Janice Walker, and later William Thompson and Darius
Clynes; Hewlett Packard Research Laboratories in Palo Alto and
Joel Birnbaum, Executive Vice President, where the principles of
SuperConductor where first realized for the Hewlett Packard
workstations with the help of Darius Clynes in 1993; to Barry
Vercoe of MIT for his help in making C Music available for
exploring the possibilities on the workstation, and most of all
Steve Sweet for making it all possible on the PC and Mac, and
developing groundbreaking new computer technology in the process.
Grants from several Australian goverment agencies were also most
helpful in furthering the research in the nineteeneighties.
Becking, G. (1928). Der musikalische Rhythmus als
Erkenntnisquelle (Filser, Augsburg, Germany).
Clynes, M. (1969). "Precision of essentic form in living
communication," in Information processing in the nervous
system, edited by K.N. Leibovic and J.C. Eccles, (Springer, New
York), pp. 177-206.
Clynes, M. (1970). "Towards a view of Man", in
Biomedical Engineering Systems, edited by M. Clynes and J. Milsum
, (McGraw-Hill, New York), pp. 272-358.
Clynes, M. (1973). "Sentics: biocybernetics of emotion
communication", Annals of the New York Academy of Sciences,
220, 3, 55-131.
Clynes, M. (1977). Sentics, the touch of emotion. 310 ps,
(Doubleday Anchor, New York.) Reprinted, Prism Press, 1989.
Clynes, M. (1983). "Expressive microstructure linked to
living qualities" in Publications of the Royal Swedish
Academy of Music, No. 39 edited by J. Sundberg pp.76-181.
Clynes, M. (1985a). "Secrets of life in music" in
Analytica, Studies in the description and analysis of music in
honour of Ingmar Bengtsson. Publication of the Royal Swedish
Academy of Music, No 47 pp. 3-15.
Clynes, M. (1985b). "Music beyond the score",
Communication and Cognit., 19, 2, 169-194.
Clynes, M. (1986). "Generative principles of musical
thought: Integration of microstructure with structure",
Comm. and Cognition, CCAI, Vol. 3, 185-223.
Clynes, M., (1987). "What a musician can
learn about music performance from newly discovered
microstructure principles, P.M. and P.A.M", in Action
and Perception of Music, edited by A. Gabrielsson (Publications
of the Royal Swedish Academy of Music, No. 55, Stockholm) pp.
201-233.
Clynes, M. (1990a). "Some guidelines for the synthesis and
testing of Pulse Microstructure in relation to musical
meaning", Music Perception, 7, 4, 403-422.
Clynes M. (1990b). "Mind-body windows and music",
Musikpaedagogische Forschung, Vol. 11, pp. 19-42, Essen, Germany:
Verlag Die Blaue Eule.
Clynes, M. (1992). "Time-forms, nature's generators and
communicators of emotion", Proc. IEEE Int. Workshop on Robot
and Human Communication, Tokyo Sept. 1992, pp. 18-31.
Clynes, M. (1993). Program of the Annual Meeting of American
Association for the Advancement of Science, (AAAS,) Boston,
February 1993 pp.. 58-59.
Clynes, M. (1994), "Entities and brain organization:
Logogenesis of meaningful time-forms", Origins: Brain and
Self Organization, ed. Karl PribramProceedings of the Second
Appalachian Conference on Behavioral Neurodynamics Erlbaum Press,
N.J. p. 604-632.
Clynes, M. (1994). Comments on "Patterns of expressive
timing in performances of a Beethoven minuet" J. Acoust.
Soc. Am, August 1994, Vol 96, (2) Pt 1
Clynes, M. (1995) "Microstructural musical linguistics:
composer's pulses are liked most by the best musicians"
Cognition Vol 55, pps 269-310
Clynes, M., in preparation, The interpretation of music for the
21st Century . MIT Press.
Clynes M., and Nettheim, N., (1982). "The living quality of
music, neurobiologic patterns of communicating feeling", in
Music, Mind and Brain: the Neuropsychology of Music edited by
M.Clynes, (Plenum, New York) pp. 47-82.
Clynes, M. and J. Walker, (1982). "Neurobiologic functions
of rhythm, time and pulse in music", in Music, Mind And
Brain: the Neuropsychology of Music edited by M. Clynes, (Plenum
New York) pp. 171-216.
Clynes, M. and J. Walker (1986). "Music as Time's
Measure", Music Perception 4,1,85-120.
Clynes, M. and Sweet, S. (1995) "SuperConductor
Interpretation Program for PC", Society for Music Perception
and Cognition, Conference 95, pps 39-40
Epstein, D., (1995), Shaping Time Schirmer, New York 598 ps.
Todd, N., A computational model of rubato Contemporary Music
Review, 3, 69 -88
US Patents Nos. and Patents applied for