Proceedings paper

Dalgarno, G.

Unit for the Study of Musical Skill and Development,

Dept. of Psychology.

University of Keele,

Staffs

ST5 5BY

g.dalgarno@keele.ac.uk

Vibrotactile and vibroacoustic chairs/couches have been used in two areas: for therapeutic use and for helping hearing impaired to perceive music. Pioneers in the therapeutic use have been Olave Skille [1] and Tony Wigram [2]. Our own work was for many years solely in the perception of music area [6], but in the last three years we have moved very much into the therapeutic area.

Systems have been produced commercially; in the UK these include those made by The Sound Beam Project and overseas by the Somatron Corporation in the USA, and a number of systems from Scandinavia.

These have fallen into two classes: (a) those where music is played though headphones and the vibration in the chair or couch is unrelated to the music (e.g. those designed or employed by Skille and Wigram); and (b) where the vibration is driven by the music itself.

Type (b) is of course the only type of interest for the perception of music by hearing impaired people. We believe it is to be preferred for therapeutic use, other than for specific physical treatments designed for example to improve joint angles in people with cerebral palsy. (The latter topic is not covered in this paper.)

Why is it then not more used? We believe this is because of the several technological problems which have to be solved to make this possible in a truly satisfactory way. The solution to these will be described. Incidentally, here is then no conflict between the requirements of designing a chair/couch for therapeutic use and for music perception; only the way of using it is different.

1. A suitable type of sensation

First, and most important, we seek a means of generation vibrations in the body which a gives a sensation which is as analogous as possible to the sensation of hearing music. Thus while it is well known that the finger tips are the part of the body most sensitive to vibration, the sensation of vibration in the finger tips is not very analogous to hearing. Our research has shown that the best part of the body to apply vibration is the back, above the lungs. However it is not a matter of creating a superficial vibration there on the skin; it has to be a deep body sensation if it is to be judged analogous to hearing. Hence we are considering large transducers and powers of 15 watts and above, rather than the small, low powered transducers which are employed to assist in the conveying of speech in a number of commercially available aids for deaf people. For a review of the latter see [3].

Having established that, it is not just a matter of putting a loudspeaker driver (or other transducer) in a box or buried in the upholstery of a chair. Doing that will certainly provide a sensation of the rhythm and a feeling of the bass notes, and indeed that is largely what is felt by using some of the existing commercial systems. But our goal is to go well beyond that and to do so we have to consider all the factors below.

2. To produce the desired sensation over the widest possible range of pitches.

Low notes are no problem. In any case it is undesirable to go below about 30 Hz: to do so is to invite feelings of discomfort even nausea. It is extremely dangerous to use frequencies below about 12 Hz because the body's internal organs can be driven into resonance. Fortunately, most if not all, commercial systems are fail safe in that respect because neither the amplifiers nor the transducer have a significant response at frequencies much below 20 Hz.

It is the high notes which are the problem. Merely embedding loudspeakers in the upholstery or mounting them in a plywood box does not give a good acoustic design, and many such systems respond effectively only up to bass notes.

Before one can discuss the upper limit of pitch response one has to define the conditions. There are several mechanisms in the body though which mechanical vibrations may be transduced, giving nerve impulses which can be received by the brain. Some of these mechanisms operate best at around low frequencies, eg 30 Hz; the one capable of best transducing the higher vibrational frequencies, up to a maximum of about 950 Hz, is through the Picinian corpuscles [4] and [5]. However the energy required to obtain a response increases very rapidly with increasing frequency. Hence if extremely high powers are used at the frequencies in question, then the result is a volume of sound as a by-product which would be considered unacceptable by most people. Indeed almost any design could be pressed into giving a higher frequency response if the resultant volume of sound were not a problem. Further as well as excessive volume of sound, it would be a sound which is distorted and unpleasant to the ear irrespective of its volume.

Hence we will define the response as that obtained using no more than 10 watts of electrical power supplied to the transducer at any frequency in the range claimed. Most of the vibroacoustic chair/couches on the market are then capable of responding up to about the note A3 (in the notation where C4 is middle C).

This is the fundamental reason why many give a sensation of little more than the rhythm and the bass at acceptable levels and quality of sound produced as a by product of the vibration.

A major part of our work has been the design of a suitable acoustic coupling system. Our most recent design enables us to go up an octave higher, to A4.

The details of this design, which is also helpful in obtaining the desirable type of sensation as described in 1., have to remain confidential until patenting is complete.

This means that the tune, rather than just the bass of a small number of original pieces of music can now be perceived. But by using arrangements of music, eg pieces written for say the violin but arranged for the 'Cello or Euphonium the repertoire of pieces can be much enlarged. Further with modest amount of pitch shifting downwards, a reasonable number of original pieces are also brought within range. It should be remembered that large amounts of pitch shifting, or transposing by more than an interval of a fourth, usually gives a result which is musically unacceptable for reasons well understood in musical acoustics. However this of course applies for listening through normal hearing and we deal with hearing through the ears separately as will be described later

 

3. To obtain an even response to different pitches over the useable range.

1. To carry out amplitude compression.

The volume of sound of symphony music, live at a classical concert, might range from 35 dB HL to 115 dB HL. While the ear can cope with such very large range, the tactile sense certainly can not and very considerable compression would be necessary - so that the range of volume variation is within 10 dB and ideally 6 dB.

In "pop" music the range is very much smaller and sometimes it is in already amplitude compressed - reducing further the already small range. Sometimes little or no additional compression is necessary.

There is no technical difficulty in carrying out the compression; a commercial dynamic range compressor such as the Behringer model MDX 1200 or the Alesis model 3630 is suitable for the purpose. If very great compression is required it may be necessary to use two compressors in series with appropriate settings on each.

 

A note on 3. & 4.

An alternative to the use of a 1/3 octave equaliser and a dynamic range compressor (on each separate channel - and there may be up to 4) is to produce a special recording on tape with this processing already done.

 

2. To have an adequate means of conveying pitch.

The following applies only to the application for the perception of music by deaf people - and then only to those with no hearing whatsoever.

Pitch perception was formerly the greatest problem, in the application for completely deafened people, because the sense of pitch, conveyed through vibration in the absence of any cochlear action, is very poor. For western art music at least, one needs to be able to discern, as a minimum, a semitone, whereas the discrimination here is about a fourth or a fifth under realistic music listening conditions. Moreover, there is no sensation of musical intervals - not even a sensation of an octave.

People who can do much better - eg who say they can discern musical intervals and to be able to discriminate semitones or better, have in the authors experience have always had some cochlear action at the frequencies in question. They might insist that they are "stone deaf" but that does not mean that they are!

Accordingly an attempt was made to represent pitch by using a spatial array of transducers. This is fully described in [6] and will be mentioned here only briefly.

Each transducer responded to one musical note, irrespective of its octave. With the frequency of vibration doubling in each octave the octave can then be discerned readily, at least over a 3 octave range.

In our first attempt the array was on the forearm (inspired by the pioneering work of Von Békésy). One forearm was used for the tune and the other for a monophonic (ie single note) accompaniment. Our second attempt was a truly extensive digital system - using the digits - the fingers! - as well as the digital electronics used in the first system.

Although both the systems worked technically in the way we had designed them, neither was practical success for reasons explained in the paper.

Hence we would not been able to produce a satisfactory system for those people who had no hearing whatsoever, had it not been that multi-channel cochlear implants came along at just this time when it appeared we were unable to provide a satisfactory solution to this pitch perception problem. These give enough hearing sensation for a large proportion of users - results varying very much for different individuals for reasons which no one seemed to understand, at least at the time. For some the results were so good that music could be enjoyed with the cochlear implant alone.

 

3. Hearing and feeling music simultaneously.

It is highly desirable to present music to the ears and to the body separately. The reason for this is that if a compromise is attempted by pitch shifting or transposing the music downwards, it will generally be too low for clarity of hearing yet remaining too high for clear and comprehensive vibrational sensations (particularly with other vibroacoustic chairs/couches). Moreover the envelopes of amplitude will not be optimal for each; and the harmonic content of the signal driving the vibration systems will generally not be low, so giving strong fundamentals.

Only very small and low powered loudspeakers are required for the ears, or if desired, headphones may be chosen. The choice is down to the individual. Some do not like the pressure or feeling of headphones on the head; some like them because they help to exclude the sound of the vibration in the rest of the chair/couch.

Which ever of these is used, their function is important: to convey the high pitched information. Remember that since even with our developments the highest note sensed by vibration alone is A4 (concert A) or at most few semitones higher..

The system we employ is different according to whether it is MIDI based music or ordinary audio.

With MIDI we produce a copy of the notes of the music an octave down (occasionally, when appropriate, 2 octaves down) and usually with different instrument sounds - chosen to have a suitable amplitude envelope and a strong fundamental pitch, giving optimum vibratory perception as well as optimal hearing. The system for this is described in [6].

With ordinary audio we cannot do this nearly so well, but by signal processing of the signals to the head speakers and for the vibrational systems separately, we do the best we can in the above direction

  

4. Simplification but maintaining the essence of the music - reduction.
   
   When perception via vibration rather than a pleasing accompanying sensation to hearing is desired, just 2 or 3 monophonic parts is ideal, particularly to start with. Later as the person builds up some experience of vibratory perception, they can perhaps go on to up to 4 parts later, but we think not beyond that number. The use of music with this restriction on the number of parts is strongly recommended for profoundly deaf people.

For therapeutic use this requirement is not necessary, which is fortunate as this would be too restrictive of the repertoire of music for this purpose. But some reduction of original music is often useful where the music is complex and this may be done by the selection of suitable arrangements of music.

5. Separation of parts.

The ear/brain is able to separate the sound of instruments wonderfully well. Thus the sound of a prominent oboe part in a symphony orchestra can be picked out against a background of perhaps 80 instruments playing at the same time. Moreover this is a task that even a room full of computers is unable to accomplish.

However, the body through the sense of touch, certainly cannot do this! Hence we believe it is very important: to simplify the body's perceptual task, and we do this by splitting up the parts and sending them to different parts of the body. For example suppose we have a trio arranged for baritone saxophone, trombone and bass guitar. We might place the vibration generated by the saxophone on the right side of the back, that from the trombone on the left back and the guitar's on the buttocks and thighs. If in addition there were another bass instrument, playing lower notes, we might place that on the feet. Not everyone would prefer that choice of positioning. In particular, some people like the tune on the left back and the counter-tune on the right back and vice versa for others. The actual choice of which part goes where can be made by the individual if desired, simply by setting 4 rotary switches.

How is this separation of parts achieved?

With a MIDI based system each part is allocated a separate MIDI channel and a MIDI sound generator is used which has a facility to allocate a separate audio output to each MIDI channel. For example the Roland D100 is very suitable for driving the vibrational systems and these are now available at very moderate cost second hand. Further details can be found in [6].

Without MIDI it is more difficult, yet having a means is important because, with MIDI the repertoire is limited, (although ever increasing) and, perhaps more importantly, many of the commercially available MIDI based performances are of a mechanical nature without expressive feeling. One solution is to make special recordings. While acceptable for duets and trios, costs (or difficulties in finding suitable volunteer musicians) would increase rapidly with larger groupings. Another difficulty is that to obtain really good separation (and this is not something which is required, or even desirable, for normal audio listening) the instruments would have to be widely spaced and with microphones very close to the instruments. (The ideal would be in separate sound booths).

A solution which we have employed is to add a part to a commercial recording (eg an ordinary CD of the music). To do this we ask a musician to play along with one of the performers on the recording on a piano type keyboard and we record this as separate track on a four track recorder, the other tracks being used for the signal processed version of the original recording. This requires some special skill (beyond being a reasonably good pianist or organist) and dedication for preparation of the part of the musical since the "doubled" musician cannot be seen, nor the conductor if there is one. There are advantages in using a MIDI based, rather than just an audio producing keyboard system for the doubling of a part: it allows the performer to hear what is being played by them at the same pitch as the doubled instrument and with a suitable aesthetic choice of timbre for the performance. Simultaneously, e a sound in a different octave, amplitude envelope and harmonic content can be generated for recording on the multi-track tape. Some performers are aided in this process by being offered a monitoring system which allows them to hear their own playing in one ear, and the original recording in the other,

6. Aesthetic, and ergonomic considerations

This paper is about design requirements and results will be mention very briefly. (These are the subject of a paper in preparation.)

People who prior to being become very deaf, were music lovers, and who have experienced other vibroacoustic systems, have without exception, said they said something like "I have never experienced music anything like so clearly". No statistic needed here!

We have tried it with 6 children who were "attention demanders" in the classroom; these children were described by their teachers as not giving them a minute's peace. Of these, 5 of were calm and relaxed and showing every signs of enjoying the experience within 2 minutes. This amazed some of the teachers - who said that has never seen these children like that, that they went to fetch colleagues.

Very good results, which could truly be described as "heart-warming" have been achieved in brightening up the lives of very severely brain injured children and young adults.

Further trails are planned or in progress, and we hope to carry out trial with people who have certain mental illnesses, in co-operation with medical colleagues.

[1] Skille, O. (1992) Vibro Acoustic Therapy : Manual & Reports. Levanger, Norway. ISVA

[2] Wigram, A.L. (1996) The Effects of Vibro Acoustic Therapy on Clinical and Non-Clinical populations. PhD Thesis. www.members.tripod.com/~quadrillo/VAT/tonyphd.htm

[3] Summers, I.R. (Ed.) (1992) Tactile Aids for the Hearing impaired. London: Whurr Publishers Ltd

[4] Bolanowski, S.R. et all (1988) Four channels mediate the mechanical aspects of touch. J Acous Soc AM 84: 1680-1694.

[5] Varillo, R.T. et all (2000) Some basics of tactile sensation: temporal and spatial considerations. ISAC'00 Univ. of Exeter.

[6] Dalgarno, G. (1989) A computer based system for music for hearing impaired people". Proceedings of the Second National Conference on Music and the Hearing Impaired. University of Kansas (1989) pp 31-42

[7 ] Wigram, A.L. & Dileo, C. (Eds.) Music Vibration . Jeffrey Books, New Jersey, USA.