US4018121
US4018121
P6 MOD. FREQ.
P8-P7). P6DEV2
(P8-P7)
P7. P6 DEV
P5 CARR FREQ.main
ATTACK O
Ø. ID 5.2
ID (2.2)
AMP FUNCTION
AMP
ATTACK STEADY STATE DECAY
0 Hz /v1. sal-TIME
?h JULUUUUUIllinn. FIG. I.
?hrime
ANADHDhn WC = 1.2
A1Ih.
? h Y:
ID = 2.0
ID2 = 5.2)
4. 11
1In
P6 MOD. FREQ.
(P8-P7). P6 Ev
P7. P6 DEV
P5 CARR, FREQ.
ana
1.2-, 2 X2
AMP
0 HZ-
NMN1. TIME
Mattit TTTTTTLILITITLETTLError
MTlin
MTTTTTrier------- WC = 1.2
Y
Mr. WM = 4
CR D X as 2.2)
tra- no app a no-amp -- run was a- - was un- a
DX 2s. 8.0
F (R
EQ. N-------------
is -----------------.
RY-------------------
o - am as us - amo am - a H - - - -
All me - - - - - - - - - - me an a- - - - - - -
FIG.3 V r"
U.S. Patent April 19, 1977 Sheet 3 of 17 4,018,121
( \,.
. . AMP
FUNCTION
5 SEC. 2 NDEX
AMP
2 HZ 4||N TIME
WC = .2
WM = 2.2
FIG. A. IDX = 2.0
------u- IDX2= .0
U.S. Patent April 19, 1977 Sheet 6 of 17 4,018,121
+TILO
U.S. Patent April 19, 1977 Sheet 7 of 17 4,018,121
AVOB
! Dae)| ! ! !
3)| | czw?=#F
C O
GN9
U.S. Patent April 19, 1977 Sheet 8 of 17. 4,018,121
U.S. Patent April 19, 1977 Sheet 9 of 17 4,018,121
U.S. Patent April 19, 1977 Sheet 10 of 17. 4,018,121
D 2 MEM 2.
XSS wDI 8 D MEM
ST 8 D 2 MEM 2
FT 8 D 3 MEM 3
MP9
53A
ST 9 x DI9
FT 9
MP 2 2 MP2 2) D 4 MEM 4
AS2-A
ST 2 2
w|DI 283 -
ST2-a-2
wDI to DrD 56 MEM 5
MEM 6
FT 2 3. YE 3 DI. 7 MEM 7
MP 3- 2 74.15 2
ST 3
FT 33-N-7 3 WENBX
X DI 359:S.
FTU -3
BA B
MP 4 M P2 DI 8 MEM 8
6 N- w DI4AO2 DI 9 MEM 9
ST 3. ST 2 D. O MEM O
FT 4 MPXR FT2 D MEM
MP 5-a 2 74I53 MP3
? EE 3.
FT 5
DI59FT3
MP 6 MP4 D 2 MEM 2
DI6AST 4 MEs
FT 6 FT 4 D. 5 MEM 5
MP 7 X5's
AO 7
ST 7
x DI7SF3
FT 5
FT 7 MA 2
MA
WS - MA 2)
WS2 GNO C --
C
WE D SQ
36
y 7474. 26
ra/ I LOAD C
U.S. Patent April 19, 1977 Sheet 12 of 17. 4,018,121
dS
C.
W~I A~NE )
U.S. Patent April 19, 1977 Sheet 13 of 17 4,018,121
|
|
18
| Sn8S
| 18
Ø
6
SngSn8sngSn8, 68 STñIE
6
4.9
O
8
Sn8Sn
G
98
,
„18
2.9
iz828
WTES
Ø gºezwu £2]©22iØ
OO
Sng
U.S. Patent April 19, 1977 Sheet 14 of 17. 4,018,121
€NEOW
TENwiOWg~—sÇ
,ETOLITOÅè?O1JI
d'N
9
O
U.S. Patent April 19, 1977 Sheet 15 of 17 4,018,121
ON™)
dW
|-
dW
dW
dW
dW
\
+
~
W
O
E
EN
U.S. Patent April 19, 1977 Sheet 16 of 17. 4,018,121
SngO]
/IIºf )IJI
U.S. Patent April 19, 1977 4,018,121
C
1 4,018,121 2
varies as a function of time in accordance the timbral
METHOD OF SYNTHESIZING A MUSICAL souND qualities of the musical sound to be synthesized.
CROSS REFERENCE TO RELATED APPLICATION In accordance with the above object, the method
This application is a continuation-in-part of applica comprises the steps of selecting carrier, co, and modu
lation, on, frequencies in the audio range. The coe and
tion Ser. No. 454,790, filed Mar. 26, 1974, and now a frequencies are modulated in accordance with
abandoned. :
e = At oth f(t) sin at
BACKGROUND OF THE INVENTION
The present invention is directed to a method of 10 where e is the instantaneous amplitude of the frequency
synthesizing a musical sound and more specifically to a modulated wave, A is the peak amplitude and I(t) is the
technique utilizing frequency modulation to provide modulation index. I(t) is selected as a predetermined
for the temporal evolution of the spectral components function to control the bandwidth of the wave and its
of a musical sound. evolution in time.
Several types of musical synthesizers are well known 15 BRIEF DESCRIPTION OF THE DRAWINGS
in the art but thus far the synthesis of natural sounds
has been elusive. In a typical type of analog synthesizer, FIG. 1 is a dynamic FM spectrum useful in under
a voltage controlled oscillator is driven by a waveform standing the invention;
of the desired shape and frequency and then filtered FIG. 2 is a circuit useful for practicing the present
and passed through an attenuator to provide the proper 20 invention as represented in MUSICV notation;
envelope to simulate a desired musical or natural FIGS. 3 and 4 are dynamic FM spectra useful in
sound. With the foregoing analog synthesizer which is understanding the invention; and
of the subtractive type, there is, of course, no evolution FIGS. 5 through 18 are block diagrams for a digital
in time of the various spectral components or partial 25 FM synthesizer embodied in the present invention.
frequencies of the final sound. DETALED DESCRIPTION OF THE PREFERRED
Synthesizers utilizing digital techniques have realized EMBODIMENTS
that to create a natural sound individual partial fre
quencies must be generated and combined. One type of In general the present invention provides a frequency
organ is based on this principle where the several par 30 modulation technique for producing a complex spec
tial frequencies are added together and then given a tra; that is, one which has a spectral evolution which
common envelope function. The combinations of such evolves in time with relative simplicity. In other words,
frequencies are based, of course, on the principles the frequency modulation technique of the present
established in Fourier analysis. invention provides for specific control of the individual
Yet another digital synthesis technique has been 35 or partial frequencies making up a total or natural
suggested by Jean-Claude Risset and Max V. Mathews, musical sound.
Specifically, this is accomplished by selecting carrier
"Analysis of Musical Instrument Tones," Physics To
and modulation frequencies in the audio range and
day, vol. 22, no. 2, pp. 23-30 (1969). Rissetestablished frequency
that the character of the temporal evolution or the modulating the carrier a with the modula
evolution in time of the spectral components of a sound 40 tion frequency, an in accordance with
is of critical importance in the determination of timbre. e re. A cut (t) sin of (1)
In other words, Risset would suggest that to simulate a
natural sound the amplitude of each harmonic should
be individually controlled as a function of time. In where e is the instantaneous amplitude of the frequency
modulated wave, A is the peak amplitude and I(t) the
addition, Rissetsuggested, at least for the production of
the trumpet tones, that the attack envelope, that is, the modulation index. Moreover, the modulation index is
initial envelope characteristic for the trumpet tone, has bandwidth ofa the
selected as predetermined function to control the
wave and its evolution in time.
a distinctive characteristic in that during the attack and
also decay portion of the sound, the energy of the vari a brass-like sound. the
FIG. 1 illustrates
The
foregoing for the production of
dynamic spectra of a typical
ous frequency components evolve in complicated ways. 50 brass tone is shown as functions
To implement the Risset theory with known tech amplitude in the Attack mode toofsteady frequency, time and
state mode and
niques requires a complex digital computer which indi into a Decay mode. The curve 10 labeled amplitude
vidually simulates each frequency component. Thus, at function is a characteristic envelope function
the present time a real time music synthesizer of the overall tone or musical sound and varies the factorofAthe in
digital type is not commercially feasible. 55 equation (1). Curve 11 labeled index function shows
The variation of bandwidth with modulation index the variation of I(t) from an initial zero point ID to the
has been illustrated in an article by Murlan S. Corring final steady state point ID. The spectral evolution
ton, "Variation of Bandwidth With Modulation Index curves of FIG. 1 are based on carrier and modulation
in Frequency Modulation," Selected Papers on Fre frequencies equal to one another. That is, they have
quency Modulation, edited by Klapper (Dover Publica 60 relative values of 1.0 as indicated in the drawing of
tions, 1970). However, this is merely a theoretical FIG. 1 with ID equal to zero and ID, equal to five. The
study of frequency modulation. overall envelope of amplitude function 10 in essence
OBJECTS AND SUMMARY OF THE INVENTION varies the peak amplitude A of equation (1). Thus, the
intensity of sound increases from a zero level as shown
It is, therefore, an object of the present invention to by curve 10 to a maximum at steady state and then
provide an improved method of synthesizing a musical decays in a somewhat linear manner.
sound composed of a plurality of partial frequencies in Risset demonstrated in his analysis of trumpetitones a
which the amplitude of each frequency individually fundamental characteristic of this class of timbres; the
3
4,018,121.
4
amount of energy in the spectrum is distributed over an Ps = 440 Hz (ratio of cfm = 1/1)
increasing band in approximate proportion to the in P =0
crease in intensity. This is illustrated in FIG. 1 where P = 5.
initially only the carrier and lower harmonics such as A standard MUSIC V program is suitable in many
the second and third have any appreciable amplitude 5 instances but depending on the musical sound being
and thereafter during the state state designated SS generated the instantaneous frequency of the modu
the higher harmonics are increased in intensity. Other lated carrier at times become negative; in other words,
characteristics of a brass tone are that frequencies in the final waveform would have a phase angle which
the spectrum are in harmonic series, both odd and even decreases with time. This condition occurs when either
numbered harmonics are sometimes present, and as O the ratio of the carrier to the modulating frequency is
stated by Risset but not specifically illustrated in FIG. very small or the modulation index is very large. Thus,
1, the rise time of the amplitude or envelope function is the unit generator U.G. 3 of FIG. 2 must be able to
rapid for a typical attack and may "overshoot' the produce a wave which results from taking the sine of an
steady state. Moreover, a comparison of the curves 10 angle which decreases as well as increases with time.
and 11, illustrates that the modulation index varies in 15 The change in code to the oscillator in MUSIC V to
direct proportion to the amplitude of a modulated car allow for decreasing angle is:
rier wave. for
The musical brass-like sound illustrated in FIG. 1 is 290 IF(SUM-XNFUN).288, 287, 287
preferably achieved by use of a special purpose com 287 SUM=SUM-XNFUN
puter or digital FM synthesizer as will be described substitute :
below. However, for demonstration purposes as to the 290 IF(SUM.GW.XNFUN) GO TO 287
effectiveness of the concept of the present invention a IF(SUM.LT.0.0) GO TO 289
typical minicomputer can be programmed with a FOR and for
TRAN IV program written in MUSIC V. MUSIC V is a GO TO293
well-known program as set out in a look by Max V. 25 292 J6-L1-3-1.
Mathews, The Technology of Computer Music (The substitute
MIT Press, Boston, 1969). The difficulty with using the GO TO 293
MUSIC V program is that it is not a real-time on-line 287 SUM-SUM-XNFUN
system. GO TO 288
In general, the MUSIC V sound synthesis program is 30 289 SUM=SUM+XNFUN
a program which generates samples of a sound wave GO TO 288
which are then stored in a memory device as computed.
Digital to analog conversion and filtering then allows FIGS. 3 and 4 illustrate respectively bell-like and
an audio system to regenerate the sound. The program clarinet-like tones. Referring to FIG. 3, the bell-like
is designed so that computation of samples is done by 35 timbre of the family of percussive sounds is developed
program blocks called unit generators abbreviated around the following two premises:
U.G. A typical unit generator is an oscillator which has 1. The spectral components are not usually in the
two inputs, an output, and a stored wave shape func harmonic series, and
tion. The first input specifies the amplitude of the out 2. The evolution of the spectrum is from the complex
put, the second input the frequency of the output, and 40 to the simple. The amplitude or envelope function of
the function determines the shape of the output. The the bell-like sound illustrated has an exponential decay
value of an input can either be specified by the user or which, for example, may terminate at a time of 15
can be the output of another unit generator, thereby seconds. The index function is directly proportional to
allowing multi-level operations on waveforms. A col the amplitude envelope. From a MUSICV standpoint,
lection of interconnected unit generators is called an 45 the parameters to produce a bell-like sound can be the
instrument which is supplied data through a set of pa following:
rameters P, through Ps Prs 15 seconds
Referring to FIG. 2, this is a suitable instrument for P = 1000
producing a FM circuit which generates a dynamic P = 200 Hz
spectra in accordance with the present invention. A 50 P = 280 Hz
unit generator 4 produces an amplitude envelope simi P =0
lar to envelope 10 of FIG. 1 and a unit generator 5 a P = 10.
modulation index envelope similar to envelope 11 of More importantly, however, the carrier and modula
FIG. 1. The parameters for the instruments have the tion frequencies are related to one another by an irra
following functions. 55 tional number or a number ratio which is not an inte
P = Begin time of instrument ger. As illustrated in FIG. 3, this causes inharmonic
P = Instrument number spectra where the components are not in a relation of
P = Duration of the "note' simple ratios. However, such irrational numbers should
P = Amplitude of the output wave be small enough to maintain the density of partial tones
P = Carrier frequency 60 to produce, for example, the bell-like sound of FIG. 3.
P = Modulating frequency The irrational ratio of colon = 11 V2, for example,
P = Initial modulation index ID which produces a nonperiodic waveform, and where
P = Final modulation index ID. the bandwidth is controlled by the index function in
The parameter values for brass-like tones such as time, can produce bell tones and other percussive tones
illustrated in FIG. 1 would be the following: 65 as shown in FIG. 3.
P = 0.6 The same technique can also be used to produce
P = 1000 (amplitude scaling) secondary features of quasi-periodic tones, such as the
P = 440 Hz "scratchiness" during the attack of a violin tone. This
5
4,018, 121
6
will be termed a "grit function' hereinafter. In this The component at 0 Hz represents a constant in the
case, this index or grit function would only be non-zero wave. The remaining lower-side frequencies are re
during the attack interval of, for example, 0.025 sec flected into the positive frequency domain with a
onds after which the spectrum would be composed change of sign (inversion of phase) and add algebra
solely of the rationally related colon ratios. This would ically to the components which are already there. For
demand two modulating oscillators and one carrier example the second lower-side frequency will add to
oscillator, where the first modulating frequency is re the carrier with like signs, therefore increasing the
lated to the carrier, olon = 1/1, and the second by energy at 100 Hz, while the third lower-side frequency
11 W2= a?on. In such as case, the FM modulation of will add to the first upper-side frequency with unlike
the present invention would occur in accordance with 10 signs, decreasing the energy at 200 Hz. The foregoing
the following equation: as well as other facets of the present invention is dis
e - Ali (a t + I(t) sin on t + 1, (t) sin at (2) cussed in an article by the present inventor in the Jour
nal of the Audio Engineering Society, September, 1973,
where I(t) and an are equivalent to I(t) and cont of 15 entitled "The Synthesis of Complex Audio Spectra by
equation (l) and on is an additional modulating fre Means of Frequency Modulation.”
quency where colon is equal to an irrational number. A formant peak may be accomplished by the FM
Thus, the first index (t) would have an envelope cordance with modulation technique of the present invention in ac
shape similar to the amplitude envelope of the brass
like sound of FIG. 2 but in addition the grit function 20 e F Ai (cuct act t (t) sin cont) (3)
would be added and would be the predetermined func
tion of the second index I,(t) and having a duration of where oc -- oc are two carrier frequencies having
less than 200 milliseconds. Thus, for a violin sound ratios with on. Such ratios typically being cofon = 7/1
along with the grit function the carrier and first modu and oc1a)n = 1/1 and the 7/1 ratio placing the formant
lation frequency would be related by a function 1/1 = 25 peak at the seventh harmonic.
a?on. The first index function would be inversely From the foregoing it is apparent that the present
proporational to the rising amplitude envelope. invention provides a simple technique for providing for
From the foregoing it is apparent that the ratio of the the timbral evolution for the various frequencies or
carrier and modulating frequencies determines the partials of a complex musical sound. Moreover, the
position of the components in the spectrum. The modu 30 present invention realizes such timbal evolution espe
lation index itself determines the number of compo cially in the attack portion of the sound often provides
nents which will have a significant amplitude. With the "signature' of the sound. In other words, this is
regard to simple frequency ratios, N/N, the following what the listener judges as the lively quality of the
generalizations can be made: sound. In contrast, it is the fixed proportion spectrum
1. The carrier is always the Nth harmonic in the 35 of most synthesized sounds that readily imparts to the
series. listener the electronic cue and lifeless quality. The FM
2. If N = 1, the spectrum contains all harmonics and synthesis technique of the present invention is far sim
the fundamental is at the modulating frequency, e.g., pler than additive or subtractive synthesis techniques
1/1,2/1. which can produce similar spectra. It is believed that
3. When N is an even number, the spectrum contains 40 the FM technique of the present invention duplicates
only odd numbered harmonics, e.g., 2, 4,3/2,%, 5/2. natural sounds more cost effectively than if a very com
4. If N = 3, every third harmonic is missing from the plex computer were utilized to control the amplitudes
series, e.g., 4, 4, 4/3, 5/3. of individual partial frequencies in a very precise man
FIG. 4 illustrates a clarinet-like timbral sound where ner. In other words, although the control of the present
the index function curve is inversely proportional to the 45 invention is seemingly limited in that precise amplitude
leading edge of amplitude function. In addition colon = control of each partial frequency cannot be varied fully
%. This, as stated above in rule (3) produces only odd as desired, this proves to be no limitation as far as the
harmonics which is a well known characteristic of the subjective musical impression is concerned.
clarinet sound. Finally, the present invention may be capable of
With respect to the grit function (t) this is of a very 50 generating "musical sounds' which have not hereto
short duration compared to the period of the carrier fore been heard by a human being. Thus, the use of the
frequency, (). term "musical sound' is not meant to be limited to the
The special richness which may be produced by the standard musical sounds now known.
technique of the present invention lies in the fact that From the foregoing, it is apparent that the frequency
there are ratios of the carrier and modulating frequen 55 modulation technique of the present invention is quite
cies and values of the index which will produce side different from the addition of a typical vibrato or peri
band components that fall in the negative frequency odic variation of a frequency around some average
domain of the spectrum. These negative components which is added to a musical sound. In vibrato, the mod
reflect around 0 Hz and "mix' with the components in ulating frequency is merely a few cycles per second and
the positive domain. The variety of frequency relations 60 thus the ear has little difficulty tracking the instanta
which result from this mix is vast and includes both neous frequency of the carrier. However, where the
harmonic and inharmonic spectra. carriers and modulating frequencies are either equal or
A simple but very useful example of reflected side of approximately the same order of magnitude, the ear
frequencies occurs if the ratio of the carrier to modu can no longer track the instantaneous change in fre
lating frequencies is unity. For the values 65 quency as a sweep frequency but rather perceives a
a = 100 Hz complex spectrum.
a = 100 Hz Although the MUSIC V program will produce musi
=4 cal sounds in accordance with the FM techniques of
4,018, 121
7 8
this invention, if real-time operation is desired, a digital word I0-115 are stored into the latches. The outputs of
FM synthesizer should be used. the latches perform the following functions. The MA0,
FIG. 5 illustrates a micro-programmed device which 1,2,3, outputs serve as a memory address for the
has a digital output on line 21 which is converted to scratchpad memory 26 (FIG. 5). WS0, 1 controls the
analog output by digital to analog converter 22 to pro selector 36 which controls what is written into the
duce the desired musical sound at loudspeaker 23. The scratchpad memory 26. In general the scratchpad
device has as its inputs a 3 bit binary number represent memory 26 can be written into from four sources.
ing the instrument or rather a selection of different Namely, the multiplier 31, adder 32, sine memory 34
timbres or tone quality, a seven bit binary number and frequency memory 37. This occurs at initialize
representing the desired frequency of the musical note 10 time discussed in conjunction with FIG. 11.
and a key bit which initiates the generation of the musi Referring now again to FIG. 6, the outputs LS0, 1,
cal sound. The synthesizer will then generate succes and 2 form a three bit binary number which decodes to
sive 16 bit binary numbers on its output 21 which rep one of either latch selects which are for the purpose of
resents the waveform at 50 microsecond intervals. If directing information to the system. Specifically, one of
the device is to be used as the sound generating part of, 15 its functions is to specify which data sink will latch the
for example, an organ all that is required is to feed the data that is on the bus. RE0, 1, 2 form a three bit binary
number of the key that is being depressed as the fre number which selects a data source to be gated onto
quency information and, of course, the actual pressing the bus. LOB is for the purpose of gating the low order
of the key. All of the blocks of FIG. 5 are shown in eight bits of the scratchpad memory onto the bus. The
detail in subsequent drawings and can be made up with 20 purpose of LOB is for sine wave interpolation where
standard off-the-shelf components. Each block in the 16 bit angle of which the sine is being taken is
FIGS. 6-18 is labeled with a standard type number divided into a high order eight bits and low order eight
which may be found in the TTL Data Book, Number bits. Since the sine memory 34 itself only accepts the
CC-411, Texas Instruments, Incorporated and INTEL high order eight bits, the interpolation is done on the
Data Catalog, October; 1973, and ordered from Texas 25 low order eight bits and the LOB signal essentially turns
Instruments Components, Group, Market Communica off the high order bits and sets them to zero when it
tions Depart., P.O. Box 5012, M.S. 84, Dallas, Texas goes onto the bus. ISEL and SSEL select which enve
75222 and Intel Corporation, 3065 Bowers Avenue, lope function is of interest. They are finally directed to
Santa Clara, California 9505l. the envelope memory 27 of FIG. 5. They can select
Referring now specifically to FIG. 5, there are illus 30 either an amplitude function or a modulation index
trated both sources of information and sinks. All data is function or the second modulation index function. The
communicated over the data bus 24. The sources of
information are a scratch pad memory 26, envelope isWEwrite signal in this instruction word from the latches 42
enable and enables writing into the scratchpad
memory 27, initialization memory 28 and a binary memory 26. Lastly, WS0, 1, provide the scratchpad
constant 29. The sinks are a multiplier 31, adder 32 and 35 memory write data and selects one of four inputs to the
the output latch for register 33. Two other sinks are not scratchpad memory which are multiplier, adder, sine
directly connected to the data bus are the sine memory table and frequency memory,
34 which is a read only memory and in addition, The remaining gating shown in FIG. 6 is AND gate
scratch pad memory 26 which can also accept data.
Information in the search pad memory 26 is inputed by 40 40, OR gating 46, and D flip-flop 47 which serve the
a four way selector 36 and a frequency memory 37. purpose of shutting off initializing after start up. During
Envelope counter memory 38 which is driven by an start up the system is in INIT state. This state is com
eight bit binary counter 39 provides a segmented enve pleted by an Init Done signal (which is an input to OR
lope as will be discussed below. gate 46) which means that the program counter has
Lastly, the actual program for the digital synthesizer 45 overflowed and the initialization program is completed.
is provided by an instruction memory 41 and its asso In general in operation if, for example, an update on
ciated latch 42. Such memory is shown in greater detail the amplitude of the output sinusoid is desired, this is
in FIG. 6 and consists of four 74186 read only memo accomplished by storing the current value of the ampli
ries that can hold 64 eight bit words each. Thus, a tude of the sinusoid and scratchpad memory 26 and
combination of two of the memories produces a stan 50 then reading out of the envelope memory 27 the incre
dard 16 bit instruction word labeled 10 through I15. ment to that value. Thus, on one instruction cycle the
The memory pair 41a is for the 64 word running pro scratchpad memory will be gated onto the data bus by
gram and the pair 41b for the start up program. The the adder latches of adder 32 and the adder will latch
start up program is initiated by a true signal on the onto the current amplitude position. On the next in
active input. After this 64 word program is run through 55 struction cycle the envelope memory 27 is gated onto
once, the running program then cycles. Each cycle is the data bus and the other adder latch activated. A few
50 microseconds in which time the running program microseconds later, the sum appears at the adder out
cycles through all 64 words and then starts at the begin put, AO0-15.
ning again. This 50 microseconds is also the sample FIG. 7 provides control logic for the latch instruction
time; that is a new sample is delivered to the D/A con 60 outputs of FIG. 6. Specifically the latch select bits LS0,
verter 22 (FIG. 5) for every sample time. Memories 1, 2 are coupled to a decoder 48 which produces when
41a, b are addressed by the program counter which enabled a signal on one of eight different lines. These
consists of blocks 43 and 44 which has as outputs PC0 include the multiplier latch signals MPL1, 2 and adder
through PC5. These signals address the instruction latch signals AL1, 2. Also, sine table latch, ST and a
memories at the inputs indicated advancing the instruc 65 signal EL which is used as a control signal to cycle the
tion and then wrapping around at 63 back to zero. The envelope memory onto the next segment; this occurs
latches 42 store on each cycle the output of the read once every 63 instructions. This is done because of a
only memories 41; that is, the 16 bits of the instruction lack of sufficient bits in the instruction word. The last
9 4,018,121 10
output "load output buffer" causes the output latch 33 is to prevent a timing problem. The timing problem is if
(FIG. 1) to store words from the data bus. the key stroke is very very short such that the device is
Control bits RE0, 1, 2 to decoder 49 determine still in the Attack sub-state but the key is lifted the
which of four data sources are gated onto the data bus. envelope control keeps it in Attack state until the at
Output SP gates the scratchpad memory onto the bus; tack is finished and then goes into Decay state; so it will
IM gates the initialization memory 28 onto the bus. never abort the attack because of a very short key
Such memory contains data as for example, the factor stroke.
by which the fundamental frequency is multiplied to The foregoing is illustrated in the state diagram of
produce either the modulating frequency or carrier FIG. 8. Changing states is done on the AND of the
frequency. As is apparent from the foregoing examples, 10 previous state and the state changing condition and the
this is generally an small integer factor of 1,2,3. This clock. For instance, to get out of idle state, there must
is, of course, done at the initialization time. The con be present the AND (gate 56) of Idle, the key has
stant C1 is % which is the difference between a sine and become depressed, and master clock (MRCLK); gate
cosine angle so that a cosine angle may be processed as 57 goes into the Init state. To get out of Init state, Init
a sine angle during sine table table interpolation. C2 is 15 Done must come up to activate gate 58. Init Done
unused. ENV gates the envelope memory 27 onto the (FIG. 6) comes true when the program counter 44
bus which, of course, contains the increment that is to reaches 63; that is, the initialize program is finished.
be added to the current value of either the modulation Run state is gotten out of by being in Run state and the
index or amplitude envelope for the next step. key being raised; that is, "Not" Key comes true and the
Memory 48 is enabled by a bus enable input which is 20 master clock closes gate 59 to go into Decay. Finally
produced by the gate 51 having as inputs load, CLR2 Idle state is returned to by being in a Run Down sub
and latch. Decoder 49 is enabled by single I Load. Both state as Envelope Done comes up, Decay is true and
load and latch are generated by the four coupled master clock. Thus, if the device is in Decay mode and
flip-flops 52 which are fired in order; that is, only one the envelope has cycled all the way through into Run
of them is true at any one time. Flip-flops 52 count the 25 Down mode and is done, signified by Envelope Done,
master clock (MRCLK) and each time the master the idle is returned to.
clock goes true the stored bit advances to the next FIG. 9 contains the blocks envelope memory 27 and
flip-flop. The first flip-flop increments the program envelope counter memory 38 of FIG. 5. These are all
counter with the output PCINC and in turn the pro INTEL (trademark) model 1302 (see INTEL Data
gram counter causes data to be produced from the read 30 Catalog) read only memories. The envelope memory
only instruction memories 41. At the next clock pulse I consists of two 1302s which provide 256 different 16
Load will be produced which latches the instruction bit words. These 16 bit words as they come out of the
word and gates the data source onto the data. The next memory are labeled Env 0, 1, through 15. They are the
clock cycle is wait and on the last clock cycle LATCH increments to the current position, e.g., the attach
will come true and will cause whatever data is present 35 amplitude or the modulation index or the second mod
on the data but to be latched into one of the data sinks. ulation index. The envelope words are the amount that
This completes an instruction cycle at which time the is added to those amplitudes at each sample cycle; that
initial flip-flop causes the program counter to incre is, at one loop of the instruction memory; (50 micro
ment another step. Thus, in operation in general at I seconds). The envelope counter memory 38 specifies
load the instruction word is produced and control sig 40 the number of sample cycles where the above incre
nals propagate around the system, the data getting ment is true; the process is a piece wise linear approxi
gated onto the bus and then latched off the bus to mation and the counter memory specifies the number
perform whatever function is desired as, for example, of samples for each piece of the piece wise linear seg
adding or multiplying ment. The number ENV 0 through ENV 15 is actually
The last two functions illustrated in FIG. 7 include 45 related to the slope of that piece wise linear segment.
the production of the master clock which is produced The counter memory generates an eight bit count CNT
by the crystal clock 53. The clock runs at a rate such 0 through 7.
that an instruction word is executed in about 800 nano All of the above is addressed by various inputs. There
seconds which thus allows 64 instruction words in 50 is instrument number which is a three bit number INS
microseconds, Finally, OR and AND gates provide a 50 0, 1 and 2 where different Attack and Decay envelopes
clear pulse so that when power is supplied to the device are selected for different instruments. With the seg
all registers are reset.
The major states of the device from a system stand on anynumber
ment SEG 0, 1, an Attack can be synthesized
instrument with up to four segments and the
point are Idle, Init, Run, and Decay. In the ldle state Decay with up to four segments. Also gated is the signal
there is no note playing and no key is depressed. When 55 "attack" which selects a different set of piece-wise
a key is depressed the device goes into Inite state and it linear segments for the attack compared to the decay.
runs the 64 word initialize program in the instruction The signal Init is ORed with Attack to ensure that the
memory. When that program is finished, it automati data will be ready as soon as the Init state is completed
cally goes into Run state and it stays in Run state until and Attack is begun. The other two bits in the eight bit
the key is lifted. When the key is lifted it goes into 60 address are SSEL and ISEL from the instruction word;
Decay state. How long it stays in Decay state is deter they select the desired envelope which may be the
mined by the Decay envelope in the envelope memory. Attack envelope, the first modulation index envelope
When it completes the Decay envelope, then the device or the second modulation index envelope.
goes back into the Idle state. There are also some sub All the other gates on FIG. 9 gate the envelopes onto
states associated with envelope control. For instance, 65 the bus. They are tristate buffers which have three
the Run state is divided into two sub-states called At states; true, false and not enabled. Data is gated onto
tack and Steady State and Decay major state is called a the bus with the OR of two signals; ENV which is from
run-down state in the envelope control. This renaming the read enable bits of the instruction word, that is
11
4,018, 121
12
decoder 49 (FIG. 7) and CNTEN or count enable. FIG. 11 is a more detailed diagram of blocks fre
Count enable is true if the device is in Attack and not quency memory 37 and initialization memory 28 of
in Decay; that is, count enable specifies that the ampli FIG. 5. The frequency memory is coupled directly to
tudes are changing. Otherwise the device is in steady the selector 36 for the scratchpad memory 26 and the
state and the amplitudes are not changing and there initialization memory 28 can be gated onto the bus.
fore zeros are gated onto the bus. Gating zeros onto the Since the initialization memory is only 8 eights, only
bus is illustrated in FIG. 7; gating the envelope incre the higher order 8bits is gated onto the bus and the low
ment itself onto the bus as shown in FIG. 9. order 8 bits are gated as zeros as illustrated in FIG. 12.
FIG. 10 illustrates envelope control and contains two Referring in detail to FIG. 11, inputed to the fre
sets of counters. The first set of counters 61 consists of 10 quency memory is a 7 bit number, FREQ 0 through 6.
three counters which take in the count from the enve In actuality, this is the number of a key starting at A
lope counter memory which is an eight bit count 0-7 natural 27.5 Hertz and every number specifies a half
and counts the number of EL signals. The EL signal is step from that low A, that is, 128 frequency numbers
a decoded latch select and there is one in every sample can specify a seven octave range above low low A. This
cycle; that is, one instruction of the 63 instructions is transformed via the frequency memory into an actual
turns the EL bit on. Thus, counters 61 essentially count frequency number. Thus the frequency number is the
samples. The number that comes out of the envelope amount that is added to the current position of the sine
counter memory is the negative or the two's comple table to get the next position in the sine table. That is,
ment of the number of samples to count until the seg 20 this is the incremental angle of which the sine is taken.
ment is completed. Counters 61 are enabled by Memory outputs FTO through FT15 are coupled into
CNTEN, count enable, and they only count if in an the selector 36 (FIG. 5) which can be selected by WS0
attack mode or run-down mode which is physically the and 1 in the instruction word which is the write select
Attack or the Decay of the signal. When counters 61 into the scratchpad memory.
overflow, CNTOV goes true, counter overflow, and 25 The initialization memory is addressed by the three
allows the other counter 62 in FIG. 10 to count. bits of the instruction and also by a four bit counter;
Counter 62 counts which of the four segments of the that is, up to 16 initialization constants can be supplied
piece-wise linear approximation is being processed. and every time the program asks for a new initialization
That is, at initialize time counter 62 is cleared and its constant with the IM signals (decoded from RE0
output is Seg 0 and Seg 1 and this goes directly into the 30 through 2) the read enable, which gates data onto the
address of the envelope memory and the envelope bus, will count counter 67 and thus go to a new initial
counter memory. In operation, some number of sam ization constant. The program essentially has to know
ples is counted and then CNTOV goes true and counts what these constants are and what order they are stored
to the next segment. A new count is reloaded and the
number of samples in that segment is counted. When 35 The kind of constants that are stored in the initializa
four segments have been counted through, envelope tion memory are the following: for most versatility, the
done comes true. That will cause, if in an Attack mode, frequency of the carrier sinusoid and the modulating
a move to Steady State; in Run Down it will cause a sinusoid are not necessarily the same frequency as the
return to Idle. fundamental frequency. The difference is that they will
There are a few other input signals. For instance, 40 in general be small integral multiples of the fundamen
load enable, LDENB, enables the loading of the sample tal frequency. Those integral multiples 0, 1, 2 or 3 or so
counter 61 by the envelope counter memory 27 and is are stored in the initialized memory to be later read
generated by the OR (gate 63) of load and CNTOV out. The other information that is stored in the initial
when the counter overflows; CNTOV has gone true ized memory are the beginning amplitudes. For exam
and a new count is ready to be accepted. However, 45 ple, some amplitude envelopes start at a non-zero num
upon going into Attack mode the count has not over ber like the bell which starts at a loud point and decay
flowed and some way of getting it started is needed. down. However, for most instruments, those numbers
Thus, when attack first goes true, Load is set. are 0. These are stored in a specific order; i.e., the
Count enable, CNTEN is generated by the OR of multiplier for the second modulating frequency, the
Attack and RUN Down (gate 64). This is essentially 50 multiplier for the first modulating frequency, and the
true during the Attack Decay portions of the waveform multiplier for the carrier frequency. Gated out of the
but it is not identical to the Decay major state. Specifi initialization memory onto the data bus (IM0-7) are
cally, four flip-flop 66 form a four-flop. Attack is gener the beginning positions for the angle of which the sine
ated by going into the Run major state, then Attack is taken, which in most cases is 0. The next three num
goes true and it also brings Load true. But Load only 55 bers gated out of the initialization memory are the first
stays true for one sample cycle whereas, Attack will modulating index, I, the second modulating index, I,
stay true until Envelope Done comes up; that is, until and the signal amplitude A. I, , and A are actually
all four segments have been counted through. Thereaf locations in the scratchpad memory. The following is a
list of the scratchpad memory locations and then the
ter, Steady State mode will be stayed in until essentially symbolic
going into Decay major state; that is until the key is 60 names.
lifted; then the device goes into Run Down mode. R
load is generated like Load; that is, it stays on momen SP locations:
tarily whereas Run Down stays on until Envelope Done pos
comes true. If the key is tapped very very shortly, the
|
fr ; Fm2
fr108 ; Fm28
major state will change to Decay but the device will 65 pos2
stay in Attack mode until Envelope Done comes true; fr2
fr208
then it will go into Steady State for just an instant and pos3
then flip into Run Down mode. fr3
4,018,121 14
-continued of those conditions which produces a signal ENBGND
O l, ; Index 2 | 8
8-15, enabled ground, on 8 through 15 and this in turn
11 ; Index 1 | 8 controls the tristate latches which will gate ground on
12
13
A
ti2
; Amplitude envelope
; Temps
the low order bits of the bus. The high order bits will
4 gate bits 0 through 5 and bits 6 through 7 slightly differ
5 ently. That is, all the bits are ground on the condition
for that of steady state for the envelope. The only rea
son bits 0 through 5 are gated differently from bits 6
Specifically, fr1 specifies the frequency of the second 10 through 7 is that the constant C1 only occupies the first
modulating index, pos1 is the angle of the second mod 6 bits and the rest of them of zeros. If C1 is true, then
ulating sinusoid; fr2 and pos2 are the frequency of the zeros are gated onto bits 6 through 15 of the bus and
first modulating waveform and the current angle of the ground is enabled; that is, zeros are enabled on 6
first modulating waveform and fr3 and pos3 are the through 7 separately; and the condition under which
carrier frequency and its position; A, I and I are the 15 zeros are gated on 6 through 7 are if LOB is true; that
amplitude envelope, index 1 and index 2, respectively. is, low order bits are being gated. Zeros are gated onto
There are three temporary registers in the scratchpad the high order bits of the data bus from the instruction
memory designated t1, t2 and ti2. Two other numbers word if ENV and not count enable, CNTEN, are true;
fr108, fr208 are the modulating frequencies times 8. that is, if there is an envelope request and are in steady
state or there is an instruction memory or an initializa
This is a constant offset and is used because in actuality 20 tion
and indexes are divided by 8 to provide scaling because memory request, or if the constant is being gated
of a fixed point system. onto the bus. Zeros will also be gated onto the bus bits
Lastly, FIG. 11 shows the tristate buffers that gate the 0 through 5 if LOB is true; that is, low order bits only
initialization memory onto the data bus. and if envelope control and steady state are true.
FIG. 12 illustrates the scratchpad memory 26 (FIG. 25 FIG. 14 illustrates the adder 32. It has three parts; a
5). This contains the scratchpad memory itself which latches 16 bit adder 70 and two input latches 71,72. The input
are four read/write memories. These are four sixteen are strobed or made to latch data from the bus,
bit memories. Data into the scratchpad memory comes and 16 bits, on the AL1 and AL2 signals; AL1 latches 71
out of selector 36. The selector is selected by WS0 and directly AL2 latches 72. The output of these latches goes
1 which comes out of the instruction word as shown in 30 into the 16 bit adder 70. The output of the
FIG. 6. This can gate in either the output of the multi adder is called AO 0 through 15. It is a 16 bit number.
plier which is MP0 through 15, the adder AO0 through The output of the adder goes to two different places;
sine table input latch and the scratchpad memory write
15, the sine table STO through 15 and the frequency select. When the angle of a sinusoid is being updated
memory FTO through 15 and the selector produces an the
intermediate signal OI0 through 15 which goes directly 35 time this dataincrement
angular is added to it and at the same
into the scratchpad memory data inputs. The scratch can the updated angle be the can go to sine table. Thus, not only
pad memory is always reading but it is also write en memory but the sine tablestored loop
back in the scratchpad
up can be started at the
abled by WE which is a bit in the instruction word; that same time.
is, write enable, WE, and its clock ILOAD. FIGS. 15 and 16 control the 16 bit multiplier 31
The write enable on the scratchpad memory is true if 40 (FIG. 5). The 16 bit multiplier has two latches, a third
WE, the write enable bit of the instruction word is true. latch
It is clocked on not I load, that is, the falling edge of the gate whichthegates
for partial product, a 16 bit adder, a 16 bit
the multiplicand into the 16 bit adder
instruction load. This will set the write enable true and
as soon as the latch timing signal comes up, the write some control logic. Theproduct
to add it to the partial when shifted and it has
enable will be turned off; that is, presumably the data 45 multiplier latch which is actually logic
control
a
is essentially the
shift register 73 and
will have been written by then. a counter 74 that will count 15 times and then stop. It
The circuit of FIG. 13 accomplishes three different
things related to the data bus. It can gate scratchpad operates as follows: on signal MPL2 the multiplicandis
memory output which is Mem 0 through 15 onto the latched MPL1 the
into one of the multiplier latches. On signal
multiplier is latched into a 16 bit shift regis
data bus and it can gate zeros onto the data bus. This is, 50 ter, and MPL1
of course, important for the envelope control during That is, the fallalso of
resets the multiplier control circuit.
MPL1 will gate MP load true which
steady state where the amplitudes and modulation indi causes on the next fall of MP
ces are not changing. Also zeros can be gated onto the The counter is loaded with +1step, to
the counter to load.
take 15 steps not 16.
high order bits of the bus if the LOB, low order bit MP step is produced by MP GO and master
signal is true in the instruction word. This is for doing 55 flop 76. However, for synchronizing purposesclock the
flip
out
sine table interpolation. Finally, the third thing which is put is gated round into the input. The sequence of
gated is a constant of 4; that is, the higher bits on 94 of operation is as follows: MPL1 (the signal which con
a rotation onto the bus. This is controlled by signal C1 trols the gating of the multiplier input latch) and that
which is one of the read enables; that is, REO through gates MP load true which causes the load enable of the
2 go into a demultiplexer which will produce one of the 60 counter to go true and then on the next master clock,
signals C1. This is a constant which is added to the the counter will get loaded with a +1 and MPGO will be
angle of the sine to get a cosine. It is essentially 3/2 pi. brought ture which will cause MP load to become false;
On the left hand side of the bus are gates 68 that then once MP GO is true, then each MP step is cycling
control operations. For instance, it is desired in a the multiplier. With the multiplier itself the serial out
steady state condition to gate 0 on the low order bits 65 put of the 16 bit input latch 77 and shift register 73 is
8-15 of the bus on the condition that the initialization called MC enable (multiplicand enable) and is coupled
memory, the IM signal, is being read from and the into a set of gates 78 (FIG. 16) which will gate the
constant C1 from the envelope is present. It is the OR multiplier into the adders 79 and which adds the partial
4,018,121 16
15
product to the multiplier. At each step the MC enable The device is also busy doing other things during that
or multiplicand enable will either gate the multiplicand time. It is slightly pipelined; that is, the multiplier may
into the adders or not. So all 15 bits of the multiplier be loaded up and while computing other things like
are stepped through. The partial product is latched by update amplitude or modulating index may be accom
latch 81 into a 16 bit register on the output. This is not 5 plished.
shown on the block diagram but is internal to the 16 bit FIG. 17 is the output register and it is gated on the
multiplier block. An output controlled by MP step. The signal "load out buf" which is produced by the latch
output of latch 84, the signals MP 0 through MP 15, is select bits LS0 through 1 from the instruction word;
the partial product which becomes the product after this grabs 16 bits off of the bus and latches them. The
the last multiplier step. The shifting right of the partial 10 program goes through computing what the output
product is accomplished in FIG. 16 simply by the or waveform should be and then at the very last step it
dering of the bits as they go back into the adder; that is, gates the waveform onto the data bus and strobes "load
the bits are shifted right by one. In the first adder 79 out buf' which causes the output word to be sent to the
there is the sign bit twice MP0 as the first two bits. That D to A converter 22 (FIG. 5).
is, an arithmetic two complement shift right where 15 FIG. 18 is a sine table. It consists of a 256 - 16 bit
copy the sign bit is copied into the vacated places; that word read only memory. The output of the read only
is, the low order bit is discarded on every cycle because memory is called ST0 through 15 and goes back into
a multiply will eventually end up with a 31 bit resultant the input selector for the scratchpad memory. The
and all that is necessary is 16 bits. The multiplier is address for the sine table comes out of the adder output
stopped by the counter 74 overflowing and this pro 20 AO0 through 7; that is, only the high order 8 bits are
duces MP clear which turns off MP 60 and the multi looked at and latched on the signal ST which is again
plier halts. The product is available at the multiplier generated from the latch select bits from the instruc
output (latch 81) as signals MP0 through 15. These tion word LS0 through 2. When ST occurs, the adder
multiplier output signals will remain true, of course, output is latched and produces internal signals which
until the next time multiplication is begun. The total 25 are SA0 through 7; that is, the sine address or sine
multiplication takes 15 clock steps. A clock step is angle. With these memories, it can take as long as a
about 200 nanoseconds so the whole multiply takes 3 microsecond to get the data out so one microsecond
microseconds or roughly five instruction cycles. Thus later or roughly two instruction cycles, the sine is avail
on the fifth instruction cycle after the multiplication is able for use.
initialized, the output can be used. 30 The running program for the foregoing is shown be
low.
In the above running program the left hand column the writing back into the scratchpad memory there is
has the actual program counter number in octal form. either AO for adder output, MO for multiplier output,
The second column is a shorthand notation for describ ST for sine table and FT for frequency table. This is,
ing where data is flowing. In general, the adder latches only in initialization, of course.
are AL1 and AL3, multiplier latches are ML1 and 65 The program of TABLE I is repeatedly processed by
ML2, the sine table latches ST, the envelope memory is the apparatus of FIG. 5. In so doing, the apparatus of
ENV. For the state of bits Isel and Ssel they are ENV FIG. 5 iteratively performs calculations which are an
(00, 01, 10, 11) to reflect the four possible states. For approximation of Equation (2). The instructions of
4,018, 121
19 20
TABLE I are stored in the instruction memory 41 of ditions, the term cos(lob: cont) is approximately equal
FIG. 5. The instructions are accessed in order, from to unity and the term sin(lob: cont) is approximately
Instruction 0 to Instruction 77, to complete one pass equal to (lob:aint) itself. Using those approximations,
through TABLE I. After each pass through TABLE I, the value of the sin(out) is given by the following
Instructions 0 through 77 are again accessed to com Equation (5). :
plete a new pass through TABLE I.
The final calculation for each pass through TABLE I sin(cut) = sin(hob: alt) -- (lob: alt)cos(hob: ,
abat) (5)
is set up by latching quantities into the input latches of
multiplier 31. Those quantities are latched during In
structions 74 and 75. The product of those quantities, 10 sineByterms
using Equation (5), the accuracy with which the
latched in the multiplier during Instructions 74 and 75, 5 had 4,000arelocations
calculated is as if sine memory 34 in FIG.
rather than just 256 locations. Of
becomes available on the multiplier 31 output (MO) course, the interpolation in accordance with Equation
approximately five instruction cycles later. Five in (5) can be avoided, merely
struction cycles later, for any given pass through the memory 34 or by acceptingbylessemploying a larger sine
TABLE I instructions, actually occurs in the next pass 15 an attendant deterioration in quality of theresults
accurate
sound
with
pro
through TABLE I. In the next pass through TABLE I, duced. . .
Instruction 4 transfers the output from multiplier 31 to In the TABLE evaluation of Equation (2), the mod
latches 33. . .
For each pass through TABLEl, a new value is gated ulation indexes I1(t) and I2(t) are divided by a con
into the output latches 33 in FIG. 5. The digital-to- 20 stant 8 to form modulation indexes I1 and I2, respec
analog converter 22 converts the data stored in latches tively. In order to restore the modulation indexes to
33 to an analog signal which in turn is converted to a their full values, the modulation indexes I1 and I2 in
musical sound in speaker 23. TABLE I are each multiplied by a scaling factor which
cancels the factor of 8 division. The scaling factors for
The TABLE
utilizing I instructions
an interpolation evaluate
technique Equation (2)
for evaluating the 5 the modulation indexes are selected as 8 times (Aalt)
sine terms in Equation (2). Also the instructions of for 1 and as 8 times (Acont) for 12.
TABLE employ scaling factors for the modulation In the evaluation of Equation (2), the indexes I1(t)
indexes. The interpolation technique, the scaling fac and 20t) are given by Equations (6) and (7).
tors and the equations actually iterated by the TABLE 3o 1(t) = (1) (8Aot) (6)
I instructions will now be described. .
Equation (2) includes three sine terms. In order to I2(r) = (I2) (8Aa) . . (7)
accurately evaluate the sine terms with a comparatively
small sine table (256-word sine memory 34 of FIG. 5), The program of TABLE I treats the amplitude A of
two of the three sine values in Equation (2) are evalu 35 Equation (2) as a function of time so that A becomes
ated using an interpolation technique. Evaluation of A( t). -.
the third sine term (sinot), however, does not employ . . Using A(t) and the modulation indexes of Equations
the interpolation technique when this term represents (6) and (7), Equation (2) becomes Equation (8) as
the grit function for producing inharmonic partials follows. ..
35
40
45
50
55
60
65