Video summary
The video explores the fascinating concept of creating sound from nothing, moving beyond traditional instruments like acoustic or electric guitars that rely on physical vibration. Instead, it focuses on synthesizers which generate voltage directly to produce audio waves. The presentation begins by explaining the fundamental building block of all sound: the sine wave, a pure tone at a single frequency. It demonstrates how complex sounds, such as chords or sawtooth waves, can be constructed by summing multiple sine waves together. This leads to an introduction of two primary synthesis methods: subtractive and FM (Frequency Modulation). Subtractive synthesis starts with a rich, complex waveform generated by oscillators and then uses filters to "carve out" unwanted frequencies, shaping the sound over time using envelopes that control volume and filter cutoffs dynamically.
The discussion then shifts to the history and mechanics of Frequency Modulation, tracing its roots back to John Ching's experiments in 1967 where he modulated one sine wave with another. This technique allows for the creation of incredibly complex timbres by having a modulator change the frequency of a carrier wave. The video highlights the Yamaha DX7 as a landmark instrument that utilized FM synthesis, capable of producing realistic piano sounds and other textures through a vast space of possible sound combinations known as algorithms. However, programming these sounds manually was extremely difficult due to the sheer number of variables, leading manufacturers to rely on presets discovered by engineers who navigated this complex sonic landscape.
To make such powerful synthesizers affordable and practical, the video delves into the sophisticated engineering tricks required to build the hardware chips inside instruments like the DX7. Since calculating sine waves in real-time was too computationally expensive for processors of that era, engineers used lookup tables and mathematical shortcuts like working in logarithmic space to turn multiplication into addition. Furthermore, they employed advanced data compression techniques such as delta encoding and planar organization within the chip's memory. By storing only the differences between values and organizing data to maximize rows of zeros, they significantly reduced the amount of physical silicon needed, shrinking the required storage size by over ten times while maintaining high performance.
In conclusion, the video summarizes the three main approaches to sound generation: additive synthesis using infinite sine waves, subtractive synthesis which filters rich waveforms, and FM synthesis where oscillators modulate each other to create complex textures. The speaker demonstrates these concepts with audio examples, including a recreation of New Order's "Blue Monday" on a Minimoog and attempts at electronic piano sounds on both a Juno 60 and the DX7. The presentation ends by inviting viewers to explore these synthesizers further, emphasizing how mathematical ingenuity and hardware optimization allowed musicians to create virtually any sound imaginable without relying on physical acoustic bodies.
Read the full video transcript
So every instrument vibrates. There's
some physical motion that causes the
sound to happen. In an acoustic guitar,
there's a sound hole and a resonating
body and the when you strum the string,
it shakes and then that sound that
vibration gets amplified into a sound
that you end up hearing. So consider an
electric guitar. Same thing but no sound
hole amplifier this time. Instead we
have a pickup which turns the uh some
motion into a voltage.
But what if I told you there is no
guitar. In fact you can just create
sound from nothing. You could just make
a voltage. So whatever circuit you can
build, you can have it make some voltage
and turn that voltage into sound. Same
way the electric guitar works just
without the guitar part.
So welcome to Sonis X Nihilo sound from
nothing.
So how do we make sound from nothing?
So it starts with a sine wave. So sine
wave is one of the simplest types of
sounds. There are two ways to look at
this. You'll see in the video. So we'll
have on the top we'll have a time domain
view they call it and on the bottom a
frequency domain. So a sign is at a a
pure tone at a single frequency. So
you'll see a single spike on the bottom
and then on the top you'll see a
familiar wavy shape.
All right. So, who here has heard of
Forier analysis? Pretty much everybody.
Wow. Awesome. Excellent. EMF. Yeah, way
to go. All right. So, uh, in case you
don't know basics, uh, very handwavy
explanation. Um, any complex waveform
can be turned into a sum of sine waves.
So for example we can make a chord
playing just sign.
[music]
So that's as if sign were your
instrument. But you can make any
complicated waveform you like. So we'll
try to make a saw to wave. We don't have
an infinite number of signs. So, we'll
be limited in how well we can reproduce
the actual sawtooth, but uh we'll we'll
give it a try. So, we'll have 32.
[music]
So, we'll we'll need a few more signs to
make a sawtooth. Exactly.
uh so that's not practical to have uh an
infinite number of oscillators in your
synthesizer if you're trying to build
one. So there are two shortcuts we can
take two two ways to design a
synthesizer and that's subtractive and
FM.
So subtractive synthesis,
there are three steps basically. You
start with a really rich complicated
waveform and then you filter out the
parts that you don't want just as if
you're uh carving or sculpting
and then after that you shape that sound
over time and it you can play something
that sounds more musical. So one of the
first synthesizers very popular to
follow this technique is called the
Minimog. Um so Minimog was a
commercialization of a previous
synthesizer that explored how to do
subtractive synthesis. It's one of the
first modular synthesizers. Um and is
quite expensive in today's money but
still cheaper than an infinite number of
sine wave oscillators.
So here's the architecture how it works.
You have basically uh three oscillators
that can play different waveforms. You
run those into a mixer to mix them all
together and then you have the Moog
ladder filter. Uh this filter is um a
low pass filter. So it cuts out things
above the cutff, but it also has a
feature called resonance where you can
emphasize uh frequencies near that cutff
point. You take the output of the
filter, run it through an amplifier. The
amplifier can also change the uh volume
of a note over time. And then that is
the sound you hear at the end.
So the raw material we have to work with
the what the minimog oscillators can do.
We'll explore each of those sounds here.
So we'll have a saw, triangle, square,
and then some noise.
[music]
Okay. But if that's not complex enough,
you can um do a few different tricks on
this synthesizer to make it even more
thick and rich. So, we'll start with a
sawtooth. Then we will add another one,
an octave up. You'll see what that
sounds like. And then we'll add a third
a little bit D-tuned.
[music]
Now for the filter. So I've explained
the resonance feature already and that
it's a cutff. The cutff is adjustable
and you can tweak a knob on the front
panel to control where that's at.
Um, so to show you what resonance sounds
like, we'll crank up that resonance dial
all the way to almost the top. Um, so
that'll turn the sine wave or sorry, the
sawtooth that we're outputting into
basically a sine wave.
[music]
Okay. And so the amplifier can also be
controlled with an envelope. So every
time you hit a key on the keyboard, you
can have it play over time different
levels of loudness.
Same thing you can do with the filter,
which we'll get to next, but I'll show
you what the amplitude envelope sounds
like u by trying to make a plucked kind
of a plucked plucked bass sound.
>> [music]
>> There's no envelope. [music]
Then with the envelope, it's plucked.
Okay. And with the filter, you can uh
simulate brassy type sounds. So if you
start with a little bit slower attack,
so a little bit slower um let's say
opening up the filter over time and then
setting the cut back uh cut off
frequency back down as the notes played
over time. So this is not loudness. The
loudness is the same level. This is just
opening and closing the filter.
>> Ha.
>> [music]
[music]
>> Um, another thing you can do with the
oscillator 3 is set it into what's
called a low frequency oscillator mode.
So, this is low enough that we can't
hear it as as humans. It's around like
10 herz, 8 herz, something like this.
And you can use that oscillator's output
to control other things in the
synthesizer. So, for example, you can
control the pitch of another oscillator
or you can control the filter itself.
Um, and if you put the filter into
almost resonance and have noise go into
it, you can make some nice wind sounds.
>> [music]
>> You can see the filter open and close on
the bottom there.
All right. So, putting all that
together, this is uh what is this? March
1983.
You can um Oh, Blue Monday released this
a new order released Blue Monday. And
this is the base riff from that on a
mini mug. [music]
All right. So, that's subtractive
synthesis.
Now, enter the Yamaha DX7. So, converted
to today's money, this is less than a
third the cost of the mini moog.
And this was only a few months uh after
New Orders Blue Monday came out.
Completely different sound.
[music]
Not many knobs or controls on this
synthesizer. quite hard to program um
but very expressive.
So where did FM come from? So we'll have
to go back in time to 1967.
Consider John Ching. He was a composer
researcher working at uh sale at
Stanford, their AI lab. he was playing
around on uh one of their old uh not at
the time old but PDP6 and music 5
program. So he would program uh the
computer, hey make me a sound. I want
two signs. I want you to control one of
the signs with the other. He'd submit
the job and wait many many minutes to
see what it came back as.
So we're going to do the same exact
thing. So we have a two signs. One is
doing modulation of a carrier. The
carrier is the note you'll hear. The
modulator adds the VA to it.
>> [music]
[music]
>> So this was the same 200 htz carrier the
whole time. So the same note. All we
were doing was changing how fast VO
goes. Once verbat got fast enough like
up into audio range we um hear it
differently. We hear it as tambber
instead of a wavy ver.
This is the mask if you're interested.
Um if you're not, plug your ears. All
right. So we have a sign inside of that
another another sign. That's how the
modulation is working. So the uh the
second sign on the inside, you have a
frequency in there. That's the the
vibrto rate, the purple number. And then
you have this plugged into the other
sign causing it to move the wave back
and forth which would be the VA. The
carrier I talked about the 200 Hz from
before is the green. And uh index is how
far you're allowed to move the FM bands
away like how far the sign will affect
the other one.
And uh a nice way to measure that is as
a portion of the modulation frequency.
Okay. So John Chin discovered a
completely new soundsscape. All kinds of
sounds you can you can make with FM.
U let's take a tour. So first we'll do
his experiment again but instead of
steps where we had to wait minutes
between each step, we'll do it smooth
and then we'll go in the other
dimension. So instead of moving
frequency, we'll move the modulation
depth. And then we'll do both at once.
[music]
Pretty crazy. Most of that sounds bad,
but there there's a few nice points in
there. You might have um heard them if
you're listening carefully.
[snorts]
All right. So, the basic building block
of FM synthesis is called the operator.
This is a combination of the oscillator
and envelope. So, you can change the
modulation over time as you play a note.
You can wire them up uh in what's called
a algorithm. The algorithm is the
mapping of how operators are connected
together.
And when you add algorithms into the mix
here, you have crazy dimensional space
of all kinds of sounds. So, let's um
pick a few points that sound nice and
hear what those are like.
[music]
>> [music and bell]
>> Yeah. So, those were those were in what
we heard before almost. You might have
missed them, but those are quite nice
sounds. Those were some of the presets
that are shipped on the Yamaha DX7. So,
the Yamaha engineers discovered these
points in that hyper space somehow and
uh after they discover them, program
them in. So, everyone uses these
presets. As I mentioned, it's quite hard
to program this thing. Finding a good
sound in that huge space is difficult as
well. So, most people just use these
presets.
All right. So, what's it take to build a
DX7? We will need uh let's say we want
to play more than one note at a time.
So, the mini moog was monophonic,
meaning one note at a time. We want to
do multiple. So, we'll do uh 16 cuz we
have lots of fingers. Okay. So if we
have 16 and uh each one of those we want
to make up of of six different FM
operators, we'll need 96 operators
total.
And to run those operators, what they do
is they do a sign calculation and they
do an envelope calculation
inside each one. So that's a lot of uh
multiplications per second to to apply
all of these operations. Just for
comparison, um we have Commodore 64 do
about 5,000 per second.
IBM 35,000. The Intel PC like 10 years
into the future still only 4 million.
The DX7 has to do 19 million of these.
So the the tricks to to build this to
make it happen is instead of computing
sign, you use a lookup table for one. So
this first trick.
So and um one lookup table can make any
frequency of sign. How you make
different frequencies is by jumping
around in different step widths.
You can also take advantage of the
quarter wave symmetry of the sine wave.
So you don't have to store one full
period of sign. You just have to store a
quarter of that and flip it around and
mirror it. And then you can have uh one
quarter reduction of the space.
Another trick, you don't have to
multiply if you can add. Uh if we use a
trick from slide rules where you take
the um work in log space. So you can
multiply numbers together just by
adding. So if you have log of two, log
of three, you can make log of six.
Now if you want to get the six back out,
you'll have to do exp, which is fine.
That's another table, right?
Or you don't have to have it another
table. You could bake it in. So could be
uh sorry exp is still separate table but
um to go to log space you don't have to
do another table lookup. You can put
that inside pre-baked into your table.
So you don't have a sign lookup table
anymore. You have a log of sine of x
table.
Um, and so all of that so far was either
like obvious, other people knew about
it, or was disclosed in Yamaha's patents
for a DX7.
But, uh, Ken Sheriff decapped the chip
that's at the heart of the DX7 and did
some very good reverse engineering work.
And I'll share with you what I thought
was interesting from from his work.
So, this is the chip.
opened up
put under the microscope
uh metal layer stripped off
and zoomed in on this log sign ROM some
interesting patterns here which I can't
read but Ken shifted and explained it
very well so top recommend his uh blog
post on the topic
all right so first thing uh they use a
trick called the delta encoding. So this
is the wave table or sorry the log sign
table on the right there. Even at the
steepest portion if you put the numbers
out each entry in that table they're
close together. So you you say we have
spatial locality here. Whenever you have
spatial locality in your function you're
trying to build a lookup table for, you
can benefit from delta encoding.
So how this works is you store every
fourth value as an absolute reference
point. So this is the full value, but
then you only have to store uh tiny
deltas which you add back on as needed
later. and storing those deltas because
you are leveraging spatial locality
because they're close by don't need as
many bits as storing the full value
every time.
So for example you have uh the value
7722
at the fourth entry you can store the
entries before it 012
with uh a little delta. So each one of
those has a different value different
delta.
Everything at the [sighs]
multiple of four position at that starts
at n equals 0 will be a delta 0. So if
we go to n equals 4 that would be added
as a delta 0. So it's like a mod 4
operation to get the which delta family
it's it's belonging to.
Uh the next trick is pl planer planer
organization. So you put you Yeah. All
right. So you put everything together
into a group. So all of the delta zeros
put them into one big delta zero array.
All of the delta 1's into one big delta
1 array. All of the delta twos into one
big delta 2 array. So forth. Once
they're together, you can take a stack
of 256 of these, lay it out as 8 by32,
and that looks like this blue square on
the right, and then a single line of
that, just to show you, you see the ones
and zeros, these are actual bits
finally.
So, the reason you'd organize it this
way is you end up with lots of zeros in
rows.
And when you're building a hardware ASIC
device, um, zeros are free. You can
leave them out. You don't need to put a
transistor down unless it needs to go
somewhere other than default. So you
take advantage of patterns of the
address lines, how things are u the
logic used to access the values from the
chip. If it would always return zero for
a class of values, you can not put
transistors there. You don't need space
in the chip. So this gave the biggest
savings.
So we went from around uh almost 60,000
bits just storing a log sign table at
full take a quarter wave um do the delta
encoding and zeroing and you end up
around 11 times smaller than than what
you started.
Okay.
Do you recognize this song?
[music]
Sound familiar? Anybody? [music]
Show of hands.
No.
Okay, good. Good. I I made up the song.
Um, but I I use the most common preset
on the DX7, which is electronic piano
number one. So, if it sounded familiar,
it's because of the sound. Um, anyway, I
didn't get any of you. All right. Worth
a try.
All right. So, in summary, we've covered
three different ways to make sounds out
of nothing. The additive synthesis,
where we have infinite number of sine
waves. We've covered subtractive where
you build a rich sound and then carve
out the bits you don't want. And then
we've also talked about FM where things
did ddle each other and it gets complex.
All right. So I will leave you with uh
two best effort pianos. So, one on a
Juno 60 synthesizer, which is using
subtractive, same as a Mini Mog, and
then the other uh DX7 trying to be a a
grand piano. Grand piano
[music]
>> [music]
>> All right. So, if you if you'd like to
play around with any of the synthesizer
stuff with me, I'll be hanging around
Trash Compactor and uh hope to catch you
there and goof around with audio. Thank
you. [applause]