Monday, July 13, 2015

Think DSP at SciPy

Last week I was at SciPy 2015 in Austin, where I presented a short talk about Think DSP and the class based on it: "Basic Sound Processing in Python".  The slides for the talk are here.  The SciPy organizers did an amazing job of editing and posting the videos, usually in less than 24 hours.  The video from my talk is here:



And the IPython notebook I presented is here.

Mark Wickert was also at SciPy to present a related talk on "Signal Processing and Communications Teaching and Research Using IPython Notebook".  His talk is here:



One other update: I sent a proposal to my friends at O'Reilly Media to see if they would like to publish Think DSP.  They have responded favorably.  We are working on a contract and a schedule, but I am aiming to be done in February 2016, with publication in March or April.

Monday, February 9, 2015

February is DSP month!

DSP month continues...

Last month I unilaterally declared that January is DSP month, and posted a series of excerpts from the book I am working on, Think DSP.  Here is the series so far:

Chapter 1: Signals and Spectrums
Chapter 2: Harmonics
Chapter 3: Chirps
Chapter 4: Noise

Chapter 5: Autocorrelation

I didn't get through all of the chapters in January, so today I am happy to announce that February is also DSP month, according to me.

To celebrate, I am posting excepts from my favorite chapter: Chapter 6, the Discrete Cosine Transform.  I present the discrete cosine transform (DCT) before the discrete Fourier transform for two reasons:

1) The DCT has many practical applications; notably, it is used by MP3 and related formats for compressing music, JPEG and similar formats for images, and the MPEG family of formats for video.

2) I found a way to present both DCT and DFT that (I think) demystifies the idea of a transform; and starting with the DCT has the advantage that everything is real-valued (no complex numbers).

My explanation follows these steps:

  1. I start with the synthesis problem: given a set of frequency components and their amplitudes, how can we construct a waveform?
  2. Next I rewrite the synthesis problem using NumPy arrays. This move is good for performance, and also provides insight for the next step.
  3. We look at the analysis problem: given a signal and a set of frequencies, how can we find the amplitude of each frequency component? I start with a solution that is conceptually simple but slow.
  4. Finally, I use some principles from linear algebra to find a more efficient algorithm.

Synthesis #1

Suppose I give you a list of amplitudes and a list of frequencies, and ask you to construct a signal that is the sum of these frequency components. Using objects in the thinkdsp module, there is a simple way to perform this operation, which is called synthesis:
def synthesize1(amps, freqs, ts):
    components = [thinkdsp.CosSignal(freq, amp)
                  for amp, freq in zip(amps, freqs)]
    signal = thinkdsp.SumSignal(*components)

    ys = signal.evaluate(ts)
    return ys

amps is a list of amplitudes, freqs is the list of frequencies, and ts is the sequence of times where the signal should be evaluated.
components is a list of CosSignal objects, one for each amplitude-frequency pair. SumSignalrepresents the sum of these frequency components.
Finally, evaluate computes the value of the signal at each time in ts.
We can test this function like this:
    amps = numpy.array([0.6, 0.25, 0.1, 0.05])
    freqs = [100, 200, 300, 400]
    framerate = 11025

    ts = numpy.linspace(0, 1, framerate)
    ys = synthesize1(amps, freqs, ts)
    wave = thinkdsp.Wave(ys, framerate)
    wave.play()

This example makes a signal that contains a fundamental frequency at 100 Hz and three harmonics (100 Hz is a sharp G2). It renders the signal for one second at 11,025 frames per second and plays the resulting wave.
Conceptually, synthesis is pretty simple. But in this form it doesn’t help much with analysis, which is the inverse problem: given the wave, how do we identify the frequency components and their amplitudes?

Synthesis #2

Here’s another way to write synthesize:

def synthesize2(amps, freqs, ts):
    args = numpy.outer(ts, freqs)
    M = numpy.cos(PI2 * args)
    ys = numpy.dot(M, amps)
    return ys

This function looks very different, but it does the same thing. Let’s see how it works:
  1. numpy.outer computes the outer product of ts and freqs. The result is an array with one row for each element of ts and one column for each element of freqs. Each element in the array is the product of a frequency and a time, f t.
  2. We multiply args by 2 π and apply cos, so each element of the result is cos(2 π f t). Since the ts run down the columns, each column contains a cosine signal at a particular frequency, evaluated at a sequence of times.
  3. numpy.dot multiplies each row of M by amps, element-wise, and then adds up the products. In terms of linear algebra, we are multiplying a matrix, M, by a vector, amps.
The following figure shows the structure of this computation.


 M
 
⎡
⎢
⎢
⎢
⎣
0.6 
0.25 
0.1 
0.05 
⎤
⎥
⎥
⎥
⎦
= amps
 
 . 
 . 
 tk 
 . 
 . 
⎡
⎢
⎢
⎢
⎢
⎢
⎣
.... 
.... 
abcd 
.... 
.... 
⎤
⎥
⎥
⎥
⎥
⎥
⎦
⎡
⎢
⎢
⎢
⎢
⎢
⎣
 . 
 . 
 e 
 . 
 . 
⎤
⎥
⎥
⎥
⎥
⎥
⎦
= ys
 
 
.fj.. 


Each row of the matrix, M, corresponds to a time from 0.0 to 1.0 seconds; tk is the time of the kth row. Each column corresponds to a frequency from 100 to 400 Hz; fj is the frequency of the jth column.
I labeled the kth row with the letters a through d; as an example, the value of a is cos[2 π (220)tk].
The result of the dot product, ys, is a vector with one element for each row of M. The kth element, labeled e, is the sum of products:
e = 0.6 a + 0.25 b + 0.1 c + 0.05 d 

And likewise with the other elements of ys. So each element of y is the sum of four frequency components, evaluated at a point in time, and multiplied by the corresponding amplitudes. And that’s exactly what we wanted. We can use the code from the previous section to check that the two versions of synthesize produce the same results.

    ys1 = synthesize1(amps, freqs, ts)
    ys2 = synthesize2(amps, freqs, ts)
    print max(abs(ys1 - ys2))

The biggest difference between ys1 and y2 is about 1e-13, which is what we expect due to floating-point errors.
One note on linear algebra vocabulary: I am using “matrix” to refer to a two-dimensional array, and “vector” for a one-dimensional array or sequence. To be pedantic, it would be more correct to think of matrices and vectors as abstract mathematical concepts, and NumPy arrays as one way (and not the only way) to represent them.
One advantage of linear algebra is that it provides concise notation for operations on matrices and vectors. For example, we could write synthesize like this:
M=cos(2 π t ⊗ f) 
y=M a
where a is a vector of amplitudes, t is a vector of times, f is a vector of frequencies, and ⊗ is the symbol for the outer product of two vectors.


Analysis #1

Now we are ready to solve the analysis problem. Suppose I give you a wave and tell you that it is the sum of cosines with a given set of frequencies. How would you find the amplitude for each frequency component? In other words, given ys, ts and freqs, can you recover amps?
In terms of linear algebra, the first step is the same as for synthesis: we compute M = cos(2 π t⊗ f). Then we want to find a so that y = M a; in other words, we want to solve a linear system. NumPy provides linalg.solve, which does exactly that.
Here’s what the code looks like:
def analyze1(ys, freqs, ts):
    args = numpy.outer(ts, freqs)
    M = numpy.cos(PI2 * args)
    amps = numpy.linalg.solve(M, ys)
    return amps

The first two lines use ts and freqs to build the matrix, M. Then numpy.linalg.solve computes amps.
But there’s a hitch. In general we can only solve a system of linear equations if the matrix is square; that is, the number of equations (rows) is the same as the number of unknowns (columns).
In this example, we have only 4 frequencies, but we evaluated the signal at 11,025 times. So we have many more equations than unknowns.
In general if ys contains more than 4 elements, it is unlikely that we can analyze it using only 4 frequencies.
But in this case, we know that the ys were actually generated by adding only 4 frequency components, so we can use any 4 values from the wave array to recover amps.
For simplicity, I’ll use the first 4 samples from the signal. Using the values of ys, freqs and tsfrom the previous section, we can run analyze1 like this:
    n = len(freqs)
    amps2 = analyze1(ys[:n], freqs, ts[:n])
    print(amps2)
And sure enough, we get
[ 0.6   0.25  0.1   0.05 ]
This algorithm works, but it is slow. Solving a linear system of equations takes time proportional to n3, where n is the number of columns in M. We can do better.

Orthogonal matrices

One way to solve linear systems is by inverting matrices. The inverse of a matrix M is written M−1, and it has the property that M−1M = I. I is the identity matrix, which has the value 1 on all diagonal elements and 0 everywhere else.
So, to solve the equation y = Ma, we can multiply both sides by M−1, which yields:
M−1y = M−1 M a 
On the right side, we can replace M−1M with I:
M−1y = I a 
If we multiply I by any vector a, the result is a, so
M−1y = a 
This implies that if we can compute M−1 efficiently, we can find a with a simple matrix multiplication (using numpy.dot). That takes time proportional to n2, which is better than n3.
In general inverting a matrix is slow, but some special cases are faster. In particular, if M is orthogonal, the inverse of M is just the transpose of M, written MT. In NumPy transposing an array is a constant-time operation. It doesn’t actually move the elements of the array; instead, it creates a “view” that changes the way the elements are accessed.
Again, a matrix is orthogonal if its transpose is also its inverse; that is, MT = M−1. That implies that MTM = I, which means we can check whether a matrix is orthogonal by computing MTM.
So let’s see what the matrix looks like in synthesize2. In the previous example, M has 11,025 rows, so it might be a good idea to work with a smaller example:
def test1():
    amps = numpy.array([0.6, 0.25, 0.1, 0.05])
    N = 4.0
    time_unit = 0.001
    ts = numpy.arange(N) / N * time_unit
    max_freq = N / time_unit / 2
    freqs = numpy.arange(N) / N * max_freq
    ys = synthesize2(amps, freqs, ts)

amps is the same vector of amplitudes we saw before. Since we have 4 frequency components, we’ll sample the signal at 4 points in time. That way, M is square.
ts is a vector of equally spaced sample times in the range from 0 to 1 time unit. I chose the time unit to be 1 millisecond, but it is an arbitrary choice, and we will see in a minute that it drops out of the computation anyway.
Since the frame rate is N samples per time unit, the Nyquist frequency is N / time_unit / 2, which is 2000 Hz in this example. So freqs is a vector of equally spaced frequencies between 0 and 2000 Hz.
With these values of ts and freqs, the matrix, M, is:
[[ 1.     1.     1.     1.   ]
 [ 1.     0.707  0.    -0.707]
 [ 1.     0.    -1.    -0.   ]
 [ 1.    -0.707 -0.     0.707]]

You might recognize 0.707 as an approximation of √2/2, which is cosπ/4. You also might notice that this matrix is symmetric, which means that the element at (j, k) always equals the element at (k, j). This implies that M is its own transpose; that is, MT = M.
But sadly, M is not orthogonal. If we compute MTM, we get:
[[ 4.  1. -0.  1.]
 [ 1.  2.  1. -0.]
 [-0.  1.  2.  1.]
 [ 1. -0.  1.  2.]]

And that’s not the identity matrix.

Making M orthogonal

But if we choose ts and freqs carefully, we can make M orthogonal. There are several ways to do it, which is why there are several versions of the discrete cosine transform (DCT).
One simple option is to shift ts and freqs by a half unit. This version is called DCT-IV, where “IV” is a roman numeral indicating that this is the fourth of eight versions of the DCT.
Here’s an updated version of test1:
def test2():
    amps = numpy.array([0.6, 0.25, 0.1, 0.05])
    N = 4.0
    ts = (0.5 + numpy.arange(N)) / N
    freqs = (0.5 + numpy.arange(N)) / 2
    ys = synthesize2(amps, freqs, ts)

If you compare this to the previous version, you’ll notice two changes. First, I added 0.5 to tsand freqs. Second, I cancelled out time_units, which simplifies the expression for freqs.
With these values, M is
[[ 0.981  0.831  0.556  0.195]
 [ 0.831 -0.195 -0.981 -0.556]
 [ 0.556 -0.981  0.195  0.831]
 [ 0.195 -0.556  0.831 -0.981]]

And MTM is
[[ 2.  0.  0.  0.]
 [ 0.  2. -0.  0.]
 [ 0. -0.  2. -0.]
 [ 0.  0. -0.  2.]]

Some of the off-diagonal elements are displayed as -0, which means that the floating-point representation is a small negative number. So this matrix is very close to 2I, which means M is almost orthogonal; it’s just off by a factor of 2. And for our purposes, that’s good enough.
Because M is symmetric and (almost) orthogonal, the inverse of M is just M/2. Now we can write a more efficient version of analyze:
def analyze2(ys, freqs, ts):
    args = numpy.outer(ts, freqs)
    M = numpy.cos(PI2 * args)
    amps = numpy.dot(M, ys) / 2
    return amps

Instead of using numpy.linalg.solve, we just multiply by M/2.
Combining test2 and analyze2, we can write an implementation of DCT-IV:
def dct_iv(ys):
    N = len(ys)
    ts = (0.5 + numpy.arange(N)) / N
    freqs = (0.5 + numpy.arange(N)) / 2
    args = numpy.outer(ts, freqs)
    M = numpy.cos(PI2 * args)
    amps = numpy.dot(M, ys) / 2
    return amps

Again, ys is the wave array. We don’t have to pass ts and freqs as parameters; dct_iv can figure them out based on N, the length of ys.
We can test dct_iv like this
    amps = numpy.array([0.6, 0.25, 0.1, 0.05])
    N = 4.0
    ts = (0.5 + numpy.arange(N)) / N
    freqs = (0.5 + numpy.arange(N)) / 2
    ys = synthesize2(amps, freqs, ts)

    amps2 = dct_iv(ys)
    print max(abs(amps - amps2))

Starting with amps, we synthesize a wave array, then use dct_iv to see if we can recover the amplitudes. The biggest difference between amps and amps2 is about 1e-16, which is what we expect due to floating-point errors.

Inverse DCT

Finally, notice that analyze2 and synthesize2 are almost identical. The only difference is that analyze2 divides the result by 2. We can use this insight to compute the inverse DCT:

def inverse_dct_iv(amps):
    return dct_iv(amps) * 2

inverse_dct_iv takes the vector of amplitudes and returns the wave array, ys. We can confirm that it works by starting with amps, applying inverse_dct_iv and dct_iv, and testing that we get back what we started with.

    amps = [0.6, 0.25, 0.1, 0.05]
    ys = inverse_dct_iv(amps)
    amps2 = dct_iv(ys)
    print max(abs(amps - amps2))

Again, the biggest difference is about 1e-16.

Summary

I hope this explanation helps to demystify the discrete cosine transform.  If you understand this way of thinking about it, you are 90% of the way to understanding the discrete Fourier transform, which is the topic for next time!

Friday, January 23, 2015

Cacophony for the whole family

Taking a break from my series of excerpts from Think DSP, I want to share an example I wrote to demonstrate some of the features in the thinkdsp library.  Last night I had the pleasure of listening to a grade school band, and during the breaks between sets, my mind wandered to the problem of synthesizing their unique sound.

With careful listening and consideration, I identified three important factors:

  1. Some of the instruments are out of tune by as much as a half step.
  2. At any point in the song, some fraction of the kids are not playing the written note, but choosing an alternative interpretation, plus or minus a major third.
  3. At any time, a kid might pop an overtone, playing a note at 2, 3, 4, or even 5 times the intended frequency (look out, David Sanborn!).
I implemented these effects with this function, which takes a MIDI number representing the written note and returns the chosen frequency:


def random_freq(midi_num):

    # simulate poor tuning by adding 
    # gaussian noise to the MIDI number
    midi_num += random.gauss(0, 0.5)
    
    # one kid out of 10 plays the wrong note
    if random.random() < 0.1:
        midi_num += random.randint(-5, 5)
        
    freq = midi_to_freq(midi_num)
    
    # and one kid in 10 pops an overtone
    if random.random() < 0.1:
        freq *= random.randint(2, 5)

    return freq

To hear what it all sounds like, check out this IPython notebook.  I hope you enjoy it as much as I enjoyed last night's performance!

Wednesday, January 21, 2015

Think DSP Chapter 5: Autocorrelation

It's time for Chapter 5: Autocorrelation!  If you don't think autocorrelation is interesting and fun, you are mistaken.  It turns out that autocorrelation, or something like it, explains the Missing Fundamental phenomenon I presented in last week's article.

Here are links to the previous installments:

Chapter 1: Signals and Spectrums
Chapter 2: Harmonics
Chapter 3: Chirps
Chapter 4: Noise

Autocorrelation

In Chapter 5, I explain autocorrelation by starting with the statistical definition of correlation, moving on to serial correlation, then generalizing to the autocorrelation function.

Signals often represent measurements of quantities that vary in time. For example, sound signals represent air pressure varying over time.
Measurements like this almost always have serial correlation, which is the correlation between each element and the next (or the previous). To compute serial correlation, we can shift a signal and then compute the correlation of the shifted version with the original.
def corrcoef(xs, ys):
    return numpy.corrcoef(xs, ys, ddof=0)[0, 1]

def serial_corr(wave, lag=1):
    n = len(wave)
    y1 = wave.ys[lag:]
    y2 = wave.ys[:n-lag]
    corr = numpy.corrcoef(y1, y2)[0, 1]
    return corr

corrcoef is a convenience function that simplifies the interface to numpy.corrcoef, which computes the correlation coefficient.
serial_corr takes a Wave object and lag, which is the integer number of places to shift the waves.  You can think of serial_corr as a function that maps from each value of lag to the corresponding correlation, and we can evaluate that function by looping through values of lag:
def autocorr(wave):
    lags = range(len(wave.ys)//2)
    corrs = [serial_corr(wave, lag) for lag in lags]
    return lags, corrs

autocorr takes a Wave object and returns the autocorrelation function as a pair of sequences: lags is a sequence of integers from 0 to half the length of the wave; corrs is the sequence of serial correlations for each lag.
The following figure shows autocorrelation functions for pink noise with three values of β (see Chapter 4: Noise). For low values of β, the signal is less correlated, and the autocorrelation function drops toward zero relatively quickly. For larger values, serial correlation is stronger and drops off more slowly. With  β=1.2, serial correlation is quite strong even for long lags; this phenomenon is called long-range dependence, because it indicates that each value in the signal depends on many preceding values.

With that introduction, you should check out this IPython notebook, which presents this example, then uses autocorrelation to estimate the fundamental frequency of a vocal recording.

The missing fundamental: found!

The previous installment included this notebook, which presents the phenomenon of the Missing Fundamental.  It turns out that autocorrelation sheds some light on this phenomenon.  You can read an explanation in this notebook.

Next week, for people who like the math, I derive the Discrete Cosine Transform.


Wednesday, January 14, 2015

Think DSP Chapter 4: Noise

In my ongoing celebration of DSP month, I am posting excerpts from Think DSP, my new introduction to Digital Signal Processing.  Here are links to the previous installments:

Chapter 1
Chapter 2
Chapter 3

Today's installment is Chapter 4: Noise.  Here's how it starts:

In English, “noise” means an unwanted or unpleasant sound. In the context of digital signal processing, it has two different senses:
  1. As in English, it can mean an unwanted signal of any kind. If two signals interfere with each other, each signal would consider the other to be noise.
  2. “Noise” also refers to a signal that contains components at many frequencies, so it lacks the harmonic structure of the periodic signals we saw in previous chapters.
This chapter is about the the second kind.

Chapter 4 took a while for me to get organized, because I was tempted to start with "white noise", until I realized that I could not explain what white noise is without getting myself tangled up.  It is easy to generate white noise, but not easy to explain why it is "white".

It made more sense (to me, at least) to start with uncorrelated, uniform (UU) noise and then explore the spectral structure.  What we discover is that UU noise has equal power at all frequencies, on average, and that's why it's called "white".

Here's the power spectrum of a segment of UU noise.
It looks like the same power at all frequencies, on average, but it's pretty noisy.  The picture is clearer if we compute the cumulative sum of power:
That smooths out the noise and makes the "whiteness" obvious.

The chapter also presents Brownian noise (aka "red noise") and pink noise, wrapping up this this figure, which shows the power spectrums for red, pink, and white noise on a log-log scale:


You can see the examples from Chapter 4 (and listen to them) in this IPython notebook.

And if you can't wait for Chapter 5, you might be interested in this notebook, which presents the phenomenon of the Missing Fundamental.

Monday, January 12, 2015

Think DSP Chapter 3: Chirps

In my ongoing celebration of DSP month, I am posting excerpts from Think DSP, my new introduction to Digital Signal Processing.  Here are links to the previous installments:

Chapter 1
Chapter 2

Today's installment is Chapter 3, of course.  But right now I am in the process of finishing Chapter 9, which is about signals and linear, time-invariant (LTI) systems.   I just finished one of the coolest examples in the book, and I can't resist jumping ahead a little.  So, before we get to Chapter 3, please check out this IPython notebook, and then come right back.

Chapter 3

Welcome back.  I hope you like the "violin in a firing range" example, and I hope it makes you want to stick around and see how it works.  Chapter 3 might not be quite as cool, but it has some good stuff, too, like chirps and the Eye of Sauron.  Here's the introduction:

"The signals we have worked with so far are periodic, which means that they repeat forever. It also means that the frequency components they contain do not change over time. In this chapter, we consider non-periodic signals, whose frequency components do change over time. In other words, pretty much all sound signals.

"We’ll start with a chirp, which is a signal with variable frequency. Here is a class that represents a chirp:

class Chirp(Signal):
    
    def __init__(self, start=440, end=880, amp=1.0):
        self.start = start
        self.end = end
        self.amp = amp

"start and end are the frequencies, in Hz, at the start and end of the chirp. amp is amplitude."

You can see (and hear) the examples from Chapter 3 in this IPython notebook.  It includes this figure:



Which is the spectrum of a chirp that sweeps from 220 Hz to 440 Hz.  The spectrum shows a "blur" of frequencies in that range, but because of leakage, it has a cool pattern across the top that looks a little bit like this:

Sauron eye barad dur

That's the Eye of Sauron at the top of Barad-dûr (a tower in the Lord of the Rings, in case you didn't know).

Chapter 3 also presents spectrograms, which are visualizations of the short-time Fourier transform (STFT).  A STFT breaks a wave up into short intervals and computes the Fourier transform of each interval.  The visualization shows time on the x-axis and frequency on the y-axis.  This example shows the spectrogram of a chirp that sweeps from 220 to 440 Hz:


This chapter also features one of my favorite exercises, the sawtooth chirp:

"Write a class called SawtoothChirp that extends Chirp and overrides evaluate to generate a sawtooth waveform with frequency that increases (or decreases) linearly.

"Draw a sketch of what you think the spectrogram of this signal looks like, and then plot it. The effect of aliasing should be visually apparent, and if you listen carefully, you can hear it."

This exercise has a punchline: if you implement the sawtooth chirp correctly, it should sound familiar, at least to viewers of a certain age.  Hint.

You can read the rest of Chapter 3 here.  In a few days I will post Chapter 4: Noise!


Wednesday, January 7, 2015

Think DSP Chapter 2: Harmonics

A few days ago I posted the first chapter from Think DSP, a new book taking a computational approach to Digital Signal Processing.  Over the next few weeks I'll post additional chapters and related blog articles.

Chapter 1 elicited some discussion on the discussion group Comp.DSP, where the focus of the conversation is the correct plural of "spectrum".  The most commonly used plural is "spectra", based on the sort-of-rule that words from Latin should use Latin plurals.  But this rule doesn't hold up very well to scrutiny, as in this video explaining why the plural of octopus would be octopodes.

So I decided to explore a radical version of an alternative rule, which is to pluralize English words using standard English endings, regardless of their etymological roots.  In Think DSP I use "spectrums";  I suspect that my former Latin teachers do not approve.

Moving on to less important matters, Chapter 2 is about the harmonic structure of certain periodic signals, and about aliasing.

Harmonics

Here's the spectrum of a triangle signal at 200 Hz.
The biggest peak is at 200 Hz, right where we expect it, but there are additional peaks at all the odd multiples of 200 Hz.  And the amplitude of those peaks is proportional to the inverse of frequency squared.

A square wave also contains odd harmonics, but they drop off more slowly, inversely proportional to frequency (not frequency squared).  A sawtooth wave contains both even and odd harmonics, and they are inversely proportional to frequency.

So that raises a question I posed in one of the exercises:  Is there a simple waveform that has even and odd harmonics that drop off like  1/ f 2?

Aliasing

The figure below shows a cosine signal at 4500 Hz sampled at 10,000 frames a second.
The figure above shows a cosine signal at 5500 Hz sampled at 10,000 frames a second.

If you compare the two figures, you'll notice the problem: at this framerate, the two signals are indistinguishable.  This result is an example of aliasing, where a signal at one frequency masquerades as another frequency.

Chapter 2 explains more about how all this works.  You can see the code and examples in

And if you want to read the rest of the book, the current draft is here:


Think DSP is a Free Book. It is available under the Creative Commons Attribution-NonCommercial 3.0 Unported License, which means that you are free to copy, distribute, and modify it, as long as you attribute the work and don't use it for commercial purposes.

In particular, I welcome comments and suggestions from readers.  You can add a comment to this blog, or send me email.  If I make a change based on your suggestion, I will add you to the book's contributor list (unless you ask me not to).

Finally, be sure to tell me what you think the plural of "spectrum" should be.  Maybe "spectropodes"?