How MP3 to MIDI Pitch Detection Works

YIN, a 266-sample lag window, and what happens at both edges of the range.

Short answer: this converter looks at 64 ms of audio at a time and asks one question - where does this signal repeat? The answer is found with YIN, searching lags from 8 to 266 samples at 16 kHz, which is 2000 Hz down to 60.15 Hz. I tested both edges with pure tones at the default sensitivity: 55-2170 Hz came back at the right pitch, 54 Hz and below produced no notes at all, and 2180 Hz and above came back one octave low instead of being thrown away.

Everything below is either read out of the page's own source or measured by me on a replica of that source. Nothing here is a description of how pitch detection "generally" works - it is how this one works, with the numbers to check.

The constants that set the whole behaviour

The converter is a single HTML file with the analysis in plain JavaScript, so every parameter is readable. These six values decide what the tool can and cannot hear:

ConstantValueWhat it does
Analysis rate16,000 HzEverything is resampled down to this first. Nyquist limit 8 kHz.
Frame length1024 samples64 ms of audio per decision. One pitch per frame.
Frame hop256 samples16 ms. Frames overlap by 75%, so note boundaries snap to 16 ms.
Lowest pitch60 HzBecomes a lag cap of 266 samples.
Highest pitch2000 HzBecomes a lag floor of 8 samples.
YIN threshold0.15The default sensitivity setting (0.85 on the slider).

Two more values matter at the edges of the analysis: a frame is discarded as silence when its RMS is below 0.006, and a candidate note is dropped when it is shorter than 60 ms - under four frames.

What YIN actually does in this page

YIN is not a neural network and it is not a lookup table. For each frame it runs four steps, and the order is the whole trick:

  1. Difference function. For every candidate lag, the frame is compared against itself shifted by that many samples, and the squared differences are summed. A lag that lines up with the period gives a small number. The sum here runs over the first half of the frame only - 512 samples, 32 ms - not the whole window.
  2. Cumulative mean normalised difference. The raw difference is divided by the running average of all the smaller lags, which stops the function from drifting upward as the lag grows. This is what makes a small dip at a long lag as meaningful as a small dip at a short one.
  3. First dip below the threshold. The search walks up from lag 8 and stops at the first lag whose normalised value drops under 0.15. That lag is taken as the period. If no lag qualifies, the frame returns nothing - it does not guess.
  4. Parabolic interpolation. The chosen lag is fitted with a parabola through its two neighbours so the period can land between samples. This is why the reported frequencies are not multiples of 16000/8.

Step 3 is the part that was arrived at the hard way. The comment above the function in the page source records that the first version used plain autocorrelation, which picks the strongest peak - and for a periodic signal the peak two periods out is often slightly stronger than the one-period peak. The result was a C major scale coming back several octaves low. Taking the first qualifying dip instead of the strongest one is what fixed it.

This is also why the tool is monophonic. There is one period per frame, so there is one note per frame. A chord does not produce three notes; it produces a period that matches none of the three.

Why the range is 60-2000 Hz and not something else

The two limits are not musical choices. They fall out of one line of arithmetic each:

minLag = floor(16000 / 2000) = 8 samples   ->  16000 / 8   = 2000.00 Hz
maxLag = floor(16000 /   60) = 266 samples ->  16000 / 266 =   60.15 Hz

Note what the ceiling is not. It is not the sample rate: at 16 kHz the audio could represent anything up to 8 kHz. It is not the frame length either, which allows lags up to 512 samples. The ceiling is simply the shortest lag the search is allowed to try. Raise the top of the pitch range and the search starts at a smaller lag; the code does not, because below about eight samples the difference function starts lining up with the waveform's own shape rather than its period.

The bottom is the mirror image. The lag cap of 266 samples is what "60 Hz" means in practice, and 16000/266 is 60.15 Hz, not 60.00. There is a second, subtler cost down there: the difference function compares only 512 samples, and at 60 Hz one period is 266 samples - so the comparison window holds under two periods. At 2000 Hz the same window holds 64 periods. The low end is working with roughly a thirty-second of the evidence the high end gets.

The resample down to 16 kHz that happens before any of this is not a neutral step, though. It is a plain linear interpolation with no anti-alias filter, so content above 8 kHz folds back down rather than being removed - and some of it lands inside the very window described above. I measured that separately in what happens to the audio before the detector runs.

The two edges, measured

I generated pure sine tones at 2 seconds each, 44.1 kHz mono, and ran them through an exact replica of the page's detector at the default settings. Each 2-second tone produces 122 frames. Here is the bottom of the range first:

Tone inFrames acceptedReported asError
45 Hz0 / 122nothing-
50 Hz0 / 122nothing-
54 Hz0 / 122nothing-
55 Hz122 / 12260.0 Hz+9.2%
57 Hz122 / 12260.0 Hz+5.3%
59 Hz122 / 12260.0 Hz+1.8%
60 Hz122 / 12260.0 Hz+0.06%
62 Hz122 / 12262.0 Hz0.00%
65 Hz122 / 12265.0 Hz0.00%

Two different failures at the bottom, not one. Everything from 54 Hz down returned nothing at all - the detector looked, found no lag below the threshold, and refused to guess. From 55 to 59 Hz it did the opposite: it answered, and the answer was pinned to the lag cap, so all five tones came back as the same note, 60.0 Hz. A 55 Hz tone reported as 60 Hz is a 9% error, which is most of a semitone and enough to land on the wrong note name.

Now the top:

Tone inFrames acceptedReported asErrorNote out
1000 Hz122 / 1221001.9 Hz+0.19%B5
1500 Hz122 / 1221502.8 Hz+0.19%F#6
1900 Hz122 / 1221908.7 Hz+0.46%A#6
2000 Hz122 / 1222013.5 Hz+0.67%B6
2100 Hz122 / 1222104.8 Hz+0.23%C7
2170 Hz122 / 1222179.5 Hz+0.44%C#7
2180 Hz122 / 1221090.4 Hz-50.0%C#6
2200 Hz122 / 1221100.4 Hz-50.0%C#6
2500 Hz122 / 1221251.0 Hz-50.0%D#6
3000 Hz122 / 1221499.2 Hz-50.0%F#6
4000 Hz122 / 1222008.4 Hz-49.8%B6

The headline number is the 2000 Hz ceiling, but the measurements show the interesting part is what happens past it. The tool does not go quiet. It reports half the frequency - the detector finds the second period of the wave instead of the first, and the note comes out exactly one octave low. That is the same octave failure the autocorrelation version had, arriving from the other direction.

Two details worth pulling out of that table. First, the ceiling is soft at the top: 2100 and 2170 Hz were still tracked correctly, because the parabolic interpolation can place the period slightly below the 8-sample floor. Second, the octave error does not stay an error - a 4000 Hz tone was reported as 2008 Hz, which is an octave low but lands back inside the range and gets written out as a plausible-looking note. A wrong answer that looks right is worse than no answer, and this is where it comes from.

Where your instrument sits

Equal-temperament frequencies, A4 = 440 Hz, against the measured working range of 55-2170 Hz:

NoteHzExampleInside the range?
B030.875-string bass, lowest stringNo - returns nothing
E141.204-string bass, lowest stringNo - returns nothing
A155.00bass A stringMarginal - reported as ~60 Hz
C265.41cello, lowest stringYes
E282.41guitar, lowest stringYes
C4261.63middle CYes
A4440.00reference pitchYes
C61046.50soprano high CYes
C72093.00top octave of an 88-key piano begins hereYes, barely
E72637.02violin, upper positions on the E stringNo - comes back an octave low
C84186.01highest note on an 88-key pianoNo - comes back an octave low

The practical consequence is that the bottom of the range bites far more often than the top. A bass line played on the low E of a four-string bass sits at 41.2 Hz, below the floor, and in testing that region returned no notes whatsoever. The top of the range is a narrow band - the last octave of a piano, or a violin in high positions - and it fails in a way you can see, because the notes come out an octave lower than what you played.

What this changes when you use it

What happens to those notes once they are written out - the file format, the tick grid, and what the file leaves out - is picked apart in what is inside the downloaded .mid file.

Where the analysis runs - on your device rather than a server - is covered in MP3 to MIDI without uploading your file. How much the input bitrate moves the result was measured in 128 kbps vs 320 kbps MP3 to MIDI.

What I did not test