A MIDI synth on a chip

This part began with a light that could be dimmed. It ends with a musical instrument: play a MIDI keyboard, and the little computer on the UPduino plays the notes, through its own sound chip, out of a headphone socket. Every piece is one from the pages before; this page puts them together, and counts what it all costs.

1 · From key to ear

A note’s journey:

A key goes down on a MIDI keyboard. It sends three bytes: note on, which note, how hard (MIDI).
The bytes travel as a current through the cable and the opto-isolator on the carrier board, into a pin of the FPGA, and the second copy of the serial port collects them at 31,250 baud.
The 6502 reads them, works out which note it is, chooses one of the sound chip’s three voices for it, and writes that voice’s frequency and gate.
The sound chip starts the voice’s envelope and makes its waveform, through the filter, a new sample every microsecond.
The one-bit DAC turns each sample into a stream of bits at 12 MHz, which the RC filter on the carrier board smooths into a voltage, through the volume control to the headphones.

How long does it take? The three bytes take 0.96 ms on the wire (320 µs each). The program takes under 0.2 ms more once the last byte has arrived (measured on the emulator: 0.18 ms to the gate). The voice then starts its attack at once. So about a millisecond from key to sound: a musician can’t feel that. (The attack itself, with the envelope settings used here, takes 16 ms to reach the top.)

2 · Three voices, ten fingers

The sound chip has three voices; a player has ten fingers. So the program has to share them out, a job called voice allocation. Its rule is the simplest that works: a new note takes a free voice if there is one; otherwise it takes the voice whose note began longest ago, cutting that note off (voice stealing). When a key comes up, the voice playing its note is let go.

The synthesiser program’s rule for voices, as it plays out. Press keys (each click holds it down, the next lets it go). The first three notes get a voice each; a fourth takes the voice of the note held longest, which goes silent though its key is still down. Let a key go and its voice is free for the next note.

Real synthesisers think harder: steal the quietest voice, or the one already releasing, or the lowest note except the bass. All of them are a few more lines in the same place.

3 · The program

The synthesiser is one 6502 program, about 240 lines of assembly (counting its tables), all of it pieces from the earlier pages:

the MIDI reader from the MIDI page, a little state machine handling running status, extended to read control changes as well as notes;
the voice allocator above;
the frequency table from the sound chip’s tune, a note’s register value for each MIDI note;
the buttons to choose the waveform, and the knob (or the keyboard’s control 74, which many keyboards send from a knob of their own) to move the filter’s cutoff;
the LEDs for which voices are playing, and the digits for the last note.

One detail matters for playing: the envelope’s quirk from the sound chip page, where a note can start up to 33 ms late if the rate counter has to go all the way round. The program gives the attack, decay and release the same rate (2), so the counter’s period never drops below where it is, and every note starts at once. (The cost: the three stages have to share one speed. A more elaborate program would use a “hard restart”, as the C64’s music players did.)

The whole synthesiser. Tick sound (in the sound chip’s box) and Run, then play the keyboard below the panel: on screen, with the computer’s keys (A, S, D… for the white notes), or a MIDI keyboard (click Use a MIDI keyboard). Up to three notes at once. Buttons 1–4 choose sawtooth, pulse, triangle or noise for the notes that follow; the knob opens and closes the filter. On the board, the same program, with a real keyboard in the MIDI socket and headphones on the jack.

4 · Fitting it all in

The UP5K has 5,280 logic cells. Here’s how the computer and its peripherals share them, each measured on its own by Yosys and nextpnr:

Part	Logic cells	Other
The computer of part 2: the 6502, RAM, ROM, LEDs, button, digits, timer, tone	1,616	16 block RAMs (the 8 KB ROM), 1 SPRAM (the 32 KB RAM), 1 DSP block
The sound chip	1,516	2 DSP blocks
Video	555	8 block RAMs, the PLL
The serial port	341	2 block RAMs
The DAC	282
MIDI	270	2 block RAMs
Buttons and the knob	172
I²C	125
The RGB LED	89	the UP5K’s LED driver
SPI	82
The 595 chain	59

All together, with the address decoding between them, placed and routed by nextpnr on the carrier board’s pins (board.v): 5,128 of the 5,280 logic cells (97%), 28 of the 30 block RAMs, 3 of the 8 DSP blocks and 35 of the 39 pins. The CPU’s 12 MHz clock could run at up to 17.0 MHz, and the video’s 25.125 MHz clock at up to 35.4. It fits, just, with no room to spare; the build can leave out the video or the sound chip to make some.

The sound chip is the biggest part after the CPU, even sharing one set of logic among its three voices: its 408 bits of voice state (three voices’ registers, oscillators, noise and envelopes) and its 24-bit filter take room whatever the logic does. And it fits at all only because of that sharing.

5 · The whole part

Looking back, every peripheral was the same three things: registers at addresses, a way to read them back, and a job done all the time. What changed was the job:

Timers and PWM: counting clocks, and switching fast to make an average;
Buttons and knobs: getting the outside world safely into a clocked circuit, and twenty outputs from three pins;
the serial port and MIDI: bytes one bit at a time, with no shared clock;
SPI and I²C: buses to other chips, one fast and one shared;
the one-bit DAC: turning numbers into a voltage with one pin;
video: a circuit fast enough to feed a monitor, beside a CPU far too slow to;
the sound chip: many voices from one set of logic.

None of it needed a bigger CPU. A 1 MHz 6502, with the right peripherals, plays music, drives a screen, talks to a terminal and to sensors: the CPU in about a fifth of a small FPGA, its peripherals in most of the rest.

Summary

A MIDI synth is a MIDI reader, a voice allocator and a sound chip, with a DAC on the end: about a millisecond from key to sound.
Three voices for ten fingers: a free voice, else steal the oldest.
The whole computer, with every peripheral, fills 97% of the UP5K, and runs on the carrier board’s pins.