MIDI Music Videos, Merry Christmas

(2017)

Back at Christmas, I heard a very strange recording of "All I Want For Christmas Is You". Somebody had taken the original song and converted it to MIDI and then played it back.

The results are quite unsettling..

Is this what they mean by "transformative art"?

Much of the song is gibberish, but the main chorus is recognizable. Partly this is our brains filling the gaps in a well-known song. But partly it's because a series of discrete notes can approximate a continuous waveform. You just need a lot of them.

So obviously the next question was: Does it work with other songs?

To find out, we started opening MP3s in GarageBand, which can convert MP3 to MIDI. Results were mixed and the encoding was slow. I wanted to do batch conversions so we could try lots of songs quickly.

Having played with audio/video encoding a little bit, I knew there are some pretty powerful CLI tools available. ffmpeg is a workhorse, but it turned out that a few other pieces of software were needed too.

  1. ffmpeg, to convert MP3 to WAV, WAV to MP3
  2. waon, to convert WAV to MIDI
  3. Timidity, to convert MIDI back to MP3
#!/bin/bash
ffmpeg -i /tmp/in.mp3 /tmp/in.wav
waon -i /tmp/in.wav -o /tmp/in.mid
timidity /tmp/in.mid -Ow -o - | ffmpeg -i - -acodec libmp3lame -ab 64k /tmp/out.mp3

The resulting file will be written to /tmp/out.mp3

Going a step further, this can be wrapped in a bash for-loop to encode an entire directory at once.

As a weird bonus, you could even add youtube-dl into the pipeline, and rewrite the MP3 back over the video file. The results are downright bizarre and I can't entirely recommend it. But it's neat to play around with.