MIDI Music Videos, Merry Christmas
Back at Christmas, I heard a very strange recording of "All I Want For Christmas Is You". Somebody had taken the original song and converted it to MIDI and then played it back.
The results are quite unsettling..
Is this what they mean by "transformative art"?
Much of the song is gibberish, but the main chorus is recognizable. Partly this is our brains filling the gaps in a well-known song. But partly it's because a series of discrete notes can approximate a continuous waveform. You just need a lot of them.
So obviously the next question was: Does it work with other songs?
To find out, we started opening MP3s in GarageBand, which can convert MP3 to MIDI. Results were mixed and the encoding was slow. I wanted to do batch conversions so we could try lots of songs quickly.
Having played with audio/video encoding a little bit, I knew there are some pretty powerful CLI tools available.
ffmpeg is a workhorse, but it turned out that a few other pieces of software were needed too.
ffmpeg, to convert MP3 to WAV, WAV to MP3
waon, to convert WAV to MIDI
Timidity, to convert MIDI back to MP3
#!/bin/bash ffmpeg -i /tmp/in.mp3 /tmp/in.wav waon -i /tmp/in.wav -o /tmp/in.mid timidity /tmp/in.mid -Ow -o - | ffmpeg -i - -acodec libmp3lame -ab 64k /tmp/out.mp3
The resulting file will be written to
Going a step further, this can be wrapped in a bash for-loop to encode an entire directory at once.
As a weird bonus, you could even add
youtube-dl into the pipeline, and rewrite the MP3 back over the video file. The results are downright bizarre and I can't entirely recommend it. But it's neat to play around with.