aural state splitter.
Intrinsic Pattern
Imagine sub-dividing a digital audio file into individual beats, where a beat is not just percussion but all audio associated with some duration of audio data. The Aural State Splitter project takes a symbolic representation of these beats, creates a time series representation of the song, and infers a statistical model of the time series called an e-machine. Beat detection is performed by the Echo Next Remix SDK for Python.
A random walk on the e-machine generates new symbols, and the beats associated with these symbols are stitched back together to form a new audio stream. In this sense, the intrinsic patterns in the song are allowed to re-write themselves. We can visualize these patterns stored in the e-machine, shown below for Radiohead's "No Surprises":
Familiar Surprise
Instead of a complete statistical re-write, we can instead use a clustering algorithm to group beats that have similar frequency content. We can then traverse the song and replace each beat with a a different one from the same cluster. The number of clusters determines how similar the re-written song will be to the original.
A visualization of the K-means clustering algorithm is given below, where K is the number of clusters to use. Imagine we set K to the number of beats in a song. Each beat will be assigned its own cluster and the new audio stream will be identical to the original. Now imagine setting K to be the number of beats n minus 1. Some cluster must then have 2 beats instead of 1.
A good starting point for adding a sprinkle of surprise is around K=0.9*n. Radiohead's "No Surprises" for K=0.8*n can be played below.
Links
A paper on e-machine inference [postscript]
A paper on clustering samples based on their frequency content [pdf]
An analysis of drumming variance using the Remix SDK [link]
A project using clustering to re-interpret video samples based on live audio input [link]