Alex Rivera | Logout

Perceptual similarity between two audio sequences

Asked 2010-07-04T18:42:25.387
19

I would like to get some sort of distance measure between two pieces of audio. For example, I want to compare the sound of an animal to the sound of a human mimicking that animal, and then return a score of how similar the sounds were.

It seems like a difficult problem. What would be the best way to approach it? I was thinking to extract a couple of features from the audio signals and then do a Euclidian distance or cosine similarity (or something like that) on those features. What kind of features would be easy to extract and useful to determine the perceptual difference between sounds?

(I saw somewhere that Shazam uses hashing, but that's a different problem because there the two pieces of audio being compared are fundamentally the same, but one has more noise. Here, the two pieces of audio are not the same, they are just perceptually similar.)

algorithm audio audio-processing
Edit
Report

1 Answer

12

The process for comparing a set of sounds for similarities is called Content Based Audio Indexing, Retrieval, and Fingerprinting in computer science research.

One method of doing this is to:

  1. Run several bits of signal processing on each audio file to extract features, such as pitch over time, frequency spectrum, autocorrelation, dynamic range, transients, etc.

  2. Put all the features for each audio file into a multi-dimensional array and dump each multi-dimensional array into a database

  3. Use optimization techniques (such as gradient descent) to find the best match for a given audio file in your database of multi-dimensional data.

The trick to making this work well is which features to pick. Doing this automatically and getting good results can be tricky. The guys at Pandora do this really well, and in my opinion they have the best similarity matching around. They encode their vectors by hand though, by having people listen to music and rate them in many different ways. See their Music Genome Project and List of Music Genome Project attributes for more info.

For automatic distance measurements, there are several projects that do stuff like this, including answered 2010-07-06T12:27:22.133

Your Answer