How Many Data Points Can You Hear?
Posted on June 10, 2011 by Dr. Greg Wilder
We’re often asked, “How many musical data points does Clio capture?” A good question, but one that doesn’t have a simple answer…
The thing is, Clio’s process is much smarter than that. Instead of mindlessly extracting a slew of spectral and temporal data points for statistical analysis, Clio intelligently adapts its attention to key aspects of the musical language present in the track — just like you and I do.
To best illustrate, let’s take a look at some pure results from Clio’s mood-based matching algorithms. This first match-list begins with a section from a randomly chosen seed track (courtesy APM Production Music) and continues with a list of close mood matches.
Listening to this match list, it’s clear Clio has identified and matched the musical elements responsible for the epic, heart-pounding, action-filled feel of the original seed track.
But, what exactly is Clio listening to?
Clio identifies and models numerous musical elements, intelligently adapting itself to focus on the aspects most critical to the mood of the starting seed track. In this case:
Rhythmic Feel: Much more than simple-minded estimations of meter and BPM, Clio hears the frenetic hi-hat and cymbal patterns and strong orchestral accents (‘stabs’) and understands their critical importance to establishing the heart-pounding feel of the seed track. Clio behaves far more intelligently than traditional pattern matching algorithms — Clio searches out similarly aggressive rhythmic motives, ensuring the original mood is present throughout the match list.
Harmonic and Melodic Archetypes: Underneath the intense, pulse-pounding rhythmic features, the presence of strong tonic foundation, thematic use of the minor third (e.g. Match #2: 0:26, Match #3: 0:41), and relentless step-wise ascending modulations (e.g. Match #4: 0:55, Match #5: 1:05) go a long way toward defining the mood of this match-list. Again, Clio reacts like a human listener, discovering similar archetypes that maintain the original mood and feel — beyond the traditional limitations of keywords and generic pattern matching algorithms.
To see just how subtle and detailed Clio’s ability to discern mood is, check out this match-list that begins with an aggressive, rock-based seed track.
Even though many of these tracks share musical elements in common with the previous match-list (i.e. frenetic rhythm, strong accents, obsessive use of the minor third, and strong tonic foundation) Clio doesn’t confuse the epic-sounding aggressive mood in the first example with the similar (yet different!) youthful, raw, guitar-based aggressive feel of this seed.
Ultimately, Clio’s close match-lists provide music supervisors options that are consistent in mood while offering possibilities that range across varied styles and sub-genres — pure, content-based music discovery at a level previously reserved for expert musicians and musicologists.