MIR Research

Fading Out

2010-03-07T02:05:00.003Z

I first started learning about MIR around spring 2001. Back then I never would have guessed what a wonderful journey I was about to embark on. The journey had way more highlights than I ever deserved and I'm very grateful to so many people I've met on the journey.

I recently returned to Vienna to be closer to my family and friends again, and to settle down and start a new chapter in Gwen's and my life. Part of this transition has also been switching jobs. I'm in telecommunications now and uncertain if I'll ever return to MIR research.

I'll always continue following all the great stuff that's going on in the community from the side lines - but my already very infrequent blogging frequency is very likely to continue to decrease. When I started writing there was basically just Paul's blog. Now there's still Paul's blog, but also lots of other great MIR related blogs. Some of which are listed in the side bar.

Updated List of MIR PhDs

2010-03-07T00:18:00.004Z

There's a long-overdue updated version of the list of MIR PhDs. The new entries I've added are Stéphane Rossignol (2000), Juan José Burred (2008), Iman S. H. Suyoto (2008), Matti Ryynänen (2008), Enric Guaus (2009), Claudio Baccigalupo (2009), Klaas Bosteels (2009), Katy Noland (2009), Tim Pohle (2010), and Jouni Paulus (2010). I've also updated some other links. Thanks to everyone who sent me information!!

I've had the pleasure of reading the thesis of Claudio and I can highly recommend it. His enthusiasm is inspiring. It's a great point to start for anyone researching playlist generation.

Another pleasure I've had was to attend the defense of Klaas' thesis. I've been closely following his work for a very long time. I've had the honor to collaborate with him on several occasions, and I'm very pleased that it's him who has taken over some of my previous responsibilities at Last.fm.

I've also been following Katy's and Tim's work for many years. Which makes me wonder how soon until all MIR PhD students I know are done. Until that happens I'll infrequently continue updating this blog and the list.

10 Reasons to Work for Last.fm

2009-10-23T11:39:00.004+01:00

I've moved to Vienna, and I'm about to leave Last.fm. Looking back, here are 10 reasons why I love to work for Last.fm:

1. Staff
My colleagues turn work into play and have created an environment that feels like a family. I've learned a lot from many of them. Some have become very close friends. All of them have inspired me with their passion, enthusiasm, motivation and curiosity for everything related to technology and music. Also they've always been very patient with me and always very helpful. I admire their intelligence, creativity, and how determined they are to make music more enjoyable.

2. Users
There is a constant flow of very valuable feedback from our users. Many are as passionate about Last.fm as we are. They have enabled us to run countless experiments (A/B tests and similar) - from which we learned more than we could have ever learned in a standard academic environment.

3. Data
If you love data mining and working with large data sets, you'll love working at Last.fm. There are a lot of fascinating things that can be learned about music from Last.fm's data. And there is a lot of fascinating things to learn about dealing with data at such a scale.

4. Technology
Last.fm has been on the front-line of some very interesting developments. Always using the tools that work best and developing new ones when needed. Always trying to push scalability limits and trying to make things work faster and better. Before joining Last.fm I didn't realize how fascinating technology can be that solves large scale problems. I've learned new programming languages, things like Hadoop, and many algorithms I had never heard of before. I caught a glimpse of the amazing technology (both software and hardware) behind serving tens of millions every month. If you love technology, you'll love working at Last.fm.

5. Openness
The data is easily available through APIs. Last.fm staffers constantly contribute to open source projects and parts of Last.fm are open source. Every staff member (even I) can post anything they want on the company blog. We are constantly attending events where we talk about the things we do and share our experiences. Also the communication within Last.fm is very open.

5. Fun
We have a table tennis table standing around in the office, a foosball table, there was always a skateboard around, and for a while we even turned one of the meeting rooms into a ball pit (and had meetings in there). There were remote controlled helicopters flying around in the office, XBoxes, we have colored bears telling us about the state of our code, highlight words on IRC include "pub", on IRC we also constantly share the newest coolest stuff we find on the Internet, servers have names like "badger", ...

6. Lunch Discussions & Techmosis
One of the things I've already started missing since I moved to Vienna are the daily (informal) lunch discussions. In particular the discussions in which Norman Casagrande was involved - he's like a living encyclopedia. We talked about why C++ templates are so useful, how to solve the problems in the middle east, inconsistencies in religious beliefs, objectivity as a concept, world history, politics, the daily show, ...

We also have something we call Techmosis, where we teach each other things. Usually Friday evenings we reserve a one hour slot to learn about something cool. Sometimes we'd also have non-staffers teach us. Today it will be a presentation on manga by Japanese manga artists. Generally I found the atmosphere of teaching each other extremely inspiring.

7. London
London is an amazing city. There is so much happening, there is more stuff to check out every day than I could absorb in a year. I'd love to move Last.fm to Vienna, but I can see why East London is the perfect place for Last.fm. As one of my colleagues says: it rubs off on you. If you haven't lived in London yet, you should definitely give it a try.

8. Free Stuff
My favorites are my Last.fm t-shirts (one of which I'm wearing right now) and the fruit. (I herewith admit that every week I probably eat about half of our weekly supplies all on my own.) Pizza lunches and the occasional pub visits with a company credit card were nice to have, too.

9. Rooftop Barbecues
We made sure our neighbors knew they lived (or worked) next to us. But I think the best part always was how the parties just seemed to spontaneously happen.

10. Music
Last on this list, but not least: If you love music, you'll have a hard time finding a better place to work than Last.fm where you are surrounded by music, and people who love music. You might also like the band practice room.

Wittgenstein Award: Gerhard Widmer (!!)

2009-10-20T17:18:00.009+01:00

The Wittgenstein award is kind of like the Austrian version of the Nobel prize. It's worth €1.5M and the most prestigious research award I'm aware of. Yesterday Gerhard Widmer received the prize for his outstanding research work.

Gerhard is one of the hardest working I know, he has an amazing talent of communicating research to non-researchers, creating very productive work environments, and bringing the right people together. Also he has conducted, supervised, and inspired lots of great research. He also gave me my first (paid) job in MIR and supervised my PhD. I've learned a lot from him and I'm very grateful. So I'm particularly happy to see Gerhard received this award he well deserves.

My favorite daily newspaper wrote about it. And here's a quick, shortened, and far from accurate translation of what they wrote:

Frustrations and Fortunes with Ludwig van Beethoven

Among Gerhard Widmer's favorite music you find Beethoven's piano sonatas. He doesn't have a preferred interpret. For some phrases he prefers Friedrich Gulda, for others Alfred Brendel, or other pianists.

It is also Beethoven who is to blame that Gerhard did not pursue a career as musician and instead became a internationally renowned researcher on algorithms to study music. Work for which he was now awarded with the Wittgenstein prize.

Gerhard was an early talent but gave up his career as pianist after frustrations with Beethoven's sonatas as a teenager. Instead he took a quick dip in Jazz and more or less randomly ended up studying computer science.

He received an MSc in Vienna and at the University of Wisconsin. It was also in Wisconsin that Gerhard briefly returned to the Jazz piano. Back in Vienna he completed his PhD in computer science which finalized his career as researcher.

Gerhard's research started with analyzing the performances of a single artist and as a side effect he tried to teach computers to interpret music. Later his group started focusing on developing algorithms that enable organizing and retrieving content from very large music collections. The work of his group can also be found in the newest devices by Bang & Olufsen.

Music has always been more than just a research subject to Gerhard. And he says that the scientific analysis of music does not take away any of its magic - instead it makes the music even more beautiful when you start to understand its structure.

Updated MIR PhD Theses List

2009-09-06T17:41:00.003+01:00

I've updated the list of MIR PhD theses to include the recent work by Antti Eronen on Signal Processing Methods for Audio Classification and Music Content Analysis. Antti covers a broad range of fascinating audio content processing topics such as: instrument classification, classification of ambient environments/backgrounds (libraries, cars, ...), chorus detection, and meter analysis.

Seems like I'm the list is still far from complete. As the following two theses indicate. If you know of any more that I'm missing please let me know. (Thank you Arijit Biswas for your help!)

One that I missed is the work of Michael J. Bruderer on Perception and Modeling of Segment Boundaries in Popular Music. Which describes an interesting series of experiments in which Michael explores which cues listeners use when segmenting music and how they can be modeled.

Another older one I've added is the work of Olivier Gillet Transcription des signaux percussifs. Application à l'analyse de scènes musicales audiovisuelles. Olivier joined Google after his PhD where he worked on the optimization of Google's ad product and on technologies now being used by Google China Music. Olivier is now joining us at Last.fm's to work on fun next generation MIR technologies.

I would also like to mention the Master thesis Implementing a scalable music recommender by Erik Bernhardsson. I really like how Spotify has given him access to their data and let him publish the results. Seems like Spotify was also very happy with his work as they have hired him.

Bang & Olufsen and OFAI

2009-08-04T11:51:00.005+01:00

This is probably very old news to some, but I'm a bit behind in blogging.

Yet another really cool product my former colleagues at OFAI have been working on over the last years: integration of content-based playlist generation technologies into the most awesome music hardware out there.

Read more about it here (English) and here (German).

If you can't read German, the perhaps most interest bits: retail price per unit €4.765, 500 pre-orders before the device was available, OFAI and B&O will continue the collaboration, size of the features representing each song: 3.2KB.

FM4 Soundpark and OFAI

2009-08-04T11:33:00.006+01:00

My former colleagues at OFAI have been experimenting with audio content based music recommendation for FM4 Soundpark for a while now. (I blogged about it here.) Now they (and in particular Martin Gasser) have taken it a few steps further and implemented a 3D exploration interface using the islands of music metaphor.

It's all in Java and requires special privileges because it needs access to low level graphics card functions. But if you get passed those hurdles here's what you'd see:

(Navigation: space bar, A,W,S,D and mouse)

There is more information about this and other recommendation tools they have developed for Soundpark here (in German).

Last.fm Artist Connections

2009-05-03T00:49:00.003+01:00

Playing with Last.fm similar artist data, trying to connect one artist to another through similar artists:

The Beatles -> John Lennon -> Sean Lennon -> Joseph Arthur -> Howie Day -> Teddy Geiger -> Jonas Brothers

ABBA -> Cher -> Madonna -> Róisín Murphy -> Björk -> Radiohead

Metallica -> Iron Maiden -> Paul Di'Anno -> Numbers From The Beast -> Scott Lavender -> Ark Sano -> Frédéric Chopin -> Wolfgang Amadeus Mozart

Brian Eno -> Roxy Music -> Bryan Ferry -> Eurythmics -> Annie Lennox -> Madonna -> Britney Spears

MIR PhD Thesis: Luís Gustavo Martins

2009-04-19T17:09:00.003+01:00

Luís Gustavo Martins recently completed his PhD thesis titled A computational Framework for Sound Segregation in Music Signals.

From the abstract: "[...] This dissertation proposes a ﬂexible and extensible Computational Auditory Scene Analysis framework for modeling perceptual grouping in music listening. The goal of the proposed framework is to partition a monaural acoustical mixture into a perceptually motivated topological description of the sound scene (similar to the way a naive listener would perceive it) instead of attempting to accurately separate the mixture into its original and physical sources. [...]"

Luis is probably best known in the MIR community for his contributions to Marsyas (the awesome open source software framework for audio processing with specific emphasis on MIR applications).

If you know of any other dissertations missing in the list of MIR PhDs please let me know.

Visual Listening Charts (Part 2)

2009-04-15T01:18:00.006+01:00

Here's a follow-up to my previous attempt to visualize my listening history differently.

Basically, I've been looking for a fun project that would give me plenty of reasons to play with matplotlib.

My preliminary conclusion is that matplotlib is awesome. They couldn't have made it much easier to use for someone already familiar with Matlab.

Recent MIR PhDs

2009-04-06T02:07:00.007+01:00

I'm slowly catching up. The following dissertations were added to the list of MIR PhDs:

"Real Time Automatic Harmonisation" by Giordano Cabra. I couldn't find a link to the thesis, but I found a video of the defense. I'd love to see more defense videos (preferably in English).

"Modeling musical anticipation: From the time of music to the music of time" by Arshia Cont.

"Music Recommendation and Discovery in the Long Tail" by Oscar Celma. I highly recommend it!

"From Sparse Models to Timbre Learning: New Methods for Musical Source Separation" by Juan Jose Burred.

UPDATE: I totally missed Yves' announcement on the Music-IR list on Friday. I've added him now too:

"A Distributed Music Information System" by Yves Raimond.

UPDATE 2: Almost forgot that Kris finished recently too:

"Novel Techniques for Audio Music Classification and Search" by Kris West.

Please send me any I might have missed - thanks!

Since ISMIR I've been listening to...

2009-04-05T23:36:00.004+01:00

I'm currently a bit fascinated with different ways of representing my listening history.

I really like LastGraph which was inspired by Lee Byron. The graphs show how my listening preferences (and in particular how often I listen to my favorite artists) change over time.

I also like these visualizations by Martin. They show how artists move up or down over time in my chart.

Recently I tried to visualize my top artist chart using artist images, and readjusting their sizes so they correspond to how often I've listened to each artist respectively. Here's what I've been listening to since my last blogpost at ISMIR 2008:

I think this montage says it all: I like black and white music from solo artists playing guitars.

C++ Software Engineer, Data & Recommendations

2009-04-05T23:02:00.004+01:00

If you are interested in data structures, algorithms, and scalability, you might also be interested in joining the data and recommendations team at Last.fm.

You'd be working with Norman Casagrande, Mark Levy, me and other highly motivated colleagues trying to solve lots of fun challenges in music information retrieval.

ISMIR 2008 Demos

2008-09-18T22:02:00.009+01:00

One of the best parts about ISMIR was the demo session.

Paul Lamere and Francois Maillet demonstrated Explaura which enables users to directly interact with tag clouds by resizing individual tags for tag-based recommendations. Increasing the size of a tag puts more emphasis on it, shrinking it reduces the impact to the point where it's ignored. Increasing the "negative size" of a tag filters results by the respective tag. They also allow combining artists with tags in the search query. Paul blogged about it. There's also a short ISMIR abstract. I like the idea of interacting with the individual tags in a tag cloud and found it very intuitive to use.

Another demo which was presented (and I already previously blogged about) was the work of Martin Gasser and his colleagues on how they integrated audio-similarity into FM4 Soundpark (a platform for independent Austrian artists). The numbers they presented show that using audio-similarity helped Soundpark users find older and more obscure items in the catalogue. It would be nice to see more real world applications using audio-similarity.

Òscar Celma and Marcelo Nunes presented GeoMuzik which allows drawing a route on a world map. Their system then generates a playlist according to this route. They implemented genre/tag filters. They can also visualize the artists in my Last.fm profile on top of a map. There's an ISMIR abstract. I liked the demo very much and would like to play a bit more with it. The picture below shows a screenshot of a playlist generated from the interface.

Luke Barrington and his colleagues demonstrated their new tagging game Herd it which they implemented in the Facebook application framework. If I understood them correctly they still want to do some testing before releasing it publicly, so I'll write more when they are ready. Anyway, I've heard from others how difficult it can be to get something to work seamlessly on Facebook. I'm impressed that Luke and his colleagues are doing this. It would be nice to see researchers use Facebook and similar platforms more frequently for their work.

One demo I unfortunately didn't have enough time to see (but at least I got the handout) was the work Anita Lillie presented. There's also a video. In the video Anita shows different ways to visualize the same music collection from different perspectives using a PCA (principal component analysis, a linear projection of a high-dimensional space onto 2-dimensions). Seems like Anita just finished her MSc thesis on MusicBox: Navigating the space of your music. I've been told the demonstration was implemented in processing. Very nice! One thing I didn't see were playcounts. It would be great if playcounts (or ratings per track) could also be visualized (size of the circles?). The picture below shows a screenshot of her MusicBox.

Another demo I really liked was Claudio Baccigalupo's work on Poolcasting. The idea is to have several people tune into the same radio station at the same time, and to have them rate the songs, and use that rating to optimize the overall listening experience. I could easily see Gwen and myself sharing the same radio stream frequently. What I found particularly interesting is how Claudio tries to maximize happiness for everyone in a situation where compromises are unavoidable and where he does not want the majority to completely ruin the experience for an individual.

Btw, if you like demos, you might also be interested in Last.fm's playground.

MIREX 2008

2008-09-17T20:05:00.003+01:00

This year's MIREX evaluation task has been one of my personal ISMIR 2008 highlights. Stephen Downie and his team computed more numbers than I could possible keep track of for lots of different algorithms in 12 different MIR tasks. That's a lot more than in any of the previous years, and that's a lot of interesting data to dig into.

I've been particularly interested in the auto-tagging task. It's the first time MIREX ran this form of task, and there have been only few research papers in the MIR community on the subject. As far as I understood there is no agreement yet as to how to exactly evaluate the algorithms, which is also reflected on the result page. Kris West has added information on the statistical significant of the results which show that none of the submissions was consistently and significantly better than others. Nevertheless, there's a lot to learn from the evaluation and I hope we'll see many more participants next year.

Paul has a good summary of the discussion of this year's MIREX panel.

1000 years of music to listen to

2008-09-17T13:01:00.004+01:00

Youngmoo Kim had asked everyone on the ISMIR recommendation panel to briefly summarize what they think will happen in the next 5 years of music recommendation. However, it's really hard to do so in less than 4 minutes.

From my perspective the most interesting development in the next 5 years will be the increase in the amount of data we will be working with. We will have a lot more of the same and we will have additional sources. Combining different sources is an interesting challenge, but the main challenge will be to scale things up.

All of this additional information will lead to much better recommendations overall, and in particular in the long tail. We'll be able to detect new trends such as an up-and-coming artists or the emergence of a new subgenre much sooner. We'll be able to localize recommendations a lot more.

At the same time there'll obviously be a lot more music to choose from. I'd roughly estimate about 200 million tracks in Last.fm's recommendation engine in the next 5 years. That's more than 1000 years of continuous listening. Subcultures and genres will emerge faster.

In 5 years recommendation engines will have a much better understanding of listeners. While Last.fm, Pandora, and others already do a lot to understand what listeners are interested in, I'm sure there is room for a lot more improvements.

Another interesting development I'm looking forward to is data portability and openness. In particular, I'm looking forward to users being able to move freely with their personal data from one site to another. Similar to how Last.fm users can already today allow other sites to access their data.

I'm also expecting to see a lot more artists and labels embrace recommendation engines. Similar to SEO (search engine optimization) more artist and labels will try to do a lot more REO (recommendation engine optimization).

Obviously mobile applications will be very important, and so will mobile music recommendations. And I have no doubts that human-to-human recommendations (which are strongly supported by Last.fm) will continue to be very important, maybe even more than they are today.

Anthony Volodkin made a great point that we'll see a lot happen in terms of user interfaces, how recommendations are represented, how recommendations are explained. I believe Paul Lamere would call that steerable and transparent recommendations. I like how Last.fm explains recommendations by explaining a recommendation in terms of a bunch of similar artists I'm familiar with. However, there's obviously room for a lot more. On the other side, I wouldn't mind no explanation at all, as long as every recommendation is spot on. Anthony also made a great point by pointing to playful discovery systems.

I believe it was Brian Whitman who said that recommendations will be a commodity. Every music site will have recommendations. Just like almost every web 2.0 site out there supports tagging. I believe Etienne Handman made a similar point when he previously explained to me why he expects the word "personalization" to fade away. Everything will be personalized, it will be the default option.

ISMIR Thoughts

2008-09-16T20:53:00.004+01:00

Inspired by Paul's constant flow of ISMIR blog posts I thought I should give it a try and post some thoughts as well.

First of all, the organizers did a wonderful job organizing ISMIR. I already wrote about what I think about the electronic proceedings. I was also very happy to see that they did not waste unnecessary resources and skipped the silly conference bag thing.

I very much enjoyed giving the social tags tutorial with Paul. It worked out really well, and although I knew Paul's slides since weeks, I found it fascinating to listen to Paul talk about them.

So far ISMIR has by far exceeded my expectations. I've had the pleasure to meet many in person that I previously hadn't had the opportunity to meet. I've also had the pleasure to see many very interesting posters. Unfortunately I've also managed to miss many that I wanted to see. I guess there's never enough time to see everything.

Last night's banquet was great too. In particular I enjoyed the conversations with Etienne from Pandora.

The recommendation panel today was fun, too.

ISMIR 2008 Tutorial: Social Tags and Music Information Retrieval

2008-09-11T22:23:00.004+01:00

When Paul originally asked me if I'd be interested in helping him put together a tutorial proposal for ISMIR I was a bit reluctant. I did an ISMIR tutorial a long time ago, and I didn't forget how much work it was (although it was only a one hour mini tutorial). In fact, only last year Paul and Oscar told me how much work it was to put together their recommendation tutorial (which I liked a lot).

Anyway, somehow I couldn't say no and looking back I don't regret it at all. I'm totally fascinated by social tags. There's no tutorial topic I'd rather talk about. Working together with Paul has been a great pleasure, and I've learned a lot. However, I'm very much looking forward to have a free weekend or even just a free evening again. (Free as in not-MIR-related.)

Paul and I have put together about 200 slides for our tutorial. I think we still need to figure out our time budget. Hopefully we'll have plenty of time for interesting discussions.

Btw, as part of the tutorial we started compiling a list of relevant papers. That list started to grow. Then we thought it would be a good idea to group papers into topics (e.g. autotagging). Then we realized that several papers were in several categories... which is when we moved everything to delicious. In particular, we started using the tag "SocialMusicResearch" to mark interesting things we found in the Internet. Here's a list of some of the items we have tagged. We hope that others will start using that tag as well.

Btw, if you are going to ISMIR, please say hi! If you don't know what I look like: try to spot the guy wearing a Last.fm t-shirt.

ISMIR Proceedings 2008! Wow!

2008-08-27T23:50:00.004+01:00

I'm extremely impressed. The ISMIR proceedings are online. Whoever wants a printed copy can organize it themselves (it couldn't be much easier). Some might also want to only print the papers they are interested in. And some might be happy to have only an electronic version.

It's always been a pain to drag the heavy ISMIR proceedings home. And it always felt like a huge waste of paper.

I heard Juan and Youngmoo talk about this idea a year ago in Vienna (at last year's ISMIR). I'm very happy to see that they found a solution that should make everyone happy.

Juan writes in his email:
We hope that you will like this new approach to printing the proceedings which we intend to be more cost effective, more convenient, and, with luck, more environmentally friendly than mass printing of proceedings for all attendees who may not wish to carry a printed copy around.

Wonderful! :-)

Librarians and Tags

2008-08-25T14:04:00.003+01:00

Yves pointed me to this really nice presentation by a librarian who seems to have a really good understanding of tagging. The only part missing in that presentation is music.

Libraries and the Hive Mind: Folksonomies and Tagging

View SlideShare presentation or Upload your own. (tags: tagging tags)

Getting Last.fm Tags for MP3s with Python

2008-08-25T03:15:00.007+01:00

Paul (who is already distributing a large chunk of Last.fm tags) and I are planing to include a few slides in our ISMIR tutorial on how to obtain tag data.

Below is some Python code that basically takes an MP3 file as input and outputs a list of Last.fm tags (for both artist and track). The MP3s don't need correct ID3 tags, but they need to be full length (clips won't work).

The Python code uses Norman's command line finger printing client to find the correct artist and track name. The path to the executable needs to be set in the code. Norman supports Win32, OSX Intel, Linux - 32.

The output is written to a file. For each MP3 file passed as argument there are up to two rows in the output file: one for the artist tags, and one for the track tags. Each row has the format: "<mp3filename> <encoded artist or artist/track name> <tag> <score> [<tag> <score> ...]". Tabs are used as delimiters.

The data from the Last.fm API is available under the Creative Commons Attribution-Non-Commercial-Share Alike License.

Btw, special thanks to Eric Casteleijn for various Python recommendations (lxml etc). (Which reminds me that I still need to fix the other Python code I posted.) As usual any feedback is much appreciated.

import subprocess, sys, re, time, urllib
from lxml import etree

FP_CLIENT_PATH = '"C:\\fpclient\\lastfmfpclient.exe"'
MAX_RETRIES_URL_OPEN = 5

def getArtistTrack(mp3FileName): # ret: (artist, track)
    command = FP_CLIENT_PATH + ' ' + mp3FileName
    pipe = subprocess.Popen(command, \
                            stdout=subprocess.PIPE).stdout
    for line in pipe:
        mo = re.search('<url>.*/([^/]+)/_/(.+)<',line)
        if mo:
            return urllib.quote(mo.group(1)), \
                   urllib.quote(mo.group(2))
    print "ERROR: failed to get artist/track for: " + \
          mp3FileName

def crawlTags(url): # ret: [(tag, count), ...]
    for i in xrange(MAX_RETRIES_URL_OPEN):
        tagCounts = []
        time.sleep(1) # be nice!
        try:
            root = etree.parse(
                urllib.urlopen(url)).getroot()
        except IOError:
            print "(%d/%d) Failed trying to get: %s." % \
                (i, MAX_RETRIES_URL_OPEN, url)
        else:
            for tag in root.iter('tag'):
                tagCounts.append(
                    (tag.find('name').text, \
                     tag.find('count').text))
            return tagCounts

def tags(prefix, items, outStream): # crawl and write
    for mp3FileName, item in items:
        url = prefix + item + '/toptags.xml'
        print url
        tagCounts = crawlTags(url)
        outStream.write('%s\t%s\t%s\n' %
                (mp3FileName, item, '\t'.join(
                    tag + '\t' + str(count)
                    for tag, count in tagCounts)))    

def main():
    if len(sys.argv)<3:
        print 'USAGE: python getTags.py ' + \
              '<outFile> <f1.mp3> [<f2.mp3> ...]'
        sys.exit(2)
    outFile = sys.argv[1]
    mp3FileNames = sys.argv[2:]
    artists = set()
    artistTracks = set()
    for mp3FileName in mp3FileNames:
        print 'Fingerprinting: ' + mp3FileName
        artist,track = getArtistTrack(mp3FileName)
        artists.add((mp3FileName, artist))
        artistTracks.add((mp3FileName,
                          artist + '/' + track))

    print 'start crawling tags'
    o = open(outFile,'w');
    tags('http://ws.audioscrobbler.com/1.0/artist/', \
         artists, o)
    tags('http://ws.audioscrobbler.com/1.0/track/', \
         artistTracks, o)    
    o.close()    

if __name__ == "__main__":
    main()

Tagging Critics

2008-08-24T11:16:00.007+01:00

I was doing some research for the ISMIR tag tutorial when I stumbled upon (via this interesting paper Playing Tag: An Analysis of Vocabulary Patterns and Relationships Within a Popular Music Folksonomy by Abbey E. Thompson):

The following expert from this paper:

[...] "tags are often ambiguous, overly personalised and inexact" [...] "The result is an uncontrolled and chaotic set of tagging terms that do not support searching as effectively as more controlled vocabularies do." [...]

This was published in the D-Lib magazine in early 2006. I wouldn't be surprised if by now the authors realized they were wrong.

But why would anyone ever want to control the vocabulary people use when describing something so extremely multifaceted and something that evolves so fast like the content on the web (delicious), or snapshots of life (flickr), or music (Last.fm)? I guess I'd need to think more like an old-skool librarian to understand that.

Tagging Games (Callabio on Facebook)

2008-08-23T01:49:00.004+01:00

Did Microsoft researchers clone the scoring system MajorMiner uses for a tagging game on Facebook? Well, maybe it's not identical, but it seems kind of similar. And I guess it could be lots of fun.

Btw, if scoring points is what drives people to play that game, what would stop them from entering a whole dictionary?

UK Hadoop User Group Meeting

2008-08-23T00:05:00.007+01:00

Last Tuesday the first UK Hadoop user group meeting took place in London. Johan did a great job in organizing it and speakers included Doug Cutting from Yahoo! (who leads the Hadoop project).

Here's are some links:

Yahoo! developer network blog mentioning the event. (Includes a video of interviews with some of the presenters.)
Doug Cutting: Hadoop overview
Tom White: Hadoop Web Services on Amazon S3/EC2
Steve Loughran: Deploying Apache Hadoop with Smartfrog
Mark Butler: Distributed Lucene for Hadoop
Last.fm related talks:

Seems like the interesting talk by Miles Osborne on "Using Nutch and Hadoop for Natural Language Processing" is still missing on skills matter's website. I'll update this blog post when the talk is added.

More MIR related PhDs

2008-08-02T20:39:00.009+01:00

I've added 5 dissertations to the incomplete list of MIR related PhDs. I'm particularly happy that Markus, Tomoyasu, and Kazuyoshi (all of them are former colleagues of mine) finished their thesis so successfully.

Tomoyasu Nakano recently finished his thesis on "A Study on Developing Applications Systems Based on Singing Understanding and Singing Expression" (in Japanese). He has now joined Masataka Goto's research group as a postdoc and is working on the CrestMuse project. He recently received a lot of attention for his work on optimizing Vocaloid parameters.

Kazuyoshi Yoshii recently finished his thesis on "Studies on Hybrid Music Recommendation Using Timbral and Rhythmic Features". Kazuyoshi was awarded a tenure position at the AIST which is pretty impressive. He joined Masataka's group and is working on CrestMuse.

Markus Schedl recently finished his thesis on "Automatically Extracting, Analyzing, and Visualizing Information on Music Artists from the World Wide Web". As part of his thesis he crawled a very impressive number of web pages to build a retrieval system for 600,000 artists. In the next months he'll be finishing his business studies.

Steven Travis Pope's thesis is already a bit older (2005) and serves as perfect example of how incomplete the list of PhDs is which I maintain. His work was on "Software Models and Frameworks for Sound Composition, Synthesis, and Analysis: The Siren, CSL, and MAK Music Languages".

The 5th PhD thesis I added to the list is Matt Wright's work on "Computer-Based Music Theory and Acoustics" which he completed at CCRMA, Stanford University.

Creepy Recommendations

2008-07-26T23:05:00.003+01:00

I just got some recommendations from an algorithm that were so good that it was creepy. (One of the recommendations was this video.)

It made me realize how such recommendations can be a nice shortcut for a machine into someones heart. (Although it only takes a few wrong recommendations to be kicked out again.)

I wonder if in the near future I'll have gotten used to the idea that an algorithm attached to my attention profile data will know me better than any human being could (and I'm not just talking about music).

The New Last.fm

2008-07-25T13:14:00.004+01:00

I guess this post is a bit off topic. But I’d argue that Last.fm is one of the main MIR related web sites out there, and I found the launch of the new site very exciting.

Here’s a link to the main announcement on our blog (with over 2000 immediate responses from users, most of them are negative). The blog post includes links to the forums where the feedback continued after the comments needed to be closed.

Here’s a Last.fm group with over 11,000 users asking to bring back the old design. An interesting read are also the forums of that group. Some user's have even worked on ways (e.g. using greasemonkey) to bring back the old look and feel.

Here is one of many youtube videos of people complaining.

And there are even some conspiracy theories.

The negative feedback was being voiced in many different places. Including, for example, in the comment section of an article by the Times Online which focused on how the changes related to advertising.

(Btw, there’s also been a lot of positive feedback, too. For example, there are some positive comments in this digg article.)

When I joined Last.fm over a year ago there were like a million things I thought that could be improved on the site. I was bugging those in charge of the web page design on a daily basis. However, they were already aware of almost everything I was pointing out, and explained that there will be a major redesign coming, and that things will be fixed then. As time went by I started to appreciate the complications of making changes to the old site. It was a site that had grown very quickly in many different directions that weren’t fitting together perfectly anymore.

So more than a year ago my colleagues had started making plans on how they would design the perfect Last.fm if they could start from scratch. I’d also like to mention that almost all of my colleagues are hardcore Last.fm users (and have lots of friends who are Last.fm users, and spend time talking to their moms (or other less technology savvy users) about what difficulties they might have using Last.fm). I think it’s fair to say that most of my colleagues have an excellent understanding of the various issues related to the user experience.

However, at the same time the old site was still in full development. A lot of new features were being launched, integrated, bugs fixed, etc. Maintaining the old site was a full time job for a small (but quickly expanding) team, and my colleagues were spending any free minute they had on completely redesigning Last.fm. I’d also like to add that when they talked about change, it wasn’t simply the design, navigation, and structure that were being considered. A lot of changes involved some serious backend changes and new features. It really was all about making the dream of a perfect Last.fm come true.

Btw, here's an interesting podcast where Hasso Plattner (founder of SAP) talks (among many other things) about the challenges of developing the next version of a product while maintaining the previous version. I'm pretty proud of how my colleagues have mastered this challenge.

When the site went live everyone knew that there was still plenty of room for improvements, but at the same time it was clear that the benefits would largely outweigh the remaining issues. And we also knew that even if the new site were absolutely perfect from the start (which it obviously wasn't), it wouldn't be easy for those who were able to use the old site blindfolded. This included myself: there were several moments of serious frustration where I knew what I wanted to do on the old site, but couldn’t instantly figure out how to do it on the new one. (Btw, continuing to maintain two sites was not really an option.)

Despite all the negative feedback we received (and despite all the effort my colleagues are currently putting into fixing issues the Last.fm community has pointed out), there are already several indicators that the new site might be an even bigger success than we would have hoped for, and most of all it has paved the way for a lot more to come. It’s never been more fun to work at Last.fm! Btw, check out the jobs at Last.fm :-)

Recommended Book: Probability and Statistics

2008-07-21T00:10:00.007+01:00

The last two weeks I was camping north of London. Because I didn't want to drag a whole library with me I decided to take along only one book on statistics. From the books next to my bed I selected the lightest one which is Schaum's Outline: Probability and Statistics. And it was one of the best packing decisions I made.

It all started in one of my favorite book stores in London where I almost ignored the book in the first place. The book stuck out in the shelf of statistics books because of it's height and because of it's ugly front cover design. The pages felt like those of a telephone book. My expectations were as low as it's price tag (which was £12) and I ignored it. However, the shop only offered a very limited selection of books on statistics. So eventually I turned back to it out of curiosity wondering how bad a book on statistics could be. And then I stumbled upon one of the many interesting problems in the book and tried to solve it, and then I found the next interesting problem, and decided to move to the attached cafe. By the time the store was about to close I didn't want to part with the book. (One example of the fun problems in the book is: given 6 randomly sampled observations from a continuous population what is the probability that the last 2 are higher than the first four? (One way to solve it is to use calculus another way is to use combinatorics.))

The book does not only feature the ugliest front cover of any of the books I ever owned, it also contains many typos. And every time I tried to use the index it seemed to point me to random pages. For example, one typo can be found in the introduction to the multinomial distribution where they forgot an important exclamation mark. I don't understand how they manged to include so many errors in this second edition. In fact sometimes I wondered if errors were included to keep the reader alert. However, none of the errors I found were hard to identify as such. (In the case of the multinomial distribution formula there is an example just a few lines below the typo which uses the correct formula.)

Overall the book is amazing. I had a hard time choosing between packing up my tent in the rain or waiting for the rain to stop while reading a few more pages and drinking some hot tea (btw, I also highly recommend Trangia stoves).

The book is definitely suitable for people (like me) who work with probabilities and statistics on a daily basis but feel like they lack a solid foundation. It helps if you've had a basic course on statistics a long time ago and just want to refresh your knowledge. However, I think it is also largely and easily accessible to anyone who has not had any courses on statistics (although some understanding of calculus will help a lot).

The best part of the book is that it features lots of practical problems that help understand the theoretical concepts. The book is also structured in a way that makes it very easy to spend 30 minutes or less at a time with it. The topics covered include nonparametric tests, curve fitting, regression, and hypothesis testing.

Btw, I can also recommend: Old Man, Loch Lomond (and the West Highland Way), walking around the beaches of Holy Island at low tide (and reading books on the beach in front of Bambourgh), walking along some mountain ridge anywhere in the Highlands, listening to the choir in Durham cathedral, sleeping next to Hadrian's Wall, extreme hill walking in the Lake District... and Gwen recommend's Bill Bryson: Notes from a small Island.

VocaListener

2008-07-01T00:50:00.003+01:00

Yesterday and today I had the pleasure to spend some time with Masataka Goto talking about how MIR technologies are changing how we create and enjoy music, and in particular what Masataka calls active music listening. It was also great to get some updates on what's happing on the other end of the world.

One thing I found particularly interesting is the VocaListener project which Tomoyasu Nakano (who recently finished his PhD) and Masataka Goto are working on. VocaListener is based on Vocaloid 2 and synthesizes a singing voice that is very hard to discriminate from a real singer.

Here is a video featuring the synthesized voice using Vocaloid 2 (and special techniques to tune the parameters).

VocaListener received a lot of coverage in Japan and some of it has been translated to English, for example: here, here, and here.

Last.fm's API, Python, and tagging behaviour (Part 2)

2008-06-29T06:48:00.006+01:00

Update: (2008/08/25) I fixed the same kind of bug I had in the previous post on this topic. While fixing it I decided to rerun it with a sample of 5000 instead of 2000 users. The code is fixed and the data is updated.

Last night I was a bit tired and quickly concluded "the numbers show...". But do they really?

To find out I use Python to do some simple statistical analysis to add some weight to the claim that older Last.fm users have a larger vocabulary and tag items more frequently.

First, I compute 95% confidence intervals for the percentage of non-taggers in each age group. Seeing the large margins (in the table below) helps explain why the age group 25-30 has a higher percentage than the age group 19-22. However, it doesn’t help explain why the age group 22-25 has a lower percentage. (I’d blame that on the relatively small and skewed sample, and I’d argue that they are still reasonably similar, both in average age and deviations of the percentages). (With the larger sample size this is not the case any longer.)

Computing the confidence interval is very easy. A user is either a tagger or not. The probability within an age group can thus be modeled with a Bernoulli distribution. The 95% confidence intervals for a Bernoulli distribution can be computed with:

z = 1.96 # norminv(0.975) for a 95% confidence interval
def binom_confidence(p,n):
    if n*p*(1-p) >= 5:
        return z*(p*(1-p)/n)**0.5

Btw, I couldn’t find the equivalent to the Matlab norminv function in Python. Any pointers would be appreciated!

To test the hypothesis that the vocabulary size of a tagger depends on her or his age I test the following: Given my observations, are all age groups likely to have the same vocabulary size, i.e, are the differences I observed just random noise? Since the distributions within each age group are far from Gaussian I can’t use a standard ANOVA. Instead I use the non-parametric version of a one-way ANOVA which is the Kruskal-Wallis test. In particular, I use the test to compute a p-value. The p-value is the probability that I would have made the same observation if the hypothesis that there is no difference between age groups would be true. (Thus smaller p-values are better. Usually one would expect at least a value below 0.05 before accepting an alternative hypothesis.) In this case the resulting p-value is nice and low indicating that it's extremely unlikely that older users don't have larger vocabularies.

Here are the results, and below is the Python code.


age   || % non taggers || tagger's median 
                          vocabulary size
14-19 ||     41.3-48.1 ||  6
19-22 ||     37.5-43.6 ||  7
22-25 ||     40.6-47.3 ||  9
25-30 ||     34.8-41.4 ||  8
30-60 ||     28.4-36.0 || 13
Kruskal-Wallis p-value: 1.04e-008

from scipy.stats.stats import kruskal
from numpy import asarray

def print_stats(age_tags):
    age_groups = (14,19,22,25,30,60)
    ll = []
    print "age   || % non taggers || " + \
           "tagger's median \n" + \
           "                          vocabulary size"
    for i in xrange(0,len(age_groups)-1):
        nonzeros = [];
        zero_count = 0
        for j in xrange(age_groups[i],age_groups[i+1]):
            for item in age_tags[j]:
                if item!=0:
                    nonzeros.append(item)
                else:
                    zero_count += 1
        conf = binom_confidence(
               zero_count/float(zero_count+len(nonzeros)), 
               zero_count+len(nonzeros))
        ll.append(nonzeros);
        print \
    "%d-%d || %8.1f-%.1f || %2d" % \
            (age_groups[i],age_groups[i+1], 
            (zero_count/float(max((1,len(nonzeros)+\
                                   zero_count)))-conf)*100, 
            (zero_count/float(max((1,len(nonzeros)+\
                                   zero_count)))+conf)*100, 
            median(nonzeros))
    p = kruskal(*(asarray(ll[i]) for i in 
                 xrange(len(age_groups)-1)))[1]
    print "Kruskal-Wallis p-value: %.2e" % p

~~Btw, that part where I use eval to convert my lists into function arguments could hardly be any uglier. I’m sure there must be a better way of doing that?~~ (Thanks Klaas!)

Last.fm's API, Python, and tagging behaviour

2008-06-29T02:05:00.013+01:00

Update: (2008/08/25) I fixed the bug pointed out by thisfred. And I noticed that what I thought was the percentage of non-taggers was actually the ratio of non-taggers vs taggers... I changed that now.

My colleagues completely redesigned the Last.fm API. Inspired by their efforts and all the amazing things the Last.fm community has already built with the old API I decided that I wanted to try doing something with the API as well. The first thing that came to my mind was to use the public API to show that younger people have a smaller tagging vocabulary than older people. I couldn't figure out how to get a user's age from the new API so I used the old one. Anyway, here are the results and I also included the Python script I used. (Btw, any feedback on my Python coding is very welcome, I'm still very much a Python newbie.)

I crawled about 2000 users starting with RJ as seed. The first column is the age group, the second column is the ratio of users who haven't used any tags vs number of users who have used tags, the last number is the median number of unique tags which users who have applied tags have used.


14-19: zeros: 0.44 (120/155), median tags: 6
19-22: zeros: 0.42 (153/215), median tags: 6
22-25: zeros: 0.38 (114/184), median tags: 9
25-30: zeros: 0.43 (141/188), median tags: 10
30-60: zeros: 0.31 (79/179), median tags: 11

The numbers show that older users tag more and apply more unique tags.

from xml.dom import minidom
from urllib import quote, urlopen
from time import sleep
from numpy import median
from collections import defaultdict

seed = 'RJ' # start with Last.fm's CTO
MAX_RETRIES_URL_OPEN = 5

def get_xml(url):
    for i in xrange(MAX_RETRIES_URL_OPEN):
        try:
            sleep(1) # be nice!
            return minidom.parse(urlopen(url))
        except IOError:
            print "(%d/%d) Failed trying to get: %s." % \
                (i, MAX_RETRIES_URL_OPEN, url)

def get_friends(user, friends, ignore_friends):
    url = u'http://ws.audioscrobbler.com/1.0/user/' \
            + quote(user) + u'/friends.xml'
    xmldoc = get_xml(url)
    xmlusers = xmldoc.getElementsByTagName("user")
    for user in xmlusers:
        u = user.getAttribute("username")
        if u not in ignore_friends:
            friends.add(u)
    print "%d/%d" % (len(friends), len(ignore_friends))
    return friends    

def get_age(user):
    ''' returns zero if user has not set his or her age '''
    url = u'http://ws.audioscrobbler.com/1.0/user/' \
            + quote(user) + u'/profile.xml'
    xmlage = get_xml(url).getElementsByTagName("age")
    if len(xmlage)==0: return 0
    return int(xmlage[0].firstChild.nodeValue)

def get_tags(user):
    url = u'http://ws.audioscrobbler.com/1.0/user/' \
        + quote(user) + u'/tags.xml'
    return len(get_xml(url).getElementsByTagName("tag"))
        
def print_stats(age_tags):
    age_groups = (14,19,22,25,30,60)
    for i in xrange(0,len(age_groups)-1):
        nonzeros = [];
        zero_count = 0
        for j in xrange(age_groups[i],age_groups[i+1]):
            for item in age_tags[j]:
                if item!=0:
                    nonzeros.append(item)
                else:
                    zero_count += 1
        print \
    "%d-%d: zeros: %.2f (%d/%d), median tags: %d" % \
            (age_groups[i],age_groups[i+1], 
            zero_count/max((1,float(len(nonzeros)+ \
                             zero_count))),
            zero_count, len(nonzeros), median(nonzeros))


users_notvisited = set([seed])
users_visited = set()

while len(users_notvisited)>0 and \
    len(users_notvisited) + len(users_visited)<2000:
    user = users_notvisited.pop()
    if user not in users_visited:
        users_notvisited = \
            get_friends(user, users_notvisited, \
            users_visited)
        users_visited.add(user)

users = users_notvisited.union(users_visited)

age_tags = defaultdict(list)
i = 0
for user in users:
    i += 1
    print "%d/%d" % (i, len(users))
    age_tags[get_age(user)].append(get_tags(user))
    if i % 5 == 0:
        print_stats(age_tags)

Matlab, Python, and a Video

2008-06-26T23:29:00.006+01:00

I've been using Matlab extensively for probably almost 10 years. I have written more lines of code in Matlab than in any other language. I always have at least one Matlab application window open. I've probably generated at least a few million Matlab figures (one of my most favorite Matlab functions is close all). I've written three small toolboxes in Matlab (and all of them have actually been used by people other than me). I've told anyone who was willing to listen that I couldn't have gotten even a fraction of my work done without Matlab. In fact, 3 times in a row I convinced the places I've been working at that I needed a (non-academic) license for Matlab and several of its toolboxes. I even had a Matlab sticker on my old laptop for a long time. I frequently visited the Matlab news group and I'm subscribed to several Matlab related blogs. If I would have needed to take a single tool with me on a remote island it would have been Matlab. I guess it's fair to say I was in love with Matlab.

However, I always felt that it wasn't a perfect relationship. Matlab is expensive. Matlab is not pre-installed on the Linux machines I remotely connect to. In fact, installing Matlab on Linux is a pain (compared to how easy it is to install it on Windows). Furthermore, not everyone has access to Matlab making it harder to share code. Finally, Matlab can be rather useless when it comes to things that are not best described in terms of matrices that fit into memory, and I can't easily run Matlab code on Hadoop.

I had been playing with Python out of curiosity (wondering why everyone was liking it so much) but I guess I was too happy with Matlab to seriously consider alternatives. But then Klaas showed me how to use Python with Hadoop. Within a very short time I've started to use Python more and more for things I usually would have done in Matlab. Now I write more Python code a day than Matlab code. I still use Matlab on a daily basis, but if I had to choose between Matlab and Python, it would be a very easy choice. SciPy and related modules are wonderful. If I'd redo my PhD thesis, it wouldn't include a single line of Matlab code and instead lot's of Python code :-)

Btw, James pointed me to the following visualization showing activities and shared code of Python developers over time. This is by far the best information visualization I have seen in a very long time. I really like the idea and implementation. I wonder if something similar could be done for a piece of music where the coders are replaced with instruments, and the files are replaced with sounds.

code_swarm - Python from Michael Ogawa on Vimeo.

ISMIR'08 Student Travel Award

2008-06-25T17:24:00.004+01:00

It's wonderful to see Sun Microsystems sponsoring student travel awards for this year's ISMIR. Submission deadline for applications is July 4th - very soon!

I highly recommend applying for an award even if it might seem like a bureaucratic burden. Sure, any student who gets an award will still need to find additional sources of funding. However, it's always easier to find smaller amounts of money, and as a researcher it is not unusual to spend a lot of time writing project proposals asking for grants. Student travel awards are a great way to start practicing! And writing one page yourself and asking your professor to write a recommendation is actually not a lot of effort. Btw, professors deal with recommendations very frequently, they shouldn't complain if you ask them to write you one :-)

I remember when I received a student travel award for the ACM KDD 2003: It was lots of fun because students who won the award also got a chance to participate in the organization. And helping in the organization of such a huge conference was a great experience.

Late Night Thoughts

2008-06-24T00:27:00.005+01:00

Last night I went to bed and fell asleep listening to a podcast that Paul recommended. I didn't stay awake for long but I remember that the interviewer seemed skeptical about some of the ideas of open research.

I guess it's natural to be skeptical when something is radically different to what we are used to. I wonder what Newton would have said if someone would have told him to publish or perish and that there are plenty of good journals that publish articles within a year of submission (including the review process). It took Newton almost 22 years to publish his findings, and that was not so unusual back in 1687.

Using those two data points (22 years in 1687 and 1 year in 2008) a simple linear model would suggest that a submit-publish cycle might take less than 3 months in 2020. While I can't see how that linear model is a good fit I could easily see how pushing a publication out within 3 months could be achieved with tools similar to what we know as blogs today.

Furthermore, looking at historic data it's also not too hard to see that unlike some might expect we will see fewer disputes with respect to who discovered what first.

Btw, tonight I'll try to stay awake as long as possible with this podcast about earworms.

More MIR related PhDs

2008-06-19T20:57:00.006+01:00

I just updated the list of MIR PhDs. The most notable update is the thesis by Adam Berenzweig. Unfortunately his thesis is not publicly available for download, but it seems like he has found an answer to why we've been observing hubs using certain similarity measures for music. Quoting from his abstract: "A practical problem with this technique, known as the hub phenomenon, is explored, and we conclude that it is related to the curse of dimensionality."

I'm sure there's still plenty of dissertations missing in the list and I'll be happy to add any that are sent to me. (I'll also be happy to update any broken links or missing information...)

Btw, the list is slowly approaching 100 entries now. The Matlab script I wrote to generate the html files and statistics now takes almost 5 seconds to complete.

Myths about Last.fm tags

2008-06-13T17:27:00.004+01:00

Today I was pointed to the following: "Last.fm has thousands of tags, unfortunately they are all pretty bad." (A statement made in this video of a very interesting talk about autotagging and applications that can be built using tags, around minute 51.)

I think this needs some clarification: Last.fm has a lot more than just a few thousand tags. The 1,000,000th unique tag applied by a Last.fm user was earthbeat about half a year ago.

Related links: fun stats on Last.fm tags, Last.fm's multi-tag search.

Machine Learning Rant

2008-06-10T10:22:00.005+01:00

This rant is inspired by this wonderful blog post which I found through Greg Linden's blog.

Most people who've worked with me might know that I'm very skeptical about using machine learning algorithms. In most of my work I've avoided using them as a solution to a problem.

My problem with machine learning algorithms is the way they are used. I think a beautiful example to illustrate this failure to use them is genre classification. Countless papers have been published claiming around 80% classification accuracy. There have even been a number of papers indicating that these 80% are close to the disagreement level between humans (i.e. the machines 80% are as good as the genre classification performance of any human).

Anyone who has seriously looked at such trained genre classifiers in more detail will have wondered why the measured accuracy is almost perfect and yet the results on a new data set are often not so satisfactory. The simple solution to this specific problem is that instead of genre classification accuracy most researchers have been measuring the artist classification accuracy because training and test set often included pieces from the same artists and most pieces of an artist belong to the same genre. (I've been arguing for the use of an artist filter since 2005, and yet I still see lots of papers published which ignore the issue completely...)

Anyway, the point is that people using machine learning algorithms often consider their problem to be solved if their model performs well on their test set. I often have the impression that no effort is made to understand what the machine has learned.

Most of the time when I explain that I'm skeptic about machine learning I'm confronted with raised eyebrows. How can someone seriously challenge best practice methods? Surely anyone who is skeptic about machine learning has not understood what it is about?

Despite all the responses I've received so far, today after having read the wonderful blog post mentioned above, I feel like there are lots of people out there (many of which are surely a lot smarter than me) who are skeptic about the use of machine learning algorithms. The next time I'm trying to explain why I think that blindly using and trusting a machine learned model is not a solution I'll point to Google's ranking of search results :-)

Having said that, I think there is a lot of potential for machine learning algorithms that generate human readable explanations for the models they generate. In particular, in the same way any human data analyst uses experience, common sense, and data to justify any decision when building a model, I'd like to see machine learning algorithms do the same. In addition, it would be nice if (like a human learner) the algorithm could point to possible limitations which cannot be foreseen based solely on the given data.

I guess I should also add that all of this is just a matter of definitions. With machine learning I mean black boxes which people use without bothering to understand what happens inside them. In contrast to my definition, many consider statistical data mining to be an important part of machine learning (which I sometimes don't, because it requires specific domain knowledge, as well as human learning, understanding, and judgment). Furthermore, I have no doubts that Google applies machine learning algorithms all the time, and that combining machine learning with human learning is a very natural and easy step, unlike my above rant might indicate.

Quaero

2008-06-09T15:25:00.005+01:00

While trying to catch up with my emails I stumbled upon Geoffrey Peeters' mail in the Music-IR list. He still has 2 open positions in the Quaro project, which is a huge project I know almost nothing about. A quick Google search revealed that Quaro is French, it's worth 200M€, will last 5 years, has 24 partners, and is Latin for "I'm searching".

I guess only a tiny fraction of Quaero will be dealing with music and MIR research. However, a tiny fraction of 200M€ is still huge, and the topics Geoffrey mentions are very interesting: genre/mood classification, music similarity, chorus detection, ... Quaero also seems to have a strong emphasis on gathering and annotating data that can be used to evaluate and improve algorithms.

Btw, there is some more information about Quaero in this presentation.

I found it interesting that the administrative work on Quaero started in August 2005. Operational work (research etc) will start November 2008. I wonder how much overhead costs they have and how much flexibility they still have?

On slide #7 the concrete innovations are mentioned (and 5 examples are given). They mention detecting soccer goals in a recording of a soccer match, identifying the songs in a sound track, and using automatic translation to enable people to search in different languages. I wonder if they had to finalize the innovative outcomes of Quaero back in 2005?

Overall it seems like part of Quaero's strategy is to enter markets which have already been covered by many startups as well as large players such as Google. In particular, it almost seems like Quaro is intended to help Exalead survive a bit longer? (Exalead is a French search engine startup. I briefly tried their search and it seemed a bit slow. When searching for "music information retrieval" they fitted 1.25 proper search results on my 15.4" screen the rest was ads and other information I don't care about... I'm not surprised they aren't doing well - not even in France.)

Anyway, IRCAM is leading Quaero's music technology, so I expect we'll see lots of great outcomes.

Open Research

2008-06-01T10:30:00.006+01:00

In response to what Paul just blogged about...

I'm curious how in the long run blogging about ongoing research will transform communication within the research community. I've already been curiously following blogs by some PhD students such as Mark or Yves who are very open about their ongoing work and ideas.

I don't think blogs can replace publishing papers at research conferences or journals. However, I wouldn't be surprised to see more and more references to blog posts in conference papers in the future.

Blogs would be the perfect communication platform for researchers if:

there would be a guarantee that a blog post will be around for ever (i.e., that researchers in 20 years from now can go back and look at it)
if it would not be possible to alter any information published on a blog (or at least to detect if something has been altered), this includes not being able to change the date when an idea or result was first published

It seems that one way to overcome both limitations would be to have authorities frequently crawl, store, and index blog posts related to research. Another option might be to have something like a "research mode" on popular blogging service providers such as blogger.com: if the blogger opts into this mode, then the researcher wont be able to ever change his blog posts (including deletion) and the blog posts will be indexed by search engines.

However, even as publishing preliminary research results on blogs becomes an accepted standard I wouldn't be surprised if some unfortunate researchers without ideas of their own consider an idea published on a blog to be not published at all and try to publish it at a conference with their own names on it without referencing the source. I'm sure though, that such attempts would ultimately fail as the blogging research community would point them out quickly.

I wouldn't be surprised if a few researchers starting to use blogs to communicate ongoing results will trigger a snowball effect. For example, if Paul starts blogging about ongoing research that someone else is currently working on, wouldn't that other researcher feel urged to publicly state that he or she is also working on the same topic? Otherwise, by the time this other person publishes results, everyone might think that those ideas were just copied from Paul's blog?

Looking at how research has developed over the past centuries, the direction we have consistently been heading in seems very obvious: more openness and getting results out faster. Research blogs seem like a very natural next step in the evolution of communication in the science community.

Things to do in London

2008-05-28T19:30:00.003+01:00

Search Sounds:
Thursday, 5 June 2008, 19:00 - 21:00

The Omras2 team is showing off some of the fun things they've been working on. Unfortunately I won't be in London on that day.

ISMIR'08 Reviewing

2008-05-22T23:21:00.007+01:00

ISMIR 2008 seems extremely well organized. I've been only watching from the side line this year (for the first time since 2002 I haven't submitted a paper myself) but the bits and pieces I've seen seem great.

I like how reviews are double blind this year.

I like how authors get a chance to respond to reviews. And reviewers get a chance to reconsider their ratings after seeing the response of the authors, and what their fellow reviewers wrote.

However, and this strikes me as fascinating: I get to see the name of my fellow reviewers! (To be more accurate: I only get the see reviews and names of reviewers who reviewed the same content I did.) First I thought it was a bug in the system (I even felt the urge to instantly report it to the program chairs). But it seems like the whole system is designed around exposing the real names of the reviewers to the fellow reviewers: I was just sent an email with the email addresses of all my fellow reviewers.

I think I've never experiences such openness in any of the review processes I've been involved in. It's fascinating, but it makes me wonder if that might lead to reviewers being more reluctant to write critical remarks in the future? Especially in such a small community as the ISMIR community, one of the fellow reviewers might be a colleague of one of the authors etc. While I think it's a good idea to publish reviews and a response to those (I did so for one of my publications last year here), I don't think it's necessarily a good idea to expose the reviewers without their consent.

Hadoop in London

2008-05-21T15:43:00.006+01:00

August 19th there will be a Hadoop workshop somewhere in East London which some of my colleagues are helping to organize. Details can be found here. Further announcements will follow. The speakers include Doug Cutting (works at Yahoo and leads the Hadoop project). Johan and Martin will be talking about how we use Hadoop at Last.fm. Applications at Last.fm range from counting how many users listened to an artist to learning from skipping behaviour to improve our radio streams.

Islands of Music

2008-05-20T18:08:00.007+01:00

It all started about 7 years ago when I was an intern working in Aberdeen (Scotland). Aberdeen is a rather quiet town. It's beach was a wonderful place to sit and think about my MSc thesis which I was working on at that time.

At first I was very enthusiastic about the idea of using a metaphor of islands to visualize a music collection. However, a few years and many not so perfect island of music maps later my enthusiasm had faded away and I had more or less given up trying to map the entire music world onto a map.

Recently at Last.fm we launched our playground and one response we heard a few times was that it wasn't very colorful. (All three initial demonstrations were basically simple lists of artists or tracks.) At the same time I had a dataset of users described in terms of tags loaded in Matlab. Previous results using simple clustering algorithms had shown that it was very easy to extract meaningful clusters.

Now there's an islands of music demonstration on playground. There's also a journal entry talking about how it was computed.

Last.fm's Playground

2008-05-14T20:16:00.005+01:00

At Last.fm we launched a public version of our playground. One of the motivations has been to be able to gather more user feedback on some of the ideas we have. In particular, to find out if ideas we have are sticky or if they are just nice ideas. Eventually, all sticky ideas will (hopefully) find their way into the main site.

I'm very excited that we now have this platform. So far we've been running most of our experiments and evaluations in the background but now we can directly interact with users and expose possibilities we have even if there is no straight forward way to integrate them into the main site.

Special thanks to Klaas Bosteels who's done some amazing work during his 3 month internship at Last.fm building up playground. Btw, if you know someone who might be a good fit for an internship at Last.fm, let me know! (Java/Hadoop, PHP, Python, and being able to work efficiently on Linux systems is more or less a requirement.)

Btw, from a MIR research perspective I believe the multi tag search is the most interesting. Primarily because it indicates how much potential tags have.

The Echotron

2008-05-13T14:49:00.004+01:00

Having read Paul's post yesterday, I instantly applied for a beta invite. In fact, while writing this blog post I'm listening to a very pleasant stream of music from the echotron. I'm impressed! It couldn't be much easier to get started. It's very easy to quickly add a bunch of artists I like to my profile, and I even get to listen to any search results. The recommendations are pretty good, definitely a lot better than many others I've seen out there... and they are sometimes rather different from the ones I get at Last.fm (in a good way). However, despite being biased, I'd still argue that Last.fm's recommendations are better ;-)

Anyway, it seems the echotron could easily turn into more than just a site built to showcase the echo nest's APIs.

Jobs@Last.fm

2008-05-12T15:17:00.004+01:00

If I'd know how to code C++ I'd instantly apply for this job:

Software Engineer, Data and Recommendations

We’re looking for someone who can help us improve our recommendation engine, someone who knows how to handle large amounts of data, and knows C++ really well. You’d be joining a small, enthusiastic, and flexible team.

Crunching terrabytes, dealing with computational complexities on a daily basis, developing applications where you can hear the difference (and can measure the feedback from millions of users), working on some of the most elegant C++ code every written, designing and building the next generation of Last.fm's recommendation engine, ...

Automatic accompaniment

2008-05-08T10:18:00.007+01:00

Here's a great video demonstrating how much fun automatic accompaniment systems can be:

The video demonstrates Andrew Robertson's (C4DM) live drumming controlled automatic accompaniment system: "James Sedwards (guitar) and Jeremy Doulton (drums) improvise a rock track by recording loops of bass and guitar. This is done in Ableton Live with B-Keeper controlling the tempo so they stay in time even when speeding up."

Here's a link to one of the publications describing the system (PDF).

Applying audio-based similarity

2008-05-06T14:29:00.008+01:00

Applications of audio-based similarity still seem rather rare nowadays. Once in a while a startup claims that they're using it for recommendations, but looking at their results might suggest that they are just using metadata instead (for example, see this example which Paul recently blogged about).

Anyway, here is an example where 100% pure audio-similarity is being used: FM4 soundpark. The feature launched today. For anyone who doesn't read German: soundpark is the number one place for new Austrian artists to expose themselves online. Soundpark has been around for ages. Long before Myspace or similar sites became popular. Soundpark has a devoted community and is well integrated into one of the most popular radio stations in Austria (the FM4 station which reaches out to younger demographics favoring alternative/indie music).

However, attention is usually only paid to new releases and old ones can get buried quickly. To help users navigate and find content researchers from the legendary OFAI (including Martin Gasser and Arthur Flexer) in collaboration with Gerhard Widmer's department of computational perception (including Dominik Schnitzer) have helped FM4 integrate audio-similarity into their system. There is a bit more information on OFAI's page here and some more information in German can be found on orf.at.

To each song on Soundpark users will get 3 acoustically similar songs, and there seems to be a feature that allows creating a playlist by defining a starting and end song (but I haven't found that feature yet).

Soundpark hosts about 5000 artists with each a few tracks in average. (If I'm not mistaken they got about 8800 tracks.) They are constantly growing. According to the press releases they have a number of plans to add additional features to make it easier to navigate their content and discover interesting artists. Great news! Makes me especially happy knowing that it's former colleagues who are building this.

Tag of the Day: Death Country

2008-04-25T22:08:00.006+01:00

Today I learned from Mischa about death country. Apparently it's related to gothic americana... which btw sounds nothing like gothic.

The Echo Nest on Techcrunch

2008-03-30T02:07:00.003+01:00

The Echno Nest made it on Techcrunch!

Interestingly Techcrunch seems to think that The Echo Nest is about genre classification (second sentence in their blog post). I doubt they heard that from Brian.

One reader raised an interesting question: "[...] do we really need a computerized pandora?" It's nice to see something like this discussed outside of the music-ir mailing list!

I really like The Echo Nest's very open approach to demonstrate the technologies they got. Besides being of interest to anyone in MIR I'm sure it also makes it a lot easier to attract interest to their APIs and communicate what kind of problems can be solved with them. But I'm not sure if I like the redesign of their webpage, I somehow had gotten used to the simplicity of their original one. I'm very curious what their next demonstrations will be... maybe some web crawling based stuff?

See also Paul's post about this is my jam.

Thoughts on Autotagging

2008-03-07T01:28:00.005Z

I've recently spent a lot of time looking at tags. I'm fascinated by how creative listeners are when it comes to inventing new tags for the music they care about. And I'm fascinated by how small communities adapt these newly invented terms as quickly as they tune into a new radio station. This constantly changing and seemingly endless vocabulary of words to describe music makes me wonder if research on automatically classifying or tagging music will ever be able to catch up.

If you think research is catching up, check out the 22,703 shades of metal, or the 11,929 shades of pop, or the 29,940 shades of rock at Last.fm.

See also Paul's experiments on creating a taxonomy of metal and gothic tags.

See also Mike's recent results on autotagging of music (which includes some impressive numbers).

Simple Stuff

2008-02-11T12:07:00.000Z

Jeff Hammerbacher from Facebook’s team on data and analytics gave an interesting talk and it can be viewed here (which might be broken, here is a direct link). I’d highly recommend it to anyone interested in what happens behind the scenes on websites dealing with a lot of interesting data.

Some of the stuff he talks about is related to what happens at Last.fm. For example, at Last.fm we also use Hadoop.

I found the part around minutes 33-38 most interesting. He talks about the skills needed to work with their data. He mentions that being able to write code is a must. He mentions that most of the people on his team write more code than they did in their previous positions. And he mentions that very simple statistical tools such as curve fitting and understanding statistical significance can be used to solve most of their learning from data challenges. He talks about how visualizing data is very important (e.g. to identify and understand outliers).

Obviously music recommendation is a much more complex problem than any of the challenges Facebook is facing. Scrobbles, tags, skipping behaviour, etc require very different treatment than the data Facebook gathers. Or maybe not?

To some of the most interesting things I’ve had the pleasure to work on at Last.fm I’ve only applied very basic statistical techniques: non-linear curve fitting and measuring the significance of improvements. However, while the “machine learning” parts could hardly be any simpler, the complexity of dealing with terabytes is something completely different.

Btw, at Last.fm we are hiring someone to work on data and analytics and we also got a position related to data warehousing. Both positions would be facing challenges very closely related to the stuff Jeff talks about. Except that the data we have is a lot more interesting! It made me a bit sad to hear that one of the things they were actually looking at is communication streams between universities… you’d think they’d have a lot more interesting insights to gain? ;-)

Web Services for Researchers

2008-02-07T00:23:00.000Z

It just occurred to me how soon every research lab might be offering a long list of web services. Bandwidth is not a limiting factor. Building a web service is not that hard as it was 5 years ago. It's a great way to share without giving away code (and IP). It's also user friendlier as it doesn't require installing someone else's most likely buggy code on your own system. And it's potentially a great way to make money, too!

I wonder if I'm the last one to realize this? :-)

Anyway, what has helped me realize this was Thomas Lidy's announcement of his teams new web service, and The Echo Nest's web services that I recently found out about through Paul. Both allow you to upload music, extract features from the audio signal, and send them back to you.

I just gave both a try and they worked very smoothly. The two pictures below show results for the same track. The first one is created with the processing music visualization tool provided by The Echno Nest, the second one using Matlab to analyze the fluctuation pattern that Tom's tool extracts.

I wonder if the Echo Nest's service would crunch 100k tracks. (I believe there are at least a few research groups already dealing with collections beyond 100k tracks.) The service Tom announced is limited to 100 tracks/day and a maximum of 300 total per voucher (which requires you to sign up with your email address). Anyway it's a great start. And it seems that Tom will soon be making more announcements on further services that allow anyone to visually organize their music collections using a metaphor of geographic maps. Nice!

Btw, the Last.fm web services also seem to be very popular amongst researchers, at least some have been hitting them very hard ;-)

And one of the most eagerly anticipated web services is probably the MIREX DIY web service which was announced at ISMIR 2007 by Stephen Downie's team. The service will allow researchers to upload their implementations and receive evaluation results in return. Which will make it very easy for researchers to test if they are heading in the right direction.

2 PhDs and 1 MSc

2008-02-05T22:30:00.000Z

Matthew Davies recently made his PhD (Towards Automatic Rhythmic Accompaniment) available online.

Arturo Camacho recently announced on the music-ir list that his PhD (SWIPE: A Sawtooth Waveform Inspired Pitch Estimator for Speech and Music) and the corresponding Matlab code are available online.

Claudia Wronski recently finished her Master's thesis (in German) on "Die veränderten Zugriffsmöglichkeiten auf die Ressource Musik – Auswirkungen auf das Kauf-, Nutzungs- und Rezeptionsverhalten der Musikkonsumenten" (freely translated: the changing access to music and its impact on shopping and consumption habits of music listeners). She covers topics such as how the music culture is changing, the crisis of the music industry, the long tail, the emancipation of artists, and the future of music.

Claudia is an active Last.fm user, and has successfully been leading a very interesting user group: The Special Interest Tag Radio Collective.

Fun with Audio Analysis

2008-01-27T16:31:00.000Z

Paul just blogged about it: The Echo Nest are demonstrating some of the stuff they have been working on. The one I like best is "automatic song" (which they say is a composition from automatically combining about 50 songs).

I'm curious what impact their API to extract features from audio will have on MIR research. Seems like they are also targeting artists who use processing to visualize music content. I'd like to see videos of their music visualizations.

The future of MIR?

2008-01-23T07:36:00.000Z

The European Commission (Unit DG INFSO/E2) is planning to invest 2B€ in research and development (2009-2010, FP7). They recently sent out emails asking for comments from researchers:

“[…] we would very much like to hear your views on what you think are the most pressing problems and ripe opportunities in your field. We are interested as much in scientific advances as in innovative applications or infrastructural initiatives in domains where they are likely to have a large positive impact.”

(The context is “knowledge technologies, interactive media and online content”.)

It’s an interesting question to think about.

I’m tempted to point them to work on unstructured collaborative tagging of music, ie, folksonomies. There are lots of interesting opportunities there, some of which might still be there by the time the FP7 projects start.

If you have any ideas, let the EC know:
http://cordis.europa.eu/ist/kct/fp7-consultation.htm

UPDATE: Jeremy points out some interesting topics the EU should funding in the comments.

London Calling

2007-12-11T17:20:00.000Z

Last.fm is expanding! We are seeking over 20 people to join in on the fun. The complete list is here.

And one of the open positions is related to MIR research! :-)

Btw, have I mentioned that working at Last.fm has been so much fun that I've totally forgotten about updating this blog? We get free organic fruit, we got a table tennis and table soccer table, we get enough data every day to keep our Hadoop clusters busy for hours analyzing it, we get to design and build the next generation of music recommenders and contribute to lots of other fun projects at Last.fm, we got users supplying us with instant feedback on any changes we make, and we also got the Best American pizza place and the Best kebab just around the corner (they actually mention they are the best in their name, so it must be right).

See also my previous post about MIR research at Last.fm.

Somewhat related: Christopher Raphael just posted a tenure-track position at Indiana university. Unfortunately no URL was given, but I guess the position only targets people who have already subscribed to music-ir. It's nice to see so many opportunities in MIR.

Life after ISMIR 2007

2007-10-01T20:18:00.001+01:00

As usual after an ISMIR I’m totally burnt out. The program was intense. My brain is still trying to absorb all the interesting conversations, ideas, results, and theories that are echoing in my ears. Some of the things I saw made me realize how close we are to reach some of the scenarios we’ve been talking about in the last 5 years (for example, Klaas Bosteels demonstrated a content-based playlist generator that learns from user feedback which he implemented on his tiny MP3 player). Other things made me reconsider assumptions I’ve made in the past (for example Hamish Allan et al. presented interesting work on music similarity: "Methodological Considerations in Studies of Musical Similarity").

It was also wonderful to have the opportunity to spend time with people that seem very familiar although I only see them once every one or two years. It’s strange how reading papers can make the authors seem so familiar.

One of the many highlights of ISMIR 2007 was when Don Byrd announced the locations of future ISMIRs. I was particularly happy to hear that ISMIR 2009 will be organized by Masataka Goto et al. (in Japan!). However, ISMIR 2008 organized by Youngmoo Kim, Dan Ellis, and Juan Bello et al. (US) and ISMIR 2010 organized by Frans Wiering et al. (Netherlands) will surely be great, too. It’s good to see ISMIR, for the first time in its history (afaik), planed out for 3 years in advance. And I heard some rumors that ISMIR 2011 will be held in Vienna again because it was such a huge success ;-)

ISMIR Highlights

2007-09-25T17:48:00.000+01:00

ISMIR is only halfway through and I can’t believe how many interesting things I’ve already missed. I guess it’s unavoidable given parallel sessions and so many poster presentations in limited time. Nevertheless, my brain is already overflowing and I feel burnt out. In fact there were so many interesting presentations, that I don’t have enough time to write all of them down (although it would be a great way to remember them).

One thing that might have a very high impact is that IMIRSEL is just about to launch their online evaluation system. Researchers will be able to submit their newest algorithms and find out how well they do compared to others. I think having an evaluation system like that is actually worth a lot to the whole community, and I can well imagine that (once all issues are solved) research labs will have something like a paid subscription which allows them to use a certain amount of CPU time on IMIRSELs clusters. However, to be truly successful they’d need to be 100% neutral and transparent. (Which I think means they shouldn’t have IMIRSEL show up in their rankings, and they should clarify how One Llama is linked to IMIRSEL.)

I also liked the poster Skowronek, McKinney, and Van de Par presented (“A Demonstrator for Automatic Music Mood Estimation”). They allowed me to test their system with one of my own MP3s which I had on my USB drive (I used a song from Roberta Sá) and it did really well. Another demo I liked a lot was the system Peter Knees presented (“Search & Select – Intuitively Retrieving Music from Large Collections”). Unfortunately I was asked to leave after I had been playing around with the demo for a bit too long, I guess. Ohishi, Goto, Itou, and Takeda (“A Stochastic Representation of the Dynamics of Sung Melody”) showed me some videos which I thought were simply amazing. Apparently it isn’t hard to compute them (once you know how to extract the F0 curve), but I’ve never seen the characteristics of a singing voice visualized that way. The demo of Eck, Bertin-Mahieux, and Lamere (“Autotagging Music Using Supervised Machine Learning”) was really impressive too… and it was interesting to learn that Ellis (“Classifying Music Audio with Timbral and Chroma Features”) found ways to use chroma information to increase artist identification performances. (And his Matlab source code is available!!) I once worked on a similar problem, but never got that far. Btw, it seems that chroma is everywhere now :-)
I was also happy to see that Flexer’s poster (“A Closer Look on Artist Filters for Musical Genre Classification”) was receiving a lot of attention. I liked his conclusions. There were also lots of interesting papers in the last two days. For example, I liked the paper presented by Cunningham, Bainbridge, and McKay (“Finding New Music: A Diary Study of Everyday Encounters with Novel Songs”). I particularly liked their discussion on how nice it would be to have a “scrobble everywhere” device that keeps track of everything I ever hear (including ring tones).

Beatles Chord Transcriptions

2007-09-25T13:22:00.000+01:00

Chris Harte from the C4DM announced last night that he completed his amazing effort of transcribing the chords for all songs on the 12 studio albums of the Beatles. The transcriptions are extremely high quality. Anyone who wants a copy just needs to contact him. This will definitely boost research in any direction related to chords (chord recognition, chord progressions, harmony analysis...). It's also a good excuse for any research lab to buy the complete Beatles collection. Btw, don't forget to cite his work when you use his annotations! ;-)

Below are excerpts from the two emails he sent to the music-ir mailing list, so that Google can index them (Afaik he hasn’t set up a website for this yet).

(Chris' email is christopher dot harte at elec dot qmul dot ac dot uk).

Chris Harte wrote in his first email (Sep 24, 2007, 9pm):

[...] I have just completed work on the full set of chord transcriptions for the beatles songs from all 12 studio albums.

The verification was done by synthesizing the transcriptions in MIDI then putting that back together with the original audio (with correct timing and tuning) so that people could spot any errors by listening through to them.

Hopefully, after the verification process that we have just completed, these transcriptions should now be accurate enough to serve as a ground truth for various kinds of chord and harmony work in the MIR field.

If you would like a copy of the new version of the collection then please let me know and I will send them to you. [...]

Chris Harte wrote in his second email (Sep 25, 2007, 4am):

[...] To clear up a few things:

The transcription files are in wavesurfer ".lab" format which is just flat text arranged like this:

Start-time end-time label
Start-time end-time label
Start-time end-time label
...

Times are in seconds.

".lab" files can be opened as a transcription pane in wavesurfer (I have made a wavesurfer conf file set up for showing these transcriptions nicely if people need one) and also in Sonic Visualiser as an annotation layer (use "A point in time" for the "each row specifies" option when loading in sonic visualiser).

The chord symbols used in the transcriptions basically conform to the syntax described in our ISMIR 2005 paper "Symbolic Representation of Musical Chords: A Proposed Syntax for Text Annotations" available here:
http://ismir2005.ismir.net/proceedings/1080.pdf

There has been one slight change to the syntax described in this paper which is that now a chord symbol, which is defined as a root and a list of component degrees, should not automatically be assumed to include the given root note unless the '1' degree is explicitly included in the list
- e.g. C major can be written C or C:maj which are both equivalent to writing C:(1,3,5) so the "major" pattern should be (1,3,5) instead of just (3,5). This makes it possible to annotate a chord where it is obvious that the intended harmony is C major even though only the notes E and G are present by using C:(3,5). I hope that makes sense...

For those who do not already know, I have written a set of tools for manipulating these chord symbols in matlab (they don't use any toolkits so I guess they should also work fine in Octave) - if you would like a copy of those then let me know. There will be an updated version of these tools available soon as well.

For more information on the chord symbols, chord tools and transcription process, my long awaited (long awaited by me at any rate...) PhD thesis will include a whole chapter about it all. I hope to submit the thesis sometime around christmas this year. [...]

One Llama, IMIRSEL, MIREX

2007-09-23T22:32:00.000+01:00

One of the interesting things I learned in the recommendation tutorial today is that IMIRSEL launched a startup called One Llama. Seems like they have some ideas on how to make money with MIR technologies. I wonder how many of the MIREX participants were aware of this before submitting their latests implementations to IMIRSEL.

ISMIR Highlight: Recommendation Tutorial

2007-09-23T21:49:00.000+01:00

Paul Lamere and Oscar Celma did a wonderful job presenting the recommendation tutorial. I wouldn't be surprised if this turns out to be my personal highlight of ISMIR 2007. They presented an overview of all the standard techniques used for recommendations, they talked about the typical (and unsolved) problems recommenders face, they had plenty of examples, and they also presented results from an evaluation of recommenders. The parts I personally liked best were the in depth analysis of tags and folksonomies, the part they called "novelty and relevance" (with interesting ideas on how to reach deeper into the long-tail), the analysis of artist similarity networks, and the evaluation of recommenders. They also made an interesting point about how nice it would be to have something like a Netflix competition for music recommendation. I'm guessing the slides of the tutorial will be online soon. I highly recommend having a look at them ;-)

I only attended the recommendation tutorial, but I've been told the other tutorials were also really well done. Seems like this year's ISMIR is not only the best in terms of number of papers submitted, number of people attending, best location ever, but also best content ever! ;-)

Btw, Paul's blogging about ISMIR in case you haven't noticed yet. And a number of pictures have already been uploaded to flickr tagged ismir2007.

Fun things to do with fingerprinting

2007-09-22T00:39:00.000+01:00

Erik Frey just posted some fun things he did using Last.fm's fingerprinter. For example, it's really easy to find out if an artist released "live" versions that are identical to the studio version except that some cheering has been added. In terms of false positives and fingerprinting he raises some interesting questions.

The most frequently cited ISMIR paper

2007-09-21T22:01:00.000+01:00

I just did a quick Google scholar search to find the most frequently cited ISMIR paper. I'm not sure if I missed any, but it seems the most frequently cited paper is "Mel Frequency Cepstral Coefficients for Music Modeling" (PDF) presented in 2000 by Beth Logan. According to Google scholar it has been cited 127 times as of today. My coauthors and I have cited that paper several times :-)

MFCCs were originally developed in the speech processing community. Back in 2000 it wasn't obvious if the same techniques could just be "copied and pasted" to music information retrieval. MFCCs are now a very standard technique that are being used to compute music similarity, classify genres, identify instruments, segment music, ... In fact, today MFCCs are so common that they are often mentioned in ISMIR papers without citing a source.

One evening and no testing

2007-09-20T03:42:00.000+01:00

It’s been a long and busy day, and it’s taken me a while to go through the flood of emails that landed in my inbox today. Several of those were related to a singing microwave which seems to be at the height of its career.

Another interesting email I found in my inbox explains how the system that scored highest in several MIREX 2007 tasks was built:

“The system was not tuned - in fact it was not tested on any dataset (from the competition or otherwise) beyond making sure it was outputting feature values into its feature files and was in fact cobbled together in one evening.”

Something that has never been tested before, and sounds like a preliminary prototype outperformed them all. Since it hasn't been tweaked yet, the system probably has a very good potential to generalize, and probably can easily be tweaked to add at least another 1 or 2 percentage points accuracy to the genre classification results. That's pretty impressive.

Talking about MIREX I would like to add the following to clarify things I have written in a previous blog post:

I highly value MIREX, it's a driving force behind advances in MIR. I've personally learned a lot from it.

I understand that IMIRSEL has sacrificed a lot to make MIREX happen. It's been an amazing effort organized by Stephen Downie and his team.

I'm sorry my comments on the conflicts of interest issues have been perceived as personal attacks. That was not my intention.

I realize that my previous blog post on the topic should have clearly stated that: I'm fully (and always have been) convinced that no one at IMIRSEL had the intention to cheat. I have absolutely no doubts about that.

However, I'm still fully convinced that IMIRSEL submissions should not be listed in the same ranking as the submission of others.

Marsyas & Music Classification

2007-09-18T22:27:00.000+01:00

One of the most interesting things I found in the MIREX results so far has been the good performance of George Tzanetakis in different categories. He scored highest in mood classification, and did well in the other classification tasks.

Since George is well known to have published some of the most frequently cited papers on music classification this isn't really interesting news. However, what's really interesting is that George did so well despite using Marsyas. Marsyas is open source and has been around in the MIR community for as long as I can remember. At the ISMIR 2004 and MIREX 2005 evaluations Marsyas didn't do too well (although, afaik it's always been by far the fastest implementation). Perhaps as a results, I've recently been seeing fewer papers on genre classification using Marsyas as baseline. But given the excellent performance this year, I think it's fair to say it has re-established itself as the baseline for any new music classification algorithm. In fact, it has done so well, that I doubt we will see any papers in the near future which can report significant gains compared to this solid baseline. (Btw, never forget to use an artist filter when evaluating genre classification performance!)

Overfitting and MIREX

2007-09-18T22:02:00.000+01:00

IMIRSEL (the organizer of MIREX) hasn't officially responded yet to the conflicts of interest of organizing a non transparent evaluation and at the same time participating in it. What I've heard from others is that they don't see any problems with it.

Btw, has anyone else noticed that they won in every classification category where overfitting is a big issue? However, in a very related category (mood classification) where overfitting isn't an issue (thanks the a human component in the evaluation) they were outperformed by several others.

Furthermore, IMIRSEL never had their name put down on the list of potential candidates. Given the lack of transparency of the respective MIREX tasks I think this is something every participant should have known before submitting their work. Btw, so far it isn't even known who the researchers are who actually did the work. AFAIK, no entry so far in the history of ISMIR evaluations has been submitted without mentioning who the authors are.

Btw, as to now, IMIRSEL are the only ones in the genre classification task who haven't published an abstract (describing what their algorithm does and how it was optimized) yet. (They also haven't submitted one yet for the other tasks they won in.)

UPDATE:
Regarding anonymous MIREX submissions I just remembered that at the ISMIR 2004 evaluation hosted by MTG allowed anonymous submissions... and some authors did choose to do so. (However, as I already mentioned in the comments of this post: MTG clearly stated that they did not participate in the tasks they organized to avoid any conflict of interest.)

Vocaloid 2 is a big hit in Japan

2007-09-17T20:40:00.001+01:00

Vocaloid 2 is Yamaha software that sings. The $100 software seems to be a big hit now in Japan. Read more about it here and here.

Congratulations to MTG who contributed largely to the development of Vocaloid! Seems like we're a lot closer now to having 5 billion new (anime) songs per week flood the MIR universe.

(Thanks Norman for the pointer!)

MIREX Results Online!

2007-09-17T09:59:00.000+01:00

The MIREX results just got posted by Stephen Downie. Interestingly the organizers scored highest in a number of categories. To be honest, if I were a participant in a task like genre classification I’d be a bit suspicious. (Knowing the distribution of the genres before hand can be a huge advantage when designing an algorithm.)

Congratulations to Tim Pohle and Dominik Schnitzer (two very clever PhD students I once worked together with in Vienna) who scored highest in the audio similarity task. I wouldn’t be surprised if they also had one of the fastest implementations. Tim also scored second highest last year in the same task. And Dominik recently made the results of his Master’s thesis available (open source playlist generation).

Congratulations also to Joan Serrà and Emilia Gomez (a former SIMAC colleague) who scored highest in the cover song identification task.

And congratulations to everyone who participated and the organizers for managing to complete all the tasks before ISMIR!

Listening

2007-09-17T00:12:00.000+01:00

The best thing about working in MIR research is that it’s part of the job to spend lots of time listening to music. Which makes me realize that I've been working very hard this weekend ;-)

I spent the last hours listening to music listening I found playing around with a recommender. It's one of those recommenders which takes one of my favorite tracks as input and returns a list of similar tracks. Seeing how amazing some of the recommendations are makes me wonder if I’ll ever again bother to browse lists of similar artists to find new music.

620, 10, 5

2007-09-17T00:07:00.000+01:00

Michael Fingerhut announced today on the music-ir list that his complete list of ISMIR papers now contains 620 entries (including this year’s papers). That’s an impressive pile of papers that the ISMIR community has produced since 2000... Btw, there have been only 10 papers so far which contained “recommend” in the title. 5 of those will be presented this year... Given that there will also be a tutorial on recommendation, and that I've mostly been blogging about recommendations recently makes me wonder if recommendations is about to establish itself as one of the core topics of MIR?

Good Recommendations (2)

2007-09-10T05:58:00.000+01:00

Inspired by Paul’s ongoing evaluation I tried my own tiny little evaluation.

As seed I used Le Volume Courbe. I recently stumbled upon Charlotte while browsing the Last.fm music profiles of friends.

I wanted to find more of the same (unfortunately she only recorded one album), and the most obvious place to start was the Last.fm similar artist list. There’s lots of good music there, but nothing that I enjoyed as much. I also browsed the top listener profiles, and profiles of people who commented on the Last.fm page for Le Volume Courbe. Again I found lots of great music, but not really more of the same.

Next obvious stop was Pandora, but they never heard of Le Volume Courbe before. So I tried iLike but they didn’t know about any similar artists and ZuKool couldn’t help either. MyStrands had a long list, but after sampling the first two on the list I had the impression that they are pointing me in the wrong direction (too much towards electronic music). Amazon had some interesting recommendations (first time I heard about shoegaze) but not really more of the same. And finally my flat mate recommended some great and related music, but also not really more of the same.

So my preliminary verdict is: either there isn't more of the same out there, or the music recommendation services I tried need to be improved.

UPDATE: I just had a look at the AMG similar artist list. There's some interesting recommendations there, some of which I had already stumbled upon while browsing similar artists on Last.fm, but still nothing that's truly more of the same.

UPDATE Part 2: I just tried the AMG Tapestry Demo suggested by Zac in the comments. It's a lot more convenient than browsing the AMG pages, and it's similar to Last.fm's similar artist radio stations (except that it's only 30 second previews). Nevertheless, there were some recommendations on the list that I appreciated (and hadn't found in the AMG list of similar artists). However, somehow the recommendations seem to be missing some of the "darkness" I like about Le Volume Courbe. Anyway, it's great to have so many nice ways of exploring similar artists.

UPDATE Part 3: I just sampled some of the artists on the list Paul posted in the comments. Whatever system he's using, it's doing a great job in surfacing very unknown artists, some of which are even hardly present on Last.fm, and most of which seem to be present on myspace (which makes me wonder if his recommendation machine is gathering information from there?). Again, I failed to find more of the same. However, a number of the recommendations were related (in particular, some were related with respect to the lo-fi, singer-songwriter, DIY aspects I like about Le Volume Courbe), and since none of the recommendations in his list had shown up in any of the previous recommendations I had seen (at least as far as I can remember) it was rather refreshing to hear them. It's really nice to see a recommendation machine that has such a strong emphasis on surfacing rather unknown artist.

UPDATE Part 4: Oscar suggested in the comments to try the hype machine, which I did. I found some interesting comments about Le Volume Courbe there, but didn't really find more of the same. I also tried the much hyped SeeqPod, but they not only failed to find music related to Le Volume Courbe, but also gave some not so trust worthy recommendations for Mozart. Others I tried and that didn't have any results were musicmatch and musicplasma.

UPDATE Part 5: Ian mentioned that ZuKool now has Le Volume Courbe in their catalog. I gave it a quick try by adding all the songs from the album into the list (because I didn't like how when I'd only choose one track all other tracks from the same album would show up in the recommendation list). The results were as refreshing as those from Paul's list (none of them I had previously seen in a recommendation list) and they were interesting to listen to. However, I couldn't find more of the same. Btw, getting Celeste Zepponi's "Jesus Is Here" recommended when searching for similar music to Le Volume Courbe suggests that ZuKool completely ignores socio-cultural information, which makes it an interesting alternative to all other music recommenders I use.

Good Recommendations

2007-09-08T16:51:00.000+01:00

Paul launched a very interesting survey on music recommendations. The results will be presented at their tutorial in 2 weeks at ISMIR and I'm sure presentation slides will be available online after that. I highly recommend participating :-)

Trying to answer the questions I realized how difficult it can be to recommend music given just one artist. Would it be good to recommend someone a rather unknown (= not so popular) artist when they are looking for something similar to an extremely popular artist (like The Beatles)? Or would it be better to recommend similar artists which are also very popular? Btw, in the case of The Beatles, would it really make sense to recommend John Lennon and other members of the group? And if someone is looking for music similar to a not so well known artist, would it make sense to recommend similar but popular artists? Or is it safe to assume that this person already knows these?

Evaluating recommendations is another very interesting topic… and I’m very curious what the outcomes of Paul’s survey will be.

Researchers far ahead of industry

2007-08-31T11:43:00.000+01:00

Delete the filter and other playlist generation tools you might have installed on your system, here comes Dominik's open source solution.

Thanks Klaas for the link!

MIR Related Books

2007-08-30T00:53:00.000+01:00

Here’s two books I’m currently reading and which I would like to recommend.

Last night a DJ saved my life
The history of the disc jockey
Bill Brewster and Frank Broughton
Website

I’m still in the late 70s, but it’s been a wonderful voyage through time so far. I think anyone working on DJ-ing algorithms might enjoy this. The book starts out with the impact recorded music had, the first broadcasts, and then dives very deep into the role of the DJ in terms of discovering and distributing rare records, breaking new records, creating new styles of music, and enabling others to have a great time. The book isn’t structured like a research paper, but it’s very well researched.

Net, Blogs and Rock ‘n’ Roll
How digital discovery works and what it means for consumers, creators and culture
David Jennings
Blog, Last.fm Profile

I just started reading this one, but it’s been fun so far. Sometimes it feels a bit like all of the Music 2.0 hype words thrown into a mixer, but beyond that there's lots of interesting information in the book about everything related to Music 2.0. This book might be a great source of inspiration for project proposals or motivations/introductions for research papers on MIR technologies that could somehow fit under the Music 2.0 umbrella.

Most of all I love contrasting the two books with each other. The two worlds couldn’t be more different and yet they are somehow about the exact same thing.

Fingerprinting and Music Recommendation

2007-08-29T17:34:00.000+01:00

It's a huge step forward for Last.fm:
Audio Fingerprinting for Clean Metadata

Soon Last.fm will stop recommending Roberta Sa because I've listened to Roberta Sá. And once Last.fm's catalog is cleaned up, all Last.fm users will be able to use our massive meta data catalog to fix and extend the tags of their own MP3 collections. Furthermore, the quality of recommendations, neighbors, the radio stations, and just everything should gradually improve as a nice side effect of this major clean up.

Btw, the fingerprint extraction code is open source :-)

Exciting times for music recommendations

2007-08-27T12:36:00.000+01:00

The Filter recently secured another round of financing worth USD 5 million. Not too long ago MyStrands secured a massive USD 25 million. And today's job announcement on the music-ir mailing list sounds like BMAT has started to build their own social music recommendation web site, too. Btw, wouldn't it make sense for BMAT and MyStrands to work together? Anyway, that's just a tiny sample of all the companies working on music recommendations, some of the startups seem very promising.

There’s still a very long way to go. But with every step forward, music listeners will find it easier to discover new artists. And artists will find it easier to find an audience. (Btw, if you use Last.fm you might notice some larger steps forward in the next months.)

It's amazing how much has happened since 2001 (when I finished my Master thesis on a related topic). It’s fun to be working in such a dynamic environment. And it's never been easier to discover amazing music.

ISMIR: Short List of Papers

2007-08-26T06:17:00.000+01:00

I just compiled my short list of papers I don’t want to miss at ISMIR 2007 which starts in about 4 weeks. Of course I’m interested in all papers, but if I run out of time while exploring posters, or need to choose between different sessions, I’ll prefer the ones listed here.

Fuzzy Song Sets for Music Warehouses
To be honest, this is just on the list because given the title I don’t have the slightest clue what this paper is about. I know what fuzzy sets are thanks to Klaas. I’m guessing that a music warehouse is a synonym for a digital library of music. I wonder if the second part of the title got lost?

Music Clustering with Constraints
Another title that puzzles me. Seems like titles have been cut off a lot. They forgot to mention according to what they are clustering the music. Number of musical notes in a piece? AFAIK, most clustering algorithms have some form of constraints. For example, in standard k-means the number of clusters is constrained. When using GMMs it is very common to constrain the minimum variance of an individual Gaussian. Anyway, I’m into clustering algorithms, so this could be an interesting presentation.

Sequence Representation of Music Structure Using Higher-Order Similarity Matrix and Maximum-Likelihood Approach
The author of this one has done lots of interesting stuff in the past. I’m curious what he’s up to this time. Music structure analysis is definitely something very interesting that could be very useful in many ways.

Algorithms for Determining and Labelling Approximate Hierarchical Self-Similarity
Again at least one of the authors has have done very interesting stuff in the past and I’m really interested in music structure analysis.

Transposition-Invariant Self-Similarity Matrices
I’m only guessing but this one could be about self-similarity with respect to melody. (I’m guessing that the previous 2 are focusing on self-similarity with respect to timbre or chroma.) Melodic similarity is a lot harder than timbre similarity. I’m curious how they did it.

A Supervised Approach for Detecting Boundaries in Music Using Difference Features and Boosting
If I miss this presentation I might upset my coauthors ;-)

Automatic Derivation of Musical Structure: A Tool for Research on Schenkerian Analysis
I had to Google Schenkerian. It sounds interesting.

Improving Genre Classification by Combination of Audio and Symbolic Descriptors Using a Transcription System
I’m very curious what kind of symbolic descriptors the authors used. Note density? I’ve seen lots of work on audio-based genre classification, and some work on using MIDI (which is usually referred to as symbolic information, but the authors could also mean something very different with symbolic). I’m pretty sure I’ve read at least one article on the combination of audio and MIDI information, but I don’t think I’ve ever seen anyone actually succeed. I’m curious what results the authors got, and I hope they used an artist filter.

Exploring Mood Metadata: Relationships with Genre, Artist and Usage Metadata
Let me guess: pop is usually happy and upbeat, and death metal is rather aggressive? :-) I wonder though what usage metadata is (if people listen to it while driving their cars, working, jogging etc?).

How Many Beans Make Five? The Consensus Problem in Music-Genre Classification and a New Evaluation Method for Single-Genre Categorisation Systems
Single-category classification? I think I’m good at that ;-) (Yes, I know that with single they mean binary classification.) Anyway, I’m curious what the authors say about genre classification and consensus. The authors probably have a very different perspective than I do.

Bayesian Aggregation for Hierarchical Genre Classification
I hope they either compare it to existing techniques, or use evaluation DBs that have been used previously. And I hope they used an artist filter. I’m very curious though what they aggregated.

Finding New Music: A Diary Study of Everyday Encounters with Novel Songs
If I had a very, very short list of papers I wouldn’t want to miss, than this would be on it :-)

Improving Efficiency and Scalability of Model-Based Music Recommender System Based on Incremental Training
Made in Japan, what else is there left to say? ;-)
This would also be on the very, very short list of presentations I wouldn’t want to miss.

Virtual Communities for Creating Shared Music Channels
I’m guessing that this could be really interesting, but I wish the title was more specific. Under the same title one could present, for example, how Last.fm groups and their group radio stations work, or how people get together on Last.fm to tag music to create their own radio stations.

MusicSun: A New Approach to Artist Recommendation
Another title that’s missing lots of information, nevertheless, I won’t skip this one.

Evaluation of Distance Measures Between Gaussian Mixture Models of MFCCs
I’m curious which approaches they tested and how and what their conclusions are.

An Analysis of the Mongeau-Sankoff Algorithm for Music Information Retrieval
Another title that sent me to Google. This time there were only 15 results, none of which did a good job in explaining it to me. Anyway it has MIR in the title, so I think I should have a look.

Assessment of Perceptual Music Similarity
Sounds like a follow-up of the work they presented last year. I’m very curious. I hope they got more than 2 pages in the proceedings. I’d love to read more on this topic.

jWebMiner: A Web-Based Feature Extractor
Sounds like there’s more great software from McGill for everyone to use.

Meaningfully Browsing Music Services
I’ve seen a demo that included Last.fm, so I really can’t miss this one.

Web-Based Detection of Music Band Members and Line-Up
Personally I would be tempted to just use MusicBrainz DB for that. I wonder how much more data the authors could find by crawling the web in general.

Tool Play Live: Dealing with Ambiguity in Artist Similarity Mining from the Web
Artist name ambiguity is an interesting problem, I wonder what solution they are presenting.

Keyword Generation for Lyrics
I’m guessing these are keywords that summarize the lyrics? I wonder if they use some abstraction as well to classify, for example, a song as a love song.

MIR in Matlab (II): A Toolbox for Musical Feature Extraction from Audio
I use Matlab everyday and I don’t think I’ve heard of this toolbox before, sounds interesting.

A Demonstration of the SyncPlayer System
I think I saw a demo of this at the MIREX meeting in Vienna. If I remember correctly the synchronization refers mainly to synchronizing lyrics with the audio but it can do lots of other cool stuff, too.

Performance of Philips Audio Fingerprinting under Desynchronisation
I have no clue what desynchronisation is, but I know that fingerprinting is relevant to what I work on.

Robust Music Identification, Detection, and Analysis
This could be another paper on fingerprinting?

Audio Identification Using Sinusoidal Modeling and Application to Jingle Detection
More fingerprinting fun.

Audio Fingerprint Identification by Approximate String Matching
Seems like fingerprinting has established itself as a research direction again :-)

Musical Memory of the World –- Data Infrastructure in Ethnomusicological Archives
It’s not directly related to my own work, but sounds very interesting.

Globe of Music - Music Library Visualization Using Geosom
A visualization of a music library using a metaphor of geographic maps? I’m curious how using a globe improves the experience.

Strike-A-Tune: Fuzzy Music Navigation Using a Drum Interface
I hope they’ll let me have a try :-)

Using 3D Visualizations to Explore and Discover Music
I believe I’ve seen this demo already, but I never got to try it out myself. I hope the waiting line won’t be too long.

Music Browsing Using a Tabletop Display
If the demo is interesting I’ll forgive them their not very informative title ;-)

Search&Select -– Intuitively Retrieving Music from Large Collections
I like the authors work. I’m very curious what he built this time.

Ensemble Learning for Hybrid Music Recommendation
It has the words music recommendation in the title, and the authors have done some interesting work in the past.

Music Recommendation Mapping and Interface Based on Structural Network Entropy
Another music recommendation paper, I’m guessing this one is about a certain MyStrand visualization. I’m particularly interested in the “structural network entropy” part.

Influence of Tempo and Subjective Rating of Music in Step Frequency of Running
My guess is that tempo has an impact and that this impact is even higher for music I like? But I wouldn’t expect the subjective rating to have a very high impact. I often notice how I start walking to the beats of music I hear even if I don’t like the music.

Sociology and Music Recommendation Systems
Another paper I’d put on the very, very short list :-)

Visualizing Music: Tonal Progressions and Distributions
Sounds great! I should check if they already have some videos online.

Localized Key Finding from Audio Using Nonnegative Matrix Factorization for Segmentation
I’m curious how the author used a nonnegative matrix factorization for this task. I’ve never used one, but I thought they are usually used for mixtures. However, segments (like chorus and instrument solos) are usually not best described as mixtures?

Invited Talk
Sounds like I’ll learn interesting things about copyright, creative commons, and other intellectual property issues involved in music information retrieval.

Audio-Based Cover Song Retrieval Using Approximate Chord Sequences: Testing Shifts, Gaps, Swaps and Beats
I mainly want to know what the author has been up to, but I’m also interested in cover song detection.

Polyphonic Instrument Recognition Using Spectral Clustering
I want to see this one too, but it’s at the same time as the previous paper. The papers use rather similar techniques and deal with rather similar problems. I don’t understand why they were put up to compete with each other. Something non-audio related would have been a much better counter part.

Supervised and Unsupervised Sequence Modelling for Drum Transcription
I wonder how good their drum transcription works. I hope they have lots of demos.

A Unified System for Chord Transcription and Key Extraction Using Hidden Markov Models
Again a paper I really don’t want to miss but it’s at the same time as the one above. There are so many papers that don’t deal with extracting interesting information from audio signals that I absolutely don’t understand why they arranged this parallel session the way they did.

Combining Temporal and Spectral Features in HMM-Based Drum Transcription
I’m not sure if I’ll check out this one or the one below. Both are really interesting.

A Cross-Validated Study of Modelling Strategies for Automatic Chord Recognition in Audio
Sounds like they might have some interesting results.

Improving the Classification of Percussive Sounds with Analytical Features: A Case Study
I must see this one because I recently did some work on drum sounds. I’m curious if the authors include all sorts of percussive instruments (such as a piano) or if it’s drums mainly.

Discovering Chord Idioms Through Beatles and Real Book Songs
I’d love to see this one, too :-(
Don’t get me wrong: I fully support parallel sessions (there isn’t really an alternative given this many oral presentations) but unfortunately the sessions weren’t split in a way that would allow me to see everything I would like to see. Why not put chords and alignment parallel to each other?? To demonstrate my point I won’t list any papers of the alignment session.

Automatic Instrument Recognition in a Polyphonic Mixture Using Sparse Representations
Another strange thing about how the sessions were split is that one parallel session always ends 15 minutes earlier than the other one. Do the organizers expect that everyone from the other session runs to the other session? I’d prefer if all sessions would end at the same time and thus make it easier to find a group to go join for lunch. Anyway, sounds like an interesting paper.

ATTA: Implementing GTTM on a Computer
It’s been a while since I first heard a presentation on GTTM. I guess it’s about time to refresh my knowledge.

An Experiment on the Role of Pitch Intervals in Melodic Segmentation
I have no clue… but segments often have different “local keys”. The chords within keys are usually clearly defined. Each chord has specific pitch intervals… I wonder what experiment they did.

Vivo - Visualizing Harmonic Progressions and Voice-Leading in PWGL
A visualization!

Visualizing Music on the Metrical Circle
Another visualization :-)

Applying Rhythmic Similarity Based on Inner Metric Analysis to Folksong Research
I’m curious how they compute rhythmic similarity. I have seen a lot of work on extracting rhythm information, but haven’t seen much on computing similarities using it.

Music Retrieval by Rhythmic Similarity Applied on Greek and African Traditional Music
Another rhythmic similarity paper :-)

A Dynamic Programming Approach to the Extraction of Phrase Boundaries from Tempo Variations in Expressive Performances
A long time ago I did some work on segmenting tempo variations… I’m curious how they represent tempo (do they apply temporal smoothing?) and how well detecting phrase boundaries works given only tempo. (Why not use loudness as well?)

Creating a Simplified Music Mood Classification Ground-Truth Set
Sounds like this might also be related to the MIREX mood classification task.

Assessment of State-of-the-Art Meter Analysis Systems with an Extended Meter Description Model
I wonder how good state-of-the-art methods work for meter detection.

Evaluating a Chord-Labelling Algorithm
Chord detection is great.

A Qualitative Assessment of Measures for the Evaluation of a Cover Song Identification System
Cover song detection is great, too.

The Music Information Retrieval Evaluation Exchange “Do-It-Yourself” Web Service
Wow! I wonder if they will have a demo ready?

Preliminary Analyses of Information Features Provided by Users for Identifying Music
I have no clue what this one is about, but it’s probably MIREX related.

Finding Music in Scholarly Sets and Series: The Index to Printed Music (IPM)
One of the many things I know nothing about, but it sounds interesting.

Humming on Audio Databases
I wonder if they provide a demo, and if they can motivate people to use it. (It will probably be more fun listening to people sing than see if their system works.)

A Query by Humming System that Learns from Experience
Would be nice to have this one right next to the previous one.

Classifying Music Audio with Timbral and Chroma Features
Another one for the very, very short list. I’m curious how the author combined the features, and if he measured improvements, and if he did artist identification or genre classification (and if he used an artist filter if so).

A Closer Look on Artist Filters for Musical Genre Classification
Sounds like something everyone should be using :-)

A Demonstrator for Automatic Music Mood Estimation
I definitely want to see this demonstration.

Mood-ex-Machina: Towards Automation of Moody Tunes
I wonder what this sounds like.

Pedagogical Transcription for Multimodal Sitar Performance
I wonder if it’s so pedagogical that I can understand it?

Drum Transcription in Polyphonic Music Using Non-Negative Matrix Factorisation
Not sure what’s new here, but I’ll be there to find out.

Tuning Frequency Estimation Using Circular Statistics
No clue what this is about. My best guess would be that it’s related to the pitch corrections I’ve seen in chord transcription systems.

TagATune: A Game for Music and Sound Annotation
Wow another music game! I haven’t heard of this one yet and Google hasn’t either. I’m very curious how it differs from the Listen Game and the MajorMinor game.

A Web-Based Game for Collecting Music Metadata
Would be great if they publish some usage statistics.

Autotagging Music Using Supervised Machine Learning
I’m very curious what results they got.

A Stochastic Representation of the Dynamics of Sung Melody
Another Japanese production :-)

Singing Melody Extraction in Polyphonic Music by Harmonic Tracking
I wonder how high the improvements were by tracking the harmony.

Singer Identification in Polyphonic Music Using Vocal Separation and Pattern Recognition Methods
I wonder how they evaluated this. Did all singers have the same background instruments and sing in the same musical style?

Transcription and Multipitch Estimation Session
I know nothing about multipitch estimation. But I hope to hear some nice demonstrations in the session.

Identifying Words that are Musically Meaningful
I wonder what the most musically meaningful word is. At Last.fm I think it’s “rock”. Another word very high up in the Last.fm ranks is “chillout” :-)

A Semantic Space for Music Derived from Social Tags
I’m curious what their tag space looks like.

The Music Ontology
I don’t know much about ontologies, but it sounds like this is the one and only one for music, so I better not miss it.

Signal + Context = Better Classification
I love this title. I hope the first author will be presenting it.

A Music Information Retrieval System Based on Singing Voice Timbre
I’ll probably be totally exhausted from having seen so many presentations and posters by this time, but I’ll try to reserve some energy to be able to concentrate on this talk.

Poster session 3 (MIREX)
Usually one of the highlights at ISMIR. I hope the MIREX teams manages to have the results ready in time. Only about 4 weeks left to get everything done.

Methodological Considerations in Studies of Musical Similarity
I wish this paper would have been published before I wrote my thesis. But I guess it's never too late to learn :-)

Similarity Based on Rating Data
Sounds like something Last.fm has been doing since years: The ratings are measured based on how often people listen to a song. Then standard collaborative filtering techniques are applied. The results are not too bad. I’m guessing that the authors used very sparse data compared to the data Last.fm has. Another paper I’d put on my very, very short list.

A Study on Attribute-Based Taxonomy for Music Information Retrieval
I wonder if this is similar to Pandora’s music genome project?

Variable-Size Gaussian Mixture Models for Music Similarity Measures
I wonder if and how the author was able to measure significant improvements.

Towards Integration of MIR and Folk Song Research
I like folk music, and I like MIR.

From Rhythm Patterns to Perceived Tempo
I’m curious how they approach this. A rhythm pattern (as defined in music books) does (AFAIK) not have any tempo information and can be played at different tempi. But I’m sure this is an interesting paper :-)

The Quest for Ground Truth in Musical Artist Tagging in the Social Web Era
The title reminds me of one of the more important papers in the short history of ISMIR. Tags are something very subjective, there is no right or wrong. You’ll always find people complaining about how other people mistagged the genre of a song. It will be interesting to see if this paper has the potential to join the ranks of the original ISMIR paper with a similar title.

Annotating Music Collections: How Content-Based Similarity Helps to Propagate Labels
Sounds like something very useful.

A Game-Based Approach for Collecting Semantic Annotations of Music
I hope they’ll present some usage statistics.

Human Similarity Judgments: Implications for the Design of Formal Evaluations
I wonder why this paper isn’t presented before the MIREX panel. Seems like it might contain a lot of information that would be useful for the discussion.

Justin Donaldson’s Blog

2007-08-25T14:43:00.000+01:00

For some reason I only found Justin’s blog today. The first time I heard of Justin was when I stumbled upon the visualizations he implemented at MyStrands. Basically he visualizes songs as little disks which are spread out on the screen according to similarity. He uses a modified version of (2-dimensional) MDS and he implemented a clever way to deal with songs that overlap spatially in the visualization. It's quite different from, for example, the way Paul uses (3-dimensional) MDS to visualize a music space.

Justin’s blog covers different topics, not all of which are related to MIR, but some of his entries are. For example, in this entry he addressed reviewer feedback regarding a paper he submitted on his MyStrands visualization. The blog entry's character is kind of similar to mine on the reviewer feedback I received for the MusicSun paper. Personally, I found it interesting to see that he was confronted with similar feedback as I have in the past: “Why is the visualization more informative/useful to a user than a ranked list of results?” In times where the simple Google style search interface dominates despite having so many nice visual alternatives (like kartOO) there is actually a need to explain this although it seems so obvious sometimes.

It seems to me that publishing a paper on a visual interface for music recommendation is much harder than publishing one about a genre classifier that achieves 100% classification accuracy (which isn't hard if the test set contains the same artists as the training set). I like how Justin implemented his visualization as part of a real music site (instead of just building a research prototype). I wonder if Justin couldn’t have easily conducted a simple evaluation using all the data gathered from that test. (Maybe just simple usage statistics over time?) I wonder if those statistics would be similar to the Alexa statistics for kartOO over the last years. Maybe visualizations that need more than one color and more than one dimension are too complex for music recommendation systems?

Btw, seems like Justin will be giving an interesting talk at the Recommendation Systems 2007 conference (as part of the doctoral symposium) and he’s also presenting a poster at ISMIR 2007.

MusicSun: A New Approach to Artist Recommendation

2007-08-13T21:13:00.000+01:00

I previously announced that I’ll be posting reviews I received for a paper my co-author Masataka Goto and I submitted to ISMIR’07. You’ll find them below. Btw, I’d like to see others publish their reviews, too. Then I could decide based on those if I want to read a paper or not. It would be like using Amazon customer reviews when deciding to buy a book or not :-)

I’ve added the paper to my list of publications, there’s a link to the PDF and to the demonstration videos.

The paper was 6 pages long when we first submitted it and accepted as a 4 page version in the proceedings. (For the final version we had to shorten almost everything a bit. The biggest part we dropped was a comparison of the interface to a simplified version which was closer to the simple Google search interface. When reading the reviews it’s important to keep in mind that the reviewers were reading a rather different version of the paper than the one which is online now.) I’ve added my remarks to the reviewers in italic.

I really like receiving feedback to my work, and usually conference and journal reviews are a wonderful source of feedback. However, one thing I missed in the reviews the program chairs sent me was the final paper length each reviewer recommended (there was a field for that in the review form). Maybe next year they could improve this.

Btw, I would like to thank reviewers 1 and 2 as they have helped improve the quality of the paper. Reviewer 3 seems to have forgotten to write something. Reviewer 4 has helped a bit, but pissed me off a bit more. However, others have told me that there is nothing wrong with his or her review. I guess my view on this is not very objective ;-)

REVIEW SUMMARY

Reviewer	R1	R2	R3	R4
Relevance	+++	++	+++	++
Originality	++	++	++	-
Quality	++	++	+++	-
Presentation	+++	+++	+++	+

Legend:
+++ Strong Accept, ++ Accept, + Weak Accept
--- Strong Reject, -- Reject, - Weak Reject

=====================================
Reviewer 1: Detailed comments

Overall: Cool interface, and a decent user study. In general, I would prefer to see a more task-specific evaluation (how long does it take a user to find music they like? do they succeed?) and for specific features how often are they used? But the self-reporting survey is a good start.

It’s always nice when a reviewer starts the review with something positive :-)

Regarding “how long does it take a user to find music they like?”: I think an interface to explore music is closer to a computer game than a tool to get work done. For games measuring how much fun the users are having is way more important than measuring how long it takes to get something done (which is one of the main criteria for tools). Nevertheless, I agree with the reviewer, the evaluation we provided is not as thorough as it could be. (Although this has been by far the most extensive evaluation of a user interface I’ve ever conducted.)

My biggest criticism is that there isn't a true baseline; users are only comparing two different versions of the author's system, and not comparing against, say a simple web search, or a competing tool like collaborative filtering.

This comparison the reviewer is referring to has been removed because we didn’t have enough space.

Regarding baselines: comparing our system against state-of-the-art recommendation engines such as the one from Last.fm wouldn’t have been a fair comparison either. We thought that since the main contributions of our paper are the new interface features, a useful evaluation would have been to remove those new elements and see what the users think. I’d be very interested to get some more advice on how to better evaluation user interfaces to explore music.

Specific comments:

- “music” and “review” as constraints for finding relevant web docs: for many ambiguous band names, this doesn't perform very well. For instance, for the band “Texas”, the query “texas music review” brings up many irrelevant pages. this is a hard problem, and for research systems probably not too important to worry about, but it may be worth mentioning.

Excellent point. My only excuse is that we didn’t have enough space to discuss everything.

- how do you generate the vocabulary lists? if it's just and ad-hoc manual process, please mention that, and perhaps suggest other ways to make it more principled and to evaluate changes to these lists.

Good question. I think we’re a bit more specific on this in the 4 page version, but maybe it got squeezed out at the end. Automatically generation such vocabularies would be really interesting. But I wouldn’t know how to do that (without using e.g. Last.fm tag data).

- Similarly, why did you choose 4 vocabularies vs some other number?

I think we didn’t have space to explain this in the 4 page version.

The number of vocabularies is more or less random. The vocabularies are based on previous work (we used them in an ISMIR’06 paper, and before that in an ECDL’05 paper). However, there’s an upper limit on the number of vocabularies that would make sense to use, and the users seemed fine with the 4 we used. (Of course it would be really nice to adapt it to different languages as well.)

- not all readers will know what “tf-idf” is. please explain or provide a reference.

We added a more explicit reference in the final version. I find things like these really hard to notice. I talk about tfidf all the time and just started assuming that the whole world talks about tfidf the whole time, too. :-)

- table 3: unnecessarily confusing to introduce the “L” and “R” just call them Easy and Hard, and explain that you grouped the top/bottom 3 points in a 7-point scale. and instead of “?”, label it “No Answer” or something

Again something that’s hard to notice if you are too deep into the material. Thanks to this reviewers comment the respective table should be more understandable.

- rather than the self-reported results about which optional features the user found useful, i think a better eval would be a count of actually how many times the user used them.

Actually the usage of features wasn’t something the users reported themselves. I was sitting next to them and taking notes while they were using the interface. Thinking of it now, I realize that maybe even in the final version this might not be clear enough :-/
However, automatically counting how often functions were used would have been better. Unfortunately, I didn’t store the log files for all users because I had some technical problems (and I though that making notes while watching them use the interface would be sufficient). Next time...

=====================================
Reviewer 2: Detailed comments

This is a well written paper... and it demonstrates a nice system for showing users new songs.

Again, it’s really nice when a reviewer starts a review with something positive.

This paper largely walks through one design, a very nice looking design, but how does this system compare to other systems? I found the evaluation in this paper weak.

How does the part of this work that combines recommendations compare to the work of Fagin (Combining Fuzzy Information From Multiple Systems )? Would that be a better approach?

I’m not familiar with Fagin. (But fuzzy combinations sound interesting.) Regarding the reviewer’s criticism of the evaluation I guess they are in line with Reviewer 1.

UPDATE: Klaas has posted a link to a really nice introduction to aggregation operators in the comments.

I really wanted to know which similarity approach worked best. This paper doesn't address that issue.

This was beyond the scope of our paper. But it surely would have been very interesting to evaluate the different individual similarities we used :-/

Testing UI design is hard.. one needs a task and then lots of users. Can you do this?

Yes, it is hard :-)
And no, it doesn’t seem like we did a good job :-/

=====================================
Reviewer 3: Detailed comments

Unfortunately this reviewer didn't explain why he or she gave us such high scores.

=====================================
Reviewer 4: Detailed comments

The paper addresses an interesting issue, recommendation systems and interfaces to support them. I found the idea of using multiple information sources very interesting, and potentially useful.

Again, it’s always nice when a reviewer finds something positive to start with. However, the idea of using multiple information sources for recommendations isn’t new, and I don’t think my co-author and I can take the credit for it. And I don't understand how someone can say that combing different sources of information is only "potentially" useful. Even if I close both of my eyes I can clearly see that there's no way around that :-)

The major problem that I have with the paper is the experimental
design: I am not quite sure what is being evaluated. Is it the
recommendation system interface or the underlying software used to
create the recommendations? Is it the recommendation system interface or the underlying software used to create the recommendations? If it is the former, which I think it is, then the design of the experiment seems to confound many issues.

I think it's difficult to separate the two. It’s not really possible to evaluate the user interface without considering limitations of the underlying recommendation system. The way the user interface deals with these limitations and presents these shortcomings to the user is a very critical aspect of systems using state-of-the-art content-based algorithms. This is something we explicitly dealt with in the long version of the paper and briefly mention in the short version (e.g. the indicators for how reliable the system thinks the recommendations are). Furthermore, the recommendation system and the interface are very closely linked to each other (e.g. the way the users are given the option to adjust the aspect of similarity they are most interested in).
But then again, as Reviewer 1 and 2 have already pointed out, there are limits to the evaluation we present in our paper.

For example, the authors do not control for user expertise, nor do
they control for system issues (e.g., the database not being large
enough to provide a user with the song he or she is seeking).

We gathered and analyzed lots of statistics on the users expertise (in terms of using computers, music interfaces, and musical knowledge), music taste, and general music listening & discovery habits, but didn’t include everything because we ran out of space. Nevertheless, we allocated some space in the 4 page version to describe the participants with more detail.

Moreover, conclusions like, and I'm paraphrasing, "the users say they would use it again" are, for the most part, without any normative value.

We never claimed that such a conclusion is normative. Of course we measured user satisfaction in different ways (direct and indirect), but most of the evaluation part of our paper deals with parts of the interface we thought users would like/understand/use but (surprisingly) didn’t. We believe the contribution of our evaluation is to point out a number of directions for future work.

Under what circumstance would they use it (e.g., if they were paid to evaluate it)? It is a stretch to conclude they would use the system if it were part of Amazon-- part of Amazon in what way; in comparison to what; etc.?

I’m really confused by the reviewer’s remarks. Just because (when asked) the users said they would like to use it doesn’t mean that users would really use it. And we never drew this conclusion. Instead we’ve pointed out several (in the longer version even more) limitations of the interface. We never tried to market MusicSun as a finished system, but rather as a prototype from which there is something to learn from.

In addition, the paper is rife with typos and stylistic problems
(e.g., citations are not a part of speech), and the reference section
relies quite heavily on the authors' own work."

It would have been nice if the reviewer would have explicitly mentioned that this is not why he voted for a weak reject. Furthermore, there’s nicer ways of putting this. Neither my co-author nor I are native speakers. It would have been more helpful if the reviewer would have pointed out some of the typos.

Regarding the self-citation: we cited everything we thought was relevant to understand the work we presented. Most of the interface is built on techniques we previously used. We didn’t have room to describe everything, so we referenced it instead. It would have been more helpful if the reviewer would have pointed us to references that are missing, or unnecessary.

=====================================

Btw, check out these links my colleague Norman Casagrande pointed me to (both from the legendary Phd comics series):
- Paper review worksheet
- Addressing reviewers comments

ISMIR 8.0

2007-07-29T23:10:00.000+01:00

Paris in 2002, Barcelona in 2004, London in 2005, and now finally Vienna! For the fourth time in its history the International Sconference on Music Information Retrieval (ISMIR) will be held in Europe. Needless to say that ISMIR in Vienna will be the best ever. Following a new all time high in the number of submitted papers rumors are quickly spreading that tickets are already almost sold out (just like in London 2005). Seems like everyone wanted to get the early registration bonus (which ends on the 31st of July). It also seems like two of my Last.fm colleagues and I have been lucky enough to grab some of the last tickets for the highly anticipated tutorial on music recommendation given by Paul Lamere and Oscar Celma. However, the other three tutorials are just as exciting, I wouldn’t be surprised if the organizers will need to find bigger lecture rooms.

There are so many reasons why ISMIR in Vienna will be the best ever that I could spend the rest of my life writing them down. Two of the main reasons are that it’s the right place and the right time.

It’s the right place because Vienna is the most beautiful city in the world with the highest living standards (and yet affordable prices). Vienna has a rich history in music, and Vienna is located in the heart of Europe which is currently one of the leading forces in music information retrieval research.

It’s the right time because music information retrieval has never been more exciting. The whole music industry is just about to undergo massive transformations driven by technological changes. Music consumption habits of the younger generation have already changed drastically, and music is surrounding us like never before.

> Join the ISMIR 2007 group on Facebook here.

> Read about ISMIR 2007 on Paul’s blog here and here.

More PhDs

2007-07-15T02:47:00.000+01:00

Hirokazu Kameoka recently completed his PhD thesis on “Statistical Approach to Multipitch Analysis” (PDF) at the University of Tokyo. Hirokazu is now working at NTT Communication Science Laboratories, Media Information Laboratory.

It’s great to see another thesis written in English emerge from Japan. Btw, so far 5 out of 10 who completed their MIR related PhD thesis this year are now living and working in Japan.

In addition, it seems that Don Byrd and Tim Crawford have pointed me to the very first PhD thesis mentioning MIR in the title. It dates back to 1988!

[Email from Tim on 11 June 2007]

Don and Elias,

I have the thesis in front of me:

Stephen Dowland Page, 'Computer Tools for Music Information Retrieval', thesis submitted for the degree of Doctor of Philosophy, Oxford, New College, 1988. 252 pp. (Available from the Bodleian Library, Oxford.)

Basically it describes a simple encoding, matching and query specification method for monophonic symbolic searching, which uses a query language based on regular expressions. He never got as far as polyphonic queries, though he had hoped to do so (no surprise there!). The introductory chapters, on the nature of the problem and on previous work, are - I would say - still of value, although the methods he describes would need a lot of work to make them useful for polyphonic music. Full code for the search engine is given (in Modula 2!). The ~40-page bibliography is useful for listing a lot of early work - though a vague reference to "Kassler's series of radio talks on the subject" is frustrating, since he gives no further details!

NB, for Don:
There is a sheet at the beginning of my copy listing copies supplied up to the day I ordered it from the Bodleian Library. Second on the list is Indiana University Libraries, Bloomington, IN 47405: 20 June 1989. (My copy was supplied on 10th January 1991.)

Tim

(The list of MIR related PhDs has been updated.)

The Future of the Paper Industry

2007-07-11T04:31:00.001+01:00

Google scholar is a wonderful way to measure the quality of scientific papers based on the number of citations. I actually find the number of citations so interesting that I display it in my list of publications (the yellow bars on the left correspond to citations).

I learned, for example, that one of my papers (which I thought was one of my better ones) never got cited (I'm guessing that's because there’s not too many people working on organizing drum sample libraries). In addition, I believe it might be useful for someone browsing my list of publications to identify which papers are probably more readable. But I'm not sure about the usability in its current form.

I also just learned that somehow my PhD thesis has made it into the top 30 results for the search term music information retrieval! :-)

Anyway, the point is that Google scholar as a form of evaluating the quality of papers is highly insufficient: the delay between having a final version of the paper and the point where reliable quality estimates can be made is way too large (often taking several years).

Given the limitations of Google scholar I’ve been thinking about what the future of the paper industry should be like (and in particular how the quality of a paper could be measured more quickly), and here are some things I’d love to see:

1. Papers should be publicly reviewed similar to the book reviews by Amazon’s customers. (Including the option to rate the usefulness of reviews.)

2. Researchers, research projects, and research teams should have blogs in which they present their findings and encourage open discussions and criticism of their work.

3. Researchers (and in particular students) should post their ideas on public sites to get instant feedback from peers (and document who came up with the idea first).

And here’s how I plan to contribute to the future:

I plan to publish some comments from reviewers, and my response for the ISMIR'07 paper for which I’m first author of (including a link to the paper and the demonstration video).

I plan to write reviews on papers that I read in the future. Maybe I’ll do so for some ISMIR'07 papers that are already online.

However, I’m also planning to spend most of my time in the next weeks helping improve how Last.fm's radio stations listen to their listeners. So it might take me a while to get my "Paper Industry 2.0" contributions started :-)

Btw, using A/B tests to evaluate algorithms on radio stations beats number of citations any time :-)
(And if you like A/B tests, you’re probably also a fan of Greg Linden’s blog.)

UPDATE:
I just read the two links posted in the comments and they are great!
The first link posted by Paul basically talks about how the Nature community is still very old school. The second link posted by Chris talks about how things will need to change in the future and why so many researchers are still very old school.

C4DM’s Listening Room Celebration

2007-06-30T23:50:00.000+01:00

Last Thursday C4DM celebrated their new listening room with an open air concert. The performances included classical guitar duets, modern arrangements of Persian music, beat boxing, electronic music, and lots of great singing. It’s amazing how many excellent musicians they got in their team. Mark Sandler mentioned that they might organize another concert next year.

Interrupting the Silence

2007-06-27T02:02:00.000+01:00

There is an unwritten rule that blogs should be updated frequently... and I’ve been silent for over two weeks now. I’ve even considered discontinuing this blog. The main reason is that I enjoy what I’m currently doing too much to even think about writing a new entry. I guess it’s a side effect of working for the most exciting music 2.0 service in the world :-)

If I’d try to write a paragraph about what it is that currently fascinates me so much I would probably include words such as: music recommendations, music similarity based on audio analysis & collaborative filtering & tags associated with music, computing audio similarity on millions of tracks, playlist generation, connecting users with users with similar tastes, millions of users, scalability issues, evaluation procedures, Hadoop, LSH and other clever ways to deal with lots of data, for example, I only recently learned that an algorithm as simple as one that efficiently computes an intersection can actually be something very interesting :-)

Furthermore, I not only get to work on fascinating problems, I’m also very lucky to be surrounded by lots of very clever and highly motivated colleagues!

Btw, if you’re a frequent Last.fm user you might have noticed some improvements of the weekly recommendations recently... Norman did a great job on that, and he’s got some further significant improvements lined up.

And of course London itself is another reason why I hardly find time to put something on this blog :-)

Despite all these reasons not to write, there is a reason to write. Webcasters in the US have been demonstrating today what might happen if the recent CRB ruling (dramatic increase in royalties) remains unchanged. Last.fm has decided not to join the protest (for an official statement read this). However, there have been ongoing discussions at the office between those who think we should join the struggles of the small webcasters (after all not too long ago Last.fm was one of them), and those arguing that our user’s are already pissed off (because we’ve had way too many technical problems recently and that we really shouldn’t upset them anymore), and those arguing that all of this won’t help us advance the social music revolution. Anyway, it’s good to see that the day of silence has apparently awakened lots of listeners (as I conclude from these comments on digg). There should be a lot more public discussions on how to compensate artists.

Digital Libraries of Music

2007-06-10T18:43:00.000+01:00

Last Wednesday I enjoyed a talk by David Bainbridge at the C4DM about the Greenstone digital library project. David and his team have been developing it for 12 years, and they seem to have been well ahead of their time. The talk inspired me to start dreaming of the ultimate capoeira music digital library. Thousands of capoeira songs and their variations and interpretations (from all well known masters from various capoeira schools) available at a single click, full length audio, rhythm and melody annotations, lyrics and their translations. It would also be great to have wiki style discussions of the meanings and origins of the respective lyrics, links to similar songs (similar with respect to the style, lyrics, melody, rhythm), pictures of people who created them, or even videos of them performing them, …

Even though capoeira music is just a tiny subset of music in general it seems like putting all this information together would be a huge project. The hardest part would probably be to gather the recordings, if they exist at all. As far as I know many songs have already died because they were never recorded or written down :-/

MusicBrainz

2007-06-01T21:24:00.000+01:00

If I were in academia and about to set up a DB for my own evaluations the first thing I’d do is to obtain the MusicBrainz IDs for each track. In the long run it seems like the best way to have a common basis for sharing annotations and other metadata. Even Last.fm will (hopefully soon) support those IDs and then it will be really easy to access the correct tag data (and all the other metadata) for the corresponding tracks. Btw, I heard McGill has already started working with them.

Last.fm and CBS

2007-05-30T11:11:00.000+01:00

Last.fm is going to be working together with CBS, which is known for great TV shows such as CSI. Announcements will follow. Seems like the social music revolution got some serious tailwinds now.

Update: Check out what BBC says about it.

Another Update: Check out blog.last.fm. You'll also find the full story about my recent late night experience at the last.fm office.

Risking our lives for the social music revolution at Last.fm

2007-05-29T01:38:00.000+01:00

This one is a bit off topic, but at least it's work related :-)

Today (Monday 28 May a bank holiday in London), around midnight I was alone at Last.fm’s office going through some comments I’ve received from reviewers (for a journal publication on computational models of similarity for drum sounds). It was nice and quiet and peaceful...

At Last.fm we have an IRC channel to communicate; we are always logged on when we are online (also when we’re not in the office). Here’s an excerpt from the IRC chat: (note that all of us have set highlights to important words such as “beer” or “pub” and some other words)

<elias> i think someone just tried to kick the last.fm office door open :-/
<elias> pub beer porn
<elias> ...
<russ> inner one or outer one?
<elias> I'm inside
<elias> there was a hell of a noise at the door
<elias> i went there, saw that part of the door closing system has come off, and there seems to be a hughe crack in the upper part, I'm not sure if that has been there before
<russ> I don't remember any huge cracks before
<elias> I guess I should have a look outside? ... are there any weapons around? :-)
<russ> this is the white one, not the black one right?
<elias> yes
<muz> :/
<russ> there's an electric drill and a pocket knife by my desk

To find out how the storied continued (obviously I have survived), check out blog.last.fm tomorrow.

Lyrics

2007-05-28T19:05:00.000+01:00

Given the lyrics of a song, algorithms can relatively easily figure out if it’s a Christmas song, a love song, if it contains explicit lyrics, or simply just identify the language it’s sung in. Such algorithms could be used to improve music discovery, recommendation, and playlist generation.

Personally I think lyrics are one of the most important parts of a song. If I don’t like the lyrics, I won’t like the song. I’d love to have lyrics on my display while listening to a song, I’d love to be able to search or listening to music with similar lyrics, there is so much that would be so easy to do if the lyrics would be available, but they are not.

There is an interesting article in the WSJ by Jason Fry on this, which I stumbled upon via this post on the Lefstez letter (which also contains some interesting thoughts on the topic). Sometimes I just don't understand how far detached some parts of the music industry have become from the artists and their fans. And I also wonder how much longer those who are battling to enforce ancient business models will survive.

CrestMuse Webpage & Video

2007-05-28T12:30:00.000+01:00

CrestMuse (the currently biggest MIR related research project in Japan) launched an English version of their project webpage about a month ago. I believe the best part of the web page is the CrestMuse symposium video. The video has been produced very professionally, offers a nice summary of the many aspects of the project, features a native English female narrator, and it even briefly features myself or at least the back of my head (IIRC). Thus, it is well worth the long wait for the download (250MB Mpeg4). My download window says it will take 5 hours until I the download is complete :-/

One of the impressions I got from Japanese researchers is that they are masters of using videos to communicate their work. Wouldn't it be nice if Omras2 would also create similar videos to communicate their work? :-)
And I guess SIMAC's only flaw was that we didn't have such fancy videos.

Music Similarity: G1C Implementation

2007-05-27T18:00:00.001+01:00

I’ve been planning to do this for over a year: The MA (Music Analysis) Toolbox for Matlab now finally includes the G1C implementation which I described in my thesis (btw, the code probably also runs in the freely available Scilab in case you don’t have Matlab). The code is packaged as required for the MIREX’06 evaluation, where the implementation was overall fastest and scored highest (but not significantly better than other submissions).

The code might be useful for those who are new to the field and just want a quick start. Btw, last October I held a presentation on music similarity which might also be helpful for starters and the best documentation and explanation of the code I can offer is my thesis.

I also hope the implementation is somehow useful for those interested in comparing their work on computational models of music similarity to work by others. I believe the best option to do so is to conduct perceptual tests similar to those I conducted for my thesis and those done for MIREX’06 (btw, I wrote some comments about the MIREX’06 evaluation here).

A much easier approach to evaluate many different algorithms is to use a genre classification scenario (assuming that pieces from the same genre are generally more similar to each other than pieces from different genres). However, this doesn’t replace perceptual tests it just helps pre-select the algorithms (and their parameters). Btw, I think it would even be interesting for those working directly on genre classification to compare G1C (combined with a NN classifier) against their genre classification algorithms.

There are lots of things to be careful about when running evaluations based on genre classes (or other tags associated with music). Most of all I think everyone should be using an artist filter: The test set and the training set shouldn’t contain music from the same artists. Some previous work reported accuracies of up to 80% for genre classification. I wouldn’t be surprised to see some of those numbers drop to 30% if an artist filter had been applied.

I first noticed the impact of an artist filter when I was doing some work on playlist generation. In particular, I noticed that songs from the same artist appeared very frequently in the top 20 most similar lists for each song, which makes sense (because usually pieces by the same artists are somehow similar). However, some algorithms which were better than others in identifying songs from the same artists did not necessarily perform better in finding similar songs from other artists. I reported the differences in the evaluation at ISMIR’05, discussed them again in my MIREX'05 submission, and later in my thesis. An artist filter was also used for the MIREX’06 evaluation. Btw, I’m thankful to Jean-Julien Aucouturier (who was one of the reviewers of that ISMIR’05 paper) for some very useful comments on that. His thesis is highly relevant for anyone working on computation models of music similarity.

Another thing to consider when running evaluations based on genre classes is to use different music collections with different taxonomies to measure overfitting. For example, one collection could be the Magnatune ISMIR 2004 training set and one could be the researcher’s private collection. It can easily happen that a similarity algorithm is overfitted to a specific music collection (I demonstrated this in my thesis using a very small collection). Although I was careful to avoid overfitting, G1C is slightly overfitted to the Magnatune collection. Thus, even if G1C outperforms an algorithm on Magnatune, the other algorithm might still be much better in general.

There’s some room for improvements of this G1C implementation in terms of numerical issues, and some parts can be coded a lot more efficiently. However, I’d recommend trying something very different. Btw, I recently noticed how much easier it is to find something that works much better when having lots of great data. I highly recommend using Last.fm’s tag data for evaluations, there’s even an API.

Semantic Music Informatics 2.0

2007-05-24T22:23:00.000+01:00

Oscar Celma posted the link to the slides of the talks of MIR related talks at the recent AES convention in Vienna (on the music-ir list).

Personally (with respect to my own work) I found the slides by Mark Sandler and Oscar most interesting. Mark describes in 12 slides the context and the goals of Omras2 in a nutshell. I don’t know if Mark has been the first to talk about “Music Informatics” but I really like that term. It seems to be a much better description of what I’ve been calling MIR. (Btw, the fonts on the slides are almost unreadable, but the content is worth putting up with that.)

In his talk Oscar makes several interesting points related to my own work at Last.fm. He gives examples for the limitations of the “wisdom of the crowds” and how content-based technologies can be used to push current limits a bit further. Btw, somehow related are two recent and interesting posts by Paul Lamere on artist recommendation and tags.

How much music is out there?

2007-05-21T21:43:00.000+01:00

I tried a quick Google search but couldn’t find what I wanted. I’d like some estimates on how many distinct music tracks are currently on peoples computers worldwide, and how many will be out there in the near future. If you have any pointers to any related estimates, please let me know.

I remember when I started working on MIR research in 2001 I had a collection of 359 (digitized) tracks. I thought that was plenty. Several of the simple computations I ran back then took longer than a night on my computer. Two years later I was dealing with 3961 tracks and thought that was a “large scale” evaluation. When I conducted some of the experiments for my thesis (2005/2006) I thought 20,000 tracks was large scale. Today my colleagues and I are talking about what to do with a few million tracks.

Looking back it seems that roughly every 2 years the number of tracks increased by one order of magnitude. With a bit of extrapolation this leads to more than a hundred million tracks in 4 years. That might sound like a lot if you think of music in terms of what is played on radio stations. But if you think of it in terms of all the music that is created every day, it seems odd that only such a tiny fraction is digitally available.

ISMIR Reviewing

2007-05-19T13:27:00.000+01:00

The ISMIR review deadline is on Monday. On the website they have a nice list showing how much of their work the reviewers have already completed. Next to each reviewer's name there are some smilies (see picture on the right). I got all of mine done, so I got 3 smiles. There are still a large number of reviewers who seem to be waiting until the last moment before submitting their reviews. I hope the acceptance notification is not delayed because of them.

My impression from the papers that I reviewed is that some authors don't seem very concerned about evaluations. In some areas there are currently so many papers that it is impossible to really read all of them. If a paper doesn't offer an evaluation that helps understand how it relates to previous work (and there is lots of previous work on the topics I've reviewed) then I have a very hard time motivating myself to read it :-/

Good News

2007-05-17T22:31:00.000+01:00

Recently the following items have been posted on the music-ir list which I found interesting but haven't had time yet to blog about.

Mark Sandler announced that the Centre for Digital Music (C4DM) has released Soundbite. It's a free playlist generation software which everyone can install on their own music collection. Given a seed song the software generates a playlist of similar songs. In addition, the software sends something like a fingerprint to C4DM's servers which they plan to use to generate recommendations. Mark writes: "and we're not saying what science we do because we'd like to see if you can guess." In terms of similarity I guess they are using Mark Levy's recent work. It's a fast and memory efficient approach and performs almost as good as some computationally much more expensive approaches. I'd love to give it a try, but unfortunately Soundbite currently only runs on Mac OS X.

Chris Cannem from the C4DM recently announced the release of Sonic Visualizer 1.0 which is a nice tool to visualize music (and has partially been developed within the legendary SIMAC project). The plugins include interesting algorithms such as: beat tracking, chromagram, constant Q spectrogram, tonal change detector, key estimation, onset detection, pitch tracking, note tracking, and tempo tracking.

Douglas Turnbull (also known for the MIR related and fun Listen Game) has announced that his team is making some of the data they collected available. In particular, they are offering the "CAL-500 data set of 1700 human-generated musical annotations that describe 500 popular western musical tracks". I haven't seen the data yet, but I'm sure there's lots of interesting things you could do with it.

And last, but not least: There is a new MIR related journal: the Journal of Interdisciplinary Music Studies (JIMS). And guess where it is being published? Izmir :-) And the articles even got Turkish abstracts!

MIR Research at Last.fm

2007-05-17T22:21:00.000+01:00

I’ve joined Last.fm’s last week. Last.fm has over 20 million users and is all about music: listening to music, discovering music, keeping track of your own listening history, showing other people what you listen to, finding other people who listen to similar music, finding out and exploring what friends are listening to, keeping track of events related to music (such as concerts), helping artists find and get in touch with their audience, watching music videos, and more. Last.fm’s team is continuously working on making it easier and more fun to enjoy and discover music. Not even a power outage can stop us ;-)

From an MIR researcher’s perspective there are two particularly notable things about Last.fm. First of all, we got the best user community which provides valuable feedback helping develop new services and improve existing ones. Evaluating MIR technologies has never been easier. Second, we’re constantly receiving information about which tracks are playable in which context, and which words (tags) can be associated with which music. It’s a wonderful playground to explore interesting MIR related questions.

More MIR PhDs

2007-05-17T22:17:00.000+01:00

Kyogu Lee recently announced the defense of his PhD on the music-ir list. The title of his work is "A System for Chord Transcription, Key Extraction, and Cadence Recognition from Audio Using Hidden Markov Models".

Fabien Gouyon pointed me to Alex Loscos’ thesis which was completed recently. The title of his work is "Spectral Processing of the Singing Voice" (abstract and PDF).

Fabien also pointed me to the thesis of Álvaro Barbosa which was completed last year (and which I hadn’t included in the list of MIR PhDs yet). The title of his work is "Computer-Supported Cooperative Work for Music Applications" (PDF).

The music technology group (MTG) in Barcelona is currently by far the most productive MIR research group in terms of completed PhDs. A MTG specific PhD list can be found here.

Recent PhDs

2007-05-03T09:42:00.000+01:00

Paul Brossier recently announced on the music-ir list that his PhD thesis is available for download (PDF). The topic of his thesis is "Automatic Annotation of Musical Audio for Interactive Applications" where he deals with implementations of algorithms that have very short latencies (i.e. they are very fast, and don't require the whole audio file as input to create useful output). Update: see also this post at nest @ The Echo Nest.

Pedro Cano recently successfully defended his thesis in Barcelona. The topic of his thesis is "Content-Based Audio Search from Fingerprinting to Semantic Audio Retrieval" (PDF). He covers a broad range of topics and discusses issues such as the semantic gap. Pedro is currently CTO of BMAT which is an amazing MIR related startup with an unbelievable amount of funding and excellent links to the impressive MIR research group at MTG.

The list of MIR PhDs has been updated.

Recent Events

2007-05-03T01:54:00.000+01:00

The MIREX preparation meeting in Vienna organized by Stephen Downie was a big success. Having the chance to discus the various issues in person is definitely a much more efficient approach than the endless flood of emails we had last year. One of the most discussed topics was audio-based music similarity. Respective discussions ranged from the need for a more specific definition of the task to technical details of large similarity matrices. Andreas Rauber’s team and in particular Thomas Lidy did a great job hosting the meeting. MIREX was also mentioned on the Boing Boing blog. Below is a picture of the participants taken by Rainer Typke. More of his pictures can be found here.

Almost at the same time the ICASSP conference took place in Hawaii which was also well attended by MIR researchers. The next get together seems to be the AES122 workshop on Music 2.0 in Vienna which is organized by Oscar Celma and Mark Sandler. (Update: see also this post by Paul Lamere.)
Btw, 214 papers were submitted to ISMIR this year, which is a 17% increase of the previous record which was set last year (183 submissions).

One a side note, Paul Lamere somehow already guessed it: I’m joining last.fm. I’m starting on Monday will try my best to support the social music revolution :-)