Friday 30 March 2007

Listen Game

Listen Game is yet another music tagging game. Similar, but very different to the Major Minor game.

So what's the difference to the Major Minor game? Instead of coming up with tags you get to choose from a predefined set (most of the time). And unlike the Major Minor game, you always get to know how others described the music you just listened to. If you liked the Major Minor game, you should definitely try the Listen Game... it's lots of fun!

I had the pleasure to work together with Doug last summer in Tsukuba. I was skeptical when he talked about the ESP game and how it would be nice to do something similar for music. I believed music and images are conceptually too different. I'm very glad my skepticism hasn't influenced him :-)

Why was I so skeptical? Well, I always am. In addition, it only takes one look to get an overview of an image, but it takes a lot longer to get an overview of, for example, a Mozart sonata or even a 3 minute pop song. Furthermore, I believe the key point about the ESP game is the interaction with another person... I thought that for music the ratio between time spent viewing/listening to the content and time spent interacting would not make it an attractive game.

However, both Listen Game and the Major Minor game use only excerpts of music. Also both games developed very clever and very different ways to allow the players to interact with each other without needing to wait for each other. I'd love to see more games like these :-)

See also Paul's comments on the game.

MIR on Wikipedia

I just noticed the music information retrieval entry on Wikipedia. It's the first hit you get when you search for music information retrieval in Google.

There are no sections on:
- open source MIR tools (e.g. Marsyas, CLAM, ...)
- relevant MIR conferences (ISMIR, ...)
- history of MIR (e.g. first TREC-like evaluation in 2004 etc.)
- ...

The Wikipedia entry on information retrieval might be a nice starting point?

Thursday 29 March 2007

MIR Research Project: OMRAS2

I'm fascinated by Omras2. They got an amazing team and 4.5M Euros* 3.7M Euros to spend in the next 3 years. They are extremely interdisciplinary with excellent connections in many directions. And they have very ambitious goals.

They talk about large collections, and with large they mean millions of tracks. They talk about software that will help "you build playlists with songs that you'll love even though you never heard them before".

However, Omras2 is not just about navigation in huge music collections, playlist generation, and music recommendation. One of their main topics is developing tools for music researchers (like musicologists) and even record producers.

I wouldn’t be surprised if they manage to achieve their goals. Somehow it just all seems to fit together perfectly.

Of particular interest I also find their project wiki. I’ve never seen such a transparent research project. You’ll find information about their meetings, their work packages, and deliverables.

Btw, talking about great MIR research projects... you might also be interested in SIMAC (2004-2006). The project I’m currently part of will soon make an English version of their website available (and I promise I’ll blog about it).

Oh, in case you have been wondering what Omras2 stands for: Online Music Recognition And Searching. The 2 is in there because it's a successor to the successful Omras project.

* Update: The grant is about 3.7M EUR (about 5M USD) and not 4.5M EUR. Sorry about that. To be precise: it's 2.5M GBP. That's still HUGE. For example, SIMAC (which was the research project with the largest impact on the MIR community in the last years) had a grant of about 3M EUR which was split between 5 partners in 4 different countries. Omras2 is focused mainly on only two research labs, both located in London.

Wednesday 28 March 2007

The Major Miner Game

I’m sure you heard of the major minor music game. It was posted on the music-ir list and Paul has blogged about it. You probably also know that Paul recently lost his lead (he had the highest overall score for at least a week I think), and that many other researchers (including Martin and myself) had made it into the top 10 at some point.

The clever game was also mentioned on the Listening Post. It would be nice to see it climb high in Digg’s playable web games section.

Maybe such games could be the solution to automatic segmentation, music classification, maybe also music recommendation, and replace various other content-based techniques we are still working on?

Instead of teaching computers how to understand music we just need to find clever ways to motivate those 8.5 millions playing World of Warcraft to play similar games. I think that might even solve the various scalability issues Jeremy and Paul mentioned recently.

Recent PhDs

Two new PhD theses have been added to the list of MIR related PhDs recently.

Not too long ago Rui Pedro Paiva announced his completed thesis on the music-ir mailing list. The title of this thesis is "Melody Detection in Polyphonic Audio" written at the University of Coimbra in Portugal. One of his conclusions is that most of the research problems involved in melody detection are complex and still open. So I hope we will see many more PhDs on this topic.

The second PhD added to the list was written by Katsuhiko Kaji from Nagoya University, Japan. (Thanks to Masataka for pointing this out to me!) He wrote a thesis in Japanese with the English title: "A Study on a Universal Platform for Digital Content Annotation and its Application to Music Information Processing". I've seen some work on annotation in the MIR research community, but I don't remember seeing anyone trying to figure out a framework that can be applied to multimedia in general (including text, video, music and so on). If you're interested check the abstract and I'm sure Katsuhiko will be happy to answer any questions you might have.

There are at least two more PhDs from Japan that I hope to add to the list soon. If you know anyone that is about to finish, or has already finished their PhD related to MIR, please let me know.

Monday 26 March 2007

ISMIR Steering Committee and Free Beer

Jeremy suggested that I should get involved in hosting ISMIRs. I'd love to do so, but I don't have the option to do so. (Unfortunately I left Vienna before they got to start organizing ISMIR. And since I'm leaving Japan now, I am not involved in plans that go beyond April.) Anyway, curious as I am I checked the last call for hosting an ISMIR, and found some documents... and now I know why we don't want any additional members on the steering committee! :-)

First I found the call for hosting an ISMIR. Where I found a link to the ISMIR booklet. On page 12 there's finances (a list of the main costs a host should anticipate), on page 13 there's the last item on the list: "Travel/accommodation costs of SC". (SC stands for steering committee.)

So an average flight might cost about 700 Euros. 5 nights accommodation could easily add up to about 500 Euros. The registration fee could be about 300 Euros (I guess that's also included in the SC package?) and that sums up to about 1500 Euros per member. If I assume that 250 people attend ISMIR, that splits to 6 Euros per ISMIR participant. So any additional member of the steering committee means one free (alcoholic) drink less at the conference :-/

Realizing this I've come to the conclusion that I'd prefer to have one more beer instead of an additional SC member... in contrast to what I suggested in my previous post. Prost! (As we will be saying in Vienna.)

Btw, that document has lots of interesting information. (Assuming that like me, you are interested in what happens backstage of ISMIR.)

First of all, as I probably could have figured myself, the SC gets to select the yearly ISMIR host and chairs of future conferences. The SC also has veto power on invited speakers and can overrule the program chair with respect to the papers that are accepted. The document also explains that the size of the committee is limited to 12, that members get voted in by other members of the SC, and that the membership doesn't expire.

So if, in the future, you hear me talking about how perfect the current size of the SC is, and how nothing should be changed... you know why ;-)

Saturday 24 March 2007

Is industry outrunning MIR research?

I feel lost when trying to get an overview of all the different MIR related services emerging monthly. Industry is moving extremely fast and coming up with solutions to problems which some researchers (like me) were trying to solve.

For example, I've done some work on algorithms that can classify genres by analyzing audio. The quality of the best algorithms I've seen is not so great. On an interesting music collection with several not completely obviously distinguishable genres (obvious like techno vs. death metal) the algorithms perform a lot worse than humans. Why bother with that if a better solution already exists? With a better solution I mean one that is not limited to only genres, one that assigns multiple categories to each item: tagging by people. If you doubt it, you might want to check out the tags. I (and many other researchers) have been checking them out recently (check out Paul’s blog for some interesting ideas and comments on tagging).

So instead of trying to do some simple mathematics to understand music, just give millions of users the option to tag songs and artists (and give them benefits for doing so). If you don't trust the masses, hire experts. If you think I'm kidding check out If my ears didn't fail me, Tim Westergreen (founder of recently said in an interview that they got 600.000 tracks in their database. He also once mentioned that it takes an expert 20 minutes to annotate each track. (100 person years of work - if it’s true. Spare a moment to think of all the wonderful MIR research things you could do with that kind of data.)

Btw, it's not just music classification that seems solved. You’ll find services that successfully create playlists, give recommendations, ...

I'd also dare to say that it's not just stuff I've been working on that seems to be outrun by industry. Have you ever tried Midomi query-by-humming (or singing, or whistling)? It works!

People in the MIR research community have been working on query-by-humming for many years. As far as I know the mainstream research direction was to extract the melody information from human input, and compare it to melody information extracted from the music, and match them.

I know nothing about query-by-humming research. However, I've seen demonstrations and the results were not so great. Some people (like me) don’t sing/hum accurately enough. Furthermore, extracting melody information from songs is a lot harder than it seems. So while researchers were working on the problem and envisioning how great a system would be that allows you to query a large archive by humming, the problem was solved by Midomi.

Midomi found a shortcut to the problem. Instead of relying on computers that understand music, they found a clever way to use the information humans can give them easily. (For an interesting analysis check out Cristian Francu’s post on the music-ir list on 2007/02/02.)

I'm not saying that any of the research was wasted. Not one second of it! Extracting (and matching) melody information is extremely interesting. It's a big step towards truly understanding music. Also developing algorithms that can analyze music and classify it into styles/genres/moods will always remain an extremely interesting research direction, despite tagging. However, it seems that in some cases the motivation for our work needs to be reformulated.

Or maybe MIR research should just give up? Wait and see what industry comes up with and then solve the problems that are left over?

Well, I got to continue writing my ISMIR paper now... which btw is my answer to all these questions :-)

And I hope everyone else is busy writing their ISMIR papers, too.

Friday 23 March 2007

ISMIR Steering Committee

ISMIR is by far my most favorite conference. I've attended all ISMIRs since 2002. My research has been revolving all around the ISMIR community, I've learned a lot from ISMIR papers (Btw, most papers I cite are ISMIR papers), I learned a lot from conversations I had at ISMIR, I got my current job through ISMIR, and even my next job... but only recently I discovered the ISMIR steering committee.

Now that I discovered it, I realize that many people that seem really important to me in the community are not included. Which seems a bit odd. In particular, I wonder why none of the following people are included: (in alphabetical order)
Anssi Klapuri
Has done some amazing research, has been very active in the community, helping many others. Has given many interesting talks, got involved in evaluation efforts like MIREX...

Dan Ellis
Another person everyone knows, great research, very active, in particular when it comes to evaluation. Has shared a lot of data, code, has written many frequently cited papers, ...

Francois Pachet
I guess I don't need to mention anything here? Has pioneered many very interesting research directions, is always happy to discuss research topics, inspirational, ...

George Tzanetakis
Another legend... also co-organizer of ISMIR last year. He gave us Marsyas, many great papers, and genre classification.

Gerhard Widmer
Co-organizer of ISMIR this year. He was my PhD supervisor, so I might be biased. But even if I look at this entirely objectively, I'd definitely vote for him being on the committee.

Mark Sandler
Co-organizer of ISMIR in London. Runs a great (and very large and growing) team of music researchers in London. Involved in some amazing MIR related projects...

Masataka Goto
Again I might be biased because I'm working with him right now. But the same applies as in Gerhard's case. Furthermore, having him in the committee would bring the Japanese MIR community closer to the international one.

Paul Lamere
Paul has been very active in the MIR community, attended the last ISMIR conferences, writes a great blog covering lots of MIR topics. Furthermore, he has a great overview of MIR topics outside of academia ...

Xavier Serra
Organizer of ISMIR 2004. Also legendary in the MIR community. Well known for his ground breaking PhD thesis and for being an excellent manager.

I should probably mention that I didn't ask any of the people I'm listing here. So some (or most of them?) might not even want to or might not have the time to be on the committee (which might explain why they aren't right now?) And I'd also like to add that this is only a very short list. There are many more I think would be very suitable to be on the committee (Andreas Rauber, Michael Casey, Simon Dixon, ...) but I felt the list was getting a bit long.

I wonder how new members get added to the committee? Can I just post some suggestions on the music-ir list? Is there a limit on the size of the committee? What about some form of society? I think Michael Fingerhut once mentioned that ISMIR could stand for the "international society of music information retrieval". That would be wonderful.

Tuesday 6 March 2007

MIR Related Newsbits

Last week the CRB approved an increase in royalty fees (proposed by the RIAA) for Internet radio (only the US is affected). I can't believe that they will succeed with this, even if the CRB has approved it. Paul wrote a nice summarizing entry in his blog.

Recently the RIAA seems to have found a very efficient way to sue people who share music via P2P networks ( They've been testing it on college students in the last weeks. Seems as though they will be using this approach with any ISP in the future. I'm guessing that more people in the US will now start to use P2P systems which make it impossible to track those sharing music. Noteworthy are also alternatives to P2P which are emerging... e.g. Amie Street has an interesting price model and sells the music DRM free. (BTW, pressure on iTunes and other closed-DRM systems is continuously mounting in Europe.)

The IFPI has put pressure on the MP3 blog the "Late Greats". If this is not a single incident then this could have a negative impact on the Hype Machine, or even interesting work within MIR (see e.g. Oscar Celma's work).

Last month has signed deals with Warner and EMI. ( is one of the few companies actively seeking cooperations with MIR researchers.)

There is some confusion with MP3 licensing (it is not clear who owns which rights of the MP3 format). Using alternatives such as Ogg seems a good solution.

Friday 2 March 2007

MIR in Japan


Japan is a wonderful place for MIR research. There is a very strong community, they got good funding, and they are moving fast.

The largest Japanese society related to MIR is SIGMUS (special interest group on music and computers). It has got over 400 members interested in topics related to music processing. They organize 5 meetings a year. Around 50-80 participants usually attend these meetings which are held at different cities in Japan.

SIGMUS Demonstration Session
[Demonstration session at a SIGMUS meeting]

The best known Japanese MIR researcher is Masataka Goto. In the last 10 years Masataka has published on various topics such as beat tracking, chorus detection, and new user interfaces to discover music and organize playlists.

[Masataka's office]

CrestMuse Logo
One of the largest MIR related projects in Japan is the CrestMuse project which brings together universities and research labs all over Japan. CrestMuse is funded by the Japanese government (about 2.5 million Euro). It's a 5 year project and started October 2005. CrestMuse is headed by Haruhiro Katayose.

If you want to do MIR research in Japan check out the JSPS fellowships. It's a lot of money, it's flexible, and it's not impossible :-)

European MIR researchers currently working in Japan include Jean-Julien Aucouturier, Bee Suan Ong, Sebastian Streich, and myself. Arthur Flexer just left a few days ago. From the Americas Douglas Turnbull worked two months in Japan last summer. J. Stephen Downie and George Tzanetakis will be stopping by soon.

Thursday 1 March 2007

Echo Nest Expansion

The Echo Nest announced that they won:
"the Young Entrepreneur Initiative (YEi), a nation-wide contest organized by the French embassy in the US, which aims to help create or develop most innovative companies in France."

Seems like they are successfully expanding to Europe! I'm very curious what Brian Whitman and his team will come up with.

Recent PhDs

Bee SuanNickRainerSebastianJan-Mark

5 PhDs were recently added to the list of MIR PhDs: Nicholas M. Collins, Sebastian Streich, Bee Suan Ong, Rainer Typke, and Johann-Markus Batke. Sebastian and Bee Suan have joined Yamaha in Japan. Rainer has joined the legendary intelligent music processing group at OFAI (who will be co-organizing ISMIR this year).

It seems like this year the number of completed PhDs could be higger than in any previous year. Other candidates in the queue include Enric Guaus, Roy Pierre, Kris West, Perfecto Herrera, Adam Berenzweig and many more.

BTW, Thomas Lidy recently completed his Master's thesis. The title is "Evaluation of New Audio Features and Their Utilization in Novel Music Retrieval Applications" (PDF). Thomas has been active in the MIR community (publishing at ISMIR and participating in MIREX for the last two years). He is a core team member of Andreas Rauber's group who will be co-organizing ISMIR this year.