ISMIR 2008 seems extremely well organized. I've been only watching from the side line this year (for the first time since 2002 I haven't submitted a paper myself) but the bits and pieces I've seen seem great.
I like how reviews are double blind this year.
I like how authors get a chance to respond to reviews. And reviewers get a chance to reconsider their ratings after seeing the response of the authors, and what their fellow reviewers wrote.
However, and this strikes me as fascinating: I get to see the name of my fellow reviewers! (To be more accurate: I only get the see reviews and names of reviewers who reviewed the same content I did.) First I thought it was a bug in the system (I even felt the urge to instantly report it to the program chairs). But it seems like the whole system is designed around exposing the real names of the reviewers to the fellow reviewers: I was just sent an email with the email addresses of all my fellow reviewers.
I think I've never experiences such openness in any of the review processes I've been involved in. It's fascinating, but it makes me wonder if that might lead to reviewers being more reluctant to write critical remarks in the future? Especially in such a small community as the ISMIR community, one of the fellow reviewers might be a colleague of one of the authors etc. While I think it's a good idea to publish reviews and a response to those (I did so for one of my publications last year here), I don't think it's necessarily a good idea to expose the reviewers without their consent.
Subscribe to:
Post Comments (Atom)
1 comment:
I generally agree with your comments here, though I am working through this new and improved ISMIR review process as an author not a reviewer.
While I like the response idea, my issue with it is that there was no previous notification that such a response period would exist or when it would be and then it was very short. This could (and I imagine did) cause an author to not respond to reviews because they were on holiday (as I was) or at a conference last week (i.e. AES or CMMR) and unable to read email or formulate a proper response. I think this problem could be mitigated in the future by putting the dates for the response period in the call and possibly extending the length of time for the response. Personally, I received my reviews on May 15th, which gave me a very short 5 days to respond, 3 of which I was traveling. Anyway, those are my Friday afternoon thoughts on the matter.
(hopefully my paper will be accepted...)
Post a Comment