Edit Event: ICML ML Evaluation 2009

Event description:

<pre> Call For Submissions to ICML 2009 Workshop The Fourth Workshop on Evaluation Methods for Machine Learning June 18, 2009 Montreal Canada. William Klement, Chris Drummond, Nathalie Japkowicz, and Sofus Macskassy. Topics ====== The fourth in a series, this workshop intends to continue the debate within the machine learning community into how we evaluate new algorithms. We aim to discuss what properties of an algorithm need to be evaluated (e.g., accuracy, comprehensibility, conciseness); to solicit views and suggestions for other approaches than those currently used; to investigate alternate methods that could be useful. In the course of three previous workshops, the debate has evolved to focus around the following issues which have captured the interest of the community: * the role of experiments in evaluation * the use of one, community wide, evaluation measure (e.g., Accuracy, AUC, F-measure) * the relevance of statistical tests to evaluation * the effectiveness of the UCI data sets for evaluation * the need for sharing and characterizing benchmark data sets in general * how to promote the views of this workshop to the rest of the community The 2008 ICML workshop concluded with agreement that we, as a scientific community, should substantially change how evaluation is performed in machine learning. We, however, disagreed on the direction that this change should take. As a continuation of the same theme, we aim to solicit views, intuitions and visions of alternatives to change existing evaluation methods. We hope to make progress but still carry forward the good methods and experiences we already have acquired. We invite position and technical papers concerning the following issues: * advantages of existing evaluation methods * critiques of current evaluation practices * intuitive and creative alternatives * performance evaluation issues of concern to the community * future directions and evolutions of evaluation methods in machine learning This list certainly does not capture all the issues worthy of discussion nor the possible positions. We expect, and very much encourage, position papers raising other issues that members of the machine learning community think are important. Format ====== The day-long workshop consists of: Invited talks: we plan to have several invited speakers. Some, from outside of our research community, will be able to criticize our accepted practices from an external point of view. Some, from inside our community, will discuss how we could improve on our current practices. Panel Discussion: our invited speakers will be asked to engage each other on the various issues surrounding the problem of evaluation in Machine Learning at the end of the workshop. The audience will be strongly encouraged to participate in the discussion. Presentations: The papers accepted to the workshop will be presented throughout the day between the various invited talks. Papers will be grouped by theme, in order to facilitate discussion at the end of each session. Workshop attendance is open to the public and is estimated at 20-25 attendees. Priority will be given to those active participants in the workshop (paper authors or speakers). Submissions =========== Authors are invited to submit papers on the issues listed above or other related positions. Technical or position reports will be considered for this workshop. To promote a lively event, with plenty of discussion, the organizers are very interested in papers taking strong positions on the above topics. Workshop papers should be at least 1 page long and should not exceed 4 pages using the ICML'09 Style. Submissions should be made electronically in PDF or Postscript format and should be sent (no later than April 8, 2009) by email to: William Klement, email: klement -at- site -dot- uottawa -dot- ca Each submissions will be reviewed by two members of the organizing committee. A notification of acceptance will be sent by: April 30, 2009. Authors will be asked to submit a final version of their submission to be posted on the workshop website. Important Dates =============== April 8, 2009 Submission due date April 30, 2009 Author notification Date June 1, 2009 Final version ready June 14-18, 2009 ICML 2009 Conference June 18, 2009 Workshop Date Organizers ========== William Klement (main contact) SITE, University of Ottawa 800 King Edward Ave. P.O. Box 450, Stn A Ottawa, ON K1N 6N5, Canada. Telephone: (613) 562-5800 ext. 6699 Fax: (613) 562-5664 E-mail: klement@site.uottawa.ca Chris Drummond NRC Institute for Information Technology, Canada Email: chris.drummond -at- nrc-cnrc -dot- gc -dot- ca Nathalie Japkowicz School of Information Technology and Engineering University of Ottawa, Canada Email: nat -at- site -dot- uottawa -dot- ca Sofus A. Macskassy Fetch Technologies United States of America Email: sofmac -at- fetch -dot- com Workshop URL ============ http://www.site.uottawa.ca/ICML09WS </pre>This CfP was obtained from [http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=4967&amp;copyownerid=3382 WikiCFP]

Summary:

This is a minor edit Watch this page

Cancel