| Abstract | At various machine learning conferences, atvarious times, there have been discussionsarising from the inability to replicate theexperimental results published in a paper.There seems to be a wide spread view that weneed to do something to address this prob-lem, as it is essential to the advancementof our field. The most compelling argumentwould seem to be that reproducibility of ex-perimental results is the hallmark of science.Therefore, given that most of us regard ma-chine learning as a scientific discipline, beingable to replicate experiments is paramount.I want to challenge this view by separatingthe notion of reproducibility, a generally de-sirable property, from replicability, its poorcousin. I claim there are important differ-ences between the two. Reproducibility re-quires changes; replicability avoids them. Al-though reproducibility is desirable, I contendthat the impoverished version, replicability,is one not worth having. |
|---|
