Category Archives: CCR January 2018

Thoughts and Recommendations from the ACM SIGCOMM 2017 Reproducibility Workshop

Damien Saucez, Luigi Iannone

Abstract

Ensuring the reproducibility of results is an essential part of experimental sciences, including computer networking. Unfortunately, as highlighted recently, a large portion of research results are hardly, if not at all, reproducible, raising reasonable lack of conviction on the research carried out around the world.

Recent years have shown an increasing awareness about reproducibility of results as an essential part of research carried out by members of the ACM SIGCOMM community. To address this important issue, ACM has introduced a new policy on results and artifacts review and badging. The policy defines the terminology to be used to assess results and artifacts but does not specify the review process or how to make research reproducible.

During SIGCOMM’17 a side workshop has been organized with the specific purpose to tackle this issue. The objective being to trigger discussion and activity in order to craft recommendations on how to introduce incentives for authors to share their artifacts, and the details on how to use them, as well as defining the process to be used.

This editorial overviews the workshop activity and summarizes the main discussions and outcomes.

Download the full article DOI: 10.1145/3211852.3211863

A Survey on Artifacts from CoNEXT, ICN, IMC, and SIGCOMM Conferences in 2017

Matthias Flittner, Mohamed Naoufal Mahfoudi, Damien Saucez, Matthias Wählisch, Luigi Iannone, Vaibhav Bajpai, Alex Afanasyev

Abstract

Reproducibility of artifacts is a cornerstone of most scientific publications. To improve the current state and strengthen ongoing community efforts towards reproducibility by design, we conducted a survey among the papers published at leading ACM computer networking conferences in 2017: CoNEXT, ICN, IMC, and SIGCOMM.

The objective of this paper is to assess the current state of artifact availability and reproducibility based on a survey. We hope that it will serve as a starting point for further discussions to encourage researchers to ease the reproduction of scientific work published within the SIGCOMM community. Furthermore, we hope this work will inspire program chairs of future conferences to emphasize reproducibility within the ACM SIGCOMM community as well as will strengthen awareness of researchers.

Download the full article DOI: 10.1145/3211852.3211864