Replicability Label

Again, this year ACM WiSec continues the effort pioneered in 2017, towards supporting greater “reproducibility” in mobile and wireless security experimental research. The goal of this process is to increase the impact of mobile and wireless security research, enable dissemination of research results, code and experimental setups, and to enable the research community to build on prior experimental results. We recognize papers whose experimental results were replicated by an independent committee and provide a “replicability label” accordance to the terminology defined by ACM.

Authors of accepted papers can participate in this voluntary process by submitting supporting evidence of their experiments’ replicability. Authors are encouraged to plan ahead when running their experiments, in order to minimise the overhead of applying for this label.

If the committee can verify that all relevant data sets were included and the graphs/tables can be regenerated based on this, the committee will grant a replicability label and also provide a report on the regeneration process.

Review process

The authors upload a VM containing all the necessary data and code required to replicate the results. At least two reviewers are asked to replicate the results for each submission. We ensure that the VMs are self contained to the maximum extent possible to eliminate any future version deprecation and backward compatibility issues. The reviewers clarify any issues directly with the authors (and may request updates to the original VM and code).

Publication of data and reproducability environments

The VMs will be made available, as always, on the ACM WiSec datasets server at wisecdata.ccs.neu.edu after the conference.