What makes computational communication science (ir)reproducible? | Amsterdam University Press Journals Online
2004
Volume 6, Issue 1
  • E-ISSN: 2665-9085

Abstract

Computational methods are in full swing in communication science. Part of their promise is to make communication research more reproducible. However, how this plays out in practice has not been systematically studied. We verify the reproducibility of the entire cohort of 30 substantive and methods papers published in the journal Computational Communication Research (CCR), the official journal of the ICA Computational Methods Division with a focus on transparency and hence a high rate of voluntary Open Science participation in the field. Among these CCR papers, we are not able to verify the computational reproducibility of 16 papers as no data and/or code were shared. For the remaining 14 papers, we attempt to execute the code shared by the original authors in a standardized containerized computational environment. We encounter a variety of issues that preclude us from reproducing the original findings, where incomplete sharing of data or code is the most common issue. In the end, we could at least partially reproduce the findings in only 6 papers (20%). Based on our findings, we discuss strategies for researchers and the subfield to correct for this disheartening state of computational reproducibility.

Loading

Article metrics loading...

/content/journals/10.5117/CCR2024.1.5.CHAN
2024-01-01
2024-07-23
Loading full text...

Full text loading...

http://instance.metastore.ingenta.com/content/journals/10.5117/CCR2024.1.5.CHAN
Loading
  • Article Type: Research Article
Keyword(s): computational communication science; Open Science; reproducibility
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error