SILA: a system for scientific image analysis

A great deal of the images found in scientific publications are retouched, reused, or composed to enhance the quality of the presentation. In most instances, these edits are benign and help the reader better understand the material in a paper. However, some edits are instances of scientific miscondu...

Full description

Saved in:
Bibliographic Details
Published in:Scientific reports 2022-10, Vol.12 (1), p.18306-18306, Article 18306
Main Authors: Moreira, Daniel, Cardenuto, João Phillipe, Shao, Ruiting, Baireddy, Sriram, Cozzolino, Davide, Gragnaniello, Diego, Abd-Almageed, Wael, Bestagini, Paolo, Tubaro, Stefano, Rocha, Anderson, Scheirer, Walter, Verdoliva, Luisa, Delp, Edward
Format: Article
Language:English
Subjects:
639/705/1046
639/705/117
639/705/258
639/705/794
Automation
Computer engineering
Computer science
Coronaviruses
Datasets
Digital imaging
Electrical engineering
Experiments
Forensic sciences
Humanities and Social Sciences
Humans
Image processing
Image Processing, Computer-Assisted
multidisciplinary
Photography
Professional misconduct
Science
Science (multidisciplinary)
Scientific Misconduct
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A great deal of the images found in scientific publications are retouched, reused, or composed to enhance the quality of the presentation. In most instances, these edits are benign and help the reader better understand the material in a paper. However, some edits are instances of scientific misconduct and undermine the integrity of the presented research. Determining the legitimacy of edits made to scientific images is an open problem that no current technology can perform satisfactorily in a fully automated fashion. It thus remains up to human experts to inspect images as part of the peer-review process. Nonetheless, image analysis technologies promise to become helpful to experts to perform such an essential yet arduous task. Therefore, we introduce SILA, a system that makes image analysis tools available to reviewers and editors in a principled way. Further, SILA is the first human-in-the-loop end-to-end system that starts by processing article PDF files, performs image manipulation detection on the automatically extracted figures, and ends with image provenance graphs expressing the relationships between the images in question, to explain potential problems. To assess its efficacy, we introduce a dataset of scientific papers from around the globe containing annotated image manipulations and inadvertent reuse, which can serve as a benchmark for the problem at hand. Qualitative and quantitative results of the system are described using this dataset.
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-022-21535-3