Visual Information Processing and Protection Group


PREserving Media trustworthiness in the artificial Intelligence ERa

  • FUNDING INSTITUTE - MIUR, Ministero dell'Istruzione dell'Università e della Ricerca
  • BUDGET -  670.000 EUR
  • PARTNERS - Università degli Studi di Siena, Università degli Studi di Firenze, Università degli Studi di Trento, Politecnico di Milano, Università degli Studi di Napoli Federico II

While the appearance of AI-based editing tools is only the last, most dramatic, step towards the final delegitimization of digital media as a trustworthy representation of reality, MultiMedia Forensics (MMF) researchers have started looking at AI as a way to preserve the dependability of digital media. In the last years, several detectors based on Convolutional Neural Networks (CNNs) have been developed to detect whether an image, or a video, has been manipulated, or to gather information about its history. 

Despite the promising results achieved so far, the application of AI-based methods for MMF is seriously hindered by a number of shortcomings including: 

  1. The necessity of training the MMF detectors on a huge amount of data which is representative of the variety of situations encountered in real-life
  2. The difficulty of interpreting the results provided by CNN detectors                                             
  3.  The lack of security of CNN detectors, as witnessed by the ease with which adversarial examples capable of deceiving them can be generated.

The goal of PREMIER is to overcome the above shortcomings and develop a new class of AI-based MMF tools, that can be successfully used to preserve the dependability of digital media. To do so, PREMIER will pursue a novel approach whereby AI-techniques are enriched with a model-based signal-processing view point. In this way, the strengths of data-driven techniques will be maintained and any available information stemming from the scenario at hand exploited. In this vein, the need for huge amount of training data will be relaxed by constraining the network structure and the training process so to orient the analysis towards task-relevant features.

The strong connection between certain classes of constrained CNN and signal processing methodologies will be exploited to ease the interpretability of the forensic analysis. The use of signal-processing-oriented structures, in fact, eases the construction of temporal and spatial heat-maps showing which parts of the analyzed media contributed most to the detector’s outcome. A better interpretation of the analysis will also ensure that MMF detectors base their decisions on task-relevant features avoiding so-called confounding factors, which can be easily attacked by a forger. Still regarding security, the joint use of self-learned and handcrafted features will be exploited to improve the resilience of the detectors against deliberate attacks.

The project will mainly focus on video forensics, due to the importance that video has in the formation of opinions and the diffusion of information, and because research in video forensics is much less advanced than for still images, and hence represents a more challenging area wherein the soundness of PREMIER viewpoint can be verified.

© Copyright 2023 VIPP Group - All Rights Reserved

Mobirise web page software - Click now