Visual Information Processing and Protection Group

Reproducible Signal Processing

Reproducible Signal Processing
M. Barni, F. Perez-Gonzalez,
Pushing Science into Signal Processing, IEEE Signal Processing Magazine 22(4):120 - 119, August 2005, DOI: 10.1109/MSP.2005.1458324

Many image processing projects lack meaningful experimental validation. Despite the promising results achieved so far, the application of AI-based methods for MMF is seriously hindered by many shortcomings: 

  1. Theoretical papers are validated by a single experiment on a single image                                
  2. Results need meaningful comparison with competing schemes                                                     
  3. Results are justified with only one similar algorithm performing worse                                        
  4. Vaguely described experimental conditions make replicability impossible                                 
  5.  Readers must simply trust the authors!

Strong experimental analysis and setup allow to achieve high reproducibility, thus algorithms and experimental setup should avoid ambiguity.  So, What is the problem?

  1. Lack of time to implement someone else's algorithms                                                                         
  2. Lack of researchers to re-implement useful algorithms                                                                                                                            
  3. Intellectual properties  of the proposed algorithms        

Reproducible Signal Processing is the solution: share the software and the data set!

Many image processing projects lack meaningful experimental validation. Despite the promising results achieved so far, the application of AI-based methods for MMF is seriously hindered by many shortcomings: 

  • Software usability: which format? Use standardized solutions
  • Software readability: can other people understand what you are doing? Document your code thoroughly
  • Licensing problems? Spread your knowledge in an open source format when possible otherwise share your binaries
  • Algorithms and experiments? Describe them with the right level of detail
  • Algorithm description? Provide a block diagram or a pseudo-code
  • Algorithm parameters? Provide a summary table or a detailed list
  • Datasets? Provide all the source data needed to obtain the same results of the original paper

© Copyright 2023 VIPP Group - All Rights Reserved

Made with Mobirise site theme