Hyper-parameter optimization tools comparison for multiple object tracking applications

Commonly, an algorithm needs a certain number of variables that control its behavior. The optimal values result in a better performance that could generate profits for companies, make the algorithm stands out from similar applications or improve its ranking in algorithm competitions. However, findin...

Full description

Saved in:
Bibliographic Details
Published in:Machine vision and applications 2019-03, Vol.30 (2), p.269-289
Main Authors: Madrigal, Francisco, Maurice, Camille, Lerasle, Frédéric
Format: Article
Language:English
Subjects:
Algorithms
Communications Engineering
Computer Science
Computer vision
Computer Vision and Pattern Recognition
Image Processing
Image Processing and Computer Vision
Impact analysis
Multiple target tracking
Networks
Optimization
Original Paper
Parameters
Pattern Recognition
Source code
Stability analysis
Tuning
Vision systems
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Commonly, an algorithm needs a certain number of variables that control its behavior. The optimal values result in a better performance that could generate profits for companies, make the algorithm stands out from similar applications or improve its ranking in algorithm competitions. However, finding this values is not straightforward because manual tuning could be a stressful and difficult task even for expert users. This paper presents, evaluates and compares 4 tools in the literature for hyper-parameter optimization, selected due to their number of citations, code availability and impact on literature: MCMC, SMAC, TPE and Spearmint. We evaluate these tools using the publicly available source code provided by the authors in a computer vision application: multiple object tracking (MOT), which is a challenging topic and the strategies rely on a set of tunable parameters. This analysis considers the impact of hyper-parameter optimization tools in terms of stability, performance, usability, among others. The evaluations are carried out using public benchmarks such as PETS09 and ETH and the results show how these tools change the performance of a MOT framework and how this would affect the results of real ranked competitions. Our goal is (1) to encourage the reader to use these tools and (2) to provide some guidelines that helps anyone when tuning his/her methods.
ISSN:0932-8092
1432-1769
DOI:10.1007/s00138-018-0984-1