Subsample, Generate, and Stack Using the Spiral Discovery Method: A Framework for Autoregressive Data Compression and Augmentation

• Published in: IEEE transactions on systems, man, and cybernetics. Systems man, and cybernetics. Systems, 2024-11, Vol.54 (11), p.7129-7142
• Main Author: Csapo, Adam B.
• Format: Article
• Language: English
• Subjects: Data compression; Data models; Dataset augmentation; dataset compression; Interpolation; Mathematical models; spiral discovery method (SDM); spiral optimization; Spirals; Training; Training data
• ISSN: 2168-2216; 2168-2232
• DOI: 10.1109/TSMC.2024.3448206

This article addresses the challenge of efficiently managing datasets of various sizes through two key strategies: 1) dataset compression and 2) synthetic augmentation. This article introduces a novel framework, referred to as subsample, generate, and stack (SGS), which can be used to implement both of these strategies while maintaining the statistical characteristics of the original data. While SGS can be paired with a variety of generative methods, this article specifically demonstrates its application using the spiral discovery method (SDM)-an autoregressive data generation model that allows for the exploratory manipulation of numerical data. The uniqueness and widespread applicability of this approach stems from its support for the fine-grained optimization of exploration versus exploitation goals through an interpretable set of hyperparameters. The effectiveness of the SGS framework combined with SDM is validated on two benchmark examples-one focusing on compression and the other on augmentation-showcasing its potential as a tool for dataset management in engineering contexts.