Visual information retrieval from historical document images

• Published in: Journal of cultural heritage 2019-11, Vol.40, p.99-112
• Main Authors: Zhalehpour, Sara, Arabnejad, Ehsan, Wellmon, Chad, Piper, Andrew, Cheriet, Mohamed
• Format: Article
• Language: English
• Subjects: Footnote detection; Historical document image classification; MFCC; SVM; Table detection; Visual information retrieval
• ISSN: 1296-2074; 1778-3674
• DOI: 10.1016/j.culher.2019.05.018

Information retrieval from documentary heritage is considered a challenging issue because of the documents’ unique structures and level of degradation. Text characters in printed documents historically are accompanied by typographical objects. Retrieving and pursuing these visual typographical elements, which inform the content of historical manuscripts can help us better understand our documentary cultural heritage. Extracting these visual objects aids us in understanding and conveying more information about different practices of representation in historical documents and their effects on the current trends of publishing. Two important typographical objects related to the history of knowledge and information are footnotes and tables; the former are one of the critical elements that demonstrate authority and link the manuscript to its sources, and the latter summarize information in a compact and organized manner essential to the growth of scientific knowledge. To the best of our knowledge, there is currently no work that considers in depth the automated detection of these two typographical objects from the large collections of historical documents that would allow further historical study. This article focuses on the problem of detecting the presence of these two visual elements from historical printed documents and establishes two frameworks. The footnote detection framework uses a set of layout-based methods to extract some features regarding the font and appearance, and the table detection framework extracts spectral-based features from the images. These frameworks are tested on a large collection of 18th-century printed documents with more than 32 million images, and the results show their effectiveness and generalization power.