A Distributed Microservice Architecture Pattern for the Automated Generation of Information Extraction Pipelines

Companies often build their businesses based on product information and therefore try to automate the process of information extraction (IE). Since the information source is usually heterogeneous and non-standardized, classic extract, transform, load techniques reach their limits. Hence, companies m...

Full description

Saved in:
Bibliographic Details
Published in:SN computer science 2023-11, Vol.4 (6), p.833, Article 833
Main Authors: Sildatke, Michael, Karwanni, Hendrik, Kraft, Bodo, Zündorf, Albert
Format: Article
Language:English
Subjects:
Advances on Enterprise Information Systems
Architecture
Automation
Choreography
Computer Imaging
Computer Science
Computer Systems Organization and Communication Networks
Customer services
Data Structures and Information Theory
Documents
Energy industry
Information retrieval
Information Systems and Communication Service
Opportunity costs
Original Research
Pattern Recognition and Graphics
Pipelines
Prices
Product information
Prototyping
Semantics
Software
Software Engineering/Programming and Operating Systems
Suppliers
Vision
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Companies often build their businesses based on product information and therefore try to automate the process of information extraction (IE). Since the information source is usually heterogeneous and non-standardized, classic extract, transform, load techniques reach their limits. Hence, companies must implement the newest findings from research to tackle the challenges of process automation. They require a flexible and robust system that is extendable and ensures the optimal processing of the different document types. This paper provides a distributed microservice architecture pattern that enables the automated generation of IE pipelines. Since their optimal design is individual for each input document, the system ensures the ad-hoc generation of pipelines depending on specific document characteristics at runtime. Furthermore, it introduces the automated quality determination of each available pipeline and controls the integration of new microservices based on their impact on the business value. The introduced system enables fast prototyping of the newest approaches from research and supports companies in automating their IE processes. Based on the automated quality determination, it ensures that the generated pipelines always meet defined business requirements when they come into productive use.
ISSN:2661-8907
2662-995X
2661-8907
DOI:10.1007/s42979-023-02256-4