A Cooperative, Self-Configuring High-Availability Solution for Stream Processing

We present a collaborative, self-configuring high availability (HA) approach for stream processing that enables low-latency failure recovery while incurring small run-time overhead. Our approach relies on a novel fine-grained checkpointing model that allows query fragments at each server to be backe...

Full description

Saved in:
Bibliographic Details
Main Authors: Jeong-Hyon Hwang, Ying Xing, Cetintemel, U., Zdonik, S.
Format: Conference Proceeding
Language:English
Subjects:
Availability
Checkpointing
Collaboration
Computer science
Costs
Delay
Patient monitoring
Runtime
Safety
Virtual prototyping
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a collaborative, self-configuring high availability (HA) approach for stream processing that enables low-latency failure recovery while incurring small run-time overhead. Our approach relies on a novel fine-grained checkpointing model that allows query fragments at each server to be backed up at multiple other servers and recovered collectively (in parallel) when there is a failure. In this paper, we first address the problem of determining the appropriate query fragments at each server. We then discuss, for each fragment, which server to use as its backup as well as the proper checkpoint schedule. We also introduce and analyze operator-specific delta-checkpointing techniques to reduce the overall HA cost. Finally, we quantify the benefits of our approach using results from our prototype implementation and a detailed simulator.
ISSN:1063-6382
2375-026X
DOI:10.1109/ICDE.2007.367863