[Back]

M. Borkowski, C. Hochreiner, S. Schulte:
"Moderated Resource Elasticity for Stream Processing Applications";
Talk: International Workshop on Autonomic Solutions for Parallel and Distributed Data Stream Processing (Auto-DaSP 2017) colocated with Euro-Par 2017, Santiago de Compostela, Spain; 2017-08-28 - 2017-09-01; in: "Euro-Par 2017: Parallel Processing Workshops - Euro-Par 2017 International Workshops, Revised Selected Papers", D. Heras, L. Bougé et al. (ed.); Springer, Cham, LNCS 10659 (2018), ISBN: 978-3-319-75177-1; 5 - 16.

In stream processing, elasticity is often realized by adapting the system scale and topology according to the volume of input data. However, this volume is often fluctuating, with a high degree of noise, which can trigger a high amount of scaling operations. Since these scaling operations introduce additional overhead and cost, systems employing such approaches are at risk of spending a significant amount of time scaling up and down, nullifying the positive effects of scalability.

To overcome this, we propose an approach for moderating the scaling behavior of stream processing applications by reducing the number of scaling operations, while still providing quick responses to changes in input data volume. Contrary to existing approaches, instead of using linear smoothing techniques, we show how to employ non-linear filtering techniques from the field of signal processing to pre-process the raw volume measurements, mitigating superfluous scaling operations, and effectively reducing the number of such operations by up to 94%.

Stream processing · Elasticity · TVD · EKF

http://dx.doi.org/10.1007/978-3-319-75178-8_1

Project Head Stefan Schulte:
Cloud-based Rapid Elastic MAnufacturing