[Back]

T. Hiessl:
"Optimizing the Placement of Stream Processing Operators in the Fog";
Supervisor: S. Schulte, C. Hochreiner; Institute of Information Systems, Distributed Systems Group, 2017; final examination: 2017-06-12.

The increasing presence of connected devices that interact with their environments, is a driving force for the Internet of Things (IoT). In the IoT, various devices are equipped with sensors to support private as well as business life by collecting and providing insightful information. Especially in the manufacturing domain, IoT devices allow for a better organization and reduce the need for human control.
To process the collected IoT data, batch jobs are often executed on large databases that are hosted in the Cloud. Nevertheless, this is not feasible when results are required in real time. Therefore, Data Stream Processing (DSP) considers to process data based on a topology of DSP operators. These DSP operators continuously query data to perform e.g., customized analysis operations. In contrast to batch job processing, DSP operators consider to process a small amount of data only once or a limited number of times in order to deliver results in real time.
IoT data is often streaming from multiple sources to Cloud Resources that enact DSP topologies. This can lead to high latencies when data is uploaded. For that, Fog Computing is used to provide a highly virtualized computing platform between the Cloud and the edge of the network. The Fog consists of a high number of heterogeneous
resources to provide scalability and on-demand accessibility. For this, device types such as routers, switches, or even end-user hardware can be used to host e.g., DSP operators. Due to the given heterogeneity of resources, the optimization of DSP operator placements is desirable to achieve a high availability, low latency, and ideally low cost.
The goal of this thesis is to show how the Fog paradigm facilitates DSP systems to improve the Quality of Service. For this, we introduce the ODR Reasoner, a loosely coupled service that is used for the optimization of DSP operator placements in Fog and Cloud environments. The ODR Reasoner continuously monitors the Fog network and DSP topology enactment to incorporate this information into a periodically solved Integer Linear Program optimization model. To investigate which criteria have to be considered in the optimization, we review state-of-the-art approaches and study DSP and Fog Computing characteristics. To evaluate the developed dynamic approach, we test operator placement optimization in a Fog-like test bed.
Our results show that response time and cost is reduced by 31.5% and 8.8% respectively. We compared our ODR approach to a baseline approach that optimizes only when the DSP topology is initially deployed. Furthermore, we learned that achieving a global optimum in operator placement problems is a difficult task for growing Fog networks.