[Back]

O. Hanappi:
"Testing Idempotence and Convergence of Automatic Configuration Scripts";
Supervisor: S. Dustdar, W. Hummer; Institut für Informationssysteme, Distributed Systems Group, 2016; final examination: 2016-03-02.

In recent years, with the rise of cloud computing and large dynamically scaling distributed applications, the demand for automatically managing IT infrastructures grew steadily and led to the adoption of so-called configuration management tools. Such tools operate on declarative descriptions of the desired state of the system. They are repeatably applied and try to configure the system accordingly. The actions performed may fail temporarily but will eventually succeed. The system is called to converge to the desired state.
A crucial property to achieve convergence is the idempotence of the executed actions, hence that repeated executions do not fail or change the system state once the action succeeded. Developing such idempotent actions is a very challenging task and requires thorough testing.
In addition to idempotence issues, different actions may influence or are in conflict with each other. In particular, a sequence of idempotent actions is not necessarily idempotent as well. Furthermore, the order in which actions are executed may not be
totally defined, but may vary from one execution to another. This stems from the idea that the configuration process is convergent and the system will eventually reach the desired state, regardless of the execution order of the underlying actions. Conflicts may only occur in some orders and could thus stay undetected.
It follows that creating idempotent and convergent automated configurations is a difficult task which requires rigorous and systematic testing. Therefore, we apply a model based test approach, propose a conceptual framework for automated idempotence and
convergence testing and present a prototypical implementation thereof based on Puppet.
Actions in Puppet scripts are only partially ordered, such that there are usually many different execution orders and testing all of them is not viable in general. Hence, we define a property called preservation, which can be tested pairwise between resources and which, assuming that it holds for all pairs of resources, implies the convergence of the corresponding configuration.
We perform an evaluation of our model based on real world Puppet scripts and show the effectiveness of the presented approach. The prototype is able to detect all idempotence and convergence related issues in a set of real world Puppet scripts with known issues as well as some hitherto undiscovered issues in a large set of arbitrarily chosen Puppet scripts.