[Back]

P. Rosenberger, D. Gerhard:
"Evaluating context-aware systems: State-of-the-art analysis and introduction of a customized framework";
Talk: 48th International Conference on Computers and Industrial Engineering, CIE 2018, Auckland, New Zealand; 2018-12-02 - 2018-12-05; in: "Proceedings of International Conference on Computers and Industrial Engineering, CIE", Proceedings of International Conference on Computers and Industrial Engineering, CIE, Volume 2018 (2018), ISBN: 9781510877399; 1228 - 1241.

In recent years, context-aware systems gained increasing popularity as these systems can sense their environment, recognize specific contexts, and provide situational assistance. A disadvantage of such systems is that their development requires substantial additional efforts due to the extended functionalities and the absence of tested development approaches. To resolve this issue, an adjusted process model is required that addresses the particularities of context-aware systems. As part of a broader effort to create such a model, this publication focuses on the evaluation of context-aware systems. The paper starts with an analysis of the existing evaluation approaches. Afterwards, a comprehensive framework for the user-centred evaluation is introduced that puts special focus on the industrial domain. As the proposed approach is independent of the current system development stage, the evaluation scope, the evaluation objectives, the targeted metrics, and the used measures, it can be applied in a wide range of scenarios. Finally, the application of the framework is demonstrated in an industrial use case. A context-aware information system is evaluated in different development stages that assists the workers at an injection moulding company.

Context-Aware Systems, Context-Aware Evaluation, Evaluation Framework, Evaluation Process Model