Talks and Poster Presentations (with Proceedings-Entry):
M. Reisinger, P. Frangoudis, S. Dustdar:
"System support and mechanisms for adaptive edge-to-cloud DNN model serving";
Poster: 9th IEEE International Conference on Cloud Engineering (IC2E 2021) - Online Conference,
San Francisco, CA, USA;
2021-10-04
- 2021-10-08; in: "Proceedings of the IEEE International Conference on Cloud Engineering (IC2E 2021)",
IEEE,
(2021),
ISBN: 978-1-6654-4971-7;
278
- 279.
English abstract:
We present an orchestration scheme for Deep Neural Network (DNN) model serving, capable of computation distribution over the device-to-cloud continuum and low-latency inference. Our system allows automated layer-wise splitting of DNN structures and their adaptive distribution over compute hosts, providing an execution environment for collaborative inference. Model deployment and its self-adaptation at runtime are implemented by optimization algorithms supported in a plug-in manner. These follow service and infrastructure provider criteria and constraints, expressed via well-defined interfaces. Our framework can serve diverse neural architectures, including DNNs with early exits, with zero to minimal modifications.
"Official" electronic version of the publication (accessed through its Digital Object Identifier - DOI)
http://dx.doi.org/10.1109/IC2E52221.2021.00046
Created from the Publication Database of the Vienna University of Technology.