Umeå universitets logga

umu.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Edge orchestration for latency-sensitive applications
Umeå universitet, Teknisk-naturvetenskapliga fakulteten, Institutionen för datavetenskap.ORCID-id: 0000-0001-9249-1633
2024 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)Alternativ titel
Orkestrering av distribuerade resurser för latenskänsliga applikationer (Svenska)
Abstract [en]

The emerging edge computing infrastructure provides distributed and heterogeneous resources closer to where data is generated and where end-users are located, thereby significantly reducing latency. With the recent advances in telecommunication systems, software architecture, and machine learning, there is a noticeable increase in applications that require processing times within tight latency constraints, i.e. latency-sensitive applications. For instance, numerous video analytics applications, such as traffic control systems, necessitate real-time processing capabilities. Orchestrating such applications at the edge offers numerous advantages, including lower latency, optimized bandwidth utilization, and enhanced scalability. However, despite its potential, effectively managing such latency-sensitive applications at the edge poses several challenges such as constrained compute resources, which holds back the full promise of edge computing.

This thesis proposes approaches to efficiently deploy latency-sensitive applications on the edge infrastructure. It partly addresses general applications with microservice architectures and party addresses the increasingly more important video analytics applications for the edge. To do so, this thesis proposes various application- and system-level solutions aiming to efficiently utilize constrained compute capacity on the edge while meeting prescribed latency constraints. These solutions primarily focus on effective resource management approaches and optimizing incoming workload inputs, considering the constrained compute capacity of edge resources. Additionally, the thesis explores the synergy effects of employing both application- and system-level resource optimization approaches together.

The results demonstrate  the effectiveness of the proposed solutions in enhancing the utilization of edge resources for latency-sensitive applications while adhering to application constraints. The proposed resource management solutions, alongside application-level optimization techniques, significantly improve resource efficiency while satisfying application requirements. Our results show that our solutions for microservice architectures significantly improve end-to-end latency by up to 800% while minimizing edge resource usage. Additionally, the results indicate that our application- and system-level optimizations for orchestrating edge resources for video analytics applications can increase the overall throughput by up to 60%. 

Ort, förlag, år, upplaga, sidor
Umeå: Umeå University, 2024. , s. 46
Serie
UMINF, ISSN 0348-0542 ; 24.05
Nyckelord [en]
Edge Computing, Resource Management, Latency-Sensitive Applications, Edge Video Analytics
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
datalogi; datorteknik
Identifikatorer
URN: urn:nbn:se:umu:diva-223021ISBN: 978-91-8070-350-5 (tryckt)ISBN: 978-91-8070-351-2 (digital)OAI: oai:DiVA.org:umu-223021DiVA, id: diva2:1849510
Disputation
2024-04-29, Hörsal UB.A.240 - Lindellhallen 4, 13:00 (Engelska)
Opponent
Handledare
Anmärkning

Incorrect date of publication on the posting sheet.

In publication: UMINF 24.04

Tillgänglig från: 2024-04-08 Skapad: 2024-04-08 Senast uppdaterad: 2024-04-23Bibliografiskt granskad
Delarbeten
1. Microsplit: efficient splitting of microservices on edge clouds
Öppna denna publikation i ny flik eller fönster >>Microsplit: efficient splitting of microservices on edge clouds
2022 (Engelska)Ingår i: 2022 IEEE/ACM 7th Symposium on Edge Computing (SEC), IEEE, 2022, s. 252-264Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Edge cloud systems reduce the latency between users and applications by offloading computations to a set of small-scale computing resources deployed at the edge of the network. However, since edge resources are constrained, they can become saturated and bottlenecked due to increased load, resulting in an exponential increase in response times or failures. In this paper, we argue that an application can be split between the edge and the cloud, allowing for better performance compared to full migration to the cloud, releasing precious resources at the edge. We model an application's internal call-Graph as a Directed-Acyclic-Graph. We use this model to develop MicroSplit, a tool for efficient splitting of microservices between constrained edge resources and large-scale distant backend clouds. MicroSplit analyzes the dependencies between the microservices of an application, and using the Louvain method for community detection---a popular algorithm from Network Science---decides how to split the microservices between the constrained edge and distant data centers. We test MicroSplit with four microservice based applications in various realistic cloud-edge settings. Our results show that Microsplit migrates up to 60% of the microservices of an application with a slight increase in the mean-response time compared to running on the edge, and a latency reduction of up to 800% compared to migrating the entire application to the cloud. Compared to other methods from the State-of-the-Art, MicroSplit reduces the total number of services on the edge by up to five times, with minimal reduction in response times.

Ort, förlag, år, upplaga, sidor
IEEE, 2022
Nyckelord
Edge clouds, Microservices, Service mesh, Louvain community detection
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
datorteknik
Identifikatorer
urn:nbn:se:umu:diva-202481 (URN)10.1109/SEC54971.2022.00027 (DOI)000918607200019 ()2-s2.0-85146644109 (Scopus ID)978-1-6654-8611-8 (ISBN)978-1-6654-8612-5 (ISBN)
Konferens
IEEE/ACM 7th Symposium on Edge Computing (SEC), Seattle, WA, USA, December 5-8, 2022
Tillgänglig från: 2023-01-16 Skapad: 2023-01-16 Senast uppdaterad: 2024-04-08Bibliografiskt granskad
2. RAVAS: interference-aware model selection and resource allocation for live edge video analytics
Öppna denna publikation i ny flik eller fönster >>RAVAS: interference-aware model selection and resource allocation for live edge video analytics
Visa övriga...
2023 (Engelska)Ingår i: 2023 IEEE/ACM Symposium on Edge Computing (SEC): Proceedings, Institute of Electrical and Electronics Engineers (IEEE), 2023, s. 27-39Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Numerous edge applications that rely on video analytics demand precise, low-latency processing of multiple video streams from cameras. When these cameras are mobile, such as when mounted on a car or a robot, the processing load on the shared edge GPU can vary considerably. Provisioning the edge with GPUs for the worst-case load can be expensive and, for many applications, not feasible. In this paper, we introduce RAVAS, a Real-time Adaptive stream Video Analytics System that enables efficient edge GPU sharing for processing streams from various mobile cameras. RAVAS uses Q-Learning to choose between a set of Deep Neural Network (DNN) models with varying accuracy and processing requirements based on the current GPU utilization and workload. RAVAS employs an innovative resource allocation strategy to mitigate interference during concurrent GPU execution. Compared to state-of-the-art approaches, our results show that RAVAS incurs 57% less compute overhead, achieves 41% improvement in latency, and 43% savings in total GPU usage for a single video stream. Processing multiple concurrent video streams results in up to 99% and 40% reductions in latency and overall GPU usage, respectively, while meeting the accuracy constraints.

Ort, förlag, år, upplaga, sidor
Institute of Electrical and Electronics Engineers (IEEE), 2023
Nyckelord
Edge Video Analytics, Model Selection, Resource Allocation, Interference-aware GPU Multiplexing
Nationell ämneskategori
Datorsystem
Forskningsämne
datorteknik
Identifikatorer
urn:nbn:se:umu:diva-220744 (URN)10.1145/3583740.3628443 (DOI)001164050000003 ()2-s2.0-85186111633 (Scopus ID)979-8-4007-0123-8 (ISBN)
Konferens
2023 IEEE/ACM Symposium on Edge Computing (SEC), Wilmington, USA, December 6-9, 2023
Tillgänglig från: 2024-02-11 Skapad: 2024-02-11 Senast uppdaterad: 2024-04-08Bibliografiskt granskad
3. CVF: Cross-Video Filtration on the edge
Öppna denna publikation i ny flik eller fönster >>CVF: Cross-Video Filtration on the edge
2024 (Engelska)Ingår i: MMSys '24: Proceedings of the 15th ACM Multimedia Systems Conference, Association for Computing Machinery (ACM), 2024, s. 231-242Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Many edge applications rely on expensive Deep-Neural-Network (DNN) inference-based video analytics. Typically, a single instance of an inference service analyzes multiple real-time camera streams concurrently. In many cases,  only a fraction of these streams contain objects-of-interest at a given time. Hence, it is a waste of computational resources to process all frames from all cameras using the DNNs. On-camera filtration of frames has been suggested as a possible solution to improve the system efficiency and reduce resource wastage. However, many cameras do not have on-camera processing or filtering capabilities. In addition, filtration can be enhanced if frames across the different feeds are selected and prioritized for processing based on the system load and the available resource capacity. This paper introduces CVF, a Cross-video Filtration framework designed around video content and resource constraints. The CVF pipeline leverages compressed-domain data from encoded video formats, lightweight binary classification models, and an efficient prioritization algorithm. This enables the effective filtering of cross-camera frames from multiple sources, processing only a fraction of frames using resource-intensive DNN models. Our experiments show that CVF is capable of reducing the overall response time of video analytics pipelines by up to 50% compared to state-of-the-art solutions while increasing the throughput by up to 120%.

Ort, förlag, år, upplaga, sidor
Association for Computing Machinery (ACM), 2024
Nyckelord
Edge, Video Analytics, Video Filtration, Codecs
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
datalogi; datorteknik
Identifikatorer
urn:nbn:se:umu:diva-223020 (URN)10.1145/3625468.3647627 (DOI)2-s2.0-85191950336 (Scopus ID)979-8-4007-0412-3 (ISBN)
Konferens
ACM Multimedia Systems Conference 2024 (MMSys ’24), Bari, Italy, April 15-18, 2024
Tillgänglig från: 2024-04-08 Skapad: 2024-04-08 Senast uppdaterad: 2024-05-16Bibliografiskt granskad
4. RAVEN: adaptive edge resource allocation with spatio-temporal multiplexing for live video pipelines
Öppna denna publikation i ny flik eller fönster >>RAVEN: adaptive edge resource allocation with spatio-temporal multiplexing for live video pipelines
Visa övriga...
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
datorteknik; datalogi
Identifikatorer
urn:nbn:se:umu:diva-223016 (URN)
Tillgänglig från: 2024-04-08 Skapad: 2024-04-08 Senast uppdaterad: 2024-04-08
5. Hedge: a real-time video analytics system for heterogeneous distributed edge with compressed feeds
Öppna denna publikation i ny flik eller fönster >>Hedge: a real-time video analytics system for heterogeneous distributed edge with compressed feeds
Visa övriga...
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
datalogi; datorteknik
Identifikatorer
urn:nbn:se:umu:diva-223017 (URN)
Tillgänglig från: 2024-04-08 Skapad: 2024-04-08 Senast uppdaterad: 2024-04-08

Open Access i DiVA

fulltext(2898 kB)152 nedladdningar
Filinformation
Filnamn FULLTEXT02.pdfFilstorlek 2898 kBChecksumma SHA-512
dd5da97de47432cb601ade9d98325072bec026ded60dcf5ac8d65fc749a62409a154bc40faeb3fea883456bf352264e4637156983f0d34f4f8bda30f6e898663
Typ fulltextMimetyp application/pdf
spikblad(155 kB)43 nedladdningar
Filinformation
Filnamn SPIKBLAD01.pdfFilstorlek 155 kBChecksumma SHA-512
f3c922e5678598115b443043b8b81b2585cc22378f128860dd85ffbb81a268b2abbe96940685f62365e4f0cdb253399f963f093dc348292c19fce1f636f35ffe
Typ spikbladMimetyp application/pdf

Person

Rahmanian, Ali

Sök vidare i DiVA

Av författaren/redaktören
Rahmanian, Ali
Av organisationen
Institutionen för datavetenskap
Datavetenskap (datalogi)

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 152 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 903 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf