umu.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Dynamic Cloud Resource Management: Scheduling, Migration and Server Disaggregation
Umeå universitet, Teknisk-naturvetenskapliga fakulteten, Institutionen för datavetenskap. (Distribuerade system)
2014 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

A key aspect of cloud computing is the promise of infinite, scalable resources, and that cloud services should scale up and down on demand. This thesis investigates methods for dynamic resource allocation and management of services in cloud datacenters, introducing new approaches as well as improvements to established technologies.Virtualization is a key technology for cloud computing as it allows several operating system instances to run on the same Physical Machine, PM, and cloud services normally consists of a number of Virtual Machines, VMs, that are hosted on PMs. In this thesis, a novel virtualization approach is presented. Instead of running each PM isolated, resources from multiple PMs in the datacenter are disaggregated and exposed to the VMs as pools of CPU, I/O and memory resources. VMs are provisioned by using the right amount of resources from each pool, thereby enabling both larger VMs than any single PM can host as well as VMs with tailor-made specifications for their application. Another important aspect of virtualization is live migration of VMs, which is the concept moving VMs between PMs without interruption in service. Live migration allows for better PM utilization and is also useful for administrative purposes. In the thesis, two improvements to the standard live migration algorithm are presented, delta compression and page transfer reordering. The improvements can reduce migration downtime, i.e., the time that the VM is unavailable, as well as the total migration time. Postcopy migration, where the VM is resumed on the destination before the memory content is transferred is also studied. Both userspace and in-kernel postcopy algorithms are evaluated in an in-depth study of live migration principles and performance.Efficient mapping of VMs onto PMs is a key problem for cloud providers as PM utilization directly impacts revenue. When services are accepted into a datacenter, a decision is made on which PM should host the service VMs. This thesis presents a general approach for service scheduling that allows for the same scheduling software to be used across multiple cloud architectures. A number of scheduling algorithms to optimize objectives like revenue or utilization are also studied. Finally, an approach for continuous datacenter consolidation is presented. As VM workloads fluctuate and server availability varies any initial mapping is bound to become suboptimal over time. The continuous datacenter consolidation approach adjusts this VM-to-PM mapping during operation based on combinations of management actions, like suspending/resuming PMs, live migrating VMs, and suspending/resuming VMs. Proof-of-concept software and a set of algorithms that allows cloud providers to continuously optimize their server resources are presented in the thesis.

Ort, förlag, år, upplaga, sidor
Umeå: Umeå universitet , 2014. , s. 26
Serie
Report / UMINF, ISSN 0348-0542 ; 2014:09
Nyckelord [en]
Cloud computing, virtualization, distributed infrastructure, live migration, scheduling
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
URN: urn:nbn:se:umu:diva-87904ISBN: 978-91-7601-038-9 (tryckt)OAI: oai:DiVA.org:umu-87904DiVA, id: diva2:712168
Disputation
2014-05-06, Naturvetarhuset, N320, Umeå universitet, Umeå, 10:15 (Engelska)
Opponent
Handledare
Tillgänglig från: 2014-04-15 Skapad: 2014-04-14 Senast uppdaterad: 2018-06-08Bibliografiskt granskad
Delarbeten
1. Evaluation of delta compression techniques for efficient live migration of large virtual machines
Öppna denna publikation i ny flik eller fönster >>Evaluation of delta compression techniques for efficient live migration of large virtual machines
2011 (Engelska)Ingår i: The 2011 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2011), New York, NY: ACM Press, 2011, s. 111-120Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Despite the widespread support for live migration of Virtual Machines (VMs) in current hypervisors, these have significant shortcomings when it comes to migration of certain types of VMs. More specifically, with existing algorithms, there is a high risk of service interruption when migrating VMs with high workloads and/or over low-bandwidth networks. In these cases, VM memory pages are dirtied faster than they can be transferred over the network, which leads to extended migration downtime. In this contribution, we study the application of delta compression during the transfer of memory pages in order to increase migration throughput and thus reduce downtime. The delta compression live migration algorithm is implemented as a modification to the KVM hypervisor. Its performance is evaluated by migrating VMs running different type of workloads and the evaluation demonstrates a significant decrease in migration downtime in all test cases. In a benchmark scenario the downtime is reduced by a factor of 100. In another scenario a streaming video server is live migrated with no perceivable downtime to the clients while the picture is frozen for eight seconds using standard approaches. Finally, in an enterprise application scenario, the delta compression algorithm successfully live migrates a very large system that fails after migration using the standard algorithm. Finally, we discuss some general effects of delta compression on live migration and analyze when it is beneficial to use this technique.

Ort, förlag, år, upplaga, sidor
New York, NY: ACM Press, 2011
Serie
ACM SIGPLAN Notices, ISSN 1523-2867 ; Vol. 46 Nr. 7
Nyckelord
Virtualization, Live migration, Compression, Performance evaluation
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
urn:nbn:se:umu:diva-40370 (URN)10.1145/2007477.1952698 (DOI)000294949300011 ()978-1-4503-0687-4 (ISBN)
Konferens
7th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
Tillgänglig från: 2011-02-22 Skapad: 2011-02-22 Senast uppdaterad: 2018-06-08Bibliografiskt granskad
2. High Performance Live Migration through Dynamic Page Transfer Reordering and Compression
Öppna denna publikation i ny flik eller fönster >>High Performance Live Migration through Dynamic Page Transfer Reordering and Compression
2011 (Engelska)Ingår i: 2011 IEEE Third International Conference on Cloud Computing Technology and Science, IEEE Computer Society, 2011, s. 542-548Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Although supported by many contemporary Virtual Machine (VM) hyper visors, live migration is impossible for certain applications. When migrating CPU and/or memory intensive VMs two problems occur, extended migration downtime that may cause service interruption or even failure, and prolonged total migration time that is harmful for the overall system performance as significant network resources must be allocated to migration. These problems become more severe for migration over slower networks, such as long distance migration between clouds. We approach this two-fold problem through a combination of techniques. A novel algorithm that dynamically adapts the transfer order of VM memory pages during live migration reduces the risk of re-transfers for frequently dirtied pages. As the amount of transferred data is thereby reduced, the total migration time is shortened. By combining this technique with a compression scheme that increases the migration bandwidth the migration downtime is also reduced. An evaluation by means of synthetic migration benchmarks shows that our combined approach reduces migration downtime by a factor 10 to 20, shortens total migration time by around 35%, as well as consumes between 26% and 39% less network bandwidth. The feasibility of our approach for real-life applications is demonstrated by migrating a streaming video server 31% faster while transferring 51% less data.

Ort, förlag, år, upplaga, sidor
IEEE Computer Society, 2011
Nyckelord
Virtual machine monitors, Performance evaluation, Platform virtualization
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
datalogi
Identifikatorer
urn:nbn:se:umu:diva-50835 (URN)10.1109/CloudCom.2011.82 (DOI)978-1-4673-0090-2 (ISBN)
Konferens
The 3rd IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2011) November 29 - December 1, Athens, Greece
Tillgänglig från: 2011-12-27 Skapad: 2011-12-27 Senast uppdaterad: 2018-06-08Bibliografiskt granskad
3. The Noble art of Live Migration
Öppna denna publikation i ny flik eller fönster >>The Noble art of Live Migration
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Abstract [en]

Although a mature technique and an important feature of virtualization, live VM migration still suffers from performance and robustness issues, limiting its usefulness. This is particularly true if the migration technique is not appropriately selected for the usage scenario. In this contribution, we define requirements for live migration and discuss some of the challenges that arise in meeting these. We investigate, categorize, and compare current approaches to live migration as well as provide guidelines for which to use in different scenarios. Our investigation is validated by a set of experiments that highlights different characteristics of these approaches. We also propose methods for hybrid live migration and an improved memory page reordering algorithm. Finally we outline the future research landscape in the area. 

Nyckelord
Virtualization, Live Migration, Survey
Nationell ämneskategori
Datorsystem
Forskningsämne
administrativ databehandling
Identifikatorer
urn:nbn:se:umu:diva-87241 (URN)
Anmärkning

Technical Report, UMINF-12.11

Tillgänglig från: 2014-03-25 Skapad: 2014-03-25 Senast uppdaterad: 2018-06-08Bibliografiskt granskad
4. Hecatonchire: Enabling Multi-Host Virtual Machines by Resource Aggregation and Pooling
Öppna denna publikation i ny flik eller fönster >>Hecatonchire: Enabling Multi-Host Virtual Machines by Resource Aggregation and Pooling
2014 (Engelska)Rapport (Övrigt vetenskapligt)
Abstract [en]

Vertical elasticity, or scale-up of individual virtual ma- chines is hard to perform in today’s cloud environments due to limitations in the amount of hardware resources available in single servers. We propose a novel approach that allows aggregation of memory, compute and I/O resources from multiple physical machines in resource pools which in turn are used to seamlessly provision vir- tual machines with the right amount of resources. We present our architecture and highlight key functionality such as transparent and resilient memory aggregation and fast live migration. Our approach is validated by a demonstration using benchmarks and a real-world big- data application. Performance results indicate a very low overhead in using aggregated memory as well as a sig- nificant improvement in live migration performance.

Ort, förlag, år, upplaga, sidor
Umeå: Department of Computing Science, Umeå University, 2014. s. 6
Serie
Report / UMINF, ISSN 0348-0542 ; 14.11
Nyckelord
Virtualization, Operating Systems, Disaggregation
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
urn:nbn:se:umu:diva-87902 (URN)
Tillgänglig från: 2014-04-14 Skapad: 2014-04-14 Senast uppdaterad: 2018-06-08Bibliografiskt granskad
5. A General Approach to Service Deployment in Cloud Environments
Öppna denna publikation i ny flik eller fönster >>A General Approach to Service Deployment in Cloud Environments
2012 (Engelska)Ingår i: Cloud and Green Computing (CGC 2012): 2012 Second International Conference on, IEEE Computer Society, 2012, s. 17-24Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

The cloud computing landscape has recently developed into a spectrum of cloud architectures, leading to a broad range of management tools for similar operations but specialized for certain deployment scenarios. This both hinders the efficient reuse of algorithmic innovations within cloud management operations and increases the heterogeneity between different management systems. Our overarching goal is to overcome these problems by developing tools general enough to support the full range of popular architectures. In this contribution, we analyze commonalities in recently proposed cloud models (private clouds, multi-clouds, bursted clouds, federated clouds, etc.), and demonstrate how a key management functionality - service deployment - can be uniformly performed in all of these by a carefully designed system. The design of our service deployment framework is validated through a demonstration of how it can be used to deploy services, perform bursting and brokering, as well as mediate a cloud federation in the context of the OPTIMIS Toolkit.

Ort, förlag, år, upplaga, sidor
IEEE Computer Society, 2012
Nyckelord
Cloud Computing, Cloud Architecture, Service Deployment
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
urn:nbn:se:umu:diva-79784 (URN)10.1109/CGC.2012.90 (DOI)978-0-7695-4864-7 (ISBN)978-1-4673-3027-5 Print (ISBN)
Konferens
the 2nd International Conference on Cloud and Green Computing, Xiangtan, 1-3 November 2012
Tillgänglig från: 2013-09-02 Skapad: 2013-09-02 Senast uppdaterad: 2018-06-08Bibliografiskt granskad
6. Cost-Optimal Cloud Service Placement under Dynamic Pricing Schemes
Öppna denna publikation i ny flik eller fönster >>Cost-Optimal Cloud Service Placement under Dynamic Pricing Schemes
2013 (Engelska)Ingår i: 6th IEEE/ACM International Conference on Utility and Cloud Computing, IEEE Computer Society, 2013, s. 187-194Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Until now, most research on cloud service placement has focused on static pricing scenarios, where cloud providers offer fixed prices for their resources. However, with the recent trend of dynamic pricing of cloud resources, where the price of a compute resource can vary depending on the free capacity and load of the provider, new placement algorithms are needed. In this paper, we investigate service placement in dynamic pricing scenarios by evaluating a set of placement algorithms, tuned for dynamic pricing. The algorithms range from simple heuristics to combinatorial optimization solutions. The studied algorithms are evaluated by deploying a set of services across multiple providers. Finally, we analyse the strengths and weaknesses of the algorithms considered. The evaluation suggests that exhaustive search based approach is good at finding optimal solutions for service placement under dynamic pricing schemes, but the execution times are usually long. In contrast, greedy approaches perform surprisingly well with fast execution times and acceptable solutions, and thus can be a suitable compromise considering the tradeoffs between quality of solution and execution time.

Ort, förlag, år, upplaga, sidor
IEEE Computer Society, 2013
Nyckelord
Cloud Computing, Dynamic Pricing, Service Placement, Deployment Optimization
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
urn:nbn:se:umu:diva-80478 (URN)
Konferens
the 6th IEEE/ACM International Conference on Utility and Cloud Computing
Forskningsfinansiär
eSSENCE - An eScience CollaborationEU, FP7, Sjunde ramprogrammet, 257115
Tillgänglig från: 2013-09-18 Skapad: 2013-09-18 Senast uppdaterad: 2018-06-08Bibliografiskt granskad
7. Continuous Datacenter Consolidation
Öppna denna publikation i ny flik eller fönster >>Continuous Datacenter Consolidation
Visa övriga...
2014 (Engelska)Rapport (Refereegranskat)
Abstract [en]

Efficient mapping of Virtual Machines (VMs) onto physical servers is a key problem for cloud infrastructure providers as hardware utilization directly im- pacts revenue. Today, this mapping is commonly only performed when new VMs are created, but as VM workloads fluctuate and server availability varies, any ini- tial mapping is bound to become suboptimal over time. We introduce a set of heuristic methods for continuous optimization of the VM-to-server mapping based on combina- tions of fundamental management actions, namely suspending and resuming physical machines, migrating VMs, and suspending and resuming VMs. Using these methods cloud infrastructure providers can continuously optimize their server resources regard- less of the predictability of the workload. To verify that our approach is applicable in real-world scenarios, we build a proof-of-concept datacenter management system that implements the proposed algorithms. The feasibility of our approach is evaluated through a combination of simulations and real experiments where our system provi- sions a workload of benchmark applications. Our results indicate that the proposed algorithms are feasible, that the combined management approach achieves the best results, and that the VM suspend and resume mechanism has the largest impact. 

Ort, förlag, år, upplaga, sidor
Umeå: Umeå universitet, 2014. s. 12
Serie
Report / UMINF, ISSN 0348-0542 ; 2014:08
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
urn:nbn:se:umu:diva-87385 (URN)
Tillgänglig från: 2014-03-31 Skapad: 2014-03-31 Senast uppdaterad: 2018-06-08Bibliografiskt granskad

Open Access i DiVA

Dynamic Cloud Resource Management(2465 kB)1778 nedladdningar
Filinformation
Filnamn FULLTEXT02.pdfFilstorlek 2465 kBChecksumma SHA-512
b743bb74500da2d2bd39e123ad2d3b7a56059ced5ac3f4674e3cef666a8152f6cb217c318b8ee014556dde002384c03eaf33704fdfd83ad0ee7393fe3e42838e
Typ fulltextMimetyp application/pdf

Personposter BETA

Svärd, Petter

Sök vidare i DiVA

Av författaren/redaktören
Svärd, Petter
Av organisationen
Institutionen för datavetenskap
Datavetenskap (datalogi)

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 1778 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 1457 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf