umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Decentralized resource brokering for heterogeneous grid environments
Umeå University, Faculty of Science and Technology, Department of Computing Science.
2006 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

The emergence of Grid computing infrastructures enables researchers to share resources and collaborate in more efficient ways than before, despite belonging to different organizations and being distanced geographically. While the Grid computing paradigm offers new opportunities, it also gives rise to new difficulties. One such problem is the selection of resources for user applications. Given the large and disparate set of Grid resources, manual resource selection becomes impractical, even for experienced users. This thesis investigates methods, algorithms and software for a Grid resource broker, i.e., a scheduling agent that automates the resource selection process for the user. The development of such a component is a non-trivial task as Grid resources are heterogeneous in hardware, software, availability, ownership and usage policies. A wide range of algorithmically difficult issues must also be solved, including characterization of jobs, prediction of resource performance, data placement considerations, and, how to provide Quality of Service guarantees. One contribution of this thesis is the development of resource brokering algorithms that enable resource selection based on Grid job performance predictions and use advance reservations to provide Quality of Service guarantees. The thesis also includes an algorithm for coallocation of sets of jobs. This algorithm guarantees a simultaneous start of each subjob, as required e.g., when running larger-than-supercomputer simulations that involve multiple resources.

We today have the somewhat paradoxal situation where Grids, originally aimed to overcome interoperability problems between different computing platforms, themselves struggle with interoperability problems caused by the wide range of interfaces, protocols and data formats that are used in different environments. The reasons for this situation are obvious, expected and almost impossible to avoid, as the task of defining appropriate standards, models and best-practices must be preceded by basic research, proof-of-concept implementations and real-world testing. We address the interoperability problem with a generic Grid resource brokering architecture and job submission service.

By using (proposed) standard formats and protocols, the service acts as an interoperability-bridge that translates job requests between clients and resources running different Grid middlewares. This concept is demonstrated by the integration of the service with three different Grid middlewares. The service also enables users to both fine-tune the existing resource selection algorithms and plug in custom brokering algorithms tailored to their requirements.

Place, publisher, year, edition, pages
Umeå: Datavetenskap , 2006. , 112 p.
Series
Report / UMINF, ISSN 0348-0542 ; 06.39
Keyword [en]
Grid computing, Resource brokering, interoperability, advance reservations, coallocation, Web services
National Category
Computer Science
Identifiers
URN: urn:nbn:se:umu:diva-966ISBN: 91-7264-225-4 (print)OAI: oai:DiVA.org:umu-966DiVA: diva2:145222
Supervisors
Available from: 2007-01-04 Created: 2007-01-04 Last updated: 2017-03-27Bibliographically approved
List of papers
1. Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions
Open this publication in new window or tab >>Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions
2008 (English)In: Future generations computer systems, ISSN 0167-739X, E-ISSN 1872-7115, Vol. 24, no 6, 585-593 p.Article in journal (Refereed) Published
Abstract [en]

We present algorithms, methods, and software for a Grid resource manager, that performs resource brokering and job scheduling in production Grids. This decentralized broker selects computational resources based on actual job requirements, job characteristics, and information provided by the resources, with the aim to minimize the total time to delivery for the individual application. The total time to delivery includes the time for program execution, batch queue waiting, and transfer of executable and input/output data to and from the resource. The main features of the resource broker include two alternative approaches to advance reservations, resource selection algorithms based on computer benchmark results and network performance predictions, and a basic adaptation facility. The broker is implemented as a built-in component of a job submission client for the NorduGrid/ARC middleware.

Place, publisher, year, edition, pages
Amsterdam: Elsevier, 2008
Keyword
resource broker, grid scheduling, runtime predictions, performance-based resource selection, advance reservations
National Category
Computer Science
Identifiers
urn:nbn:se:umu:diva-9612 (URN)10.1016/j.future.2007.06.001 (DOI)000255841300010 ()
Available from: 2008-05-04 Created: 2008-05-04 Last updated: 2017-12-14Bibliographically approved
2. An Interoperable Standards-based Grid Resource Broker and Job Submission Service
Open this publication in new window or tab >>An Interoperable Standards-based Grid Resource Broker and Job Submission Service
2005 (English)In: e-Science 2005: First IEEE Conference on e-Science and Grid Computing, IEEE Computer Society Press , 2005, 212-220 p.Chapter in book (Other academic)
Abstract [en]

We present the architecture and implementation of a grid resource broker and job submission service, designed to be as independent as possible of the grid middleware used on the resources. The overall architecture comprises seven general components and a few conversion and integration points where all middleware-specific issues are handled. The implementation is based on state-of-the-art grid and Web services technology as well as existing and emerging standards (WSRF, JSDL, GLUE, WS-Agreement). Features provided by the service include advance reservations and a resource selection process based on a priori estimations of the total time to delivery for the application, including a benchmark-based prediction of the execution time. The general service implementation is based on the Globus Toolkit 4. For test and evaluation, plugins and format converters are provided for use with the NorduGrid ARC middleware

Place, publisher, year, edition, pages
IEEE Computer Society Press, 2005
Identifiers
urn:nbn:se:umu:diva-5638 (URN)10.1109/E-SCIENCE.2005.17 (DOI)
Available from: 2007-01-04 Created: 2007-01-04 Last updated: 2011-02-22Bibliographically approved
3. A standards-based Grid resource brokering service supporting advance reservations, coallocation and cross-Grid interoperability
Open this publication in new window or tab >>A standards-based Grid resource brokering service supporting advance reservations, coallocation and cross-Grid interoperability
2009 (English)In: Concurrency and Computation, ISSN 1532-0626, E-ISSN 1532-0634, Vol. 21, no 18, 2298-2335 p.Article in journal (Refereed) Published
Abstract [en]

The problem of Grid-middleware interoperability is addressed by the design and analysis of a feature-rich, standards-based framework for all-to-all cross-middleware job submission.The architecture is designed with focus on generality and flexibility and builds on extensive use, internally and externally, of (proposed) Web and Grid services standards such asWSRF, JSDL, GLUE, and WS-Agreement. The external use providesthe foundation for easy integration into specific middlewares,which is performed by the design of a small set of plugins for each middleware. Currently, plugins are provided for integrationinto Globus Toolkit 4 and NorduGrid/ARC. The internal use of standard formats facilitates customizationof the job submission service by replacement of custom components for performing specific well-defined tasks.Most importantly, this enables the easy replacement of resource selection algorithms by algorithms that addresses the specific needs of a particular Grid environment and job submission scenario.By default, the service implements a decentralized brokering policy, strivingto optimize the performance for the individual user by minimizing the response time for each job submitted. The algorithms in our implementation perform resource selectionbased on performance predictions, and provide support for advance reservations as well as coallocation of multiple resources for coordinated use.The performance of the system is analyzed with focuson overall service throughput (up to over 250 jobs per minute)and individual job submission response time (down to under one second).

Keyword
Grid resource broker; standards-based infrastructure; interoperability; advance reservations; coallocation; service-oriented architecture (SOA); Globus Toolkit
National Category
Computer Science
Research subject
Computing Science
Identifiers
urn:nbn:se:umu:diva-5639 (URN)10.1002/cpe.1441 (DOI)
Available from: 2007-01-04 Created: 2007-01-04 Last updated: 2017-12-14Bibliographically approved

Open Access in DiVA

fulltext(239 kB)764 downloads
File information
File name FULLTEXT01.pdfFile size 239 kBChecksum SHA-1
ca4750ccffdb4ad8d1110540afca68fa3dd937bc17f39ec3141b7d9d1f5038d21d4fbbbd
Type fulltextMimetype application/pdf

By organisation
Department of Computing Science
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 764 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 419 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf