umu.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Portable Tools for Interoperable Grids: Modular Architectures and Software for Job and Workflow Management
Umeå universitet, Teknisk-naturvetenskaplig fakultet, Institutionen för datavetenskap. (gird)
2009 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

The emergence of Grid computing infrastructures enables researchers to shareresources and collaborate in more efficient ways than before, despite belongingto different organizations and being geographically distributed. While the Gridcomputing paradigm offers new opportunities, it also gives rise to newdifficulties. This thesis investigates methods, architectures, and algorithmsfor a range of topics in the area of Grid resource management. One studiedtopic is how to automate and improve resource selection, despite heterogeneityin Grid hardware, software, availability, ownership, and usage policies.Algorithmical difficulties for this are, e.g., characterization of jobs andresources, prediction of resource performance, and data placementconsiderations. Investigated Quality of Service aspects of resource selectioninclude how to guarantee job start and/or completion times as well as how tosynchronize multiple resources for coordinated use through coallocation.Another explored research topic is architectural considerations for frameworksthat simplify and automate submission, monitoring, and fault handling for largeamounts of jobs. This thesis also investigates suitable Grid interactionpatterns for scientific workflows, studies programming models that enable dataparallelism for such workflows, as well as analyzes how workflow compositiontools should be designed to increase flexibility and expressiveness. We today have the somewhat paradoxical situation where Grids, originally aimed tofederate resources and overcome interoperability problems between differentcomputing platforms, themselves struggle with interoperability problems causedby the wide range of interfaces, protocols, and data formats that are used indifferent environments. This thesis demonstrates how proof-of-concept softwaretools for Grid resource management can, by using (proposed) standard formatsand protocols as well as leveraging state-of-the-art principles fromservice-oriented architectures, be made independent of current Gridinfrastructures. Further interoperability contributions include an in-depthstudy that surveys issues related to the use of Grid resources in scientificworkflows. This study improves our understanding of interoperability amongscientific workflow systems by viewing this topic from three differentperspectives: model of computation, workflow language, and executionenvironment. A final contribution in this thesis is the investigation of how the design ofGrid middleware tools can adopt principles and concepts from softwareengineering in order to improve, e.g., adaptability and interoperability.

Ort, förlag, år, upplaga, sidor
Umeå: Umeå Universitet, Institutionen för Datavetenskap , 2009. , 254 s.
Serie
Report / UMINF, ISSN 0348-0542 ; 09.08
Nyckelord [en]
Grid computing, scheduling, resource brokering, performance predictions, advance reservations, coallocation, standards-based infrastructure, interoperability, service-oriented architecture, job management, workflows, data flow
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
URN: urn:nbn:se:umu:diva-19630ISBN: 978-91-7264-754-1 (tryckt)OAI: oai:DiVA.org:umu-19630DiVA: diva2:202198
Disputation
2009-04-03, MA121, MIT-huset, SE 901 87, Umeå, 13:15 (Engelska)
Opponent
Handledare
Tillgänglig från: 2009-03-13 Skapad: 2009-03-09 Senast uppdaterad: 2009-03-13Bibliografiskt granskad
Delarbeten
1. Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions
Öppna denna publikation i ny flik eller fönster >>Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions
2008 (Engelska)Ingår i: Future generations computer systems, ISSN 0167-739X, E-ISSN 1872-7115, Vol. 24, nr 6, 585-593 s.Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

We present algorithms, methods, and software for a Grid resource manager, that performs resource brokering and job scheduling in production Grids. This decentralized broker selects computational resources based on actual job requirements, job characteristics, and information provided by the resources, with the aim to minimize the total time to delivery for the individual application. The total time to delivery includes the time for program execution, batch queue waiting, and transfer of executable and input/output data to and from the resource. The main features of the resource broker include two alternative approaches to advance reservations, resource selection algorithms based on computer benchmark results and network performance predictions, and a basic adaptation facility. The broker is implemented as a built-in component of a job submission client for the NorduGrid/ARC middleware.

Ort, förlag, år, upplaga, sidor
Amsterdam: Elsevier, 2008
Nyckelord
resource broker, grid scheduling, runtime predictions, performance-based resource selection, advance reservations
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
urn:nbn:se:umu:diva-9612 (URN)10.1016/j.future.2007.06.001 (DOI)000255841300010 ()
Tillgänglig från: 2008-05-04 Skapad: 2008-05-04 Senast uppdaterad: 2017-03-27Bibliografiskt granskad
2. An Interoperable, Standards-Based Grid Resource Broker and Job Submission Service
Öppna denna publikation i ny flik eller fönster >>An Interoperable, Standards-Based Grid Resource Broker and Job Submission Service
2005 (Engelska)Ingår i: First International Conference on e-Science and Grid Computing, 2005, 212-220 s.Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

We present the architecture and implementation of a grid resource broker and job submission service, designed to be as independent as possible of the grid middleware used on the resources. The overall architecture comprises seven general components and a few conversion and integration points where all middleware-specific issues are handled. The implementation is based on state-of-the-art grid and Web services technology as well as existing and emerging standards (WSRF, JSDL, GLUE, WS-Agreement). Features provided by the service include advance reservations and a resource selection process based on a priori estimations of the total time to delivery for the application, including a benchmark-based prediction of the execution time. The general service implementation is based on the Globus Toolkit 4. For test and evaluation, plugins and format converters are provided for use with the NorduGrid ARC middleware

Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
administrativ databehandling
Identifikatorer
urn:nbn:se:umu:diva-9304 (URN)10.1109/E-SCIENCE.2005.17 (DOI)0-7695-2448-6 (ISBN)
Tillgänglig från: 2008-05-05 Skapad: 2008-05-05 Senast uppdaterad: 2011-02-22Bibliografiskt granskad
3. A standards-based Grid resource brokering service supporting advance reservations, coallocation and cross-Grid interoperability
Öppna denna publikation i ny flik eller fönster >>A standards-based Grid resource brokering service supporting advance reservations, coallocation and cross-Grid interoperability
2009 (Engelska)Ingår i: Concurrency and Computation, ISSN 1532-0626, E-ISSN 1532-0634, Vol. 21, nr 18, 2298-2335 s.Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

The problem of Grid-middleware interoperability is addressed by the design and analysis of a feature-rich, standards-based framework for all-to-all cross-middleware job submission.The architecture is designed with focus on generality and flexibility and builds on extensive use, internally and externally, of (proposed) Web and Grid services standards such asWSRF, JSDL, GLUE, and WS-Agreement. The external use providesthe foundation for easy integration into specific middlewares,which is performed by the design of a small set of plugins for each middleware. Currently, plugins are provided for integrationinto Globus Toolkit 4 and NorduGrid/ARC. The internal use of standard formats facilitates customizationof the job submission service by replacement of custom components for performing specific well-defined tasks.Most importantly, this enables the easy replacement of resource selection algorithms by algorithms that addresses the specific needs of a particular Grid environment and job submission scenario.By default, the service implements a decentralized brokering policy, strivingto optimize the performance for the individual user by minimizing the response time for each job submitted. The algorithms in our implementation perform resource selectionbased on performance predictions, and provide support for advance reservations as well as coallocation of multiple resources for coordinated use.The performance of the system is analyzed with focuson overall service throughput (up to over 250 jobs per minute)and individual job submission response time (down to under one second).

Nyckelord
Grid resource broker; standards-based infrastructure; interoperability; advance reservations; coallocation; service-oriented architecture (SOA); Globus Toolkit
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
administrativ databehandling
Identifikatorer
urn:nbn:se:umu:diva-5639 (URN)10.1002/cpe.1441 (DOI)
Tillgänglig från: 2007-01-04 Skapad: 2007-01-04 Senast uppdaterad: 2012-11-23Bibliografiskt granskad
4. Designing general, composable, and middleware-independent Grid infrastructure tools for multi-tiered job management
Öppna denna publikation i ny flik eller fönster >>Designing general, composable, and middleware-independent Grid infrastructure tools for multi-tiered job management
Visa övriga...
2007 (Engelska)Ingår i: Towards Next Generation Grids / [ed] T. Priol and M. Vaneschi, Springer-Verlag , 2007, 175-184 s.Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

We propose a multi-tiered architecture for middleware-independent Grid job management. The architecture consists of a number of services for well-defined tasks in the job management process, offering complete user-level isolation of servicecapabilities, multiple layers of abstraction, control, and fault tolerance. The middleware abstraction layer comprises components for targeted job submission, job control and resource discovery. The brokered job submission layer offers a Grid view on resources, including functionality for resource brokering and submission of jobs to selected resources. The reliable job submission layer includes components for fault tolerant execution of individual jobs and groups of independentjobs, respectively. The architecture is proposed as a composable set of tools rather than a monolithic solution, allowing users to select the individual components of interest. The prototype presented is implemented using the Globus Toolkit 4, integrated with the Globus Toolkit 4 and NorduGrid/ARC middlewares and based on existing and emerging Grid standards. A performance evaluation reveals that the overhead for resource discovery, brokering, middleware-specific format conversions, job monitoring, fault tolerance, and management of individual and groups of jobs is sufficiently small to motivate the use of the framework.

Ort, förlag, år, upplaga, sidor
Springer-Verlag, 2007
Nyckelord
Grid Job management infrastructure, standards-based architecture, fault-tolerance, middleware-independence, Grid ecosystem
Identifikatorer
urn:nbn:se:umu:diva-9305 (URN)978-0-387-72497-3 (ISBN)
Konferens
CoreGRID Symposium 2007
Tillgänglig från: 2008-05-05 Skapad: 2008-05-05 Senast uppdaterad: 2012-02-06
5. A light-weight Grid workflow execution service enabling client and middleware independence
Öppna denna publikation i ny flik eller fönster >>A light-weight Grid workflow execution service enabling client and middleware independence
2008 (Engelska)Ingår i: Parallel Processing and Applied Mathematics: 7th International Conference on Parallel Processing and Applied Mathematics (PPAM 2007), Springer-Verlag , 2008, 754-761 s.Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

We present a generic and light-weight Grid workflow execution engine made available as a Grid service. A long-term goal is to facilitate the rapid development of application-oriented end-user workflow tools, while providing a high degree of Grid middleware-independence. The workflow engine is designed for workflow execution, independent of client tools for workflow definition. A flexible plugin-structure for middleware-integration provides a strict separation of the workflow execution and the processing of individual tasks, such as computational jobs or file transfers. The light-weight design is achieved by focusing on the generic workflow execution components and by leveraging state-of-the art Grid technology, e.g., for state management. The current prototype is implemented using the Globus Toolkit 4 (GT4) Java WS Core and has support for executing workflows produced by Karajan. It also includes plugins for task execution with GT4 as well as a high-level Grid job management framework.

Ort, förlag, år, upplaga, sidor
Springer-Verlag, 2008
Serie
Lecture Notes in Computer Science, ISSN 0302-9743 ; 4967
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
administrativ databehandling
Identifikatorer
urn:nbn:se:umu:diva-9615 (URN)10.1007/978-3-540-68111-3_79 (DOI)
Tillgänglig från: 2008-05-05 Skapad: 2008-05-05 Senast uppdaterad: 2011-02-22Bibliografiskt granskad
6. Combining local and grid resources in scientific workflows (for Bioinformatics)
Öppna denna publikation i ny flik eller fönster >>Combining local and grid resources in scientific workflows (for Bioinformatics)
Visa övriga...
2009 (Engelska)Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

We examine some issues that arise when using both local and Gridresources in scientific workflows. Our previous work addresses and illustratesthe benefits of a light-weight and generic workflow engine that manages andoptimizes Grid resource usage. Extending on this effort, we hereillustrate how a client tool for bioinformatics applications employs the engine tointerface with Grid resources. We also explore how to define data flowsthat transparently integrates local and Grid subworkflows. In addition, the benefits of parameter sweep workflows are examined and a means for describing this type of workflows in an abstract and concise manner is introduced. Finally, the above mechanisms are employed to perform an orthology detection analysis.

Ort, förlag, år, upplaga, sidor
Springer-Verlag, 2009
Serie
Lecture Notes in Computer Sciencce, ISSN 0302-9743 ; 6126, 6127
Nyckelord
Grid, workflow, data flow, parameter sweep, bioinformatics
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
administrativ databehandling
Identifikatorer
urn:nbn:se:umu:diva-19545 (URN)
Konferens
The 9th International Workshop on State-of-the-Art in Scientific and Parallel Computing, May 13-16 2008, NTNU, Trondheim, Norway
Anmärkning
accepted 2009Tillgänglig från: 2009-03-09 Skapad: 2009-03-06 Senast uppdaterad: 2012-08-15Bibliografiskt granskad
7. Three fundamental dimensions of scientific workflow interoperability: model of computation, language, and execution environment
Öppna denna publikation i ny flik eller fönster >>Three fundamental dimensions of scientific workflow interoperability: model of computation, language, and execution environment
2010 (Engelska)Ingår i: Future generations computer systems, ISSN 0167-739X, Vol. 26, nr 2, 245-256 s.Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

We investigate interoperability aspects of scientific workflow systems and argue that the workflow execution environment, the model of computation (MoC), and the workflow language form three dimensions that must be considered depending on the type of interoperability sought: at the activity, sub-workflow, or workflow levels. With a focus on the problems that affect interoperability, we illustrate how these issues are tackled by current scientific workflows as well as how similar problems have been addressed in related areas. Our long-term objective is to achieve (logical) interoperability between workflow systems operating under different MoCs, using distinct language features, and sharing activities running on different execution environments.

Ort, förlag, år, upplaga, sidor
Elsevier, 2010
Nyckelord
Scientific workflows; Workflow interoperability; Workflow languages; Model of computation; Grid interoperability
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
administrativ databehandling
Identifikatorer
urn:nbn:se:umu:diva-19544 (URN)10.1016/j.future.2009.08.011 (DOI)000271985200007 ()
Tillgänglig från: 2009-03-06 Skapad: 2009-03-06 Senast uppdaterad: 2012-01-13Bibliografiskt granskad
8. Designing service-based resource management tools for a healthy grid ecosystem
Öppna denna publikation i ny flik eller fönster >>Designing service-based resource management tools for a healthy grid ecosystem
2008 (Svenska)Ingår i: Parallel processing and applied mathematics: 7th International Conference on Parallel Processing and Applied Mathematics, Springer-Verlag , 2008, 259-270 s.Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

We present an approach for development of Grid resource management tools, where we put into practice internationally established high-level views of future Grid architectures. The approach addresses fundamental Grid challenges and strives towards a future vision of the Grid where capabilities are made available as independent and dynamically assembled utilities, enabling run-time changes in the structure, behavior, and location of software. The presentation is made in terms of design heuristics, design patterns, and quality attributes, and is centered around the key concepts of co-existence, composability, adoptability, adaptability, changeability, and interoperability. The practical realization of the approach is illustrated by five case studies (recently developed Grid tools) high-lighting the most distinct aspects of these key concepts for each tool. The approach contributes to a healthy Grid ecosystem that promotes a natural selection of “surviving” components through competition, innovation, evolution, and diversity. In conclusion, this environment facilitates the use and composition of components on a per-component basis.

Ort, förlag, år, upplaga, sidor
Springer-Verlag, 2008
Serie
Lecture Notes in Computer Science, ISSN 0302-9743 ; 4967
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
administrativ databehandling
Identifikatorer
urn:nbn:se:umu:diva-9614 (URN)10.1007/978-3-540-68111-3_28 (DOI)
Konferens
7th International Conference on Parallel Processing and Applied Mathematics, 2008
Tillgänglig från: 2008-05-05 Skapad: 2008-05-05 Senast uppdaterad: 2011-04-11Bibliografiskt granskad

Open Access i DiVA

fulltext(573 kB)918 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 573 kBChecksumma SHA-512
6bb7ab990cd929dfa5de13ae50fa124c6f0c0cef082067b955ec799e17b056d0ad75aa123f4dfcaa27ca7843e2318340a5050685934a168114d88659060e6b71
Typ fulltextMimetyp application/pdf

Personposter BETA

Tordsson, Johan

Sök vidare i DiVA

Av författaren/redaktören
Tordsson, Johan
Av organisationen
Institutionen för datavetenskap
Datavetenskap (datalogi)

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 918 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 1852 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf