Dynamic Resource Extension for Data Intensive Computing with Specialized Software Environments on HPC Systems

DSpace Repositorium (Manakin basiert)


Dateien:

Zitierfähiger Link (URI): http://hdl.handle.net/10900/87665
http://nbn-resolving.de/urn:nbn:de:bsz:21-dspace-876657
http://dx.doi.org/10.15496/publikation-29051
Dokumentart: Konferenzpaper
Erscheinungsdatum: 2019-04
Sprache: Englisch
Fakultät: 7 Mathematisch-Naturwissenschaftliche Fakultät
Fachbereich: Informatik
DDC-Klassifikation: 004 - Informatik
Schlagworte: Hochleistungsrechnen
Freie Schlagwörter: bwHPC Symposium
HPC
HEP
LHC
NEMO
Zur Langanzeige

Abstract:

Modern High Energy Physics (HEP) requires large-scale processing of extensive amounts of scientific data. The needed computing resources are currently provided statically by HEP specific computing centers. To increase the number of available resources, for example to cover peak loads, the HEP computing development team at KIT concentrates on the dynamic integration of additional computing resources into the HEP infrastructure. Therefore, we developed ROCED, a tool to dynamically request and integrate computing resources including resources at HPC centers and commercial cloud providers. Since these resources usually do not support HEP software natively, we rely on virtualization and container technologies, which allows us to run HEP workflows on these so called opportunistic resources. Additionally, we study the efficient processing of huge amounts of data on a distributed infrastructure, where the data is usually stored at HEP specific data centers and is accessed remotely over WAN. To optimize the overall data throughput and to increase the CPU efficiency, we are currently developing an automated caching system for frequently used data that is transparently integrated into the distributed HEP computing infrastructure.

Das Dokument erscheint in: