Journée Présentation de l’ANR
-
Upload
jerome-paul -
Category
Documents
-
view
39 -
download
3
description
Transcript of Journée Présentation de l’ANR
Journée Journée Présentation de l’ANRPrésentation de l’ANR
In conjunction with Perpi’2006RenPar'17 / SympA'2006 / CFSE'5 / JC'20063 octobre 2006
2
Teams LIP/INRIA: Projet GRAAL
Anne Benoît Raphaël Bolze Yves Caniou Eddy Caron Pushpinder Kaur Chouhan Frédéric Desprez Jean-Sébastien Gay Cédric Tedeschi
IRISA/INRIA: Projet PARIS Gabriel Antoniu Luc Bougé Hinde Bouziane Loïc Cudennec Mathieu Jan Sébastien Monnet Christian Perez Thierry Priol
LaBRI/INRIA: Projet RUNTIME Olivier Aumage Alexandre Denis
ENSEEIHT: IRIT Michel Daydé Marc Pantel Daniel Hagimont
CERFACS Eric Maisonnave
ENS-Lyon: CRAL Hélène Courtois Julien Devriendt Romain Teyssier
3
Middleware
Components
Deployment Communications
Data management
Scheduling
The Concept
LEGO: League for Efficient Grid Operation.
Workflow
4
Components
Advanced Component Model
Components and data-sharing service Composition based on data-access
Data port Use of JuxMem
Components and master-worker paradigm Collection + request scheduling Use of Diet
Components and workflow What’s mean dependency for the component model?
Components and legacy code No code re-writing Mechanism to deal between application and scheduler
BA
Data
data_ref
workermaster
5
DIET Architecture
LA
MA
LA
LALA
Server front end
Master Agent
Local Agent
Client
MA
MA
MA
MA
CORBA or JXTA
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.
Middleware
6
Peer
Peer
Peer Peer
Peer
PeerPeer
Peer
PeerPeer
Peer
Peer
FirewallPeer
PeerTCP/IP
HTTP
Peer ID
Peer ID
Peer ID
Peer ID
Peer ID
Peer ID
Peer ID
Peer ID
Firewall
Toolbox for the development of P2P applications
Set of protocols
One peer Unique ID Several communication
protocols (TCP, HTTP, …)
JuxMem: a Grid Data-Sharing Service
A peer-to-peer architecture for a data-sharing service in memory
Persistence and data coherency mechanism Transparent data localization
Datamanagement
7
Communication Brick
Communication for multi-paradigm programming model Message passing Remote procedure calls Distributed/Shared memory
Cluster view: High-speed network Hardware heterogeneity
Myrinet, Quadrics, Infiniband, SCI Gigabit Ethernet
Software heterogeneity GM, MX Elan, Elan4 Sisci Sockets
Contribution Madeleine library
Communications
NetworkNetwork
ProgrammingenvironmentsProgrammingenvironments
Genericcommunication support
Genericcommunication support
Messagepassing
Serviceinvocation(RPC, RMI)
Madeleine
ApplicationprocessesApplicationprocesses
Ethernet Myrinet SCI Quadrics …
Distributedshared memory
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
8
Communications
Grid Communication: PadicoTM Grid communications between site
Wide communications Specific communications Connectivity: firewalls, none-routed network, etc. Performance: High latency, low bandwidth Security: protection,
accounting Middleware and applications
integration Middleware upgrading for
Madeleine? Existing code?
Contribution A high-performance
communication framework for Grids: PadicoTM
PARIS project (2000-2004) andRUNTIME (since 2004)
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
9
Scheduling Brick: into DIET
Plug-in Scheduler Existing plug-in scheduling facilities Application-specific definition of appropriate performance metrics An extensible measurement system Tuneable comparison/aggregation routines for scheduling Composite requirements enables various selection methods
basic resource availability processor speed, memory database contention future requests
CORI Collector: an easy interface to gathering performance and load
information for a specific SeD Two modules (currently): CoRI-Easy and FAST Possible to extend (new modules): Ganglia, Nagios, R-GMA,
Hawkeye, INCA, MDS, …
Scheduling
10
Scheduling Brick: Workflow Workflow management using
component model Workflow and DIET
Simple and high level API for the client Workflow description based on XML
Use of different scheduling algorithms (RR, HEFT, etc.)
Ability for the client to use its own workflow scheduler
Automatic rescheduling mechanisms Support multi-workflows scheduling DIET hierarchy extended with a special
agent: MADAG
Two execution modes of the MADAG Complete scheduling provided : task
priorities and resources mapping Partial scheduling provided : only task
priorities
Workflow
Exemple from Cosmological Application
11
Deployment Brick: ADAGE
Automatic deployment tool for grid environment
Only one command to deploy 3 kinds of input information Resource description
application description control parameter
Planning model (random, round-robin), …
Plug-in for each application Description convector Configuration of application
CCM, MPICH-P4, MPICH-G2, JXTA Plug-in: from 400 to 1200 C++ lines
Deployment
12
Ocean-atmosphere Numerical Simulations
Energy transport: Equator Pole World climate behavior Platform
Supercomputer approach large simulation (1000 years)
Grid approach parameterization design independent and simultaneous simulations
Code coupling ARPEGE v4.5 (atmospheric modelisation) OPA v9 +LIM (ocean modelisation) OASIS v3 (code coupling)
Application
QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.
13
Cosmological Simulation RAMSES
Computes the evolution of dark matter particles starting from the early universe's structure
GALICS Performs structure detection (halos of dark matter) Builds the evolution tree of the particles Generates galaxies
Application
Simulation 1st part, 1 submission from the client:
generating low resolution IC RAMSES post-processing with GALICS, results are sent
back to the user 2nd part, n submissions from the client:
generating high resolution IC centered on the wanted part of the universe
RAMSES post-processing with GALICS, results are sent
back to the user
QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.
QuickTime™ et undécompresseur Vidéo
sont requis pour visionner cette image.
14
Application
Sparse Direct Solvers Sparse direct solvers in a client-server
environment (DIET) Provide remote access to the algorithms we develop (e.g. MUMPS) Easy to use from a light client Data persistency on the servers is crucial
Application: an expertise site for sparse linear algebra: ACI GRID TLSE (coordinated by ENSEEIHT-IRIT, Toulouse)
On a user’s specific problem, compare execution time / accuracy / memory usage / … of various solvers:
public domain … as well as commercial, sequential … as well as parallel
Find best parameter values / reordering heuristics on a given problem Also bibliography, matrix collections, …
All elementary requests executed on the/a GRID through DIET Must be highly evolving (new solvers with new parameters, new
scenarii)
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
15
Conclusion
Programming model brick Component model
Grid middleware brick GridRPC environment: DIET.
Data management brick Data-sharing system: JuxMeM
Communication bricks Intra-cluster: Madeleine Grid communication: PadicoTM
Scheduling brick DIET’s Plug-in scheduler
Workflow bricks DIET’s DAG management Component management
Deployment brick ADAGE
Applications brick Ocean-atmosphere Numerical
Simulations Cosmological Simulation Sparse Direct Solvers
Questions?Questions?
http://graal.ens-lyon.fr/LEGO
QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.
QuickTime™ et undécompresseur TIFF (non compressé)
sont requis pour visionner cette image.