CERN [email protected] juin-00 La Politique du tout PC au CERN CUIC – Arcachon – 21-22 juin...
Transcript of CERN [email protected] juin-00 La Politique du tout PC au CERN CUIC – Arcachon – 21-22 juin...
CERN
[email protected] juin-00
La Politique du tout PC au CERNLa Politique du tout PC au CERN
CUIC – Arcachon – 21-22 juin 2000
Les Robertson
CERN/IT – Genève
CERN
juin-00 - #2les robertson - cern/it
SommaireSommaire
Le problème La stratégie Les difficultés
CERN
juin-00 - #4les robertson - cern/it
Architectures & operating systems supported at end 1999
AIX
Windows NT
Irix
Sola
ris
Digital Unix
HP-
UXM
AC-
OS
Linu
x
Windows
95
SPARCMIPS
Inte
l IA-3
2
PA-RISCPower
PC
Alpha Windows 2000The legacy of ten yearsThe legacy of ten years
of RISC computingof RISC computing
CERN
juin-00 - #5les robertson - cern/it
Combien d’architectures et systèmes d’exploitation sont vraiment nécessaire?
Combien coûte le support? Combien vaut la diversité?
Comment imposer des limitations de choix dans un environnement de recherche scientifique ?
CERN - The European Organisation for Nuclear Research
The European Laboratory for Particle Physics
Fundamental research in particle physics Financed by 20 European countries 6,000 users (researchers) from all over the world
LHC accelerator under construction Proton-proton collider 27 km of super-conducting magnets Target date for first beams - 2005 Four experiments
2000 physicists, 150 universities
CERN
CMS detector- as big as a 6-storey office block,- costing ~FF 2.000M- 1 PetaByte of filtered data per year
level 1 - special hardware
40 MHz (40 TB/sec)level 2 - embedded processorslevel 3 - PCs
75 KHz (75 GB/sec)5 KHz (5 GB/sec)100 Hz(100 MB/sec)data recording &
offline analysis
The LHC DetectorsCMS
ATLAS
LHCb3.5 PetaBytes / year
~108 events/year
CERN
juin-00 - #9les robertson - cern/it
Performance or Throughput?Performance or Throughput?
High Throughput Computing mass of modest problems throughput rather than performance resilience rather than ultimate reliability
Ten years of experience in exploiting inexpensive mass market components
But we need to marry these with inexpensive highly scalable management
tools
Much in common with data mining, Internet computing facilities, ……
Estimated CPU Capacity at CERN
0
500
1,000
1,500
2,000
2,500
1998 1999 2000 2001 2002 2003 2004 2005 2006
year
K S
I95
~10K SI951200 processors
Non-LHC
technology-price curve (40% annual price improvement)
LHC
10-20Kcpus?
Estimated DISK Capacity ay CERN
0
200
400
600
800
1000
1200
1400
1600
1800
1998 1999 2000 2001 2002 2003 2004 2005 2006
year
Ter
aByt
es
Non-LHC
technology-price curve (40% annual price improvement)
LHC
lmr for Monarc study- april 1999
250 Gbps
0.8 Gbps
8 Gbps
…………3000 processors1500 boxes160 clusters40 sub-farms
24 Gbps*
960 Gbps*
6 Gbps*
1.5 Gbps
100 drives
12 Gbps
5400 disks
340 arrays
……...
LAN-WAN routers
CERN
CMS Offline Farmat CERN circa 2006
tapes
0.8 Gbps (daq)
0.8 Gbps
5 Gbps
disks
processors
storage network
storage network
farm network
lmr for Monarc study- april 1999
250 Gbps
0.8 Gbps
8 Gbps
…………3000 processors1500 boxes160 clusters40 sub-farms
24 Gbps*
960 Gbps*
6 Gbps*
1.5 Gbps
100 drives
12 Gbps
5400 disks
340 arrays
……...
LAN-WAN routers
CERN
CMS Offline Farmat CERN circa 2006
tapes
0.8 Gbps (daq)
0.8 Gbps
5 Gbps
disks
processors
storage network
storage network
farm networkLHC physics facility – 4 experiments
2 M SPECint9510-20K processors
2 PByte disk>20 K disks
CERN
juin-00 - #14les robertson - cern/it
Summary of the problemSummary of the problem
HEP is using far too many operating systems in many cases with only slightly different functionality or hardware
cost benefits and at a high cost for users and support teams
The scale of LHC computing - massive numbers of processors/boxes integration of regional computing centres and CERN
problem is how to manage on this scale while limiting costs of equipment, management &
support
We must reduce the diversity while retaining flexibility to use low-cost, mass market componentsand adapt rapidly to changing physics needs
CERN
juin-00 - #16les robertson - cern/it
OpportunityOpportunity
PCs + { Linux ¦ Windows } offer an historic opportunity to reduce the solution set
Costs and performance PCs will consistently be among the very best
price/performers for HEP codes They may not be the fastest,but they are fast
enough
Linux -a non-proprietary operating system compatible with the recent Unix history
Windows – a mass market alternative – widely used on the desktop
CERN
juin-00 - #17les robertson - cern/it
PolicyPolicy
Restrict ourselves to PC hardware with Linux or Windows 2000
Develop a migration plan - progressively freeze support for other Unixes,
announcing end-dates which are reasonable for old experiments,
strongly discourage further investments in RISC systems by current and future experiments
install large Linux public facility, testbed for future experiments
Concentrate investment in Linux and Windows bring support up to the standards of proprietary
Unixes tackle the problems of scaling the management
and performance of physics farms and desktops seek HEP-wide consensus
CERN
juin-00 - #18les robertson - cern/it
But do not be unrealistic ----But do not be unrealistic ----
This is a convergence policy which looks realistic now and will provide a single starting point
for LHC computing but we can be sure that the industry
will not stand still, and we shall sooner or later have to expand the systems and architectures supported
AIX
WNTIrix
Sola
ris
Digital Unix
HP-
UX
MAC-
OS
Linu
x
Windows
95
SPARC
MIPS In
tel I
A-32
PA-RISC
Power
PC
Alpha LinuxWindows
2000
Intel IA-64- - - ?
CERN
[email protected] juin-00
Les difficultésLes difficultés et l'état de la migrationet l'état de la migration
CERN
juin-00 - #20les robertson - cern/it
Difficulties - IDifficulties - I
Physics – (almost) entirely Unix based Linux is not quite ready
(Too) wide a choice of kernels, compilers Poor debugging Different versions supported by different applications Complex packages (Oracle, AFS) – better go with the
standard platform Stability problems under load Who provides in-depth, on-site Linux systems support?
Solution: Standard Linux Package certified for all CERN
applications Solaris/SPARC for special purposes Open posts for Linux experts
CERN
juin-00 - #21les robertson - cern/it
Difficulties - IIDifficulties - II
In a research environment Easy to estimate the costs of systems support Hard to estimate the cost of application migration
The application experts have already moved on The developers have other (more interesting)
problems to solve The problem is not only to port the code but (more important) to acquire confidence in the
physics results Compiler, architecture, old bugs But there are signs that Linux+Intel are as good as
any! In the past, the production use of multiple
architectures was an important factor in finding bugs
CERN
juin-00 - #22les robertson - cern/it
Current Status – PhysicsCurrent Status – Physics
For older experiments – Strong resistance to aggressive migration
proposal Now aiming at complete freeze on all proprietary
Unixes during 2003
For future experiments (not yet collecting data) – General agreement, but reserve position on
alternative platform for validation
For new experiments collecting data now Easy to calculate the benefits Have already completed migration
CERN
juin-00 - #23les robertson - cern/it
Current Status – other applicationsCurrent Status – other applications
Engineering applications Aggressive migration plan to Windows NT/2000
& Linux, with some residual SUN Major exception is mechanical CAE (Euclid +
Digital Unix)
Administration Database (Oracle) on SUN Clients Web-based (Netscape) Strong pockets of MAC resistance
– led by the Directorate
CERN
juin-00 - #24les robertson - cern/it
ConclusionsConclusions
Les besoins énormes du LHC exigent la standardisation et l’utilisation des composants bon-marché
Opportunité –Linux + Windows 2000 avec Intel IA32/64 & Ethernet
Grande inertie (résistance?) de la part des « vielles » expériences - il faudra 4 années pour terminer la migration
Mais – déjà plus de la moitié des systèmes installés et 75% de la capacité sont Linux/Intel