PS INFN GRID · Taiwan – Academia SInica (Taipei) UK – CLRC (Oxford) US – FermiLab (Illinois)...

PS INFN GRID

Obiettivi ScientificiStato del progettoStato del progetto

Connessione con attività ATLASPersone e risorsePersone e risorse

5-luglio-2007 Milano-LASA L.Perini 1

Obiettivi PS INFN GRID• Coordinare la partecipazione INFN ai progetti Computing

Grid– Ottenere così il finanziamento delle persone che sviluppano eOttenere così il finanziamento delle persone che sviluppano e

mantengono mw e infrastruttura Grid con fondi extra INFN• EU ma anche italiani extra INFN

• Per INFN interesse specifico di GRID èPer INFN interesse specifico di GRID è– Il suo utilizzo per il calcolo esperimenti LHC– Assunzione di leadership in campo tecnologico promettente

• La principale fonte di fondi è oggi EGEE II• La principale fonte di fondi è oggi EGEE-II• L’utilizzazione di Grid per LHC è coordinata nel progetto

LHC Computing Grid (LCG)


INFNINFN GRIDGRIDINFNINFN--GRIDGRID

INFN-GRID è il Progetto Speciale INFN “contenitore” di tutte le attività GRID comprese quelle degli esperimenti: i ricercatori-tecnologi afferenti agli esperimenti che si impegnano in GRID lo fanno per implementare il calcolodel loro esperimento, quindi la loro % INFN GRID è inclusa in quella dell’esperimentoData di inizio: giugno 2000http://grid.infn.ithttp://grid.infn.it

LCGLCGLHC Computing Grid H L’ bi tti di li i f t tt (h/ / ) il l l di t ib itHa L’obiettivo di realizzare una infrastruttura (h/w e s/w) comune per il calcolo distribuito degli esperimenti LHC ( 4 aree: Grid Tecnology, Grid Deployment, Fabric, Applications)Fase 1 (2002-2005): realizzazione prototipoFase2 (2006-2008): realizzazione e gestione del “servizio di calcolo”( ) g

per LHC http://www.cern.ch/LCG

5-luglio-2007 3

LCG WLCG Collaboration

The Collaboration~100 computing centres12 large centres

(Tier-0, Tier-1)38 federations of smaller

“Tier-2” centres20 countries

M d f U d t diMemorandum of UnderstandingAgreed in October 2005

ResourcesC i d h O b f h i Commitment made each October for the coming year5-year forward look

HEPiX Rome 05apr06 [email protected]

LCG LCG Service HierarchyTier-0 – the accelerator centre

Data acquisition & initial processingLong-term data curationDistribution of data Tier-1 centres

Tier-1 – “online” to the data acquisition process high availabilityManaged Mass Storage –

Canada – Triumf (Vancouver)France – IN2P3 (Lyon)Germany – Forschunszentrum KarlsruheItaly – CNAF (Bologna)N th l d Ti 1 (A t d )

Spain – PIC (Barcelona)Taiwan – Academia SInica (Taipei)UK – CLRC (Oxford)US – FermiLab (Illinois)

Managed Mass Storage grid-enabled data

serviceData-heavy analysisN ti n l i n l s pp tNetherlands Tier-1 (Amsterdam)

Nordic countries – distributed Tier-1 US – FermiLab (Illinois)

– Brookhaven (NY) National, regional support

Tier-2 – ~100 centres in ~40 countriesSi l ti


SimulationEnd-user analysis – batch and interactive

LCG

Summary of Computing Resource RequirementsAll experiments - 2008From LCG TDR - June 2005

CERN All Tier-1s All Tier-2s TotalCPU (MSPECint2000s) 25 56 61 142Disk (PetaBytes) 7 31 19 57

CPU Disk Tape

Disk (PetaBytes) 7 31 19 57Tape (PetaBytes) 18 35 53

CERN18%

All Tier-2s43%

CERN12%

All Tier-2s33%

CERN34%

p

All Tier-1s39%

All Tier-1s55%

All Tier-1s66%


LCG LCG depends on two major sciencegrid infrastructures grid infrastructures ….

EGEE - Enabling Grids for E-SciencegOSG - US Open Science Grid


LCGProblem Response Time and Availability targets

Tier-1 CentresLCG

Service

Maximum delay in responding to operational problems (hours)

AvailabilityDegradation of theService Availability Service

interruption

gservice

> 50% > 20%

Acceptance of datafrom the Tier-0 Centreduring accelerator operation

12 12 24 99%operation

Other essential services– prime service hours 2 2 4 98%p

Other essential services– outside prime

service hours24 48 48 97%


LCGLCG

Problem Response Time and Availability targetsTier-2 Centres

Maximum delay in responding

Service

y p gto operational problems

availabilityPrime time Other periods

E d l i 2 h 72 h 95%End-user analysis facility

2 hours 72 hours 95%

Other services 12 hours 72 hours 95%


Enabling Grids for E-sciencE

Project GoalsFrom EGEE Final EU Review May 2006

Within a four year programme:

• Build, deploy and operate a consistent, robust and secure grid that attracts new computing resourcessecure grid that attracts new computing resources

• Improve and maintain the middleware in order to pdeliver a reliable service to users

• Attract new users from science and industry and ensure training and support for them

INFSO-RI-508833 10


Activities Definition

• Network Activities– NA1: Project Management– NA2: Dissemination and Outreach– NA3: User Training and Induction– NA4: Application Identification and Support– NA5: Policy and International Cooperation

• Service Activities– SA1: Grid Support, Operation and Management– SA2: Network Resource Provision– SA2: Network Resource Provision

• Joint Research Activities– JRA1: Middleware Reengineering + Integrationg g g– JRA2: Quality Assurance– JRA3: Security– JRA4: Network Services Development

Emphasis in EGEE is on operating a production

id d ti th

INFSO-RI-508833 Milano-LASA L.Perini 11

grid and supporting the end-users


What has EGEE delivered?I f t t ti• Infrastructure operation– Currently includes >180 sites across 40 countries build

on GEANT network– Continuous monitoring of grid services & automated site

fi ti / tconfiguration/management

• Middleware– Production quality middleware distributed under q y

business friendly open source licence

• User Support - Managed process from first contact through to production usagethrough to production usage– Training– Documentation – Expertise in grid-enabling applications

Online helpdesk– Online helpdesk– Networking events (User Forum, Conferences etc.)

• Interoperability


– Expanding interoperability with related infrastructures– Major contributor to grid standardisation


Related Projects

EGEE as incubator & supporter of related projects


DiligentA DIgital Library Infrastructureon Grid ENabled Technology

Etc.


Manpower levels

• Further refined PPT tool to monitor project effort consumption (timesheets)– Over 1000 people registered across Europe and Russiap p g p– Total equivalent of ~ 460 FTEs

Legend:

Total (F+UF) Effort in PM

12000

PM

Legend:• F: Funded• U: Unfunded• PM: Person Months8000

10000

• PM: Person Months• FTE: Full time equiv.

6000

8000

Consumed F+UF EffortExpected F+UF Effort

Consumed Effort

Funded Unfunded Total

112% 122% 114%2000

4000


0Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8


The EGEE-II projectEGEE• EGEE– 1 April 2004 – 31 March 2006– 71 partners in 27 countries, federated in regional Grids

• EGEE-II– 1 April 2006 – 31 March 2008– 91 partners in 32 countries91 partners in 32 countries – 13 Federations

• ObjectivesL l d ti lit– Large-scale, production-quality infrastructure for e-Science

– Attracting new resources and users from industry as well asusers from industry as well asscience

– Maintain and further improve“gLite” Grid middleware


gLite Grid middleware


Sustainability: Beyond EGEE-II

• Need to prepare for permanent Grid infrastructure– Maintain leading position in global science Grids

Ensure a reliable and adaptive support for all sciences– Ensure a reliable and adaptive support for all sciences– Independent of short project funding cycles– Modelled on success of GÉANT

Infrastructure managed in collaboration with national grid initiatives



• PoW of technical activities is coordinated by the Technical Coordination Group (TCG)

• Makes decision process clearer and ensures application driven progress

• Started in November 2005

• Main accomplishments– Definition of gLite 3.0

Clarification and prioritization of main– Clarification and prioritization of main application requirements

– Draft short term workplans for mw development and integration


p g


Middleware structure• Applications have access

both to Higher-level Grid Services and to Foundation Grid MiddlewareGrid Middleware

• Higher-Level Grid Services are supposed to help the users building theirusers building their computing infrastructure but should not be mandatory

• Foundation Grid Middleware will be deployed on the EGEE infrastructure– Must be complete and robust

Should allow interoperation– Should allow interoperation with other major grid infrastructures

– Should not assume the use of


Higher-Level Grid Services


Highlights of EGEE-II

EGEE Production Grid InfrastructureSteady growth over the lifetime of the projectImproved reliability

Data TransferM

B/s

M

04/2006 08/2006

EGEE-II INFSO-RI-031688 Milano-LASA L.Perini 19


CPU, countries, sites

CERN; 5943 SWE; 1593

Russia; 583 A‐P; 1632

CPU / ROC

CERN; 4France; 1

De/CH; 2

Italy; 1

A‐P; 8

Countries / ROC

France; 2700

SEE; 2568

Italy; 1

UK/I; 2

CE; 7

SWE; 2

Russia; 2

De/CH; 3364CE; 1875

NE; 3031

NE; 8

SEE; 8

Sites / ROC

Italy; 3628

UK/I; 7720

CERN; 12France; 10

De/CH; 14

SWE; 15

Russia; 15

A‐P; 20

Sites / ROC

UK/I; 7720

Italy; 37

SEE; 3835000 CPU45 countries (31 partner countries)

EGEE-II INFSO-RI-031688 Milano-LASA L.Perini 20SA1 - Ian Bird - EGEE-II 1st EU Review - 15-16 May 2007 20

UK/I; 25

CE; 24NE; 27

( p )237 sites (131 partner sites)


Workload

2000000

2500000

3000000No. jobs / month ‐ all 98000 jobs/day

500000

1000000

1500000 OPS

Non‐LHC

LHC

0

250000

300000

350000

400000

450000No. jobs / month ‐ exc. LHC + Ops Other VOs

planck

magic

4

13000 jobs/day

0

50000

100000

150000

200000geant4

fusion

esr

egrid

egeode


egeode

compchem


CPU time delivered

6000000

7000000

8000000

9000000

10000000Normalised CPU hours ‐ all

14000 CPU-month/month

2000000

3000000

4000000

5000000

6000000OPS

Non‐LHC

LHC

0

1000000

apr‐06 mag‐06 giu‐06 lug‐06 ago‐06 set‐06 ott‐06 nov‐06 dic‐06 gen‐07 feb‐07 mar‐07

3000000

2000000

2500000

3000000Normalized CPU hours ‐ exc. LHC + Ops OPS

Other VOs

planck

magic

3600 CPU-month~ 1/3 of total

500000

1000000

1500000 geant4

fusion

esr

egrid

d


0

apr‐06 mag‐06 giu‐06 lug‐06 ago‐06 set‐06 ott‐06 nov‐06 dic‐06 gen‐07 feb‐07 mar‐07

egeode

compchem


Regional distributionRegional share of jobs

AsiaPacific, 3.5% CentralEurope, 6.5%

CERN; 7,0%UKI; 22,2%

Regional share of jobs

France; 12,6%SouthWest 6.4%

Asia Pacific, 2.8% CentralEurope; 4,9%

Regional share of CPU deliveredGermany/Switzerland, 8.6%

Italy; 15,8%NorthernEurope;

6,7%

Russia; 3,9%

South East, 6.9%

CERN; 6,9%

France; 11,2%

UKI; 30,3%

GermanySwitzerland; 7,8%

SouthWesternEurope; 7 9%


Italy; 11,1%

NorthernEurope; 5,4%Russia; 2,1%

SouthEasternEurope; 9,7%

ope; 7,9%

Ruolo e persone Milano GRID • Milano ha ruolo molto rilevante, essendo:

– Tier2 – sj ( dettagli in presentazione ATLAS-GRID)• Interfaccia locale GRID Luca Vaccarossa, finanziato 100% da SA1 EGEE

– Sede di uno dei cluster di sviluppo del m/w in EGEE• F.Prelz (50%) (membro del advisory board internazionale) e personale al

100% finanziato da EGEE: art.23 (M. Mezzadri, D.Rebatto) ( )– Sede “experimental services”

• WMS gLite 3.1 è stato portato a production level in collaborazione da ATLAS e SA3 EGEE ( E. Molinari art.23) e i servers WMS sono ora 2 al CNAF e 1 a Milano

• A Milano è presente anche OMII (Open Mw Initiative) – Progetto Eu collegato ad EGEE per costruire e mantenere un set di mw e

servizi a disposizione di tutte le comunità Grid. G. Fiorentino art.23 . In i l BLAH i i d l i i CE ilparticolare per BLAH componente critica del servizio CE, sviluppata a

Milano in EGEE.• Persone degli esperimenti ( 0% formalmente)


Da EGEE2 a EGEE3• EGEE2 ha appena superato con la valutazione migliore fra

quelle possibili la review dopo il suo primo anno– Ormai la grid è l’infrastruttura per il calcolo distribuito di LHC eOrmai la grid è l infrastruttura per il calcolo distribuito di LHC e

non solo…• EGEE3 e’ in fase di negoziazione, è certo che sarà

finanziato con livello simile a quello di EGEE2 circa 35 Mfinanziato con livello simile a quello di EGEE2, circa 35 M Euro– Potrebbero essere però 30 o 38 e fa una bella differenza

• Altri progetti GRID in cui INFN (Milano) è:• Altri progetti GRID in cui INFN (Milano) è:– EuChina, EuMed, EuIndia, EELA, LIBI, GRIDCC, CoreGrid,

ETICS, OMII, EGI/IGI


Persone di Milano e sinergie conPersone di Milano e sinergie con ATLAS-Grid

• Funded da progetti Grid EU– S. Resconi NA4-ATLAS – D. Rebatto JRA1 EGEE2 (blah) ATLAS-Lexor (TF)– L. Vaccarossa SA1 EGEE2 ( Tier2, Atlas-TF)– E. Molinari SA3 EGEE2 (ATLAS-TF)– M. Mezzadri JRA1 EGEE2 (blah e info.system)

I l 50% l l• Internal agreement 50% serv.calcolo– G. Fiorentino OMII (blah)

• Unfunded 50% F P l JRA1 INFN– 50% F. Prelz JRA1-INFN

– 50% L. Perini TCG (COLG INFN GRID) in uscita…– Probabile che ci venga richiesto in EGEE3 aumento unfunded

(tipicamente da serv. Calcolo)


(tipicamente da serv. Calcolo)

Necessità PS GRID 2007 e oltre• Per i fondi si tratta principalmente di missioni e consumo, riporto

richieste 2008 ancora tentative..– MI 17 kEuro

• Inclusi 3 kEuro Perini (deputy responsabile nazionale)• Inclusi 3 kEuro Perini (deputy responsabile nazionale)– ME 43 kEuro

• Inclusi 5 k Euro per L.Perini EGI/IGI– Consumo 5 kEuro– Inv. 2.5 k Euro

• EGEE2 termina a fine marzo 2008. – Negoziazione avanzata e positiva per EGEE3 e OMII2, contiamo di potere

avere 6 persone funded a Milano anche per i 2 anni successivi• Non ci sarà EGEE4, si stabilirà un consorzio fra Grid nazionali su

modello GARR per retiEGI/IGI è il progetto EU approvato per studiare come strutturare e far– EGI/IGI è il progetto EU approvato per studiare come strutturare e far partire in tempo questo consorzio

– INFN in EGI/IGI: M. Mazzucato (con supporto D.Cresti) e L.Perini


PS INFN GRID · Taiwan – Academia SInica (Taipei) UK – CLRC (Oxford) US – FermiLab (Illinois)...

Documents

Transcript of PS INFN GRID · Taiwan – Academia SInica (Taipei) UK – CLRC (Oxford) US – FermiLab (Illinois)...