Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004 Il progetto EGEE Luciano...

Workshop della Commissione Calcolo e RetiCastiadas, 25-27 maggio 2004

Il progetto EGEELuciano Gaido

Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

Il contesto

01/01 – 03/04

12/01 – 03/04

2002-2005 + 2006-2008

06/00 - ......

Grid.it (FIRB) 01/03 – 12/05


Risultati dei progetti precedenti

• Release di middleware• EDG 2.x

• LCG 2.0 (basata su EDG 2.0)

• INFN-GRID 2.0 (basata su LCG 2.0)

• EGEE-0 (aka LCG 2.0)

• Infrastrutture• Testbed EDG (development, application)

• Testbed INFN-GRID

• Testbed DataTAG

• Grid di produzione italiana (INFN-GRID, GRID-IT)

• Infrastruttura di produzione LCG


Le infrastrutture Le infrastrutture

TB0 TB1 TB2 TB3

Mar 2004

2001 2002 2003

Globus 1 EDG 1 EDG 1.4 EDG 2

2004

AssessmentR&D

Jan 2001

LCG-1 LCG-2

INFNGRID-1 INFNGRID-2Figura presa dalla pres. di M. Parsons a Cork


Il progetto EGEE

• Obiettivo:

“EGEE aims to integrate current national, regional and thematic Grid efforts in order to create a seamless European Grid Infrastructure. This infrastructure will build on the underlying EU Research Network GEANT and exploit Grid expertise that has been generated by projects such as the European Union DataGrid project, other EU supported Grid projects and the national Grid initiatives such as UK e-Science, InfnGrid and NorduGrid.”

• Iniziato il 1 aprile 2004, con durata biennale

• Previsto un rinnovo per un secondo biennio

• 70 partner, 32 MEuro

• Kick-off meeting a Cork (19-22 aprile 2004)

Applications

Geant network

Grid infrastr.


Organizzazione di EGEE

I 70 partner sono stati divisi in 12 federazioni:

1. CERN2. Central Europe including Austria, Czech Republic, Hungary, Poland, Slovakia and Slovenia3. France4. Germany and Switzerland5. Ireland and the United Kingdom6. Italy7. Northern Europe including Belgium, Denmark, Finland, the Netherlands, Norway and Sweden8. Russia9. South-East Europe including Bulgaria, Cyprus, Greece, Israel and Romania10. South-West Europe including Portugal and Spain11. NRENS (National Research and Education Networks)12. United States


Struttura di EGEE

Tre filoni di attività:

• Joint Research Activities:• JRA1 – Middleware Engineering and IntegrationJRA1 – Middleware Engineering and Integration• JRA2 – Quality Assurance • JRA3 – Security• JRA4 - Network Services Development

• Service Activities:• SA1 – Grid Operation and SupportSA1 – Grid Operation and Support• SA2 – Network Resource Provision

• Networking Activities:• NA1 – Management of I3 (Integrated Infrastructure Initiative)• NA2 – Dissemination and Outreach• NA3 – User Training and Induction• NA4 – Application Identification and SupportNA4 – Application Identification and Support• NA5 – Internal Cooperation


EGEE Implementation

• From day 1 (1st April 2004)Production grid service based on the LCG infrastructure running LCG-2 grid middleware (SA)

LCG-2 will be maintained until the new generation has proven itself (fallback solution)

• In parallel develop a “next generation” grid facility (JRA)Produce a new set of grid services according to evolving standards (Web Services)

Run a development service providing early access for evaluation purposes

Will replace LCG-2 on production facility in 2005

Globus 2 based Web services based

EGEE-2EGEE-1LCG-2LCG-1

EDGVDT . . .

LCG

EGEE

. . .AliEn

Dalla pres. di Bob Jones a Cork


EGEE and LCG

EGEE builds on the work of LCG to establish a grid operations service• LCG: a worldwide collaboration of

• The LHC experiments• The Regional Computing Centres• Physics institutes

• Mission:• Prepare and deploy the computing environment that will be used by the

experiments to analyse the LHC data

• Strategy:• Integrate thousands of computers at dozens of participating institutes

worldwide into a global computing resource• Rely on software being developed in advanced grid technology projects, both

in Europe and in the USA

• Status:• LCG service up and running with LCG-2 mware – successfully being used for

LHC data challenges

Dalla pres. di Bob Jones a Cork


JRA1: Middleware Engineering and Integration

L’attività di sviluppo è distribuita su vari cluster:

• UK

• IT/CZ (F. Prelz, M. Sgaravatto)

• CERN

• Nordic

Ogni cluster ha un suo development testbed distribuito

L’integrazione e testing del m/w e lo sviluppo di tools è responsabilità del CERN.

Del cluster italo/ceco fa parte anche Datamat



Punto di partenza per la reingegnerizzazione del m/w: • Design e componenti grid basati su quanto deciso dall’RTAG* ARDA (A

Realization of Distributed Analysis)

• Design Team costituito da S. Fisher (UK), F. Prelz (IT/CZ), D. Groep (Nordic), M. Livny (VDT), P. Buncic (CERN), P. Kunszt (CERN), F. Hemmer (CERN) e E. Laure (CERN)

• Responsabilità del CE (IT/CZ)

• Esperienza e componenti da AliEn, VDT e EDG

• Web Services * Requirements and Technical Assessment Group (LCG)



• Key deliverables/milestones

• Jun’04 Tools for mware engineering and integration deployed (MJRA1.1)

• Jun’04 Architecture and Planning Document - Rel 1 (DJRA1.1)

• Jun’04 SW cluster develop. and testing infrastructure available (MJRA1.2)

• Aug’04 Integration and testing infrastructure in place with test plans – Rel 1 (MJRA1.3)

• Aug’04 Design Document for grid services - Rel 1 (DJRA1.2)


NA4 Work Breakdown Structure

EGEE

NA4NA3

NA3 liaison

Test team

HEPapplications

Biomedicalapplications

Genericapplications

Industryforum

Application specificsoftwares

Gridinterfaces

Test suitesMeetings,

reportsWeb site,

deliverables, internal notes

http://alipc1.ct.infn.it/grid/egee/na4Dalla pres. di R. Barbera a Cork


NA4 Generic Applications goals

Activity NA4 focuses on the identification and support of early-user and established applications for use on the EGEE infrastructure. It has the following objectives:

To identify through the dissemination partners and a well defined integration process a portfolio of early user applications from a broad range of application sectors from academia, industry and commerce.

To support development and production use of all of these applications on the EGEE infrastructure and thereby establish a strong user base on which to build a broad EGEE user community.

To initially focus on two well-defined application areas – Particle Physics and Life sciences.

The expected outcome of the activity will be the establishment of a broad portfolio of applications across a wide range of sectors suited to execution on the EGEE infrastructure meeting the needs of a broad collection of user groups from many sectors across Europe as illustrated in Figure (note the timeline for the introduction of each application domain is purely illustrative).

Dalla pres. di R. Barbera a Cork


Conclusions and outlook for GILDA

• GILDA is a complete suite of grid elements (test-bed, CA, VO, monitoring system, web portal) and applications fully dedicated to dissemination purposes.

• GILDA runs and will run the last production (stable) version of the grid middleware (currently Grid.it 2.0.0 based on LCG 2.0.0).

• GILDA is the dissemination tool which will be used by NA3 during induction courses and tutorials (first in Edinburgh next week!).

• Next future (2-3 weeks):• adding new sites: NeSC, InterQuanta (private company in Spain), any other

interested;• finalize and advertise the “24x7” public “grid demonstrator”: a customized

version of the full GENIUS web portal from where all users belonging to the GILDA VO can submit a pre-defined set of applications to the GILDA Testbed;

• publish all the documentation on the GILDA web site.



The GILDA Tutorials (1/2)

• Edinburgh, 7 April 2004

• Tunis, 22-23 April 2004

• Edinburgh, 26-28 April 2004, registration and program

• CERN, 17-19 May 2004


http://www.nesc.ac.uk/esi/events/411/


SA1: obiettivi

• Core Infrastructure services: • Operate essential grid services

• Grid monitoring and control: • Proactively monitor the operational state and performance, • Initiate corrective action

• Middleware deployment and resource induction: • Validate and deploy middleware releases• Set up operational procedures for new resources

• Resource provider and user support: • Coordinate the resolution of problems from both Resource Centres and users• Filter and aggregate problems, providing or obtaining solutions

• Grid management: • Coordinate Regional Operations Centres (ROC) and Core Infrastructure Centres (CIC) • Manage the relationships with resource providers via service-level agreements.

• International collaboration: • Drive collaboration with peer organisations in the U.S. and in Asia-Pacific • Ensure interoperability of grid infrastructures and services for cross-domain VO’s• Participate in liaison and standards bodies in wider grid community


SA1: strutture (1)

RC (Resource Centres):

• Provide computing and storage resources for the grid

OMC (Operation Management Center):

• Manage the operations of the entire EGEE grid from a single centralised location (CERN)

CIC (Core Infrastructure Center):

• Provide the basic service infrastructure of the Grid

• Operate the key services which connect users with resources

• Support the Regional Operation Centers

Numeri: RC (ad libitum), OMC (1), ROC (9), CIC (4)


SA1: strutture (2)

ROC (Regional Operation Center):• Customize and certificate the middleware releases• Provide the middleware release and up-to-date documentation to a set of

Resource Centers (RC)• Validate the set of grid services and components to form a functioning

Grid to serve user communities• Validate installation/upgrade procedures and documentation• Develop/Run test suites for components and Grid services to certify a

site installation• Refer and escalate middleware problems to developers• Collaborate with CIC to have monitoring systems and performance

evaluator running • Define the tools to measure the service level provided by resource

centers• Manage the access policies and sharing of the resources • Provide 24x7 support service for users and local administrators


SA1: distribuzione geografica delle strutture

10 federazioni:• CERN OMC + CIC

• Francia ROC + CIC

• Italia ROC + CIC

• Germania e Svizzera ROC

• UK e Irlanda ROC + CIC

• Northern Europe ROC

• South West Europe ROC

• South East Europe ROC

• Central Europe ROC

• Russia ROC


SA1 in Italia: da dove partiamo

La grid in italia (InfnGrid) esiste dal 2001:

• E’ diventata una infrastruttura di produzione (Grid-It) nel 2003

• 22 siti operativi (alcuni fanno parte anche dell’infrastruttura LCG2)

• Usata per le attività degli esperimenti (Data Challenge e produzioni):

• CMS preproduction (INFN-GRID_1.0 aka LCG_1.0, winter 2003)• ALICE DC prima fase (febbraio-maggio 2004)• ATLAS DC e produzioni di VIRGO e BABAR in partenza• Prove di produzione di LHCb in corso

• Release: si usa INFN-GRID 2.0.0 (basata su LCG_2.0.0, 100% compatibile)


http://grid-it.cnaf.infn.it


SA1 in Italia

Chi siamo:

• INFN-GRID• Firb (Grid.it) tramite INFN• ENEA• UniLE*• UniNA*• UniCal*

* SPACI (Southern Partnership for Advanced Computational Infrastructures)

Responsabili ROC italiano: C. Vistoli, L. Gaido


SA1-IT: da dove partiamo

The existing italian Grid Operation Center is the first step towards the EGEE ROC + CIC. It will evolve according to the EGEE plans.

• Existing teams:

• Release and Documentation Team - “builds” the INFN-GRID release, starting from the LCG release and (up to now) - provides the LCFGng profiles - provides documentation and a sw repository

• Central Management Team - Takes care of the deployment of the release, certificates the sites and manages the grid services


SA1-IT: da dove partiamo (cont.)

• Existing teams (cont.): • Monitoring Team

GridICE development and support

Provides a customized view for – VO, – Sites, – Grid Operation Center (+ services),– Users

• Support Team

Provides support through a web portal (interim solution: OneorZero

with a ticketing system and a knowledge base system)


SA1: dove andiamo

Per costruire una grid europea (EGEE), risultato dell’interconnessione delle

grid nazionali, le strutture di operation, management e supporto esistenti nelle

differenti regioni/federazioni devono armonizzarsi ed integrarsi evolvendo in

una struttura comune.

Percorso: • definire un modello generale per ROC e CIC• identificare le differenti esigenze dei ROC (e quindi dei Rersouce Center)• Trovare il minimo comune denominatore (dei requirement) per definire

il contenuto della release EGEE (in collaborazione con JRA1)• Organizzarsi perche’ il resto (quello che non farà parte delle release

ufficiale) venga prodotto e supportato da SA1 e/o da uno o più ROC

fondamentale evitare divergenze


SA1: cosa stiamo facendo?

I primi passi:

• Continuiamo a gestire le infrastrutture nazionali esistenti

• Creato gruppo ROC managers:• Riunione telefonica settimanale• Primo meeting ‘de visu’: milano 10-11 maggio• Questionario sulle piattaforme hw e sistemi operativi• Molte altre cose da definire.....

• In via di definizione i ruoli di CIC e ROC

• Definizione Execution Plan

• Definizione dei diversi testbed necessari (vedi figura)

EGEE Certification, Testing and Release Cycle

CERTIFICATIONTESTING SERVICES

Integrate

BasicFunctionality

Tests

Run testsC&T suitesSite suites

RunCertification

Matrix

Releasecandidate

tag

PR

E-P

RO

DU

CT

ION

PR

OD

UC

TIO

N

APPINTEGR

Certifiedrelease

tag

DE

VE

LO

PM

EN

T &

IN

TE

GR

AT

ION

UN

IT &

FU

NC

TIO

NA

L T

ES

TIN

G

DevTag

JRA1

HEPEXPTS

BIO-MED

OTHERTBD

APPSSW

Installation

DE

PL

OY

ME

NT

PR

EP

AR

AT

ION

Deploymentrelease

tag

DEPLOY

SA1

Productiontag


Domande?

Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004 Il progetto EGEE Luciano...

Documents

Transcript of Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004 Il progetto EGEE Luciano...