New EMI Requirements and Work Plans
Taken from an email from Andres Aeschlimann (to
operations@swing-grid.ch, Feb 7th 14:51), for the EMI Roadmap there is
https://wiki.egi.eu/wiki/EGI_Operations#EMI
Here is the EMI 1 release plan:
https://twiki.cern.ch/twiki/bin/view/EMI/InternalDeliverableEmi1ReleaseDevPlans
Release schedule of different components:
https://savannah.cern.ch/task/?group=emi-rel-sched&func=browse&set=open&msort=0&advsrch=0&morder=bug_id%3C&order=bug_id#results
EMI Services documentation:
https://twiki.cern.ch/twiki/bin/view/EMI/EMIServicesDocs
Compute Area Work Plan
ARC, gLite CREAM (with CEMon and BLAH) and UNICORE provide the same core functionality; they are implemented in different and often incompatible ways. They want to address some of these issues. In particular the plan is:
- To define and adopt job execution standard/agreed interfaces, with all three representatives, work in progress.
- To adopt information service related standards. GLUE v. 2.0.
- To harmonize the used authorization mechanisms. gLite Argus service.
- To harmonize the parallel and MPI job support. Provision of a standard EMI unified layer
- To consolidate and harmonize the compute area clients and APIs. The main objectives are to improve: usability, maintainability and portability to different architectures and operating systems.
- To assess the use of a messaging system between the EMI job management components. It's not clear if it's needed, but it's under investigation (maybe messages between WMS and Cream?)
- To remove GSI, and replace it with standard SSL/TLS. An overall strategy of the EMI project is to remove the Globus proprietary GSI protocol in the EMI components and replace it with standard SSL/TLS readily available in target operating systems
The document further describes how to accomplish these targets for each CE model, please take a look at it for further details.
Evolution plans:
- ARC-CE: Dynamic management of RTEs through the environment Janitor, accounting hooks, and Data Staging components need a serious evolution (this is a pure research and development activity)
- ARC clients: Extending the resource discovery module and enhancing the job submission interface, language binding on non-linux platforms, enhancement and extension to support the wide area of storage protocols and information systems, and support job management for non Grid resources (local, for testing)
- gLite Job Management services: Prevent jobs starting many hours after submission (WMS should be able to migrate jobs within different CEs, with a timeout), better support for MPI jobs, see below.
- gLite MPI: Better control of user processes (how to map the processes to physical resources), better hooking management in MPI-Start, enhancement of scheduler module (better support for different LRMSs).
- Unicore: more flexible generation of site-specific scripts.
Data Area Work Plan
These documents are to be published as of final versions by the middle of 2011. Migration of components begin afterwards, more or less within one year. For precise references and plans please check the official document.
There is a Harmonization process that consists of:
- Catalogue syncronization, within SEs and LFCs, using a new, under development, message-passing mechanism.
- Consolidation of Storage Resource Manager (SRM) protocol, EMI data will clarify the SRM v2.2 specification by adding more user-friendly documentation
- Replacing the Globus httpg security protocol with the SSL/X509 (https) standard. They want to move some components to SSL/X509 away from GSI. the outcome of the latter goal is not clear, but EMI will investigate solutions to guarantee interoperability
- Providing standard access to data through a mounted file system (NFS4.1)
- Providing standard access to data via http(s) and WebDAV. May not affect us.
- Publishing GLUE 2.0 information
- Integration of the ARGUS EMI authorization system. All components agreed to access the ARGUS authorization at least for obtaining user blacklisting, Some SE's can go even further.
- Consolidating data access client libraries. Together with ARC. This includes but is not limited to storage control (SRM), storage access (e.g. gsiFTP and http) and information protocols.
Evolution Plans:
- Monitoring and accounting. Consolidate activities and offer mechanisms to better monitor user access and user accounting. Also proper definition of what accounting would mean for data is still missing.
- Maintainability and Usability. Most of the services are planning to provide or improve Web interfaces or Command Line interfaces, work done by themselves.
- Evolution in wide area protocols. Gridftp v2 under development in Globus, and available in dCache. NFSv4.1 not considered.
Infrastructure Area Work Plan
TO BE DONE.
3. Status 9
3.1. Information Services 9
3.1.1 GLUE Information Model 9
3.1.1.1 Information Validation 9
3.1.2 gLite Information System 9
3.1.2.1 BDII Service (resource-level) 10
3.1.2.2 BDII Service (site-level) 10
3.1.2.3 BDII Service (top-level) 10
3.1.2.4 Client Side Tools 10
3.1.3 ARC LDAP-based Infosys 10
3.1.3.1 Classic Infoserver 10
3.1.3.2 Classic Infoindex (EGIIS) 11
3.1.4 UNICORE Registry 11
3.1.5 Common Information Provider (CIP) 11
3.2. Logging and Bookkeeping 11
3.3. Accounting 12
3.3.1 APEL Client 12
3.3.2 DGAS Client 12
3.4. Messaging 12
3.5. Service Monitoring and Management 12
3.6. Virtualization and clouds 13
4. Plans 14
4.1. Information Services 14
4.1.1 gLite Information System Development 14
4.1.2 ARC LDAP-based Infosys Development 14
4.1.3 Common Information Provider Development 15
4.1.4 EMI Registry 15
4.2. Logging & Bookkeeping Development 15
4.3. Accounting 16
4.3.1 APEL Client Development 16
4.3.2 DGAS Client 17
4.4. Messaging Development 18
4.5. Service Monitoring and Management 19
4.6. Virtualization and clouds 19
Security Area Work Plan
TO BE DONE.
4.1. HARMONIZATION....................................................................................................................................11
4.1.1 ARC Security Utils Product Team .............................................................................................11
4.1.2 Argus Product Team ..................................................................................................................11
4.1.3 VOMS Product Team .................................................................................................................12
4.1.4 gLite Security Product Team.....................................................................................................13
4.1.5 UNICORE Security Product Team ............................................................................................14
4.1.6 CESNET Security Product Team ...............................................................................................14
4.1.7 GSI Removal ..............................................................................................................................15
4.1.8 Common Authentication Libraries.............................................................................................15
4.1.9 Common SAML Profile ..............................................................................................................16
4.1.10 Compute Area Authorization .....................................................................................................16
4.2. EVOLUTION............................................................................................................................................17
4.2.1 ARC Security Utils Product Team .............................................................................................17
4.2.2 Argus Product Team ..................................................................................................................17
4.2.3 VOMS Product Team .................................................................................................................18
4.2.4 gLite Security Product Team .....................................................................................................18
4.2.5 UNICORE Security Product Team ............................................................................................18
4.2.6 CESNET Security Product Team ...............................................................................................19
--
PabloFernandez - 2011-02-08