Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 1
Lezione 7a - 9 Dicembre 2009
Il materiale didattico usato in questo corso è stato mutuato da quello utilizzato da Paolo Veronesi per il corso di Griglie Computazionali per la Laurea Specialistica in Informatica tenuto nell’anno accademico 2008/09 presso l’Università degli Studi di Ferrara.
Paolo [email protected], [email protected]://www.cnaf.infn.it/~pveronesi/unife/
Università degli Studi di Bari – Corso di Laurea Specialistica in Informatica
“Tecnologia dei Servizi “Grid e cloud computing” A.A. 2009/2010
Giorgio Pietro Maggi [email protected], http://www.ba.infn.it/~maggi
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 2
Referenze
gLite doc http://glite.web.cern.ch/glite/documentation/default.asp
gLite userGuide https://edms.cern.ch/file/722398//gLite-3-UserGuide.pdf
VOMS Guide https://edms.cern.ch/file/973684/1/voms-guide.pdf
VOMS Admin User Guide https://edms.cern.ch/file/974094/1/voms-admin-user-guide.pdf
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 3
Make a proxy
Inspecting personal certificate (grid-cert-info). Creation of a proxy without voms extensions and check
your proxy (grid-proxy-init) Creation of a proxy with voms extensions and check
your proxy (voms-proxy-init)
Use <command> -help to check how to use them
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 4
grid-cert-info Ispeziona il certificato pubblico in $HOME/.globus Nessuna communicazione di rete Stesso risultato che si ottiene ispezionando con I
comandi openssl o verificando il certificato nel sito della Certification Authority
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 5
grid-proxy-init Crea un proxy senza estensioni voms (password richiesta perchè si
usa la chiave privata) Durata predefinita: 12 ore Si possono creare proxy di durata superiore alla validità del certificato! Verifica del proxy: grid-proxy-info –all Posso solo autenticarmi, le autorizzazioni sono basate sulle
estensioni VOMS
$ grid-proxy-info -allsubject : /C=IT/O=GILDA/OU=Personal Certificate/L=unife/CN=Paolo Veronesi/CN=1777588616issuer : /C=IT/O=GILDA/OU=Personal Certificate/L=unife/CN=Paolo Veronesiidentity : /C=IT/O=GILDA/OU=Personal Certificate/L=unife/CN=Paolo Veronesitype : Proxy draft (pre-RFC) compliant impersonation proxystrength : 512 bitspath : /tmp/x509up_u11397timeleft : 1513:49:55 (63.0 days)
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 6
voms-proxy-init Equivalente a grid-proxy-init se non si specifica la VO; Viene contattato il voms server per recuperare le estensioni;
Verifica con voms-proxy-info –all (da notare i due campi timeleft distinti, cosa succede quando uno dei due scade?)
$ voms-proxy-info -allsubject : /C=IT/O=GILDA/OU=Personal Certificate/L=unife/CN=Paolo Veronesi/CN=proxyissuer : /C=IT/O=GILDA/OU=Personal Certificate/L=unife/CN=Paolo Veronesiidentity : /C=IT/O=GILDA/OU=Personal Certificate/L=unife/CN=Paolo Veronesitype : proxystrength : 512 bitspath : /tmp/x509up_u11397timeleft : 11:58:17=== VO gilda extension information ===VO : gildasubject : /C=IT/O=GILDA/OU=Personal Certificate/L=unife/CN=Paolo Veronesiissuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.itattribute : /gilda/Role=NULL/Capability=NULLtimeleft : 11:58:17
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 7
Requesting Voms Group/Role in ProxyOne of the main features of VOMS is its capability to create groups and roles which
allows VO administrator to differentiate users' privileges and right. Users, if already belonging to a group, or already assigned to a Role, can apply the request while creating the proxy with voms-proxy-init command. In this way, the information will be signed by the VOMS server and inserted in the proxy AC; resources will be able to parse them assigning to the user the expected rights.
Syntax The group/role request is done by users appending a request command to the --voms option of
voms-proxy-init voms-proxy-init --voms YourVO:/YourVO/Desired-Group in case of group request; voms-proxy-init --voms YourVO:/YourVO/Role=Desired-Role in case of role request; voms-proxy-init --voms YourVO:/YourVO/Desired-Group/Role=Desired-Role in case of
mixed (group + role) request.
Let's make it plain by some examples
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 8
Group request Suppose you want to create a voms proxy for the gilda VO, requesting the
membership of generic-users group. Then you have to just to run voms-proxy-init --voms gilda:/gilda/generic-users
Of course this wouldn't work if you don't belong to the gilda VO or either you don't belong to the generic-users group. You can verify that the command has run successfully with voms-proxy-info command :
=== VO gilda extension information === VO : gilda subject : /C=IT/O=GILDA/OU=Personal
Certificate/L=CATANIA/CN=CATANIA49/[email protected] issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it attribute : /gilda/generic-users/Role=NULL/Capability=NULL attribute : /gilda/Role=NULL/Capability=NULL timeleft : 11:59:37
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 9
Role request If you belong to gilda VO and want to get the Role GenericRole within your proxy,
you have to run just voms-proxy-init --voms gilda:/gilda/Role=GenericRole
You can verify then with voms-proxy-info –all=== VO gilda extension information === VO : gilda subject : /C=IT/O=GILDA/OU=Personal
Certificate/L=CATANIA/CN=CATANIA49/[email protected] issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it attribute : /gilda/Role=GenericRole/Capability=NULL attribute : /gilda/Role=NULL/Capability=NULL attribute : /gilda/generic-users/Role=NULL/Capability=NULL timeleft : 11:57:56
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 10
Group + Role (1/2) You may have noticed that both when requesting a Role or a VO
membership, the first part of the command you append starts always with /gilda : that's because the group with the VO name is the default group where all the VO members belong; even if it's a default, it is to be always specified.
As a consequence, the request :/gilda/Role=GenericRole and /gilda/generic-users/Role=GenericRole are different, as you can verify by executing them and confronting the first
attribute inserted in the created voms proxies. In the former you are requesting the Role within the default group, while in
the latter you're requesting the Role within the group generic-users. By the way, you will notice that the syntax is /group/subgroup1/..../Role
Of course, to be working, the request has to be consistent with the privileges that the VO-Admin has given you.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 11
Group + Role (2/2) Here an example of a Role requested within a subgroup : you can compare
the output of a successive voms-proxy-info and see the differences with the one obtained in the paragraph before.
$voms-proxy-init --voms gilda:/gilda/generic-users/Role=GenericRole
[…]=== VO gilda extension information === VO : gilda subject : /C=IT/O=GILDA/OU=Personal
Certificate/L=CATANIA/CN=CATANIA49/[email protected] issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it attribute : /gilda/generic-users/Role=GenericRole/Capability=NULL attribute : /gilda/Role=NULL/Capability=NULL attribute : /gilda/generic-users/Role=NULL/Capability=NULL timeleft : 11:59:36
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 12
Credential StorageMyProxy
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 13
What is MyProxy?
An Online Certificate Authority Issues short-lived X.509 End Entity Certificates Avoid need for long-lived user keys
An Online Credential Repository Issues short-lived X.509 Proxy Certificates Long-lived private keys never leave the server
Supporting multiple authentication methods
Open Source Software
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 14
MyProxy Logon
Authenticate to retrieve PKI credentials End Entity or Proxy Certificate Trusted CA Certificates Certificate Revocation Lists (CRLs)
MyProxy maintains the user’s PKI context Users don’t need to manage long-lived credentials Enables server-side monitoring and policy enforcement (ex. passphrase
quality checks) CA certificates & CRLs updated automatically at login
MyProxy integrates with existing authentication systems Providing a gateway to grid authentication
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 15
Scenario 1: Users already have PKI credentials MyProxy repository can help users manage the
credentials by: Securing private keys in a professionally managed server Obtaining credentials when/where needed Using credentials with MyProxy-enabled applications
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 16
Scenario 2: Users have site logons but no PKI credentials Users have site logons but no PKI credentials
MyProxy CA can provide the bridge
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 17
Scenario 3: Users need to register to obtain PKI credentials
User registration portals provide a MyProxy interface
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 18
Scenario 4: Users need run Grid jobs longer than the typical proxy life A myproxy server is used to create and store a long
term proxy which is used to renew short term proxies when they are going to expire
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 19
MyProxy Repository Policies
Who can store credentials? Restrict to specific users or CAs Restrict to administrator only
Who can retrieve credentials? Allow anyone with correct password Allow only trusted services / portals
Maximum lifetime of retrieved credentials
server-wide and per-
credential
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 20
MyProxy in EGEE
EGEE security based on proxy certificates often carrying VOMS attribute certificates
MyProxy used for several purposes: Solution for portals (P-GRADE, Genius)
a common way of using MyProxy Long-running jobs and data transfers
credential renewal t-Infrastructure CA
formalized on-line CA based on MyProxy
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 21
Long-running Jobs Jobs require valid credentials
e.g. to access GridFTP data repositories on the user‘s behalf these operations must be secured, using the users‘ credentials
Job's lifetime can easily exceed the lifetime of a proxy consider waiting in the queues, possible resubmissions, computation
time, data transfers, etc. also VOMS certificates have limited lifetime
Impossible to submit a job with sufficiently long credentials the overall job lifetime not known in advance violation of the meaning of short-time proxies increased risk when the credential is stolen might be unacceptable for the end resources
How to provide jobs with a valid short-lived credential throughout their run?
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 22
Proxy Renewal Service Periodical renewal of credentials
maintains a list of jobs' proxy certificates to be kept valid using MyProxy repository
server specified by user in the job description uses the renewal mode authenticates using the WMS credential AND authorizes using the
proxy being renewed Support for renewal of VOMS attributes
Part of the broker node (WMS) A proxy of a job is registered upon submission It is renewed whenever it is going to expire
several attempts done until renewal succeeds After renewal a new proxy is pushed to the computing resource,
where the job is running After the job completion the proxy is unregistered
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 23
Proxy Renewal Service
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 24
Proxy Renewal Service Ensures that jobs always have a valid short-time proxy Users have full control over their proxies and renewal
Using the MyProxy repository Support for VOMS All operations are logged
allows an audit Stolen credentials can't be renewed easily
the WMS credential are necessary for renewal An older (still valid) proxy must be available for renewal
reduces the risk when services are compromised Developed in EU Datagrid, in production use in EGEE
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 25
Long-term Data Transfers
EGEE applications often need to move large amount of data
The File Transfer Service (FTS) is used to handle such file movement requests
Similar problem as in the case of jobs the transfer can last long time, can be rescheduled etc.
FTS currently uses a password based retrieval from MyProxy
Support for renewal is currently being added based on routines from the renewal service
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 26
Credential Delegation through MyProxy
MyProxyUse Register a long living proxy in the MyProxy server
grid001.ct.infn.it (myproxy-init) Gather information about the proxy in the MyProxy
server (myproxy-info) Get a delegated proxy from the Myproxy server
(myproxy-get-delegation) Destroy remote proxy (myproxy-get-destroy)
Use <command> -help to check how to use them
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 27
myproxy-init Crea archivia un proxy senza estensioni voms sul server myproxy (il proxy è protetto da password diversa da quella della chiave privata); Durata di default: 7 giorni Verifico con myproxy-info
$ myproxy-initYour identity: /C=IT/O=GILDA/OU=Personal Certificate/L=unife/CN=Paolo VeronesiEnter GRID pass phrase for this identity:Creating proxy .................................................... DoneProxy Verify OKYour proxy is valid until: Tue May 12 14:03:12 2009Enter MyProxy pass phrase:Verifying - Enter MyProxy pass phrase:A proxy valid for 168 hours (7.0 days) for user veronesi now exists on myproxy.ct.infn.it.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 007a 28
myproxy-get-delegation Recupero un proxy precedentemente salvato su un server myproxy voms-proxy-init -noregen --voms gilda -cert /tmp/x509up_u11397 Aggiungo le estensioni voms ad un proxy senza rigenerare il proxy stesso => rinnovo delle
estensioni VOMS