1 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus GRACE General Meeting, September 2004, Brussels D6.1 Integration with the European DataGrid Infrastructure Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus (CERN) GRACE General Project Meeting Brussels, 20-21 September 2004
15
Embed
CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus GRACE General Meeting, September 2004, Brussels 1 D6.1 Integration with the European DataGrid.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
D6.1Integration with the European DataGrid Infrastructure
Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus (CERN)
GRACE General Project MeetingBrussels, 20-21 September 2004
2 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
10 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
D6.1 Infrastructure
1. Integration with the Grid (GILDA)2. Sites testing (certification)3. Conclusions
A big effort was necessary for setting up the infrastructure mainly due to the transition from EDG to EGEE and different middleware releases. With the effective support of EGEE NA4 team the set up and integration of GRACE grid nodes was completed successfully.
11 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
12 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
D6.1 Toolkit
1. Integration with the Grid (GRACE Grid jobs)2. Testing (integration tests and performance
tests)3. Conclusions
GRACE Grid components have been integrated with EGEE mw, “easy-deployment” solution has been adopted, integration tests performed by testval group, performance considerations
13 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
14 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
D6.1 Content Sources
1. Definition of the CSs of interest for GRACE 2. CSs configuration (see list)3. CSs configuration tests (automatic scripts)4. Conclusions
Risk of not having enough CS for Computing and Engineering KDs (subscription mechanism not implemented), it has been decided to concentrate on General and Physics KDs
15 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
D6.1 Missing parts
1. Final performance tests (started last Friday) and considerations
16 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
GRACE Grid Jobs Testing in GILDA
• Tests in GILDA testbed (LCG 2.0/2.2)• Sandra Application: Includes Text
Normalizing and Categorizing• One job per search• Application submitted in the InputSandbox
17 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
GRACE Grid Jobs Performances (un-optimized)
0
2,5
5
7,5
10
12,5
15
17,5
20
22,5
25
0 500 1000 1500 2000 2500 3000
Input File Size in KB
Exe
cutio
n T
ime
in H
ours
(with
out G
rid o
verh
ead)
18 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
GRACE Grid Jobs Performance (optimized)
0
2,5
5
7,5
10
12,5
15
17,5
20
22,5
25
0 500 1000 1500 2000 2500 3000
Input File Size in KB
Exe
cutio
n T
ime
in H
ours
(with
out G
rid o
verh
ead)
19 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
GRACE Grid Jobs Performances (details)
0
1
2
3
4
5
6
7
8
9
10
0 500 1000 1500 2000 2500 3000
Input File Size in KB
Exe
cutio
n T
ime
in H
ours
(with
out G
rid o
verh
ead)
Text Normalizing
Categorizing
20 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus
GRACE General Meeting, September 2004, Brussels
GRACE Grid Job Outlook
• Typical search result still needs O(hours) of processing• Possible Solution: Job splitting• Distribute Text Normalizing across many jobs• Merge Results for Categorization• Use of Job dependencies model (e.g. DAGMan)
• Tests in LCG 2.2 with optimized Sandra application ongoing (started on 17 Sep)
• First test in new EGEE middleware gLite successful