Jan 03, 2016
Agenda
•Grid portals•PROGRESS HPC Portal•Specialized application portlets
•Conclusions
Grid portals: today
• Target: access to distributed grid services from multiple portals
• TACC GridPort 3.0• NCSA Alliance Portal• PROGRESS HPC Portal
PROGRESS
Grid Service Provider
• Job Submission Service– computing job building, submitting jobs to the grid for
execution and monitoring the execution– grid job descriptions in the XRSL language are transferred to
the GRB for the execution of the job– possibility to prepare workflows
• Application Management Service– manages the PROGRESS application repository– an application descriptor contains a reference to the
application’s executable and a set of its available, required or optional arguments, required environment variables as well as input and output files
– one executable may be referenced by many applications– different application configurations are recognized as
independent applications
Data Management System
• A distributed system enabling the management of grid data files
• Storing files in distributed storage modules of various types: generic filesystems, archivers, relational databases
• Uses metadata to describe files• Allows access to data banks like a mirror of
Sequence Retrieval System• Exposes its functionality within the Data
Broker Service
PROGRESS HPC Portal
Core portlets
• “My computing jobs”:– creation, configuration and execution of grid jobs on the grid– downloading and visualizing the results– utilizes the functionality of the GSP Job Submission and
Application Management services and the DMS Data Broker. • “Applications”:
– management of grid-enabled applications within the GSP application repository
– co-operates with the GSP Application Management Service and the DMS Data Broker.
• “My data”:– management of the files stored within the DMS– uploading and downloading files from the DMS– communicates with the DMS Data Broker.
“My computing jobs” portlet (1)
“My computing jobs” portlet (2)
“My computing jobs” portlet (3)
“My computing jobs” portlet (4)
“My computing jobs” portlet (5)
“My computing jobs” portlet (6)
“My computing jobs” portlet (7)
“My computing jobs” portlet (8)
“My computing jobs” portlet (9)
“My computing jobs” portlet (10)
“My computing jobs” portlet (11)
• Exposes the whole functionality of the Job Submission Service
• Enables submission of any job utilizing any available application
• Great for experienced users allowing them to fulfill their requirements
• Too complicated for use by non-advanced users
• Solution: specialized application portlets utilizing the same services as core portlets and familiar with the corresponding grid-enabled application structure
“Gaussian” portlet (1)
“Gaussian” portlet (2)
“Gaussian” portlet (3)
“Gaussian” portlet (4)
“DNA Assembly” portlet (1)
“DNA Assembly” portlet (2)
“DNA Assembly” portlet: Visualization
Application portlets
• Provide users with interfaces that are easy to use and require no grid knowledge
• Grid jobs may be configured very quickly with an intuitive wizard
• Easy and quick access to results: application portlets are familiar with corresponding applications
• Problem: How to quickly build such specialized portlets?
PROGRESS Portlet Framework
• The Framework allows preparing a high-level interface to Web Services with special beans to store the data obtained from WS responses and classes used to translate HTTP requests into the invocation of proper WS methods.
• This allows to quickly build new portlets on top of those classes to utilize the already existing functionality that they provide.
Provider ...Provider Provider ProviderProvider
HTTP
Content Generator
Content Generator
Content Generator
Content Generator
Content Generator
Request Handler
Request Handler
Request Handler
Request Handler
Request Handler
Request Handler...
Web Service Proxy ...Web Service
ProxyWeb Service
ProxyWeb Service
ProxyWeb Service
Proxy
...
Web Services Invocation Framework
SOAP (to GSP, DMS)
Portlets
Communication with Web Services (1)
• Web Services Invocation Framework is used for communication with Web ServicesObject executeMethod(
String methodName, ArgumentTable methodArguments)
• AXIS toolkit is used to generate the binding classes for complex class types mapping
• Each Web Service is described in an XML configuration file
<?xml version="1.0" encoding="UTF-8"?><web-services> <web-service id="appmgmt"> <name>Application Management</name> <wsdl-location>ApplicationManagementWS.wsdl</wsdl-location> <port-type-name>
http://viola.man.poznan.pl:5021/axis/services/ApplicationManagement </port-type-name>
<port-type-namespace>ApplicationCtrl</port-type-namespace> <description>GSP Application Management Service</description> </web-service> <web-service id="jobs"> <name>Job Submission</name> <wsdl-location>JobSubmission.wsdl</wsdl-location> <port-type-name>
http://viola.man.poznan.pl:5021/axis/services/JobSubmission </port-type-name>
<port-type-namespace>JobSubmissionCtrl</port-type-namespace> <description>GSP Job Submission Service</description> </web-service> <web-services>
Communication with Web Services (2)
Content generation (1)
• The content is generated based on two HTTP request parameters (page and action). 1. page e.g. ApplicationList, JobList2. action e.g. removeApplication, submitJob
• Actions are defined in actions.xml, pages in pages.xml (the content of those files is read by appropriate classes, ContentGenerator reads Action and Page objects from those classes). Actions are executed before page content is generated; pages usually contain default actions to generate their content.
• Actions are usually page independant, they can also replace the default page action
• An action is associated with one or more RequestHandler methods,or with methods of many RequestHandlers
• The Java reflection mechanism is used to call RequestHandler methods
Content generation (2)
• Page content is delivered by RequestHandlers in form of PageBean classes
• Each page can contain one or more classes implementing the PageBean interface (PageBean classes know how to deliver their content in form of a DOM)
• The page generates its content by converting its PageBeans into a DOM and then transforming it using an XSLT template
Page
PageBean1
RequestHandler1
RequestHandler2
RequestHandler3
PageBean2
PageBean3
PageBean4
DOM1
DOM2
DOM3
DOM4
DOM
Content Generator
HTML
Content generation (3)
• Pages are defined in an XML config file:
<pages>
<page name="ApplicationList">
<display-name>Application List</display-name>
<description>List of PROGRESS applications</description>
<xsl-file>application-list.xsl</xsl-file>
<action name="ApplicationList">
<request-handlers>
<request-handler class="ApplicationGetRequestHandler">
<method name="getApplicationList"/>
</request-handler>
</request-handlers>
</action>
</page>
...
</pages>
Content generation (4)
• Invocation of many RequestHandlers
<page name="EditApplicationExecutable">
<display-name>Edit Application Executable</display-name>
<description>
Edit application executable file - choose/change the file.
The executable can also be uploaded to SZD.
</description>
<xsl-file>edit-application-executable.xsl</xsl-file>
<action name="EditApplicationExecutable">
<request-handlers>
<request-handler class="ApplicationGetRequestHandler">
<method name="getApplication"/>
</request-handler>
<request-handler class="SzdGetRequestHandler">
<method name="getUserDirectoryListing"/>
</request-handler>
</request-handlers>
</action>
</page>
Content generation (5)
• RequestHandler ”workflow” <page name="JobList">
<display-name>Job List</display-name>
<description>List of jobs for a given user</description>
<xsl-file>job-list.xsl</xsl-file>
<scripts>
<script language="JavaScript" src="jscript/job-submission/job_list.js" />
</scripts>
<action name="JobList">
<request-handlers>
<request-handler class="JobGetRequestHandler">
<method name="getJobsOfUser" output-name="userJobs"/>
<method name="filterJobs"
input-name="userJobs" output-name="filteredJobs"/>
<method name="sortJobs" input-name="filteredJobs"/>
</request-handler>
</request-handlers>
</action>
</page>
Content generation (6)
• Page error redirection – defined in an error-redirections.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<redirections><error-redirection-group id="EditJob:saveJob">
<error-redirection type="validation" redirect-to-page="EditJob" />
</error-redirection-group>
<error-redirection-group id="EditTaskInfo:saveTaskInfo">
<error-redirection type="validation" redirect-to-page="EditTaskInfo" />
</error-redirection-group>
</redirections>
Framework advantages
• It’s easy to reuse the portlets in different portal frameworks and on various portals and websites– The top-level Provider class can be easily adjusted to
implement any portal framework API– The content generated by a portlet can be easily
changed by playing with configurations files
• The same RequestHandlers can be used by mutliple portlets thus enabling easy creation of new portlets communicating with the same WS services– specialized application portlets, e.g.: “Gaussian” portlet,
“DNA Assembly” portlet
Conclusions
• Our experience shows it’s important to provide users with interfaces to grid-enabled applications that are easy to use and do not require deep knowledge about the grid and/or the application.
• It’s now easy to create such interfaces having powerful grid services on hand and tools like PROGRESS Portlet Framework that facilitate building these interfaces.