Imaging Node Meeting Atlas II Status and Plans August 2, 2006
Dec 30, 2015
Imaging Node MeetingAtlas II Status and Plans
Imaging Node MeetingAtlas II Status and Plans
August 2, 2006August 2, 2006
In the Beginning…AtlasIn the Beginning…Atlas
Original Atlas was CGI-BIN based Perl Usually one mission per application
Though code became fairly multi-mission in flavor Copy and tweak
Perl::DBI interface to various databases Decision made to make completely multi-mission and augment
with OODT software Always knew this was necessary, but other priorities usurped
development MER Archive support allowed for developer focus
Original Atlas was CGI-BIN based Perl Usually one mission per application
Though code became fairly multi-mission in flavor Copy and tweak
Perl::DBI interface to various databases Decision made to make completely multi-mission and augment
with OODT software Always knew this was necessary, but other priorities usurped
development MER Archive support allowed for developer focus
Atlas II and OODTAtlas II and OODT Atlas II takes much experience learned from the original
Atlas with the following changes: Split into distinct layers and functions (at various levels of
readiness) Web Service (use of OODT here)
Performs searches across all data sets within the system. System can be distributed. As a WS, this part is intended for machine/code use.
User Interface (aka Forms) The human useable part that utilize the others
Processing Calibration, mosaic, geometry, etc…
Retrieval (use of OODT here) Data download Shopping cart style bulk ordering
Atlas II takes much experience learned from the original Atlas with the following changes: Split into distinct layers and functions (at various levels of
readiness) Web Service (use of OODT here)
Performs searches across all data sets within the system. System can be distributed. As a WS, this part is intended for machine/code use.
User Interface (aka Forms) The human useable part that utilize the others
Processing Calibration, mosaic, geometry, etc…
Retrieval (use of OODT here) Data download Shopping cart style bulk ordering
Atlas II Web Service and OODTAtlas II Web Service and OODT
The Atlas II Web Service performs the meta-data search functions of the Atlas II system. As a distributed system it utilizes (at least) two ways to access meta-data: OODT DIS query via the OODT Profile Server SQL
Both allowed so that we can talk to most repositories in the Imaging Node domain. Some facilities not running OODT, but allow SQL Some facilities don’t allow SQL, but are willing to run OODT
Profile Servers We’re kinda hosed of neither is allowed…
The Atlas II Web Service performs the meta-data search functions of the Atlas II system. As a distributed system it utilizes (at least) two ways to access meta-data: OODT DIS query via the OODT Profile Server SQL
Both allowed so that we can talk to most repositories in the Imaging Node domain. Some facilities not running OODT, but allow SQL Some facilities don’t allow SQL, but are willing to run OODT
Profile Servers We’re kinda hosed of neither is allowed…
Atlas II Web Service and OODT (2)Atlas II Web Service and OODT (2)
Imaging node currently interacts with several database and schema configurations: MySQL (IN, UofA, probably everybody else
on the planet ;-)) Sybase (e.g. MIPL, Geo) Oracle (e.g. USGS)
Imaging node currently interacts with several database and schema configurations: MySQL (IN, UofA, probably everybody else
on the planet ;-)) Sybase (e.g. MIPL, Geo) Oracle (e.g. USGS)
Atlas II Web Service and OODT (3)Atlas II Web Service and OODT (3)
Why not just use OODT Profile searches directly? OODT searches syntax, not semantics
Is longitude East positive? Is projection planetocentric? Planetodetic? What is the equivalence of the Green filter on VGR to the
wavelength of filter 3 on GLL? What is the SCET of FDS 453245?
These are the kinds of things that Atlas II worries about. Once the user-perspective query is translated, it is shipped off the the various OODT Profile servers configured for use.
Why not just use OODT Profile searches directly? OODT searches syntax, not semantics
Is longitude East positive? Is projection planetocentric? Planetodetic? What is the equivalence of the Green filter on VGR to the
wavelength of filter 3 on GLL? What is the SCET of FDS 453245?
These are the kinds of things that Atlas II worries about. Once the user-perspective query is translated, it is shipped off the the various OODT Profile servers configured for use.
OODT Profile Server configurationOODT Profile Server configuration
So, how do you know how to query a meta-data collection? Especially one that is not local? Profile Servers have an XML configuration file that Atlas II uses (and
augments with its own spicings) Mapping of schema Searchable parameters (make sure only non-pathological items are
queried). Return-ables parameters (default, required, possible)
Atlas II knows how to use these configurations to do many things within the system: Query construction (OODT DIS or SQL) User interface creation
So, how do you know how to query a meta-data collection? Especially one that is not local? Profile Servers have an XML configuration file that Atlas II uses (and
augments with its own spicings) Mapping of schema Searchable parameters (make sure only non-pathological items are
queried). Return-ables parameters (default, required, possible)
Atlas II knows how to use these configurations to do many things within the system: Query construction (OODT DIS or SQL) User interface creation
Atlas II Retrieval and OODTAtlas II Retrieval and OODT
Downloading of data via (at least) two mechanisms: OODT Product server
Understands that a set of data may be required to fulfill order for an “EDR”. (E.g., data files and labels bundled together into a ZIP archive).
‘wget’ style URL scripting straight to a web server (Atlas II to generate script for client)
In all cases, URL accesses the data and we intend for them to be immutable.
Downloading of data via (at least) two mechanisms: OODT Product server
Understands that a set of data may be required to fulfill order for an “EDR”. (E.g., data files and labels bundled together into a ZIP archive).
‘wget’ style URL scripting straight to a web server (Atlas II to generate script for client)
In all cases, URL accesses the data and we intend for them to be immutable.
Atlas II StatusAtlas II Status
Currently we are supporting both Cassini and MER operationally and have a development version for HiRISE.
Atlas II support both mission specific and multi-mission queries.
Mission specific queries: http://pdsimg.jpl.nasa.gov/search/index.jsp?mission=cassini http://pdsimg.jpl.nasa.gov/search/index.jsp?mission=mer http://atlasweb-dev.jpl.nasa.gov/search/index.jsp?mission=mro (This is
password protected)
Currently we are supporting both Cassini and MER operationally and have a development version for HiRISE.
Atlas II support both mission specific and multi-mission queries.
Mission specific queries: http://pdsimg.jpl.nasa.gov/search/index.jsp?mission=cassini http://pdsimg.jpl.nasa.gov/search/index.jsp?mission=mer http://atlasweb-dev.jpl.nasa.gov/search/index.jsp?mission=mro (This is
password protected)
Atlas II Status (cont.)Atlas II Status (cont.)
Atlas II Status (cont.)Atlas II Status (cont.)
Atlas II Status (cont.)Atlas II Status (cont.)
Multi-mission query: The same code base is used for the multi-
mission view Common parameters of each logical grouping
(data set) is dynamically found and displayed and searched across missions.
http://pdsimg.jpl.nasa.gov/search/index.jsp?MULTI_MISSION=true
Multi-mission query: The same code base is used for the multi-
mission view Common parameters of each logical grouping
(data set) is dynamically found and displayed and searched across missions.
http://pdsimg.jpl.nasa.gov/search/index.jsp?MULTI_MISSION=true
Atlas II Status (cont.)Atlas II Status (cont.)
Atlas II ConfigurationAtlas II Configuration Profile Server and/or SQL (JDBC) access to
catalog We could mirror the meta-data here, but why?
UofA dedicated to uptime preservation.
XML Profile server configuration to be shared with IN. HiRISE drives this. This encapsulates the schema so we know the mapping of the various fields to the PDS keywords.
Profile Server and/or SQL (JDBC) access to catalog
We could mirror the meta-data here, but why? UofA dedicated to uptime preservation.
XML Profile server configuration to be shared with IN. HiRISE drives this. This encapsulates the schema so we know the mapping of the various fields to the PDS keywords.
Atlas II and OODT OverviewAtlas II and OODT Overview
Data Migration PlanData Migration Plan What is required?
Sybase to MySQL to increase performance Schema optimization Configuration file to integrate with Atlas II and OODT
Order of data migration from Atlas I web-interface to Atlas II:
THEMIS, Galileo, MPF, Voyager, Viking Orbiter, MGS, ODY (GRS), MEX, Clementine, Magellan
What is required? Sybase to MySQL to increase performance Schema optimization Configuration file to integrate with Atlas II and OODT
Order of data migration from Atlas I web-interface to Atlas II:
THEMIS, Galileo, MPF, Voyager, Viking Orbiter, MGS, ODY (GRS), MEX, Clementine, Magellan
Additional FeaturesAdditional Features HiRISE interface to JPEG2000 server
Necessary to support RDR retrieval of large files.
External processing interface. This is the ability to remotely process images, i.e. stretching, multi-format downloads for reasonably sized images.
Plan to incorporate image processing via MIPL/JAI software.
HiRISE interface to JPEG2000 server Necessary to support RDR retrieval of large
files. External processing interface. This is the ability
to remotely process images, i.e. stretching, multi-format downloads for reasonably sized images.
Plan to incorporate image processing via MIPL/JAI software.