ISO/IEC JTC 1/SC 29/WG1N5018 Date: 2009-05-20 ISO/IEC JTC 1/SC 29/WG 1 (ITU-T SG16) Coding of Still Pictures JBIG JPEG Joint Bi-level Image Joint Photographic Experts Group Experts Group TITLE: CD of JPSearch Part 2 (24800-2) Registration, Identification, and Management of Metadata Schema and Ontology SOURCE: JPSearch Part-2 Editors PROJECT: JPSearch STATUS: CD REQUEST ACTIONS: For ballot DISTRIBUTION: WG1 Distribution List CONTACT: ISO/IEC JTC 1/SC 29/WG 1 Convener - Dr. Daniel T. Lee eBay Inc., 2145 Hamilton Avenue, San Jose, California 95125,USA eBay China Development Center, Shanghai German Center, 88 Keyuan Road, Zhangjiang Hi-Tech Park, Pudong, Shanghai 201203, China Document type: Document subtype: Document stage: Document language:
40
Embed
ISO/IEC JTC 1/SC 29 N - itscj.ipsj.or.jp€¦ · Web viewISO/IEC JTC 1/SC 29/WG1N5018 Date: ... the first letter of each word is capitalized, ... Input Wraps the user register request
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ISO/IEC JTC 1/SC 29/WG1N5018 Date: 2009-05-20
ISO/IEC JTC 1/SC 29/WG 1
(ITU-T SG16)
Coding of Still PicturesJBIG JPEG
Joint Bi-level Image Joint Photographic
Experts Group Experts Group
TITLE: CD of JPSearch Part 2 (24800-2) Registration, Identification, and Management of Metadata Schema and Ontology
SOURCE: JPSearch Part-2 Editors
PROJECT: JPSearch
STATUS: CD
REQUEST ACTIONS:
For ballot
DISTRIBUTION: WG1 Distribution List
CONTACT: ISO/IEC JTC 1/SC 29/WG 1 Convener - Dr. Daniel T. Lee
eBay Inc., 2145 Hamilton Avenue, San Jose, California 95125,USA
eBay China Development Center, Shanghai German Center, 88 Keyuan Road, Zhangjiang Hi-Tech Park, Pudong, Shanghai 201203, China
Information technology — JPSearch — Part 2: Registration, identification and management of schema and ontology
Élément introductif — Élément central — Partie 2: Titre de la partie
Warning
This document is not an ISO International Standard. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an International Standard.
Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation.
ISO/IEC CD 24800-2
Copyright notice
This ISO document is a working draft or committee draft and is copyright-protected by ISO. While the reproduction of working drafts or committee drafts in any form for use by participants in the ISO standards development process is permitted without prior permission from ISO, neither this document nor any extract from it may be reproduced, stored or transmitted in any form for any other purpose without prior written permission from ISO.
Requests for permission to reproduce this document for the purpose of selling it should be addressed as shown below or to ISO's member body in the country of the requester:
[Indicate the full address, telephone number, fax number, telex number, and electronic mail address, as appropriate, of the Copyright Manger of the ISO member body responsible for the secretariat of the TC or SC within the framework of which the working document has been prepared.]
Reproduction for sales purposes may be subject to royalty payments or a licensing agreement.
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 24800-2 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
ISO/IEC 24800 consists of the following parts, under the general title Information technology — JPSearch:
Part 1: System framework and components
Part 2: Registration, identification and management of schema and ontology
Part 3: Query format
Part 4: File format for metadata embedded in image data (JPEG and JPEG 2000)
Part 5: Data interchange format between image repositories
This part of ISO/IEC 24800 provides a standardized set of technologies for JPSearch Core Metadata. The standard addresses the issues related to defining the JPSearch Core Metadata by providing a metadata system that will be embedded in an image and also be used for image search within JPSearch framework.
Information technology — JPSearch — Part 2: Registration, identification and management of schema and ontology
1 Scope
1.1 Organization of the document
ISO/IEC 24800 specifies a metadata system for describing an image. This Part of ISO/IEC 24800 specifies the description tools that comprise ISO/IEC 24800-2. The following subclauses are provided for specifying JPSearch Core Metadata scheme, where optional subclauses are indicated as (optional):
Syntax: specifies the normative syntax of the description tool using XML. Semantic: specifies the normative semantics of the description tool and each of its components (attributes
and elements) Informative examples (optional): provides informative examples that illustrate the use of provided scheme
and possible correspondence based relationships between JPSearch Core Metadata and the other metadata scheme for images.
1.2 Overview of JPSearch Core Metadata description schemes
The following basic elements are defined: schema tools, basic datatypes, linking and media localization tools, basic description tools (language, text, classification schemes).
<Editors note>: where are they defined? TODO
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 8601, Data elements and interchange formats — Information interchange — Representation of dates and timesISO 639 (all parts), Codes for the representation of names of languagesISO 3166-1, Codes for the representation of names of countries and their subdivisions — Part 1: Country codesISO 3166-2, Codes for the representation of names of countries and their subdivisions — Part 2: Country subdivision codeXML, Extensible Markup Language (XML) 1.0, 6 October 2000 http://www.w3.org/TR/2000/REC-xml-20001006XML Schema, W3C Recommendation, 2 May 2001 http://www.w3.org/XML/SchemaXML Schema Part 0: Primer, W3C Recommendation, 2 May 2001 http://www.w3.org/TR/xmlschema-0/XML Schema Part 1: Structures, W3C Recommendation, 2 May 2001 http://www.w3.org/TR/xmlschema-1/XML Schema Part 2: Datatypes, W3C Recommendation, 2 May 2001 http://www.w3.org/TR/xmlschema-2/XPath, XML Path Language, W3C Recommendation, 16 November 1999 http://www.w3.org/TR/1999/RECxpath-19991116
NOTE These documents are maintained by the W3C (http://www.w3.org).
3 Terms and definitions
3.1 Conventions
3.1.1 Description tool
<Editor’s Note> Specific structure of description system will be described here.
3.1.2 Naming convention
In order to specify the JPSearch Core metadata description scheme, this part of ISO/IEC 24800 uses constructs provided by XML such as "element" and "complexType." The names associated to these constructs are created on the basis of the following conventions:
If the name is composed of multiple words, the first letter of each word is capitalized, with the exception that the capitalization of the first word depends on the type of construct as follows: <Editors note>TODO
Element naming: the first letter of the first word is capitalized (e.g. Identifier element of JPSearchCoreType).
complexType naming: the first letter of the first word is capitalized, and the suffix "Type" is used at the end of the name (e.g. JPSearchCoreType).
3.1.3 Documentation convention
The syntax of each description is specified using the constructs provided by XML, and is presented in this document using a specific font and background as shown in the following example:
The semantics of each description tool is specified in text using a table format, where each row contains the name and a definition of a type, element or attribute as shown in the following example:
Name DefinitionExampleType Specifies an …element1 Describes the …attribute1 Describes the …
3.2 Wrapper of the schema
The description examples and syntax of description tools specified in this document assume that a schema wrapper is provided which identifies the XML Schema namespace (XML Schema) and JPSearch namespace:
For the purposes of ISO/IEC 24800-2, the following terms and definitions apply.
3.3.1 Schema-related terminology
<Editor’s Note> schema related terms will be selected and defined.
4 Overview
4.1 JPSearch Architecture Overview and Scenarios
As illustrated in ISO/IEC 24800-1 (JPSearch – Part1- System framework and components), JPSearch consists of 5 parts, namely an overall description (part 1), the definition of a core schema and transformation rules (part 2), a query format (part 3), metadata embedded image file format (part 4) and a data interchange format (part 5). The main components and its interplay have been demonstrated in part 1. In this context, an overview of the involved components of this part and its interplay is presented.
Core Schema: The core schema serves as metadata basis supporting interoperability during search among multiple image retrieval systems. The core schema is used by clients to formulate in combination with the JPEG Query Format search requests to JPSearch compliant search systems. Note, that only metadata described by the core schema is guaranteed to be processed by JPSearch compliant systems.
Transformation Rules: Transformation rules are used in order to map the metadata of a query of incoming search requests to compatible search requests of native systems.
Metadata Registration authority: The metadata registration authority is hosted by JPEG and supports the registration and request of metadata schemas and its transformation rules or links to them. It is necessary that every participating content provider registers their schema and transformation rules or a link to them at this authority. In case the JPSearch compliant retrieval system is operated in offline mode, the necessary information (target schema, transformation rules, etc.) should be available at the respective system itself.
JPQF Query Reformulation: After applying the transformation rules on the metadata content of a query, the query itself is target for transformation. This transformation can be effected by the enlargement or combination of metadata information or by any other reason such as adding user preferences, etc.
JPQF Aggregation Service: In a retrieval scenario where multiple search engines are involved, multiple search results will occur for one query request. In order to forward only one single result set to a client, these individual result sets need to be aggregated. This task is accomplished by the aggregation service.
4.2 Simple Scenario (1 Retrieval System is involved)
The simple scenario describes a JPSearch compliant system that forwards the incoming JPQF query to exactly one target database. In our example, the target metadata format is MPEG-7 but any other XML based description format can be considered as well.
Figure 1 — Workflow of a simple retrieval scenario
Workflow of the system:
1. JPQF query receives
2. Metadata based on core schema is transformed to the metadata of the native schema
3. JPQF query reformulation (optional)
4. Forward the query to the native system where the interpreter transforms it to the native query language and executes it.
5. Transforms the result set to the metadata of the core schema
4.3 Complex Scenario (N Retrieval Systems are involved)
The complex scenario describes a JPSearch compliant system that forwards the incoming JPQF query to two or more target databases. In our example, the target metadata format of the involved databases is MPEG-7 but any other XML based description format can be considered as well.
Figure 2 — Workflow of a complex retrieval scenario
Workflow of the system:
1. JPQF query receives
2. Metadata based on Core Schema is transformed to the metadata of the N native schemas
3. Additional Metadata is transformed to N native Schemas (if possible else the information is discarded)
4. N times Query reformulation (optional)
5. Forward N queries to N native systems where the interpreter transforms it to the native query language and executes it
6. Transforms N individual result sets to core schema.
7. Aggregate N result sets to 1 result set
8. Forward final result set to user
4.4 Utilization of the core schema for JPSearch as guideline
The integration of JPEG files annotated by proprietary image description formats or the transformation of JPQF queries should be realized by a transformation process (see Error: Reference source not found for a query approach) where the normative parts concentrate on the definition of the query format, a description language for transformation rules and the core schema which is the starting point for every request. The main aim of the JPSearch transformation module is the adaptation of the original query to a query (in JPQF) targeting on the schema of the target retrieval system.
Transformation Rules (JPTRDL) for mapping JPRMS to Target Schema X
JPQF Query Transformation
Figure 3 — JPQF Query Transformation Process
4.5 Aggregation Service
Goal: establish a final correct ranking and sorting of the individual result items
Minimum Inputs:
N result sets (after transformation),
N service capability descriptions (for links to additional information)?
An aggregation service must consider the following elements of the JPQF during the merge process.
GlobalComment: The GlobalComment elements of all individual result sets are simply concatenated as they are represented as strings. In doing so, the original information of the services is not lost.
SystemMessage: Since it is hard to merge multiple SystemMessage elements to a single one, a new SystemMessage is created where all individual messages are combined. Unfortunately, the information which service has stated which system message is lost. In order to keep this information, one solution is to modify the description of the system message which is not standard conform.
currPage: This attribute is supposed to be the same for all individual results, so simply one of the present values is chosen.
totalPages: This attribute indicates the total number of pages retrieved from a service. During the aggregation process this value is set to the sum of the single total pages that were returned. Thus, the final value of totalPages might be bigger than the actual amount of pages. The overall amount of result items should not exceed the given maxItemCount and maxPageEntries values of the input query. Therefore, the new totalPages is accumulated by the sum of all individual totalPages, but must hold the following condition: totalPages <= maxItemCount/maxPageEntries
expirationDates: This attribute describes the date when the service expires the result. The new expiration date is set to the next date to take place.
recordNumber: The attribute recordNumber is specified to be a numeric value in strictly ascending order. This value has to be adjusted when several result sets are merged to a single one. Its occurrence is required and after the aggregation took place, it has to be strictly ascending again. This uniqueness is necessary since the recordNumber is referenced when relevance feedback is used. As mentioned previously, the overall amount of result items should not exceed the maxItemCount. Therefore, a merging process can only select items from the individual result sets until this upper boundary is reached. How this selection is realized, depends on the used aggregation algorithm. Another problem tackles the process of an ensuing relevance feedback operation. A relevance feedback operation selects several result items by its recordNumber value and activates a new retrieval based on the given good and/or bad examples. As the record number is adjusted at the merged result set, a mapping table has to be maintained in order to extract the original record numbers. Nevertheless, this solution only solves a part of the problem. A relevance feedback request only contains a collection of record numbers and it assumes that the target MMRS has the original result set by hand and can extract additional information of the selected items for retrieval. This assumption is only true in a single MMRS scenario, but not in a scenario where multiple services are involved and the result merging process is realized by a third party implementation (as in our scenario).
originID: The attribute originID has to point to the respective MMRS wherefrom the result was retrieved. Its assignment is not mandatory for the JPQF standard. If the MMRS already assigned its ID, it does not have to be changed during the result aggregation. Otherwise it is activated. This is necessary since the client needs to know the origin of a result item, e.g. for a following relevance feedback.
jpqfID: This attribute identifies a single message within the communication process between clients, services and middleware components. During the result aggregation process, the framework is confronted with a set of different jpqfIDs as response to one query request. Therefore, a new ID needs to be generated which serves as identifier for the merged result set. In this case, several issues are related. On the one side, in asynchronous communication mode the client is not able to fetch the individual result sets on its own and on the other side a direct relevance feedback operation to the involved MMRS is also not possible. Therefore, a mapping (merged result set ID to the IDs of the individual messages) needs to be maintained at the JPSearch compliant search system until the expiration time is reached and all proximate messages need to be processed by the framework.
5 JPSearch core metadata schema
This clause specifies the schema that facilitates the composition of descriptions. The following description tools are specified: (1) the type hierarchy of the schema defined in ISO/IEC 24800-2 and the root element.
JPSearch core metadata schema contains four types: PersonNameType, SourceType, PublisherType and JPSearchCoreType.
Name DefinitionSourceType Specifies a type of image source that can be an original image
or an object in the form such as painting, book and so on.Relation Describes a relationship between the image and the object that
the image is created from. The definition of the relationship should be selected from definitions in ControlledRelationTermType.
<Editor’s Note>
1. In the original design, the source was defined directly without SourceType. For structural pattern, SourceType is created and modified the original schema.
2. Since there are two “controlledTerm” for Source and Rating, in order to control the terms, two subtypes are created and needs to be defined.
Modifiers Describes a modifier’s name or a list of names who changed the original image resulting in the creation of the image (optional).
Creators Describes a person’s name or a list of the names who created the image or made contributions in the creation of the image (optional).
Publisher Describes information about the publishing people or organization of the image
CreationDate Describes the date when the image is created.
ModfiedDate Describes the date when the image is modified.
Description Specifies the content of the image in the form of text.
RightsDescription Describes the right related information. <Editors note> Need to be more precise!!
Source Describes a source of the image. It can be another image or an object in the form of such as painting, book and so on.
Keyword Describes a list of keywords that characterize the image (optional).
Title Describes the title of the image (optional).
CollectionLabel Describes user provided labels that can be used for the purpose of collection and categorization of images (optional).
PreferenceValue Describes the value of the preference of the image in the form of integer value.
Rating Describes the rating results that should be one of the corresponding controlled terms. The definition of the terms is provided by JPSearch.
OriginalImageIdentifier Describes the identifier of the original image from which the image is created
<Editor’s note>
The following elements are under discussion.
1. Place can be where the picture is taken or where the picture is trying to express.2. Person can be who is shown in the picture.3. Event is shown in the picture object which is shown in the picture. 4. Regionlocator is simple bounding box represented by imageWidth that is the number of pixels and
imageHeigth that is the number of pixels.5. Orientation is the choice of portrait or landscape make model.
6 Management of core schema and transformation rules
This part targets on the definition of the complex types for the registration process of the schema, its transformation rules and contact information. The process of registration is mandatory for all schemas that can be addressed within a retrieval operation. The standard supports two scenarios. First, a global authority for ontologies and their transformation rules will be established where all JPSearch compliant retrieval applications can obtain the needed information. This global authority is hosted by JPEG. Second, in case the retrieval application operates in offline mode, the schema and their transformation rules shall be located at the application itself.
6.1 Root Element
6.1.1 Introduction
The SchemaManagement element serves as the root element of the JPSearch Management Process. The root element shall be used as the topmost element in all messages transmitted. This applies on all operations in the corresponding message direction (input or output).
Name DefinitionRequestInputType Specifies the RequestInputType type which is used for describing all
information that is necessary during the request of schemas and/or transformation rules.
SchemaID Requests the schema information that is registered by the specified URI.
TransformationRuleFrom
Requests the information for the transformation rules that are registered for the transformation between a proprietary schema to the core schema. The proprietary schema is identified by its schema ID.
TransformationRuleTo Requests the information for the transformation rules that are registered for the transformation between the core schema to a proprietary schema. The proprietary schema is identified by its schema ID.
Name DefinitionSchemaType Specifies a SchemaType type which is used for describing all
information in order to register a schema and its transformation rules.SchemaInformation Specifies the description of a certain schema.
TransformationRules Specifies the set of transformation rules which are necessary for reformulating a query from the reference metadata model to the registered target metadata model.
TBD : provides similar as the SchemaInformationType information about the name, description, identification location and direction of transformation rules
TBD: Add replacement operation for transformation rules only!!!
6.8 SchemaInformationType
6.8.1 Introduction
The SchemaInformationType type provides information to identify the schema and its different versions.
Description Describes the details of the information.
6.9.4 Example
Editors Note: TODO
7 JPSearch Transformation Rules Declaration Language (JPTRDL)
7.1 TransformationRulesType
7.1.1 Introduction
The TransformationRulesType type provides information in order to define a set of transformation rules that provide the transformation strategies between the core schema and a registered target metadata schema.
Specifies the TransformationRulesType type that allows the definition of a set of transformation rules providing means for transforming JPQF queries from the core schema to a registered target schema.
TransformationRule Describes one transformation rule. fromFormat Identifies (by the registered URI) the source schema. toFormat Identifies (by the registered URI) the target schema.
7.1.4 Example
Code 1: Transformation rule body (from core schema to Dublin Core)
The following abstract types support the composition of a modular architecture in order to combine individual source and target types during the configuration of certain transformation rules.
Specifies the OneToOneFieldTransformationType type which realizes the one-to-one pattern for transformation rules. In this case one field element of the source schema will be transformed into one element of the target schema.
FromField This element references to one of the subtypes of the SourceType type and points to a specific element of the source schema.
ToField This element references to one of the subtypes of the TargetType type and points to a specific element of the target schema.
7.3.4 Example
Code 2: Example one-to-one transformation rule (from core schema to MPEG-7)
7.4 ManyToOneFieldTransformationType
7.4.1 Introduction
This type realizes the many-to-one pattern of a transformation rule.
Semantics of the ManyToOneFieldTransformationType type:
Name DefinitionManyToOneFieldTransformationType
Specifies the ManyToOneFieldTransformationType type which realizes the many-to-one pattern for transformation rules. In this case at least 2 field elements of the source schema will be transformed into one element of
FromField This element references to one of the subtypes of the SourceType type and points to a specific element of the source schema. At least two elements shall occur.
ToField This element references to one of the subtypes of the TargetType type and points to a specific element of the target schema. Only one element is allowed.
7.4.4 Example
Code 3: Example many-to-one transformation rule (from core schema to MPEG-7)
7.5 OneToManyFieldTransformationType
7.5.1 Introduction
This type realizes the one-to-many pattern of a transformation rule.
Semantics of the OneToManyFieldTransformationType type:
Name DefinitionOneToManyFieldMappingType Specifies the OneToManyFieldTransformationType
type which realizes the one-to-many pattern for transfromation rules. In this case one element of the source schema will be transformed into at least 2 elements of the target schema.
FromField This element references to one of the subtypes of the SourceType type and points to a specific element of the source schema. Only one element is allowed.
ToField This element references to one of the subtypes of the TargetType type and points to a specific element of the
Name DefinitionSourceFieldType Specifies the SourceFieldType type which allows the
selection of a certain element of the source metadata format by an XPath expression. The type is an extension of the abstract SourceType type.
XpathExpression This element allows the definition of an XPath expression which points to a single element in the source metadata format.
7.7 FilteredSourceFieldType
7.7.1 Introduction
This type defines all information for pointing to an element of the source metadata format. In addition, some filter conditions can be applied which are performed during the transformation process.
Name DefinitionFilteredSourceFieldType Specifies the FilteredSourceFieldType type which
allows the selection of a certain element of the source metadata format. The type is an extension of the SourceFieldType type.
FilterWithRegExpr This element defines a filter condition by using regular expressions. By evaluating this regular expressions the content of the addressed element in the source schema is divided into tokens, which can be bound to variables.
VariableBinding This element is the main section to bind the extracted tokens to particular variables.
ExplicitBinding This element may be used without a regular expression. It simply binds the addressed metadata element / graph pattern to a variable.
ExplicitPrefixBinding This element is used in order to bind the first token defined by a regular expression to a variable.
ListBinding This element binds all tokens defined by a regular expression to a list of variables. Note that only those tokens are considered that are not target of any other binding.
ExcplicitPostfixBinding This element is used in order to bind the last token defined by a regular expression to a variable.
Name DefinitionTargetFieldType Specifies the TargetFieldType type which allows the
selection of a certain element of the source metadata format. The type is an extension of the abstract TargetType type.
XPathExpression This element allows the definition of an XPath expression which points to a single element in the target metadata format.
7.9 FormattedTargetFieldType
7.9.1 Introduction
This type defines all information for pointing to an element of the target metadata format. In addition, some filter conditions can be applied which are performed during the transformation process in order to format the content correctly.
Name DefinitionFormattedTargetFieldType Specifies the FormattedTargetFieldType type which
allows the selection of a certain element of the target metadata format. The type is an extension of the TargetFieldType type.
ReplaceWithRegExpr This element defines a format condition by using regular expressions. By evaluating this regular expression the content of the source schema is modified and transformed respectively in order to fit to the addressed element in the target metadata format.