Cuypers: Smart Style on the Semantic Web

Cuypers: Smart Style on the Semantic Web

Jacco van OssenbruggenLynda HardmanCWI Amsterdam

2

Talk overview

1.Requirements for 2nd & 3rd generation multimedia– why multimedia is different from text

2.Smart style – a document engineer meets a graphic

designer

3.Overview of CWI’s Cuypers engine – testbed for multimedia transformations

4.Future directions – towards 3rd generation multimedia

3

Talk overview

1.Requirements for 2nd & 3rd generation multimedia– why multimedia is different from text

2.Smart style – a document engineer meets a graphic

designer

3.Overview of CWI’s Cuypers engine – testbed for multimedia transformations

4.Future directions – towards 3rd generation multimedia

4

The Web in three generations1. Hand-coded (HTML) Web content

– easy access through uniform interface– huge authoring and maintenance effort– hard to deal with dynamically changing content

2. Automated on-the fly content generation– based on templates filled with database content– later extended with XML document

transformations

3. Automated processing of content– The Semantic Web– explicit meta-data instead of “screen scraping”– agreed upon semantics (RDFS, DAML+OIL)

5

Multimedia on the Web• Real multimedia Web content is still rare

– Mostly bells & whistles to enhance HTML text …– … or mono-media AV-streams

• Virtually all presentations are hand-authored– proprietary formats that are hard to generate – limited support for dynamic content and

multichanneling– most Web technology is text/page-oriented …– … with SMIL as one of the few exceptions

• Conclusion:Multimedia has hardly caughtup with the 1st generation Web!

6

2nd generation multimedia

•Adapts to end-user’s platform capabilities– multichanneling: PC, PDA, mobile, voice-only,

…

•Adapts to the network resources available – bandwidth and other QoS parameters

•Personalization– language, abilities, level of expertise, ...

7

But multimedia is not text...• Different document and presentation abstractions

– hard to separate style from structure

• Formatting is not based on text flow– no pages or scrollbars, no line-breaking or hyphenation– templates often do not work well either

• Feedback from the formatting back-end required– need to check whether proposed layout is feasible– layout of media items is less flexible than text-based

layout

• Transformations are hard in a functional language– need to try out designs and backtrack when necessary

9

Talk overview

1.Requirements for 2nd & 3rd generation multimedia– why multimedia is different from text

2.Smart style – a document engineer meets a graphic

designer

3.Overview of CWI’s Cuypers engine – testbed for multimedia transformations

4.Future directions – towards 3rd generation multimedia

10

Smart style (our proposed solution)•The creation of a web-based infrastructure that is – an extension of the current

document engineering perspective

– taking into account the graphic design perspective

11

Document Engineering Perspective• Content is selected & structured• Mappings are defined to a

new presentation structure• Styles (such as color and font) can be applied

• The transformation process is linear and assumes that – Content/document structure, – presentation structure and, – style

are independent of each other.

12

Document Engineering Perspective

Finalform

Documentstructure

Presentationstructure

Styledpresentation

Tra

nsfo

rmatio

nSheet (X

SLT

)

Sty

lesh

eet

(CSS

)

Renderin

g(W

eb-d

evice

)

HTML

WML

PostScript

13

Graphic Design Perspective

•Basically:presentation structure, content and style depend on one another.

•In multimedia presentations, spatio-temporal layout gives meaning to the presentation (in contrast to the "linear" nature of text-flow).

14

Pres

enta

tion

stru

ctur

e Style

Content

Grouping deter-

mines selection

of media item

s

Grouping depends on

semantic relations

among m

edia items

Styl

e de

term

ined

by

sele

cted

med

ia it

ems

Styl

e de

term

ines

sele

ctio

n of

med

ia it

ems

Top-level order/groupingaffects local arrangements

Overall style determineslocal style of media items

Grouping deter-mines layout style

Layout aestheticsdetermines grouping

Internal design dependencies

15

Example

16

External Forces on Design Process• Content provider

– Mission (make profit, promote image)– Limited resources (cost)– Preferences (company colors)

• End-user– Goals, needs– Delivery context limitations (time, environment)– Preferences (images vs text, audio vs visual)

• Designer– Design experience– Resource limitations

17

Requirements for Smart Style

Integrate Semantic Web technology with current Web engineering technology:

• Communicating delivery contexts• Supporting metadata for content and

description of its function• Encapsulating explicit design knowledge

Processing delivery context, metadata and design knowledge within the document engineering pipeline.

18

Talk overview

1.Requirements for 2nd & 3rd generation multimedia– why multimedia is different from text

2.Smart style – a document engineer meets a graphic

designer

3.Overview of CWI’s Cuypers engine – testbed for multimedia transformations

4.Future directions – towards 3rd generation multimedia

19

Cuypers transformation engine• Cuypers testbed is based on

– media independent presentation abstractions– transformation rules with built-in backtracking

andconstraint solving

Sem

an

tic

str

uctu

re

Dis

cou

rse

str

uctu

re

Pre

sen

tati

on

str

uctu

re

Con

str

ain

t la

yer

Fin

al fo

rm

pre

sen

tati

on

Mu

ltim

ed

ia IR

syst

em

HTTP C

lient

20

Cuypers transformation engine• Cuypers testbed is based on

– media independent presentation abstractions– transformation rules with built-in backtracking

andconstraint solving

… but all knowledge is • implicit and hidden in the rules• lost in the generated Web presentation • not reusable for other Web sites

21

Talk overview

1.Requirements for 2nd & 3rd generation multimedia– why multimedia is different from text

2.Smart style – a document engineer meets a graphic

designer

3.Overview of CWI’s Cuypers engine – testbed for multimedia transformations

4.Future directions – towards 3rd generation multimedia

22

User Query

MMDB

MetadataDB

Cuypers Knowledge Sources

DesignModel

Cuypers Engine

DiscourseModel

DomainModel

UserProfile

DeviceProfile

23

Cuypers & the Semantic Web

• Domain model– Reuse existing domain ontologies, RDFS/OWL

• Device profile– CC/PP

• Design model– ? (local rules, global constraints, templates?)

• Discourse model– ? (RST, discourse ontologies, narration models)

• User model– ? (AHAM/AHA!/MAO)

24

Conclusions

• Multimedia is mostly first generation– hand-authored presentations

– common Web-tools are too text-centric

• Cuypers system realises second generation– on-the-fly multimedia generation

– media-centric transformations

• Third generation is around the corner– generating annotated multimedia

– reusing knowledge available on the Semantic Web