SceneMaker: Multimodal Visualisation of Natural Language Film Scripts Dr. Minhua Eunice Ma School of Computing & Intelligent Systems Faculty of Computing & Engineering University of Ulster, Magee, Northern Ireland [email protected], {p.mckevitt, tf.lunney, j.condell}@ulster.ac.uk Eva Hanser Prof. Paul Mc Kevitt Dr. Tom Lunney Dr. Joan Condell School of Computing and Mathematics Faculty of Business, Computing and Law University of Derby, England [email protected]
16
Embed
SceneMaker: Multimodal Visualisation of Natural Language Film Scripts
SceneMaker: Multimodal Visualisation of Natural Language Film Scripts. Dr. Minhua Eunice Ma. Eva Hanser Prof. Paul Mc Kevitt Dr. Tom Lunney Dr. Joan Condell. School of Computing & Intelligent Systems Faculty of Computing & Engineering University of Ulster, Magee, Northern Ireland - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
SceneMaker:Multimodal Visualisation
of Natural Language Film Scripts
Dr. Minhua Eunice Ma
School of Computing & Intelligent SystemsFaculty of Computing & EngineeringUniversity of Ulster, Magee, Northern Ireland
• Automatically generate well-designed and affective virtual scenes from screenplays
• Realistic visualisation of emotional aspects
• Multimodal representation with 3D animation, speech, audio and cinematography
• Enhance believability of virtual actors and scene presentation
: AIMS & OBJECTIVES
Input:Screen-play SceneMaker
SystemOutput: Animation
OBJECTIVES
• Processing/inferencing emotions and semantic information within story context
• Common sense, affective and cinematic knowledge ontologies reflecting human cognitive reasoning rules
• Automatic genre recognition from text
• Design, implementation and evaluation of SceneMaker
: AIMS & OBJECTIVES
• Standardized format and language of screenplays
• Automatic annotation of formal screenplay elements (Jhala 2008)
• Semantic information on location, timing, props, actors, events, manners, dialogue and camera direction
SEMANTIC TEXT PROCESSING: RELATED PROJECTS
INT. M.I.T. HALLWAY -- NIGHT Lambeau and Tom come around a corner. His P.O.V. reveals a figure in silhouette blazing through the proof on the chalkboard. There is a mop and a bucket beside him. As Lambeau draws closer, reveal that the figure is Will, in his janitor's uniform. There is a look of intense concentration in his eyes.
LAMBEAUExcuse me!
WILL
Oh, I'm sorry.
LAMBEAUWhat're you doing?
WILL
(walking away)I'm sorry.
Screenplay Extract from ‘Good Will Hunting (1997)’
ARCHITECTURE OF SCENEMAKER: DESIGN AND IMPLEMENTATION
SOFTWARE AND TOOLS: DESIGN AND IMPLEMENTATION
LVSR(2)
Lexical Visual Semantic
Representation
Script Format
Ontology
Unity(6)
3D Engine(JavaScript,XML)
MSML(5)
/SMIL
ConceptNet(3)
Common Sense Knowledge
Gate(1)
ANNIEOnto-Gazetteer
GenreOntologyRDFS/OWL
MovieOntologyRDFS/OWL
WordNet-Affect(4)
Festival(7)
Speech Synthesiser
Natural Language Processing & Script Segmentation
Context + Emotion Reasoning
Event Synchronisation
3D Rendering + Multimedia
3D Models(3D Studio Max)
MovieScript
AutomaticSound & Music
Selection
(1) http://gate.ac.uk (2) Ma 2006 (3) Liu and Singh 2004 (4) Strapparava and Valitutti 2004 (5) Van Rijsselbergen et al. 2009 (6) http://unity3d.com (7) http://www.cstr.ed.ac.uk/projects/festival
Evaluating 4 aspects of SceneMaker:
EVALUATION OF SCENEMAKER
Aspect EvaluationCorrectness of screenplay analysis & visual interpretation
Hand-animating scenes
Effectiveness of output scenes
Existing feature film scenes
Suitability for genre type Scenes of unknown scripts categorised by readers
Functionality of interface Testing with drama students and directors
: DESIGN AND IMPLEMENTATION
Text to Animation System Year Text Input: Genre Context Emotion AnimationMovie Script Reasoning (3D )
CONFUCIUS (Ma 2006) 2006 – – – – ScriptViz (Liu and Leung 2006) 2007 – – – – CAMEO (Shim and Kang 2008) 2007 – P&E Engine (Su et al. 2007) 2007 – – P&E rules Behaviour Generation System (Breitfuss et al. 2007) 2007 – (Dialogue) – – MSML (Van Rijsselbergen et al. 2009) 2009 – – – externalSceneMaker
RELATION TO OTHER WORK
CONCLUSION AND FUTURE WORK
• Automatic expressive multi-media animation of screenplays
• Focus on:– automatic reasoning about story context and emotional interpretation – based on world knowledge and context memory – emotions influencing scene compositions and event execution– scene direction refined by genre-specifics
• Analysis of script format to access semantic information
• Automatic genre specification from script
• Heightened expressiveness, naturalness and artistic quality
• Assist directors, actors, drama students, script writers
• Future work: Implementation & Testing of SceneMaker