Sketch-Based Interaction for Planning-Based Interactive Storytelling

Drawings have been used for thousands of years as a visual complement to oral and written storytelling. The evolution of technology and the advent of interactive narratives brings the possibility of exploring drawings and storytelling in new ways. This paper presents a new sketch-based interaction m...

Full description

Saved in:
Bibliographic Details
Main Authors: de Lima, Edirlei Soares, Gheno, Felipe Joao, Viseu, Ana
Format: Conference Proceeding
Language:English
Subjects:
automated planning
Games
interactive narratives
interactive storytelling
Planning
Servers
sketch recognition
sketch-based interaction
Solid modeling
Tablet computers
Three-dimensional displays
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Drawings have been used for thousands of years as a visual complement to oral and written storytelling. The evolution of technology and the advent of interactive narratives brings the possibility of exploring drawings and storytelling in new ways. This paper presents a new sketch-based interaction method for planning-based interactive storytelling systems, which uses a deep learning model based on a Convolutional Neural Network to recognize digital hand-drawn sketches. By combining real time sketch recognition with a planning-based plot generation algorithm, the proposed system allows users to interact with narratives by sketching objects on smartphones or tablet computers, which are then recognized by the system and converted into virtual objects in the story world, thereby affecting the plot of the narrative. Preliminary results show that the sketch recognition model has a remarkable accuracy for small sets of sketch classes (accuracy of 95.1 % for 14 classes), which are sufficient to provide a good variety of interaction options. In addition, it can also be extended to more complex scenarios while maintaining a considerable accuracy (87.4% for 172 classes and 71.6% for 345 classes).
ISSN:2159-6662
DOI:10.1109/SBGames51465.2020.00029