Multimodal Fusion, Fission and Virtual Reality Simulation for an Ambient Robotic Intelligence

• Main Authors: Adjali, Omar, Hina, Manolo Dulva, Dourlens, Sebastien, Ramdane-Cherif, Amar
• Format: Conference Proceeding
• Language: English
• Subjects: ambient intelligence; multimodal fission; multimodal fusion; multimodality; robotic intelligence; virtual reality
• ISSN: 1877-0509; 1877-0509
• DOI: 10.1016/j.procs.2015.05.060

In this paper, we present an architecture that demonstrates multimodal fusion and fission using semantic agents and web services that interacts with worldwide-web computers, services and users. This solution extracts the meaning of a situation using the semantic memory of agents to manage the interaction process involved. The fusion of values from different sensors produces an event that needs implementation. The fission process suggests a detailed set of actions that are for implementation. Before such actions are implemented by actuators, these actions are first evaluated in a virtual environment which mimics the real-world environment. If no danger arises from such virtual evaluation, then implementation is feasible. Otherwise, there might be a need to add one or more smaller actions to render the action safe and free from danger. Our work presents the following contributions: (i) a design of agent memory and a model of the world environment using a knowledge representation language that is compatible with the current standards, (ii) creation of a pervasive architecture with several scenarios of composition and adaptation, (iii) presentation of how agents and services interact to provide support in a real-world, and (iv) simulation of an event in a virtual environment to assess the feasibility of the event's implementation.