On Determinism of Game Engines Used for Simulation-Based Autonomous Vehicle Verification

Game engines are increasingly used as simulation platforms by the autonomous vehicle community to develop vehicle control systems and test environments. A key requirement for simulation-based development and verification is determinism, since a deterministic process will always produce the same outp...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on intelligent transportation systems 2022-11, Vol.23 (11), p.20538-20552
Main Authors: Chance, Greg, Ghobrial, Abanoub, McAreavey, Kevin, Lemaignan, Severin, Pipe, Tony, Eder, Kerstin
Format: Article
Language:English
Subjects:
Autonomous driving
Autonomous vehicles
Computational modeling
Computer & video games
Configurations
Data models
Determinism
Domains
Engines
game engines
Games
Initial conditions
physics engines
Simulation
Software development tools
Testing
Verification
verification and validation (V&V)
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Game engines are increasingly used as simulation platforms by the autonomous vehicle community to develop vehicle control systems and test environments. A key requirement for simulation-based development and verification is determinism, since a deterministic process will always produce the same output given the same initial conditions and event history. Thus, in a deterministic simulation environment, tests are rendered repeatable and yield simulation results that are trustworthy and straightforward to debug. However, game engines are seldom deterministic. This paper reviews and identifies the potential causes and effects of non-deterministic behaviours in game engines. A case study using CARLA, an open-source autonomous driving simulation environment powered by Unreal Engine, is presented to highlight its inherent shortcomings in providing sufficient precision in experimental results. Different configurations and utilisations of the software and hardware are explored to determine an operational domain where the simulation precision is sufficiently high i.e. variance between repeated executions becomes negligible for development and testing work. Finally, a method of a general nature is proposed, that can be used to find the domains of permissible variance in game engine simulations for any given system configuration.
ISSN:1524-9050
1558-0016
DOI:10.1109/TITS.2022.3177887