Neural Network Generalization: The Impact of Camera Parameters

We quantify the generalization of a convolutional neural network (CNN) trained to identify cars. First, we perform a series of experiments to train the network using one image dataset - either synthetic or from a camera - and then test on a different image dataset. We show that generalization betwee...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2020, Vol.8, p.10443-10454
Main Authors: Liu, Zhenyi, Lian, Trisha, Farrell, Joyce, Wandell, Brian A.
Format: Article
Language:English
Subjects:
Artificial neural networks
Automobiles
autonomous driving
camera design
Cameras
convolutional neural network
Datasets
Image acquisition
Image processing
Imaging systems
network generalization
Neural networks
Parameters
physically based ray tracing
Pixels
Prototyping
Railroad cars
Ray tracing
Software
Training
Urban areas
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We quantify the generalization of a convolutional neural network (CNN) trained to identify cars. First, we perform a series of experiments to train the network using one image dataset - either synthetic or from a camera - and then test on a different image dataset. We show that generalization between images obtained with different cameras is roughly the same as generalization between images from a camera and ray-traced multispectral synthetic images. Second, we use ISETAuto, a soft prototyping tool that creates ray-traced multispectral simulations of camera images, to simulate sensor images with a range of pixel sizes, color filters, acquisition and post-acquisition processing. These experiments reveal how variations in specific camera parameters and image processing operations impact CNN generalization. We find that (a) pixel size impacts generalization, (b) demosaicking substantially impacts performance and generalization for shallow (8-bit) bit-depths but not deeper ones (10-bit), (c) the network performs well using raw (not demosaicked) sensor data for 10-bit pixels.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2020.2965089