Encoding a Categorical Independent Variable for Input to TerrSet’s Multi-Layer Perceptron

The profession debates how to encode a categorical variable for input to machine learning algorithms, such as neural networks. A conventional approach is to convert a categorical variable into a collection of binary variables, which causes a burdensome number of correlated variables. TerrSet’s Land...

Full description

Saved in:
Bibliographic Details
Published in:ISPRS international journal of geo-information 2021-10, Vol.10 (10), p.686
Main Authors: Evenden, Emily, Pontius Jr, Robert Gilmore
Format: Article
Language:English
Subjects:
Algorithms
categorical variable
Continuity (mathematics)
Dependent variables
encoding
Geology
Independent variables
land change modeler
Learning algorithms
Machine learning
Methods
Multi-Layer Perceptron
Multilayer perceptrons
Neural networks
Population
Population Evidence Likelihood
Probability theory
Sample Empirical Probability
Variables
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The profession debates how to encode a categorical variable for input to machine learning algorithms, such as neural networks. A conventional approach is to convert a categorical variable into a collection of binary variables, which causes a burdensome number of correlated variables. TerrSet’s Land Change Modeler proposes encoding a categorical variable onto the continuous closed interval from 0 to 1 based on each category’s Population Evidence Likelihood (PEL) for input to the Multi-Layer Perceptron, which is a type of neural network. We designed examples to test the wisdom of these encodings. The results show that encoding a categorical variable based on each category’s Sample Empirical Probability (SEP) produces results similar to binary encoding and superior to PEL encoding. The Multi-Layer Perceptron’s sigmoidal smoothing function can cause PEL encoding to produce nonsensical results, while SEP encoding produces straightforward results. We reveal the encoding methods by illustrating how a dependent variable gains across an independent variable that has four categories. The results show that PEL can differ substantially from SEP in ways that have important implications for practical extrapolations. If users must encode a categorical variable for input to a neural network, then we recommend SEP encoding, because SEP efficiently produces outputs that make sense.
ISSN:2220-9964
2220-9964
DOI:10.3390/ijgi10100686