Constructing Organism Networks from Collaborative Self-Replicators

We introduce organism networks, which function like a single neural network but are composed of several neural particle networks; while each particle network fulfils the role of a single weight application within the organism network, it is also trained to self-replicate its own weights. As organism...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-02
Main Authors: Illium, Steffen, Zorn, Maximilian, Lenta, Cristian, Kölle, Michael, Linnhoff-Popien, Claudia, Gabor, Thomas
Format: Article
Language:English
Subjects:
Neural networks
Organisms
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We introduce organism networks, which function like a single neural network but are composed of several neural particle networks; while each particle network fulfils the role of a single weight application within the organism network, it is also trained to self-replicate its own weights. As organism networks feature vastly more parameters than simpler architectures, we perform our initial experiments on an arithmetic task as well as on simplified MNIST-dataset classification as a collective. We observe that individual particle networks tend to specialise in either of the tasks and that the ones fully specialised in the secondary task may be dropped from the network without hindering the computational accuracy of the primary task. This leads to the discovery of a novel pruning-strategy for sparse neural networks
ISSN:2331-8422