R. Grosu:

"ResNets, NeuralODEs and CT-RNNs are Particular Neural Regulatory Networks";

ArXiv,.(2020), 9 pages.

This paper shows that ResNets, NeuralODEs, and CT-RNNs, are par-

ticular neural regulatory networks (NRNs), a biophysical model for the

nonspiking neurons encountered in small species, such as the C.elegans

nematode, and in the retina of large species. Compared to ResNets, Neu-

ralODEs and CT-RNNs, NRNs have an additional multiplicative term in

their synaptic computation, allowing them to adapt to each particular

input. This additional

exibility makes NRNs M times more succinct

than NeuralODEs and CT-RNNs, where M is proportional to the size of

the training set. Moreover, as NeuralODEs and CT-RNNs are N times

more succinct than ResNets, where N is the number of integration steps

required to compute the output F(x) for a given input x, NRNs are in

total M N more succinct than ResNets. For a given approximation task,

this considerable succinctness allows to learn a very small and therefore

understandable NRN, whose behavior can be explained in terms of well

established architectural motifs, that NRNs share with gene regulatory

networks, such as, activation, inhibition, sequentialization, mutual exclu-

sion, and synchronization. To the best of our knowledge, this paper uni es

for the rst time the mainstream work on deep neural networks with the

one in biology and neuroscience in a quantitative fashion.

https://publik.tuwien.ac.at/files/publik_293059.pdf

Created from the Publication Database of the Vienna University of Technology.