[Back]

R. Grosu:
"ResNets, NeuralODEs and CT-RNNs are Particular Neural Regulatory Networks";
ArXiv, . (2020), 9 pages.

This paper shows that ResNets, NeuralODEs, and CT-RNNs, are par-
ticular neural regulatory networks (NRNs), a biophysical model for the
nonspiking neurons encountered in small species, such as the C.elegans
nematode, and in the retina of large species. Compared to ResNets, Neu-
ralODEs and CT-RNNs, NRNs have an additional multiplicative term in
their synaptic computation, allowing them to adapt to each particular
input. This additional
exibility makes NRNs M times more succinct
than NeuralODEs and CT-RNNs, where M is proportional to the size of
the training set. Moreover, as NeuralODEs and CT-RNNs are N times
more succinct than ResNets, where N is the number of integration steps
required to compute the output F(x) for a given input x, NRNs are in
total M N more succinct than ResNets. For a given approximation task,
this considerable succinctness allows to learn a very small and therefore
understandable NRN, whose behavior can be explained in terms of well
established architectural motifs, that NRNs share with gene regulatory
networks, such as, activation, inhibition, sequentialization, mutual exclu-
sion, and synchronization. To the best of our knowledge, this paper uni es
for the rst time the mainstream work on deep neural networks with the
one in biology and neuroscience in a quantitative fashion.

https://publik.tuwien.ac.at/files/publik_293059.pdf