New framework lets neural networks grow in breadth, depth, and time

In a new paper on arXiv, researchers Eivinas Butkus, Kedar Garzón Gupta, and Nikolaus Kriegeskorte propose a framework that treats a recurrent convolutional neural network (R-CNN) as a finite subset of an infinite lattice. They define differentiable cost terms for three fundamental resources: breadth (width of layers), depth (number of layers), and time (number of recurrent steps). These costs are optimized jointly with task error via backpropagation, allowing the network to spontaneously evolve its architecture during training. By applying different pressures to each resource, the team observes diverse computational graphs emerging, from shallow-and-wide to deep-and-narrow, all achieving similar accuracy through trade-offs.

The experiments reveal several intriguing behaviors. Networks grow in all three dimensions as task complexity increases, and they automatically take more recurrent steps when inputs are partially occluded – a form of adaptive computation. Most strikingly, the model's time usage (recurrent steps) correlates with human reaction times in an object recognition task, suggesting that biological brains might also optimize similar resource constraints. This framework provides a normative account of how spatial and temporal pressures shape neural architectures, connecting AI research to questions about the diversity of neural solutions in nature. It could inspire more efficient, adaptable AI systems that self-structure under real-world constraints.