G-ICSO-NAS: Shifting Gears between Gradient and Swarm for Robust Neural Architecture Search

A team of researchers has introduced G-ICSO-NAS, a novel hybrid framework for Neural Architecture Search (NAS) that aims to solve a critical bottleneck in automated machine learning. NAS is the process of automatically designing optimal neural network architectures, but existing methods are stuck in a trade-off: evolutionary algorithm (EA)-based approaches find high-quality designs but are computationally prohibitive, while gradient-based methods like DARTS are fast but prone to failure. G-ICSO-NAS bridges this gap by implementing a smart, three-stage strategy that shifts gears between different optimization techniques.

First, a 'Warm-up Phase' uses efficient gradient descent to pre-train the model's weights. Then, the core 'Exploration Phase' begins a hybrid co-optimization: an Improved Competitive Swarm Optimizer (ICSO)—a type of evolutionary algorithm—navigates the architecture space for the best design, while gradient descent concurrently fine-tunes the weights. This synergy allows ICSO to provide robust, global search capabilities to avoid dead ends, while gradients handle local efficiency. A final 'Stability Phase' uses fine-grained gradient search to converge on the optimal architecture.

The results are striking for their efficiency. In the standard DARTS search space, G-ICSO-NAS achieved a top-tier accuracy of 97.46% on the CIFAR-10 dataset using a computational budget of only 0.15 GPU-days. This represents a massive reduction in cost compared to purely evolutionary methods, which can require thousands of GPU-days. The method also showed strong transfer learning performance, scoring 83.1% on CIFAR-100 and 75.02% on ImageNet, and delivered state-of-the-art results across all datasets in the NAS-Bench-201 benchmark. The paper has been accepted for presentation at the International Joint Conference on Neural Networks (IJCNN) in 2026.