Evolutionary Mixture-of-LoRA study: router tweak helps, lifecycle penalizes, pre-alignment needed

Kumaresan's paper decomposes an evolutionary mixture-of-LoRA system on a 150M-parameter, widened-D (D=1536, V=32000) substrate into three factors: a router rewrite, a per-domain evaluation scope, and a lifecycle of death, alpha-blend inheritance, SVD mutation, and slot reallocation. Using a 5-of-8 partial 2^3 factorial design with 3 seeds and 25,000 adaptation steps per cell, the study attributes the full system's improvement over a static B3 baseline primarily to the router rewrite (+0.0426 nats balanced log-PPL improvement, t=12.86, p=0.006). The per-domain evaluation scope had null effect at seed resolution, while the lifecycle was a significant net drag of -0.028 nats (t=-4.46, p=0.047). An auxiliary counterfactual on alpha=0 inheritance was inconclusive.

The paper also introduces a controllable synthetic sandbox to explore substrate-conditional boundaries. A base-perturbation probe directionally refutes a "genomic-context" reframe of the lifecycle role. The sandbox reveals a sharp regime boundary: evolutionary search on the routing channel is load-bearing only when adapters are pre-aligned to the task. In every other regime tested—random adapters, misaligned tasks, or gradient-based initialization—the evolutionary approach underperformed, tied, or actively degraded gradient solutions. This suggests that naive application of evolutionary methods to mixture-of-LoRA systems can hurt performance unless careful pre-alignment is ensured.