Speak-to-Objective: AI turns voice commands into microscopic particle assembly

Light-based advanced manufacturing has long faced a bottleneck: translating human design intent into machine-executable objectives for micro-scale tasks. Researchers from Karlsruhe Institute of Technology and University of Cambridge now introduce Speak-to-Objective, an agentic pipeline that bridges this gap. By feeding spoken or written commands into a conditioned large language model (LLM), the system generates fully differentiable objective functions for assembling microparticles. These objectives are then solved by a constraint-aware SLSQP solver and executed on an experimental optofluidic platform using laser-induced thermoviscous flows. The modular loop—perceive, compose, propose, act, report & learn—keeps the objective as the key interface, separating what to pattern from how to actuate, and learns from user feedback in real time.

The pipeline demonstrates robust assembly from partial traces, recovery after perturbations, and explicit placement of particle patterns—all without actuator-specific code. It composes geometry, spacing, and topology terms to generate descriptive objectives. In experiments, the team used laser-induced thermoviscous flows to assemble microfluidic particle patterns, turning natural language into precise microscale manipulation. Beyond microassembly, this work points toward self-driving AI-assisted optical manufacturing platforms, coupling natural language, differentiable objectives, and laser-based actuation into a reusable digital workflow. The reduced-complexity optofluidic setup serves as a proof-of-concept for broader applications in programmable manufacturing, lab-on-chip devices, and precision biomedical engineering.