AI models struggle with complex logic needed for computer chip design

Researchers created a new automated benchmark, LLM-FSM, to test how well large language models can understand state-based logic and generate correct hardware code from natural language descriptions. They found that even the most advanced models' accuracy drops sharply as the logical complexity increases. The study also showed that targeted training and increased computational power during testing can improve the models' performance on these specialized engineering tasks.