"Second Thoughts" Been playing with adding a small transformer that reads output near the end of generation, and feeds it back near the top as a refinement loop. A quick test of 1.7B model showed drastic improvement in focused tasks (like coding)

An independent AI researcher known as bigattichouse has shared a promising new technique for improving small language models. They added a small transformer that reads the model's output near the end of generation and feeds a refinement signal back near the top of the network, creating a loop. In initial tests with a 1.7B parameter model, the approach delivered a 'drastic improvement' on focused tasks like coding, even though the base model was tiny. The researcher is now scaling up by training a 9B model and plans to run the full HumanEval benchmark (previously only the first 20 samples were tested).

The technique was inspired by neuroanatomy findings from 'Repeat Yourself' (dnhkng.github.io/posts/rys/), which gave the researcher a starting point and endpoint to attach the 'reverse LLM' side car. The side car reads from the end of the output and injects its processed signal back at the top, cycling through a loop focused on syntax. The result is a significant boost in code generation accuracy without requiring a larger base model. The researcher will clean up and release code on GitHub shortly, inviting the community to replicate and build on the work.