Refine Thought: New test-time method boosts embedding model reasoning

A new paper from researchers Guangzhi Wang, Kai Li, Yinghao Jiao, and Zhi Liu introduces Refine Thought (RT), a test-time inference method designed to improve the semantic reasoning capability of text embedding models. Unlike traditional approaches that require fine-tuning or architectural changes, RT simply runs multiple forward passes of the existing embedding model and combines the intermediate representations into a final semantic vector. The method is shown to be especially effective for decoder-only embedding models such as Qwen3-Embedding-8B, which already possess latent reasoning abilities from pretraining but need an inference-time boost to surface them. The key insight is that these models can 'refine' their understanding iteratively, similar to chain-of-thought reasoning in large language models, but applied to the embedding space.

Experiments demonstrate that RT achieves significant gains on the BRIGHT benchmark (designed for semantic reasoning) and the PJBenchmark for person-job matching, while preserving performance on general-purpose semantic understanding tasks like C-MTEB. This indicates that RT selectively enhances reasoning without degrading other capabilities. The method is computationally heavier due to multiple passes, but requires no additional training data or parameter updates, making it a practical drop-in enhancement. The paper positions RT as a test-time inference method—a growing trend in AI that trades compute for improved performance without altering model weights. Future work could explore optimizing the number of passes or applying similar techniques to other embedding architectures.