Use more text than one token to avoid neuralese

A viral technical post argues that compressing a transformer's output vector into a single token for the next input is a massive information bottleneck, calling it a 'pretty big squash.' The proposed solution, dubbed 'neuralese,' involves passing the full vector or multi-token sequences to preserve bandwidth. This challenges the assumption that scaling AI requires abandoning natural language, suggesting richer text intermediates could unlock more capability without losing interpretability.