What are people using for low-latency autocomplete in production? [P]

A recent Reddit thread on autocomplete/typeahead systems has sparked a debate among developers about the best approaches for production environments where latency is critical, such as search-as-you-type or RAG pipelines. The discussion highlights three main strategies: full search backends like Elasticsearch and Meilisearch, which offer robust indexing but can be heavy; LLM-based suggestions, which provide flexibility but are too slow for per-keystroke use; and simpler prefix or n-gram systems, which are fast but limited in quality. The community consensus leans toward classical methods for pure speed, with some experimenting with hybrid retrieval+reranking to balance latency and suggestion quality.

The original poster shares a local Python package called query-autocomplete (available on GitHub and PyPI) for lightweight experimentation, but emphasizes it's not a production replacement. Developers note that real-world tradeoffs depend on infrastructure overhead and dataset size. For instance, Meilisearch handles typo-tolerant autocomplete with sub-50ms latency, while Elasticsearch requires careful tuning for similar performance. LLM-based approaches are largely dismissed for real-time use due to inference latency, though they excel in complex suggestion tasks. The thread underscores a practical divide: most production systems still rely on classical methods, with hybrid models emerging only where quality demands justify the latency cost.