Estimating near-verbatim extraction risk in language models with decoding-constrained beam search

A research team from Stanford, Cornell, and other institutions has developed a breakthrough method for detecting when large language models (LLMs) like GPT-4 and Claude memorize and can reproduce copyrighted or private training data. The new technique, called 'decoding-constrained beam search,' addresses a critical blind spot in existing detection methods, which only looked for verbatim (exact word-for-word) memorization. The researchers found that near-verbatim memorization—where models reproduce content with minor variations—poses similar privacy and copyright risks but was previously too computationally expensive to detect reliably.

Standard Monte Carlo estimation required approximately 100,000 samples per text sequence to accurately assess risk, making large-scale audits impractical. The new deterministic method provides reliable lower-bound risk estimates at a cost comparable to just 20 samples, making it 5,000 times more efficient. When applied, it revealed that many more sequences are extractable than verbatim methods showed, with substantially larger per-sequence extraction mass. The research also uncovered clear patterns in how this near-verbatim extraction risk manifests across different model sizes and types of training text, providing crucial data for AI developers and regulators.