CW-Ghost boosts CPU speeds 1.54x with smarter prefetching

Helper-thread prefetching is a classic technique to hide the high latency of irregular memory accesses by running a lightweight thread ahead of the main computational thread. However, its effectiveness depends critically on how far ahead the helper thread runs—too far and it wastes cache capacity or introduces memory pollution, too short and it fails to hide latency. Existing approaches either use a fixed coverage range, which cannot adapt to different workloads and processors, or exhaustively search over candidate configurations, which is costly and impractical for production systems.

CW-Ghost, introduced by Ya Zhang and colleagues from the National University of Defense Technology and other institutions, solves this by taking a single offline profiling run of the target region. It estimates the average demand cache line fill volume per iteration and then combines that with a user-defined cache capacity budget to compute a 'Capacity Window'—the precise number of future iterations the helper thread should cover per chunk. A bounded chunk-level synchronization mechanism further limits how many chunks the helper can outrun the main thread. Evaluated across 14 diverse workloads on both Intel and AMD platforms, CW-Ghost delivered geometric mean speedups of 1.54x (Intel) and 1.33x (AMD) over the original programs. Compared to Ghost Threading, a state-of-the-art framework, CW-Ghost improved performance by 15.8% and 10.8% on the two platforms, respectively. Remarkably, it achieved over 99% of the performance of the best hand-picked configuration from an exhaustive set, demonstrating that cache capacity constraints alone are a surprisingly effective guide for granularity selection.