Branch Flip Analysis finds 21 hidden performance bugs in major DBMSs

Researchers Jinsheng Ba and Zhendong Su propose a novel white-box methodology called Branch Flip Analysis (BFA) to systematically uncover performance issues in database management systems (DBMSs). Unlike traditional black-box methods that compare execution consistency, BFA flips code branches to enforce or disable specific optimizations. The key insight: if disabling an optimization significantly improves performance, a performance bug exists because the optimization is actually degrading performance. This fine-grained approach allows testing of optimization logic in ways not possible before.

The team implemented BFA in a prototype called QueryZen and evaluated it on four mature and widely-used DBMSs: PostgreSQL, MySQL, CockroachDB, and MariaDB. Using the industry-standard benchmarks TPC-H and TPC-DS, QueryZen discovered 21 previously unknown and unique performance issues. The methodology is designed to be simple and broadly applicable, extending beyond databases to analyze performance issues in other complex software systems. As performance is critical for fundamental infrastructure, BFA offers a new perspective for developers to identify and fix dormant code paths that cause suboptimal performance before they impact users.