LayoutBench benchmark shows tar archives fastest for cloud ML data retrieval
Benchmark of 11 queries on ImageNet finds tar layouts beat Parquet for latency
LayoutBench is introduced as the first benchmark designed to evaluate how cloud storage layouts affect multimedia data retrieval for ML workloads. It compares three layout strategies—individual objects (L1), tar archives (L2), and Parquet columns (L3)—using 11 queries on ImageNet across six AWS EC2 instance configurations. The results show that L2 achieves lower latency than L1 and L3 through connection reuse, but loses that advantage for very large retrievals. L3 is fastest for very large retrievals, but transfers substantially more data across all query sizes and requires significantly more memory. Across all layouts, data transfer cost dominates total expenditure, with L3 costing an order of magnitude more than L1 or L2.
- LayoutBench is the first benchmark for cloud storage layout performance in multimedia ML, testing 3 layouts across 11 queries and 6 EC2 instance types
- Tar archives (L2) deliver lowest latency via connection reuse for small-to-medium retrieval, but lose the edge for very large result sets
- Parquet (L3) is fastest for large retrievals but transfers up to 10x more data, making storage cost an order of magnitude higher than tar or individual objects
Why It Matters
Helps ML engineers choose storage layouts that cut retrieval latency and cloud costs for large-scale multimedia datasets.