ServerlessT2I boosts text-to-image serving 2x with same GPU budget

Text-to-image (T2I) workflows on serverless platforms have traditionally been deployed as monolithic GPU functions, bundling all models together. This obscures the workflow structure, increases scaling overhead, and forces users to manage low-level GPU coordination. ServerlessT2I, presented by researchers, decomposes these workflows into loosely coupled model functions that can be independently managed and scheduled. This decomposition enables per-model scaling, declarative workflow composition, transparent GPU-resident communication, and fairness-aware scheduling in multi-tenant clusters.

To make this decomposition efficient without sacrificing performance, ServerlessT2I harvests slack GPU memory left idle by compute-bound T2I inference. It builds a data plane that reduces model loading and data communication overheads. The system also introduces a fair scheduler for multi-tenant serving. In tests with production traces, ServerlessT2I sustains up to 2× higher request rates than existing T2I workflow serving systems using the same GPU budget. Alternatively, for a fixed request rate, it saves up to 3× GPU resources while still satisfying service level objectives (SLOs). This approach promises significant cost and performance improvements for AI image generation in cloud environments.