Amazon SageMaker Async Inference now supports inline payloads, skipping S3 uploads

Amazon has announced inline payload support for SageMaker AI Async Inference, allowing customers to send inference payloads directly in the request body of the InvokeEndpointAsync API. Previously, every async invocation required two steps: upload the payload to an S3 bucket, then call the API with the S3 URI as InputLocation. The new Body parameter accepts raw bytes up to 128,000 bytes, and is mutually exclusive with InputLocation. This removes an entire network round-trip, simplifies client-side code (no S3 client, UUID generation, or IAM permissions for the input path), and reduces operational overhead for small payload workloads. The output behavior remains unchanged—results are still written to the configured S3 OutputLocation. The feature works with existing async endpoints without model or container changes, and is available in 31 commercial AWS regions. It is particularly beneficial for applications with small input payloads (in KB) that need longer processing times than real-time inference allows, such as bursty or batch-style workloads with automatic scaling to zero.

From a developer experience perspective, the difference is stark. Before, a Python invocation required importing boto3 S3 and sagemaker-runtime clients, uploading a payload with a UUID key to an S3 bucket, and then calling invoke_endpoint_async with InputLocation. Now, a single call to invoke_endpoint_async with Body does the job—no extra imports, no S3 bucket or IAM grants needed. This reduces latency and cost per request, and eliminates the need to manage stale input objects. For teams running many small async inference calls, the savings in code complexity and operational burden are significant. AWS notes that payloads under 128KB (like JSON prompts) are the sweet spot, while larger payloads should still use the S3 path. Error handling is synchronous: requests that exceed the size limit or provide both Body and InputLocation return a ValidationError immediately. This update makes SageMaker Async Inference more accessible for serverless, event-driven AI workloads where simplicity and speed matter.