StepX-Edge's 0.9B UI model runs on phones with 98 tok/s and 1.4 GB memory
A tiny 0.9B model matches 2.3B rivals on screen understanding and OCR.
Deploying a vision-language model with full UI understanding on mobile devices has long been a trade-off between accuracy and resource constraints—until now. StepX-Edge, a 0.9B-parameter model, smashes this barrier using a three-layer co-design of architecture, training, and deployment. Architecturally, it introduces UI-aware Layered Visual Encoding (ULVE) and a Progressive Dimensionality Projection (PDP) connector to handle extreme screen aspect ratios and fine-grained perception. A five-stage StepX-Curriculum training framework ensures all four UI subtasks (OCR, screen understanding, VQA, element grounding) grow synergistically despite the tight parameter budget.
On deployment, a module-wise differentiated two-stage PTQ-to-QAT quantization scheme keeps accuracy loss under 1%. The result: StepX-Edge outperforms all models under 1B parameters and matches 1.3B–2.3B general VLMs on benchmarks like RefCOCO (92.0%) and OCRBench v1 (831). After W4A16+KV8 quantization, it runs on Snapdragon 8 Gen5 devices with ~0.84s time-to-first-token, 98 tokens/second decode, and only 1.4 GB peak memory. The team plans to open-source training data, the full recipe, and the deployment pipeline, making on-device UI AI practical for the first time.
- 0.9B-parameter model achieves 88.76 F1 on ScreenQA and 57.25 on Chinese OCRBench v2, surpassing all <=1B models and matching 2.3B baselines.
- Runs on Snapdragon 8 Gen5 with 0.84s TTFT, 98 tok/s decode, and only 1.4 GB peak memory after W4A16+KV8 quantization.
- Open-source release includes training data, full recipe, and quantization pipeline for on-device deployment.
Why It Matters
Brings full UI understanding to mobile devices without cloud dependency, enabling faster, private screen assistants.