[D] How's MLX and jax/ pytorch on MacBooks these days?

Apple's MLX framework has emerged as a game-changer for AI development on MacBooks, specifically optimized for the company's M-series silicon. Unlike generic frameworks like PyTorch or JAX, MLX is designed from the ground up to leverage Apple's unified memory architecture and Neural Engine accelerators. This means developers can now train and fine-tune medium-sized language models (like 7B-13B parameter LLMs) directly on their laptops, with performance that often surpasses what's possible with adapted versions of other frameworks. The key advantage is native integration with Apple's hardware, allowing efficient use of both GPU cores and the dedicated neural processing units.

For hardware selection, the consensus among developers is clear: the M4 Max with its higher GPU core count (up to 30 cores) and significantly faster memory bandwidth (400GB/s) provides substantially better performance for ML workloads compared to the M5 Pro. While the M5's CPU improvements are marginal, the Neural Engine enhancements do accelerate specific matrix operations common in transformer models. However, for most training and inference tasks, the M4 Max's raw GPU power and memory throughput deliver better results. This makes modern MacBooks, particularly Max variants, surprisingly capable machines for local AI experimentation, fine-tuning, and running multiple AI agents simultaneously.

The practical implications are significant for software developers and ML practitioners. Instead of relying solely on cloud GPUs for development work, they can now prototype, test, and even deploy smaller models entirely locally. This reduces costs, improves iteration speed, and maintains data privacy. While large-scale training of foundation models still requires cloud infrastructure, the ability to work with 7B-13B parameter models locally represents a major shift in accessibility. Combined with containerization tools that now support Apple Silicon, developers can create complete AI development environments on their MacBooks.