What does the self-hosted ML community use day to day?

A viral post on a self-hosted AI community has revealed the practical, budget-conscious setups powering daily AI use outside of expensive cloud subscriptions. The original poster, a developer, detailed a system built around a Mac Mini M4 with 32GB of memory and a separate server laptop, with total hardware costs under $1,100. His stack uses Ollama and mlx-server to run smaller, efficient models like Gemma3:4b for quick queries and the more capable Qwen3.5-35B (in a 4-bit quantized version) for in-depth or coding tasks. For the most complex work, he strategically routes requests to Anthropic's paid Claude model via Model Context Protocol (MCP) integration, creating a cost-effective hybrid system.

The post has ignited a significant discussion, with dozens of users sharing their own configurations, highlighting a growing movement towards accessible local inference. Common themes include using quantized models (like 4-bit versions) to run larger models on consumer hardware, frameworks like Ollama and LM Studio for easy management, and a clear division of labor: local models for fast, private, or token-intensive tasks, and frontier cloud models (like GPT-4o or Claude 3.5) for top-tier reasoning. This community-driven exploration is proving that with tools like Qwen 2.5, Llama 3.2, and DeepSeek, professional-grade AI assistance is increasingly feasible without a massive GPU cluster, lowering the barrier to entry and fostering innovation in personal AI workflows.