M4 Mac mini 24GB local AI: What models can it run?

A developer running local LLMs on the new M4 Mac mini with 24GB of memory reports that 7B and 13B models run smoothly, but 30B+ models crash the system. This reveals a clear memory ceiling that Apple's unified architecture cannot overcome without a hardware upgrade.

A developer tested local LLMs on an M4 Mac mini with 24GB unified memory, finding that 7B and 13B parameter models run well but 30B+ models are impossible due to memory constraints.
This confirms that Apple's unified memory architecture is excellent for small, private inference but cannot match Nvidia's VRAM capacity for large models.
The key tension: Apple offers a seamless, low-power local AI experience for developers, but the 24GB ceiling limits its utility for serious AI research or production workloads.

How far can a 24GB M4 actually run local models?

According to a developer post on Hacker News, the M4 Mac mini with 24GB of unified memory can run a 7B parameter model (like Llama 3 8B) at 4-bit quantization with a token generation speed of ~40 tokens per second. A 13B model (like CodeLlama 13B) at 4-bit also runs, but at a slower ~20 tokens per second. The developer reported that the system uses about 12-16GB of memory for these models, leaving only 8-12GB for macOS and other applications. This is a comfortable operating zone for single-user, interactive use.

However, the developer stated that attempting to load a 30B+ model (like Yi-34B) at 4-bit quantization failed outright, with the system running out of memory and crashing the model loader. The 24GB ceiling is a hard limit: to run a 30B model, you need at least 16-20GB of contiguous VRAM, and the M4's unified memory must also host the OS and browser, leaving insufficient headroom.

M4 Mac mini 24GB: Local AI is real, but hits a 24GB wall

Why does Apple's unified memory matter for LLMs?

Apple's unified memory architecture is the secret sauce. Unlike a typical PC where the CPU and GPU have separate RAM pools (e.g., 32GB system RAM + 8GB VRAM), the M4's 24GB is a single pool accessible by both the CPU and GPU at full bandwidth (~120 GB/s on the M4 Pro). This eliminates the need to copy data between pools, which is a major bottleneck on Nvidia systems. The developer noted that this unified access allows for faster prompt processing and lower latency for small models, as the entire model can fit in the shared memory without paging.

But the trade-off is stark: you cannot upgrade the memory. The developer explicitly said, "If you want to run 30B+ models, you need a Mac with at least 64GB of memory, which means jumping to a Max or Ultra chip." This is a deliberate product segmentation by Apple, forcing users who need more capacity to pay a significant premium for higher-tier chips.

How does the M4 stack up against an Nvidia RTX 4090?

For local AI, the direct competitor to the M4 Mac mini is a PC with an Nvidia RTX 4090 (24GB VRAM). Both have 24GB of memory, but the architectures are fundamentally different. The RTX 4090 has 24GB of dedicated GDDR6X VRAM with a bandwidth of ~1 TB/s, which is nearly 10x the M4's bandwidth. For large models that fit within 24GB, the RTX 4090 is dramatically faster for token generation (150+ tokens per second for a 13B model).

However, the M4's advantage is power consumption and ecosystem. The M4 Mac mini idles at ~10W and peaks at ~65W, while an RTX 4090 system can draw 450W at load. For always-on, local inference, the M4 is far more energy-efficient. The developer noted that for interactive coding or chat, the M4's ~20 tokens per second is "fast enough" for a natural conversation, and the silence and low heat are significant benefits for a desktop environment.

Metric	M4 Mac mini (24GB)	PC + RTX 4090 (24GB)
Memory Type	Unified (CPU+GPU)	Dedicated GDDR6X VRAM
Memory Bandwidth	~120 GB/s	~1,008 GB/s
Max Model Size (4-bit)	13B (practical)	30B (practical)
Token Speed (13B model)	~20 tok/s	~150 tok/s
Power Consumption (load)	~65W	~450W
Verdict	Best for low-power, small-model inference	Best for high-speed, large-model inference

My thesis: The M4 Mac mini with 24GB is a fantastic local AI appliance for the enthusiast developer, but it is not a serious workstation for AI research or production.

In the short term, this device will enable a wave of experimentation with small, private models for coding assistants, local chat, and data analysis. The developer community will embrace it for its silence, low power, and seamless macOS integration. Tools like Ollama, LM Studio, and llama.cpp already support Apple Silicon well, and the M4's performance for 7B-13B models is genuinely good.

But in the long term, the 24GB ceiling is a trap for anyone who wants to scale. The developer's report of a 30B model crashing is a clear warning. Apple's product segmentation means that moving to 48GB or 64GB requires buying a much more expensive M4 Max or Ultra chip, which may not offer a proportional performance gain for the price. The real winner here is Nvidia, which continues to offer the only path to running larger models locally without a hardware upgrade. The loser is the developer who buys the 24GB M4 hoping to "grow into" larger models — they will hit a wall.

My concrete prediction: Within 12 months, Apple will be forced to offer a 32GB option on the base M5 Mac mini to stay competitive for the local AI crowd, or risk losing this emerging market to Windows-on-ARM laptops with Snapdragon X Elite chips that may offer more memory flexibility.

Apple will release a 32GB configuration of the M5 Mac mini by Q2 2026 to address the memory ceiling for local AI workloads.
By December 2025, at least two major open-source model providers (e.g., Meta with Llama 4, or Mistral) will release models specifically optimized for 24GB Apple Silicon, using 3-bit or 2-bit quantization to fit larger architectures.
The Snapdragon X Elite will capture 15% of the local AI developer market by 2027 if Qualcomm offers 32GB+ memory configurations at a lower price point than Apple's Max/Ultra chips.

Maximum Practical Model Size (4-bit) by Hardware (estimated)

The M4's 24GB limit is a hard ceiling for 30B+ models, making it unsuitable for serious AI work beyond experimentation.
Apple's unified memory advantage is real but bounded; bandwidth is the bottleneck, not just capacity.
Nvidia remains the undisputed king of local AI for anyone who needs speed or capacity beyond 13B models.
The developer who buys the 24GB M4 for local AI should plan for it to be a single-user, small-model appliance, not a future-proof workstation.
Apple's product segmentation is a deliberate strategy to push power users toward the more expensive Max/Ultra chips, but it risks ceding the mid-range AI market to ARM competitors.