How DeepSeek Runs a 284B LLM on a Laptop

“A 284-billion-parameter AI model running on a laptop?”

A year ago, that sentence would have sounded like clickbait.

Today, it’s reality.

The surprising part isn’t just that DeepSeek V4 Flash (284B parameters) runs locally, it’s that it runs at usable speeds on a 128GB MacBook Pro without needing a data center or a rack of GPUs.

Even more surprising?

The secret isn’t more powerful hardware.

It’s better engineering.

Thanks to DS4 (DwarfStar 4), created by Redis creator Salvatore Sanfilippo (antirez), one of the biggest AI models available can now run locally using techniques that completely rethink how inference engines should work. DS4 isn’t another wrapper around an existing runtime — it’s a purpose-built inference engine focused on one model: DeepSeek V4 Flash.

Let’s break down why this is such a big deal.

When people hear 284 billion parameters, they immediately think:

Normally, that’s true.

Traditional dense language models activate every parameter for every generated token.

That means:

Running something this large on consumer hardware would normally be impossible.

Yet DeepSeek somehow does it.

The answer starts with its architecture.

One of the biggest misconceptions is assuming that a 284B model computes all 284 billion parameters every time it generates a token.

It doesn’t.

DeepSeek V4 Flash is a Mixture-of-Experts (MoE) model.

Instead of activating every expert, the router chooses only a small subset for each token.

In practice, only around 13B parameters become active during inference, while the remaining experts stay idle. That gives developers the knowledge capacity of a massive model without paying the computational cost of using all 284B parameters at once.

Think of it like a hospital.

You don’t call every doctor every time a patient arrives.

You route the patient to the specialist they actually need.

DeepSeek does exactly that with neural networks.

This is the first reason a 284B model becomes practical.

Even activating only 13B parameters is expensive.

That’s where DS4 introduces another clever optimization.

Instead of quantizing the entire model equally, DS4 applies asymmetric quantization.

Rather than compressing everything, it compresses only the routed MoE experts aggressively while leaving the important shared components, routing layers, and projections at higher precision. The routed experts are stored using specialized 2-bit formats, while critical components remain at 8-bit to preserve quality.

This selective approach dramatically reduces memory usage without the severe quality loss you’d expect from uniform low-bit quantization.

Because not every parameter contributes equally.

Some tensors are critical.

Others are surprisingly tolerant to compression.

DS4 takes advantage of this observation.

Instead of asking:

It asks:

That mindset changes everything.

The result is a model that occupies dramatically less memory while maintaining impressive coding and reasoning quality.

Many people initially assumed DS4 was simply another frontend for llama.cpp.

It isn’t.

In fact, the project intentionally avoids becoming a generic inference framework.

Instead, DS4 is laser-focused on one model:

This specialization allows optimizations that general-purpose engines can’t easily implement.

Sometimes doing one thing exceptionally well beats supporting everything.

Another feature that deserves attention is DeepSeek’s 1 million token context window.

Normally, long contexts explode memory usage because the KV cache grows continuously.

Most inference engines assume:

DS4 challenges that assumption.

Instead of treating RAM as the only place for the cache, it treats fast SSD storage as a first-class citizen for persistent KV cache when appropriate. Combined with DeepSeek’s already compressed KV cache, this makes much longer conversations practical on personal machines.

It’s a subtle architectural shift with huge practical implications.

Benchmarks alone rarely tell the full story.

What matters is whether developers can actually use the model.

DS4 reports generation speeds of around 26 tokens per second on a 128GB MacBook Pro M3 Max, which is fast enough for interactive coding, chat, and agent workflows.

That means:

For many developers, that’s the difference between experimentation and daily usage.

The real innovation isn’t just another inference engine.

It’s the philosophy behind it.

For years, local AI focused on shrinking models.

DS4 flips the problem.

Instead of making models smaller, it asks:

That’s a completely different direction.

As larger MoE models become more common, we may see more inference engines optimized for specific architectures rather than generic runtimes.

One interesting detail from the project is that antirez openly states DS4 was developed with strong assistance from GPT-5.5, while humans remained responsible for the ideas, testing, debugging, and overall direction. He also emphasizes that the project builds on the foundations laid by llama.cpp and GGML.

It’s a practical example of how experienced engineers can use AI to accelerate development without replacing engineering judgment.

DS4 teaches an important engineering lesson.

Performance isn’t always about buying better hardware.

It’s often about asking better questions.

Instead of accepting conventional assumptions, the project rethinks:

Those questions produced a breakthrough.

Running a 284B LLM on a laptop sounds impossible until you understand the engineering behind it.

The magic isn’t hidden inside expensive GPUs.

It’s a combination of:

Together, these ideas make frontier-scale local AI far more accessible than many developers imagined.

The biggest lesson isn’t that DeepSeek runs on a laptop.

It’s that thoughtful systems engineering can redefine what’s possible often more than raw hardware ever could.

https://github.com/antirez/ds4

https://antirez.com/news/165 

I hope you found it helpful and informative. If you have any questions or feedback, feel free to leave a comment below. Your support and engagement mean a lot to me.