With a decent computer, you can run a local LLM.

The full DeepSeek R1 requires 768GB of memory, which is beyond the reach of most home users.

But that doesn't mean that you can't get started with "distilled" models, which can still pack a punch.

Especially, if you run specialized models. Imagine the full R1 model with 671B parameters. How much of that is programming knowledge?

There are dedicated coding models that are pretty usable.

The big 671B model has also been used to train smaller variants. DeepSeek is available as a, 1.5b, 7b, 8b, 14b, 32b, 70b, and 671b variants.

The easiest way to get started is by installing ollama. Simply download for your architecture and you can use the command line to access many of the models.

Use ollama pull to download a model. The deepseek-r1:14b is around ~9GB.

Once it's down loaded, simply start it with ollama run deepseek-r1:14b and you're greeted by a prompt. That's it. You can start chatting.

The R1 is a thinking model, like ChatGPT o1, but with a difference that you can actually see the full thinking process.

Pretty cool.

Now the command line is a fast way to get started, but not the most convenient.

For that I've got two favorites right now.

​LM Studio, which allows you to easily install local models from a wide selection of different general purpose and specialized ones, in different sizes, depending on your hardware spec.

And another tool which I like, is Chatbox, which is not only used for local models, but you can connect to your favoritee spyware model that sucks your data too, like ChatGPT or Claude.

It's fine to play around, and although, you'll get better results with a paid online model, the privacy benefits to having a local LLM is worth it for some tasks.

But the coolest thing is that local LLMs are becoming better and better and with optimizations trickling down, we might see open-source model take over.

And with the reduced compute requirements and better quality distilled models, we're not far from a future where running your own AI model locally is a part of a your normal workflow.

P.S. I forgot to mention that some of these models can act like a server where you can connect a coding assistant to work with it through VS Code. But that's for a future article.

Yours,

Taj