Could one theoretically install DeepSeek onto a home server, thereby having a completely private LLM for themselves? Oh the possibilities (if so). Future seems interesting from those who’ve invested in having a smart home. - ThreadSky

mitchellwatts.com • 108 days ago

Could one theoretically install DeepSeek onto a home server, thereby having a completely private LLM for themselves?

Oh the possibilities (if so). Future seems interesting from those who’ve invested in having a smart home.

Comments

bartboeman.bsky.social•108 days ago

It is the preferred way. The quality (/precision) is a bit lower with one of the smaller distilled DeelSeek-R1 model versions, but completely save.

Several ways, but most simple: Install Ollama (https://ollama.com/) and pick a model that fits. A proper GPU and lots of VRAM helps.

https://ollama.com/library/deepseek-r1

mitchellwatts.com•108 days ago

I just installed DeepSeek onto my MacAir this afternoon (have yet to put it through its paces). But the thought of running this as a private assistant for home and personal use in the near future is an exciting proposition.

bartboeman.bsky.social•108 days ago

With a 'private' solution you can be pretty sure data is not stored and used by others. That can be different with online Chat and through their API:
https://chat.deepseek.com/downloads/DeepSeek%20Privacy%20Policy.html

mitchellwatts.com•108 days ago

I am fairly certain that I have a completely private installation, having utilized Ollama to pull DeepSeek's R1:32b model. However, I'm admittedly very new to this (installing an LLM via terminal, for instance), so my ignorance could prove me wrong here.

bartboeman.bsky.social•108 days ago

I'll think you manage. If you have the hardware for the 32B version, you could use the given uncensored.

For inference you can use the default or https://chatboxai.app/en or https://github.com/gluonfield/enchanted or - what many like - https://openwebui.com etc.

Some config for a home network:
https://github.com/ollama/ollama/issues/703

bartboeman.bsky.social•108 days ago

PS

To use that given model, try something like:

ollama run https://hf.co/bartowski/deepseek-r1-qwen-2.5-32B-ablated-GGUF:Q5_K_M

mitchellwatts.com•108 days ago

Ooo, I haven’t heard of enchanted. Thank you! I’m currently exploring similar to help make the IX of using these models more pleasant.

toddcoleman.bsky.social•108 days ago

like JARVIS?

mitchellwatts.com•108 days ago

Exactly!

I wonder how long until someone creates an open source way to speak with it, or any other open source LLM.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply