For years, we’ve been reliant on massive cloud-based AI models like ChatGPT. They’re powerful, sure, but they’re also dependent on huge data centres, raise privacy concerns, and can be subject to outages (either physically or because we don’t have signal). Cloud-based models and systems are typically centralised and at the moment this makes sense given the processing required for their usage.
But its very conceivable that you could in the not so distant future have a local LLM which rivals even some of the cloud based models in terms of capability (thinking of LLM distillation methods). Improvements in hardware and tweaks and optimisations of the models could see them running on smaller devices. New hardware like the 8060S chipsets from AMD are perhaps the beginning of that trend?
We’ve already seen this for some time in the Local LLM community – where edge models like 1B, 2B and 4B models have been made available for edge computing. I think that’s the exciting direction Ollama (and other frameworks) and the rise of Open LLMs are pointing us towards. A peer-to-peer world where you won’t be reliant on the Internet to have information available locally and information which you can query using natural language processes.
If you’ve not tried it out yet – Ollama and LM Studio are fantastic, open-source projects making it incredibly easy to download, run, and experiment with powerful Open LLMs directly on your computer. Think of it as a user-friendly interface for deploying models like Llama (Meta), Gemma (Google), Deepseek, Mistral, and many others. It handles the complexities of setting up and managing these models, letting you focus on using them. Open LLMs are Large Language Models that are released with their vectors and weights publicly available. Unlike proprietary models like GPT-4, you can actually see and understand how these models work. This transparency is crucial for research, development, and building trust. I’ve personally built a homelab workstation dedicated to LLM execution using Open models and there is plenty of support on Github projects.
Ollama and similar technologies are perhaps laying the groundwork for a future where LLM and agents aren’t just on your phone, but potentially connected to a decentralized network and peer-to-peer network of agents on various devices around you.
Check out Ollama.
Check out the model library.