Today, Guest Speaker Justice Peltier gave a talk on hosting LLMs on your own local device! He gave us an intro on benefits of self-hosting, different self-hosting methods (bare metal and containerized), models and model runners, adding a GUI, and some ways you put your local models to use.
He also gave us a demo of different open models, such as Ollama, DeepSeek, and HuggingFace models.
If you would like an overview of the talk, please see Justice’s GitHub Repository, which has a guide to self hosting LLMs and other helpful resources!