Skip to content

Ollama: Local AI Model Server

Ollama is a tool that allows you to run large language models (LLMs) locally on your own server or computer. It simplifies the process of downloading, setting up, and interacting with powerful open-source AI models, providing AI capabilities without relying on third-party cloud services and ensuring data privacy.

Key Features

  • Run LLMs Locally: Host and run various open-source large language models (like Llama, Gemma, Mistral, etc.) on your own hardware.
  • Simple CLI: Easy-to-use command-line interface for downloading models (ollama pull), running them (ollama run), and managing them (ollama list).
  • API Server: Ollama serves models through a local API, allowing other applications (like OpenWebUI) to interact with them.
  • Data Privacy: Since models run locally, your data doesn't leave your server when you interact with them.
  • Growing Model Library: Access a growing library of popular open-source models.
  • Customization: Create custom model files (Modelfiles) to tailor model behavior.

Getting Started with Ollama (within Changemaker)

Ollama itself is primarily a command-line tool and an API server. You typically interact with it via a terminal or through a UI like OpenWebUI.

Managing Ollama via Terminal (e.g., in Code Server)

  1. Access a Terminal:

    • Open the integrated terminal in Code Server.
    • Alternatively, SSH directly into your Changemaker server.

  2. Common Ollama Commands:

    • List Downloaded Models: See which models you currently have. 

      docker exec -it ollama-changemaker ollama list
      (The docker exec -it ollama-changemaker part is necessary if Ollama is running in a Docker container named ollama-changemaker, which is common. If Ollama is installed directly on the host, you'd just run ollama list.)

    • Pull (Download) a New Model: Download a model from the Ollama library. Replace gemma:2b with the desired model name and tag. 

      docker exec -it ollama-changemaker ollama pull gemma:2b
      (Example: ollama pull llama3, ollama pull mistral)

    • Run a Model (Interactive Chat in Terminal): Chat directly with a model in the terminal. 

      docker exec -it ollama-changemaker ollama run gemma:2b
      (Press Ctrl+D or type /bye to exit the chat.)

    • Remove a Model: Delete a downloaded model to free up space. 

      docker exec -it ollama-changemaker ollama rm gemma:2b

Interacting with Ollama via OpenWebUI

For a more user-friendly chat experience, use OpenWebUI, which connects to your Ollama service. See the apps/openwebui.md documentation for details.

Use Cases within Changemaker

  • Powering OpenWebUI: Ollama is the backend engine that OpenWebUI uses to provide its chat interface.
  • AI-Assisted Content Creation: Generate text, summaries, ideas, or code snippets with privacy.
  • Custom AI Applications: Developers can build custom applications that leverage the Ollama API for various AI tasks.
  • Offline AI Capabilities: Use AI models even without an active internet connection (once models are downloaded).

Editing the Site

Ollama is an AI model server. It is not used for editing this documentation site. Site editing is done via Code Server.

Further Information