Run o1 at Home Privately: Think-Respond Pipe Tutorial with Open-webui & Ollama

Running advanced AI models like o1 on your personal computer is now more accessible than ever, thanks to tools like Open WebUI and Ollama and models like QwQ. This guide will walk you through setting up the Think-Respond Chain Pipe, enabling you to perform complex reasoning tasks locally.

Prerequisites

Before diving into the setup, ensure your system meets the following requirements:

• Hardware: At least 8 GB of RAM and a modern multi-core processor and a beefy GPU.

• Software:

  • Docker: For containerized application deployment.

  • Ollama: For managing large language models locally.

If you already have the open-webui and Ollama setup skip to step 4

Step 1: Install Docker

Docker simplifies application deployment by packaging applications into containers. To install Docker on linux, for windows go here :

1. Update Package Index:

    sudo apt update
   

2. Install Docker:

    sudo apt install docker.io
   

3. Start and Enable Docker Service:

    sudo systemctl start docker
    sudo systemctl enable docker
   

For more detailed instructions, refer to the Docker installation guide for Ubuntu.

Step 2: Install Ollama

Ollama is a command-line tool that simplifies working with large language models. To install Ollama, or go here to the ollama set up page:

1. Download and Run the Installation Script:

    curl -fsSL https://ollama.com/install.sh | sh
   

2. Verify Installation:

    ollama --version
   

For additional guidance, consult the Ollama installation tutorial.

Step 3: Install Open WebUI

Open WebUI provides a user-friendly interface to interact with AI models. To set it up or go here to the open-webui setup:

1. Pull the Open WebUI Docker Image:

    docker pull ghcr.io/open-webui/open-webui:main
   

2. Run the Open WebUI Container:

    docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main
   

This command sets up Open WebUI to be accessible at http://localhost:3000. For more information, visit the Open WebUI documentation.

Step 4: Configure the Think-Respond Chain Pipe

With Open WebUI and Ollama installed, proceed to configure the Think-Respond Chain Pipe:

1. Access the Admin Panel:

   • Navigate to http://localhost:3000 in your web browser.

   • Log in with your credentials.

2. Add the Pipe Manifold:

   

   • Click on the top right with your initials.

     

   • Go to the "Functions" tab in the Admin Panel.

   • Click the '+' button to add a new function.

   • Enter the following details:

     • Name: Think-Respond Chain Pipe

     • Description: Pipe that performs internal reasoning steps before producing a final response.

     • Copy latest version of the Code: https://openwebui.com/f/latentvariable/o1_at_home

       

3. Enable the Pipe Manifold:

   • After adding the function, enable it to make it active.

     

4. Customize Settings:

   • Click the settings cog next to the newly added function.

     

   • Configure the following options:

     

     • Select Models: Choose your desired thinking and responding models.

     • Toggle API: Choose between OpenAI API or Ollama API.

     • Show Reasoning: Enable or disable the display of the reasoning process.

     • Set Thinking Time: Specify the maximum time (in seconds) allowed for the reasoning model to process.

5. Save and Apply:

   • Save your settings to apply the changes.

   • The Think-Respond Chain Pipe should now be available in your dropdown menu.

Conclusion

By following these steps, you've set up a robust environment to run o1 at home using the Think-Respond Chain Pipe. This setup leverages Open WebUI and Ollama to perform complex reasoning tasks locally, ensuring both efficiency and privacy.

For further assistance or to explore advanced configurations, refer to the Open WebUI documentation and the Ollama tutorial.

Note: Ensure you have the necessary API keys and access rights for the models you intend to use.

Author: https://github.com/latent-variable