Kuzco Docker

Welcome to the Kuzco Docker setup guide. This document provides detailed instructions to deploy Kuzco Workers using Docker. If you are unfamiliar with Docker, please refer to the Docker documentation (opens in a new tab). Join us on Discord if you need further assistance.

Requirements

• Windows or Linux operating system
• Any NVIDIA GPU found in our list of supported hardware
• Docker Desktop (Windows) or Docker Engine (Linux)

Getting Started

Linux Installation

1. Download and install Docker Engine for Linux (opens in a new tab) (follow the official installation guide for Ubuntu)
2. Install NVIDIA Driver using terminal:

sudo apt update
sudo apt install ubuntu-drivers-common
sudo ubuntu-drivers autoinstall

Or manually select and install a specific driver version:

sudo apt update
sudo apt install ubuntu-drivers-common
ubuntu-drivers devices
sudo apt install nvidia-driver-XXX  # Replace XXX with the recommended version number

You can check the recommended driver version for your GPU at NVIDIA's driver download page (opens in a new tab)

3. Install NVIDIA Container Toolkit for Linux (opens in a new tab)
4. Register a Kuzco account at https://kuzco.xyz/register (opens in a new tab)
5. Verify your email and connect your Discord account in Settings
6. On the dashboard, navigate to the "Workers" tab on the left
7. Click "Create Worker" in the top right-hand corner
8. Enter a name for your worker, make sure "Docker" is selected, and click "Create Worker"
9. On the Worker Details page, click "Launch Worker" in the top right-hand corner
10. Open Terminal and run the Docker container with your worker ID and code:

docker run --rm --runtime=nvidia --gpus all -e CACHE_DIRECTORY=/root/models -v ~/.kuzco/models:/root/models kuzcoxyz/amd64-ollama-nvidia-worker --worker <workerid> --code <code>

Windows Installation

1. Download and install Docker Desktop for Windows (opens in a new tab)
2. Download and install the NVIDIA Driver (opens in a new tab) for your GPU
3. Install NVIDIA Container Toolkit for Windows (opens in a new tab)
4. Register a Kuzco account at https://kuzco.xyz/register (opens in a new tab)
5. Verify your email and connect your Discord account in Settings
6. On the dashboard, navigate to the "Workers" tab on the left
7. Click "Create Worker" in the top right-hand corner
8. Enter a name for your worker, make sure "Docker" is selected, and click "Create Worker"
9. On the Worker Details page, click "Launch Worker" in the top right-hand corner
10. Open PowerShell or Windows Terminal and run the Docker container with your worker ID and code:

docker run --rm --runtime=nvidia --gpus all -e CACHE_DIRECTORY=/root/models -v ${HOME}/.kuzco/models:/root/models kuzcoxyz/amd64-ollama-nvidia-worker --worker <workerid> --code <code>

Once your worker is started, you will see it enter the "Initializing" state on the dashboard. This means that your worker is preparing to accept tasks. Depending on the speed of your GPU, this process may take up to 10 minutes, but generally only takes a minute or two.

Best Practices

Below are a few best practices to help you get the most out of your Kuzco Worker and avoid common issues.

Automatic Recovery

Docker natively supports restarting a container when it crashes via the --restart=always flag. If you would like to automatically restart your worker when it crashes, you can include this flag when you start the container.

For Linux (Terminal):

docker run --restart=always --rm --runtime=nvidia --gpus all -e CACHE_DIRECTORY=/root/models -v ~/.kuzco/models:/root/models kuzcoxyz/amd64-ollama-nvidia-worker --worker <workerid> --code <code>

For Windows (PowerShell):

docker run --restart=always --rm --runtime=nvidia --gpus all -e CACHE_DIRECTORY=/root/models -v ${HOME}/.kuzco/models:/root/models kuzcoxyz/amd64-ollama-nvidia-worker --worker <workerid> --code <code>

Make sure to replace <workerid> and <code> with your actual worker id and code.

Running Multiple Instances

If you have multiple GPUs, you can run multiple instances of your worker by starting a container for each GPU and passing the --gpus flag to the container.

For Linux (Terminal):

docker run --rm --runtime=nvidia --gpus "device=0" -e CACHE_DIRECTORY=/root/models -v ~/.kuzco/models:/root/models kuzcoxyz/amd64-ollama-nvidia-worker --worker <workerid> --code <code>

For Windows (PowerShell):

docker run --rm --runtime=nvidia --gpus "device=0" -e CACHE_DIRECTORY=/root/models -v ${HOME}/.kuzco/models:/root/models kuzcoxyz/amd64-ollama-nvidia-worker --worker <workerid> --code <code>

You will need to run this command for each GPU you have, updating the number in the device= flag to the appropriate GPU number.

Useful Commands

For both Linux (Terminal) and Windows (PowerShell):

To list all running Docker containers:

docker ps

To stop all running Docker containers:

docker stop $(docker ps -q)

To remove all stopped containers:

docker container prune

To force stop and remove all containers (running and stopped):

docker rm -f $(docker ps -aq)

Deprecation Notice

At the start of Epoch 2 (Dec 2024), we have deprecated the kuzcoxyz/worker image. Please use the kuzcoxyz/amd64-ollama-nvidia-worker image instead.