Kuzco Docker

Welcome to the Kuzco Docker setup guide. This document provides detailed instructions to deploy Kuzco Workers using Docker. If you are unfamilar with Docker, please refer to the Docker documentation (opens in a new tab). Join us on Discord if you need further assistance.

Requirements

• Linux OS
• Any NVIDIA GPU found in our list of supported hardware
• Docker.

Getting Started

1. Register a Kuzco account at https://kuzco.xyz/register (opens in a new tab)
2. Verify your email
3. On the dashboard, navigate to the "Workers" tab on the left.
4. Click "Create Worker" in the top right-hander corner.
5. Enter a name for your worker, make sure "Docker" is selected, and click "Create Worker".
6. On the Worker Details page, click "Launch Worker" in the top right-hander corner.
7. Follow the instructions in the pop-up window to run the Kuzco Docker container.
8. The last step should be running the docker run ... command with your worker id and code.

Once your worker is started, you will see it enter the "Initializing" state on the dashboard. This means that your worker is preparing to accept tasks. Depending on the speed of your GPU, this process may take up to 10 minutes, but generally only takes a minute or two.

Best Practices

Below are a few best practices to help you get the most out of your Kuzco Worker and avoid common issues.

Automatic Recovery

Docker natively supports restarting a container when it crashes via the --restart=always flag. If you would like to automatically restart your worker when it crashes, you can include this flag when you start the container.

For example:

 docker run --restart=always --rm --runtime=nvidia --gpus all -e CACHE_DIRECTORY=/root/models  -v ~/.kuzco/models:/root/models kuzcoxyz/amd64-ollama-nvidia-worker --worker <workerid> --code <code>

Make sure to replace <workerid> and <code> with your actual worker id and code.

Running Multiples Instances

If you have multiple GPUs, you can run multiple instances of your worker by starting a container for each GPU and passing the --gpus flag to the container.

To start a container with GPU #0, you can use the following command:

docker run --rm --runtime=nvidia --gpus "device=0" -e CACHE_DIRECTORY=/root/models  -v ~/.kuzco/models:/root/models kuzcoxyz/amd64-ollama-nvidia-worker --worker <workerid> --code <code>

You will need to run this command for each GPU you have, updating the number in the device= flag to the appropriate GPU number.

Useful Commands

To list all running Docker containers:

docker ps

To stop all running Docker containers:

docker stop $(docker ps -q)

Deprecation Notice

As the start of Epoch 2 (Dec 2024), we have deprecated the kuzcoxyz/worker image. Please use the kuzcoxyz/amd64-ollama-nvidia-worker image instead.