Installation

Get igllama up and running on your system. This guide covers installation on Windows, Linux, and macOS, including GPU backend configuration for accelerated inference.

Prerequisites

Before installing igllama, ensure you have the following dependencies:

  • Zig 0.15 or later - Required for building from source
  • Git - For cloning the repository with submodules
  • CMake - Required for some GPU backend builds
  • C++ compiler - GCC, Clang, or MSVC depending on platform

Installing Zig

Linux

# Using package manager (Ubuntu/Debian)
sudo apt install zig

# Or download pre-built binary
wget https://ziglang.org/download/0.15.0/zig-linux-x86_64-0.15.0.tar.xz
tar -xf zig-linux-x86_64-0.15.0.tar.xz
sudo mv zig-linux-x86_64-0.15.0 /opt/zig
sudo ln -s /opt/zig/zig /usr/local/bin/zig

macOS

# Using Homebrew
brew install zig

# Or download from ziglang.org
brew install --cask zig

Windows

# Using Windows Package Manager
winget install zig.zig

# Or download installer from ziglang.org
# Add to PATH manually if needed
setx PATH "%PATH%;C:\Program Files\zig"

Verify installation:

zig version  # Should show 0.15.0 or later

Quick Start

The fastest way to get started:

# Clone with submodules (required for GGUF support)
git clone --recursive https://github.com/bkataru/igllama.git
cd igllama

# Build release version
zig build -Doptimize=ReleaseFast

# Run with a GGUF model
./zig-out/bin/igllama --model path/to/model.gguf

Note: GGUF stands for “Georgi Gerganov Unified Format”, the standard model format for llama.cpp-compatible inference engines.

Platform-Specific Instructions

Linux Installation

Ubuntu/Debian

# Install dependencies
sudo apt update
sudo apt install -y build-essential cmake git zig

# Clone and build
git clone --recursive https://github.com/bkataru/igllama.git
cd igllama
zig build -Doptimize=ReleaseFast

# Optional: Install to system path
sudo cp ./zig-out/bin/igllama /usr/local/bin/

Fedora/RHEL

sudo dnf install -y zig cmake git gcc-c++
git clone --recursive https://github.com/bkataru/igllama.git
cd igllama
zig build -Doptimize=ReleaseFast

macOS Installation

# Install Xcode Command Line Tools
xcode-select --install

# Install dependencies via Homebrew
brew install zig cmake git

# Clone repository
git clone --recursive https://github.com/bkataru/igllama.git
cd igllama

# Build with Metal acceleration (macOS only)
zig build -Doptimize=ReleaseFast -Dgpu=metal

# Test installation
./zig-out/bin/igllama --version

Windows Installation

Using PowerShell

# Install zig via winget (if not already installed)
winget install zig.zig

# Clone repository
git clone --recursive https://github.com/bkataru/igllama.git
cd igllama

# Build with Visual Studio toolchain
zig build -Doptimize=ReleaseFast -Dtarget=x86_64-windows

# Run
.\zig-out\bin\igllama.exe --model models\model.gguf

Using WSL2

# Install WSL2 dependencies
sudo apt update
sudo apt install -y build-essential cmake git zig

# Build as Linux binary
git clone --recursive https://github.com/bkataru/igllama.git
cd igllama
zig build -Doptimize=ReleaseFast

GPU Backend Configuration

igllama supports multiple GPU backends for accelerated inference. Choose based on your hardware.

Metal (macOS)

Metal is the recommended backend for Apple Silicon and Intel Macs.

# Build with Metal support
zig build -Doptimize=ReleaseFast -Dgpu=metal

# Verify Metal devices
./zig-out/bin/igllama --info

Requirements:

  • macOS 10.15 (Catalina) or later
  • Xcode Command Line Tools

Vulkan (Linux/Windows)

Vulkan provides cross-platform GPU acceleration for NVIDIA, AMD, and Intel GPUs.

Linux Vulkan

# Install Vulkan SDK
sudo apt install -y libvulkan-dev mesa-vulkan-drivers

# Or for latest SDK
wget -qO - https://packages.lunarg.com/lunarg-signing-key-pub.asc | sudo apt-key add -
sudo apt install -y vulkan-sdk

CUDA (Experimental)

CUDA backend for NVIDIA GPUs (experimental support).

# Prerequisites
# - NVIDIA GPU with CUDA support
# - CUDA Toolkit 12.0+
# - NVIDIA driver 525+

# Install CUDA Toolkit (Linux example)
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt update
sudo apt install -y cuda-toolkit-12-0

# Build with CUDA
zig build -Doptimize=ReleaseFast -Dgpu=cuda -Dcuda-arch=sm_80

# Set CUDA path if not in default location
export CUDA_HOME=/usr/local/cuda

Supported CUDA architectures:

  • sm_70 - Volta (V100)
  • sm_75 - Turing (T4, RTX 2000)
  • sm_80 - Ampere (A100, RTX 3000)
  • sm_86 - Ampere (RTX 3000 mobile)
  • sm_89 - Ada Lovelace (RTX 4000)

Build Configuration Options

Customize your build with these flags:

# Debug build with symbols
zig build -Doptimize=Debug

# Release with fast optimizations
zig build -Doptimize=ReleaseFast

# Release with small binary size
zig build -Doptimize=ReleaseSmall

# Custom target triple
zig build -Dtarget=aarch64-macos

# Disable specific features
zig build -Dgpu=none -Dfeatures=none

# Static linking (Linux)
zig build -Doptimize=ReleaseFast -Dtarget=x86_64-linux-musl

Verification

After installation, verify everything works:

# Check version
igllama --version

# List available GPU devices
igllama --info

# Test inference with small model
igllama --model models/tinyllama-1.1b.gguf --prompt "Hello" --n-predict 10

Troubleshooting

Common Issues

Zig version too old:

error: Zig 0.15.0 or later is required

Solution: Install Zig 0.15+ from https://ziglang.org

Missing submodules:

error: unable to find gguf module

Solution: Run git submodule update --init --recursive

GPU not detected:

# Check GPU backend
igllama --info

# Verify drivers installed
nvidia-smi        # CUDA
vulkaninfo        # Vulkan
system_profiler   # Metal (macOS)

Getting Help