Installation

Complete installation guide for BitNet on all supported platforms

System Requirements

Required

  • Python 3.9 or higher (Python 3.9 recommended)
  • CMake 3.15 or higher
  • C++ Compiler:
    • Linux: GCC 9+ or Clang 10+
    • Windows: Clang with MS-Build support or Visual Studio 2022
    • macOS: Clang (Xcode Command Line Tools)
  • Git for cloning the repository
  • Conda (highly recommended) or pip

Optional but Recommended

  • CUDA-capable GPU for optimal performance
  • NVIDIA CUDA Toolkit 11.0+ for GPU acceleration
  • cuDNN for optimized neural network operations

Installing Dependencies

Linux (Ubuntu/Debian)

Install Clang

Ubuntu/Debian - Install Clang 
bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"

Alternatively, you can install from your distribution's repositories:

Ubuntu/Debian - Alternative Installation 
sudo apt-get update
sudo apt-get install clang cmake git python3 python3-pip

Windows

On Windows, you need to use a Developer Command Prompt or PowerShell for VS2022. Make sure to initialize your environment properly.

Command Prompt

Windows - Command Prompt Setup 
"C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\Tools\VsDevCmd.bat" \
  -startdir=none -arch=x64 -host_arch=x64

PowerShell

Windows - PowerShell Setup 
Import-Module "C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\Tools\Microsoft.VisualStudio.DevShell.dll"
Enter-VsDevShell 3f0e31ad -SkipAutomaticLocation -DevCmdArguments "-arch=x64 -host_arch=x64"

For more Windows-specific issues, check our FAQ Page.

macOS

Install Xcode Command Line Tools:

macOS - Install Command Line Tools 
xcode-select --install

Install CMake and other dependencies via Homebrew:

macOS - Install via Homebrew 
brew install cmake git python@3.9

Building from Source

Step 1: Clone the Repository

Clone BitNet Repository 
git clone --recursive https://github.com/microsoft/BitNet.git
cd BitNet

Important: Use --recursive to include submodules.

Step 2: Create Conda Environment

We highly recommend using conda for environment management:

Create Conda Environment 
conda create -n bitnet-cpp python=3.9
conda activate bitnet-cpp

Step 3: Install Python Dependencies

Install Requirements 
pip install -r requirements.txt

Step 4: Download and Setup Model

Download a pre-quantized model from HuggingFace:

Download Model 
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf \
  --local-dir models/BitNet-b1.58-2B-4T

For a complete list of available models, see our Models Page.

Step 5: Setup BitNet Environment

Setup Environment Script 
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s

setup_env.py Options

Option Description Values
--hf-repo, -hr Model used for inference Various model names
--model-dir, -md Directory to save/load the model Path to model directory
--log-dir, -ld Directory to save logging info Path to log directory
--quant-type, -q Quantization type i2_s or tl1
--quant-embd Quantize embeddings to f16 Flag (no value)
--use-pretuned, -p Use pretuned kernel parameters Flag (no value)

Verifying Installation

To verify your installation, try running inference:

Test Installation 
python run_inference.py \
  -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf \
  -p "Hello, world!"

Converting Models from .safetensors

If you have a .safetensors checkpoint, you can convert it to GGUF format:

Step 1: Download .safetensors Model

Download .safetensors Model 
huggingface-cli download microsoft/bitnet-b1.58-2B-4T-bf16 \
  --local-dir ./models/bitnet-b1.58-2B-4T-bf16

Step 2: Convert to GGUF

Convert to GGUF Format 
python ./utils/convert-helper-bitnet.py ./models/bitnet-b1.58-2B-4T-bf16

Troubleshooting

If you encounter issues during installation, check our FAQ Page for solutions to common problems. Common issues include:

  • Build errors with llama.cpp
  • CUDA compatibility issues
  • Windows environment setup problems
  • Missing dependencies

Next Steps

Once installation is complete, you can:

Additional Resources