candle

Candle is a minimalist machine learning framework for Rust, developed by Hugging Face, with a strong focus on performance (including GPU support) and ease of use. It enables serverless inference by deploying lightweight binaries, eliminating Python overhead and accelerating instance creation on clusters. Candle supports a wide array of state-of-the-art ML models, including large language models, computer vision, and audio processing, and offers WebAssembly compatibility for browser-based execution.

• Minimalist ML framework natively in Rust
• High-performance inference with CPU, GPU (CUDA/cuDNN) support
• Supports a wide range of state-of-the-art pre-trained models (LLMs, CV, Audio)
• Lightweight binary deployment for serverless inference
• WebAssembly (WASM) compatibility for browser execution