Architecture Engineering Example About GitHub ↗
About Haelixe

Why we went
bare-metal.

Most deep learning frameworks are a thin Python layer over someone else's C++ and CUDA. Haelixe takes the harder path: a compute engine written entirely in Rust, down to its own memory allocator.

The premise

Understanding a system means owning every layer of it.

Wrapping an existing runtime is the fast way to ship a framework. It is also the fast way to stop understanding what your framework actually does — every bug becomes a question about someone else's C++, and every optimization is bounded by an API you don't control.

Haelixe is built the slower way on purpose. The autograd engine, the tensor layout, the memory allocator, and the GPU dispatch layer are all written from scratch in Rust. When something is slow or wrong, the answer lives in this codebase — not three layers down in a vendored library.

The payoff is a framework that can be reasoned about end to end: deterministic memory reclamation, zero-copy tensor views, and GPU kernels that run unmodified on Vulkan, Metal, or DirectX through a single WGSL source.

Principles

Four decisions that shape everything else.

Determinism over convenience

RAII-based VRAM reclamation via Arc reference counting means a slab is freed exactly when the last autograd reference dies — never earlier, never silently.

Zero-copy by default

Views, transposes, and slices manipulate shape and stride arrays only. The physical buffer moves once, at allocation — not on every reshape.

One shader, three backends

Compute kernels are written once in WGSL and dispatched through wgpu to Vulkan, Metal, or DirectX 12 — no per-platform kernel forks to maintain.

Research-grade rigor

Every architectural claim is checked against a real convergence test — the haelixe-lab workspace exists specifically to validate the math, not just the API surface.

Inside the workspace

One crate, mapped by concern.

Haelixe is a Cargo workspace. haelixe-lab sits downstream as a real consumer of the public API, so ergonomics get tested the same way an external user would hit them.

src/tensor.rs The core Tensor struct and public API surface.
src/autograd.rs Computation graph (DAG) and reverse-mode autodiff.
src/ops/ Autograd operations — forward and backward pass logic.
src/nn/ Network layers: Linear, MultiHeadAttention, TransformerBlock.
src/kernels/ Bare-metal compute: matmul, reduce, binary ops, RoPE, RMSNorm.
src/gpu/ wgpu initialization, the binning slab allocator, and WGSL shaders.
src/device.rs CPU / GPU abstraction and device dispatch.
src/layout.rs Shape, strides, and contiguous memory mapping.
src/storage.rs Physical memory backings and mixed-precision slice accessors.
src/data/ Memory-mapped datasets and zero-copy dataloaders.
src/optim.rs Optimizers — AdamW with cosine annealing, CPU/GPU dispatch.
haelixe-lab/ Downstream consumer workspace for API ergonomics and convergence testing.
Rust Core language
Vulkan / Metal / DX12 GPU backends via wgpu
BF16 / F32 Mixed precision
1.70+ Minimum Rust version
Get involved

Read the code. Run the lab. File the issue.

Haelixe is under active development on GitHub. The clearest way to understand it is still to clone haelixe-lab and watch the loss curve converge.