Bitwise Identical AI

mojolearn is an open-source Mojo machine-learning library designed for bitwise-reproducible training and inference across supported Apple, NVIDIA and AMD GPUs.

Visit Website
Bitwise Identical AI

Introduction

Overview

mojolearn is an open-source machine-learning library focused on bitwise-identical results across hardware. Its Apache 2.0-licensed Mojo codebase supports more than 100 algorithms and model components in 13 families, ranging from tree ensembles to transformers and hybrid attention and state-space models.

The project separates code portability from numerical reproducibility: Mojo targets Metal, CUDA and HIP, while mojolearn defines arithmetic rules intended to keep supported computations identical down to the bits.

Key Features

  • Runs one Mojo implementation across supported Apple, NVIDIA and AMD GPUs, with CPU execution also available.
  • Uses fixed reduction and accumulation rules plus portable numerical routines to control operation order and rounding.
  • Supports full-FP32 neural training and inference rather than converting those computations to integers.
  • Provides verification checks for representative inference and classical training workloads, with neural training available as a separate opt-in check.
  • Allows result hashes from two machines to be compared without a GPU, network connection or published reference results.

Reproducibility Evidence

The published experiment trained a 162-million-parameter language model twice from the same initialization. Each route processed 2.62 billion FineWeb-Edu tokens while moving through different combinations of Apple, NVIDIA and AMD hardware.

Both runs matched across all 5,000 optimizer steps in training state, summed gradients, per-shard losses and learning rate. They also produced identical final checkpoint files despite ending on different GPU platforms.

Use Cases and Scope

The numerical contract can support exact reproduction of fine-tuning, replay of model outputs, cross-vendor checkpoint resumption and pre-deployment checks across GPU fleets. Models trained in other frameworks can also be loaded for consistent inference on supported devices.

Identity applies when the release, input bytes, model state, settings and supported execution configuration match. Results may differ between releases, and bitwise agreement does not by itself establish accuracy, fairness or suitability for a particular application. The project is currently described as alpha software.